This article outlines several major enhancements to the "VPop Inference" wizard in QSP Designer. The most significant addition is the "treatment-response" feature, which enables the VPop Inference fitting and analysis stages to evaluate each inputted individual against multiple defined treatments within the clinical data file.
Additionally, new methods for defining gates are introduced, along with various improvements to the filtering and analysis stages. The inclusion probability calculation stage has been enhanced with an additional empirical distribution type and the capability to view a plot of sampled individuals using calculated inclusion probabilities.
Several other minor improvements are also detailed throughout the article.
Enhancements to VPop Inference in QSP Designer
Virtual Populations (VPops) are supported in various areas of the QSP software. They are principally used to run simulations for multiple individuals with per-individual overrides for certain inputs. Individuals can also be considered as parameter sets.
VPop Inference (formerly ABC) is a feature used to check whether some individuals satisfy a set of constraints with respect to quantity values over time. These constraints are specified using "gate" definitions, which constrain the min/max values of a particular dependent variable at a given time point.
This document describes several extensions to VPop Inference:
Treatment Response Functionality:
The major goal of this project is the addition of “Treatment response” functionality.
This mode allows the user to simulate/analyze each VPop subject against multiple defined treatments.
A treatment is a set of clinical observations in the data file, as well as a set of starting parameters (that do not overlap with the parameters defining the individual itself).
This new ability works in both VPop Inference fitting mode (where the individual’s parameters are mutated to adhere to the observations) and during non-fitting (filter-only) mode (where the individual is simply determined to either adhere or not adhere to the observations).
Gate Definitions:
The ability to define gates manually (independently of any observations within the clinical data file) is added.
Gates can also be defined using error margins specified in the data file.
These improvements enhance flexibility when specifying and re-using gate definitions.
Gates can also be defined on a per-treatment basis when combined with the treatment response functionality.
Analysis Stage Enhancements:
The user may now (optionally) run the analysis stage following a fitting-based simulation.
Individuals that have had their input parameters mutated can be analyzed for acceptance/rejection based on more relaxed filter criteria than previously.
The threshold for the number of passed gates can be set on a per DV basis rather than a single threshold that applies globally to each DV.
Users can control whether to automatically run the filtering stage rather than requiring manual intervention following a simulation.
More options for plotting simulation results against clinical observations and gate definitions have been added.
The ability to either copy or clear the contents of the console area on this screen has been added.
Inclusion Probability Calculation:
The previous development cycle added inclusion probability calculation, which calculates a probability for each individual that passed the analysis stage.
This stage has seen a couple of improvements:
Generalizing the multivariate empirical distribution so that each variable/dimension may be either a normal distribution or a distribution of binned data (referred to as a “Frequency” distribution).
The ability to plot each of the univariate distributions in the inclusion screen as well as calculated samples of the virtual population using the calculated inclusion probabilities.
This allows the user to see how accurate the calculated inclusion probabilities are at reproducing the empirical distribution when individuals are sampled according to these probabilities.
Parameter Selection Interface:
The screen that allows the user to select the individual parameters that should be fitted is changed to more closely mimic the interface used in the Parameter Estimation wizard.
New Treatment Response Mode in VPop Inference
The ability to run VPop Inference in “treatment response” mode has been added.
Here are the key features and changes:
New Option: Defines Treatments from INIT Columns:
Added to the VPop Inference import page (the first screen of the wizard).
Available only in “New Simulation” mode, not in “Existing simulation results database”.
Operation Without the New Option:
If the option is not selected, the VPop Inference wizard operates as previously.
Each subject in the VPop file is fitted/simulated/analyzed as normal.
Data file restrictions remain the same as before.
Operation With the New Option:
The data file has different restrictions:
Must have at least one INIT/quantity-name column, where quantity-name is a species/parameter not listed in the VPop file.
The ID column must be present, with each ID number defining a new treatment.
For every ID number, there must be exactly one row where the EVENT type is INIT, with values for all INIT columns.
Multiple rows with the same ID and EVENT type INIT are not allowed, and values for INIT columns cannot be missing.
Optionally, the Label column may be present.
If present, each ID number must have exactly one row where EVENT type is LABEL and the Label column contains a non-empty string.
Treatments are referred to by names in the Label column or by ID numbers if labels are not available (e.g., "Treatment 4", "Treatment 5").
Gate Definition Screen:
Contains a drop-down box labeled “Treatment” next to the “DV Quantity” drop-down.
Lists all defined treatments from the data file (by label if present, or by number).
Switching between treatments shows different sets of gates.
Gates are linked to a treatment, not just a DV.
For example, with 3 treatments ("Cake", "Chocolate", "Crisps") and 2 DVs ("dv1", "dv2"), there will be 6 sets of gates ("dv1: Cake", "dv1: Chocolate", "dv1: Crisps", "dv2: Cake", "dv2: Chocolate", "dv2: Crisps").
Observations in the data file are also linked to the treatment, resulting in multiple sets of observations.
Simulation and Analysis:
An individual will only pass the criteria if it passes for all treatments.
Each DV must pass the minimum number of gates against each treatment.
Plots During Simulation/Analysis:
Plots will take treatments into account.
Inclusion Statistics Calculation:
Each variable used for calculation is {Treatment}: {DV}: {time}.
Values are taken from simulations corresponding to the relevant treatments.
Inclusion statistic calculations behave the same as in non-treatments mode.
New Gate Loading Functionality
There is now the ability to load gates from the loaded data file instead of specifying them one at a time in the interface. This option is only available when running the wizard in "New simulation" mode. It is disabled for "Existing simulation results database" mode. A button “Load From Data File” has been added to the gate definitions overview screen.
Clicking the button will:
Clear any existing gates that are defined.
Load any gates defined in the data file's ERROR columns.
This occurs as follows:
Gates are automatically inferred from the value of a DV's corresponding ERROR column (symmetrically around the observation) or from the combination of ERROR_UPPER and ERROR_LOWER columns.
For example, given the existence of a DV called
d:If the column
ERROR/dis present, then for every row whereEVENT=DVand there is a value present for theERROR/dcolumn (call thise):A gate will be loaded at the time given in the
Timecolumn on the current row, where:min = (the value of DV/d on the current row - e)max = (the value of DV/d on the current row + e)
If the columns
ERROR_LOWER/dandERROR_UPPER/dare both present, then the same as above except that the min value of the gate is simply the value ofERROR_LOWER/don the current row, and the max value of the gate is simply the value ofERROR_UPPER/don the current row.
For any given DV (for example
d), you cannot haveERROR/dpresent ifERROR_UPPER/dorERROR_LOWER/dare both present (and vice versa). Only one of the two methods may be used for a given DV.
If the "Defines treatments from INIT columns" option was selected, then the inferred gates are associated with the matching treatment specified by the
IDcolumn in the data file (see user story 124105 for more information on how treatments work).
The “new gate definition” popup window now allows users to define gates manually, without taking the min/max values from the loaded data file.
If the user selects "From observations", then gates are determined as before, where the user enters a "Time" and "Tolerance" and the software calculates the min/max from the data file.
If the user selects "Manual", then this exposes the new functionality. The user can simply enter a Time, Min, and Max values. If these are valid (positive time, and max > min), then the user can add the gate to the definitions list. The data file is not used to determine min or max values when using this option.
VPop Inference: Enhanced Gate Criteria and Analysis Options
Previously, when running VPop Inference in "fitting mode" (i.e., the option "Attempt to fit individuals so that they pass criteria" was selected), an individual was considered to "pass/succeed" only if they passed all gates. The ability to specify that only some threshold of gates needs to be passed for an individual to succeed was limited to non-fitting mode.
This restriction has been relaxed so that once the fitting process has finished, the user can choose to explicitly Analyse with less strict gate criteria, similar to non-fitting mode (i.e., where they can specify that only X number of gates needs to pass).
Improved Filter Criteria Specification
The method for specifying filter criteria has also been improved for both fitting and non-fitting modes. The analysis options have been expanded to be more customizable.
Rather than the simple option of "All gates" or "Threshold per DV" as seen in previous versions, we now have a dedicated screen for configuring thresholds. This screen is accessible via a new analysis options button (inactive when currently simulating or analyzing).
It opens the following window:
If "All DVs" is selected:
The user can specify either All gates or a number of gates to pass, identical to the previous version's behavior (i.e., the settings apply to all DVs).
If "Chosen DVs" is selected:
The interface changes, allowing the user to add any of the DVs from the left-hand side to the right-hand side, selecting them for filtering.
If a DV is not selected for filtering, its values will be completely ignored during the analysis stage (an individual may fail all gates for that DV, and it will not affect whether they succeed or fail filtering).
A chosen DV can be set to require All Gates or only a specified number of gates. This means different DVs can have different thresholds to determine whether the overall individual passes the filter.
For example, an individual requires that
ID0passes all of its gates, butIDonly needs to pass 6 gates. If both conditions hold, the individual passes the filter. Otherwise, the individual fails the filter. The values ofBeta ID,KMS,MD,MSfor the individual are ignored, and the gates are not considered.
Auto-Run Analysis Stage
The ability to auto-run the Analysis stage of VPop Inference following simulation has also been added.
If this option is checked:
When the Simulation stage finishes, Analysis is run automatically.
If this option is unchecked:
It follows the previous behavior where the user must click "Analyse" following completion of simulation.
The default option is ON. This setting is not stored with the workspace and will reset once the VPop Inference window is closed.
Plotting Improvements
There have also been improvements to plotting on the simulation/analysis screen.
When not running in "treatments mode":
A single DV, and a single individual.
A single DV and all individuals.
In both modes, users can choose to either hide or show gates and hide or show observations loaded from the data file for the current DV.
When running the wizard in "treatments mode"
(see User Story 124105):
When running the wizard in "treatments mode" (see User Story 124105):
A single DV, and a single individual, running against a single treatment.
A single DV, and a single individual, running against all treatments.
A single DV, and all individuals, running against a single treatment.
A single DV, and all individuals, running against all treatments.
Gates or observations can be shown/hidden as before.
In both cases (treatments mode and non-treatments mode), plot settings can be modified similarly to the simulation outputs window, e.g., toggle grid lines, switch from linear to log scale, edit colors, etc., and export to Plotly HTML. This makes the plot behave similarly to other parts of the software, such as the simulation outputs window or the parameter scanner.
The graph can also be zoomed, and certain legends toggled on/off. Additionally, hovering over parts of the graph provides extra information.
Console Message Options
The options to either Copy (to clipboard) or Clear the console messages in the VPop Inference analysis screen have also been added.
Introducing Frequency Distributions for Inclusion Statistics
A new type of distribution, referred to in the UI as a "Frequency" distribution, can now be selected for each variable on the inclusion statistics calculation screen.
Under "Selected variables":
Next to each selected variable name, there is a new combo box drop-down that lists either "Normal" or "Frequency".
Selecting "Normal" shows the controls for specifying a univariate normal distribution as previously.
If all selected variables are set to "Normal", the variance-covariance matrix can be specified as before. This is only possible if all variables are "Normal," as this means the overall distribution is a "multivariate normal" distribution.
Selecting "Frequency" exposes the new "Frequency" distribution type for that variable.
Add a series of rows, where each row defines a "bin" for that distribution.
A bin includes a name (string), a minimum value (number), a maximum value (number), and a weighted frequency (positive number greater than zero).
The maximum must be greater than the minimum for each bin.
No bins may have overlapping ranges. Note that the minimum value is inclusive and the maximum value is exclusive, so you can define something like the following without it being considered an overlap:
Bin 0 - Min: 0, Max: 1
Bin 1 - Min: 1, Max: 2
Weighted frequencies do not need to add up to 1, as they are normalized during the calculations. It is only necessary that all weighted frequencies are greater than 0.
Any errors in the specification of a frequency distribution are displayed at the top of the screen in red text, similar to errors in the specification of normal distributions. This also prevents the calculation of the inclusion statistics as it did before.
Any combination of normal and frequency distributions can be used.
Additionally, the specification of a frequency distribution can be saved or loaded from a CSV file. This is done using the CSV "Load" and "Save" buttons that appear separately for each selected variable when it is set to "Frequency" mode.
The overall set of distributions can also be saved or loaded using the CSV "Load" and "Save" buttons located at the top-right of the "Selected variables" box.
Enhanced Plotting and Comparison for Inclusion Statistics
The ability to plot the specified univariate distributions in the inclusion screen, as well as test samples of the virtual population using the calculated inclusion probabilities, has been added.
On the inclusion screen, a new button "Compare" has been introduced. This button becomes active once the proportionality constant has been fitted.
This opens a new window that allows the user to sample the population using the calculated inclusion probabilities.
The window's top-left provides options to select one of the distributions or variables defined on the main screen.
Clicking Acceptance Sampling or Weighted Sampling will initiate a sampling of the successful individuals from the VPop Inference analysis stage, using the same algorithms employed in the VPop Management window during a VPop Simulation. Clicking these buttons repeatedly will re-sample using fresh random values.
Different options are available depending on whether the selected distribution/variable is Normal or Frequency.
The screenshot above shows:
A plot of the Normally distributed variable
S1's specified empirical distribution (green line) - i.e., the normal distribution defined on the inclusion screen.The sampled individuals binned into a series of 100-width intervals (specified at the top of the screen), with the resulting densities of those bins plotted (blue bars).
The bin width can be adjusted to more appropriate values to simplify visualization of their distribution. For example, in the below screenshot, the bin width has been changed to 10.
A line of best fit for the estimated distribution of the sampled individuals (red curve).
If the selected variable is a Frequency distribution, there are two binning modes available for the sampled individuals: "Distribution bins" or "Fine-grained bins".
The screenshot above illustrates the plot when viewing the sampled individuals from a Frequency distribution in the context of the Distribution bins. It shows:
A plot of the Frequency distributed variable
S's specified empirical distribution (the thick outlines colored blue, red, and green) - i.e., the Frequency distribution defined on the inclusion screen.The sampled individuals binned into these empirically defined bins, with the resulting bins' densities plotted (the solid bars). In the screenshot above, only the red and green bins have sampled individuals in them, resulting in non-zero densities. The blue bin has no sampled individuals, hence a zero density.
The screenshot above illustrates the plot when viewing the sampled individuals from a Frequency distribution in the context of fine-grained bins. It shows the individuals binned into a series of fine-grained bins of 25-width, with the resulting densities of those bins plotted.
Fine-Grained Width Adjustment: Similarly to viewing a Normally distributed sample, the fine-grained width can be adjusted.
Color-Coded Bins: The fine-grained bins are colored to reflect the Frequency distribution's empirical bins that they fall into.
In this screenshot, no individuals fall into the blue empirical bin, so there are no finer-grained bins of the same color.
There are also finer-grained bins within the red and green empirical bins that contain no individuals, creating gaps between the bins (these are simply bars of height zero).
In all cases, these plots can be exported to Plotly and aspects of the plot can be configured, as seen in the above screenshots. This functionality is consistent with the plots in other parts of QSP Designer.
Enhanced Parameter Selection Interface in VPop Inference Wizard
The parameter selection interface on the VPop Inference wizard has been updated to more closely resemble the selection interface in the Parameter Estimation wizard. The selected parameters are now presented in a table structure.
Parameter Selection
Parameters can be selected from the bottom and added to the list of Selected Parameters using the arrows.
Bound Enforcement
The Min/Max checkboxes are replaced with "Enforce Lower Bound" and "Enforce Upper Bound". Similarly, the minimum and maximum input boxes are replaced with "Lower Bound" and "Upper Bound".
Parameter Estimation
All parameters in the Selected Parameters list are estimated during fitting. Previously, it was possible to check/uncheck "Estimate" to switch a parameter on or off.
Save/Load Functionality
The selection can be saved or loaded from a CSV file using the new Save and Load buttons.