A Review and Analysis of Economic Models of Prevention Benefits. A Review and Analysis of Economic Models of Prevention Benefits : Table 1


: Table 1

Type of ThreatImportanceMethods of Detection and/or Amelioration
Threats to Fidelity  
Programming errorsHigh■    Systematically vary model parameters to extreme values to identify model processes causing errors. ■    Budget appropriate time for model testing and debugging. ■    Utilize personnel with programming expertise.
Unrealistic, inaccurate, or overly-simplified disease natural history modelVery High■    Solicit clinical and other subject matter expertise to review model structure and assumptions. ■    Focus on natural history changes that result in a change of symptoms or costs.  Stages that do not directly influence a change of symptoms or costs can be combined with other stages.
Threats to Internal Validity  
Systematic bias in selection of parametersVery High■    Solicit expert review of parameters. ■    Be alert for omission of important studies. ■    If systematic bias is identified, heavily discount results.
Use of low-quality data as the result of lack of informationLow■    Solicit expert review of parameters. ■    It is not uncommon to have weak data sources for some model parameters.  Well-designed studies will include sensitivity analyses that test the impact of weakly measured parameters.
Model does not recreate data used to program itModerate■    Evaluate information comparing the model’s epidemiological inputs to the comparable estimates created by the model. ■    Consider causal relationships in the model, it is possible that the correct parameter value is being misinterpreted or used incorrectly. ■    Model calibration can improve the model’s internal predictive performance.
Poor cross-model validation. Model does not produce similar results to other simulation models.Moderate to Very High■    Evaluate reasons why the models predictions would differ such as whether model uses different input data, or whether the model uses different disease natural history assumptions. ■    If different, evaluate if different data or model formulation is a suitable alternative to existing models.
Threats to External Validity       
Model does not predict out of sample epidemiological dataModerate■    Adjust model’s confounding parameters to match external data set. ■    Consider if differences are reasonable given uncertainty in model’s epidemiological parameters. ■    Calibrate model if differences are extreme.
Model utilization rates do not match community estimates utilization ratesLow■    Assess the degree to and manner in which community utilization rates influence the model’s results. ■    Consider whether the utilization rates used in the model are logical and internally consistent across utilization, effectiveness, and costs. ■    Assess how model results apply to settings in which community rates of service utilization differ.