Assessing the Performance of a Computer-Based Policy Model of HIV and AIDS

Background: Model-based analyses, conducted within a decision analytic framework, provide a systematic way to combineinformation about the natural history of disease and effectiveness of clinical management strategies with demographic andepidemiological characteristics of the population. Among the challenges with disease-specific modeling include the need toidentify influential assumptions and to assess the face validity and internal consistency of the model. Methods and Findings: We describe a series of exercises involved in adapting a computer-based simulation model of HIVdisease to the Women’s Interagency HIV Study (WIHS) cohort and assess model performance as we re-parameterized themodel to address policy questions in the U.S. relevant to HIV-infected women using data from the WIHS. Empiric calibrationtargets included 24-month survival curves stratified by treatment status and CD4 cell count. The most influentialassumptions in untreated women included chronic HIV-associated mortality following an opportunistic infection, and intreated women, the ‘clinical effectiveness’ of HAART and the ability of HAART to prevent HIV complications independent ofvirologic suppression. Good-fitting parameter sets required reductions in the clinical effectiveness of 1st and 2nd line HAARTand improvements in 3rd and 4th line regimens. Projected rates of treatment regimen switching using the calibrated cohortspecificmodel closely approximated independent analyses published using data from the WIHS. Conclusions: The model demonstrated good internal consistency and face validity, and supported cohort heterogeneitiesthat have been reported in the literature. Iterative assessment of model performance can provide information about therelative influence of uncertain assumptions and provide insight into heterogeneities within and between cohorts.Description of calibration exercises can enhance the transparency of disease-specific models.