Nicole G Campos1,2, Philip E Castle3, Mark Schiffman4, Jane J Kim1. 1. Center for Health Decision Science, Harvard School of Public Health, Boston, MA (NGC, JJK) 2. Center of Excellence for Health Disparities Research–El Centro, School of Nursing and Health Studies, University of Miami, Coral Gables, FL (NGC) 3. American Society for Clinical Pathology Institute, Washington, DC (PEC) 4. Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, MD (MS)
Abstract
BACKGROUND: Although the randomized controlled trial (RCT) is widely considered the most reliable method for evaluation of health care interventions, challenges to both internal and external validity exist. Thus, the efficacy of an intervention in a trial setting does not necessarily represent the real-world performance that decision makers seek to inform comparative effectiveness studies and economic evaluations. METHODS: Using data from the ASCUS-LSIL Triage Study (ALTS), we performed a simplified economic evaluation of age-based management strategies to detect cervical intraepithelial neoplasia grade 3 (CIN3) among women who were referred to the study with low-grade squamous intraepithelial lesions (LSIL). We used data from the trial itself to adjust for 1) potential lead time bias and random error that led to variation in the observed prevalence of CIN3 by study arm and 2) potential ascertainment bias among providers in the most aggressive management arm. RESULTS: We found that using unadjusted RCT data may result in counterintuitive cost-effectiveness results when random error and/or bias are present. Following adjustment, the rank order of management strategies changed for 2 of the 3 age groups we considered. CONCLUSIONS: Decision analysts need to examine study design, available trial data, and cost-effectiveness results closely in order to detect evidence of potential bias. Adjustment for random error and bias in RCTs may yield different policy conclusions relative to unadjusted trial data.
RCT Entities:
BACKGROUND: Although the randomized controlled trial (RCT) is widely considered the most reliable method for evaluation of health care interventions, challenges to both internal and external validity exist. Thus, the efficacy of an intervention in a trial setting does not necessarily represent the real-world performance that decision makers seek to inform comparative effectiveness studies and economic evaluations. METHODS: Using data from the ASCUS-LSIL Triage Study (ALTS), we performed a simplified economic evaluation of age-based management strategies to detect cervical intraepithelial neoplasia grade 3 (CIN3) among women who were referred to the study with low-grade squamous intraepithelial lesions (LSIL). We used data from the trial itself to adjust for 1) potential lead time bias and random error that led to variation in the observed prevalence of CIN3 by study arm and 2) potential ascertainment bias among providers in the most aggressive management arm. RESULTS: We found that using unadjusted RCT data may result in counterintuitive cost-effectiveness results when random error and/or bias are present. Following adjustment, the rank order of management strategies changed for 2 of the 3 age groups we considered. CONCLUSIONS: Decision analysts need to examine study design, available trial data, and cost-effectiveness results closely in order to detect evidence of potential bias. Adjustment for random error and bias in RCTs may yield different policy conclusions relative to unadjusted trial data.