Identifying confounding by indication through blinded prospective review.

Confounding by indication is a relentless threat to validity in observational studies of treatment effects. Multivariable models allow adjustment for known and readily measurable prognostic factors, but they may incompletely or inaccurately represent the underlying overall perceived risk of treatment. To incorporate practitioners' judgments about treatment indication and preprocedural prognosis into an observational study of cerebral aneurysm treatments, the author and colleagues presented patient characteristics and radiographic images from 179 aneurysm cases (University of California, San Francisco, 1990--1997) to panels of practitioners who were blinded as to actual treatment selection and outcome. In this way, the review process was designed to recreate the presentation of information in a prospective study. Judgments about inclusion and prognosis were reproducible. Perceived prognosis correlated with complication rates and provided information not present in a multivariable model including all available clinical characteristics. The association between treatment modality and outcome was examined while stratifying and adjusting for differences in perceived prognosis. Blinded prospective review may provide an unbiased observational study design with which to define a cohort that could have received any of the treatments being compared and to measure and adjust for overall perceived procedural risk.