Saerom Youn1,2, Shannon Avery Wong3, Caitlin Chrystoja1, George Tomlinson1, Harindra C Wijeysundera1,4, Chaim M Bell1,4, Anna R Gagliardi1,5, Nancy N Baxter1,5,6,7, Julie Takata8, Lakhbir Sandhu5, David Robert Urbach9,10,11,12,13. 1. Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada. 2. Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada. 3. College of Medicine and Dentistry, James Cook University, Parkville, QLD, Australia. 4. Department of Medicine, University of Toronto, Toronto, Canada. 5. Department of Surgery, University of Toronto, Toronto, Canada. 6. Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada. 7. School of Population and Global Health, University of Melbourne, Melbourne, VIC, Australia. 8. Women's College Hospital Research Institute (WCRI), Toronto, ON, Canada. 9. Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada. david.urbach@wchospital.ca. 10. Department of Surgery, University of Toronto, Toronto, Canada. david.urbach@wchospital.ca. 11. Women's College Hospital Research Institute (WCRI), Toronto, ON, Canada. david.urbach@wchospital.ca. 12. Department of Surgery, Women's College Hospital, 76 Grenville St, Room 8332, M5S 1B2, Toronto, ON, Canada. david.urbach@wchospital.ca. 13. Women's College Hospital Institute for Health System Solutions and Virtual Care (WIHV), Toronto, ON, Canada. david.urbach@wchospital.ca.
Abstract
BACKGROUND: Paucity of RCTs of non-drug technologies lead to widespread dependence on non-randomized studies. Relationship between nonrandomized study design attributes and biased estimates of treatment effects are poorly understood. Our purpose was to estimate the bias associated with specific nonrandomized study attributes among studies comparing transcatheter aortic valve implantation with surgical aortic valve replacement for the treatment of severe aortic stenosis. RESULTS: We included 6 RCTs and 87 nonrandomized studies. Surgical risk scores were similar for comparison groups in RCTs, but were higher for patients having transcatheter aortic valve implantation in nonrandomized studies. Nonrandomized studies underestimated the benefit of transcatheter aortic valve implantation compared with RCTs. For example, nonrandomized studies without adjustment estimated a higher risk of postoperative mortality for transcatheter aortic valve implantation compared with surgical aortic valve replacement (OR 1.43 [95% CI 1.26 to 1.62]) than high quality RCTs (OR 0.78 [95% CI 0.54 to 1.11). Nonrandomized studies using propensity score matching (OR 1.13 [95% CI 0.85 to 1.52]) and regression modelling (OR 0.68 [95% CI 0.57 to 0.81]) to adjust results estimated treatment effects closer to high quality RCTs. Nonrandomized studies describing losses to follow-up estimated treatment effects that were significantly closer to high quality RCT than nonrandomized studies that did not. CONCLUSION: Studies with different attributes produce different estimates of treatment effects. Study design attributes related to the completeness of follow-up may explain biased treatment estimates in nonrandomized studies, as in the case of aortic valve replacement where high-risk patients were preferentially selected for the newer (transcatheter) procedure.
BACKGROUND: Paucity of RCTs of non-drug technologies lead to widespread dependence on non-randomized studies. Relationship between nonrandomized study design attributes and biased estimates of treatment effects are poorly understood. Our purpose was to estimate the bias associated with specific nonrandomized study attributes among studies comparing transcatheter aortic valve implantation with surgical aortic valve replacement for the treatment of severe aortic stenosis. RESULTS: We included 6 RCTs and 87 nonrandomized studies. Surgical risk scores were similar for comparison groups in RCTs, but were higher for patients having transcatheter aortic valve implantation in nonrandomized studies. Nonrandomized studies underestimated the benefit of transcatheter aortic valve implantation compared with RCTs. For example, nonrandomized studies without adjustment estimated a higher risk of postoperative mortality for transcatheter aortic valve implantation compared with surgical aortic valve replacement (OR 1.43 [95% CI 1.26 to 1.62]) than high quality RCTs (OR 0.78 [95% CI 0.54 to 1.11). Nonrandomized studies using propensity score matching (OR 1.13 [95% CI 0.85 to 1.52]) and regression modelling (OR 0.68 [95% CI 0.57 to 0.81]) to adjust results estimated treatment effects closer to high quality RCTs. Nonrandomized studies describing losses to follow-up estimated treatment effects that were significantly closer to high quality RCT than nonrandomized studies that did not. CONCLUSION: Studies with different attributes produce different estimates of treatment effects. Study design attributes related to the completeness of follow-up may explain biased treatment estimates in nonrandomized studies, as in the case of aortic valve replacement where high-risk patients were preferentially selected for the newer (transcatheter) procedure.
Entities:
Keywords:
Aortic stenosis; Bias; Meta-epidemiological; Meta-regression; Non drug health technologies; Nonrandomized studies; Randomized controlled trials; SAVR; Study design attributes; TAVI
Authors: J P Ioannidis; A B Haidich; M Pappa; N Pantazis; S I Kokori; M G Tektonidou; D G Contopoulos-Ioannidis; J Lau Journal: JAMA Date: 2001-08-15 Impact factor: 56.272
Authors: Thao Huynh; Stephane Perron; Jennifer O'Loughlin; Lawrence Joseph; Michel Labrecque; Jack V Tu; Pierre Théroux Journal: Circulation Date: 2009-06-08 Impact factor: 29.690
Authors: Robert M Califf; Deborah A Zarin; Judith M Kramer; Rachel E Sherman; Laura H Aberle; Asba Tasneem Journal: JAMA Date: 2012-05-02 Impact factor: 56.272
Authors: Ajay J Kirtane; Anuj Gupta; Srinivas Iyengar; Jeffrey W Moses; Martin B Leon; Robert Applegate; Bruce Brodie; Edward Hannan; Kishore Harjai; Lisette Okkels Jensen; Seung-Jung Park; Raphael Perry; Michael Racz; Francesco Saia; Jack V Tu; Ron Waksman; Alexandra J Lansky; Roxana Mehran; Gregg W Stone Journal: Circulation Date: 2009-06-15 Impact factor: 29.690