Atle Fretheim1, Fang Zhang2, Dennis Ross-Degnan2, Andrew D Oxman3, Helen Cheyne4, Robbie Foy5, Steve Goodacre6, Jeph Herrin7, Ngaire Kerse8, R James McKinlay9, Adam Wright10, Stephen B Soumerai2. 1. Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue, Boston, MA 02215, USA; Global Health Unit, Norwegian Knowledge Centre for the Health Services, PO Box 7004, St. Olavs pl, 0130 Oslo, Norway; Department of Community Medicine, Institute of Health and Society, University of Oslo, PO Box 1130 Blindern, 0318 Oslo, Norway. Electronic address: atle.fretheim@nokc.no. 2. Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue, Boston, MA 02215, USA. 3. Global Health Unit, Norwegian Knowledge Centre for the Health Services, PO Box 7004, St. Olavs pl, 0130 Oslo, Norway. 4. Nursing Midwifery and Allied Health Professions Research Unit, University of Stirling, Stirling FK9 4LA, UK. 5. Academic Unit of Primary Care, Leeds Institute of Health Sciences, University of Leeds, Charles Thackrah Building, 101 Clarendon Road, Leeds LS2 9LJ, UK. 6. School of Health and Related Research (ScHARR), University of Sheffield, Regent Court, Regent Street, Sheffield, S1 4DA, UK. 7. Division of Cardiology, Yale University School of Medicine, Yale University, 333 Cedar St, New Haven, CT 06510, USA; Health Research & Educational Trust, 155 N Wacker, Suite 400, Chicago 60606, IL, USA. 8. School of Population Health, University of Auckland, Private Bag 9201, Auckland, New Zealand. 9. Health Information Research Unit, Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada. 10. Department of Medicine, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115, USA.
Abstract
OBJECTIVES: There is often substantial uncertainty about the impacts of health system and policy interventions. Despite that, randomized controlled trials (RCTs) are uncommon in this field, partly because experiments can be difficult to carry out. An alternative method for impact evaluation is the interrupted time-series (ITS) design. Little is known, however, about how results from the two methods compare. Our aim was to explore whether ITS studies yield results that differ from those of randomized trials. STUDY DESIGN AND SETTING: We conducted single-arm ITS analyses (segmented regression) based on data from the intervention arm of cluster randomized trials (C-RCTs), that is, discarding control arm data. Secondarily, we included the control group data in the analyses, by subtracting control group data points from intervention group data points, thereby constructing a time series representing the difference between the intervention and control groups. We compared the results from the single-arm and controlled ITS analyses with results based on conventional aggregated analyses of trial data. RESULTS: The findings were largely concordant, yielding effect estimates with overlapping 95% confidence intervals (CI) across different analytical methods. However, our analyses revealed the importance of a concurrent control group and of taking baseline and follow-up trends into account in the analysis of C-RCTs. CONCLUSION: The ITS design is valuable for evaluation of health systems interventions, both when RCTs are not feasible and in the analysis and interpretation of data from C-RCTs.
OBJECTIVES: There is often substantial uncertainty about the impacts of health system and policy interventions. Despite that, randomized controlled trials (RCTs) are uncommon in this field, partly because experiments can be difficult to carry out. An alternative method for impact evaluation is the interrupted time-series (ITS) design. Little is known, however, about how results from the two methods compare. Our aim was to explore whether ITS studies yield results that differ from those of randomized trials. STUDY DESIGN AND SETTING: We conducted single-arm ITS analyses (segmented regression) based on data from the intervention arm of cluster randomized trials (C-RCTs), that is, discarding control arm data. Secondarily, we included the control group data in the analyses, by subtracting control group data points from intervention group data points, thereby constructing a time series representing the difference between the intervention and control groups. We compared the results from the single-arm and controlled ITS analyses with results based on conventional aggregated analyses of trial data. RESULTS: The findings were largely concordant, yielding effect estimates with overlapping 95% confidence intervals (CI) across different analytical methods. However, our analyses revealed the importance of a concurrent control group and of taking baseline and follow-up trends into account in the analysis of C-RCTs. CONCLUSION: The ITS design is valuable for evaluation of health systems interventions, both when RCTs are not feasible and in the analysis and interpretation of data from C-RCTs.
Authors: Adam L Sharp; Brian Z Huang; Tania Tang; Ernest Shen; Edward R Melnick; Arjun K Venkatesh; Michael H Kanter; Michael K Gould Journal: Ann Emerg Med Date: 2017-07-21 Impact factor: 5.721
Authors: Michael Von Korff; Sascha Dublin; Rod L Walker; Michael Parchman; Susan M Shortreed; Ryan N Hansen; Kathleen Saunders Journal: J Pain Date: 2015-10-22 Impact factor: 5.820
Authors: Lauryn P Roth; Jessica N Sanders; Rebecca G Simmons; Holly Bullock; Elizabeth Jacobson; David K Turok Journal: Contraception Date: 2018-03-21 Impact factor: 3.375
Authors: Adam L Sharp; Aileen S Baecker; Ernest Shen; Rita Redberg; Ming-Sum Lee; Maros Ferencik; Shaw Natsui; Chengyi Zheng; Aniket Kawatkar; Michael K Gould; Benjamin C Sun Journal: Ann Emerg Med Date: 2019-02-21 Impact factor: 5.721
Authors: Marguerite E Burns; Haiden A Huskamp; Jessica C Smith; Jeanne M Madden; Stephen B Soumerai Journal: Med Care Date: 2016-09 Impact factor: 2.983
Authors: Arthur Bagonza; Freddy Eric Kitutu; Stefan Peterson; Andreas Mårtensson; Milton Mutto; Phyllis Awor; David Mukanga; Henry Wamani Journal: Health Sci Rep Date: 2021-05-07