Carrie E Fry1, Laura A Hatfield2. 1. Interfaculty Initiative in Health Policy, Harvard University, Cambridge, Massachusetts, USA. 2. Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, USA.
Abstract
OBJECTIVE: To formalize comparative interrupted time series (CITS) using the potential outcomes framework; compare two version of CITS-a standard linear version and one that adds postperiod group-by-time parameters-to two versions of difference-in-differences (DID)-a standard version with time fixed effects and one that adds group-specific pretrends; and reanalyze three previously published papers using these models. DATA SOURCES: Outcome data for reanalyses come from two counties' jail booking and release data, Medicaid prescription drug rebate data from the Centers for Medicare and Medicaid Services (CMS), and acute hepatitis C incidence from the Centers for Disease Control and Prevention. STUDY DESIGN: DID and CITS were compared using potential outcomes, and reanalyses were conducted using the four described pre-post study designs. DATA COLLECTION/EXTRACTION METHODS: Data from county jails were provided by sheriffs. Data from CMS are publicly available. Data for the third reanalysis were provided by the authors of the original study. PRINCIPAL FINDINGS: Though written differently and preferred by different research communities, the general version of CITS and DID with group-specific pretrends are the same: they yield the same counterfactuals and identify the same treatment effects. In a reanalysis with evidence of divergent preperiod trends, failing to account for this in standard DID led to an 84% smaller effect estimate than the more flexible models. In a second reanalysis with evidence of nonlinear outcome trends, failing to account for this in linear CITS led to a 28% smaller effect estimate than the more flexible models. CONCLUSION: We recommend detailing a causal model for treatment selection and outcome generation and the required counterfactuals before choosing an analytical approach. The more flexible versions of DID and CITS can accommodate features often found in real data, namely, nonlinearities and divergent preperiod outcome trends.
OBJECTIVE: To formalize comparative interrupted time series (CITS) using the potential outcomes framework; compare two version of CITS-a standard linear version and one that adds postperiod group-by-time parameters-to two versions of difference-in-differences (DID)-a standard version with time fixed effects and one that adds group-specific pretrends; and reanalyze three previously published papers using these models. DATA SOURCES: Outcome data for reanalyses come from two counties' jail booking and release data, Medicaid prescription drug rebate data from the Centers for Medicare and Medicaid Services (CMS), and acute hepatitis C incidence from the Centers for Disease Control and Prevention. STUDY DESIGN: DID and CITS were compared using potential outcomes, and reanalyses were conducted using the four described pre-post study designs. DATA COLLECTION/EXTRACTION METHODS: Data from county jails were provided by sheriffs. Data from CMS are publicly available. Data for the third reanalysis were provided by the authors of the original study. PRINCIPAL FINDINGS: Though written differently and preferred by different research communities, the general version of CITS and DID with group-specific pretrends are the same: they yield the same counterfactuals and identify the same treatment effects. In a reanalysis with evidence of divergent preperiod trends, failing to account for this in standard DID led to an 84% smaller effect estimate than the more flexible models. In a second reanalysis with evidence of nonlinear outcome trends, failing to account for this in linear CITS led to a 28% smaller effect estimate than the more flexible models. CONCLUSION: We recommend detailing a causal model for treatment selection and outcome generation and the required counterfactuals before choosing an analytical approach. The more flexible versions of DID and CITS can accommodate features often found in real data, namely, nonlinearities and divergent preperiod outcome trends.
Authors: Atle Fretheim; Fang Zhang; Dennis Ross-Degnan; Andrew D Oxman; Helen Cheyne; Robbie Foy; Steve Goodacre; Jeph Herrin; Ngaire Kerse; R James McKinlay; Adam Wright; Stephen B Soumerai Journal: J Clin Epidemiol Date: 2014-12-11 Impact factor: 6.437
Authors: Peter C Rockers; John-Arne Røttingen; Ian Shemilt; Peter Tugwell; Till Bärnighausen Journal: Health Policy Date: 2014-10-22 Impact factor: 2.980
Authors: Peter Craig; Srinivasa Vittal Katikireddi; Alastair Leyland; Frank Popham Journal: Annu Rev Public Health Date: 2017-01-11 Impact factor: 21.981