PURPOSE: The choice of propensity score (PS) implementation influences treatment effect estimates not only because different methods estimate different quantities, but also because different estimators respond in different ways to phenomena such as treatment effect heterogeneity and limited availability of potential matches. Using effectiveness data, we describe lessons learned from sensitivity analyses with matched and weighted estimates. METHODS: With subsample data (N = 1292) from Sequenced Treatment Alternatives to Relieve Depression, a 2001-2004 effectiveness trial of depression treatments, we implemented PS matching and weighting to estimate the treatment effect in the treated and conducted multiple sensitivity analyses. RESULTS: Matching and weighting both balanced covariates but yielded different samples and treatment effect estimates (matched RR 1.00, 95% CI: 0.75-1.34; weighted RR 1.28, 95% CI: 0.97-1.69). In sensitivity analyses, as increasing numbers of observations at both ends of the PS distribution were excluded from the weighted analysis, weighted estimates approached the matched estimate (weighted RR 1.04, 95% CI 0.77-1.39 after excluding all observations below the 5th percentile of the treated and above the 95th percentile of the untreated). Treatment appeared to have benefits only in the highest and lowest PS strata. CONCLUSIONS: Matched and weighted estimates differed due to incomplete matching, sensitivity of weighted estimates to extreme observations, and possibly treatment effect heterogeneity. PS analysis requires identifying the population and treatment effect of interest, selecting an appropriate implementation method, and conducting and reporting sensitivity analyses. Weighted estimation especially should include sensitivity analyses relating to influential observations, such as those treated contrary to prediction.
RCT Entities:
PURPOSE: The choice of propensity score (PS) implementation influences treatment effect estimates not only because different methods estimate different quantities, but also because different estimators respond in different ways to phenomena such as treatment effect heterogeneity and limited availability of potential matches. Using effectiveness data, we describe lessons learned from sensitivity analyses with matched and weighted estimates. METHODS: With subsample data (N = 1292) from Sequenced Treatment Alternatives to Relieve Depression, a 2001-2004 effectiveness trial of depression treatments, we implemented PS matching and weighting to estimate the treatment effect in the treated and conducted multiple sensitivity analyses. RESULTS: Matching and weighting both balanced covariates but yielded different samples and treatment effect estimates (matched RR 1.00, 95% CI: 0.75-1.34; weighted RR 1.28, 95% CI: 0.97-1.69). In sensitivity analyses, as increasing numbers of observations at both ends of the PS distribution were excluded from the weighted analysis, weighted estimates approached the matched estimate (weighted RR 1.04, 95% CI 0.77-1.39 after excluding all observations below the 5th percentile of the treated and above the 95th percentile of the untreated). Treatment appeared to have benefits only in the highest and lowest PS strata. CONCLUSIONS: Matched and weighted estimates differed due to incomplete matching, sensitivity of weighted estimates to extreme observations, and possibly treatment effect heterogeneity. PS analysis requires identifying the population and treatment effect of interest, selecting an appropriate implementation method, and conducting and reporting sensitivity analyses. Weighted estimation especially should include sensitivity analyses relating to influential observations, such as those treated contrary to prediction.
Authors: Tobias Kurth; Alexander M Walker; Robert J Glynn; K Arnold Chan; J Michael Gaziano; Klaus Berger; James M Robins Journal: Am J Epidemiol Date: 2005-12-21 Impact factor: 4.897
Authors: A John Rush; Maurizio Fava; Stephen R Wisniewski; Philip W Lavori; Madhukar H Trivedi; Harold A Sackeim; Michael E Thase; Andrew A Nierenberg; Frederic M Quitkin; T Michael Kashner; David J Kupfer; Jerrold F Rosenbaum; Jonathan Alpert; Jonathan W Stewart; Patrick J McGrath; Melanie M Biggs; Kathy Shores-Wilson; Barry D Lebowitz; Louise Ritz; George Niederehe Journal: Control Clin Trials Date: 2004-02
Authors: A John Rush; Madhukar H Trivedi; Stephen R Wisniewski; Jonathan W Stewart; Andrew A Nierenberg; Michael E Thase; Louise Ritz; Melanie M Biggs; Diane Warden; James F Luther; Kathy Shores-Wilson; George Niederehe; Maurizio Fava Journal: N Engl J Med Date: 2006-03-23 Impact factor: 91.245
Authors: Jeff Y Yang; Michael Webster-Clark; Jennifer L Lund; Robert S Sandler; Evan S Dellon; Til Stürmer Journal: Gastrointest Endosc Date: 2019-04-30 Impact factor: 9.427