Christine Y Lu1, Robert B Penfold2, Sengwee Toh1, Jessica L Sturtevant1, Jeanne M Madden1,3, Gregory Simon4, Brian K Ahmedani5, Gregory Clarke6, Karen J Coleman7, Laurel A Copeland8, Yihe G Daida9, Robert L Davis10, Enid M Hunkeler11, Ashli Owen-Smith12,13, Marsha A Raebel14, Rebecca Rossom15, Stephen B Soumerai1, Martin Kulldorff16. 1. Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA. 2. Department of Health Services Research, Kaiser Permanente Washington Health Research Institute, University of Washington, Seattle, WA. 3. School of Pharmacy, Northeastern University, Boston, MA. 4. Kaiser Permanente Washington Health Research Institute, Seattle, WA. 5. Center for Health Policy and Health Services Research and Behavioral Health Services, Henry Ford Health System, Detroit, MI. 6. Center for Health Research, Kaiser Permanente Northwest, Portland, OR. 7. Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA. 8. Center for Applied Health Research, Baylor Scott & White Health jointly with Central Texas Veterans Health Care System, Temple, TX. 9. Center for Health Research, Kaiser Permanente Hawaii, Honolulu, HI. 10. Center for Biomedical Informatics, University of Tennessee Health Science Center, Memphis, TN. 11. Emeritus, Division of Research, Kaiser Permanente, Oakland, CA. 12. Health Management & Policy, Georgia State University School of Public Health, Atlanta, GA. 13. Kaiser Permanente Georgia, The Center for Clinical and Outcomes Research, Atlanta, GA. 14. Kaiser Permanente Colorado, Institute for Health Research, Denver, CO. 15. HealthPartners Institute, Bloomington, MN. 16. Department of Medicine, Division of Pharmacoepidemiology and Pharmacoeconomics, Harvard Medical School and Brigham and Women's Hospital, Boston, MA.
Abstract
BACKGROUND: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation. OBJECTIVE: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth. METHOD: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides. RESULTS: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship. CONCLUSIONS: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.
BACKGROUND: New health policies may have intended and unintended consequences. Active surveillance of population-level data may provide initial signals of policy effects for further rigorous evaluation soon after policy implementation. OBJECTIVE: This study evaluated the utility of sequential analysis for prospectively assessing signals of health policy impacts. As a policy example, we studied the consequences of the widely publicized Food and Drug Administration's warnings cautioning that antidepressant use could increase suicidal risk in youth. METHOD: This was a retrospective, longitudinal study, modeling prospective surveillance, using the maximized sequential probability ratio test. We used historical data (2000-2010) from 11 health systems in the US Mental Health Research Network. The study cohort included adolescents (ages 10-17 y) and young adults (ages 18-29 y), who were targeted by the warnings, and adults (ages 30-64 y) as a comparison group. Outcome measures were observed and expected events of 2 possible unintended policy outcomes: psychotropic drug poisonings (as a proxy for suicide attempts) and completed suicides. RESULTS: We detected statistically significant (P<0.05) signals of excess risk for suicidal behavior in adolescents and young adults within 5-7 quarters of the warnings. The excess risk in psychotropic drug poisonings was consistent with results from a previous, more rigorous interrupted time series analysis but use of the maximized sequential probability ratio test method allows timely detection. While we also detected signals of increased risk of completed suicide in these younger age groups, on its own it should not be taken as conclusive evidence that the policy caused the signal. A statistical signal indicates the need for further scrutiny using rigorous quasi-experimental studies to investigate the possibility of a cause-and-effect relationship. CONCLUSIONS: This was a proof-of-concept study. Prospective, periodic evaluation of administrative health care data using sequential analysis can provide timely population-based signals of effects of health policies. This method may be useful to use as new policies are introduced.
Authors: R Moynihan; L Bero; D Ross-Degnan; D Henry; K Lee; J Watkins; C Mah; S B Soumerai Journal: N Engl J Med Date: 2000-06-01 Impact factor: 91.245
Authors: Robert J Valuck; Anne M Libby; Heather D Orton; Elaine H Morrato; Richard Allen; Ross J Baldessarini Journal: Am J Psychiatry Date: 2007-08 Impact factor: 18.112
Authors: Christine Y Lu; Christine Stewart; Ameena T Ahmed; Brian K Ahmedani; Karen Coleman; Laurel A Copeland; Enid M Hunkeler; Matthew D Lakoma; Jeanne M Madden; Robert B Penfold; Donna Rusinak; Fang Zhang; Stephen B Soumerai Journal: Pharmacoepidemiol Drug Saf Date: 2014-02 Impact factor: 2.890
Authors: Mark C Hornbrook; Gene Hart; Jennifer L Ellis; Donald J Bachman; Gary Ansell; Sarah M Greene; Edward H Wagner; Roy Pardee; Mark M Schmidt; Ann Geiger; Amy L Butani; Terry Field; Hassan Fouayzi; Irina Miroshnik; Liyan Liu; Robert Diseker; Karen Wells; Rick Krajenta; Lois Lamerato; Christine Neslund Dudas Journal: J Natl Cancer Inst Monogr Date: 2005
Authors: John March; Susan Silva; Stephen Petrycki; John Curry; Karen Wells; John Fairbank; Barbara Burns; Marisa Domino; Steven McNulty; Benedetto Vitiello; Joanne Severe Journal: JAMA Date: 2004-08-18 Impact factor: 56.272
Authors: Anne M Libby; David A Brent; Elaine H Morrato; Heather D Orton; Richard Allen; Robert J Valuck Journal: Am J Psychiatry Date: 2007-06 Impact factor: 18.112
Authors: Bo Zhang; Wei Liu; Stephenie C Lemon; Bruce A Barton; Melissa A Fischer; Colleen Lawrence; Elizabeth J Rahn; Maria I Danila; Kenneth G Saag; Paul A Harris; Jeroan J Allison Journal: J Eval Clin Pract Date: 2019-08-19 Impact factor: 2.431
Authors: Wei Liu; Shangyuan Ye; Bruce A Barton; Melissa A Fischer; Colleen Lawrence; Elizabeth J Rahn; Maria I Danila; Kenneth G Saag; Paul A Harris; Stephenie C Lemon; Jeroan J Allison; Bo Zhang Journal: Contemp Clin Trials Commun Date: 2019-10-16