Aman Prasad1, Max Shin1, Ryan M Carey2, Kevin Chorath2, Harman Parhar2, Scott Appel3, Alvaro Moreira4, Karthik Rajasekaran3. 1. Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America. 2. Department of Otorhinolaryngology, University of Pennsylvania, Philadelphia, PA, United States of America. 3. Biostatistics Analysis Center, University of Pennsylvania, Philadelphia, PA, United States of America. 4. Department of Pediatrics, University of Texas Health San Antonio, San Antonio, TX, United States of America.
Abstract
BACKGROUND: Propensity score techniques can reduce confounding and bias in observational studies. Such analyses are able to measure and balance pre-determined covariates between treated and untreated groups, leading to results that can approximate those generated by randomized prospective studies when such trials are not feasible. The most commonly used propensity score -based analytic technique is propensity score matching (PSM). Although PSM popularity has continued to increase in medical literature, improper methodology or methodological reporting may lead to biased interpretation of treatment effects or limited scientific reproducibility and generalizability. In this study, we aim to characterize and assess the quality of PSM methodology reporting in high-impact otolaryngologic literature. METHODS: PubMed and Embase based systematic review of the top 20 journals in otolaryngology, as measured by impact factor from the Journal Citations Reports from 2012 to 2018, for articles using PSM analysis throughout their publication history. Eligible articles were reviewed and assessed for quality and reporting of PSM methodology. RESULTS: Our search yielded 101 studies, of which 92 were eligible for final analysis and review. The proportion of studies utilizing PSM increased significantly over time (p < 0.001). Nearly all studies (96.7%, n = 89) specified the covariates used to calculate propensity scores. Covariate balance was illustrated in 67.4% (n = 62) of studies, most frequently through p-values. A minority (17.4%, n = 16) of studies were found to be fully reproducible according to previously established criteria. CONCLUSIONS: While PSM analysis is becoming increasingly prevalent in otolaryngologic literature, the quality of PSM methodology reporting can be improved. We provide potential recommendations for authors regarding optimal reporting for analyses using PSM.
BACKGROUND: Propensity score techniques can reduce confounding and bias in observational studies. Such analyses are able to measure and balance pre-determined covariates between treated and untreated groups, leading to results that can approximate those generated by randomized prospective studies when such trials are not feasible. The most commonly used propensity score -based analytic technique is propensity score matching (PSM). Although PSM popularity has continued to increase in medical literature, improper methodology or methodological reporting may lead to biased interpretation of treatment effects or limited scientific reproducibility and generalizability. In this study, we aim to characterize and assess the quality of PSM methodology reporting in high-impact otolaryngologic literature. METHODS: PubMed and Embase based systematic review of the top 20 journals in otolaryngology, as measured by impact factor from the Journal Citations Reports from 2012 to 2018, for articles using PSM analysis throughout their publication history. Eligible articles were reviewed and assessed for quality and reporting of PSM methodology. RESULTS: Our search yielded 101 studies, of which 92 were eligible for final analysis and review. The proportion of studies utilizing PSM increased significantly over time (p < 0.001). Nearly all studies (96.7%, n = 89) specified the covariates used to calculate propensity scores. Covariate balance was illustrated in 67.4% (n = 62) of studies, most frequently through p-values. A minority (17.4%, n = 16) of studies were found to be fully reproducible according to previously established criteria. CONCLUSIONS: While PSM analysis is becoming increasingly prevalent in otolaryngologic literature, the quality of PSM methodology reporting can be improved. We provide potential recommendations for authors regarding optimal reporting for analyses using PSM.
Authors: Thérèse A Stukel; Elliott S Fisher; David E Wennberg; David A Alter; Daniel J Gottlieb; Marian J Vermeulen Journal: JAMA Date: 2007-01-17 Impact factor: 56.272
Authors: Richard Blake Ross; Shlomo A Koyfman; Chandana A Reddy; Narcissa Houston; Jessica L Geiger; Neil M Woody; Nikhil P Joshi; John F Greskovich; Brian B Burkey; Joseph Scharpf; Eric D Lamarre; Brandon Prendes; Robert R Lorenz; David J Adelstein; Matthew C Ward Journal: Laryngoscope Date: 2017-10-31 Impact factor: 3.325