Jennifer L Hefner1, Timothy R Huerta1,2, Ann Scheck McAlearney1,3, Barbara Barash1, Tina Latimer4, Susan D Moffatt-Bruce4,5. 1. Department of Family Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA. 2. Department of Biomedical Informatics, College of Medicine, The Ohio State University, Columbus, Ohio, USA. 3. Division of Health Services Management and Policy, College of Public Health, The Ohio State University, Columbus, Ohio, USA. 4. Quality and Operations, The Ohio State University Wexner Medical Center, Columbus, Ohio, USA. 5. Department of Surgery, The Ohio State University Wexner Medical Center, Columbus, Ohio, USA.
Abstract
Objective: Agency for Healthcare Research and Quality (AHRQ) software applies standardized algorithms to hospital administrative data to identify patient safety indicators (PSIs). The objective of this study was to assess the validity of PSI flags and report reasons for invalid flagging. Material and Methods: At a 6-hospital academic medical center, a retrospective analysis was conducted of all PSIs flagged in fiscal year 2014. A multidisciplinary PSI Quality Team reviewed each flagged PSI based on quarterly reports. The positive predictive value (PPV, the percent of clinically validated cases) was calculated for 12 PSI categories. The documentation for each reversed case was reviewed to determine the reasons for PSI reversal. Results: Of 657 PSI flags, 185 were reversed. Seven PSI categories had a PPV below 75%. Four broad categories of reasons for reversal were AHRQ algorithm limitations (38%), coding misinterpretations (45%), present upon admission (10%), and documentation insufficiency (7%). AHRQ algorithm limitations included 2 subcategories: an "incident" was inherent to the procedure, or highly likely (eg, vascular tumor bleed), or an "incident" was nonsignificant, easily controlled, and/or no intervention was needed. Discussion: These findings support previous research highlighting administrative data problems. Additionally, AHRQ algorithm limitations was an emergent category not considered in previous research. Herein we present potential solutions to address these issues. Conclusions: If, despite poor validity, US policy continues to rely on PSIs for incentive and penalty programs, improvements are needed in the quality of administrative data and the standardized PSI algorithms. These solutions require national motivation, research attention, and dissemination support.
Objective: Agency for Healthcare Research and Quality (AHRQ) software applies standardized algorithms to hospital administrative data to identify patient safety indicators (PSIs). The objective of this study was to assess the validity of PSI flags and report reasons for invalid flagging. Material and Methods: At a 6-hospital academic medical center, a retrospective analysis was conducted of all PSIs flagged in fiscal year 2014. A multidisciplinary PSI Quality Team reviewed each flagged PSI based on quarterly reports. The positive predictive value (PPV, the percent of clinically validated cases) was calculated for 12 PSI categories. The documentation for each reversed case was reviewed to determine the reasons for PSI reversal. Results: Of 657 PSI flags, 185 were reversed. Seven PSI categories had a PPV below 75%. Four broad categories of reasons for reversal were AHRQ algorithm limitations (38%), coding misinterpretations (45%), present upon admission (10%), and documentation insufficiency (7%). AHRQ algorithm limitations included 2 subcategories: an "incident" was inherent to the procedure, or highly likely (eg, vascular tumor bleed), or an "incident" was nonsignificant, easily controlled, and/or no intervention was needed. Discussion: These findings support previous research highlighting administrative data problems. Additionally, AHRQ algorithm limitations was an emergent category not considered in previous research. Herein we present potential solutions to address these issues. Conclusions: If, despite poor validity, US policy continues to rely on PSIs for incentive and penalty programs, improvements are needed in the quality of administrative data and the standardized PSI algorithms. These solutions require national motivation, research attention, and dissemination support.
Authors: Hude Quan; Bing Li; L Duncan Saunders; Gerry A Parsons; Carolyn I Nilsson; Arif Alibhai; William A Ghali Journal: Health Serv Res Date: 2008-08 Impact factor: 3.402
Authors: Carolyn De Coster; Hude Quan; Alan Finlayson; Min Gao; Patricia Halfon; Karin H Humphries; Helen Johansen; Lisa M Lix; Jean-Christophe Luthi; Jin Ma; Patrick S Romano; Leslie Roos; Vijaya Sundararajan; Jack V Tu; Greg Webster; William A Ghali Journal: BMC Health Serv Res Date: 2006-06-15 Impact factor: 2.655
Authors: Qi Chen; Amresh Hanchate; Michael Shwartz; Ann M Borzecki; Hillary J Mull; Marlena H Shin; Amy K Rosen Journal: Am J Med Qual Date: 2013-08-22 Impact factor: 1.852
Authors: Andrew D Boyd; Young Min Yang; Jianrong Li; Colleen Kenost; Mike D Burton; Bryan Becker; Yves A Lussier Journal: J Am Med Inform Assoc Date: 2014-09-03 Impact factor: 4.497