Adrian V Hernandez1,2, Yuani M Roman1,2, C Michael White3,4,5. 1. School of Pharmacy, University of Connecticut Evidence-based Practice Center, Storrs, CT, USA. 2. Department of Pharmacy, Hartford Hospital, Hartford, CT, USA. 3. School of Pharmacy, University of Connecticut Evidence-based Practice Center, Storrs, CT, USA. Charles.white@uconn.edu. 4. Department of Pharmacy, Hartford Hospital, Hartford, CT, USA. Charles.white@uconn.edu. 5. University of Connecticut School of Pharmacy, Storrs, CT, USA. Charles.white@uconn.edu.
Abstract
BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria. METHODS: We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria. RESULTS: The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (P < 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (P < 0.001). However, the final revision was not significantly improved over the first revision (P = 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable). CONCLUSION: Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ's support in that regard.
BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) could devote resources to collate and assess quality improvement studies to support learning health systems (LHS) but there is no reliable data on the consistency of data extraction for important criteria. METHODS: We identified quality improvement studies and evaluated the consistency of data extraction from two experienced independent reviewers at three time points: baseline, first revision (where explicit instructions for each criterion were created), and final revision (where the instructions were revised). Six investigators looked at the data extracted by the two systematic reviewers and determined the extent of similarity on a scale of 0 to 10 (where 0 represented no similarity and 10 perfect similarity). There were 42 assessments for baseline, 42 assessments for the first revision, and 42 assessments for the final revision. We asked two LHS participants to assess the relative value of our criteria. RESULTS: The consistency of extraction improved from 1.17 ± 1.85 at baseline to 6.07 ± 2.76 after revision 1 (P < 0.001) and to 6.81 ± 1.94 out of 10 for the final revision (P < 0.001). However, the final revision was not significantly improved over the first revision (P = 0.14). One key informant rated the difficulty in finding and using quality improvement studies a 6 (moderately difficult) while the other a 4 (moderately difficult). When asked how valuable it would be if AHRQ found and collated the demographic information about the health systems and the interventions used in published quality improvement studies, they rated it a 9 (highly valuable) and a 6 (moderately valuable). CONCLUSION: Creating explicit instructions for extracting data for quality improvement studies helps enhance the consistency of data extraction. This is important because it is difficult for LHS to vet these quality improvement studies on their own and they would value AHRQ's support in that regard.
Entities:
Keywords:
data extraction; learning health system; quality improvement
Authors: Greg Ogrinc; Louise Davies; Daisy Goodman; Paul Batalden; Frank Davidoff; David Stevens Journal: J Surg Res Date: 2015-09-28 Impact factor: 2.192
Authors: Tammy C Hoffmann; Paul P Glasziou; Isabelle Boutron; Ruairidh Milne; Rafael Perera; David Moher; Douglas G Altman; Virginia Barbour; Helen Macdonald; Marie Johnston; Sarah E Lamb; Mary Dixon-Woods; Peter McCulloch; Jeremy C Wyatt; An-Wen Chan; Susan Michie Journal: BMJ Date: 2014-03-07
Authors: Susanne Hempel; Paul G Shekelle; Jodi L Liu; Margie Sherwood Danz; Robbie Foy; Yee-Wei Lim; Aneesa Motala; Lisa V Rubenstein Journal: BMJ Qual Saf Date: 2015-08-26 Impact factor: 7.035
Authors: Hilary Pinnock; Melanie Barwick; Christopher R Carpenter; Sandra Eldridge; Gonzalo Grandes; Chris J Griffiths; Jo Rycroft-Malone; Paul Meissner; Elizabeth Murray; Anita Patel; Aziz Sheikh; Stephanie J C Taylor Journal: BMJ Open Date: 2017-04-03 Impact factor: 2.692