Rebecca Brundin-Mather1, Andrea Soo2, Danny J Zuege3, Daniel J Niven4, Kirsten Fiest2, Christopher J Doig2, David Zygun5, Jamie M Boyd2, Jeanna Parsons Leigh6, Sean M Bagshaw7, Henry T Stelfox8. 1. W21C Research & Innovation Centre, Cumming School of Medicine, University of Calgary, GD01-TRW Building, 3280 Hospital Dr NW, Calgary, AB T2N 4Z6, Canada. 2. Department of Critical Care Medicine, University of Calgary, Foothills Medical Centre, Ground Floor-McCaig Tower, 1403-29 St NW, Calgary, AB T2N 5A1, Canada. 3. Department of Critical Care Medicine, University of Calgary, Foothills Medical Centre, Ground Floor-McCaig Tower, 1403-29 St NW, Calgary, AB T2N 5A1, Canada; eCritical Alberta Program, Alberta Health Services, Alberta, Canada. 4. Department of Critical Care Medicine, University of Calgary, Foothills Medical Centre, Ground Floor-McCaig Tower, 1403-29 St NW, Calgary, AB T2N 5A1, Canada; O'Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, 3rd Floor TRW Building, 3280 Hospital Dr NW, Calgary, AB T2N 4Z6, Canada; Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10, 3280 Hospital Dr NW, Calgary, Alberta T2N 4Z6, Canada. 5. Department of Critical Care Medicine, Faculty of Medicine and Dentistry and School of Public Health, University of Alberta, 2-124E Clinical Sciences Building, 8440-112 St NW, Edmonton, Alberta T6G 2B7, Canada; Alberta Health Services, Alberta, Canada. 6. Department of Critical Care Medicine, University of Calgary, Foothills Medical Centre, Ground Floor-McCaig Tower, 1403-29 St NW, Calgary, AB T2N 5A1, Canada; Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10, 3280 Hospital Dr NW, Calgary, Alberta T2N 4Z6, Canada. 7. Department of Critical Care Medicine, Faculty of Medicine and Dentistry and School of Public Health, University of Alberta, 2-124E Clinical Sciences Building, 8440-112 St NW, Edmonton, Alberta T6G 2B7, Canada; School of Public Health, University of Alberta, 3-300 Edmonton Clinic Health Academy, 11405-87 Ave Edmonton, Alberta T6G 1C9, Canada. 8. Department of Critical Care Medicine, University of Calgary, Foothills Medical Centre, Ground Floor-McCaig Tower, 1403-29 St NW, Calgary, AB T2N 5A1, Canada; O'Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, 3rd Floor TRW Building, 3280 Hospital Dr NW, Calgary, AB T2N 4Z6, Canada; Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3D10, 3280 Hospital Dr NW, Calgary, Alberta T2N 4Z6, Canada; Alberta Health Services, Alberta, Canada. Electronic address: tstelfox@ucalgary.ca.
Abstract
PURPOSE: This study measured the quality of data extracted from a clinical information system widely used for critical care quality improvement and research. MATERIALS AND METHODS: We abstracted data from 30 fields in a random sample of 207 patients admitted to nine adult, medical-surgical intensive care units. We assessed concordance between data collected: (1) manually from the bedside system (eCritical MetaVision) by trained auditors, and (2) electronically from the system data warehouse (eCritical TRACER). Agreement was assessed using Cohen's Kappa for categorical variables and intraclass correlation coefficient (ICC) for continuous variables. RESULTS: Concordance between data sets was excellent. There was perfect agreement for 11/30 variables (35%). The median Kappa score for the 16 categorical variables was 0.99 (IQR 0.92-1.00). APACHE II had an ICC of 0.936 (0.898-0.960). The lowest concordance was observed for SOFA renal and respiratory components (ICC 0.804 and 0.846, respectively). Score translation errors by the manual auditor were the most common source of data discrepancies. CONCLUSIONS: Manual validation processes of electronic data are complex in comparison to validation of traditional clinical documentation. This study represents a straightforward approach to validate the use of data repositories to support reliable and efficient use of high quality secondary use data.
PURPOSE: This study measured the quality of data extracted from a clinical information system widely used for critical care quality improvement and research. MATERIALS AND METHODS: We abstracted data from 30 fields in a random sample of 207 patients admitted to nine adult, medical-surgical intensive care units. We assessed concordance between data collected: (1) manually from the bedside system (eCritical MetaVision) by trained auditors, and (2) electronically from the system data warehouse (eCritical TRACER). Agreement was assessed using Cohen's Kappa for categorical variables and intraclass correlation coefficient (ICC) for continuous variables. RESULTS: Concordance between data sets was excellent. There was perfect agreement for 11/30 variables (35%). The median Kappa score for the 16 categorical variables was 0.99 (IQR 0.92-1.00). APACHE II had an ICC of 0.936 (0.898-0.960). The lowest concordance was observed for SOFA renal and respiratory components (ICC 0.804 and 0.846, respectively). Score translation errors by the manual auditor were the most common source of data discrepancies. CONCLUSIONS: Manual validation processes of electronic data are complex in comparison to validation of traditional clinical documentation. This study represents a straightforward approach to validate the use of data repositories to support reliable and efficient use of high quality secondary use data.
Authors: Samiha Mohsen; Stephana J Moss; Filipe Lucini; Karla D Krewulak; Henry T Stelfox; Daniel J Niven; Khara M Sauro; Kirsten M Fiest Journal: Crit Care Med Date: 2022-08-26 Impact factor: 9.296
Authors: Kirsten M Fiest; Henry T Stelfox; Anmol Shahid; Victoria S Owen; Bonnie G Sept; Shelly Longmore; Andrea Soo; Rebecca Brundin-Mather; Karla D Krewulak; Stephana J Moss; Kara M Plotnikoff; Céline Gélinas Journal: Pilot Feasibility Stud Date: 2022-07-16
Authors: Sean van Diepen; Tim Coulson; Xiaoming Wang; Dawn Opgenorth; Danny J Zuege; Jo Harris; Malik Agyemang; Daniel J Niven; Rinaldo Bellomo; Stephen E Wright; Paul J Young; Sean M Bagshaw Journal: Eur J Cardiothorac Surg Date: 2022-07-11 Impact factor: 4.534
Authors: K M Sauro; A Soo; A Kramer; P Couillard; J Kromm; D Zygun; D J Niven; S M Bagshaw; H T Stelfox Journal: Neurocrit Care Date: 2019-04 Impact factor: 3.210
Authors: Henry T Stelfox; Rebecca Brundin-Mather; Andrea Soo; Jeanna Parsons Leigh; Daniel J Niven; Kirsten M Fiest; Christopher James Doig; Danny J Zuege; Barry Kushner; Fiona Clement; Sharon E Straus; Deborah J Cook; Sean M Bagshaw; Khara M Sauro Journal: Intensive Care Med Date: 2019-02-01 Impact factor: 17.440
Authors: Filipe R Lucini; Karla D Krewulak; Kirsten M Fiest; Sean M Bagshaw; Danny J Zuege; Joon Lee; Henry T Stelfox Journal: J Am Med Inform Assoc Date: 2021-03-01 Impact factor: 4.497
Authors: Kyla N Brown; Andrea Soo; Peter Faris; Scott B Patten; Kirsten M Fiest; Henry T Stelfox Journal: Crit Care Date: 2020-07-31 Impact factor: 9.097