OBJECTIVES: We sought to test reliability of two approaches to classify adverse events (AEs) associated with helicopter EMS (HEMS) transport. METHODS: The first approach for AE classification involved flight nurses and paramedics (RN/Medics) and mid-career emergency physicians (MC-EMPs) independently reviewing 50 randomly selected HEMS medical records. The second approach involved RN/Medics and MC-EMPs meeting as a group to openly discuss 20 additional medical records and reach consensus-based AE decision. We compared all AE decisions to a reference criterion based on the decision of three senior emergency physicians (Sr-EMPs). We designed a study to detect an improvement in agreement (reliability) from fair (kappa = 0.2) to moderate (kappa = 0.5). We calculated sensitivity, specificity, percent agreement, and positive and negative predictive values (PPV/NPV). RESULTS: For the independent reviews, the Sr-EMP group identified 26 AEs while individual clinician reviewers identified between 19 and 50 AEs. Agreement on the presence/absence of an AE between Sr-EMPs and three MC-EMPs ranged from κ = 0.20 to κ = 0.25. Agreement between Sr-EMPs and three RN/Medics ranged from κ = 0.11 to κ = 0.19. For the consensus/open-discussion approach, the Sr-EMPs identified 13 AEs, the MC-EMP group identified 18 AEs, and RN/medic group identified 36 AEs. Agreement between Sr-EMPs and MC-EMP group was (κ = 0.30 95%CI -0.12, 0.72), whereas agreement between Sr-EMPs and RN/medic group was (κ = 0.40 95%CI 0.01, 0.79). Agreement between all three groups was fair (κ = 0.33, 95%CI 0.06, 0.66). Percent agreement (58-68%) and NPV (63-76%) was moderately dissimilar between clinicians, while sensitivity (25-80%), specificity (43-97%), and PPV (48-83%) varied. CONCLUSIONS: We identified a higher level of agreement/reliability in AE decisions utilizing a consensus-based approach for review rather than independent reviews.
OBJECTIVES: We sought to test reliability of two approaches to classify adverse events (AEs) associated with helicopter EMS (HEMS) transport. METHODS: The first approach for AE classification involved flight nurses and paramedics (RN/Medics) and mid-career emergency physicians (MC-EMPs) independently reviewing 50 randomly selected HEMS medical records. The second approach involved RN/Medics and MC-EMPs meeting as a group to openly discuss 20 additional medical records and reach consensus-based AE decision. We compared all AE decisions to a reference criterion based on the decision of three senior emergency physicians (Sr-EMPs). We designed a study to detect an improvement in agreement (reliability) from fair (kappa = 0.2) to moderate (kappa = 0.5). We calculated sensitivity, specificity, percent agreement, and positive and negative predictive values (PPV/NPV). RESULTS: For the independent reviews, the Sr-EMP group identified 26 AEs while individual clinician reviewers identified between 19 and 50 AEs. Agreement on the presence/absence of an AE between Sr-EMPs and three MC-EMPs ranged from κ = 0.20 to κ = 0.25. Agreement between Sr-EMPs and three RN/Medics ranged from κ = 0.11 to κ = 0.19. For the consensus/open-discussion approach, the Sr-EMPs identified 13 AEs, the MC-EMP group identified 18 AEs, and RN/medic group identified 36 AEs. Agreement between Sr-EMPs and MC-EMP group was (κ = 0.30 95%CI -0.12, 0.72), whereas agreement between Sr-EMPs and RN/medic group was (κ = 0.40 95%CI 0.01, 0.79). Agreement between all three groups was fair (κ = 0.33, 95%CI 0.06, 0.66). Percent agreement (58-68%) and NPV (63-76%) was moderately dissimilar between clinicians, while sensitivity (25-80%), specificity (43-97%), and PPV (48-83%) varied. CONCLUSIONS: We identified a higher level of agreement/reliability in AE decisions utilizing a consensus-based approach for review rather than independent reviews.
Authors: P Daniel Patterson; Judith R Lave; Christian Martin-Gill; Matthew D Weaver; Richard J Wadas; Robert M Arnold; Ronald N Roth; Vincent N Mosesso; Francis X Guyette; Jon C Rittenberger; Donald M Yealy Journal: Prehosp Emerg Care Date: 2013-09-04 Impact factor: 3.077
Authors: T A Brennan; L L Leape; N M Laird; L Hebert; A R Localio; A G Lawthers; J P Newhouse; P C Weiler; H H Hiatt Journal: N Engl J Med Date: 1991-02-07 Impact factor: 91.245
Authors: G Ross Baker; Peter G Norton; Virginia Flintoft; Régis Blais; Adalsteinn Brown; Jafna Cox; Ed Etchells; William A Ghali; Philip Hébert; Sumit R Majumdar; Maeve O'Beirne; Luz Palacios-Derflingher; Robert J Reid; Sam Sheps; Robyn Tamblyn Journal: CMAJ Date: 2004-05-25 Impact factor: 8.262
Authors: Magnus Andersson Hagiwara; Carl Magnusson; Johan Herlitz; Elin Seffel; Christer Axelsson; Monica Munters; Anneli Strömsöe; Lena Nilsson Journal: BMC Emerg Med Date: 2019-01-24