Douglas J Lorenz1, Mary Clyde Pierce2, Kim Kaczor3, Rachel P Berger4, Gina Bertocci5, Bruce E Herman6, Sandra Herr7, Kent P Hymel8, Carole Jenny9, John M Leventhal10, Karen Sheehan2, Noel Zuckerbraun4. 1. Department of Bioinformatics and Biostatistics, School of Public Health and Information Sciences, University of Louisville, Louisville, KY. Electronic address: djlore01@louisville.edu. 2. Division of Emergency Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL; Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL. 3. Division of Emergency Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL. 4. Department of Pediatrics, University of Pittsburgh, Children's Hospital of Pittsburgh of UPMC, Pittsburgh, PA. 5. Department of Bioengineering, J.B. Speed School of Engineering, University of Louisville, Louisville, KY. 6. Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, UT. 7. Division of Pediatric Emergency Medicine, University of Louisville, Louiville, KY. 8. Department of Pediatrics, Division of Child Abuse Pediatrics, Hershey, PA. 9. Department of Pediatrics, University of Washington, Seattle Children's Hospital, Seattle, WA. 10. Department of Pediatrics, Yale School of Medicine, New Haven, CT.
Abstract
OBJECTIVE: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information. STUDY DESIGN: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS. RESULTS: The interrater reliability of the panelists was high. The Kendall coefficient (95% CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status. CONCLUSIONS: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.
OBJECTIVE: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information. STUDY DESIGN: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS. RESULTS: The interrater reliability of the panelists was high. The Kendall coefficient (95% CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status. CONCLUSIONS: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.
Authors: Morgan A Valley; Kennon J Heard; Adit A Ginde; Dennis C Lezotte; Steven R Lowenstein Journal: Ann Emerg Med Date: 2012-03-07 Impact factor: 5.721
Authors: Mary C Pierce; Julia N Magana; Kim Kaczor; Douglas J Lorenz; Gabriel Meyers; Berkeley L Bennett; John T Kanegaye Journal: Ann Emerg Med Date: 2015-07-29 Impact factor: 5.721
Authors: Kent P Hymel; Amanda K Fingarson; Mary Clyde Pierce; Kim Kaczor; Kathi L Makoroff; Ming Wang Journal: Pediatr Emerg Care Date: 2022-03-02 Impact factor: 1.602