Sherif El-Kady1, Dara Petel1, Robert Baird2. 1. Department of Pediatric General and Thoracic Surgery, The Montreal Children's Hospital, McGill University, Montreal Quebec, Canada; McGill University, Montreal Quebec, Canada; Department of Radiology, The Montreal Children's Hospital, McGill University, Montreal Quebec, Canada. 2. Department of Pediatric General and Thoracic Surgery, The Montreal Children's Hospital, McGill University, Montreal Quebec, Canada; McGill University, Montreal Quebec, Canada; Department of Radiology, The Montreal Children's Hospital, McGill University, Montreal Quebec, Canada. Electronic address: robert.baird@mail.mcgill.ca.
Abstract
PURPOSE: Abdominal radiographs are frequently employed in the surveillance of patients with necrotizing enterocolitis (NEC), with typical findings well described. Clinicians interpret and act upon these films at different intervals, however, and inter-rater agreement has not been evaluated to date. METHODS: Thirty abdominal radiographs of premature infants were distributed to attending radiologists (4), pediatric surgeons (4), and trainees (4), who evaluated for findings of NEC from a list of eight potential choices (1=normal, 8=perforation). Fleiss's Kappa (FK) was used to evaluate concordance between multiple raters with 0-0.2=slight association and 0.8-1=almost perfect agreement. RESULTS: Practicing surgeons had an FK of 0.77 overall (95% CI: 0.67-0.87), but demonstrated poor agreement when evaluating decubitus films (FK: 0.39, 95% CI:0.12-0.65). Radiologists had excellent inter-rater agreement (FK: 0.81, 95% CI: 0.74-0.88), but had only modest agreement with surgeons (FK: 0.59, 95% CI: 0.56-0.63) and poor agreement for decubitus films (FK: 0.15, 95% CI: 0.47-0.26). Surgical and radiology trainees had fair agreement with their respective attendings (0.60, 95% CI: 0.55-0.65 and 0.64, 95% CI: 0.60-0.69, respectively). CONCLUSIONS: While inter-rater agreement was good-excellent among attending staff, it was only moderate between radiologists and surgeons and between trainees and their attendings. This highlights the importance of inter-disciplinary and hierarchical communication to optimize clinical decision-making. Decubitus films may be of limited value in evaluating patients with NEC.
PURPOSE: Abdominal radiographs are frequently employed in the surveillance of patients with necrotizing enterocolitis (NEC), with typical findings well described. Clinicians interpret and act upon these films at different intervals, however, and inter-rater agreement has not been evaluated to date. METHODS: Thirty abdominal radiographs of premature infants were distributed to attending radiologists (4), pediatric surgeons (4), and trainees (4), who evaluated for findings of NEC from a list of eight potential choices (1=normal, 8=perforation). Fleiss's Kappa (FK) was used to evaluate concordance between multiple raters with 0-0.2=slight association and 0.8-1=almost perfect agreement. RESULTS: Practicing surgeons had an FK of 0.77 overall (95% CI: 0.67-0.87), but demonstrated poor agreement when evaluating decubitus films (FK: 0.39, 95% CI:0.12-0.65). Radiologists had excellent inter-rater agreement (FK: 0.81, 95% CI: 0.74-0.88), but had only modest agreement with surgeons (FK: 0.59, 95% CI: 0.56-0.63) and poor agreement for decubitus films (FK: 0.15, 95% CI: 0.47-0.26). Surgical and radiology trainees had fair agreement with their respective attendings (0.60, 95% CI: 0.55-0.65 and 0.64, 95% CI: 0.60-0.69, respectively). CONCLUSIONS: While inter-rater agreement was good-excellent among attending staff, it was only moderate between radiologists and surgeons and between trainees and their attendings. This highlights the importance of inter-disciplinary and hierarchical communication to optimize clinical decision-making. Decubitus films may be of limited value in evaluating patients with NEC.
Authors: Priya G Sharma; Dhanashree A Rajderkar; Christopher L Sistrom; Roberta M Slater; Anthony A Mancuso Journal: Br J Radiol Date: 2022-01-27 Impact factor: 3.629