Clotilde Gomond-Le Goff1, Laura Vivalda1, Silvia Foligno1, Barbara Loi1, Nadya Yousef1, Daniele De Luca2. 1. Division of Pediatrics and Neonatal Critical Care, "A. Béclère" Medical Center, South Paris University Hospitals, APHP, Paris, France. 2. Division of Pediatrics and Neonatal Critical Care, "A. Béclère" Medical Center, South Paris University Hospitals, APHP, Paris, France; Physiopathology and Therapeutic Innovation Unit-INSERM U999, South Paris-Saclay University, Paris, France. Electronic address: dm.deluca@icloud.com.
Abstract
BACKGROUND: The effect of different probes and operator experience on the reliability of lung ultrasound (LU) interpretation has not been investigated. We studied the effect of probes and operator experience on the interpretation reliability of LU in critically ill neonates. METHODS: This was a prospective, blind, cohort study enrolling patients with basic patterns ("B," "severe B," consolidation). Patients were scanned with microlinear (15 MHz; L15), phased-array sectorial (6-12 MHz; S7), and microconvex (8 MHz; C8) probes, in random order. Static images were acquired in high resolution, anonymized, and included in a pictorial database in random sequences. Seventeen clinicians with different LU experience were asked to blindly assess the pictorial database. Interrater agreement and interpretation reliability were analyzed. Subanalyses according to expertise and probe, and multivariate linear regression (including an "expertise × probe" interaction factor), were also performed. RESULTS: The agreement tends to be lower and more heterogeneous for residents (intraclass correlation coefficient [ICC], 0.82 [95% CI, 0.74-0.9], P < .001; I2, 67%, P = .04) and for fellows (ICC, 0.93 [95% CI, 0.9-0.97], P < .001; I2, 69%, P = .04), especially when using nonlinear probes, compared with senior physicians (ICC, 0.95 [95% CI, 0.93-0.96], P < .001; I2, 0%, P = .433). Area under the curve (AUC) values were high for all probes (L15, 0.96 [95% CI, 0.93-0.99]; C8, 0.91 [95% CI, 0.85-0.98]; S7, 0.86 [95% CI, 0.82-0.91]) and physicians (senior physicians, 0.95 [95% CI, 0.83-0.99]; fellows, 0.95 [95% CI, 0.75-0.99]; residents, 0.86 [95% CI, 0.5-0.99]). Worse reliability and higher heterogeneity were found when the evaluation was performed by residents (AUC, 0.9 [95% CI, 0.85-0.94], P < .01; I2, 93.6%, P < .001) than by fellows (AUC, 0.99 [95% CI, 0.9-0.999], P < .001; I2, 34.3%, P = .09) and/or by senior physicians (AUC, 0.99 [95% CI, 0.9-0.999], P < .001; I2, 18%, P = .236). The "expertise × probe" interaction factor was associated with lower ICC (standardized regression coefficient β, -0.69; P < .0001; adjusted R2, 0.99) and AUC (standardized regression coefficient β, -0.76; P < .0001; adjusted R2, 0.98). CONCLUSIONS: LU interpretation in neonates shows good interrater agreement and reliability, irrespective of the probe and rater expertise. The use of nonlinear probes by novice operators is associated with the lowest agreement and reliability.
BACKGROUND: The effect of different probes and operator experience on the reliability of lung ultrasound (LU) interpretation has not been investigated. We studied the effect of probes and operator experience on the interpretation reliability of LU in critically ill neonates. METHODS: This was a prospective, blind, cohort study enrolling patients with basic patterns ("B," "severe B," consolidation). Patients were scanned with microlinear (15 MHz; L15), phased-array sectorial (6-12 MHz; S7), and microconvex (8 MHz; C8) probes, in random order. Static images were acquired in high resolution, anonymized, and included in a pictorial database in random sequences. Seventeen clinicians with different LU experience were asked to blindly assess the pictorial database. Interrater agreement and interpretation reliability were analyzed. Subanalyses according to expertise and probe, and multivariate linear regression (including an "expertise × probe" interaction factor), were also performed. RESULTS: The agreement tends to be lower and more heterogeneous for residents (intraclass correlation coefficient [ICC], 0.82 [95% CI, 0.74-0.9], P < .001; I2, 67%, P = .04) and for fellows (ICC, 0.93 [95% CI, 0.9-0.97], P < .001; I2, 69%, P = .04), especially when using nonlinear probes, compared with senior physicians (ICC, 0.95 [95% CI, 0.93-0.96], P < .001; I2, 0%, P = .433). Area under the curve (AUC) values were high for all probes (L15, 0.96 [95% CI, 0.93-0.99]; C8, 0.91 [95% CI, 0.85-0.98]; S7, 0.86 [95% CI, 0.82-0.91]) and physicians (senior physicians, 0.95 [95% CI, 0.83-0.99]; fellows, 0.95 [95% CI, 0.75-0.99]; residents, 0.86 [95% CI, 0.5-0.99]). Worse reliability and higher heterogeneity were found when the evaluation was performed by residents (AUC, 0.9 [95% CI, 0.85-0.94], P < .01; I2, 93.6%, P < .001) than by fellows (AUC, 0.99 [95% CI, 0.9-0.999], P < .001; I2, 34.3%, P = .09) and/or by senior physicians (AUC, 0.99 [95% CI, 0.9-0.999], P < .001; I2, 18%, P = .236). The "expertise × probe" interaction factor was associated with lower ICC (standardized regression coefficient β, -0.69; P < .0001; adjusted R2, 0.99) and AUC (standardized regression coefficient β, -0.76; P < .0001; adjusted R2, 0.98). CONCLUSIONS: LU interpretation in neonates shows good interrater agreement and reliability, irrespective of the probe and rater expertise. The use of nonlinear probes by novice operators is associated with the lowest agreement and reliability.
Authors: Ryan L DeSanti; Eileen A Cowan; Pierre D Kory; Michael R Lasarev; Jessica Schmidt; Awni M Al-Subu Journal: J Ultrasound Med Date: 2021-08-11 Impact factor: 2.754
Authors: Javier Rodriguez-Fanjul; I Jordan; M Balaguer; A Batista-Muñoz; M Ramon; S Bobillo-Perez Journal: Eur J Pediatr Date: 2020-07-24 Impact factor: 3.183