Le Chi Chiu1, Mathew Leonardi1, Chuan Lu2, Brendan Mein3, Batool Nadim4, Shannon Reid1,5, Joanne Ludlow6, Ishwari Casikar1, George Condous1. 1. Acute Gynecology, Early Pregnancy, and Advanced Endosurgery Surgery Unit, Sydney Medical School Nepean, University of Sydney, Nepean Hospital, Kingswood, New South Wales, Australia. 2. Department of Computer Sciences, Aberystwyth University, Aberystwyth, Wales. 3. Perinatal Ultrasound Department, Nepean Hospital, Kingswood, New South Wales, Australia. 4. Nepean Clinical School, University of Sydney, Kingswood, New South Wales, Australia. 5. Department of Obstetrics and Gynecology, Liverpool Hospital, Liverpool, New South Wales, Australia. 6. Department of Obstetrics and Gynecology, Royal Prince Alfred Hospital, Sydney, New South Wales, Australia.
Abstract
OBJECTIVES: To investigate the diagnostic accuracy and interobserver agreement among sonologists when assessing offline ultrasound (US) video sets of the "sliding sign" and among gynecologic surgeons when assessing corresponding laparoscopic video sets to predict pouch of Douglas (POD) obliteration and to compare the performance of the groups. METHODS: A diagnostic and reproducibility study was conducted, including 15 observers in 4 groups: (1) senior sonologists, (2) junior sonologists, (3) general gynecologists, and (4) advanced laparoscopists. The sonologists viewed 25 offline preoperative US video sets of the sliding sign, and the surgeons viewed the corresponding intraoperative laparoscopic videos of the same patients. Each observer was asked to classify POD obliteration in the video sets and was compared to the reference standard POD state determined at real-time laparoscopy by a single investigator (G.C.). The interobserver correlation and diagnostic accuracy were evaluated among the 15 observers and 4 groups. The Cohen κ coefficient and Fleiss κ coefficient were used for the analysis. RESULTS: The overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for senior sonologists were 93.3%, 100%, 89.6%, 84.4%, and 100%, respectively; for junior sonologists, 70.0%, 88.9%, 59.4%, 55.2%, and 90.5%; for general gynecologists, 75.2%, 88.1%, 78.1%, 69.8%, and 91.9%; and for advanced laparoscopists, 82.4%, 91.9%, 90.8%, 82.9%, and 95.8%. The overall agreement between senior sonologists was almost perfect (Fleiss κ = 0.876); for junior sonologists and general gynecologists, it was moderate (Fleiss κ = 0.589 and 0.528); and for advanced laparoscopists, it was substantial (Fleiss κ = 0.652). CONCLUSIONS: Interobserver agreement was superior among senior sonologists. Prediction of POD obliteration using offline US videos by senior sonologists is comparable to offline assessments of laparoscopic videos by advanced laparoscopists for prediction of POD obliteration.
OBJECTIVES: To investigate the diagnostic accuracy and interobserver agreement among sonologists when assessing offline ultrasound (US) video sets of the "sliding sign" and among gynecologic surgeons when assessing corresponding laparoscopic video sets to predict pouch of Douglas (POD) obliteration and to compare the performance of the groups. METHODS: A diagnostic and reproducibility study was conducted, including 15 observers in 4 groups: (1) senior sonologists, (2) junior sonologists, (3) general gynecologists, and (4) advanced laparoscopists. The sonologists viewed 25 offline preoperative US video sets of the sliding sign, and the surgeons viewed the corresponding intraoperative laparoscopic videos of the same patients. Each observer was asked to classify POD obliteration in the video sets and was compared to the reference standard POD state determined at real-time laparoscopy by a single investigator (G.C.). The interobserver correlation and diagnostic accuracy were evaluated among the 15 observers and 4 groups. The Cohen κ coefficient and Fleiss κ coefficient were used for the analysis. RESULTS: The overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for senior sonologists were 93.3%, 100%, 89.6%, 84.4%, and 100%, respectively; for junior sonologists, 70.0%, 88.9%, 59.4%, 55.2%, and 90.5%; for general gynecologists, 75.2%, 88.1%, 78.1%, 69.8%, and 91.9%; and for advanced laparoscopists, 82.4%, 91.9%, 90.8%, 82.9%, and 95.8%. The overall agreement between senior sonologists was almost perfect (Fleiss κ = 0.876); for junior sonologists and general gynecologists, it was moderate (Fleiss κ = 0.589 and 0.528); and for advanced laparoscopists, it was substantial (Fleiss κ = 0.652). CONCLUSIONS: Interobserver agreement was superior among senior sonologists. Prediction of POD obliteration using offline US videos by senior sonologists is comparable to offline assessments of laparoscopic videos by advanced laparoscopists for prediction of POD obliteration.