OBJECTIVE: The aim of this study was to measure interobserver agreement among cytologists on using a set of digital images. METHODS: A set of 90 selected Papanicolaou-stained cervical smears were digitalized and the digital images circulated among 117 readers, from laboratories spread across almost all Italian regions. Three representative fields of each smear were displayed at 20× and 40× magnification (overall six images for each case). The diagnoses made by the cytologists who provided the images were taken as target diagnoses. RESULTS: The κ values were: very low for the categories atypical squamous cells of undetermined significance (ASC-US), and atypical squamous cells - cannot exclude high-grade squamous intraepithelial lesion (ASC-H); poor for the categories atypical glandular cells (AGC), high-grade squamous intraepithelial lesion (HSIL) and invasive cancer; and fair to good for the categories negative and low-grade squamous intraepithelial lesion (LSIL). However, we found a cluster of 42 best concordant readers. The overall κ value and overall weighted κ with the target diagnosis for these 42 readers were 0.45 and 0.66, respectively. This finding is in contrast with the overall κ value and overall weighted κ for the other readers of 0.39 and 0.59, respectively. CONCLUSIONS: As this finding is an estimate of the accuracy of the readers, we can infer that it will be very important to reach this level of concordance for all the participating readers. Future effort will facilitate common experiences in order to improve the reproducibility of diagnostic criteria. Digital images could be the key to reach this aim.
OBJECTIVE: The aim of this study was to measure interobserver agreement among cytologists on using a set of digital images. METHODS: A set of 90 selected Papanicolaou-stained cervical smears were digitalized and the digital images circulated among 117 readers, from laboratories spread across almost all Italian regions. Three representative fields of each smear were displayed at 20× and 40× magnification (overall six images for each case). The diagnoses made by the cytologists who provided the images were taken as target diagnoses. RESULTS: The κ values were: very low for the categories atypical squamous cells of undetermined significance (ASC-US), and atypical squamous cells - cannot exclude high-grade squamous intraepithelial lesion (ASC-H); poor for the categories atypical glandular cells (AGC), high-grade squamous intraepithelial lesion (HSIL) and invasive cancer; and fair to good for the categories negative and low-grade squamous intraepithelial lesion (LSIL). However, we found a cluster of 42 best concordant readers. The overall κ value and overall weighted κ with the target diagnosis for these 42 readers were 0.45 and 0.66, respectively. This finding is in contrast with the overall κ value and overall weighted κ for the other readers of 0.39 and 0.59, respectively. CONCLUSIONS: As this finding is an estimate of the accuracy of the readers, we can infer that it will be very important to reach this level of concordance for all the participating readers. Future effort will facilitate common experiences in order to improve the reproducibility of diagnostic criteria. Digital images could be the key to reach this aim.