Reliability of a computer-aided manual procedure for segmenting optical coherence tomography scans.

PURPOSE: To assess the within- and between-operator agreement of a computer-aided manual segmentation procedure for frequency-domain optical coherence tomography scans.
METHODS: Four individuals (segmenters) used a computer-aided manual procedure to mark the borders defining the layers analyzed in glaucoma studies. After training, they segmented two sets of scans, an Assessment Set and a Test Set. Each set had scans from 10 patients with glaucoma and 10 healthy controls. Based on an analysis of the Assessment Set, a set of guidelines was written. The Test Set was segmented twice with a ≥1 month separation. Various measures were used to compare test and retest (within-segmenter) variability and between-segmenter variability including concordance correlations between layer borders and the mean across scans (n = 20) of the mean of absolute differences between local border locations of individual scans, MEAN{mean( ΔLBL )}.
RESULTS: Within-segmenter reliability was good. The mean concordance correlations values for an individual segmenter and a particular border ranged from 0.999 ± 0.000 to 0.978 ± 0.084. The MEAN{mean( ΔLBL )} values ranged from 1.6 to 4.7 μm depending on border and segmenter. Similarly, between-segmenter agreement was good. The mean concordance correlations values for an individual segmenter and a particular border ranged from 0.999 ± 0.001 to 0.992 ± 0.023. The MEAN{mean( ΔLBL )} values ranged from 1.9 to 4.0 μm depending on border and segmenter. The signed and unsigned average positions were considerably smaller than the MEAN{mean( ΔLBL )} values for both within- and between-segmenter comparisons. Measures of within-segmenter variability were only slightly larger than those of between-segmenter variability.
CONCLUSIONS: When human segmenters are trained, the within-and between-segmenter reliability of manual border segmentation is quite good. When expressed as a percentage of retinal layer thickness, the results suggest that manual segmentation provides a reliable measure of the thickness of layers typically measured in studies of glaucoma.