What levels of agreement can be expected between histopathologists assigning cases to discrete nominal categories? A study of the diagnosis of hyperplastic and adenomatous colorectal polyps.

AIMS: To assess the levels of agreement between histopathologists for a two-class nominal categorization process--the discrimination between hyperplastic and adenomatous colorectal polyps.
METHODS: Fifty hyperplastic and 50 adenomatous polyps received consecutively in the laboratory were categorized by nine histopathologists, and the level of agreement between all observers and the original diagnosis was assessed using kappa statistics.
RESULTS: For the eight observers with 11 months or more experience in histopathology, there was a high level of agreement with kappa statistics ranging from 0.84 to 0.98. This process was performed rapidly with an average of 13 to 22 seconds spent on each case. One observer with only 6-weeks' experience of histopathology had a lower overall level of agreement with kappa statistics ranging from 0.46 to 0.54, but the performance on the later cases was much higher.
CONCLUSIONS: The level of agreement in the distinction between hyperplastic and adenomatous colorectal polyps is high among histopathologists with at least moderate amounts of experience in histopathology. The one virtually naïve observer showed a marked learning response during the study without feedback on case outcome. This suggests that histopathologists are very reliable in assigning cases to distinct nominal categories and that learning of these processes occurs early in a histopathologist's career.