Interval estimation under two study designs for kappa with binary classifications.

Cornfield's test-based method of setting a confidence interval on a parameter associated with a two-by-two contingency table is adapted for use with the measure of agreement kappa. One-sided confidence intervals derived in this way are compared to other intervals proposed for kappa under two study designs. Both designs involve two ratings per subject on a dichotomous scale. In one design the same two raters make all evaluations; in the other, possibly different pairs of raters evaluate different subjects, or the same rater carries out a pair of independent assessments for each subject. It is shown through simulation that lower bounds based on Cornfield's test-based method attain the nominal coverage probability more often than other intervals proposed in the literature.