Assessing interrater agreement from dependent data.

Estimation of interrater agreement for ordered categorical data is examined when the same sample is being assessed by various raters with different methods. We investigate the use of a latent model proposed by Qu, Piedmonte, and Medendorp (1995, Biomerics 51, 268-275) to estimate the correlation between raters for each method, and test for their equality. For each of the assessment methods, these correlations can be interpreted as the variance components of random effects representing subject and rater. This method is applied to an HIV study, in which the amount of ectopy on a woman's cervix is measured by both direct visual assessment and a computer planimetry method.