BACKGROUND: We present an approach to examiner calibration study design where the number of calibration subjects is based on a specified margin of error (half-width of the 95% confidence interval [CI]) of the percentage of agreement (exact and within 1 mm) for both intra- and interexaminer reliability assessments. METHODS: An experienced standard examiner (S) trained three dental hygienists (A, B, and C) in correct procedures for obtaining a variety of periodontal measures. Duplicate measurements of probing depth (PD [mm]) and the free gingival margin to the cemento-enamel junction (CEJ-GM [mm]) were obtained in a pilot study to design a formal examiner calibration study, where sample sizes were adjusted for the effects of within-subject clustering of binary indices of agreement. RESULTS: Within-subject clustering of agreement indices resulted in an approximate four-fold increase in the variance of the estimates of percentage of agreement with the standard. PD and CEJ-GM percentage of exact agreement measurements (95% CI) for each examiner-standard pair, respectively, were as follows: AS=55% (48%, 61%) and 70% (62%, 78%); BS=52% (45%, 59%) and 73% (63%, 82%); and CS=55% (50%, 61%) and 72% (65%, 79%). The corresponding 95% CIs unadjusted for the effects of clustering underestimated the margin of error associated with the estimates of exact agreement by as much as 57% for PD and 68% for CEJ-GM. CONCLUSION: Failure to account for dependence among site-level agreement indices results in a false sense of precision in the resulting reliability estimates and can lead to faulty inference.
BACKGROUND: We present an approach to examiner calibration study design where the number of calibration subjects is based on a specified margin of error (half-width of the 95% confidence interval [CI]) of the percentage of agreement (exact and within 1 mm) for both intra- and interexaminer reliability assessments. METHODS: An experienced standard examiner (S) trained three dental hygienists (A, B, and C) in correct procedures for obtaining a variety of periodontal measures. Duplicate measurements of probing depth (PD [mm]) and the free gingival margin to the cemento-enamel junction (CEJ-GM [mm]) were obtained in a pilot study to design a formal examiner calibration study, where sample sizes were adjusted for the effects of within-subject clustering of binary indices of agreement. RESULTS: Within-subject clustering of agreement indices resulted in an approximate four-fold increase in the variance of the estimates of percentage of agreement with the standard. PD and CEJ-GM percentage of exact agreement measurements (95% CI) for each examiner-standard pair, respectively, were as follows: AS=55% (48%, 61%) and 70% (62%, 78%); BS=52% (45%, 59%) and 73% (63%, 82%); and CS=55% (50%, 61%) and 72% (65%, 79%). The corresponding 95% CIs unadjusted for the effects of clustering underestimated the margin of error associated with the estimates of exact agreement by as much as 57% for PD and 68% for CEJ-GM. CONCLUSION: Failure to account for dependence among site-level agreement indices results in a false sense of precision in the resulting reliability estimates and can lead to faulty inference.
Authors: Hon K Yuen; Nicole M Marlow; Susan G Reed; Samantha Mahoney; Lisa M Summerlin; Renata Leite; Elizabeth Slate; Richard M Silver Journal: Disabil Rehabil Date: 2011-09-27 Impact factor: 3.033
Authors: Jyotika K Fernandes; Ryan E Wiegand; Carlos F Salinas; Sara G Grossi; John J Sanders; Maria F Lopes-Virella; Elizabeth H Slate Journal: J Periodontol Date: 2009-07 Impact factor: 6.993