INTRODUCTION: This study continues our assessment of agreement and disagreement among 25 Chinese and 20 US orthodontists in the ranking for facial attractiveness of end-of-treatment photographs of randomly sampled growing Chinese and white orthodontic patients. The main aims of this article were to (1) measure the overall pattern of agreement between the mean rankings of US and Chinese orthodontists, and (2) measure the strength of agreement between the rankings of the US and Chinese orthodontists for each patient. METHODS: Each judge independently ranked standard clinical sets of profile, frontal, and frontal-smiling photographs of 43 US patients and 48 Chinese patients. For each patient, a separate mean rank was computed from the responses of each group of judges. Pearson correlations between the mean ranks of the 2 groups of judges were used to measure their overall agreement. Paired and unpaired t tests were used to measure the agreement between the judges of the 2 groups for each patient. RESULTS: The overall agreement between the mean rankings of the US and Chinese judges was very high. For the US patients, the correlation between the Chinese and US judges means was r = 0.92, P <0.0001. For the Chinese patients, the analogous value was r = 0.86, P <0.0001. Agreement between the 2 groups of judges concerning each patient was also generally strong. For two thirds of the patients, the mean ranks of the US and Chinese judges differed by less than 1 unit in a scale of 12. However, for 6 patients considered individually (5 Chinese and 1 US), the assessment of the 2 groups of judges was statistically significantly different at P values ranging from 0.02 to less than 0.0001, even after the Bonferroni correction. CONCLUSIONS: These findings demonstrate that orthodontic clinicians can reliably identify and rank subtle differences between patients, and that differences between judges and between patients can be distinguished at a high level of statistical significance, given appropriate study designs. However, the reasons clinicians give for the differences in their judgments are more difficult to investigate and will require further study.
INTRODUCTION: This study continues our assessment of agreement and disagreement among 25 Chinese and 20 US orthodontists in the ranking for facial attractiveness of end-of-treatment photographs of randomly sampled growing Chinese and white orthodontic patients. The main aims of this article were to (1) measure the overall pattern of agreement between the mean rankings of US and Chinese orthodontists, and (2) measure the strength of agreement between the rankings of the US and Chinese orthodontists for each patient. METHODS: Each judge independently ranked standard clinical sets of profile, frontal, and frontal-smiling photographs of 43 US patients and 48 Chinese patients. For each patient, a separate mean rank was computed from the responses of each group of judges. Pearson correlations between the mean ranks of the 2 groups of judges were used to measure their overall agreement. Paired and unpaired t tests were used to measure the agreement between the judges of the 2 groups for each patient. RESULTS: The overall agreement between the mean rankings of the US and Chinese judges was very high. For the US patients, the correlation between the Chinese and US judges means was r = 0.92, P <0.0001. For the Chinese patients, the analogous value was r = 0.86, P <0.0001. Agreement between the 2 groups of judges concerning each patient was also generally strong. For two thirds of the patients, the mean ranks of the US and Chinese judges differed by less than 1 unit in a scale of 12. However, for 6 patients considered individually (5 Chinese and 1 US), the assessment of the 2 groups of judges was statistically significantly different at P values ranging from 0.02 to less than 0.0001, even after the Bonferroni correction. CONCLUSIONS: These findings demonstrate that orthodontic clinicians can reliably identify and rank subtle differences between patients, and that differences between judges and between patients can be distinguished at a high level of statistical significance, given appropriate study designs. However, the reasons clinicians give for the differences in their judgments are more difficult to investigate and will require further study.