H Bella1, N S S Kornmann1, J T Hardwicke2, K L Wallis1, C Wearn2, T-L Su3, B M Richard4. 1. Birmingham Institute for Paediatric Plastic Surgery (BIPPS), Birmingham Children's Hospital NHS Foundation Trust, Steelhouse Lane, Birmingham, UK. 2. Birmingham Institute for Paediatric Plastic Surgery (BIPPS), Birmingham Children's Hospital NHS Foundation Trust, Steelhouse Lane, Birmingham, UK; School of Clinical and Experimental Medicine, University of Birmingham, Edgbaston, Birmingham, UK. 3. School of Dentistry, University of Manchester, Manchester, UK. 4. Birmingham Institute for Paediatric Plastic Surgery (BIPPS), Birmingham Children's Hospital NHS Foundation Trust, Steelhouse Lane, Birmingham, UK. Electronic address: bruce.richard@bch.nhs.uk.
Abstract
BACKGROUND AND AIM: The reproducible measurement of aesthetic outcomes after cleft lip and palate (CLP) surgery remains elusive, and there is no internationally recognised system. The aim of this pilot study was to better understand how humans rate post-operative aesthetic outcome after unilateral cleft lip and palate (UCLP) repair using a novel web-based rating platform with an extended panel of surgeon raters. METHODS: Cropped images of 5-year-old UCLP patients were arranged in a randomly generated sequence within a web-based aesthetic scoring tool as part of an agreement/reliability study. Assessors rated the appearances of patients using a five-point Likert-type scale on two occasions. A mixed-effect statistical model was adopted to analyse the effects of rater, image and timing. RESULTS: Images of 76 patients were scored by 29 UK-based cleft surgeons. Intra-rater variability was found, and the linear weighted kappa was 0.56. This allowed identification of the most and least consistent raters. The random image effect (p < 0.001) suggested that a broad range of aesthetic outcomes were included in the current study. Surgeon raters in this study were likely to score the images more preferably at the second assessment. CONCLUSIONS: A web-based scoring system provides extended data capture, and mixed-effect statistical modelling reveals the effect that time, image and rater have on the scorings. The selection and training of raters, in combination with an exemplary yardstick, might improve inter- and intra-rater agreement. The development of objective measures based upon digital facial recognition can replace the highly variable subjective human influence on rating the aesthetic outcome.
BACKGROUND AND AIM: The reproducible measurement of aesthetic outcomes after cleft lip and palate (CLP) surgery remains elusive, and there is no internationally recognised system. The aim of this pilot study was to better understand how humans rate post-operative aesthetic outcome after unilateral cleft lip and palate (UCLP) repair using a novel web-based rating platform with an extended panel of surgeon raters. METHODS: Cropped images of 5-year-old UCLP patients were arranged in a randomly generated sequence within a web-based aesthetic scoring tool as part of an agreement/reliability study. Assessors rated the appearances of patients using a five-point Likert-type scale on two occasions. A mixed-effect statistical model was adopted to analyse the effects of rater, image and timing. RESULTS: Images of 76 patients were scored by 29 UK-based cleft surgeons. Intra-rater variability was found, and the linear weighted kappa was 0.56. This allowed identification of the most and least consistent raters. The random image effect (p < 0.001) suggested that a broad range of aesthetic outcomes were included in the current study. Surgeon raters in this study were likely to score the images more preferably at the second assessment. CONCLUSIONS: A web-based scoring system provides extended data capture, and mixed-effect statistical modelling reveals the effect that time, image and rater have on the scorings. The selection and training of raters, in combination with an exemplary yardstick, might improve inter- and intra-rater agreement. The development of objective measures based upon digital facial recognition can replace the highly variable subjective human influence on rating the aesthetic outcome.
Authors: Nirvana S S Kornmann; Robin A Tan; Frans J Mulder; Joseph T Hardwicke; Bruce M Richard; Brian B Pigott; Ronald W Pigott Journal: Cleft Palate Craniofac J Date: 2018-11-21