CONTEXT: Only a patient and his or her family can judge many of the most important aspects of the doctor-patient interaction. This study evaluates the feasibility and reliability of children and their families assessing the quality of paediatricians' interactions using a rating instrument developed specifically for this purpose. METHODS: A reliability analysis using generalisability theory on the ratings from 352 doctor-patient interactions across different speciality clinics. RESULTS: Ratings were normally distributed. They were highest for 'overall' performance, and lowest for giving time to discuss the families' agenda. An appropriate sample of adults' ratings provided a reliable score (G = 0.7 with 15 raters), but children's ratings were too idiosyncratic to be reproducible (G = 0.36 with 15 raters). CONCLUSIONS AND FURTHER WORK: Accompanying adults can provide reliable ratings of doctors' interactions with children. Because an adult is usually present at the consultation their ratings provide a highly feasible and authentic approach. Sampling doctors' interactions from different clinics and with patients of both genders provides a universal picture of performance. The method is ideal to measure performance for in-training assessment or revalidation. Further work is in progress to evaluate the educational impact of feeding ratings back to the doctors being assessed, and their use in a range of clinical contexts.
CONTEXT: Only a patient and his or her family can judge many of the most important aspects of the doctor-patient interaction. This study evaluates the feasibility and reliability of children and their families assessing the quality of paediatricians' interactions using a rating instrument developed specifically for this purpose. METHODS: A reliability analysis using generalisability theory on the ratings from 352 doctor-patient interactions across different speciality clinics. RESULTS: Ratings were normally distributed. They were highest for 'overall' performance, and lowest for giving time to discuss the families' agenda. An appropriate sample of adults' ratings provided a reliable score (G = 0.7 with 15 raters), but children's ratings were too idiosyncratic to be reproducible (G = 0.36 with 15 raters). CONCLUSIONS AND FURTHER WORK: Accompanying adults can provide reliable ratings of doctors' interactions with children. Because an adult is usually present at the consultation their ratings provide a highly feasible and authentic approach. Sampling doctors' interactions from different clinics and with patients of both genders provides a universal picture of performance. The method is ideal to measure performance for in-training assessment or revalidation. Further work is in progress to evaluate the educational impact of feeding ratings back to the doctors being assessed, and their use in a range of clinical contexts.