OBJECTIVE: The quality of clinical care is often assessed by retrospective examination of case-notes (charts, medical records). Our objective was to determine the inter-rater reliability of case-note audit. METHODS: We conducted a systematic review of the inter-rater reliability of case-note audit. Analysis was restricted to 26 papers reporting comparisons of two or three raters making independent judgements about the quality of care. RESULTS: Sixty-six separate comparisons were possible, since some papers reported more than one measurement of reliability. Mean kappa values ranged from 0.32 to 0.70. These may be inflated due to publication bias. Measured reliabilities were found to be higher for case-note reviews based on explicit, as opposed to implicit, criteria and for reviews that focused on outcome (including adverse effects) rather than process errors. We found an association between kappa and the prevalence of errors (poor quality care), suggesting alternatives such as tetrachoric and polychoric correlation coefficients be considered to assess inter-rater reliability. CONCLUSIONS: Comparative studies should take into account the relationship between kappa and the prevalence of the events being measured.
OBJECTIVE: The quality of clinical care is often assessed by retrospective examination of case-notes (charts, medical records). Our objective was to determine the inter-rater reliability of case-note audit. METHODS: We conducted a systematic review of the inter-rater reliability of case-note audit. Analysis was restricted to 26 papers reporting comparisons of two or three raters making independent judgements about the quality of care. RESULTS: Sixty-six separate comparisons were possible, since some papers reported more than one measurement of reliability. Mean kappa values ranged from 0.32 to 0.70. These may be inflated due to publication bias. Measured reliabilities were found to be higher for case-note reviews based on explicit, as opposed to implicit, criteria and for reviews that focused on outcome (including adverse effects) rather than process errors. We found an association between kappa and the prevalence of errors (poor quality care), suggesting alternatives such as tetrachoric and polychoric correlation coefficients be considered to assess inter-rater reliability. CONCLUSIONS: Comparative studies should take into account the relationship between kappa and the prevalence of the events being measured.
Authors: Amirta Benning; Mary Dixon-Woods; Ugochi Nwulu; Maisoon Ghaleb; Jeremy Dawson; Nick Barber; Bryony Dean Franklin; Alan Girling; Karla Hemming; Martin Carmalt; Gavin Rudge; Thirumalai Naicker; Amit Kotecha; M Clare Derrington; Richard Lilford Journal: BMJ Date: 2011-02-03
Authors: Amirta Benning; Maisoon Ghaleb; Anu Suokas; Mary Dixon-Woods; Jeremy Dawson; Nick Barber; Bryony Dean Franklin; Alan Girling; Karla Hemming; Martin Carmalt; Gavin Rudge; Thirumalai Naicker; Ugochi Nwulu; Sopna Choudhury; Richard Lilford Journal: BMJ Date: 2011-02-03
Authors: D Weller; P Vedsted; G Rubin; F M Walter; J Emery; S Scott; C Campbell; R S Andersen; W Hamilton; F Olesen; P Rose; S Nafees; E van Rijswijk; S Hiom; C Muth; M Beyer; R D Neal Journal: Br J Cancer Date: 2012-03-13 Impact factor: 7.640
Authors: Helen Hogan; Frances Healey; Graham Neale; Richard Thomson; Charles Vincent; Nick Black Journal: BMJ Qual Saf Date: 2012-09 Impact factor: 7.035