Andrew J A Keen1, Mark H Freeston. 1. Child and Family Mental Health Service, Royal Aberdeen Children's Hospital, Westburn Road, Aberdeen. Andrew.Keen@ARH.grampian.scot.nhs.uk
Abstract
BACKGROUND: Postgraduate courses on cognitive-behavioural therapy (CBT) assess various competencies using essays, case studies and audiotapes or videotapes of clinical work. AIMS: To evaluate how reliably a well-established postgraduate course assesses CBT competencies. METHOD: Data were collected on two cohorts of trainees (n=52). Two examiners marked trainees on: (a) two videotapes of clinical practice; (b) two case studies; and (c) three essays. RESULTS: Essay examinations were more reliable than case studies, which in turn were more reliable than videotaped assessments. The reliability of the latter two assessments was considerably lower than that commonly expected of high-stakes examinations. To assess reliably standard CBT competencies, postgraduate courses would need to examine about 5 essays, 12 case studies and 19 videotapes. CONCLUSIONS: Reliable assessment of standard competencies is complex and resource intensive. There would need to be a marked increase in the number of samples of clinical work assessed to be able to make reliable judgements about proficiency.
BACKGROUND: Postgraduate courses on cognitive-behavioural therapy (CBT) assess various competencies using essays, case studies and audiotapes or videotapes of clinical work. AIMS: To evaluate how reliably a well-established postgraduate course assesses CBT competencies. METHOD: Data were collected on two cohorts of trainees (n=52). Two examiners marked trainees on: (a) two videotapes of clinical practice; (b) two case studies; and (c) three essays. RESULTS: Essay examinations were more reliable than case studies, which in turn were more reliable than videotaped assessments. The reliability of the latter two assessments was considerably lower than that commonly expected of high-stakes examinations. To assess reliably standard CBT competencies, postgraduate courses would need to examine about 5 essays, 12 case studies and 19 videotapes. CONCLUSIONS: Reliable assessment of standard competencies is complex and resource intensive. There would need to be a marked increase in the number of samples of clinical work assessed to be able to make reliable judgements about proficiency.
Authors: Lily A Brown; Michelle G Craske; Daniel E Glenn; Murray B Stein; Greer Sullivan; Cathy Sherbourne; Alexander Bystritsky; Stacy S Welch; Laura Campbell-Sills; Ariel Lang; Peter Roy-Byrne; Raphael D Rose Journal: Depress Anxiety Date: 2012-12-05 Impact factor: 6.505
Authors: Marloes J Huijbers; Rebecca S Crane; Willem Kuyken; Lot Heijke; Ingrid van den Hout; A Rogier T Donders; Anne E M Speckens Journal: Mindfulness (N Y) Date: 2017-01-12