Patient involvement in assessment: How useful is it?

A scoping review published in this issue of Medical Education by Khalife et al. highlights the benefits of involving patients in assessing postgraduate medical learners. The authors set the scene by stating that the authentic patient voice is still merely a whisper in contemporary competency‐based assessment in medical education despite the global movement towards greater patient partnership in defining a holistic medical learners' competence for 21st century health care. They postulate that patients can be effective assessors given the high correlations among themselves in rating the humanistic attributes of residents in professional behaviours, advocacy and communication skills and the lower correlations with clinicians who prioritise medical knowledge. The authors argue that there is a misalignment between the perspectives of patients and clinicians and that the extent to which unique insights from patients are taken up in postgraduate medical education may be dependent on the readiness of assessment systems, particularly the readiness of clinicians, to partner with patients as assessors.

Authentic patient voice is still merely a whisper in contemporary competency‐based assessment in medical education.

Although clinicians, as the central actors in assessment systems, have expressed uncertainty and concerns regarding ability, expertise and potential bias of patients in assessment, studies have shown that clinicians themselves are not totally free of bias in their judgement. As such, Khalife et al. reignite the debate about the usefulness of patient involvement in assessment within medical education. In this commentary, I will further contrast and compare the decision making of patients (standardised or real) with the processes employed by clinically trained examiners in an effort to help elucidate the usefulness of the patient's voice in competency‐based assessment and guide future research in this area.

Clinicians, as the central actors in assessment systems, have expressed uncertainty and concerns regarding ability, expertise and potential bias of patients in assessment.

Clinical performance assessment is a complex cognitive task that requires processing and assimilation of initial impressions with a pro‐active detection and selection of relevant performance elements that can burden working memory. Objective Structured Clinical Examinations (OSCEs) are commonly used for this purpose to remove the priority of patient care and to enable the evaluation of students using consistently presented scenarios. Our recent study confirms that medical schools are still keen to retain OSCEs, despite COVID related restrictions, due to their ability to enforce rigour and standardisation in the assessment process. In this context, assessor judgements are guided by prescribed expectations and scoring criteria, provided by assessment academics, outlined on the OSCE sheet. The judgements offered are not ‘objective’, however, as my colleagues and I have noted that the process of making judgements about student performance involves combining analytical and affective elements with an intuitive drive to rate candidates against a personal construct of a ‘prototypical intern’. In making such a comparison, the specific, time‐limited, and pre‐scripted nature of OSCE encounters may require assessors to adapt their expectations of learner performance to take into account situational constraints. In a follow‐up study, we explored OSCE assessors' perceptions of the ‘prototypical’ construct by applying a theoretical framework (Cultural Historical Activity Theory [CHAT]) that enabled examination of the complexity inherent in making assessment decisions. Interestingly, we found that clinical assessors make judgements that combine, to a varying extent, two interacting sets of roles and experiences that have different origins: The academic construct, achievement of expected graduate learning outcomes, and the clinical workplace construct, the ability to successfully and safely work within professional settings. If a goal is to evaluate professional expectations such as rapport‐building and patient safety, which emphasise person‐centred care, it makes sense to involve the patient in the decision‐making process. However, there may be important differences in the cognitive processes standardised patients (SPs) use in OSCE settings relative to those utilised by real patients in the workplace‐based assessment (WBA) setting.

If a goal is to evaluate professional expectations such as rapport‐building and patient safety, which emphasise person‐centred care, it makes sense to involve the patient in the decision‐making process.

Although SPs have been known to provide sound judgements on professional skills such as empathy and communication skills, , they do so through the lens of a list of medically enculturated checklists provided by faculty members with a focus on achieving standardisation, rather than prioritising the SPs' authentic opinions. As a consequence, other researchers have suggested that real patients outside the educational environment provide more valuable feedback that better prepares medical students for clinical practice. In fact, an exploration of the potential role of patients in assessment has revealed a disconnect between the views of real patients and medically enculturated examiners, with patients being able to better identify deficiencies in students' communication skills. Interestingly, this study has generated a lot of interest among the medical student body, who endorsed the involvement of patients in assessment but identified crucial areas of consideration, such as it being imperative for cultural factors to be considered if the data are to be used summatively. Based on anecdotal observation, these authors reported that the ratings they received from patients were less favourable when compared to peers from more extroverted cultures. In further consideration of relevant perspectives, Harris et al. challenged the non‐involvement of student opinions by pointing out that further research is needed to explicate the subtle differences in patients', examiners', and students' viewpoints ; patients were deemed critical stakeholders in clinical assessments if we are to optimise preparative training for patients and aid an equitable implementation of patient involvement.

The medical student body endorsed the involvement of patients in assessment but identified crucial areas of consideration.

Juxtaposing these findings with those of Khalife et al. unveils a field of enquiry that will elucidate a pathway to identifying best practices for involving real patients in OSCE and WBA competency‐based assessments. Greater understanding of the socio‐cultural perspectives, cognitive processes and sources of bias of all key stakeholders will feed into innovations that effectively increase assessment authenticity with the added potential to appropriately drive learning, professional development and clinical practice. Involving a diverse patient population in competency‐based assessment will provide a more authentic representation of the diverse society in which medical trainees will work.

Greater understanding of the socio‐cultural perspectives, cognitive processes and sources of bias of all key stakeholders will feed into innovations that effectively increase assessment authenticity.