CONTEXT: Ratings awarded by standardised patients (SPs) in UK objective structured clinical examinations (OSCEs) are typically based on humanistic (non-technical) skills and are complementary to clinician-examiner ratings. In psychometric terms, SP ratings appear to differ from examiner ratings and improve reliability. For the first time, we used qualitative methods from a constructivist perspective to explore SP experiences of rating, and consider how these impact our understanding of assessment. METHODS: We used constructivist grounded theory to analyse data from focus groups and individual semi-structured interviews with 38 SPs and four examiners. Inductive coding, theoretical sampling and constant comparison continued until theoretical saturation was achieved. RESULTS: Standardised patients assessed students on the core process of relationship building. Three theoretical categories informed this process. The SP identity was strongly vocational and was both enacted and reinforced through rating as SPs exerted their agency to protect future patients by promoting student learning. Expectations of performance drew on individual life experiences in formulating expectations of doctors against which students were measured, and the patient experience was a lens through which all interactions were refracted. Standardised patients experienced the examination as real rather than simulated. They rated holistically, prioritised individuality and person-centredness, and included technical skill because the perception of clinical competence was an inextricable part of the patient experience. CONCLUSIONS: The results can be used to reframe understanding of the SP role and of the psychometric discourse of assessment. Ratings awarded by SPs are socially constructed and reveal the complexity of the OSCE process and the unfeasibility of absolute objectivity or standardisation. Standardised patients valued individuality, subjective experience and assessment for learning. The potential of SPs is under-used their greater involvement should be used to promote real partnership as educators move into a post-psychometric era. New-generation assessments should strive to value subjective experience as well as psychometric data in order to utilise the significant potential for learning within assessment.
CONTEXT: Ratings awarded by standardised patients (SPs) in UK objective structured clinical examinations (OSCEs) are typically based on humanistic (non-technical) skills and are complementary to clinician-examiner ratings. In psychometric terms, SP ratings appear to differ from examiner ratings and improve reliability. For the first time, we used qualitative methods from a constructivist perspective to explore SP experiences of rating, and consider how these impact our understanding of assessment. METHODS: We used constructivist grounded theory to analyse data from focus groups and individual semi-structured interviews with 38 SPs and four examiners. Inductive coding, theoretical sampling and constant comparison continued until theoretical saturation was achieved. RESULTS: Standardised patients assessed students on the core process of relationship building. Three theoretical categories informed this process. The SP identity was strongly vocational and was both enacted and reinforced through rating as SPs exerted their agency to protect future patients by promoting student learning. Expectations of performance drew on individual life experiences in formulating expectations of doctors against which students were measured, and the patient experience was a lens through which all interactions were refracted. Standardised patients experienced the examination as real rather than simulated. They rated holistically, prioritised individuality and person-centredness, and included technical skill because the perception of clinical competence was an inextricable part of the patient experience. CONCLUSIONS: The results can be used to reframe understanding of the SP role and of the psychometric discourse of assessment. Ratings awarded by SPs are socially constructed and reveal the complexity of the OSCE process and the unfeasibility of absolute objectivity or standardisation. Standardised patients valued individuality, subjective experience and assessment for learning. The potential of SPs is under-used their greater involvement should be used to promote real partnership as educators move into a post-psychometric era. New-generation assessments should strive to value subjective experience as well as psychometric data in order to utilise the significant potential for learning within assessment.