Workplace-based licensing assessments: an idea worth considering?

Increasing complexity in clinical practice is raising uncertainties with current assessments of clinical competence such as Objective Structured Clinical Examinations (OSCEs) that take place away from the clinical environment and thus may not adequately assess the spectrum of competencies which are inherent to a real patient encounter. Workplace-based assessments (WBAs) offer the advantage of being authentically embedded in real clinical practice and should be considered for high-stakes summative decisions such as medical licensure. However, this requires rethinking the dual-purposing of WBAs which are implemented as part of programmatic assessment and where the distinction between formative and summative assessment of learners has been blurred.

Rethinking dual-purposing assessment activities

In educational practice, learners and faculty remain unclear about the formative or summative nature of WBAs, leaving confusion as to whether learner performances during a WBA should be viewed as growth or performance events.[1] Conceptually, a growth event or orientation is one where the learner is willing to reveal performance gaps in order to further their professional development whereas a performance event or orientation is one where the learner minimizes any performance gaps in order to ‘pass’ the assessment. One solution to clear up the confusion that may arise from dual-purposed WBAs is to a provide a clear demarcation between when WBAs are formative and permit learners to engage in pure growth orientations from when WBAs are summative, high-stakes assessment activities that may result in performance orientations.[1] To support both growth and performance orientations, and to provide opportunities for Workplace-Based Licensing Assessments (WBLAs), we propose separating the formative and summative intent of WBAs by creating two distinct phases of assessment.

Foregrounding WBLAs as summative, performance-oriented, high-stakes assessment events holds the potential to free up the rest of the educational system to focus on formative, growth-oriented moments.[2] Educational programs could carve out clear developmental space for learning that is separated from summative assessment. This would flatten the power differential between learner and faculty supervisors, encouraging supervisors to be coaches and not judges and encouraging learners to seek out observation and feedback.[3] A WBLA model would include processes that permit a performance orientation for the system, where healthcare setting, educational institution, learners, faculty, and licensing body are aware that a specific encounter is part of a summative high-stakes assessment process; all other assessment instances and faculty roles are clearly growth and progress oriented.

A potential WBLA model

WBLAs should take place in the clinical workplace, using real patients and trained, dedicated and independent assessment faculty with clearly defined roles. The WBLAs should be implemented against an assessment blueprint, focused on the relevant competency framework. We anticipate multiple assessments taking place over time, across multiple patient encounters pre-selected for the candidate in their clinical workplace and assessed by dedicated assessment faculty. Each WBLA would be clearly labelled as summative assessment. This does not preclude learners benefitting formatively, but it emphasizes the primary summative intent.

WBLAs could be implemented at key developmental stages as outlined in competency frameworks. This approach embraces the concept of longitudinal assessment across a developmental growth-oriented learning trajectory[4] and positions multiple high-stakes summative assessments across the learning trajectory rather than a single high-stakes summative hurdle. We envision a process whereby educational programs use current assessment practices to provide the data needed to determine if a learner is ready for their WBLA at each developmental stage. When ready, the WBLA occurs and if successful, the learner progresses to the next developmental stage and is one step closer to licensure. If unsuccessful, remediation is undertaken and the WBLA re-attempted. WBLA could be implemented as one component of programmatic assessment informing licensure, in conjunction with other assessment modalities such as simulation, MCQs etc., that complete the assessment blueprint.

Validity and other considerations

A WBLA model is not without its challenges. First, while it is outside the scope of this commentary to deeply examine the validity argument in support of a distinct high-stakes summative WBA for medical licensure, we can broadly state that most existing WBAs were not created with the intent (in Kane’s parlance, an interpretation/use argument[5]) of assessing readiness for independent practice. To use WBAs with this summative intent would require collecting new validity evidence. Second, newer philosophical positions informing assessment[6] (e.g., interpretive or constructivist views) may need to be more widely adopted and provide an opportunity to revisit assumptions that have historically threatened WBLA proposals. For example, these broader philosophical positions could open up discussion around key assessment features such as whether rater variability is considered as richness or error, or around indicators of quality assessment, such as de-emphasizing reliability and emphasizing triangulation. Third, there are a variety of practical issues to consider (e.g., logistical issues, costs, overcoming reliability indicators as gold standard quality indicators), which we believe are ultimately surmountable.

Let’s move forward

The attention and advances brought by competency-based medical education, programmatic assessment, broadening philosophical positions on assessment, increasing complexity of practice, and the dual-purposing conundrum are a few of the reasons why it may be time to move forward with WBLA. Individual programs would still engage in programmatic assessment for the purposes of assessing growth through training, including maintaining local competence committees to assess learner progress. But by clearly signposting for all stakeholders the workplace assessments that are explicitly high-stakes, summative and performance-oriented, WBLA holds the potential to open up and protect other workplace assessments to be truly formative and growth-oriented.