Workplace based assessment: A review of available tools and their relevance.

Workplace-based assessment (WPBA) appears to be a promising tool for more comprehensive assessment of the learners. Relevant literature in this direction was collated and analyzed for its relevance, salience, and merit. Many WPBA tools are being used by various educational institutions which spans across multiple domains and over the entire duration of the workplace learning. It caters to holistic assessment with structured measures, real-time feedback, and continuous professional development. It is being used more for formative assessment and has limited utility in summative assessment as on date. WPBA tools have promising prospect in bringing novelty, objectivity, and holistic approach in assessment. Copyright:

Science is an alibi to many researches/endeavors which gave excellent result at inception in ideal test scenario; but failed to deliver or shimmer to expectation in the real-time working environment. Any person or entity is understood holistically in the context of its complex/dynamic interaction with the environment. In fact, often mere observation of a person' s naturally occurring learning in complex interplay with its environment, give the fair assessment of the shape the clay is going to take under guided hands of the potter and the shimmering heat of the furnace. Workplace based assessment (WPBA) is borne out of such desire to assess the true and holistic competence of learner in its working milieu which might not be assessed by a discipline-specific classroom setting paper-based assessment.[1]

SCOPE

It is quite applicable to post graduate residents working at various clinical and paraclinical steam who are shaped under supervision in real-time working environment from a novice to a competent practitioner. On any routine working day a medical or surgical resident would go through many a learning event/opportunity. At an outpatient department, he/she would be conducting targeted physical examination of patients, making treatment plan and rendering it, explaining to individual and family about illness and nuances of treatment and interacting with fellow residents and staff in the team. They would revisit specific components of history and examination to ascertain the progress. If required, they would deliver bad news with desired diligence and empathy. Pending discharge of the patient, the resident would be addressing predischarge administrative and medical formalities and rendering advice with respect to various psychosocial and medical management during follow-up. During all this while he/she would be liaising or interacting with colleagues and faculty of other stream for various multidisciplinary component in management requiring a specific set of communication skills and respect for each other.[2]

ADVANTAGES

There are many advantages of WPBA:[34567]

The assessment of individual in its working environment. It brings the assessment closer to the true environment and reduces artificiality

Assessment may be woven around the work without much disruption in work or any additional effort

Assessment of multitude of skill over variety of cases and settings possible

Holistic assessment of competencies are possible across preventive to therapeutic to rehabilitative spectrum; education, management to follow-up, patient orientation to family-community orientation, etc.,

Easy to assess higher order thinking and echelons in Miller's pyramid

Assessment of learners' addressal of ethical and legal issues in their work

Assessment is continuous in nature through a course of time and complexity in issues of management

Gives emphasis on other aspect of learning necessary for holistic management, i.e., record keeping, taking the team along, communicating with patients and peers, understanding larger view of health than addressing mere illness

Gives opportunity of assessment and feedback my multiple assessors

Gives opportunity for active involvement of learner in patient selection, decision making. Lerner-centric

It also gives structure to “naturally occurring learning” in work environment for observation and assessments.

Things which are required to be kept in mind during the implementation of WPBA in an institution:[67891011]

It should be able to address the desired competencies

It should be able to yield data or feedback for reflection and appropriate plan for future

It should be able to help in certification of certain competencies

It should be able to identify gap in knowledge and skill for continuous development

The practice environment should be safe and free of any possible litigation

It should be easy to conduct and acceptable to broad range of participants (patients, learners, assessors)

Designed tool should be adequately reliable and valid

It ranges across a wide matrix of competencies.

UTILITY INDEX

As per Van der Vleuten, one should CARVE an effective assessment strategy. The mnemonics stand for following:[12]

Cost-effectiveness - With regards to infrastructure, workforce training, etc.

Acceptability - To all involved in the process of assessment.

Reliability - It should yield consistent result across measure and time.

Validity - Ability of tool to measure what is intended to.

Educational impact - The assessment (formative/summative) should result in the promotion of learning or translation to competent practice.

C × A × R × V × E.

TYPES OF WPBA TOOLS

There are various types of WPBA tools:[26713141516171819]

Mini clinical evaluation exercise (Mini CEX): A direct observation tool of a cross-sectional encounter; commonly used in clinical setting. Structured pro forma for assessment of skill, attitude, and behavior. Immediate feedback is given to the learner by the assessor[67]

Direct observation of procedural skill: A direct observation tool of a cross-sectional encounter; commonly used in the assessment of procedures in clinical and paraclinical setting. Structured pro forma for the assessment of skill. Immediate feedback is given to the learner by the assessor[6]

Clinical encounter card: Cards to record salient aspects of examination and management of patients seen during the work for later perusal, assessment, and feedback by the faculty[214]

Clinical work sampling: Direct observation, assessment and feedback by faculty on the learner's patient management across multiple domains of patient care (i.e., admission, management in the ward, and predischarge)[14]

Acute care assessment tool: Structured assessment of learner's ability during the period of work at acute ward, night shift. The observation is longitudinal in nature with feedback by the assessor given at the end[67]

Consultation observation tool (COT): Similar to Mini CEX. Often used in primary care setting. Structured assessment and immediate feedback by assessor on the performance of a learner in a face-to-face consultation[7]

Audio-COT: It is COT conducted on leaner performing telephonic consultation

Clinical examination and procedural skill: Direct observation and assessment of learner's interpretation of physical examination and conduct of procedural skill. Commonly used in the assessment of examination of breast, male/female genitalia, rectum, prostate, etc.[7]

Case-based discussion: Structured discussion of case management involving data collection, decision-making, administrative and team management, addressing ethic, and holistic approach[67]

Multisource feedback (MSF): Sometime referred to as 360° feedback, in this, there is structured assessment of performance of learner by the peer, team member or even patient with regards to his/her professional conduct and relationship. Anonymized feedback is given for the learner to reflect on and make suitable improvement. Various other structured tool addressing this domain are mini peer assessment tool, team assessment of behavior, and patient satisfaction questionnaire[67]

Quality improvement project/audit assessment tool: It measures learners' ability to successfully conduct a quality improvement project/audit. Various descriptors for the assessment of this competency could be topic selection, identification of right criteria/measures, methodology used, changes brought into implementation, evaluation of change and future implication, presentation skill and quality of project in general. Feedback is provided based on the given criteria[67]

Leadership MSF: It can be used for the assessment of learner's/trainee's organizational skills, owning of responsibility, effective management of team, etc., With leadership MSF anonymized feedback may be taken from the peer and team for reflection and improvement on part of the learner[7]

Clinical supervisor report: A report given by clinical supervisor about a learner work in the hospital based on short structured pro forma. Four domain clusters which are addressed are relationship, diagnostic, management, and professionalism. Feedback on the observation is given[7]

Educational supervisor's review: Periodic review by educational supervisor to assess overall progress, identifies area of focused input and feedback; based on performance on various WPBA tools and “naturally occurring evidence”[7]

Learning logs: It is a comprehensive record of various works done in pursuance of learning competencies, reflection on the work/feedback given, evidence of continuous professional development, and relevant supportive document. It gives a longitudinal overview of quantity and quality of progress[27]

Portfolio: It stores all documents generated through workplace-based assessments, record of clinical experiences, reflections, relevant meetings attended, informal of formal training events, critical incidents, etc., Structure and purpose of this tool are to show overall quality and quantity of learning and progress. Portfolio can be paper (diary, etc.,) or electronic (e-portfolio, etc.,) one. Based on the functionality, these may be classified into reflective, developmental, assessment, or showcase portfolio.[27]

Other WPBA tools:

Nontechnical skill for surgeons: It is a rating of intraoperative nontechnical skill vital to safety of patients, i.e., situational awareness, decision-making, leadership, communication, and team work[20]

Anesthetist nontechnical skill: It is assessments of anesthetist's intraoperative nontechnical skill related to patient's safety. It emphasizes the importance of vigilance, clarity of communication, effective coordination, and timely anticipation[21]

Chart-stimulated recall: The use of charts of learner's patient encounters to explore clinical reasoning and knowledge application[2]

Objectively structured assessment of technical skills: Evaluator rated performance on a platform consisting of various technical skill sets.[22]

The comparative overview of relevant WPBA tools are as brought out in Table 1.

Table 1

Overview of salient features of common workplace-based assessment tools

Type of tools	Type of observation	Type of assessment	Temporal context	Type of learning event	Feedback
Mini CEX	Direct	Structured	Cross-sectional	Clinical skills	Immediate
		By assessor
DOPS	-do-	-do-	-do-	Procedural skills	-do-
CWS	-do-	-do-	Longitudinal	Case management	On completion
COT	-do-	-do-	Cross sectional	Face-to face consultation	Immediate
Audio-COT	-do-	-do-	-do-l	Teleconsultation	-do-
CEPS	-do-	-do-	-do-	Clinical and procedural skill	On completion
CEC	Input based	-do-	-do-	Clinical encounter	-do-
CbD	-do-	-do-	-do-	Holistic case management	-do-
MSF	-do-	Structured	longitudinal	Professional conduct and relationship	On completion
		By peer, team, patient			Anonymized
Leadership MSF	-do-	-do-	-do-	Leadership ability	-do-
QIPAT	-do-	Structured	longitudinal	Quality improvement or audit	On completion
		By assessor
CSR	Overall	Structured	-do-	Various domains of clinical learning	-do-
		By clinical supervisor
ESR	-do-	Structured	-do-	Overall academic progress	At periodic interval
		By educational supervisor	Periodic
Learning logs	-do-	Semi structured	Longitudinal	Record of progress	At regular intervals
		By team of faculty
Portfolios	-do-	-do-	-do-	Documents pertaining to learning events and trajectory	-do-

CEX – Clinical evaluation exercise; DOPS – Direct observation of procedural skill; CWS – Clinical work sampling; COT – Consultation observation tool; CEPS – Clinical examination and procedural skill; CEC – Clinical encounter card; CbD – Case based discussion; MSF – Multisource feedback; QIPAT – Quality improvement project/audit assessment tool; CSR – Clinical supervisor report; ESR – Educational supervisor’s review

DIVERSITY OF EXPERIENCE GROUP

The workplace-based exposure and assessment may further be contextualized to various other groups as under:[7]

Pediatric group

Geriatric group

Chronic care group

Emergencies

Vulnerable and disabled group

Population health.

GRADATION

Although the grading may be tailor made to the tools chosen, the learner may be graded under following broad heads:[7]

Much below the desired level of expectation

Below the desired level of expectation

Desired level of expectation

Above the desired level of expectation

Much above the desired level of expectation.

CONSTRAINTS AND CHALLENGES

Following constraints and challenges exit in this paradigm:[67232425]

Inability of assessors to spare time for supervision and WPBA from their busy schedule

Lack of adequate number of trained supervisors

Inadequate alignment of assessment tools with workplace culture and setting

Lack of desired standardization of tool for the given cultural or educational context

Lack of communication of change in the method of assessment to the concerned party. Poor change management

Conflicting evidence of effectiveness of reliable assessment tool and perceived unnecessary replacement of time-tested old methods of assessment

Variability in assessment over time, domains, and multiple assessors

Uneasiness of students and faculty toward acceptance of WPBA.

CURRENT STATUS OF WORKPLACE-BASED ASSESSMENT

WPBA is effective in day-to-day assessment of student's learning in naturally working environment. Significant benefit lies in the assessment of multiple domains including those which are not possible by traditional modalities.[19232627] The workplace-based approach in assessment is being used extensively by many educational institutes across the world including various Royal Colleges. The feedback by trainers, reflection by student and subsequent steps taken for onward learning gives the continuity in learning and has changed the concept form “assessment driven learning” to “learning driven assessment.” Although the WPBA tools have found its place in various formative assessments, the use of the same as summative assessment is still debated, limited to few situations only and yet not accepted by many an educational institution in its entirety. It is felt that its utility as a summative assessment tool will only be possible when these measures of assessments establishes their robustness reliability and validity; and the trainers are adequately competent in assessing students with these tools. Given the growing recognition of these tools and the benefits associated, that day does not seem too far.

CONCLUSION

WPBA tools are structured to semi-structured format to assess a range of knowledge and skills applications by a learner in its working environment. It gives emphasis to holistic concept of management, feedback-based reflection, continuous professional development, effective multidisciplinary integration, and wellness centric management of illness. It has shown promises as effective formative assessment tool but needs further time and validation for broader acceptance as summative assessment tool. Regardless of this debate, the WPBA tools has promising prospect in bringing novelty, structure, appreciation of finer nuances of concept while understanding holistic approach to management in a real work environment with true patients and ramification to the family/community at large. Addition of these tools in the current armamentarium of assessment tool is worthwhile.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.