Workplace-based assessment: how to use case-based discussion as a formative assessment.

Workplace-based assessments are increasingly used as a way of gaining insight into clinician performance in real-life situations. Although some can be used to inform a summative (pass/fail) assessment, many have a much greater role in the formative assessment of trainees, and can be used as tools for teaching and training and in identifying the development needs of trainees. There is considerable variation between different European countries in the use of formative, workplace-based assessment, such as a structured case-based discussion (CbD), during training. This article gives an overview of how to use CbD as a formative assessment for higher specialist trainees, and gives access to a downloadable record form which can be used by trainers.

Introduction

The HERMES (Harmonising Education in Respiratory Medicine for European Specialists) project was launched by the European Respiratory Society in 2005, with the aim of promoting harmonised education and training in respiratory medicine for European specialists [1]. Acceptable methods of assessing trainees in clinical competencies, including skills, attitudes and behaviours, were outlined by the paediatric HERMES task force in 2009, when they suggested an “assessment toolbox”, a series of tools which could be used to assess trainees in the workplace [2]. However, while a few countries in Europe use these workplace-based assessment tools routinely in training, many others still have no access to such methods, and no national requirements to use them. A recent needs assessment suggested that there was considerable enthusiasm among respiratory trainers to gain access to some of these assessment tools, and training in how to use them. This article explains the use of case-based discussion (CbD), and outlines its potential benefits to trainees and trainers.

What is formative assessment?

All medical graduates are used to summative assessment: the final examinations at the end of medical school, postgraduate examinations or an oral defence of a research thesis. The objective of a summative examination is a simple, usually binary outcome, pass or fail. In clinical practice the question is whether the trainee is competent to progress to the next stage of their career. Formative assessments are more of a training tool, used to identify the strengths of a trainee, and more importantly, the areas in which they need to improve their performance and develop their skills. They fulfil a teaching function in which the trainee is an active participant but can also be used to feed into an assessment of competency. While these types of assessments are probably happening informally during many of the normal working interactions between the supervisor and trainee, they can be much more useful if they are formalised and there is some record of the interaction. Formative assessment is most useful when it is a direct assessment of real-life functioning in the workplace, i.e. workplace-based assessment (WBA).

The conventional model of different levels of assessment is Miller's pyramid (figure 1) in which the lowest level is factual knowledge (“knows”), followed by integrated knowledge (“knows how”), then “shows how”, demonstrating competence in a simulated situation, and finally “does” [3]. It is this final level which we attempt to assess in WBA, exploring the way in which a clinician performs in normal practice. The advantage of this type of assessment is that it can take into account knowledge, skills and attitudes, and gives a realistic picture of actual performance, so it has a high validity. The disadvantage is that it is less reproducible than a simple cognitive assessment such as a multiple-choice examination and introduces the subjectivity of the assessor.

Figure 1

Miller's pyramid and prism of assessment. Reproduced and modified from [3] and [4], with permission from the publisher. DOPS: direct observation of procedural skill; OSCE: objective structured clinical examination; MCQ: ­multiple-choice question.

In most countries, to enter a programme of higher specialist training the trainee will have gone through a series of summative assessments, including entry to medical school, completion of medical training, performance during foundation training after graduation, and often postgraduate examinations before going through a selection process for the specialist training programme. We can therefore assume that almost all trainees have the ability to become specialists, given the appropriate training, and with a reasonable amount of effort on their part. Formative assessment is not primarily concerned with detection of the “failing” trainee (although it can perform that function); it is aimed at ensuring that the trainee is being helped to maximise their potential, broaden their skills and experience so that they emerge from training with as few gaps in their clinical competencies as possible.

What is case-based discussion?

Almost everyone involved in higher specialist training discusses cases with their trainees as part of the training process. Informally, this will be done during or after a ward round, or in an outpatient clinic: the trainee presents a summary of the case to his supervisor, who critiques and approves or adjusts the decision-making. It is often an opportunity to teach at the same time. There may be departmental meetings in which trainees present a case (often selected for clinical interest or rarity) for discussion with their colleagues and supervisors. How is this concept different?

CbD builds on these traditional methods, by identifying a period of protected time to carry out a focused, private, one-to-one discussion of a case with the trainee, identifying the trainee's strengths, and also making suggestions for development or further learning. The encounter should normally be scheduled to last 20–30 min, and will usually focus on one or two particular aspects of the case, e.g. clinical reasoning, management planning or communication.

How is it done?

The case is usually chosen by the trainee, but the supervisor can select a case if he or she is aware of an issue which would benefit from discussion. It should be a case seen recently (in the past week), and it should be one in which the trainee felt there was uncertainty, or a conflict in decision making. If the schedules permit it, the meeting can take place after a clinic or ward round, but can also occur at a pre-planned time, with a case chosen from the recent workload. It should take place in a private, protected environment, since the discussion needs to be free from constraints. The trainee brings the case notes, and presents a summary of the case, and the supervisor and trainee agree an area to focus on. The supervisor should try to explore the trainee's thinking and decision making, rather than giving a didactic tutorial. Prompts should encourage reflection and should be open and gently probing questions such as: “What diagnoses did you consider, and how did you reach the one you did?”, “What factors did you take into account in deciding on the treatment?”, “How did you feel the communication went?”, and perhaps “What might you do differently on reflection?”. The supervisor should avoid knowledge-based questions like: “What is the commonest cause of…?”, or “What is the most important side-effect of…”.

At the end of the session, the supervisor should spend a few minutes giving feedback to the trainee about what was done well and what might have been done better or differently. It is then important to suggest and then agree what might be useful actions for learning or development. It is often helpful to start this discussion by asking the trainee for their views on what they did well and what they were less happy with, as they are often aware of their own development needs.

The supervisor finally records the encounter on a structured form, which documents the seniority of the trainee, the setting (inpatient, outpatient, etc.), and the complexity of the case. The supervisor writes down the strengths and the suggestions for development, and finally gives an overall rating of the trainee's competence, based on this case discussion. Based on the concept of “entrustable professional activities” [5], this might involve an assessment of the level of supervision needed to manage a similar case in future (see table 1). The record is signed by both parties, and a copy is kept by both. The trainee can keep the record in their portfolio, and the trainee's educational supervisor should keep a copy. If the supervisor performing the CbD is not the normal educational supervisor, they should send the copy to the normal supervisor.

Table 1

An example of a supervisor subjective rating of level of trust

How much supervision do you feel this trainee needs for a future case of similar complexity?

Discuss each case with a supervisor before executing decisions.

Discuss all cases with a supervisor subsequently.

Discuss a case with a supervisor only if the trainee wishes.

No supervision needed. Can function as specialist.

Giving feedback

Many trainees are excessively self-critical, and will often focus on the negative aspects of the feedback. It is helpful to start with the strengths (what was done well) before dealing with any weaker areas, which can be referred to more positively as “areas for development”. When concluding it is a good idea to finish by reminding the trainee of the strengths, so they leave in a positive frame of mind. This “feedback sandwich” approach does not mean that the supervisor should not be rigorous in identifying areas for improvement. Research has shown that trainees value the CbD process most when the supervisor has identified these areas, and agreed specific action points for the trainee to pursue [6, 7]. Suggestions for development should be specific, realistic and measurable. “Learn more about asthma” is too vague to be helpful, whereas “update myself on the recommendations for biological agents in asthma” or “familiarise myself with the current BTS/SIGN/GINA guidelines on asthma” might be appropriate.

Why should we bother?

First, this form of formalised CbD is highly valued by trainees as a learning process [6-8], thus functioning as a formative assessment. It is also regarded as a valuable teaching process by most supervisors [8]. Importantly, the degree to which the trainees value the process is dependent on the supervisors' commitment to it, and their understanding of the need to give constructive and specific feedback.

Secondly, it allows the trainee and supervisor to have a record of strengths, weaknesses and level of performance, which can identify if there are consistent deficits or flaws which need to be addressed.

Thirdly, when training programmes move towards competency-based summative assessments it can be used by the trainee as evidence of competence: if the most recent CbDs are all at the level of trusting the trainee without supervision, then this is evidence that they are ready for independent practice.

Who should do it?

Ideally, it is most useful if a number of supervisors perform CbDs with the same trainee at different times. This allows more objectivity in the assessment component of the process, and may allow the detection of a consistent fault in the trainee which cannot be blamed on the likes and dislikes of a single observer. Obviously, this will depend on the number of supervisors available in any training institution.

Who should initiate it, and how often?

In most competency-based training programmes that use WBA, the trainee has a requirement to have completed a certain number of CbDs in each module, covering a broad range of case types and areas of the training curriculum. In this situation, the trainee is usually the initiator. However, a supervisor can trigger a CbD, especially if they feel that a trainee demonstrated a developmental need during an informal case review. Where there is no existing national training requirement, the introduction of CbD will probably be an individual decision by a supervisor or a training centre, and there should be some clear agreement from the outset about the frequency of CbDs and the responsibility for initiating them. The simplest way to ensure that they occur is to inform the trainee that they need to do a certain number to be signed off by the supervisors for that training period!

Time constraints are the usual factor limiting the numbers of CbDs being undertaken. One session of 20–30 min every 2 or 3 weeks might be an achievable goal. Of course, the informal case discussions will continue as before; this form of structured CbD is an additional tool which can help to structure and document training and identify developmental needs.

The proforma for recording a CbD (see the online supplementary material) is also available for download at: www.ers-education.org/cbd; the downloadable form can be used as a printed form or an electronic record. It is recommended that before introducing CbD to a training unit, at least one of the supervisors should have undergone some training in the process, and in particular in how to deliver effective feedback.

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Case-based discussion form EDU-0209-2019_Supplementary_material

Supplementary material

This article has supplementary material available from breathe.ersjournals.com