Incorporating value-based healthcare projects in residency training: a mixed-methods study on the impact of participation on understanding and competency development.

OBJECTIVES: Stimulating the active participation of residents in projects with societally relevant healthcare themes, such as value-based healthcare (VBHC), can be a strategy to enhance competency development. Canadian Medical Education Directions for Specialists (CanMEDS) competencies such as leader and scholar are important skills for all doctors. In this study, we hypothesise that when residents conduct a VBHC project, CanMEDS competencies are developed. There is the added value of gaining knowledge about VBHC.
DESIGN: An explorative mixed-methods study assessing residents' self-perceived learning effects of conducting VBHC projects according to three main components: (1) CanMEDS competency development, (2) recognition of VBHC dilemmas in clinical practice, and (3) potential facilitators for and barriers to implementing a VBHC project. We triangulated data resulting from qualitative analyses of: (a) text-based summaries of VBHC projects by residents and (b) semistructured interviews with residents who conducted these projects.
SETTING: Academic and non-academic hospitals in the Netherlands. PARTICIPANTS: Out of 63 text-based summaries from residents, 56 were selected; and out of 19 eligible residents, 11 were selected for semistructured interviews and were included in the final analysis.
RESULTS: Regarding CanMEDS competency development, the competencies 'leader', 'communicator' and 'collaborator' scored the highest. Opportunities to recognise VBHC dilemmas in practice were mainly stimulated by analysing healthcare practices from different perspectives, and by learning how to define costs and relate them to outcomes. Finally, implementation of VBHC projects is facilitated by a thorough investigation of a VBHC dilemma combined with an in-depth stakeholder analysis.
CONCLUSION: In medical residency training programmes, competency development through active participation in projects with societally relevant healthcare themes-such as VBHC-was found to be a promising strategy. From a resident's perspective, combining a thorough investigation of the VBHC dilemma with an in-depth stakeholder analysis is key to the successful implementation of a VBHC project. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The residents included were from diverse specialties, both academic and non-academic, making this research relevant for a broad spectrum of doctors and educators.

This is a practice-oriented study, easily applicable in current medical practice.

Data from the semistructured interviews supported data from the text-based summaries, although the sample size is small.

Canadian Medical Education Directions for Specialists competency development was self-reported.

Only projects that were thought to be viable for implementation and led by residents with the intrinsic motivation to make a change were publicised and therefore eligible for selection, which could lead to overestimation in the results.

Introduction

The Canadian Medical Education Directions for Specialists (CanMEDS) project contributed to a major change in medical education (undergraduate and postgraduate), moving away from a time-based learning system to a competency-based learning system.1 It describes important competencies residents should master during their training, such as communicator, collaborator, leader and health advocate.2 3 However, it appears that the ‘soft’ competencies such as ‘leader’ and ‘health advocate’ are neither easy to teach nor to assess.4 Educational efforts that specifically train these competencies are therefore appreciated.

In addition to the CanMEDS competencies, and to ensure that physicians’ competencies align with evolving health systems, overarching themes focusing on societal needs and future demands are increasingly integrated into medical education.5 Examples of such themes are patient safety, shared decision-making and value-based healthcare (VBHC).6–8 In the Netherlands, educators undertook action to combine these developments, resulting in the CanBetter project, which started in 2015.8 That project involved linking the development of all CanMEDS competencies with teaching residents about societally relevant themes. VBHC is one of the current relevant themes the CanBetter project focuses on, because healthcare expenditure is rising and medical professionals need to take their responsibility for keeping costs down.9

VBHC is defined as the health outcome that matters to the patient, relative to the costs of achieving this outcome.9 Health outcomes can be, for example, disease-free survival for patients with cancer, or the time needed to regain functionality after a knee joint replacement.9 As well as achieving the health outcomes, the professionals are also responsible for the stewardship of resources, which requires an entirely new way of managing.10 Therefore, instruction, training and fundamental knowledge of VBHC are required.11–13 As recent research had illustrated, past efforts to teach residents about cost-effective care have unfortunately not always been as effective as intended,13–15 and that key elements of learning to deliver VBHC are knowledge transmission, appropriate role modelling, reflection and the presence of a supportive environment.13 Medical students and residents must be educated and trained in settings where they have opportunities to develop and use VBHC, preferably a clinical setting.16–18 Among others, a specific training programme, incorporating formal and informal learning, is necessary to learn how to deliver VBHC.19 The residency training programmes are believed to be one of the best places to initiate VBHC education because residents are adaptable, highly educated and motivated. In addition, it has been shown that what residents learn during their residency has a significant impact on how they treat their patients when they become medical specialists.20

In our regional organisation of teaching hospitals, a strategy was set up that combined formal and informal training within the residency training programme, getting residents involved with VBHC. Residents were stimulated to critically assess daily healthcare and recognise potential VBHC dilemmas and transform all this into a VBHC project. Such VBHC projects are resident led and practice based.

This study evaluates the impact of incorporating VBHC projects within the residency training programme. The following research questions were formulated: (1) Which CanMEDS competencies do residents develop when conducting VBHC projects in residency training?; (2) Is recognition of VBHC dilemmas in medical practice facilitated when residents conduct VBHC projects during their residency training programme?; (3) Which facilitators for and barriers to VBHC project implementation can we detect when conducting a VBHC project during residency training?

Methods

Study design

An explorative mixed-methods design was used. The study entailed an analysis of retrospective information from text-based summaries of VBHC projects conducted by residents between 2014 and 2018, an analysis of semistructured interviews with residents and a comparison of data of both analyses. The text-based summaries were retrieved via the ‘standard format for VBHC projects’ (see online supplemental appendix 1 for the format), which are publicly available in Dutch via a webpage.21

Setting

This study focused on the postgraduate training setting (residency training) in the southeast region of the Netherlands. Table 1 presents a general overview of the Dutch medical undergraduate and postgraduate training programmes. The region of study has chosen to apply a multifaceted approach of incorporating VBHC in residency training by having residents conduct small, pragmatic initiatives called ‘VBHC projects’. At the time of the study, VBHC training and projects were not mandatory, though greatly encouraged as a method for residents to learn about VBHC in different ways.

Table 1

Overview of general characteristics of the Dutch undergraduate and postgraduate training programmes

Name and structure	Duration	Qualification after graduation
Preclinical training (bachelor)	3 years	Bachelor of Medicine
Clinical training (master)	3 years	Physician, MD
Resident not in training	Optional	–
Residency training or training for general practitioner	3–6 years	Medical specialist, general practitioner

The VBHC projects

Residents were stimulated to critically assess daily healthcare and recognise potential VBHC dilemmas in practice. These dilemmas entail a wide range of problems or possibilities for improvement that residents would come across; for example, inefficiency in logistics, unnecessary costs spent on diagnostics or a new treatment with fewer complications. Residents were then guided step by step. First, they were offered the knowledge and skills to identify, measure and evaluate both costs and outcome. Second, they were encouraged to discuss VBHC dilemmas with the relevant stakeholders, consequently assessing the dilemma from multiple perspectives. These perspectives could be the patient’s, the doctor’s, other healthcare professionals’ and/or that from the healthcare organisation as a whole. Third, residents had to determine goals for improvement or search for an effective alternative for current clinical practice. Finally, they reported to each other how they implemented their change or planned to implement their change, what facilitators and barriers they foresaw or encountered (potential or actual), and what they learnt during the process. Altogether, this resulted in a VBHC project. In order to enhance the chance of successfully conducting such a project, the residents were encouraged to keep the projects relatively small scale, that is, with the goal to finish it within 3–6 months.

Data collection and analyses

Data collection and analyses of text-based summaries

VBHC projects were included in this study when a standard format was completed and published online on the publicly available webpage,21 with a clear problem statement and potential costs and effects identified as measured and valued from multiple perspectives (at least 2). If the data were incomplete, an email and one reminder email were sent to request the missing information. If the data remained incomplete, the VBHC project was excluded.

The final set of included VBHC projects was assessed by two independent researchers (CYGN, SV). Data extraction of these projects involved a summary of the setting, medical specialty, focus of the project, anticipated and achieved results, and learning effects in terms of the residents’ self-perceived CanMEDS competency development. Data extraction yielded a score for the learning effects and were compared with data from the semistructured interviews. Any disagreements were resolved through discussion. The focus of the projects was categorised after an inductive process as medical education, medical care and/or organisational efficiency. The medical education category included projects that aimed to improve education for residents, either in terms of practical skills or theoretical knowledge. The medical care category included projects that focused on improvement in the current quality of care and/or direct reduction of healthcare costs. The organisational efficiency category included projects that aimed to optimise processes in healthcare.

Data collection and analyses of semistructured interviews

Semistructured interviews were held with residents who conducted a VBHC project and published it in the publicly available online database.21 After the initial contact with a resident, a reminder request was sent if there was no response. All interviews were performed by telephone or face-to-face by one investigator (MK) under the supervision of a trained interviewer (CYGN). There was no prior relationship between the interviewer and the interviewees before the start of the study. Prior to performing the interviews, a series of pilot interviews was held to train the interviewer and assess the quality and feasibility of the topic list. The final topic list for the semistructured interviews can be found in online supplemental appendix 2. The residents were asked to provide a general reflection of the process, and to reflect on both successful and unsuccessful processes of implementation. If it appeared that no implementation attempt was made, based on the interview data, the interview data were not included in the analyses. Individual interview data were recorded and transcribed verbatim (MK).

The data were analysed by summarising information on the residents’ self-perceived learning of CanMEDS competencies and compared with data from the text-based summaries. Furthermore, data were analysed assessing the learning effects of VBHC, and gathering detailed information on implementation facilitators and barriers. Data were clustered by two independent researchers (CYGN, SV) applying the generally accepted principles of primary, secondary and tertiary coding, in a constant comparison,22 iterative approach. Regarding potential facilitators for and barriers to implementation, this means reading through the transcripts inductively to find patterns.23 24 Finally, all interview data were searched for these patterns. These steps were executed by two researchers (SV, CYGN), and in case of disagreement, dialogue followed until consensus was reached.

Patient and public involvement

None.

Results

Text-based summaries

From the 63 VBHC projects identified in the publicly available online database,21 56 VBHC projects were included and analysed (see figure 1). Fifty VBHC projects (89%) included in this study were implemented in practice.

Figure 1

Flow chart on selection and exclusion of text-based summaries. VBHC, value-based healthcare.

In table 2, a general description of the 56 included VBHC projects is presented. For 14 residents, the focus of their VBHC project was on more than one goal (eg, medical education and medical care, or organisational efficiency and medical care).

Table 2

Specifics of the VBHC projects included

Method		Text-based summaries	Interviews
Setting	University medical centre	38 (68%)	6 (55%)
	Non-academic	18 (32%)	5 (45%)
Medical specialty	Anaesthesiology	4 (7%)	2 (18%)
	Cardiology	1 (2%)	–
	Cardiothoracic surgery	2 (4%)	–
	Clinical pharmacology	3 (5%)	–
	Clinical genetics	3 (5%)	–
	Internal medicine	1 (2%)	–
	Neurology	1 (2%)	2 (18%)
	Neurosurgery	4 (7%)	1 (9%)
	Obstetrics and gynaecology	7 (13%)	1 (9%)
	Orthopaedics	6 (11%)	–
	Ophthalmology	2 (4%)	–
	Paediatrics	1 (2%)	2 (18%)
	Plastic surgery	1 (2%)	–
	Psychiatry	3 (5%)	1 (9%)
	Pulmonary medicine	3 (5%)	–
	Radiology	4 (7%)	–
	Rehabilitation medicine	1 (2%)	2 (18%)
	Surgery	7 (13%)	–
	Urology	1 (2%)	–
	Vascular medicine	1 (2%)	–
Focus of the projects	Organisational efficiency	14 (25%)	–
	Medical education	2 (4%)	1 (9%)
	Medical care	25 (45%)	1 (9%)
	More than one goal	14 (25%)	9 (82%)

VBHC, value-based healthcare.

Individual semistructured interviews

Out of the 19 residents approached for the interview, 11 were included and analysed in this study (see figure 2). To some extent (n=5), both the residents and their projects were included in the text-based summaries and the interviews.

Figure 2

Flow chart on selection and exclusion of individual semistructured interviews.

Descriptive details of the VBHC projects conducted by the 11 residents interviewed are summarised in table 2. More detailed information can be found in online supplemental appendix 3.

CanMEDS competency development

Data analysis of the text-based summaries and interviews revealed that self-perceived learning effects regarding CanMEDS roles were present for at least two competencies. See table 3 and figure 3 for details, distribution and comparison.

Table 3

Self-perceived learning regarding the different CanMEDS competencies

	Medical expert	Scholar	Leader	Collaborator	Communicator	Professional	Health advocate
Text-based summaries	18%	23%	89%	86%	86%	34%	43%
Interviews	9%	45%	91%	100%	82%	36%	100%

CanMEDS, Canadian Medical Education Directions for Specialists.

Figure 3

Self-perceived learning regarding the different CanMEDS competencies. CanMEDS, Canadian Medical Education Directions for Specialists.

Data from the text-based summaries showed that self-perceived learning effects regarding CanMEDS roles were evident in all respondents for at least two competencies. In addition, 93% of the residents trained three or more CanMEDS competencies when conducting a VBHC project. In order of frequency, the best trained competencies were ‘leader’ (n=50), ‘communicator’ (n=48) and ‘collaborator’ (n=48).

The interview data showed a somewhat similar distribution of self-perceived learning regarding CanMEDS roles as the data from the text-based summaries (see table 3 and figure 3). In order of frequency, the best trained competencies found in the data from the interviews were ‘collaborator’ (n=11), ‘leader’ (n=10) and ‘communicator’ (n=9). A difference in the distribution of self-perceived learning was noted when comparing the data from the text-based summaries with the data from the interviews for the health advocate competency (n=11) and, to some extent, scholar (n=5), which were more often developed according to the interviewed residents. See text below for illustrative quotes on self-perceived learning regarding CanMEDS roles.

Supporting quotes were found for the most frequently trained competencies. R10 comment on scholar: “because I did a complete literature investigation to support the new protocol I implemented.” R3 comment on leader: “I learned a lot about the organisation of healthcare and how many possibilities there still are for improvement.” R7 comment on collaborator: “I learned that supportive departments, for example the financial administration, are easily accessed, which gave me great insight into the organisational structure, but also showed me that communication is sometimes lacking.” R1 comment on communicator: “[…] communication as well, by the high frequency of presentations I had to do.” R1 comment on organisation: “[…] because I gained great insight into the structure of our organisation, financial background, who different stakeholders are, and how choices are made.” R4 comment on health advocate: “I learned about health advocacy because I tried to reduce health care costs for the community without loss of quality of care.”

Learning concepts of VBHC

Concerning the concepts of VBHC, our data revealed a few important aspects of the residents’ learning process. First, the resident needs to recognise the VBHC dilemma in clinical practice. Next, they need to examine this dilemma from multiple perspectives. Finally, they need to balance costs versus outcome regarding this dilemma from these multiple perspectives.

Recognition of VBHC dilemmas in clinical practice

Conducting VBHC projects included a step-by-step guidance and completion of the format. This allowed the VBHC dilemma to be linked to clinical practice, facilitating the recognition of VBHC—or its lack—in practice. All text-based summaries of VBHC projects included in this study (N=56) described the VBHC dilemma by defining the costs and outcomes from multiple perspectives and the goal for improvement or alternatives for practice (see online supplemental appendix 1, items 2 and 3). The interviewed residents (N=11) helped us to gain insight since they could explain how this format helped them to recognise the VBHC dilemma. R12: “the format obliges you to walk through the process step by step. We often say we do, but this helps us actually to do so, because you have an anchor.”

Learning multiple perspectives concerning VBHC

All interviewed residents (n=11) could explain the multiple perspectives they took into account when analysing their VBHC problem. R5: “I learned to reduce waste and made the process of patient letters more efficient and improved the quality of care in the same process. So, patient, doctor and organisation benefit.”

Learning process of costs versus effects

All interviewed residents (n=11) explained how they became more aware of the relationship between both costs and effects (value) when conducting their VBHC project. For example, R8: “we measured the number of no-shows in the intervention group vs the group that received standard care, and measured the revenues we missed out on because of the no-shows.” R7: “I reduced costs without loss of quality of care for the patient by reducing standardised laboratory tests upon admission.” A self-perceived learning effect could also be described as an improvement in awareness and identification of costs in a broader perspective. R4: “We had a reduction in leading time for patient letters to the general practitioner, from an average of a few weeks to at least 80% finished and sent within 5 days after discharge. But the cost reduction and quality improvement I envisioned were not just the reduction in leading time of the patient letter, but also in better care when the GP [general practitioner] has adequate information as soon as possible.” The interview data also revealed that the residents were able to explain their learning regarding VBHC in clinical practice as well. R10: “[…] every randomised controlled trial we use or refer to in clinical practice that refers to cost in relation to quality reflects a moment where we reflect on VBHC […] we don’t always recognise it, but it is the basis of VBHC.”

Facilitators for implementation

Residents mentioned a number of facilitators for implementation, in part related to the step-by-step process we use (see the Methods section for details). First, a thorough problem analysis from multiple perspectives is important. Second, the involvement of all important stakeholders seems essential. Third, an intrinsic motivation to resolve the problem at hand is a strong facilitator. Finally, support from a supervisor or mentor seems important.

Problem analysis

Residents (n=6) stated that detecting and defining the problem is a very important facilitator and relates to the first step in the process of conducting a VBHC project (see online supplemental appendix 1, points 2 and 3), in which residents are stimulated to critically assess daily healthcare and recognise potential VBHC dilemmas and assess all this from multiple perspectives. R4: “A thorough investigation of the problem and making sure we knew why and for whom the problem was relevant made implementation a lot easier.”

Stakeholder involvement

An important item to facilitate implementation (n=11) turned out to be the involvement and support of all stakeholders as defined during the second step in the guidance by experts: stakeholder analyses. R4: “Involve all relevant stakeholders and create a sense of urgency and relevance.” R2: “Everybody (doctors, nurses, secretarial staff) recognised the problem of a language barrier and wanted a change to be able to take better care of patients who speak a different language […].”

Intrinsic motivation

Intrinsic motivation to solve the problem was a huge facilitator as well for many residents (n=5). R8: “[…] my interest and motivation turned a small project into a hospital-wide project.”

Support

A final facilitator many residents (n=7) mentioned was a mentor or supervisor who helped them with potential barriers, and that they probably would not have been able to change anything without their support. R6: “The educational committee supported my project and supported the different residents to investigate and implement possibilities. That was really helpful.”

Barriers to implementation

Self-perceived barriers were often the opposite of the facilitators related to the VBHC dilemma. For example, there was no clear VBHC dilemma or no clear problem definition. A few distinct barriers were mentioned, namely a lack of support, involvement of too many stakeholders and problems with the magnitude of the project.

A major barrier for implementation was felt to involve the preconditions and technical aspects, for instance, information and communication technology (ICT) or secretarial support to implement the VBHC project. R11: “In the end, the ICT application was not realised and therefore I could not implement my project.” Another barrier mentioned was the lack of a supportive environment by a sufficient number of the stakeholders involved (n=6). Too few could also be experienced as a barrier. R10: “in the end I did not have enough support from the other residents to implement my change although the educational committee was on board.”

Stakeholders

Involvement of too many stakeholders was described as a barrier. R11: “I needed not only other residents, but also different supervisors to be on-board with the plan, which I understood along the way.”

Magnitude

Finally, the extent of the project made it more difficult to implement, for instance, when the whole organisation was involved instead of just their own department (n=3). R11: “because it became a hospital-wide project, it is currently still not implemented.”

Discussion

In this explorative mixed-methods study, different aspects of learning were identified from the residents’ perspective when conducting small, pragmatic VBHC projects. First, conducting VBHC projects was shown to contribute to developing different CanMEDS competencies, especially communicator, collaborator, leader and health advocate. Second, residents learnt to recognise VBHC dilemmas in clinical practice. Finally, facilitators for and barriers to implementing VBHC projects were explored. In the subsequent sections, these findings will be discussed more extensively.

Our study has shown that by conducting VBHC projects, residents were provided with opportunities to further develop at least two different CanMEDS competencies. Our data showed that the ‘leader’, ‘collaborator’ and ‘communicator’ competencies were most often developed. These are examples of the ‘soft’ competencies.25 Mastering these competencies is helpful when implementing any change project, and this motivates residents to acquire them.26 In this study, the ‘scholar’ and ‘health advocate’ competencies were often mentioned, especially by the participants of the interviews, more than in the text-based summaries. This was an unexpected and remarkable finding: young medical specialists feel inadequately prepared for these specific competencies.4

We showed that residents are facilitated to learn to recognise a VBHC dilemma in clinical practice when conducting a VBHC project. Additionally, they perceived an improvement in awareness regarding VBHC, which is an important step in teaching residents to deliver VBHC.27 Our data support that when residents conduct VBHC projects, recognition of VBHC dilemmas in medical practice is facilitated if they: (1) perform a thorough investigation of the problem, (2) explore potential barriers and (3) set up a viable project for implementation. The data might overestimate the learning effects because we only included those residents who believed their projects were viable for implementation and those with the intrinsic motivation to make a change since participation in the VBHC projects was voluntary. Intrinsic motivation has a known positive effect on change projects in healthcare.28 For future research, it could be interesting to investigate if VBHC projects are fruitful learning opportunities when mandatory in any postgraduate curriculum.

Finally, the self-perceived facilitators for and barriers to implementing a VBHC project during residency training were extracted from the interviews. We believe them to be crucial for the successful implementation of a VBHC project. The information revealed many similarities to the contemporary literature on facilitators and barriers in change management.29–31 First, a thorough investigation of the problem is crucial,31 specifically concerning the VBHC dilemma. This means investigating the costs and outcomes from multiple perspectives (such as patient, organisation, doctor, nurses and supportive staff) and balancing these costs versus outcomes.32 This is crucial in our opinion to establish a sense of urgency and/or relevance, which in turn is essential for a successful implementation.33 34 Next, it is important to identify stakeholders and get them involved.35 36 This is only possible via ‘a clear VBHC dilemma’, of relevance for those who are affected by it.34 Subsequently, find a supervisor or mentor with enough influence to make the change happen and someone who has control of resources; in a hospital, this could be several different people.29 Then establish a plan of action, keep it small and simple, within your own scope of influence.37 Finally, describe how the effects are going to be measured (ie, in terms of costs and effects) and demonstrate intermediate results.36 Ultimately, you need to consolidate and secure your new way of working.29 Inspired by the work of other researchers in the field of change management,29–31 we summarised the tips from this study in an overview checklist (see online supplemental appendix 4). We hope to inspire and guide residents who want to implement a VBHC project. We would advise any educator who is about to support residents when implementing VBHC projects to use a similar format and combine structured teaching with support by an expert in the field.19 This strategy adheres to the necessary requirements of knowledge, support, role models and reflection when it comes to teaching the concepts of VBHC.19

Conclusion

This study revealed the positive impact of incorporating VBHC projects within the residency training programme. VBHC projects are resident led and practice based and proved to be an effective educational method to learn and develop CanMEDS competencies and practise the concepts of VBHC via learning by doing. Residents learn to recognise VBHC dilemmas, how to implement an effective change and that they have the potential to influence medical care, medical education or efficiency in healthcare. According to the residents, important facilitators for successfully implementing a change are a thorough investigation of the VBHC dilemma combined with a thorough stakeholder analysis. Practice points are presented in box 1.

Introducing a societally relevant theme such as value-based healthcare (VBHC) in residency training programmes is a promising strategy to enhance competency-based education.

Residents develop different Canadian Medical Education Directions for Specialists competencies when conducting a VBHC project, especially those of ‘leader’, ‘communicator’ and ‘collaborator’.

Residents learn to recognise VBHC dilemmas in practice when combining formal teaching with conducting VBHC projects.

A thorough investigation of a VBHC dilemma combined with an in-depth stakeholder analysis are key to successfully implementing a VBHC project.