Balancing student/trainee learning with the delivery of patient care in the healthcare workplace: a protocol for realist synthesis.

INTRODUCTION: A national survey was recently conducted to explore medical education research priorities in Scotland. The identified themes and underlying priority areas can be linked to current medical education drivers in the UK. The top priority area rated by stakeholders was: 'Understanding how to balance service and training conflicts'. Despite its perceived importance, a preliminary scoping exercise revealed the least activity with respect to published literature reviews. This protocol has therefore been developed so as to understand how patient care, other service demands and student/trainee learning can be simultaneously facilitated within the healthcare workplace. The review will identify key interventions designed to balance patient care and student/trainee learning, to understand how and why such interventions produce their effects. Our research questions seek to address how identified interventions enable balanced patient care-trainee learning within the healthcare workplace, for whom, why and under what circumstances. METHODS AND ANALYSIS: Pawson's five stages for undertaking a realist review underpin this protocol. These stages may progress in a non-linear fashion due to the iterative nature of the review process. We will: (1) clarify the scope of the review, identifying relevant interventions and existing programme theories, understanding how interventions act to produce their intended outcomes; (2) search journal articles and grey literature for empirical evidence from 1998 (introduction of the European Working Time Directive) on the UK multidisciplinary team working concerning these interventions, theories and outcomes, using databases such as ERIC, Scopus and CINAHL; (3) assess study quality; (4) extract data; and (5) synthesise data, drawing conclusions. ETHICS AND DISSEMINATION: A formal ethical review is not required. These findings should provide an important understanding of how workplace-based interventions influence the balance of trainee learning and service provision. They should benefit various stakeholders involved in workplace-based learning interventions, and inform the medical education research agenda in the UK. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The realist synthesis is well suited to evaluation of complex interventions in the healthcare workplace.

The use of grey literature enables data triangulation from multiple sources.

A multidisciplinary research team lends broad experience and more comprehensive data interpretation.

The choice of search terms may limit the material included in the review.

The geographic search area is limited to the UK.

Introduction

Medical education research priorities

In 2014, Dennis et al1 conducted a national survey in order to explore the priorities for medical education research in Scotland as perceived by a variety of stakeholders. The priority setting exercise identified 21 priority areas for Scottish medical education research, falling into five broad research themes: (1) The culture of learning together in the workplace; (2) Enhancing and valuing the role of educators; (3) Curriculum integration and innovation; (4) Bridging the gap between assessment and feedback; (5) Building a resilient workforce. These themes and their underlying priority areas can be linked to current medical education drivers in the UK (eg, Tomorrow's Doctors,2 the Shape of Training review,3 and Promoting Excellence: standards for medical education and training4). The reasons given by participants for prioritising items included patient safety, quality of care, investing in the future, policy/political agendas and evidence-based education.1

The top priority area rated by stakeholders was ‘Understanding how to balance service and training conflicts’, concerning ‘the pressures that exist or are perceived to exist between the delivery of service to patients and the provision of training.’1 However, a preliminary scoping exercise to identify literature reviews relevant to the priority areas revealed the least activity in this area. Figure 1 shows the wide variation in literature review activity across the different priority areas. It was therefore decided to proceed with a literature review for the top-rated priority area.

Figure 1

Literature reviews pertaining to each of the 21 priority areas identified in the priority-setting exercise by Dennis et al.1 Note that some literature reviews are relevant to more than one priority area. (a) Vertical integration of undergraduate and postgraduate curricula.

Challenges of evaluating complex interventions in the workplace

The balance of service delivery and education operates within a complex environment. It is present in a variety of care contexts; it involves different stakeholders (eg, members of multidisciplinary teams, patients, managers, etc), and the dynamic of the balance will vary between sites (eg, different models of care in different regions) and at different times of the day, week and year, depending on demand.5 Interventions such as protected study time are dependent both on these variations and on the ways in which such interventions are implemented.6 The factors affecting this balance are similarly complex and dynamic, although many of the factors themselves (such as workplace learning culture and capacity/capability) appear to be common across healthcare settings.1 5

Narrative reviews summarise a range of material in order to construct holistic conclusions. These conclusions are informed by the research team's own experience and by drawing on existing theories.7 Robustly conducted narrative reviews can be useful depending on their intended purposes, but they are commonly criticised from a positivist perspective as being vulnerable to bias.8–10 Systematic reviews involve “empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question”.11 They can be useful for assessing simple interventions, employing robust and replicable methods, and providing a comprehensive review of available peer-reviewed literature.9 12 13 By focusing only on peer-reviewed literature, however, we risk missing out on potentially valuable information collected by other means such as grey literature (ie, that which lies outside academic or commercial publication). The scoping exercise mentioned previously has also indicated that much of the literature dealing with the balance of service and training conflicts appears to be embedded in papers with different primary foci, such as occupational stress or evaluation of clinical teaching.

A realist review would be suitable for more complex interventions such as those encountered in healthcare, affording us an explanatory focus and the ability to include a wide range of evidence sources (including primary qualitative and quantitative research, secondary research and grey literature).14 The realist approach described by Pawson et al15 enables us to develop theories that consider the context-mechanism-outcome (C-M-O) approach, that is, how these contexts mobilise resources through which interventions work or do not work and their ability to promote the balance of service delivery and training. It is worth noting here that the C-M-O approach is not necessarily a linear one, even though it is ultimately expressed as such; for example, interventions may work in more than one way in a particular context, or a programme theory may be based on an existing outcome. Despite realist reviews not being able to cover every eventuality, they are able to shed light on complex situations and to provide contextual explanations, which are arguably more useful in a healthcare policy-making context.15

Research questions and aim

The research questions arising from the scoping exercise were as follows:

▸ How can the delivery of service to patients and of training be simultaneously facilitated in the healthcare workplace?

▸ What are the key complex interventions which are designed to help achieve/maintain this balance?

▸ In what ways do successful interventions enable this balance within the healthcare workplace, and in what context?

Our study therefore aims to address the ways in which identified interventions enable balanced patient care-trainee learning within the healthcare workplace, for whom, why and under what circumstances.

Methods and analysis

Study design

Pawson et al15 describe five stages for undertaking a realist review, which have been used to underpin the design of this study. Since the review process is iterative, it may not necessarily follow this linear progression neatly:

1. Clarify the scope of the review by identifying: (a) relevant interventions; and (b) existing programme theories, so as to understand how these interventions act to produce their intended outcomes. The scoping exercise mentioned above generated a number of possible search terms which could be used both to refine the purpose of the review and as the basis for formulating key theories, and may help to address the issue that much of the research in this area is embedded in other literature.16–18 These search terms (and derivations thereof) include: (1) protected study time; (2) workplace-based learning (or workplace learning); (3) workplace-based assessment (or workplace assessment); (4) clinical learning environment; (5) clinical placement; (6) supervised learning events; (7) bedside teaching encounters; (8) continuing professional development; (9) barriers/facilitators to learning; (10) interprofessional/multiprofessional learning; (11) capacity and capability; (12) workforce planning; (13) cost-effectiveness; (14) organisational need; (15) context-sensitive learning.

A number of possible programme theories were considered when developing the protocol, and based on the scoping exercise a speculative candidate theory was identified as follows: protected learning time can be an effective intervention for postgraduate medical trainees in the primary care setting in the quest to balance the requirements of service delivery and of training depending on logistical pressures, learner motivation and attitude and the social environment. The mechanism is possibly due to access to education and perceived value/leadership.

2. Search for empirical evidence concerning these interventions, theories and outcomes. Note that evidence may be supportive, contradictory or act to modify the theories identified in stage 1. It is anticipated that the search strategy will involve: (1) Searching for peer-reviewed literature using electronic databases; (2) Snowballing and citation tracking; (3) Grey literature searching. See table 1 for a summary of the types of literature and sources to be searched.

Table 1

Summary of types of literature and sources to be searched

Type of literature	Sources
Journal articles	Searching databases ERIC, Scopus, CINAHL, Web of Science and PsychInfo; checking the reference lists of included papers and contacting authors where appropriate.
Grey literature	Publications from the General Medical Council, Health and Care Professions Council, Nursing and Midwifery Council, etc.
UK websites	Eg, NHS Education for Scotland, Health Education England, Royal Colleges of Physicians/Nursing/Midwifery, etc.

Terminology will initially be refined using Medical Subject Headings (MeSH) to ensure a systematic and comprehensive search of the literature. The authors have elected to start with Medline and to use the list of search terms arising from the initial scoping exercise in order to elicit the MeSH terms that will be used. A similar strategy will be employed for CINAHL, which uses a similar control language but with a different set of descriptors. The resulting list of search terms will then be used for those databases requiring a free text approach, such as Scopus and Web of Science. Table 2 summarises the type and detail of inclusion criteria to be used, which are explained in more detail below.

Table 2

Inclusion criteria

Type	Criterion
Topic	Literature should relate directly to one or more of the research questions (see earlier).
Recency	Literature published from 1998 onwards (see also box 1).
Geographic spread	Literature should relate to studies carried out in the UK.

Since all of the papers identified during the scoping exercise employed a UK-wide focus, and since many of the factors considered to influence the balance of service and education are common to all regions of the UK, the proposed review will be UK-wide. While the findings may have relevance to other national health systems, studies involving them will be excluded from this review as different health systems may be influenced by different contextual factors such as healthcare funding and educational pathways. This is not to say that mechanisms identified in other countries would not be helpful to those in the UK healthcare workplace; however; they may indeed be transferable in this context.

Postgraduate education tends to be workplace-based and often involves interprofessional learning; therefore, the proposed review will include multidisciplinary teams as well as individual learning, rather than being limited to doctors only, particularly since we are seeking to understand in which circumstances (ie, for which ‘populations’) mechanisms are effective.19–23 These populations include students (ie, those who have not yet gained their initial qualification but who undergo some of their training as part of a team in the healthcare workplace) and trainees (ie, individuals post-qualification but not yet at the end of their training). See box 1 for a brief timeline of relevant NHS milestones, which allowed us to choose an appropriate timescale for the literature search.

1990: NHS & Community Care Act—health authorities manage own budgets

1991: NHS Trusts established following NHS Community Care Act24

1997: Scottish white paper: Designed to Care—47 Trusts become 28; integration of services; managed clinical networks25

1998: European Working Time Directive (EWTD) introduced for all except junior doctors26

1999: NHS Scotland control handed over to Scottish Government

2002: NHS Education for Scotland established

2004: EWTD extended to cover junior doctors

Abolition of NHS Trusts by NHS Reform (Scotland) Act27

2008: Lord Darzi report: Our NHS, Our Future28

2009: GMC report: Tomorrow's Doctors2

2012: 2020 Vision and Strategic Narrative29

2013: Greenaway report: Shape of Training3

Since ‘time’ was the most frequently cited influencing factor in the 25 studies found in the scoping exercise, it is proposed that the literature search be conducted from 1998, the introduction of the European Working Time Directive.

3. Assess quality of studies

Literature will be assessed for relevance and rigour according to RAMESES publication standards.14

Relevance—papers will be screened first for relevance, that is, those which “provide data that inform programme theory development and refinement”.14 30 Initial assessment of relevance will be carried out by reviewing abstracts using preliminary inclusion criteria. Any ambiguities at this stage will be checked by an additional researcher. Depending on the quantity of studies found, it is likely that a two-stage review process will be carried out; first to identify the main interventions that are relevant, and then to prioritise one or two interventions which will be the focus of the main study. Double-checking will be carried out and discussed for 10–25% of the citations, along with a number of papers previously excluded (for quality control purposes).30

Rigour—assessment of rigour will follow the same process, this time employing a review of the whole paper, to determine “whether the methods used to generate the relevant data are credible and trustworthy”.30 Any differences will be resolved between the two analysts through discussion and if this is not possible, then a third analyst will be brought in to adjudicate. The application of inclusion/exclusion criteria will be an iterative process, as will be the testing and refinement of programme theories that are generated during this stage. Figure 2 shows a summary of the searching and selection process.

Figure 2

PRISMA diagram of the searching and selection process.

4. Extract and analyse data

Realist review data are characterised by annotation rather than list extraction,14 30 and a thematic approach will be adopted here. The process of analysis will pursue the following iterative progression: (1) Reading a sample of the data to identify codes for contexts, mechanisms and outcomes; (2) Developing a coding framework including descriptive elements and more analytic C-M-O configurations; (3) Applying the coding framework to the rest of the data; and (4) Interrogating the codes using ATLAS.ti software in order to look for patterns and organise codes. Discussion of the data between researchers allows continuation of the testing and refinement of programme theories at this stage.

5. Synthesise data and draw conclusions

The purpose of the realist review should drive the process of synthesis.15 Relevant annotated evidence will be used to test each aspect of the programme theory (or theories). The process of synthesis will include the following considerations30: (1) Reconciling and consolidating contradictory evidence—evidence which does not support a theory can enable useful insights about its implementation;14 (2) Consideration of the relative methodological strengths/weaknesses of evidence; (3) Findings of one study that may allow insights into another; (4) Maintenance of the context of evidence sources when drawing conclusions and presenting findings. See box 2 for an understanding of how internal and external validity will be addressed.

Construct validity: by using multiple sources of evidence (data triangulation)

Internal validity:    by pattern matching (relating several different pieces of information from the study to a theoretical concept)

External validity:  by employing theory in the study design and by testing empirical evidence against programme theory

The research team is multidisciplinary in background, including clinically qualified individuals, social scientists, healthcare education researchers and managers. We anticipate that this broad range of experience will lend itself to a more comprehensive interpretation of the data.

Dissemination

Results will be written up according to RAMESES publication standards14 and disseminated via the Scottish Medical Education Research Consortium (http://www.smerc.org.uk) at conferences and in peer-reviewed journals, and to NHS Education for Scotland, Health Education England (in particular Postgraduate Deans), regulatory and professional bodies, etc. These findings will provide important insights into how workplace-based interventions influence the balance of service delivery and education. They will benefit multiple stakeholders involved in developing, implementing and receiving workplace-based learning interventions, as well as continue to inform the medical and health professions education research agenda for the UK.