Supervision training interventions in the health and human services: realist synthesis protocol.

INTRODUCTION: Supervision training aims to develop workplace supervisory competencies. Despite extensive supervision literature, including literature reviews, the processes through which supervision training interventions produce their effects, for whom and under what circumstances is not clearly delineated. The purpose of this study is to explain the effect of contextual factors on the underpinning mechanisms of supervision training outcomes. METHODS AND ANALYSIS: We propose to examine supervision training interventions across the health and human services workforce using realist methods. Pawson's five stages for undertaking a realist synthesis will be followed: (1) clarifying the scope of the review; (2) determining the search strategy; (3) study selection; (4) extracting data and (5) synthesising the evidence and drawing conclusions. Extracted data will include study characteristics, characteristics of participant cohort, intervention type, contextual factors, underlying mechanisms and supervision training outcomes. Patterns in context-mechanism-outcome configurations will be identified. Initial programme theories will be developed based on a comprehensive search of the literature, which will include key terms relating to supervision and training. The search strategy will involve: (1) electronic database searching using Medline, Cumulative Index to Nursing and Allied Health Literature, Social Services Abstracts, Educational Resources Information Center, PsycINFO and Australian Public Affairs Information Service and (2) hand and citation searching. We will also contact authors where necessary and discuss identified literature among the project team with extensive expertise in supervision training. ETHICS AND DISSEMINATION: The realist synthesis will propose an evidence-informed theory of supervision training interventions (ie, what interventions work for whom and why). The findings will be disseminated in peer-reviewed journals and presentations and through discussions with relevant organisations and stakeholders. The research will be used by educators to develop evidenced-based supervision training interventions. It will also help workplace supervisors to better understand what types of supervision training might work most optimally for them and their colleagues. Other researchers could use the synthesis findings to guide future supervision research. PROSPERO REGISTRATION NUMBER: CRD42018094186. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This synthesis will identify how and why components of supervision training are effective (or not) within various settings in the health and human services workforce rather than merely focusing on whether supervision training is effective.

A large multidisciplinary research team lends high-quality relevance and rigour checks and more comprehensive data interpretation.

The realist synthesis is well suited to the evaluation of complex interventions such as supervision training.

Only English language studies will be included, so findings might lack transferability to supervision training interventions in non-English speaking countries.

Search terms for supervision that are less frequently used in the health and human services (eg, critical companion, coaching, facilitation, practice development) may be omitted, meaning that some evidence may be missed.

Introduction

Supervision is a distinct professional activity characterised by the training and professional development of supervisees,1 2 and in some literature is understood to be partly hierarchical and evaluative.3 Although the definition of supervision varies in terms of its purposes, arguably one of the most widely adopted models describing the purpose and functions of supervision is that of Proctor,4 who outlined three functions of supervision: normative (managerial), formative (educational) and restorative (supportive). The normative function depicts the role that supervision plays as a process for ensuring ethical, acceptable and quality practice.5 The formative function of supervision encompasses developmental aspects such as gaining knowledge, communication skills, trust in self and confidence.6 The restorative function focuses on providing support to alleviate burnout and stress.7 All three functions highlight supervision as a protective factor in professional practice.8 Note that Proctor’s depiction of supervision has been used in this study to help guide our understanding and definition of supervision.

A guideline for what constitutes effective supervision for both the supervisor and supervisee has been outlined in the literature.9 Effective supervision has been described to include but is not limited to: supervisees choosing their own supervisors10; deciding on the most suitable types of supervision (eg, one-to-one or peer group supervision or a combination of both)11; establishing a supervision agreement or contract and using a supervision agenda12; choosing venues away from supervisees’ workplaces to conduct supervision sessions; having an optimal meeting length and frequency (eg, at least 1 hour every 4 weeks)13; using effective communication and feedback14; facilitating reflective practice; use of more than one mode for distance supervision; building a positive supervisory relationship (eg, one that is positive, supportive, trustworthy, non-judgmental and encouraging)10 12; separating clinical supervision from line management; undertaking training in supervision and evaluating supervision (either through formal or informal methods)9 15

Effective supervision is that which has positive outcomes across all levels of an organisation,16 including for the client/patient (eg, improved client/patient outcomes) and the organisation as a whole (eg, improved standards and quality of service delivery).9 12 Benefits of effective supervision for practitioners include developing capability and knowledge,17 reducing burnout18 and assisting practitioners to cope better with their work and workplace.13 When conducted effectively, supervision can provide an opportunity for practitioners to reflect on their practice and establish new approaches to supervision.9 While effective supervision and its outcomes have been clearly delineated in the literature,19–21 the same attention has not been afforded to what constitutes effective training for supervisors.2 22

This realist synthesis protocol focuses on supervision practised in both the broad domains of health and human services given that it is part of a broader programme of research investigating supervision training in these wide-ranging settings including: health, housing, children services, youth and family services, alcohol and drug services and mental health. Although Proctor’s model of supervision has been used to guide our understanding of supervision in the health and human services, any outcomes of supervision training will be explored in this study including both positive and negative outcomes at individual, interpersonal and organisation levels. What follows is a brief outline of: (1) the characteristics of supervision within these settings and (2) the different modes of supervision training commonly implemented to support supervisors within these settings.

Supervision in the health and human services

Literature within health and human services predominately describes supervision as an educative process. For the most part, the literature embraces the formative and restorative functions of Proctor’s model, thereby characterising supervision as both a platform for practitioners to develop knowledge and a method of professional support.18 23 However, within mental health settings, the normative function of supervision is typically privileged, with supervision considered as an important factor in ensuring staff ability and enabling best practice outcomes for consumers and carers.24 Other allied health professions, such as physiotherapy and speech therapy, report using discipline-specific supervisory models for a combination of normative, formative and restorative functions.10 25 26

In terms of the human services workforce, in some areas of the workforce like alcohol and other drug services, the empirical evidence for the function of supervision is sparse.27 Supervision for human services workers appears more complex due to the vast array of diverse settings in which these professionals work. Consequently, there seems to be a multiplicity of definitions, models and applications of supervision within this literature,28 29 mirroring the broader supervision literature discussed above.

Supervision training

Supervision training is a common way to support supervisors and prepare them for their supervisory roles within health and human services. Supervision training is typically a process for enhancing supervisor behaviour to improve the outcomes of the supervisory process.30 31 Current literature suggests that there is a variety of training methods used to support the development of effective supervisors.32 33 Supervision training can be in the form of formal training (eg, face-to-face or online workshops and/or seminars), training received informally (eg, role modelling and observation) and/or self-directed learning (eg, guided reading).33 Supervisors typically receive little or no formal training for their supervision role and little is known about how supervisors acquire knowledge and aptitude in the supervisory role.33 Supervisor training is rarely empirically or theoretically grounded,3 and it appears that most professionals learn supervisor skills by assimilating their own experiences as supervisees or through informal methods (eg, observation). The process of learning to be a supervisor therefore often occurs by default.34

Inadequate attention to supervision outcomes, supervision training and supervision research, however, has started to raise concerns about the quality of supervision delivered across a number of different disciplines.27 35–37 The assumption that experience as a practitioner or a supervisee translates into becoming an effective supervisor is, of course, erroneous.38 Indeed, there is a distinction between practitioner skills and supervisor skills,33 and specific training to develop supervisory skills is crucial.

While there is broad understanding about what constitutes effective supervision,19–21 little attention has been paid to what constitutes effective training of supervisors.2 22 Therefore, insufficient clarity still exists regarding how supervisors should be trained to become effective supervisors.33 It has been said that ‘research on the effectiveness of supervision training remains virtually non-existent’(p219).39

Three reviews (two systematic, one narrative) regarding supervision training have been published since 2004,33 38 40 and these provide a useful starting point to understand supervision training. First, Gonsalvez and Milne38 conducted a narrative review of clinical supervision training changes in professional psychology within Australia, both examining and addressing solutions to emerging problems. Gonsalvez and Milne identified a need for better regulation of supervisory practice within psychology and recommended that supervisory practice be supported with enhanced development of resources (eg, manuals, workbooks and better psychometric tools to evaluate supervisory processes). Second, Milne et al 33 conducted a systematic review of controlled trials of supervision interventions and found clear empirical support for supervisor training; training which when effective could help to bridge the gap between policy and practice and ensure that therapies within the mental health field are implemented with fidelity. This review also helps us to understand the complexity and diversity of supervisor training. For example, they found 56 supervisor-training methods were used across the 11 studies reviewed. Training methods included workshops, role play, modelling, guided reading, direct observation and/or teaching (through verbal instruction). Finally, Tsutsumi’s40 systematic review, including seven controlled studies assessing the effect of supervisor training on the mental health of subordinate workers, proposed an evidenced-based guideline for supervisor training, which included: what training content should include, the aim (eg, behaviour modification among supervisors), frequency of training (ie, yearly) and the target population for training (eg, all supervisors). The guideline aimed to improve the mental health of workers by ensuring that all supervisors were trained and would therefore be better equipped to support the mental health of their workers.

While these reviews have made a helpful start in illustrating what effective supervision training can look like, they have their limitations. For example, these reviews do not include recently published papers (ie, papers published from 2010 onwards), they only include a small number of papers (ie, two systematic reviews with 7 and 11 controlled studies, respectively,33 40 and one narrative review which did not clearly indicate the number of papers reviewed).38 Additionally, these previous reviews tend to focus either on the mental health setting only or on supervisory support for workers’ mental health, thereby failing to explore supervision training across a diverse range of health and human services professions. Furthermore, none of these reviews explore the extent to which supervision training interventions produce their effects, how, for whom and under what contexts. Therefore, the synthesis described in this protocol aims to address this gap in knowledge. Using a realist approach, this study will allow interpretability of findings across different contexts and will address the question of what supervision training interventions work, for whom, under what circumstances, how and why.

Realist synthesis methodology

Traditional methodological approaches to evaluating interventions, such as randomised controlled trials, are orientated towards conceptually simple questions of causality such as: ‘is this intervention effective or not?’ Such a question of causality tends to be better suited to clinical treatments rather than complex health and education interventions such as supervision training. Realist methodology, therefore, is being used increasingly to study the workings of complex health and education interventions.41–46 Realist approaches privilege context in the evaluation of interventions. For example, a realist approach acknowledges that an intervention that works well within any given healthcare service (eg, supervision training for doctors) may not work well within a human service setting in the community (eg, supervision training for housing or child protection professionals). Therefore, understanding contextual variation and how this impacts the effectiveness (or otherwise) of an intervention is crucial to the development and implementation of complex interventions. This review will be underpinned by scientific realism, which is a logic of inquiry that generates distinct research strategies and uses research methods and techniques within these to ascertain an in-depth understanding of causality.47 Such realism will help us to unpack and synthesise hidden mechanisms at play within primary studies, thus allowing us to gain deeper understandings about how supervision training as a complex programme works or fails to work and why in our chosen different settings.38

Study aim and research questions

This study aims to address the ways in which supervision training interventions impact and influence supervisory practice, for whom and under what circumstances, how and why. The research questions for this realist synthesis protocol are as follows:

In what ways do supervision training interventions enable or inhibit successful supervision, for whom and in what contexts?

What are the key mechanisms that influence supervision training outcomes?

Methods and analysis

The protocol for this review is registered with the International Prospective Register for Systematic Reviews (PROSPERO).

Study design

This study will be guided by Pawson’s46 five stages for conducting a realist review: (1) clarifying the scope of the review; (2) determining the search strategy; (3) study selection; (4) extracting data and (5) synthesising the evidence and drawing conclusions. Although presented in a linear way here, the review process is iterative in nature and therefore may not follow this process precisely. The findings of this synthesis will be written up according to the Realist and MEta-narrative Evidence Synthesis: Evolving Standards (RAMESES) publication standards for realist syntheses.48

Clarifying scope of the synthesis/Identify existing theories

The first stage of our realist synthesis will be to (a) refine the purpose of the synthesis and (b) identify any key theories to be explored. The research team, with the help of a medical librarian, will refine the purpose and identify any key theories through first scoping the current supervision training literature. This will involve creating a matrix identifying existing primary literature, literature reviews, search terms and their synonyms. The lead author will work with the medical librarian to run pilot searches through several databases to test search terms, Boolean operators and proximity searching. These pilot searches will be used to help refine the research questions based on any gaps in knowledge in the empirical literature. This scoping exercise will help in starting to identify the circumstances for supervision training use and the nature of supervision training in the health and human services workforces. While this scoping exercise may start to identify initial programme theories about how and why interventions work (or fail to work) and in what contexts, we anticipate developing programme theories through later stages of our realist synthesis (for example, see ‘extraction’ later).

Search strategy

A comprehensive search of the literature will be conducted in Medline (Ovid), Cumulative Index to Nursing and Allied Health Literature Plus (Ebsco), Scopus, PsycINFO (Ovid), Australian Public Affairs Information Service (Informit), Social Services Abstracts (ProQuest) and Educational Resources Information Center (ProQuest) by the lead author, with input from a medical librarian and coauthors. We do not plan to limit our searches by date. Citations and reference lists of included studies will be checked to identify additional relevant studies.

An extensive range of search terms relating to ‘supervisors’ or ’supervision' and ‘training’ (table 1) will be tested and combined with proximity searching, Boolean operators, truncations and asterisks. An example of a Medline (Ovid) search strategy is included in box 1.

Table 1

Search terms to be used for the databases

Supervisor terms	Training terms
Supervisor*, Supervision	Education
Practice educator*	Professional development
Clinical educator*	Educating
Mentor, Mentoring, Mentors	Faculty development
Trainer*	‘Train the trainer*’
Fieldwork educator*	Personal development
Instructor	CPD
Preceptor, preceptors	Workshop
Placement educator
Clinical teacher*

*This symbol is used to search for both plural and non-plural, for example, trainer* would search for both trainer and trainers.

Searches will be adapted to account for the different functions of each database (see box 1 for an illustrative Medline search).

(supervisor* OR mentors OR mentor OR mentoring OR instructor* OR placement educator* OR practice educator* OR trainer* OR preceptor OR preceptors OR clinical teacher* OR clinical educator* or fieldwork educator*) ADJ3 (training OR education OR educating OR workshop*)

OR

Supervision ADJ (training OR education OR educating OR workshop*)

OR

‘train the trainer*’

OR

(professional development OR faculty development OR personal development OR CPD) ADJ3 (supervisor* OR mentors OR mentor OR mentoring OR instructor* OR placement educator* OR practice educator* OR trainer* OR preceptor OR preceptors OR clinical teacher* OR clinical educator* OR fieldwork educator*)

Study selection

Initial assessment of relevance will be carried out by multiple researchers. Titles and/or abstracts of studies retrieved using the search strategy, and those from additional sources against preliminary inclusion criteria, will be reviewed using the platform Covidence.

The inclusion criteria for studies are as follows: (1) directly relate to one or more of the research questions; (2) relate to the following populations: health, housing, children services, youth and families services, alcohol and drug services and mental health disciplines within the health and human services workforces; (3) directly relate to supervision training interventions; (4) literature must be written in the English language; and (5) should be primary research and peer-reviewed articles only. Note that while we decided to include only peer-reviewed articles in our synthesis rather than grey literature (due to the vast literature on supervision training), we intend to employ grey literature to make sense of our synthesis findings, plus we intend to use grey literature for a subsequent realist evaluation of a supervision training programme. The key exclusion criterion is literature relating to research supervision training interventions.

The relevance of an article will be judged against the synthesis questions and in terms of whether papers can contribute to theory building.48 49 A second independent researcher will check any ambiguities at this stage (ie, articles selected as ‘maybe’ in Covidence). After relevance checks, the full text of the remaining articles will then be retrieved and independently assessed for rigour (researchers will ask, ‘are the methods used to generate data credible and trustworthy?’).49 Multiple researchers will check rigour using either the Medical Education Research Study Quality Instrument (for quantitative studies) 50 51 or the Critical Appraisal Skills Programme qualitative checklist (for qualitative or mixed methods studies) 52. Any disagreements over eligibility based on quality will be resolved through discussion between the researchers, and if this is not possible, an extra reviewer will be brought in to adjudicate. Double checking will be carried out and discussed for approximately 5% of included and excluded papers at the relevance stage and 10% of the included and excluded papers at the rigour stage for quality control purposes.53 The assessment of eligibility for inclusion will be an iterative process as the synthesis progresses and new or refined elements of theory may be required to explain different aspects of the developing theory/theories. Figure 1 outlines the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) process for study selection. The PRISMA diagram will be employed to allow for a clear articulation of the systematic process for this literature review.

Figure 1

PRISMA diagram for realist synthesis of supervision training interventions. APAIS, Australian Public Affairs Information Service; CASP, Critical Appraisal Skills Programme; CINAHL, Cumulative Index to Nursing and Allied Health Literature; ERIC, Educational Resources Information Center; MERSQI, Medical Education Research Study Quality Instrument; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Extracting data

An Excel spreadsheet will be used to document the extracted data from included studies. Given that this study is following a realist synthesis method, the extracted data will include: study characteristics (eg, publication year, study methodology, etc); types of participants; intervention characteristics (eg, workshop, reflective practice, seminar, etc); contexts (eg, study setting); mechanisms and outcomes and any Context-Mechanism-Outcome (CMO) configurations (CMOCs). A team of reviewers will extract data with a minimum of 10% of the papers being double checked, with any discrepancies being resolved through discussion (with a different reviewer if necessary).

The researchers will first go through the full text of each selected paper adding e-notes identifying sections of text that may be interpreted as functioning as contexts, mechanisms, outcomes and any CMOCs for each supervision training intervention. Inspired by other realist syntheses,49 we aim to answer structured questions as part of this extraction process: (1) interpretation of meaning (eg, does the relevant text provide sufficient data that could be interpreted as operating as contexts, mechanisms and/or outcomes?); (2) judgements about possible CMOCs (eg, what is the partial or complete CMOC for the relevant text? Is this CMOC found elsewhere in the same or other documents?) and (3) interpretations about programme theory/theories (eg, how does this partial or full CMOC interplay with developing programme theory/theories?). This extracted data will then be transferred to the Excel spreadsheet, thus collating CMOs and CMOCs both within and across the sample of papers.

It is acknowledged that some articles may not fully describe or provide interventional components or contexts sufficiently, thereby not fully answering the research questions. To elicit this relevant but unpublished information, the research team will contact the study authors to request detailed information on certain aspects of interventions, particularly information on contextual factors and mechanisms that have not been reported. Such information will be added to the extraction spreadsheet if made available to authors.

Synthesise findings and draw conclusions

An Excel spreadsheet will be used to manage the data, the researchers employing realist logic analysis to interpret the data to make inferences about CMOCs and synthesise the findings. Here, we will look for any patterns across the included papers in terms of recurrent CMOCs (so-called ‘demi-regularities’).41 This team-based analysis process will enable discussion of the data between researchers to allow further examination and the development of initial programme theory/theories. The process of synthesis will include: (1) comparison of findings from different studies; (2) using findings from studies to address the purpose(s) of the synthesis; (3) seeking both confirmatory and contradictory findings; (4) beginning refinement of programme theories in light of evidence and; (5) disseminating the review with findings, conclusions and recommendations.47 It is worth highlighting here that while we plan to develop initial programme theories through this realist synthesis, we will be primarily testing and modifying these initial programme theories through a subsequent realist evaluation of an Australian supervision training programme for health and human services workers.

Patient and public involvement

Given that this paper is a protocol for a realist synthesis of supervision training interventions, patients and the public were not involved in the design of the synthesis nor will they be involved in conducting and disseminating the synthesis.

Potential limitations of the realist synthesis

We acknowledge several potential limitations of the proposed realist synthesis. First, while we plan to review supervision training literature from both health and human services aligned with our funding, we recognise that these settings are wide ranging, meaning that our interpretation of context is likely to be challenging. Second, while we will pilot and refine search terms, Boolean operators and proximity searching with the assistance of a medical librarian, we will inevitably omit terms associated with supervision and/or training, for example, critical companion, coaching, facilitation and practice development, meaning that some important evidence may be missed. Third, while we have decided to include only peer-reviewed papers due to the vast supervision training literature, we realise that this excludes potentially important non peer-reviewed grey literature that could facilitate the development of initial programme theory/theories. Finally, while we have decided to employ rigour checking as part of our screening/eligibility stages like other researchers,42 43 45 we know that some realist scholars do not advocate employing measures of rigour.54 While our rigour checking will help to reduce the amount of literature to be synthesised, we will prioritise the realist relevance of papers and how they can contribute to theory building.

Ethics and dissemination

Ethics approval is not required for this study as it pertains to a review of existing evidence. The findings of this synthesis will be documented using the RAMESES publication standards for realist syntheses.48 We expect to disseminate findings via a peer-reviewed journal article, conference presentation(s) and a report to the Victorian Department of Health and Human Services, Australia as the project funding body. The findings have the potential to benefit multiple stakeholders involved in developing, implementing and receiving supervision training, plus they can be used to facilitate continuous quality improvement of supervision training interventions and guide further supervision training research in the health and human services professions.