Impact of a videoconferencing educational programme for the management of concurrent disorders on nurses' competency development and clinical practice: protocol for a convergent mixed methods study.

INTRODUCTION: Extension for Community Healthcare Outcomes (Project ECHO©) is an innovative model for continuing professional development that uses videoconferencing technology to support and train general practitioners remotely. The model has been replicated to a variety of settings and locations for capacity building in healthcare professionals caring for patients with chronic and complex health conditions. Limited research has been conducted so far on the impact of ECHO in the field of concurrent mental health and substance use disorders (ie, concurrent disorders (CDs)). Therefore, this mixed methods study aims to develop a comprehensive understanding of an ECHO programme impact for CD management on nurses' competency development and clinical practice. METHODS AND ANALYSIS: The proposed mixed methods study, based on a convergent parallel design, will be conducted in the province of Quebec, Canada, to collect, analyse and interpret quantitative (QUAN) and qualitative (QUAL) data from a specific ECHO Program on CDs. In the QUAN component, an observational prospective cohort study will be conducted over a 12-month period. All nurses who participated in the programme between 2018 and 2020 and who consent to research will be recruited to collect data on the extent of their learning and practice outcomes at three time points. Alongside the surveys, nurses will be invited to participate in individual semistructured interviews. In-depth QUAL data will be subjected to a thematic analysis and will assist in exploring how and in which conditions nurses developed and mobilised their competencies in clinical practice. A comparison-of-results strategy will be used in the final integration component of the study. ETHICS AND DISSEMINATION: This study protocol was approved by the Ethics Committee of the Université de Montréal Hospital Center (#19.295) and the Université de Montréal Ethics Committee (CERSES-20-017 R). We aim to disseminate the findings through international academic conferences, international peer-reviewed journals and professional media. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To the best of our knowledge, this is the first mixed methods study aiming to understand the long-term impact of an ECHO programme for CD management on nurses’ competency development and clinical practice.

In accordance with guidelines for reporting a mixed methods study, the proposed convergent parallel design will rely on a closely coordinated combination of methods and on an explicit wording of when and how integration will occur.

Using an interpretive description approach, the qualitative method will expand on the process of competency development in the context of a videoconferencing educational programme, as well as facilitators and barriers to their mobilisation in clinical practice.

The non-experimental nature of the quantitative method and the anticipated small sample size will limit the results’ interpretation by preventing any causal association and generalisability.

Guided by an innovative social constructionism conceptual framework, this mixed methods study will combine a diversity of outcomes and perceptions to make recommendations about how to effectively support nurses in acquiring a high level of competency.

Introduction

Concurrent disorders (CDs)

CDs refers to co-occurring mental health (MH) and substance use disorders (SUD).1 In addition to their high prevalence,2 CDs are frequently associated with higher rates of relapse, worse psychiatric symptoms, increased odds of unplanned rehospitalisation, poorer treatment compliance and prognosis compared with MH only.3 4 Furthermore, individuals with CDs often experience higher risks of suicide, violence or delinquency, cardiovascular illnesses, as well as transmitted infections.5 CDs are also correlated with poor psychosocial outcomes including homelessness, social isolation, stigma and incarceration.6 A typical example illustrating this clinical problem is a person suffering from a dual diagnosis of schizophrenia and opioid use disorder who also contracted the hepatitis C virus and uses cannabis on a regular basis.7 This complex relationship between MH, SUD and related psychosocial problems highlights the specific needs of individuals living with CDs, which can pose a significant challenge for healthcare professionals attempting to care for them.8

Despite evidence supporting the efficiency of integrated treatment of both MH and SUD,9 10 it has been suggested that individuals living with CDs have poor access to care.11 A report from the Substance Abuse and Mental Health Services Administration12 revealed that among American individuals living with CDs who sought health services in the last 12 months, only 7.4% of them received care for at least one of their conditions. Furthermore, it appears that 55% of the patients’ group received no treatment at all. This gap in access to care is associated with issues in healthcare provision, for example, the lack of competent human resources and suggests a clear need to improve care and patients’ outcomes.13

Nurses’ competencies in caring for individuals with CDs

As the largest professional group within MH and addiction primary care services,14 nurses are uniquely positioned to provide care for patients with CDs. Indeed, nurses are qualified in assessment and intervention delivery and spend most of their time in direct contact with patients.15 However, nurses’ difficulties in caring for patients with MH problems who also problematically use alcohol or drugs have been highlighted in numerous studies.16–18 For example, Wener and Woodgate19 reported that feelings of ambivalence, frustration and powerlessness are common among Canadian primary care nurses who encounter individuals living with CDs. This situation is further exacerbated in rural and remote areas,20 whereas many nurses frequently feel ill-equipped and emotionally exhausted in addressing the care needs of complex and severe cases of patients with CDs.21 22

Consequently, caring for this challenging population can create a significant pressure on nurses, which has been associated with a higher vulnerability to burnout syndrome, low job satisfaction and poor employee retention.23–25 Research also indicates that nurses have suboptimal competencies for screening mental disorders in active users, offering appropriate interventions in crisis situations and coordinating care.11 26 In addition, multiple studies have shown that nurses may hold negative or judgemental attitudes towards patients with CDs.17 27 Results from a systematic review (n=28) indicated that healthcare professionals holding stigmatised attitudes towards SUD have a more task-oriented approach and perceived their knowledge and skills as inadequate.28

Continuing professional education in CDs

In light of the complexities related to CDs, combined with the challenges faced by professionals in providing appropriate care for this priority patient population, there has been a growing recognition that further educational opportunities are needed.29–31 A recent scoping review (n=32) underlines the growing body of evidence with regard to the effectiveness of continuing education in improving attitudes, increasing knowledge and confidence and supporting clinical practice change in nurses working with individuals with CDs.15 In this context, continuing education can be understood as a planned and systematic effort to support healthcare professionals in developing a higher level of competency in order to improve clinical performance and patients’ health outcomes.32

A potential intervention for developing nurses’ competencies in CDs is Extension for Community Healthcare Outcomes (ECHO), a promising model in continuing education using videoconferencing (VC) technology to establish networks between specialists from centralised academic centres and other healthcare professionals.33 ECHO provides telementoring and opportunities for sharing best practices on the management of a wide range of topics and conditions. Ultimately, the programme seeks to reduce variation in care delivery and improve patients’ health outcomes. The ECHO model considers factors affecting professionals’ participation in continuing education activities such as cost, lack of time, long-distance travelling, ability to access and use technology and credential incentives.34 Thus, it constitutes an interesting, flexible and convenient online learning intervention. Indeed, the ECHO model offers professionals the opportunity to discuss real patient cases, share common realities with peers, practise what they have learnt in a safe environment and be positively reinforced.

A systematic review (n=52) reported a positive impact of the ECHO model on healthcare professionals’ outcomes such as satisfaction, knowledge and confidence.35 Furthermore, there is growing evidence in support of the acceptability and feasibility of the ECHO programme, notably for reducing professionals’ sense of isolation in rural settings and in terms of cost-effectiveness.36 There is also a body of research showing the ECHO model effectiveness in terms of providing comparable care to specialists in the case of hepatitis C treatment,37 38 chronic pain management39–41 and geriatric care.42 43 Despite those promising results,35 36 the evidence regarding the impact of ECHO in the field of CDs remains limited. For instance, we found only four studies reporting on MH and/or addiction-specific ECHO programmes.44–47 In three of these studies, increases in professionals’ knowledge using objective measures and self-efficacy using survey rating scales were reported after a 6-month,46 47 and 8-month periods.45 However, these results were statistically significant only in Mehrotra et al.47

In conclusion, the few studies discussed previously included quantitative observational designs, of which we noted several limitations: small sample sizes, lack of longitudinal data, interpretation of findings independently of professional groups and lack of control for cofounders.35 48 The majority of these studies contain insufficient information on how the implemented programme was replicated or adapted and how well it was delivered.48 49 Although few studies have been conducted to assess the impact of ECHO on nurses,50 51 these studies were not specific to CDs and have not adequately captured nurses’ views on how and in which conditions they developed their competencies. Indeed, limited research has been conducted so far to explore ECHO’s impact on competency development, and how these new acquisitions were applied into clinical nursing practice. Thus, there is a need to obtain quantitative (QUAN) results on the long-term impact and to enrich our comprehension of these results using detailed qualitative (QUAL) findings. The present study aims to bridge these gaps by developing a comprehensive understanding of an ECHO programme impact for CD management on nurses’ competency development and clinical practice.

Philosophical and conceptual foundations

Ian Hacking’s social constructionism,52 which emphasises a pluralistic-embedded ontological and epistemological inquiry, will guide this study. This worldview is well suited for planning, conducting and appraising mixed methods research.53 54 Hacking’s proposal for a social theory of science has inspired the development of a conceptual framework for the purpose of achieving a comprehensive understanding of a VC educational programme impact for CD management on nurses’ competency development and clinical practice. Our conceptualisation stipulates that studying the impact of a programme implies the combination of different forms of evidence, each of them producing a mixed kind of evidence using a looping effect (feedback effect). This conceptualisation is illustrated in figure 1 and summarised below.

Figure 1

Integrative conceptual framework for studying the impact of a videoconference educational programme for the management of concurrent disorders on nurses’ competency development and clinical practice.52 56 57

In accordance with Gregor’s typology,55 our conceptual framework provides a description (how we will understand) of the phenomena of interest (the programme impact), the categories of outcomes of continuing education (Moore et al’s56 first five levels of assessment) and the main concepts (competency development and their mobilisation in clinical practice).57 Outcomes and concepts will be described using QUAN and QUAL evidence, respectively. The framework represents the related inputs (data source, collection and analysis), the interactions between the constructs’ ‘outcomes and competencies’ (looping effect between QUAN and QUAL evidence) and how these might be understood (integration of QUAN and QUAL methods). This conception is innovative because it builds on existing and traditional evidence of the ECHO model, while it also suggests an alternative perspective to the current dominance of ‘gap-spotting’ in the literature.58

Methods and analysis

The proposed study seeks to answer the following three overarching research questions:

Q1. QUAN question: what is the evolution in nurses’ outcomes over a 12-month period of participation in a Canadian ECHO programme for CD management?

Q2. QUAL question: how and in which conditions nurses mobilised in their clinical practice the competencies they perceived to have developed through their participation?

Q3. Mixed QUAN and QUAL question: in what ways the evolution in nurses’ outcomes over a 12-month period of participation in a Canadian ECHO programme for CD management is related to the development and mobilisation of their competencies in clinical practice?

This protocol was organised according to the Good Reporting of A Mixed Method Study checklist,59 the Strengthening the Reporting of Observational studies in Epidemiology guidelines,60 the Consolidated Criteria for REporting Qualitative research checklist61 and Phillips et al’s62 guideline for Reporting Evidence-based practice Educational interventions and Teaching. The items of the checklists are fulfilled in online supplemental file 1.

Study design

This study will use a mixed methods convergent parallel design for the purpose of complementarity and comparison between QUAN and QUAL methods.63 We will collect and analyse QUAN and QUAL data simultaneously and independently. Then, we will combine QUAN and QUAL results in the final integration of the study (see figure 2). Integration—the merging or dialogue between the QUAN and QUAL component of the study—will allow the QUAL results to give meaning and depth to the QUAN results collected over a long period.64 In the integration component of the study, the knowledge yield via QUAN and QUAL methods undertaken independently will achieve a ‘whole greater than the sum of the parts’.64 In other words, we do not aim to explain QUAN results and do not need to wait for them to undertake the QUAL component of the study.

Figure 2

Visual model for the mixed methods convergent parallel design. QUAL, qualitative; QUAN, quantitative.

As shown in figure 2, the planned convergent parallel design involves three components, C1, C2 and C3, which address the research questions Q1, Q2 and Q3, respectively.

(C1) QUAN data collection and analysis: Among nurses who participated in a Canadian ECHO programme for CD management, we intend to examine the evolution of nurses’ outcomes over a 12-month period. This will be achieved using an observational prospective single group cohort study design (QUAN method).

(C2) QUAL data collection and analysis: we will conduct individual semistructured interviews using an interpretive description approach65 with nurses who participated in the ECHO programme. We aim to explore the competencies that nurses perceive to have developed in CDs and to enlighten our understanding of how and in which conditions they mobilised these competencies in their clinical practice (QUAL method).

(C3) Integration of QUAN and QUAL results: the integration aims to merge the QUAN and QUAL methods.54 The final product of the integration, which is also the final result of the study, will be obtained by considering the insights gained from merging nurses’ outcomes on the programme impact (QUAN results) with nurses’ perceptions of their competency development (QUAL results).

Setting, VC educational programme and technical features

Based on the ECHO model, this study will use VC technology to connect professionals across multiple settings and disciplines in real-time educative sessions with an interdisciplinary team of experts in CDs. The programme was developed in 2017 and implemented in September 2018 in a Canadian tertiary hospital in the province of Quebec. One of the promising elements of the programme is the group modality, according to the premise that social interactions are essential conditions for effective learning to occur.66 Thus, the ECHO model endorses that sharing professional experiences improves and reinforces learning, while peer support enhances one’s motivation and self-perception of being able to perform new behaviours.67 To consolidate learning, opportunities for practice and feedback in authentic settings are also provided.

The educative programme is organised in scheduled virtual sessions with the Zoom platform (Zoom Video Communications Inc, 2016), which allows professionals to connect with and see each other in real time from their work or home environment. PowerPoint presentations can be shared throughout the sessions, and a chat forum is also available to participants. To run a virtual session online, each participant needs a desktop or a laptop computer (Windows, macOS or Linux), a reliable internet connection (broadband wired or wireless), speakers and a microphone (built-in, USB plug-in or wireless Bluetooth) and a webcam or HD cam (built-in, USB pug-in, an HD cam or HD camcorder with a video-capture card). Participants also have the possibility to join the virtual sessions from their tablet, phone or any mobile device. Prior to the beginning of the programme, information and tips about running a Zoom session are provided to participants by email. Before each virtual session, an encrypted Zoom meeting link with an encryption used is sent only to registered participants to prevent hacking of the platform. A computer scientist is also available in person for the expert team and online for other participants to resolve any technical issues. Registered participants are not required to hold a Zoom licence to run the virtual sessions online. More information about VC equipment and other technical features of the VC setting can be found in Phillips et al’s62 guideline (see online supplemental file 1).

Each curriculum of the programme includes a total of 20 sessions delivered over a 10-month period (September–June). The sessions last 1 hour and a half and are held at 2-week intervals. During each session, a professional or a team of professionals present a patient case for management guidance. The case-based discussions are supplemented with expert recommendations and short didactic presentations on a disease-related topic that are both tailored to professionals’ needs along the curriculum (see online supplemental file 2).

C1: quantitative method

Participants and recruitment

All nurses who participated in the ECHO programme on CDs for the 2018–2019 and 2019–2020 curricula will be invited to participate in the study (n=65). Nurses will be eligible for inclusion if they hold a valid licence of practice. Participation will be considered if nurses have attended at least one virtual session during an entire curriculum. This will ensure that eligible participants have a minimal exposition of the educative intervention,35 without excluding those who participated more passively and may potentially benefit from the program.68

Outcomes and measurements

We intend to collect a number of outcomes as part of a self-administered online survey (SurveyMonkey, 2019–2020) including nurses’ participation, satisfaction and acceptability regarding the programme, attitudes towards patients with CDs, knowledge in CDs, self-efficacy in CDs management and perceived clinical performance.56 Sociodemographic and practice information will also be collected. Table 1 summarises the QUAN outcomes’ description and operationalisation, with corresponding sources of data to be captured.

Table 1

QUAN outcomes’ description and operationalisation, with corresponding sources of data

Assessment levels and desirable outcomes in Moore et al’s56 framework	Outcomes measurement in this study	Corresponding sources of QUAN data	Number of items or questions	Internal consistency
Level 1Participation	Participation	Attendance records within the last 6 months. Learning objectives and motivations for attending the programme. Number of virtual sessions in which nurses interacted (verbal interaction or within the chat forum) with the other participants within the last 6 months. Number of virtual sessions in which nurses presented a patient case within the last 6 months. Perceived level of participation within the last 6 months.	Three questions	Study-specific questionnaire
Level 2Satisfaction	Satisfaction and acceptability regarding the ECHO programme	A self-reported questionnaire including measurement of the degree of nurses’ satisfaction and acceptability regarding the programme. The questionnaire comprises the following dimensions: quality of the information (2), quality of the technological infrastructure (1), general satisfaction (2) and perceived usefulness of the programme (8).81	13 items	NA
Level 3ADeclarative knowledge	Declarative and procedural knowledge: Attitudes towards patients with CDs. Knowledge in CDs.	A French version of the Comorbidity Problems Perceptions Questionnaire to measure nurses’ attitudes towards patients with CDs.82	33 items	α=0.9083–85
Level 3BProcedural knowledge		A declarative and procedural knowledge test in CDs consisting of four patient vignettes.	16 multiple-choice questions	Study-specific questionnaire
Level 4Competence	Self-efficacy in CD management.	A self-reported questionnaire measuring nurses’ level of confidence in CDs management.86 87	19 items	Study-specific questionnaire
Level 5Performance	Perceived clinical performance	A self-reported questionnaire measuring: (1) the number of patients with CDs seen or followed in the last 6 months, (2) the number of patients with CDs that nurses were able to manage without referring them to a specialised service in CDs and (3) the extent of nurses’ application of the experts’ recommendations and/or new learning acquisitions in their clinical practice within the last 6 months.	Three questions	Study-specific questionnaire
Level 6Patient health	NA
Level 7Community health	NA

Note: adapted from ‘Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities’, by D E Moore, J S Green and H A, Gallis, 2009, Journal of Continuing Education in the Health Professions, 29(1), p. 3. This study will not report on patient and community health outcomes. While Moore et al’s56 framework has seven levels of assessment, this study will focus on the first five levels as part of an initial programme investigation. We have added attitudes towards people with CDs since they are known to play a central role in the quality of care delivery.22

α, Cronbach’s alpha; CDs, concurrent disorders; NA, not applicable; QUAN, quantitative.

A visual representation of when each outcome will be measured during the study is shown in online supplemental file 3. At each data collection point, a link to the survey will be provided to nurses, and they will be asked to complete it within a 3-week period. The instruments to be used in the survey were developed with a panel of experts in CDs and inspired from previous studies on the ECHO programme.33 45 Further detail regarding each instrument is provided in online supplemental file 4. QUAN data from nurses will be collected at three time points (before attending the programme (T0), 6 (T1) and 12 (T2) months after the first programme attendance). The developed survey was pilot tested with a small sample of healthcare professionals prior to its use in the study.69

Sample size consideration

We aimed for a sample size of 30 participants with completed questionnaires at T0, T1 and T2. Considering the total available population (n=65) and previous ECHO studies recruitment rates, we assume: (1) 60% of the total hospital nurse population will be recruited and will complete the baseline questionnaire; (2) 90% of those recruited will remain in the study at 6 months and will complete the evaluation questionnaires; and (3) 90% of those who participated at 6 months will remain in the study and will complete the final 12-month questionnaires. As this study is descriptive and exploratory, we do not plan to test any specific statistical hypothesis and will not make sample size estimates from a power analytic perspective. The sample size of 30 will allow us to estimate the binary outcomes of participation with no more than 18% precision using the normal approximation to the 95% CI.

Statistical analysis plan

We will use simple univariate descriptive statistics to evaluate the ECHO programme impact. The available sociodemographic and the evolution in nurses’ outcomes (participation, satisfaction, knowledge, attitudes, self-efficacy and clinical performance) at each time point will be calculated. For continuous variables, descriptive statistics (mean, SD, median, IQR, minimum and maximum) will be provided. For categorical variables, the number of patients and the percentages will be reported. Paired t-test will be used for changes from baseline measures. If we have enough data, the bivariate analyses will examine the sociodemographic and professional level factors associated with study outcomes. The evaluation in time for all longitudinal variables will be presented in graphical form. The effect sizes of the ECHO programme impact with 95% CI will be extracted from the respective statistical models. The SAS V.9.4 statistical software will be used for all calculations (SAS Institute Inc, 2020).

C2: qualitative method

All nurses with a valid licence of practice who participated in the ECHO programme between 2018 and 2020 for at least one virtual session and who consent to research will be included in the QUAL component. Each potential participant will be recruited by the first author (GC) through email or by phone, according to their preferred contact information. According to Thorne,65 data saturation is not a desired outcome in the interpretive description approach since practical disciplines such as nursing tend to appreciate that subjective experience can theoretically possess infinite variation. Instead, interpretive description focuses on obtaining a deeper understanding of participants’ perspective while recognising that variation in perceptions may exist.70 Considering that QUAL strategies for data collection are time consuming and based on prior studies in the field of nursing education,70 we anticipate that 50% of the whole population (n=65) will agree in taking part of the QUAL component (n=±32). However, sample size will be evaluated on an ongoing basis as participants volunteer: by noting nurses’ characteristics (ie, registration year in the ECHO programme, academic background, past and current clinical experiences and healthcare settings) and by ensuring that the data gathered from participants are rich enough to answer the QUAL research question (Q2).

Semistructured interviews

We will conduct individual semistructured interviews with the Zoom platform for the purpose of exploring nurses’ perceptions of how they developed their competencies over their participation in the ECHO programme and how they mobilised them into clinical practice when attempting to care for individuals with CDs. We also intend to collect data on nurses’ perceptions of the conditions that might have foster or hinder their competency development (eg, personal or contextual factors, educational programme components and technical setting). Interviews will allow us to investigate nurses’ perceptions of the programme benefits for their competency development and to have the opportunity to explore how the programme could be further improved. Interviews will be guided by prompts as well as open-ended questions (see online supplemental file 5). Nurses will be encouraged to give significant examples from their clinical practice and to reflect on their clinical decisions and actions. Interviews will last approximately 1 hour and will be conducted by the first author (GC) at the time each participant will deem convenient.

Qualitative data analysis

QUAL data will be subjected to a thematic analysis.71 In interpretive description, data collection and analysis occur concurrently each informing the other in an iterative process.70 Therefore, the analytic process will begin after conducting a first interview and will approximately last 9 months. This will enable prolonged and close contact with the data. Once the interviews are transcribed by a researcher assistant, the first author (GC) will be listening to the audio recordings to familiarise herself with the data and to verify transcripts’ accuracy. Data will be imported, coded and managed using the QUAL data analysis software tool MAXQDA Standard (MAXQDA – Distribution by VERBI GmbH, 1995–2020).

The following step will be taken to scrutinise each interview transcript for meaningful units (quotations) and to generate initial codes. The codes will be subsequently classified under broad categories for further analysis. In each category, codes will be examined, and common themes among the codes will be searched to create a first set of themes representing the data patterns. The next step implicates to make sense of the data by reviewing the meaningful themes within each interview transcripts and across datasets. In this way, a hierarchy of themes pinpointing relationships between themes will first be created, and the themes will be refined and renamed until the emergent results contribute to answer Q2.

Confirmation of coding and themes will be performed at each stage of the process with two other researchers (JC and JP) in order to prevent researcher bias. Trustworthiness of the QUAL findings will also be validated using a member checking technique,72 in which the analysed data (ie, emergent themes) will be returned to each participant to check for accuracy and resonance with his or her perceptions.

C3: integration of QUAN and QUAL results

In the integration component of the study, we will adopt a comparison strategy to combine and contrast the QUAN and QUAL results.54 In this strategy, the resource composed of the QUAN and QUAL results obtained through separated data collection and analysis, and the final product constitute the mixed methods inferences. Figure 3 summarises the overall process and procedures to be applied in the comparison strategy.73

Figure 3

Mixed methods procedures for integrating QUAN and QUAL results using a comparison of results’ strategy.54 73 CDs, concurrent disorder; ECHO, Extension for Community Healthcare Outcomes; QUAL, qualitative; QUAN, quantitative.

To achieve this comparison, we will use a matrix technique to summarise and display the QUAN and QUAL results, thereby allowing consideration of both methods simultaneously.74 This technique will assist in conducting a ‘third effort’ of analysis75 and deepen our understanding of the overall results. Statistical results from surveys and emergent themes from the interviews will be charted in order to concisely map the main study findings on the same page. Information will be organised in a results-by-participant fashion.64 This organisation will allow us to pay attention in unexpected relationships and paradoxes between QUAN and QUAL results shared within a single participant, and then search for patterns across all participants.76 Findings from each component of the study will be assessed for agreement (convergence and similarities) and complementarity. We will meticulously scrutinise any apparent divergences (contradiction, discrepancy or dissonance), and consider possible explanations to resolve them.77 Figure 4 shows an example of a matrix.

Figure 4

Example of a matrix to compare QUAN and QUAL results. QUAL, qualitative; QUAN, quantitative.

Patient and public involvement

Patients and the public will not be involved in the design of, recruitment for and conduct of this study. However, nurses will be invited to reflect on their appreciation of the programme content, structure and delivery (eg, didactic presentations, teaching methods, experts’ recommendations, time allowed for case-based discussions, schedule, VC equipment, material and other resources) through an online survey after each virtual session (as part of the programme requirements to obtain continuing education credentials) and as part of the online satisfaction and acceptability questionnaire developed for this study. The programme has also implemented an ‘ECHO participant committee’ in which voluntary nurses are asked for suggestions and/or modifications in order to better adapt the programme to their learning and practice needs (ie, programme improvements). Education for healthcare professionals has been identified as a priority research area for its impact on the quality and security of care and relation to patients’ outcomes.78 We expect that the study findings will inform on the conditions and strategies that are likely to strengthen the effectiveness of educational programmes in CDs.

Current status

This mixed methods study is an addition to an ongoing prospective observational cohort study for which ethical approval has been obtained in December 2018 from the Ethics Committee of the Université de Montréal Hospital Center (#18.245). As of December 2020, we are currently collecting QUAN and QUAL data, and we will continue collecting through 2021.

Ethics and dissemination

Ethical considerations

This study protocol was approved by the Ethics Committee of the Université de Montréal Hospital Center on 14 January 2020 (#19.295) and from the Université de Montréal Ethics Committee (CERSES-20–017 R) on 30 January 2020. The proposed study follows the Canadian Tri-Council Policy Statement ethical guidelines79 as described in the paragraphs below.

First, written informed consent will be obtained from nurses for both QUAN and QUAL data collection. On registration to the ECHO programme, a first electronic consent form will be sent to nurses informing them on the nature and specific objectives of the QUAN component of the study. Nurses will also be asked their contact information and if they agree to be contacted for further research projects. This information will be stored in an encrypted and separated document on a secure server of the tertiary hospital research centre. The contact information will be maintained until the QUAL data collection stage, so that surveys can be sent and linked with responses and interviews’ transcripts. Contact information will be removed and replaced with a random identification code after interview completion. A second electronic informed consent form will be sent to nurses engaging in the semistructured interviews. Before each interview, the first author (GC) will make sure that nurses are aware that the discussion will be audio recorded.

Second, several precautions will be taken to ensure that nurses understand that their participation in the study is voluntary and outside of the ongoing programme. Both consent forms will explain that nurses’ participation is confidential and that they have the right to withdraw at any time throughout the study. We will also ensure that nurses are aware that refusing to participate will have no effect on their registration in the programme. Solicitation for the interviews will not take place during the videoconference sessions. This will ensure fairness between nurses and other healthcare professionals and distinction between the study and the programme. All the interviews will be conducted outside working hours, and nurses who accept to participate will be offered a $C50 voucher for their contribution.

Third, this study will collect a combination of electronic and material-based data that will be securely and safely stored throughout the study. All electronic data, including signed consent forms, survey responses, audio files and transcribed interviews, will be stored on a secured server. Answers from completed surveys will be uploaded from SurveyMonkey and organised in a protected-encrypted Excel document by a research assistant. Physical documentation including observation notes taken during the interviews will be exclusively kept within the hospital research centre in a locked drawer of a locked room. Only the first author (GC), members of the research team (JC, JP and DJ-A) and those involved in interview transcription will have access to the raw data. All electronic and material documents will be kept for a 7-year period and destroyed afterwards, in line with the ethics committee’s policies. We also recognise that nurses may be anxious or fearful of reprisal when completing a survey or participating in an interview that explores their knowledge and competencies in CDs. However, nurses will be made aware that the collected data will in no way be used to assess employee performance.

Finally, a key issue concerning linking outcomes from the surveys with verbatim from transcripts interview is whether a participant is identifiable from the information collected. To manage this risk, sociodemographic and practices’ outcomes will be collected and reported in broad categories (eg, area of practice will be collected instead of specific names of work settings). Any information that could identify a particular participant or setting during the interviews will be removed from transcripts. Also, the observation notes will not contain any information related to the participants’ identity, as they will only be referred to by their role (eg, registered, assistant or specialist nurse). The thematic analysis will provide a certain level of abstraction within the raw data so that any association between emergent themes and participants will be prevented. Any identifiable information (ie, interview transcripts and quotations) will be retrieved from the final study report and will not be disseminated as per the plan described below.

Dissemination plan

The dissemination plan will include both traditional (ie, academic publication, scientific conferences and professional congresses) and innovative (eg, Project ECHO Institute software repository, links with local and regional agencies and web portals) means of ensuring the study findings are communicated locally, regionally and internationally. The plan will ensure that the results from this study are accessible to a broad audience and transdisciplinary communities including scholars, students, practitioners, citizens, policymakers and other stakeholders. A summary report will be shared through emails to the participating nurses and available online (http://www.ruis.umontreal.ca/CECTC_echo; https://labo-jutras-aswad.ca/). Finally, it is expected that the results of this study will be suitable for publication in relevant peer-reviewed journals; we plan to prepare a first manuscript detailing the QUAL method and results and a second manuscript including the QUAN and MIXED components of the study.

Strengths and limitations

This study will add to the existing body of evidence regarding the ECHO programme’s impact on nurses’ competency development and clinical practice. Also, we proposed an integrative conceptual framework to guide the proposed study, and we expect that this creative conceptualisation might shed new light on the ECHO programme’s impact. A key strength of this study is that the planned mixed methods convergent design relies on a thoughtful, strategic and synergic use of two methods, which will enhance the overall comprehensiveness of the study results.

However, this study has limitations. Given the observational nature of the QUAN method, a single cohort study design was chosen. Hence, the absence of a control group will limit the QUAN results’ interpretation at the description level, preventing any causal association between nurses’ participation in the ECHO programme and the evolution in their learning and practice outcomes. In terms of generalisability, this study will focus on nurses so that findings may not be broadly applicable to other professional groups in contact with patients with CDs in their clinical practice. Since many of the instruments we intend to use in the surveys are not validated, it is possible that the measurements may not truly reflect the outcomes targeted by this study. Given that participation in this study is voluntary, a selection bias may exist as nurses who do not wish to participate may not have similar outcomes and/or perceptions that we will not be able to capture.

Other limitations include small sample size, loss to follow-up and missing data. Efforts to minimise loss to follow-up will include offering a $C50 voucher as acknowledgement of participants’ contribution, multiple reminders and flexible hours for data collection. It is also possible that participants may over-rate the programme impact in terms of learning and practise benefits due to the presence of the researcher that would challenge the internal validity of the reported results. Nonetheless, these biases should be limited based on the breadth and depth of comprehensiveness that we intend to gain in mixing both QUAN and QUAL methods (corroboration of results).

Finally, we anticipate that divergences between QUAN and QUAL results are a potential mixed methods’ threat to internal validity in the case of a convergent design.63 To address and reflect on this issue, the matrix technique will help to identify those contradictions and clarify how we will resolve them in a transparent and rigorous manner. From another angle, discrepancy can also generate unexpected discoveries, pave the way to new ideas and raise challenging questions that may contribute to further research and practice recommendations. Discussing how divergence issues will be approached throughout this mixed methods study will offer appropriate and meaningful guidance in conducting and reporting convergent designs and perhaps contribute to methodology advancement.80

Despite those limitations, we expect that this study will expand our current knowledge on how effectively the ECHO model supports nurses in developing higher levels of competency in CDs. Furthermore, this study will contribute to ensuring that new ECHO programmes in the field of CDs and future larger controlled studies assessing their efficacy are well designed. There is the potential both to provide guidance to educators, clinical leaders and researchers and ultimately to positively impact access to adequate care by people living with CDs.