Evaluating the delivery of Problem Management Plus in primary care settings in rural Rwanda: a study protocol using a pragmatic randomised hybrid type 1 effectiveness-implementation design.

INTRODUCTION: Evidence-based low-intensity psychological interventions such as Problem Management Plus (PM+) have the potential to expand treatment access for depression and anxiety, yet these interventions are not yet effectively implemented in rural, public health systems in resource-limited settings. In 2017, Partners In Health adapted PM+ for delivery by primary care nurses in rural Rwanda and began integrating PM+ into health centres in collaboration with the Rwandan Ministry of Health, using established implementation strategies for mental health integration into primary care (Mentoring and Enhanced Supervision at Health Centers for Mental Health (MESH MH)). A gap in the evidence regarding whether low-intensity psychological interventions can be successfully integrated into real-world primary care settings and improve outcomes for common mental disorders remains. In this study, we will rigorously evaluate the delivery of PM+ by primary care nurses, supported by MESH MH, as it is scaled across one rural district in Rwanda. METHODS AND ANALYSIS: We will conduct a hybrid type 1 effectiveness-implementation study to test the clinical outcomes of routinely delivered PM+ and to describe the implementation of PM+ at health centres. To study the clinical effectiveness of PM+, we will use a pragmatic, randomised multiple baseline design to determine whether participants experience improvement in depression symptoms (measured by the Patient Health Questionnaire-9) and functioning (measured by the WHO-Disability Assessment Scale Brief 2.0) after receiving PM+. We will employ quantitative and qualitative methods to describe and evaluate PM+ implementation outcomes using the Reach, Effectiveness, Adoption, Implementation and Maintenance framework, using routinely collected programme data and semistructured interviews. ETHICS AND DISSEMINATION: This evaluation was approved by the Rwanda National Ethics Committee (Protocol #196/RNEC/2019) and deemed exempt by the Harvard University Institutional Review Board. The results from this evaluation will be useful for health systems planners and policy-makers working to translate the evidence base for low-intensity psychological interventions into practice. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The proposed study protocol will be one of the first to rigorously evaluate the clinical outcomes of and describe the implementation processes for the real-world delivery of a low-intensity psychotherapy for depression within resource-limited public primary care centres.

We use a stepped-wedge design nested within routinely delivered services, which will allow comparison of treatment versus non-treatment within individual and between cohorts of individuals over time, potentially improving statistical power relative to traditional randomised controlled trials. At the same time, the stepped wedge design ensures that each eligible patient receives the evidence-based intervention for their mental health condition, an ethical imperative within routinely delivered services.

Our ability to describe implementation processes quantitatively is limited by the use of routine data, with attendant challenges in data quality and completeness.

To overcome this limitation, we will use multiple data sources to describe implementation using the Reach, Effectiveness, Adoption, Implementation and Maintenance framework, including both quantitative routine data as well as qualitative semistructured interviews of programme staff.

Introduction

Depression and anxiety are leading causes of disability worldwide,1 yet in low-income and middle-income countries up to 90% of individuals suffering from these disorders receive no treatment.2 To address this treatment gap, there has been a global drive to adapt first-line, evidence-based psychological interventions for delivery by non-specialists. For example, the WHO has developed a brief psychological intervention called Problem Management Plus (PM+),3 a five-session psychotherapeutic intervention based on cognitive-behavioural strategies which can be used by lay persons in multiple contexts. PM+ has been shown to improve depressive symptoms and functioning when delivered by community health workers (CHWs) to women living in urban poverty in Nairobi, Kenya, compared with enhanced care as usual.4 Other lay person-delivered psychological interventions have proven effective to reduce depressive symptoms and improve functioning in a variety of settings.5 6 Evidence-based, manualised low-intensity psychological interventions have the potential to expand the cadres of health workers able to manage common mental disorders such as depression and anxiety across the globe and to significantly improve the lives of people living with these disorders.

Despite growing evidence from clinical trials in low-income countries of the efficacy of task shared psychological interventions for depression and anxiety, strategies for implementing these interventions in resource-limited settings outside of a research environment, especially within government health systems where most healthcare is delivered, are urgently needed. Implementing a new psychological intervention within a government health system is inherently complex and is likely affected by a multitude of factors including human resources shortages, lack of service integration, limited supervision and mentorship and other contextual influences. In order to translate the growing global knowledge on task shared psychotherapies from research environments to practice, effective strategies for implementation and scale-up and the clinical effectiveness of such interventions in real-world settings must be documented.

Partners In Health (PIH), a non-profit organisation working in 10 countries, has supported the public health delivery system in three rural districts of Rwanda for 15 years, through its sister organisation Inshuti Mu Buzima (IMB). Since 2012, PIH has collaboratively implemented the Mentoring and Enhanced Supervision at Health Centers for Mental Health (MESH MH) programme with the Rwandan Ministry of Health (MoH) in one rural district. MESH MH is a set of implementation strategies which facilitates Rwandan front-line public primary care nurses to provide basic mental healthcare for patients with mental health conditions and epilepsy. Integrating mental healthcare, particularly for common mental health conditions such as depression, into primary care is one approach to ensure access to effective mental healthcare.7 However, primary care systems must be supported and strengthened for integration of mental healthcare to be a viable approach to increasing access to care. The MESH MH programme was originally based on an effective implementation programme designed to strengthen healthcare delivery for HIV/AIDS at primary healthcare facilities.8 MESH MH incorporates four specific strategies: (1) decentralised training of health centre nurses in evidence-based mental healthcare guidelines and practices; (2) mentorship of primary care nurses by experienced psychiatric nurses; (3) audit and feedback to primary care nurses for performance improvement and (4) systems based quality improvement (QI) interventions. MESH MH has been described in detail elsewhere as an implementation model.9 From 2014 to 2016, MESH MH was supported by a Grand Challenges Canada implementation grant and mental healthcare for patients with severe mental disorders and epilepsy has now been integrated into the existing government healthcare delivery infrastructure in one PIH-supported district (Burera district) using the MESH MH implementation model. This work was done in close partnership with the Rwanda Biomedical Center Mental Health Division.

Implementation of a low-intensity psychotherapy for depression/anxiety

Outcome data from early years of MESH MH supported mental health service delivery by primary care nurses at health centres have shown significant improvements in symptom burden and functioning for service users, as well as good fidelity to the implementation model as MESH MH was scaled throughout Burera district.10 Given the success of MESH MH as an implementation framework for delivering evidence-based packages of mental healthcare within primary care settings for severe mental disorders and epilepsy, MESH MH had the potential to be a useful delivery strategy for other types of mental health interventions in similar settings, including the delivery of psychotherapy for common mental health conditions by primary care nurses. In 2016, PIH Rwanda adapted the WHO’s PM+ for the Rwandan context. Existing PIH care guidelines for depression, articulated initially by PIH and Zanmi Lasante (PIH’s sister organisation in Haiti), which identify and match key triage rules based on depression screening scores,1112 were also adapted for use in the Rwandan context. PM+ was initially piloted at four health centres beginning in October 2017, using MESH MH implementation strategies, with plans for a sequential rollout of the intervention at district health centres until district coverage (19 health centres) is complete. As primary care nurses were integrating PM+ delivery into their routine activities, the initial pilot programme included the delivery of PM+ during one clinical day per week (approximately 4–5 patients daily) for each participating health centre nurse.

Study rationale and design

The overall aim of our study is to evaluate the clinical effectiveness and implementation of PM+ delivery at Burera district health centres, using the established MESH MH implementation strategies. This evaluation will contribute important insight into the scaling-up of task-shared, low-intensity psychotherapy within government primary care settings in a resource-limited, rural area.

We will use a hybrid type 1 randomised effectiveness-implementation design to perform this evaluation. Hybrid type 1 effectiveness-implementation designs focus on testing the effects of a clinical intervention on relevant outcomes while observing and gathering information on implementation, in order to more rapidly move interventions from effectiveness testing through implementation to public health impact.13 We chose to use this design as PM+ has shown efficacy in clinical trials but effectiveness has not yet been demonstrated in primary care settings in Rwanda and little information exists on implementation for low-intensity psychotherapy delivery in low-resource settings outside of a research environment.

To study the clinical effectiveness of PM+, we will use a pragmatic, randomised multiple baseline design to determine whether participants experience improvement in depression symptoms and functioning after receiving PM+. We will use routinely collected data and qualitative interviews to describe implementation outcomes for PM+ at health centres in Burera district. These outcomes will be defined by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, a framework designed to evaluate multiple dimensions of evidence-based intervention implementation in order to determine the public health impact and to shape scale-up processes (table 1).14

Table 1

RE-AIM framework applied to implementation evaluation

RE-AIM dimension	Definition	Implementation question
Reach	Does the intervention reach the targeted population?	Does PM+ delivered in health centres reach those who might benefit from it?
Effectiveness	What is the impact of the intervention?	Do patients receiving PM+ experience improvements in symptoms and functioning? How do patients describe their experience of PM+?
Adoption	Is the intervention adopted by target staff, settings or institutions?	Do health centres adopt PM+ into their service delivery platform?
Implementation	Is the intervention delivered properly?	Is PM+ delivered with fidelity? How do health centre staff experience the delivery of PM+?
Maintenance	Does the intervention become institutionalised?	Do health centres continue delivering PM+ after 1 year?

PM+, Problem Management Plus; RE-AIM, Reach, Effectiveness, Adoption, Implementation and Maintenance.

Specific aims

Aim 1

Clinical effectiveness: Assess whether patients with moderate or severe depression (Patient Health Questionnaire (PHQ)-9 Score≥5) who receive PM+ at select health centres experience clinical remission (defined as a 50% improvement in PHQ-9 Score) and functional improvement (defined as a 20% improvement in WHO-Disability Assessment Scale (DAS) Brief Score) following participation in PM+.

Aim 2

Implementation: Describe and assess the implementation outcomes of PM+ delivery in Burera district health centres using the RE-AIM framework. This framework will use a mixed methods approach, including routinely collected service use data and routine PM+ clinical supervision checklists, as well as semistructured interviews of key stakeholders, including patients, health centre directors and primary care nurses.

Methods and analysis

Setting

The MESH MH implementation programme is now used at all 19 health centres in Burera district to support primary care-delivered services for severe mental disorders and epilepsy and is currently used at eight health centres to support the delivery of PM+, with plans to scale to all 19 health centres in the district. In addition to the 19 primary care health centres, the district health system also includes a 150-bed general hospital (Butaro Hospital), which operates as the district referral centre for acute medical and psychiatric problems. Butaro Hospital houses an outpatient specialty mental health clinic staffed by several government psychiatric nurses and a government psychologist. The district health system also includes approximately 1500 CHWs who are based within villages and affiliated with health centres as community linkages to the formal health system.

Psychotherapeutic intervention: PM+

PM+ is a five-session weekly intervention designed for use in individuals impaired by distress, developed by the WHO and is comprised of four evidence-based techniques for the management of mood/anxiety: (a) stress management through relaxation; (b) practical problem management; (c) behavioural activation and (d) accessing social support. Each session lasts approximately 1 hour. PM+ has been adapted for context by the IMB mental health team, following the basic steps outlined in WHO guidelines for translation of instruments.15 Further description about the linguistic and cultural adaptation of the PM+ manual has been detailed elsewhere.16 First, the PM+ manual was translated into Kinyarwanda by an external translation company familiar with mental health terminology and principles. The manual was then back translated by the IMB mental health team, with any discrepancies in language or terminology resolved by a panel including members of the IMB mental health team and other IMB staff with experience in English–Kinyarwanda translation. Following this process, PM+ was initially pilot tested with 10 Kinyarwanda speaking patients with depression at the district outpatient mental health clinic to ensure face validity for the rural Rwandan context and to make any final linguistic clarifications, as well as to informally determine acceptability and feasibility for delivery in rural Rwanda.

Patient and public involvement

Patients were not involved in setting the evaluation questions or the outcome measures, but during initial pilot testing of the intervention, feedback was sought from patients participating in PM+ to ensure appropriateness and acceptability of the intervention in the rural Rwandan context.

MESH MH implementation strategies for PM+

Strategy 1: training

Each primary care nurse (two per health centre) who will deliver PM+ receives an initial 40 hours training focused on PM+, taught by IMB staff psychologists and government psychologists based at Butaro Hospital. The training curriculum focuses on screening and diagnosing depression using the PHQ-9, PM+ and its clinical foundation and the PM+ techniques. The training also includes sessions designed to improve nurses’ general clinical acumen such as communication skills and developing rapport with PM+ recipients, managing challenging emotional situations, and responding effectively to a variety of challenging clinical situations. The training curriculum has been adapted from the published PM+ intervention,3 materials obtained from the WHO and World Vision used during initial efficacy trials and existing MoH and PIH guidelines.

Strategy 2: mentorship

Immediately following the training, health centres newly participating in the PM+ programme establish a PM+ delivery day. On that day, each participating primary care nurse receives a clinical supervisory-mentorship visit by a psychologist mentor from the district hospital or IMB. The programme goal is to complete at least three supervisory visits per month at each participating health centre for at least 6 months, then to taper to monthly visits over the course of a year. Supervisory visits include direct clinical observation of PM+ delivered by the primary care nurse, individual case review, documentation review and brief didactic sessions on relevant topics.

Strategy 3: audit and feedback

A clinical supervisory checklist has been developed to track each nurse’s provision of PM+ at health centres. The goal is to complete a checklist for at least one PM+ session during each supervisory session by the psychologist mentor. The checklist contains dichotomous scoring of key observable features of basic evaluations for depression, including aspects of assessment, treatment and follow-up planning and referral procedures, as well as the components of PM+. Mentors and nurses discuss checklist scores on a regular basis and primary care nurses are provided specific feedback on clinical strengths and areas for improvement based on their checklist scores.

Strategy 4: QI

During each supervision session, the psychologist mentor also uses a structured QI process, a short-term rapid learning approach, to facilitate system improvements for primary care-delivered mental healthcare. The MESH QI process uses continuous plan–do–study–act cycles, which devise health centre derived indicators, identify specific addressable problems and implement, monitor and modify solutions as needed based on the chosen indicators.17 The PM+ mentor uses the QI process to stimulate discussion with a designated health centre clinical and administrative team around systems based performance issues and mental healthcare quality gaps. After gaps are identified, the mentor works together with the health centre staff to formulate and record the specific solutions to improving quality gaps, in order to return to them frequently until resolutions are found.

Study measures: effectiveness

The effects of PM+ on depression symptoms and functioning of participating patients at eight health centres will be determined using a stepped wedge design between cohorts within each health centre.18 The eight health centres have been chosen for convenience as these health centres represent the first set of health centres in the district rollout plan to adopt PM+. Stepped wedge design studies are time-series studies where the treatment is rolled out over multiple time periods. The start time of treatment relative to enrolment therefore differs by cohort, but measurements are taken at the same time relative to enrolment time for all cohorts. This design allows comparison of treatment versus non-treatment both within individual (measurements before vs after treatment for individuals in the cohort with the later treatment start time) and between cohorts, potentially improving statistical power relative to traditional randomised controlled trials. At the same time, the stepped wedge design ensures that each participant receives the treatment by the end of the overall intervention. This design was preferable for reasons of feasibility, as PM+ is a newly implemented intervention and overall clinical capacity at each health centre to implement the intervention at any given time is limited. The design has also been selected given that PM+ is currently the only structured psychotherapeutic intervention offered at heath centres in rural Rwanda, and as the evaluation occurs in a real-world clinical setting, it would be unethical to withhold the intervention from one group of individuals for comparison.

PM+ eligibility criteria

All individuals eligible to receive the PM+ intervention at health centres will be invited to participate in the evaluation. Eligibility criteria for PM+ includes: (1) ≥18 years old and (2) score of ≥5 on the PHQ-9. Those who (1) present a current safety risk (risk of harm to self, violence in the home), (2) are within 1 month of a major traumatic event or personal loss or (3) have active psychotic symptoms, cognitive impairment or other acute neurologic, medical or psychiatric disorder necessitating immediate hospitalisation will be referred appropriately to MESH MH-supported or district hospital mental health services and will not be enrolled in PM+ delivery or the evaluation.

Recruitment, enrolment and randomisation

Brief informational sessions about depression and the availability of PM+, conducted by primary care nurses, psychologist or psychiatric nurse mentors, will be delivered to primary care attendees at the eight health centres in Burera district participating in the PM+ evaluation. Following the information session, patients can self-identify for depression screening to determine their eligibility to participate in PM+. The screening process will occur at a private area of the clinic and will be performed by the primary care provider, psychologist or psychiatric nurse mentor and will include the PHQ-9 and a basic assessment to assess eligibility for inclusion. For those who meet inclusion criteria, informed consent for participation in the PM+ evaluation will be obtained. Those who are excluded for a psychiatric (eg, compromised safety) or neurologic reason will be referred for further evaluation within the existing mental health services delivered at the health centre or directly to district mental health services. Those scoring <5 on the PHQ-9 will be referred to existing primary care and/or CHWs for regular follow-up.

The stepped wedge design will be applied to four cohorts in total over a 1-year period, with each cohort randomly divided into two groups of four patients each, as each health centre can accommodate four patients simultaneously for delivery of PM+. At the start of the intervention in each health centre, recruitment sessions will be held weekly until eight participants screening ≥5 on the PHQ-9 who meet inclusion criteria at the initial recruitment session are enrolled for the first delivery cycle. Using a computerised generation of random numbers, these eight participants will be randomly assigned to two groups comprising the first pair (figure 1). In each pair, group A (four patients) will start PM+ immediately; group B (four patients) will start PM+ after 5 weeks have elapsed, at which time PM+ for group A will be completed. All participants will additionally be offered usual care at the MESH MH clinic, including ongoing supportive care at regular intervals and psychoeducation, as well as medication management of mood for those scoring >15 on the PHQ-9.

Figure 1

Flowchart of enrolment per PM+ delivery cycle at each health centre. PHQ-9, Patient Health Questionnaire-9.

In the weeks leading up to the end of delivery of PM+ for group B, recruitment sessions will resume until eight more eligible patients are enrolled for the next delivery cycle. This process will repeat through the remainder of the study (figure 2). We anticipate that one to three recruitment sessions will be needed to recruit eight individuals at each health centre. A cohort of eight was chosen to ensure that no individual was asked to wait longer than 5 weeks to begin PM+ and instead recruitment sessions would reoccur approximately every 8–10 weeks to enrol subsequent cohorts.

Figure 2

Assessment and PM+ delivery timeline at each health centre. DAS, Disability Assessment Scale; PHQ-9, Patient Health Questionnaire-9; PM+, Problem Management Plus.

Outcome measures

The primary outcomes will be depressive symptoms and daily functioning, measured every 5 weeks beginning from t=0 until all individuals in each cohort have finished with PM+ (figure 2). Depressive symptoms will be determined by the PHQ-9. Functioning will be measured by the WHO-DAS II Brief Scale, a general measure of functioning and disability across a variety of domains relevant to mental illness. Both scales have demonstrated high levels of validity and reliability across multiple cultures and countries and have been translated into Kinyarwanda and used in previous studies in Rwanda.192021 Routine demographic information will also be recorded at each data collection point.

Data collection

The mental health programme manager will maintain the list of individuals who completed a baseline screen during the initial recruitment session. Two trained data collectors will each travel to four health centres weekly to perform the assessments for each PM+ evaluation participant. After each cohort of eight is complete, all evaluation participants will complete a PHQ-9 and WHO-DAS Brief assessment according to the schedule outlined in figure 2. Thus, cohort 1 patients will complete PHQ-9 and WHO-DAS Assessment 1 at t=0 weeks, Assessment 2 at t=5 weeks and Assessment 3 at t=10 weeks. Cohort 2 patients will complete Assessments 1, 2 and 3 at t=10, t=15 and t=20 weeks and so forth. For participants who do not return to follow-up after recruitment, a CHW will be contacted to visit the patient at home and encourage the patient to return to care, as is the procedure during routine services.

Data analysis

We will test the following hypotheses using two separate linear mixed effects regression models, one with PHQ-9 Score as the dependent variable and one with WHO-DAS Brief Score as the dependent variable. For each regression model, we use fixed effects for the predictors corresponding to each hypothesis, health centre and cohort as random effects grouping factors and maximal random effects structure with respect to the hypothesis-critical predictors:2223

Are scores different between groups A and B at Assessment 1? We should see no significant differences between groups, as Assessment 1 is pre-PM+ delivery for both groups and assignment to group is random.

Are group B scores different between Assessment 1 and Assessment 2? There should be no large differences since group B will not have received PM+ before either assessment, but some small differences may be possible due to selection effects of enrolment.

Are group A scores lower than group B scores at Assessment 2? Group A should have significantly lower scores than group B, as group A but not group B has received PM+ at this point.

For group B, is the Assessment 2–3 difference score significantly larger than the Assessment 1–2 difference score? It should be, because group B receives PM+ between Assessments 2 and 3, but does not receive PM+ between Assessments 1 and 2.

Are group A scores different between Assessments 2 and 3? There should be no large differences as PM+ has concluded by Assessment 2, but it is possible that Assessment 3 scores may be lower (suggesting further time-delayed benefits of PM+) or higher (suggesting some degree of relapse of depression symptoms) than Assessment 3.

Are scores different between groups A and B at Assessment 3? We expect no large differences, since PM+ is concluded for both groups at this point, but small differences are possible per the above.

We will also test for interactions with health centre, cohort and data collector for all the above comparisons. Any significant results will be followed up by further analyses treating these as fixed effects rather than random effects grouping factors in order to understand the nature of the interaction.

Sample size and power analysis

We focus on PHQ-9 scores for our power analysis as the PHQ-9 assessment is most widely used in comparable settings and base our analysis on Rahman et al who obtained effect sizes for individual PM+ on PHQ-9 scores of 0.87 (post treatment) and 0.73 (3-month follow-up).24 We take our groups A–B comparison at Assessment 2 as the key test on which we base our power analysis and use the more conservative 0.73 effect size to determine minimum sample size. This is a two-sample test and a sample size of 50 patients in each of group A and group B is sufficient for 95% power. It should be noted, however, that this analysis does not take into account the possibility of variation in treatment effect size across cohorts, health centres or data collectors. For this reason, because this evaluation is part of a real-service delivery of PM+ to as many patients as possible and because each of the eight health centres delivering PM+ can accommodate four PM+ participants simultaneously, we will aim to enrol participants over eight delivery cycles of PM+, for a total of 256 participants. This enrolment goal also will account for an expected drop-out rate of 10%–20% based on previous evaluations.10

Study measures: implementation

We will employ quantitative and qualitative methods to describe and evaluate implementation outcomes using the RE-AIM framework, including routinely collected programme data and semistructured interviews. Implementation outcome measures, collection and analysis are specified by each component of RE-AIM below and are summarised in table 2.

Table 2

RE-AIM implementation outcome measures

RE-AIM dimension	Implementation question	Outcome measure(s)
Reach	Does PM+ delivered in health centres reach those who might benefit from it?	Total eligible patients after screening who complete at least one session of PM+
Effectiveness	Do patients receiving PM+ experience improvements in symptoms and functioning? How do patients describe their experience of PM+?	PHQ-9 and WHO-DAS Brief scores following participation in PM+ Qualitative semistructured interviews of patient participant experience in PM+
Adoption	Do health centres adopt PM+ into their service delivery platform?	Proportion of health centres delivering PM+ compared with total number of health centres
Implementation	Is PM+ delivered with fidelity? How do health centre staff experience the delivery of PM+?	Number of completed PM+ protocols compared with number initiated Qualitative semistructured interviews to elicit implementation experiences of staff
Maintenance	Do health centres continue delivering PM+ after 1 year?	Proportion of health centres delivering PM+ at the beginning and end of the study period

DAS, Disability Assessment Scale; PHQ-9, Patient Health Questionnaire-9; PM+, Problem Management Plus; RE-AIM, Reach, Effectiveness, Adoption, Implementation and Maintenance.

Reach

The reach of PM+ will be measured by comparing the absolute number of participants completing at least one session of PM+, with the total number of individuals screened for PM+ participation and with the total number meeting PM+ entry criteria at the four health centres participating in the effectiveness evaluation. The reach of PM+ will be compared with reported contact coverage for depression treatment in other resource-limited primary care settings.25 26

Data collection

Routine PHQ-9 screening data for entry into PM+ will be collected from the paper registry of all persons participating in the screening process, currently maintained by the primary care nurses in collaboration with IMB. Routine PM+ programme monitoring data, including PM+ session level data, are currently recorded in a health centre-based electronic medical record (EMR). Each month of the evaluation, the data officers will extract deidentified screening data from the paper registries and deidentified session completion data from the EMR and export these data into a password-protected database currently in use by the IMB team for tracking routine programme process indicators. These data will be collected for the entire PM+ effectiveness evaluation period, approximately 1 year.

Data analysis

We will report the proportion of total patients screened and total eligible patients, who complete at least one session of PM+. These proportions will be compared with participation rates in other studies or implementation programmes of PM+ in resource-limited settings.27

Effectiveness

Following the stepped wedge effectiveness evaluation, a subset of approximately 10 PM+ participants in the quantitative evaluation will be chosen purposively and invited to participate in a qualitative evaluation of PM+ effectiveness. Participants will be chosen to reflect a maximal variety of demographics including age, gender and health centre, as well as a variety of PM+ outcome scores (including those who did not experience significant clinical improvement as measured by the PHQ-9, those who achieved average improvement and those who achieved maximal clinical improvement).

Demographic data of participants will be obtained. Semistructured individual interviews will be conducted by the mental health clinician researcher. The interviews will be conducted in Kinyarwanda and audio recorded and the interviewer will take notes for context and non-verbal communication. The interview guide will be developed through an emergent design including qualitative approaches to use of the RE-AIM framework,28 insights gained from the investigators' clinical and programmatic mental healthcare experiences and the literature on the delivery of PM+.

The semistructured interviews will include sections to elicit perceptions and attitudes of participants in PM+ towards access to and uptake of PM+, acceptability of PM+ delivery in primary care settings, quality of care delivery and outcomes for participants in PM+, as well as the health centre nurse as an agent of PM+ delivery. Interviews will be translated into English and transcribed for analysis.

A framework analysis will be conducted.29 Each transcript will be analysed iteratively and themes related to the central effectiveness and implementation evaluation questions, guided by the RE-AIM framework, will be identified. Transcript content will then be coded by using the thematic framework while allowing inductive themes to emerge.

Adoption

The adoption of PM+ will be measured by documenting the total number of Burera district health centres which have delivered at least one session of PM+, compared with the total number of health centres in the district. For the four health centres participating in the effectiveness evaluation, we will also compare the number of clinic days per month over the evaluation period (approximately 1 year) that PM+ is actually delivered to recipients, compared with the expected number of PM+ delivery days during the same period, as determined by IMB programme goals (1 day weekly).

Routine PM+ programme monitoring data are recorded in each health centre’s EMR. Each month of the evaluation, the data officers will extract deidentified session completion data from the EMR and export these data into the password-protected database currently in use by the IMB team for tracking routine programme process indicators. These data will be collected monthly for the entire PM+ evaluation period, approximately 1 year.

We will report the proportion of health centres in the district ever delivering PM+ over the course of the evaluation period. We will also report descriptive statistics for expected PM+ delivery days completed at each health centre and for all health centres.

Implementation

Implementation fidelity will be measured quantitatively by the number of completed PM+ treatment protocols (five complete sessions) at each of the four health centres participating in the effectiveness evaluation, compared with the number of treatment protocols initiated.

We will also qualitatively assess implementation facilitators and barriers to integrating PM+ into primary care from the perspective of health centre directors and primary care nurses.

Data collection (quantitative)

Routine PM+ programme monitoring data are recorded in each health centre’s EMR. Each month of the evaluation, the data officers will extract deidentified PM+ protocol initiation and completion data from the EMR and export these data into the password-protected database currently in use by the IMB team for tracking routine programme process indicators. These data will be collected monthly for the entire PM+ evaluation period, approximately 1 year.

Data analysis (quantitative)

We will report the proportion of completed PM+ protocols at each health centre and across all health centres.

Data collection (qualitative)

Two focus groups, one consisting of four health centre directors and one consisting of four to six primary care nurses, will be conducted by the IMB mental health programme manager. The groups will be conducted in Kinyarwanda and audio recorded and the interviewer will take notes for context and non-verbal communication. A semistructured interview guide will be developed similarly to the interview guide for assessing effectiveness: we will use an emergent design including qualitative approaches to using the RE-AIM framework, insights gained from the investigators' clinical and programmatic mental healthcare experiences in Burera district and the implementation literature for delivery of evidence-based low-intensity psychotherapies in resource-limited settings.

The semistructured interviews will be designed to elicit perceptions and attitudes of primary care staff towards the integration of psychotherapy (PM+) into primary care settings, perceived participant reach and facilitators and barriers to the implementation of PM+ within health centres. Interviews will be translated into English and transcribed for analysis.

Each transcript will be analysed iteratively using a framework analysis, similarly to the qualitative effectiveness evaluation, and themes related to the central implementation evaluation questions, guided by the RE-AIM framework, will be identified. Transcript content will then be coded by using the thematic framework while allowing inductive themes to emerge.

Maintenance

The maintenance of PM+ delivery in the district will be measured by comparing the total number of Burera district health centres which have delivered at least one complete PM+ protocol within the first 3 months of the evaluation period (approximately 1 year), with the number of those health centres delivering at least one complete PM+ protocol within the last 3 months of the evaluation period.

Routine PM+ programme monitoring data are recorded in each health centre’s EMR. Each month of the evaluation, the data officers will extract deidentified session completion data from the EMR and export these data into the password-protected database currently in use by the IMB team for tracking routine programme process indicators. These data will be collected monthly for the entire PM+ evaluation period, approximately 1 year.

We will report the proportion of total health centres in the district which continue to deliver PM+ each month throughout the evaluation period.

Ethics and dissemination

This evaluation has been approved by the Rwanda National Ethics Committee and deemed exempt by the Harvard University Institutional Review Board. During the screening process for entry into the PM+ programme, potential evaluation participants will receive a verbal description of the programme evaluation, including the purpose, voluntariness and confidentiality issues and written consent will be obtained from those agreeing to participate.

There is limited evidence documenting the effective delivery of task-shared psychotherapy outside of research settings in resource-limited areas globally. Our study to evaluate the implementation and effectiveness of PM+ in government-run, primary care settings in one rural district of Rwanda will be among the first of its kind and will provide practical, instrumental information for programmes and governments seeking to deliver non-pharmacologic care for common mental disorders within usual care settings. Demonstrating clinical improvement for individuals receiving PM+ delivered by primary care nurses would strengthen the evidence for the potential public health impact of PM+ when delivered in real-world settings. Improvements in clinical outcomes for patients receiving PM+ at health centres in Burera district would also strengthen the evidence for the MESH MH implementation programme as a scalable set of implementation strategies which are effective in delivering interventions for a wide range of mental health conditions. Results from this evaluation will be submitted for peer-reviewed journal publication, presented at conferences and disseminated to communities served by the programme. If results are positive, recommendations for the use of MESH MH strategies to deliver PM+ through task-sharing in resource limited settings will be developed and presented to key community, government and NGO stakeholders in Burera district, in Rwanda and in other parts of Africa.