Advancing Implementation Science Measurement for Global Mental Health Research.

The Need for Implementation Science for Global Mental Health Impact

There is a critical gap in access to mental health care in low- and middle-income countries (LMICs) and among underserved groups in high-income countries (HICs). Although considerable research progress has been made on the efficacy of mental health interventions, the “know-do” gap persists, limiting the availability of evidence-based care. Challenges contributing to the severity and complexity of the mental health implementation gap include social determinants of mental health, the impact of stigma, limited engagement and participation of key stakeholders including service users, and the need for improved mental health measurement and data collection. The mental health impacts of complex challenges including the COVID-19 pandemic and the effects of climate change further emphasize the urgency of addressing this gap.

Implementation science (IS) in mental health research has the potential to advance the uptake and availability of evidence-based care, and yet investment in IS for global mental health (GMH) remains severely low. Improved IS methodology for GMH is needed to advance the understanding of implementation processes, how they interact with complex mental health-specific challenges, and to ultimately enable the design and delivery of evidence-based care that can be effectively scaled-up to reach those in need. Here we describe ongoing work to advance IS methodology for GMH and suggest next steps to further the potential for real-world impact.

Implementation Science and the Role of Funding Agencies

Funding agencies are crucial to support IS research for GMH. The Global Alliance for Chronic Diseases (GACD) brought together major funding agencies from 12 countries and the European Union to identify shared priorities and fund mental health-specific implementation research in 2018. This GACD initiative represents an important step to place IS at the core of funded global health research. The funded mental health and projects offer a template for ensuring harmonized progress that is global in scope, with 62% of mental health projects taking place in LMICs and 38% in HICs.

Networks supported by funding agencies can play a key role in advancing IS. The GACD supports working groups that promote knowledge sharing and data synthesis across communities of GMH researchers. The GACD Mental Health Data Optimization Working Group (“the Working Group”) is made up of GACD-funded researchers from several countries (four HICs and three LMICs). The goals of the Working Group are to map IS methods and create a standardized overview of implementation assessment across a diverse portfolio of GACD-funded GMH initiatives, thereby advancing the development of methodological frameworks to measure implementation outcomes in GMH. This work builds on previous efforts to standardize implementation measures by the Mental Health Innovation Network (MHIN, mhinnovation.net), funded by Grand Challenges Canada (GCC), which advanced the collection and standardization of GMH implementation metrics via the development of a Core Metrics Framework for GCC grantees.

Mapping Implementation Metrics Across a Portfolio of Projects

In partnership with global funding agencies, the GACD has funded 34 mental health projects across a spectrum of mental health conditions, target populations, and implementation contexts (see Supplemental Materials). This range of projects represents both a challenge and opportunity for implementation data collection and standardization. The GACD Working Group carried out a survey in the first year of mental health project funding to discern the core metrics domains (project development, delivery, evaluation, and context) as defined by MHIN, and to identify unique and common implementation and clinical measures across projects. The results showed great diversity in approaches and measures.

Given this diversity, the Working Group decided to map measurement approaches onto an IS framework to help to summarize these approaches across a standardized set of implementation domains. The Reach, Effectiveness - Adoption, Implementation, Maintenance (RE-AIM) framework is one of the most commonly used IS frameworks in the health and behavioral sciences, with domains capturing individual, organizational, and broader systemic levels. Box 1 briefly outlines the RE-AIM domains. Initial findings showed that the “evaluation” and “delivery” core metrics mapped well onto RE-AIM, but that there were fewer core metrics that mapped onto the “adoption” component of RE-AIM, suggesting less emphasis on implementation measures compared with more traditional outcome metrics that assess efficacy and reach.

RE-AIM Domains

Reach-The intervention reaches the target population

Effectiveness-The intervention performs as intended

Adoption-The intervention is taken up by service users, staff, communities and health systems

Implementation-Consistency, cost and adaptations considerations support intervention delivery

Maintenance -The intervention performs as intended and is sustained over time in desired settings

Adapted from re-aim.org.

The following year, the Working Group conducted a follow-up survey for projects to map their study methods and measurement tools directly onto the RE-AIM domains. This survey revealed a more equal distribution across all RE-AIM domains (see Table 1), and a more comprehensive approach to implementation evaluation. This could be due to the second survey being administered after projects had more time to fully develop their study methodology and to consider emerging results of formative research.

Table 1.

RE-AIM Dimensions Measured in the GACD Mental Health Projects.

REAIM domain	Element	Project count
Reach	No. of participants	23
	Participant characteristics	26
	Inclusion criteria	23
	Exclusions	18
	Reasons for exclusion	18
	Attrition	18
	Reason for attrition	17
Effectiveness	Improvements in health/well being	19
	Service proxies for improvement	10
	Quality of life	14
	Experiences of care	11
	System improvements	10
	Economic or resource utilisation	16
	Community level changes	13
Adoption	Setting adoption/participation	20
	Setting exclusions	8
	Setting characteristics	18
	Staff characteristics	17
	Staff adoption/participation	18
	Setting attrition	10
	Setting reasons for attrition	6
	Individual attrition	11
	Individual reasons for attrition	10
Implementation	Adherence to intervention components	21
	Consistency of intervention	15
	Characteristics of intervention delivery	19
	Barriers and/or facilitators	23
	Adaptations of the intervention	20
	Time and/or costs	19
	Contextual factors	18
Maintenance	Short term follow-up (6 months) of outcomes	13
	Medium term follow-up (12 + months) of outcomes	14
	Sustainability	10
	Barriers and/or facilitators	20
	Satisfaction—setting or individual	15
	Negative outcomes	12
	Program adaptations	15
	Alignment of intervention to organisation	8

The results of the RE-AIM survey are being analyzed to extract the quantitative tools that are being used across each of the RE-AIM domains. These tools will be listed and summarized, and different versions and linguistic translations will be identified and made available. The context of use of each tool and user experience based on feedback from investigators will also be assessed and summarized. We are also exploring options to collate qualitative study instruments and approaches, which are often developed in a unique context and are therefore more challenging to standardize. This work is expected to identify a suite of implementation and clinical measurement tools that are applicable at each stage of the RE-AIM framework, and across the implementation trajectory. This inventory of tools will be of use to researchers in the GACD network and beyond, and will itself constitute a major contribution to IS measurement and methodology for GMH. While the RE-AIM framework was chosen due to its frequent use in health implementation research and among the GACD-funded mental health studies, there are numerous IS frameworks that may be applied in GMH research. This work is also relevant when employing these frameworks, as many of the RE-AIM elements and related tools are applicable to broader IS concepts.

Next Steps to Strengthen Implementation Impact in Global Mental Health

Advances in methodology that address the specific challenges of GMH implementation are clearly needed to support improved implementation outcomes. A network approach to capturing key GMH implementation mechanisms has been previously recommended. Existing portfolios of funded GMH projects present an opportunity to capture and collate key findings related to IS methodology and real-world experiences related to GMH implementation. For example, a series of papers on GMH implementation barriers and drivers based on the MHIN core metrics domains captures mixed methods findings across GCC funded GMH projects. The ongoing work of this GACD Working Group will further contribute to this work.

Despite increased attention to IS for GMH, scale-up and sustainability of interventions following the completion of funded research remains a substantial challenge that impedes real-world impact. This suggests that beyond methodological advances in IS for GMH, support for facilitating mechanisms such as meaningful and long-term engagement of stakeholders including service users, providers, and policy makers is needed. Findings related to stakeholder engagement in GMH identify the need among researchers for time, resources, and capacity building to enable stakeholder engagement in research as a key facilitating factor in promoting sustainable uptake of GMH initiatives. Short-term funding cycles which are designed to support effectiveness research may be insufficient to enable the engagement needed to promote successful and sustainable implementation.

Increased research and investment in IS for GMH is an important step towards closing the “know-do” gap. To promote real-world impact and sustainability in GMH, methodological advances, learning across networks, and funding structures that enable facilitating factors such as stakeholder engagement will be essential.

Click here for additional data file.

Supplemental material, sj-docx-1-cpa-10.1177_07067437221078411 for Advancing Implementation Science Measurement for Global Mental Health Research by Jill K. Murphy, Pallab K. Maulik, Keith Dobson, Ishtar Govia, Raymond W. Lam, Candelaria I. Mahlke, Annabel S. Müller-Stierlin, Ionela Petrea, Carla Aparecida Arena Ventura and Melissa Pearson in The Canadian Journal of Psychiatry