Sustainability of 'mHealth' interventions in sub- Saharan Africa: a stakeholder analysis of an electronic community case management project in Malawi.

Background: The global health community and funding agencies are currently engaged in ensuring that worthwhile research-based programmes are sustainable. Despite its importance, few studies have analysed the sustainability of global health interventions. In this paper, we aim to explore barriers and facilitators for the wider implementation and sustainability of a mobile health (mHealth) intervention (Supporting LIFE Community Case Management programme) in Malawi, Africa.
Methods: Between January and March 2017, a qualitative approach was used to carry out and analyse 13 in-depth semi-structured interviews with key stakeholders across all levels of healthcare provision in Malawi to explore their perceptions with regards to the implementation and sustainability of the mHealth programme. Data were analysed thematically by two reviewers.
Results: Overall, our analysis found that the programme was successful in achieving its goals. However, there are many challenges to the wider implementation and sustainability of this programme, including the absence of monetary resources, limited visibility outside the healthcare sector, the lack of integration with community-based and nationwide programmes, services and information and communication technologies, and the limited local capacity in relation to the maintenance, further development, and management. Conclusions: Future developments should be aligned with the strategic goals and interests of the Ministry of Health and engage with national and international stakeholders to develop shared goals and strategies for nationwide scale-up. These developments should also focus on building local capacity by educating trainers and ensuring that training methods and guidelines are appropriately accredited based on national policies. Our findings provide a framework for a variety of stakeholders who are engaged in sustaining mHealth programmes in resource-poor settings and can be used to develop an evidence-based policy for the utilization of technology for healthcare delivery across developing countries.

Introduction

Despite global progress in reducing mortality and morbidity in children aged under 5 years of age, an estimated 5.9 million children died in 20151. The majority of these deaths were concentrated in sub-Saharan Africa (SSA) and were attributed to acute conditions (e.g. malaria and pneumonia) that could be prevented with better access to appropriate healthcare services. In order to reduce these preventable deaths, the World Health Organization (WHO) and the United Nation Children's Fund (UNICEF) developed Community Case Management (CCM), a tool for improving standards and equitable access to first-level healthcare in low- and middle-income countries (LMICs) for children under 5 years of age. In 2007, the Integrated Management of Childhood Illness (IMCI) Unit of the Malawian Ministry of Health (MoH) introduced CCM with support from international and non-governmental organisations (NGOs). The national rollout of CCM began in 2009, and by 2011, CCM coverage across the targeted areas reached 76%2.

Implemented by community health workers who operate in village clinics, known locally as Health Surveillance Assistants (HSAs) in Malawi, CCM, by entering simple signs and symptoms, supports HSAs to better identify common illnesses and refer children exhibiting danger signs to higher level facilities2,3. However, ensuring correct referral is dependent upon the proficiency of HSAs, compliance with CCM guidelines, and the recording of all relevant and correct information in a paper-based tool4–6. Incorrect utilisation of the guidelines required by the paper-based tool can result in over- or under-referral to higher level facilities7–9. Given that an estimated 30–35% of the Malawian population reside more than 8 km from a health centre2,9, and because financial difficulties are major obstacles for patients seeking orthodox care by care-givers10, it is imperative to equip HSAs with the appropriate resources to be able to make better diagnoses and ensure appropriate decisions and referrals.

Mobile Health (mHealth) technology offers a number of solutions to circumvent the existing limitations of CCM delivery, such as real-time supervision and the electronic transmission of data to local Health Information Systems (HIS)11. The Supporting Low-cost Intervention For Disease Control (Supporting LIFE) programme developed a CCM mobile application for use by HSAs12. During this 4-year programme, the Supporting LIFE electronic Community Case Management Application (SL eCCM App) was developed and vigorously tested through a series of HSA workshops, a feasibility study13,14 and a larger stepped-wedge cluster randomised controlled trial (featuring approximately 7000 children) in Northern Malawi14. Following the successful completion of this trial, interest shifted to the wider implementation and sustainability of this intervention.

The framework for conceptualising programme sustainability

The WHO defines sustainability as ‘the ability of a project to function effectively, for the foreseeable future, with high treatment coverage, integrated into available healthcare services, with strong community ownership using resources mobilised by the community and government’15. Research on the sustainability of health projects in LMICs is scarce as most projects investigate the initial adoption and implementation of an intervention. The broader scale-up of interventions is rarely investigated, largely due to constrained timeframes and/or funding for research16,17. Given that funders and policy-makers require evidence to inform decisions relating to resource allocation, it is imperative for research-based projects to investigate their ability to be maintained and adapted for wider use within the healthcare system following expiration of funding. One of the most widely used frameworks in community-based interventions, such as CCM, is the framework for conceptualising programme sustainability18. Chosen for its multidimensional concept towards sustainability, this framework measures the health benefits, continuation (institutionalisation) and capacity-building of a specific health programme. Three major groups of factors are emphasised as potential influences on sustainability: (1) sustainability can take the form of continued benefits to those who received health services during the implementation phase and after funding ends (i.e. project design and implementation factors); (2) continued implementation of programme activities in an organisation following the discontinuation of financial support for the programme (i.e. factors within the organisational setting); and (3) community empowerment to improve health by continuing the activities of a programme (i.e. factors in the broader community environment) (Figure 1). Collectively, these measures allow one to plan for ‘what is to be sustained, how or by whom, how much and by when’19.

Figure 1

Shediac-Rizkallah and Bone framework conceptualising programme sustainability19

Objectives

The general aim of this study was to explore the views of key stakeholders with regards to the factors perceived to affect the wider implementation and sustainability of the Supporting LIFE (SL) programme in Malawi. Stakeholders are framed as actors who have an interest and are relevant to the inception, development and maintenance of the programme. Informed by this framework, our study-specific objectives were to: (1) explore the perceived barriers and facilitators to the wider implementation, scale-up and sustainability of the SL programme in Malawi, (2) identify whether changes need to be made to the SL eCCM App to allow integration with existing systems to meet the needs of the local context and (3) explore the level of organisational and community support required for the national implementation and sustainability of the programme.

Methods

Study design

This was a qualitative study design designed to elicit the perceptions of selected stakeholders regarding the factors influencing the implementation and sustainability of the SL eCCM App in Malawi.

Study population

A purposive sample of predetermined important criteria was used to identify participants (Table 1). We then carried out in-depth semi-structured interviews with selected participants20. A total of 13 interviews were conducted with stakeholders in Malawi, including two HSAs, two district health officers, a zonal health officer, an Integrated Disease Surveillance and Response (IDRR) programmer, a Central Monitoring and Evaluation Division (CMED) officer, an IMCI official from the MoH, a senior member involved in research from Mzuzu University, and four members from three NGOs.

Table 1

The sampling strategy used to identify participants

Objective	Procedure
Criteria	Participants were selected on the basis of being directly/indirectly involved in the design, implementation, evaluation and/or maintenance of the SL programme, spoken fluency in English, and who occupy a role across different levels within the Malawian healthcare system, including HSAs, district health officers, information technology (IT) specialists, MoH officials and NGOs
Method	Participants who met the above criteria were contacted by email and telephone to ascertain their interest in contributing to this study
Interview	In-depth and semi-structured

The Supporting LIFE programme

The SL programme involved the following components: (a) the development of a CCM mobile health application for use by HSAs to assess and treat children with acute conditions in village clinics in Northern Malawi; (b) the design and delivery of workshops and training courses to equip HSAs and their supervisors with the knowledge and skills to use smartphones and computers in everyday practice and (c) the identification of key elements for disease surveillance and the development of a framework for national rollout.

The SL eCCM App was developed as an Android smartphone application that replicates the CCM decision aid tool routinely used by HSAs in Malawi. This app enables HSAs to enter the same information (usually gathered using a paper-based CCM form), including personal details (e.g. gender), clinical symptoms (e.g. fever) and clinical measurements (e.g. breathing rate). Data is entered directly into the application. The app then provides the user with the recommended treatment for the child, such as treatment at home with medication, or referral to a higher-level clinical facility. Completion of all clinical questions and assessment items is mandatory for the app to generate a CCM clinical decision. Further information regarding the app and the SL programme can be found elsewhere13,21.

Five workshops and three courses22 were conducted to equip HSAs and Senior HSAs with adequate knowledge and skills to use smartphones and computers. These also aimed to enhance the participants' understanding of the potential application of eHealth and mHealth in health care service provision and public health policy.

Data collection

In-depth semi-structured interviews were conducted with 13 stakeholders between January and March 2017. Each interview lasted between 20 and 40 minutes. Eleven interviews were conducted in person, one via Skype, and another over the telephone. All but one participant provided consent to record the interview in an audio-digital format; this particular interview was thus recorded using paper and pen. All interviews were conducted in English and were transcribed verbatim.

Interview guide

An interview guide was developed with questions derived from the Shediac-Rizkallah and Bone framework19 and developed in three stages. First, a literature review was undertaken to identify an appropriate framework for community-based interventions in LMICs. Second, based on this framework, an interview guide was developed through consensus amongst a multidisciplinary team of researchers from universities and NGOs in Malawi and universities in Europe and the United States. Third, the interview guide underwent internal validation via face validity assessment in a series of meetings over a 3-month period.

Data analysis

Data was analysed thematically using the Ritchie and Spencer's Framework Approach23. Analysis was conducted independently by two researchers from Imperial College London using a series of well-defined steps to ensure consistency, credibility and the traceability of findings. First, the analysts familiarised themselves with the data by reading through the transcripts and making notes. Next, they developed a thematic framework by identifying a priori themes from the interview guide, emergent issues raised by the participants and analytical themes arising from the recurrence of particular experiences. Then, the framework was applied to the data in its textual form using index prefixes and annotations. Fourth, we formed charts by rearranging the data according to major thematic categories from the framework. Finally, the data was collated to interpret the entire dataset and provide explanations for the findings.

Results

Of the 13 stakeholders interviewed, only one was female; this reflects the disproportionate under-representation of women in decision-making positions24. Participants were exposed in different ways to the SL programme. Four were involved from the inception phase of the programme, two were members of the SL consortium based in Malawi, two individuals utilised the app in the trial, and the remaining stakeholders had experiences of the technology through training and/or dissemination workshops.

Programme design and implementation factors

Programme negotiation

Only four participants were involved in initial discussions during the project negotiation phase: two members from NGOs, a district health officer and an MoH official. Their primary role was to engage all the relevant stakeholders in Malawi, check that correct ethical protocols were followed, and ensure that the objectives of the SL programme were in alignment with the context of Malawi. Other members were involved during the inception of this 4-year project; their departure or transfer from these roles prevented their inclusion in this study.

Programme effectiveness

Overall, the programme was thought to be effective at providing an incentive for HSAs in service provision and collecting evidence on the impact of mobile health interventions in Malawi; this was considered highly useful for the MoH in terms of decision-making and resource allocation. Other benefits that were highlighted included the fact that caregivers attended village clinics rather than going directly to higher level facilities, thus saving time and costs. Another benefit was that the app improved the status of HSAs who were now perceived by many caregivers as being equal to doctors (Table 2). Although the participants noted a greater number of desirable effects, one of the undesirable effects highlighted was that a large number of HSAs and sites were excluded. In addition, the app did not support data retrieval, thus HSAs had to rely on their paper-based records (i.e. a village register) when a child returned to the clinic. Other undesirable effects related to the maintenance of telephones (e.g. the availability of chargers) and general issues related to infrastructure (e.g. connectivity) (Table 2). In addition, although all major healthcare stakeholders were aware of the programme, participants noted that greater visibility was needed in other sectors (e.g. other CCM projects and the IT industry).

Table 2

A priori and emergent themes derived from the framework and transcripts

	Priori Themes	Emergent Themes	Quotes
Programme Design and Implementation factors	Programme Negotiation Process	Consistent engagement with relevant stakeholders at all levels of service provision	“approve NGOs or research being done, so all those processes were done” so please receive them, accommodate them, so it involved the community which was a good thing”
	Programme Effectiveness	Vital information is missed on the paper forms but this is eliminated with the app	“when there is a certain part you have missed, the phone doesn't turn you to go to the next page. In the registers, you can miss some information and you can just go straight to other pages”
		Improvement in accuracy in treatment and case management	“advantage is to do with the accuracy that the app provides is in terms of case management process”
		Reduction in caregivers seeking care in higher level facilities saving time and money	“so some of them did not even come here, they would just proceed. But now they see me using the smartphone and they are flocking here. To them, the quality of service is the same as Bolero, the way they look at it”
		Limited resources impacted the visibility of the project	“other organisations that are dealing with the mHealth or community mobile solutions, they are mainly working in the central and southern regions we didn't put too much energy or emphasis to sort build close relationships with them, we would probably do more of this type of engagement but also depends on resources”
	Programme Duration and Financing	Financial sustenanace through the MoH may, initially, be difficult	“for the MoH, I have my doubts because the ministry is overwhelmed with the basics, the drugs, the equipment, and that”
		MoH in favour of mHealth technology to reach communities	“there is a lot of reaching out to the communities, so I think there is a chance where the mobile application can be easily funded”
		Need to build local technical capacity to maintain the project	“have an external stakeholder to provide technical services and build capacity by having the locals learn from them. These locals can be incorporated into the system to continue the work”
	Training	Need for Training of trainers (ToT) to ensure practices and knowledge of project are transferred	“train them as part of the project. If the project goes, and if there is a need for a refresher, who refreshes them? We could've had a team within a system, who are like trainers of trainers, who can refresh them at a later date”
Factors Within the Organisational Setting	Integration with Existing Programmes	Imperative to integrate with DHIS2 system to address the lack of data entry	“responsible for punching data are the HMIS manager/officer, but since there are so many other programmes, sometimes he gets overwhelmed and the work is too much for one officer”
Factors in the Broader Community Environment	Socioeconomic and Political Considerations	Insufficient human resources	“one HSA has to serve a huge number. The population is widening”
	Community Participation	Communities felt empowered through the introduction of the app	“so that means that our village is intending to be developed So they feel like very empowered, yeah”
		Caregivers felt the HSAs were more knowledgeable from using the app	“that I have gained a certain knowledge from somewhere by using the phones”

Programme duration and financing

Overall, participants identified the MoH as the main facilitator for sustainability and noted that the MoH had the capacity in Malawi to fix problems when identified. Furthermore, the participants felt that the MoH is motivated to commission mHealth technologies but long-term sustainability is dependent on multiple factors. These factors include cooperation and engagement with different ministerial departments, support from external organisations involved in CCM, such as UNICEF and Save the Children, and making the app a more open source in line with the MoH eHealth framework, which was created to ensure that new projects follow standard practice and aligned with Malawi's specific interests.

Training

The participating HSAs found the training very helpful, but also highlighted that HSAs unfamiliar with smartphones needed more time to become adept with the technology and, therefore, were more reluctant to adjust to the new system. Although the project recruited specialists from Mzuzu University to handle technical troubleshooting, one stakeholder revealed that existing district data managers and coordinators should have been consulted for these tasks. One participant noted the importance of having ‘a pool within the system of who can train new ones… training of trainers’ to build capacity and ensure longer-term sustainability.

Factors within the organisational setting

Institutional strength

Overall, the stakeholders agreed that the SL programme would need support from local, national and international organisations to achieve successful scale-up. If the app was adapted to include additional features, then the relevant organisations would have to be included.

Integration with existing programmes

All respondents mentioned various suggestions to integrate the SL eCCM App with other programmes and health services that are offered by the HSAs in the communities, and to be able to treat neonates and adults. Other suggestions included integrating with a ‘referral system to enable the follow-up of patients’ via synchronisation between health providers. With limited human resources to enter data, and considering that HSAs are ‘first responders in the community’, all respondents highlighted the importance of integrating the ‘data from the app to the District Health Information System 2 (DHIS2)’ to enable public health surveillance in real time by the MoH.

Programme leadership

Participants felt that the SL programme has been supported by numerous organisations within the MoH. One stakeholder mentioned that supporting the programme has allowed capacity-building within their institution. They also mentioned that certain departments within the MoH need to lead the wider implementation for this programme to continue to be successful, but also stated that this may not be a ‘realistic option given the number of projects managed by the government’ and issues related to infrastructure.

Factors in the broader community environment

Socioeconomic and political considerations

One participant highlighted that mHealth technology is the only feasible option to reach and collect data from communities; five others noted that the present political environment is in favour of such projects. Despite limitations that are beyond our control, such as the availability of power, the lack of resources (particularly finances) is the greatest limiting factor that can influence the sustainability of this project. One participant highlighted the notable deficiency in human resources including HSAs and staff for data entry, leading to overburdening of responsibilities.

Community participation

Of the interviewed participants, only four were aware of community engagement with the trial. For example, the HSAs noticed an increase in attendance from ‘the average was something like 15’ to ‘it's like something like 25 or even 30’ at the village clinic following the utilisation of the smartphone app. The community members felt that the HSAs, by using the app, were more knowledgeable and, as a result, received better care. Other communities felt ‘empowered that their village is intending to be developed’ by participating in such programmes.

Discussion

With the Malawian MoH wanting to optimise mHealth technology for the delivery of healthcare services, this paper examined the sustainability and wider implementation of the SL programme. To our knowledge, this is the first study to report the perspective of key stakeholders on the barriers and facilitators to sustainability of mHealth intervention focusing on CCM in Malawi.

Almost all the participants felt that the SL consortium were successful in achieving two of the three components of the programme. However, at the time of the interviews, they had not yet been able to observe the impact of the surveillance framework. Participants expressed an overall satisfaction with the design of the programme, the HSAs found the app easy to use and other participants noted that the work of the HSAs had become more streamlined.

Two stakeholders noted an increase in attendance at the village clinics following the incorporation of the app by HSAs. This important finding supports the literature in that healthcare is often sought at higher-level facilities outside village clinics, which involves travelling for at least 8 km to access care9,25. Reflecting the geographical distance and the level of care outside village clinics, the stakeholders suggested that the increase in attendance could be due to caregivers perceiving that HSAs are more knowledgeable with the app and that the service provided is similar to that in hospitals.

With many health service providers familiar with mHealth, and the MoH in favour of using technology to reach communities, there is optimism that such mHealth initiatives can endure in Malawi26. Despite the clear motivation towards mHealth technology, this study revealed that the lack of internal funds, limited resources, and concerns over the broader coordination of this project may impact the sustainability of the SL programme beyond its period of funding. When analysing policy adoption in Malawi, Rodríguez et al. noted the substantial donor support required to fund the CCM programme2, which supports the emerging theme from our interviews that the MoH is overwhelmed by the management of fundamental issues, such as drug stock-outs to allocate funds for such programmes.

Despite the success of training methods within the SL programme, a vital theme emerging from our stakeholders was the missed opportunity to adopt a training of trainers (ToT) approach. Studies have noted the importance of developing an infrastructure by delegating responsibilities for facilitating the sustainability of intervention. Therefore, establishing a ToT within the MoH would have enforced standard guidelines, the continuous training of staff to ensure consistent delivery, along with the correct use and maintenance of the SL eCCM App27.

With the existence of paper-based registries and records at the community level (facilitators of intervention sustainability), the stakeholders made various suggestions to integrate the app and strengthen the utilisation of this programme within the health system28. The MoH has adopted the DHIS2 to create an electronic centralised monitoring and reporting system to promote the real-time use of the data collected. As this system provides an infrastructure for the maintenance of intervention, the sustainability of the SL programme is dependent on the integration of the app with DHIS2.

Stakeholders further noted that the wider implementation of this programme should be led by certain divisions of the MoH to ensure commitment. However, with the government being overwhelmed with other projects, the lack of organisational leadership suggests that these factors will influence the wider implementation and long-term sustainability of the SL eCCM App.

Stakeholders suggested that social and ecological challenges in Malawi can be overcome by reaching communities through mHealth technology. Recognising the benefits of mHealth, our interviews revealed a collaboration of technocrats from the healthcare and technology sectors in the development of an eHealth framework strategy and standard operating procedures to ensure that future projects in this domain are aligned with the interests of Malawi29. This expression of interest in utilising health technology in the health system to improve access to data for evidence-based policy decisions is an important step towards sustaining the SL programme. Furthermore, with many African health systems experiencing a shortage of qualified staff, mHealth has been shown to support, motivate and build capacity for health workers30.

As highlighted by Iwelunmor and colleagues, a key facilitator for sustaining the intervention is community participation and engagement. The engagement of key leaders in communities prior to initiating the programme led to the increased participation of community members. Furthermore, our participants noted that communities felt empowered and that the government is engaged in improving the healthcare provided in their villages16. A previous study investigating the impact of malaria bed nets in Benin found that community members felt pride by participating and felt empowered to make changes, thus resulting in increased levels of motivation to continue the programme31. Therefore, engaging community members in the development of the SL programme and creating a sense of ownership could be important factors for the future success of the programme.

Limitations

There are some limitations to our study which need to be considered. First, the sample size was low, although we believe that this was sufficient to ascertain fine-grained themes32; furthermore, key members in healthcare provision and MoH were consulted to gain a ‘real-world’ understanding of factors influencing sustainability and the implementation of the SL programme. Another limitation of this study was the loss of stakeholders who participated in the initial development of the programme. Furthermore, there is the possibility of potential bias because two of the interviewees were members of the SL consortium. Another limitation was that we did not include the perceptions of the communities; this was because we felt that community members may not be fully aware of the full impact of the project.

Policy considerations

The existing literature relating to public health and the information and communications technology (ICT) sector in Malawi highlights the following challenges: poor infrastructure, the lack of appropriate policy and regulation, a lack of human skills and development, a lack of financial resource and poor ICT governance33. In this paper, we present a detailed view of the potential facilitators and barriers to the sustainability of one such mHealth intervention, the SL programme in Malawi, for future policy development. Our data indicate that an infrastructure exists with regards to guidelines and policies from the MoH with which to deploy mHealth interventions and projects. We propose that future mHealth interventions in Malawi should consider the association between socio-ecological and organisational factors, and utilize and build upon existing capacity to ensure that the intervention continues to be sustained without external resources. To that end, our findings will be presented to the Malawian Ministry of Health via its mHealth Working group. Thus, going forward, as the mHealth working group seeks to deploy new mHealth platforms, it may utilise the findings from this study to ensure that its national policies are more inclusive and therefore help guarantee sustainable and long-term mHealth interventions.

Conclusions

Our findings demonstrate that there are many critical factors in Malawi that could affect the logistical and financial efforts made by the MoH to achieve sustainability of the SL eCCM App, including socio-economic influences and integration with a centralised server. Developing countries, such as Malawi, have begun to recognise the potential of mHealth and started to integrate and create strategies to use such technologies to improve the delivery, monitoring and reporting of healthcare services. To support this effort, our present study examined both barriers and facilitators for the sustainability and wider implementation of mHealth projects in SSA, using the SL programme as an example. Although this study is specific to Malawi, many SSA countries experience similar challenges within their health systems. Consequently, the findings and concepts discussed in this report may be relevant and applicable to projects in other countries currently engaged in sustaining mHealth interventions.