Reaching further: Lessons from the implementation of the WHO Basic Emergency Care Course Companion App in Tanzania.

INTRODUCTION: The World Health Organization's (WHO's) Basic Emergency Care (BEC) course was developed to address training gaps in low- and middle-income countries (LMICs). Simultaneously, LMICs have experienced an unprecedented increase in the number of cell phone and internet users. We developed a mobile application adjunct to the BEC course (BEC app) and sought to assess the reach of the BEC app.
METHODS: Forty-six BEC course participants, made up of doctors and nurses from three hospital sites in Tanzania, were given access to the BEC app with download instructions. Moderators tracked mobile access characteristics and barriers. This is a descriptive study outlining the implementation of the BEC app and associated findings from the process.
RESULTS: Fewer than 10% of participants were able to independently download and use the application. The download process revealed three key barrier areas: accessibility (no smartphone, smartphone without charge, no access to data/WiFi to download app, increased cost of data), technical (outdated operating system, inconsistent access to data/WiFi to run the app, insufficient phone storage), and participant-related characteristics (variability in smartphone literary, language discordance, smartphone turnover). Of the 46 participants, 29 (63%) were able to download and use the BEC app successfully with moderator support.
CONCLUSIONS: There is potential utility of mobile health in LMICs. However, barriers still exist to reaching the largest possible audience for these initiatives. The importance of app compatibility with a broad range of operating systems and limitation of the amount of data needed to download and use the app was underscored by our study. Moreover, creative solutions are needed to facilitate large-scale roll-outs of mobile health interventions, such as a distribution model that relies on super user and peer support rather than an individual moderator. Additional local perspectives on the download process and the utilisation and acceptance of the application post-implementation are needed.

African relevance

Health systems in low- and middle-income countries (LMICs), including numerous counties in Africa, face considerable challenges in providing high-quality, affordable and universally accessible care in the setting of limited access to training and educational materials.

There is potential utility of mobile health (mHealth) in this setting given the unprecedented increase in the number of users of cell phone and internet technologies.

While mHealth has offered great hope, significant barriers still exist in Africa despite a large uptake in the technology.

These barriers must be understood and overcome to maximize the reach of mHealth interventions and address the training gap.

Introduction

Health systems in remote or low- and middle-income countries (LMICs) continue to face considerable challenges in providing high-quality, affordable and universally accessible care [1], with limited access to training and educational materials identified as a key barrier. An open-access, Basic Emergency Care (BEC) course for frontline emergency care providers was developed by the World Health Organization (WHO) in collaboration with the International Committee of the Red Cross (ICRC) (and endorsed by the International Federation for Emergency Medicine (IFEM)) to provide standardised training in life-saving techniques. While the course was shown to be effective in knowledge transfer in LMICs [2], adult learning theory suggests that a lecture-based format alone may not promote long-term knowledge retention [3].

Simultaneously, LMICs have experienced an unprecedented increase in the number of users of cell phone and internet technologies, as well as a decline in the price of devices and services [2]. Therefore, there has been much promise on the use of mobile health (mHealth) applications to improve care in these areas [3,4]. This, combined with user feedback from BEC participants requesting a mobile adjunct, led to the development of the BEC software application (BEC app).

However, in LMICs and remote communities, access to consistent, robust internet is still often limited despite access to mobile devices [5,6]. Furthermore, though the penetrance of mobile devices to these communities may be high, a significant proportion of users may use old or out of date devices or have bandwidth limitations at a rate higher than in the US or other high-income countries [7].

Improving the efficacy of health interventions relies not only on their impact but also on their reach; thus, it is important to support as many participants as possible [8]. Our study objective was to advance the understanding of end-user characteristics and implementation dynamics. This is critical to facilitate planning and design of future interventions that will meet the varied needs of diverse participants.

Methods

The BEC course materials were used as a framework for BEC app content, with a particular focus on the airway, breathing, circulation, disability and exposure (ABCDE) pathway commonly used in emergency care systems to intervene on critical medical conditions. The ABCDE pathway was identified by course participants as particularly difficult to manage due to numerous branch points and interventions and thus ideal for an algorithmic approach through an app. Thus user-centred design and consensus opinion were used to wireframe and prototype the app which was then built to operate natively on Android and iOS operating systems. The BEC app was developed in English (one of six official WHO languages; the others being Arabic, Chinese, French, Russian and Spanish) to align with the BEC course and WHO educational materials. During pilot study of the efficacy of the BEC app, three hospital sites in Tanzania (Mbeya Zonal Referral Hospital in Mbeya Region; Sinza Health Centre in Ubungo District, Dar es Salaam; and Vijibweni Health Centre in Kigamboni District, Dar es Salaam) were randomised to receive the BEC app along with the structured classroom-based BEC course versus the coursework alone (data from the larger study are currently being analysed for publication). There were 46 participants in the app intervention arm, selected from a sample of convenience and made up of doctors and nurses from each of the three BEC app sites (Table 1). Participants were primarily Swahili speaking with variable levels of English proficiency.

Table 1

Demographic characteristics by site.

	Overall	Mbeya	Sinza	Vijibweni
	(N = 46)	(N = 16)	(N = 13)	(N = 17)
Site characteristics
Facility type	–	Zonal Referral Hospital	Health Centre	Health Centre
Location	–	Mbeya, Tanzania	Dar es Salaam, Tanzania	Dar es Salaam, Tanzania
Full service emergency dept?	–	Yes	No	No
Participant characteristics
Age, mean (range)	32.8 (23–55)	30.4 (23–55)	36.6 (24–50)	32.2 (23–48)
Gender, no (%)
Female	27 (59)	8 (50)	7 (54)	12 (71)
Male	19 (41)	8 (50)	6 (46)	5 (29)
Cadre, no (%)
Doctor	15 (33)	5 (31)	7 (54)	3 (18)
Nurse	31 (67)	11 (69)	6 (46)	14 (82)
Work experience, No (%)
<5 years	28 (61)	15 (94)	4 (31)	9 (50)
>5 years	18 (39)	1 (6)	9 (69)	9 (50)

The app download process took place in July 2019, first during the initial participant onboarding (1–2 weeks before the BEC course was initiated) and then repeated on the first day of the 5-day BEC course for those who were unable to download during the first session or did not bring their phone with them initially. Participants at the BEC app sites were provided with written instructions in English for downloading the app. These included step-by-step instructions with web-links for either Android or iOS operating systems. The app was not distributed to the Android or iPhone stores until completion of the pilot study. This was an effort to ensure that any broken links, confusing phrasing or other errors could be addressed before wider distribution.

Moderators at the sites worked with the participants to identify and record mobile phone access characteristics of end-users as well as challenges and barriers to the download process. The following specific data elements were gathered by moderators for each participant: smart phone availability (Yes/No), phone type (Android/iPhone), operating system of smart phone, ability to download app independently with instructions (Yes/No), operating system compatible with app (Yes/No), sufficient phone storage to download the app (Yes/No), and ability to open and use app (Yes/No). Data was collected in a Microsoft Excel spreadsheet during the download process (Table 2), while participants were gathered at the BEC course location. Simultaneously notes were taken by moderators to document other types of challenges encountered that resulted in barriers to downloading the app. A crash monitoring application, Crashlytics™, was used to determine the cause of a crash if one occurred after app download and installation. Participants who did not own smartphones were allowed to continue the course and trial the app with partners who did have a working phone.

Table 2

Template of data collection table.

Participant name	Has smartphone? (Y/N)	Phone type (Android/iPhone)	Operating system	Able to download? (Yes/No)	Operating system compatible with app? (Yes/No)	Sufficient phone storage to download app? (Yes/No)	Able to open and use app? (Yes/No)
[Participant 1]
…
…
…
…
…
…
…
[Participant 46]

Data gathered was subsequently analysed in a descriptive fashion, in order to summarise the number of participants who encountered each barrier and who were able to ultimately download the app. This data was supplemented by notes taken during the process to provide additional context to barriers experienced. Rigor in the findings was ensured through structured and meticulous documentation of barriers as they occurred during the download process. Participants were individually interviewed to collect this data and ensure accuracy of collection.

This is a descriptive study outlining the implementation of the BEC app and associated findings from the process.

Results

Of the 46 BEC course participants at app sites, 29 (63%) were able to download and use the BEC app successfully. The app download process revealed three key barrier areas: accessibility, technical, and participant-related characteristics (Table 3). Moderators attempted to address each barrier in real-time and through contact with the app developer who was consulting remotely and addressing issues as able (Fig. 1).

Table 3

Summary of barriers to BEC app installation.

	Barrier identified during implementation	Real-time resolution (if applicable)	Possible pitfalls (if applicable)
Accessibility	No smartphone	Provide smart phone to users without one (not employed)	N/A
	Smartphone with no charge	Obtain charger to charge phone	No availability of charger could mean missed opportunity for download during facilitated download session
	Insufficient data/no access to WiFi to download app	Hotspot from moderators' device	Variable access to data outside of the facilitated download session could limit updates or downloads on newly obtained phones
Technical	Older/outdated operating system	Create backwards compatibility for all possible prior operating systems	Users with operating systems prior to Android Lollipop will not be able to access the app due to lack of backward compatibility
	Inconsistent access to internet through limited data/unreliable WiFi connectivity	App is native app that could run locally without continuous internet; app is <30 MB in total	Contracting with a native app developer can be prohibitively costly.
	Insufficient space on phone to download app	Deletion of other apps to free-up space	Participant who is not familiar with the process of freeing up space may terminate download process at this point
Participant-related	Variability in smartphone literacy	Facilitated download process whereby moderators assisted with installation and issue troubleshooting	In absence of moderator or other assistance, participants may not have independent ability to successfully complete steps to install app
	Language discordance	Swahili-speaking moderator translated instructions verbally	In absence of moderator or materials in their native language, participants may not have understood download instructions
	Smartphone turnover	Participants with new phones at second touch-point went through the process anew	User who obtains a new phone would need to complete installation process again and may encounter the barriers described here

Fig. 1

Flow diagram of the BEC App download process.

Accessibility

To initiate download of the BEC app, participants needed to have a working, charged smartphone with internet connectivity to access the app as well as sufficient storage space to begin the download.

Four (10%) pilot participants could not access the app due to lack of a smartphone whereas all other participants had phones with Android operating systems. Some participants with smartphones reported low battery levels and were given the opportunity to charge their phones if needed.

Of those with a smartphone, many participants reported having limited mobile data plans. Other participants had their phones on “data save” mode - whereby the ability to use a browser and initiate the app install was impaired. Brief discussions with participants revealed that many purchase mobile data in small aliquots as needed in the setting of substantial increases of data bundle costs locally. Participants reported being less likely to use a web-based app or one that required large amounts of data (>50 megabytes (MB)) to be downloaded from their network. While WiFi connectivity was sometimes available at their place of work, they reported that access was unreliable. This was an anticipated issue before implementation and thus the app was designed to occupy <30 MB of memory after initial download and then run without internet connectivity. Moderators brought mobile hotspots, which allowed participants with insufficient mobile data to connect their smartphones to WiFi, enabling all 42 participants with smartphones to have the connectivity necessary to initiate the app download process.

Technical

For successful download of the app, participants needed to have an operating system compatible with the app technical standards. At least 16 participants were unable to download the app during onboarding due to the age of their operating system. Of these participants, all were using the Android Marshmallow (Android 6.0) and its preceding operating systems, which were at least 2 years old at the time of the pilot study and no longer supported by Google. Moderators communicated this information in real-time to the app developer, who altered the compilation of the BEC app libraries to be backward compatible to Android Lollipop and its successors during the time between the initial onboarding and the start of the BEC course when the next set of installs occurred. Of the 42 participants with smartphones, four (10%) were using a pre-Lollipop operating system for which libraries were no longer available and backward compatibility was not possible; these participants were unable to download the app.

Upon initiation of the download, at least 4 (10%) of participants with smartphones received a notification that they had insufficient space on their phones to download the app. Moderators worked with the participants to identify and delete applications that were not regularly used to allow for sufficient space to download the BEC app. Similar to the above-mentioned bandwidth issues, we addressed space issues by limiting unnecessary images and compressing files, making the app <30 MB in total. Two participants (4% of total participants with smartphones) had phones without any built-in storage capacity and were unable to download the app.

A total of 36 (86%) of the 42 participants with smartphones were able to download and install the application. Among these participants, seven (19%) experienced crashes upon launching the application. Ultimately, 29 (63%) were able to download and use the app by the end of the implementation.

The cause of the crashes were identified by reviewing Crashlytics™ after the implementation was completed and application updates were subsequently made to address them for future updates and new downloads. All of the crashes were identified as related to backward compatibility issues with application libraries. Updates were made to account for these issues.

Participant-related

Participants with working, charged smartphones with internet connectivity were given download instructions written in English. Few participants were able to follow the instructions to download and launch the app without support from moderators. Concerns ranged from specific installation questions to general requests for help. Challenges brought forward by participants included the need for assistance with finding a browser, initiating installation, adjusting settings to allow for installation, addressing storage limitations, troubleshooting other issues or notifications that came up, and signing up for a new BEC app account once the app was installed. Moderators worked closely with participants to address questions, assist with the installation, and troubleshoot issues as needed. While language was one possible barrier to independent installation as instructions were in English, the process also highlighted variability in smartphone literacy among participants. The challenges described occurred even when instructions were translated verbally into Swahili by moderators.

Variable rates of phone turnover were also observed. For example, some participants who had no smartphones at the initial onboarding were able to obtain and bring in a smartphone by the beginning of the BEC course, when the second round of installation took place. Yet others who had phones at the onboarding obtained new phones by the second round, requiring them to repeat the installation process.

Discussion

The initial BEC app intervention was a simple, web-link based download with libraries supporting current phone operating systems similar to most mHealth app interventions. With this initial approach, <10% of users with smartphones were able to successfully download and use the application. We imagine that many other LMIC mHealth developers will encounter similar issues with implementation. The importance of app compatibility with a broad range of operating systems and, given rising costs of data, limitation of the amount of data needed to download and use the app was underscored by our study. The discordance in language between participants and the download instructions was another clear barrier to the implementation, highlighting the importance of language accessibility. While it is often a considerable expense to build an app with multiple languages, for those who are multilingual, having instructions in the common local language offers at least an initial place to begin to improve reach.

During the download process of this pilot, we relied on real-time access to the app developer, provision of internet connectivity, and user support in translation of written materials, installation and troubleshooting to address the installation barriers encountered and ensure the majority of attempted users were able to install and use the app. Our reach increased from <10% to 63% once we incorporated these strategies.

Thus, we believe that focused user-centred development, robust backward compatibility and in-person support by individuals fluent in the language of participants may drastically increase the reach of mHealth interventions in LMICs. Additionally, continued crash monitoring with a service like Crashlytics™ helps to provide important insights into necessary technical amendments and continual updates to a given mHealth intervention, which in turn may support the future growth of the user base.

We recognise that such an intervention may be neither scalable nor feasible for mass distribution. As such, creative solutions are needed to facilitate large-scale roll-out of such interventions. These may include continued attempts to reduce the amount of data needed to access the app; simplified wording and instructions in the primary language of participants; continued easing of the download processes and installation workflow; and a distribution model that relies on organised health facility-wide initiatives with super-user and peer support rather than a single moderator assisting individual healthcare workers.

Our findings should be considered in the context of several limitations. First, advancements in mobile technologies and data infrastructures (including 5G or satellite) are continuous and present an ever-changing variable for mobile interventions. Access and costs therein may at any time change. Authors of this paper report that the expense of data has greatly fluctuated since just last year. However, this underscores the importance of developing broad accessibility through multiple modalities which is highlighted in this paper.

Second, our study was limited to workers in three healthcare settings, potentially limiting the generalisability of our results. However, representation of multiple facility types (one zonal referral hospital and two health centres) and different geographies (Mbeya, Ubungo District of Dar es Salaam, and Kigamboni District in Dar es Salaam Dar es Salaam) offsets this limitation. Finally, our study was based on moderator observations of the implementation process without a formal qualitative component. This was addressed by incorporating informal participant feedback obtained by the moderators throughout the implementation process, however, further study with a formal qualitative component would provide opportunity to obtain additional local perspective.

Conclusions

There is great hope and hype for the utility of mobile health in LMICs and the penetration of smartphones in these locations is near that of high-income countries. However, barriers still exist to reaching the largest possible audience for these initiatives, which can limit their impact. Findings from our study provide important considerations for both the development and implementation of such interventions. Additional local perspectives on the download process and usability of the application can better inform mHealth implementation strategies and may help streamline the distribution for optimal reach. Future work should aim to assess the utilisation and acceptance of the application during the several months after the implementation period.

Dissemination of results

Results from this study were shared with staff members at the data collection sites through an informal debrief after conclusion of the download process.

Authors' contribution

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting of the work or revising it critically for important intellectual content: AG contributed 50%; CR 25%; AT 10%; and PN, HS and JM contributed 5% each. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Declaration of competing interest

The authors declare no conflicts of interest.