Using a mHealth system to recall and refer existing clients and refer community members with health concerns to primary healthcare facilities in South Africa: a feasibility study.

Background: Lay health workers (LHWs) are critical in linking communities and primary healthcare (PHC) facilities. Effective communication between facilities and LHWs is key to this role. We implemented a mobile health (mHealth) system to improve communication and continuity of care for chronically ill clients. The system focused on requests from facility staff to LHWs to follow up clients and LHW referrals of people who needed care at a facility. We implemented the system in two rural and semi-rural sub-districts in South Africa.Objective: To assess the feasibility of the mHealth system in improving continuity of care for clients in PHC in South Africa.Method: We implemented the intervention in 15 PHC facilities. The clerks issued recalls to LHWs using a tablet computer. LHWs used smartphones to receive these requests, communicate with clerks and refer people to a facility. We undertook a mixed-methods evaluation to assess the feasibility of the mHealth system. We analysed recall and referral data using descriptive statistics. We used thematic content analysis to analyse qualitative data from semi-structured interviews with facility staff and a researcher fieldwork journal.
Results: Across the sub-districts, 2,204 clients were recalled and 628 (28%) of these recalls were successful. LHWs made 1,085 referrals of which 485 (45%) were successful. The main client group referred and recalled were children under 5 years. Qualitative data showed the impacts of facility conditions and interpersonal relationships on the mHealth system.
Conclusion: Using mHealth for recalls and referrals is probably feasible and can improve communication between LHWs and facility staff. However, the low success rates highlight the need to assess facility capacity beforehand and to integrate mHealth with existing health information systems. mHealth may improve communication between LHWs and facility staff, but its success depends on the health system capacity to incorporate these interventions.

Background

South Africa is a middle-income country with a high degree of economic inequality [1]. The disease burden of the country is characterised by high rates of communicable, non-communicable, maternal and perinatal and injury-related deaths, also unequally distributed within the population [2]. The leading causes of death are communicable diseases (33.6% of all deaths are caused by HIV/AIDS and TB), and non-communicable diseases, such as cerebrovascular and ischaemic heart diseases, diabetes, and hypertension, account for 19.3% of deaths [2]. This epidemiological situation requires a comprehensive response at all levels of the health system, including strengthening district-based primary healthcare (PHC) and developing innovative interventions [3]. Despite many innovations to improve the public health services in South Africa, such as the National Health Insurance scheme [4], and the PHC re-engineering programme [5], substantial challenges in service provision remain. An important challenge in South Africa is the shortage of skilled health professionals [6], which often leaves existing skilled staff overburdened and demotivated [7].

To address this challenge, a task-shifting approach is widely used in South Africa. One component of this is the use of lay health workers (LHWs) to support nurses and other healthcare professionals in implementing primary healthcare (PHC) at the community level [8]. Currently, LHWs play a critical role in extending the PHC system into communities [5]. South Africa has approximately 72,000 LHWs organised in ward-based outreach teams [9]. They perform a variety of tasks, such as medication administration, child health surveys, referring people with illness symptoms to facilities, and recalling clients to for various reasons, for instance, to receive tests results or medication [10]. LHWs are also key to ensuring that clients are not lost to follow-up [10,11]. Despite these important functions being performed by LHWs, their turnover is high [12], resulting in a need for constant retraining of staff [13]. This turnover can partly be influenced by the characteristics of LHWs’ work: long distances between communities and facilities to receive instructions [14-16], or to report on their work [17], that they usually have to travel on foot. This is added to by other issues, including inadequate reimbursement and poor communication between LHWs and facility staff [15].

Mobile health (mHealth) technologies, defined as medical and public healthcare practices supported by mobile devices such as mobile – and smartphones, client-monitoring devices, and tablets [18], are increasingly used in low- and middle-income countries (LMICs) to help solve health system challenges [17,19,20]. The technology has the potential to address some of the issues that LHWs experience, including traveling time, administrative tasks, and communication with facility staff [17,19-21]. There is also potential for these technologies to improve clients’ access to care and potentially reduce LHWs’ workload [17,22]. mHealth interventions can also contribute to improving the continuity of care for chronic conditions [23-26]. Given this potential, we developed and implemented a mHealth system within a rural and semi – rural PHC programme in South Africa. The programme aimed to improve continuity of care for clients, while improving communication between community-based LHWs and health facilities. This paper reports on a formative evaluation of this programme, focusing on the feasibility of implementing a mHealth system to improve the continuity of care for PHC service users. The findings of the evaluation contribute to understanding the feasibility of these programmes and factors that may affect their sustainability.

Methods

Aim

To assess the feasibility of a mHealth system to improve the continuity of care for clients in a PHC programme in South Africa.

Study design

This was a mixed-methods evaluation study, implemented over 10 months, from June 2015 to March 2016. We used both quantitative and qualitative methods of data collection and analyses to assess the feasibility of the multi-component mHealth system [27,28]. This required a research team experienced in both approaches [29]. We used an embedded design in which the primary focus was on the quantitative dataset. We then used the qualitative data to understand and explain the quantitative findings [30].

Setting

We planned to implement the mHealth system in a rural setting, as we thought that these settings would benefit most from the improved communication offered through a mobile health system. Following consultations, the Western Cape Department of Health suggested that the study be conducted in two rural sub-districts in the Eden Health district (Flowchart 1), one of the seven health districts in the Western Cape Province of South Africa. These two study sub-districts were selected because each already had a well-functioning team of LHWs.

The Eden district has a total estimated population size of 613,124 [31]. The two study sub-districts represented approximately 21% of the total district population and were substantially different in size (the population of Sub-district 1 estimated at 26,064 and Sub-district 2 at 101,298) [32,33]. Most residents’ first language is Afrikaans (73%), followed by isiXhosa (25%) and English (2%) [34]. In 2016, 40.5% of the population in the district lived below the poverty line (USD 320 per month) [31]. The Eden district performs slightly worse on key health indicators, compared to the Provincial average. For example in 2015, 16% of newborns were underweight in the district, compared to the 14.5% average in the Western Cape, and the maternal mortality rate was higher in Eden (69.9/100 000 live births) when compared to the Provincial rate (58.3/100 000) [31].

PHC facilities in the Eden sub-districts provide basic healthcare services, including treatment for TB, HIV/AIDS and non-communicable diseases, and maternal and child health services. The hospital in Sub-district 1 that participated in the study provides maternity services, basic surgery, and emergency services. Clients who needed to consult a doctor had to book an appointment in advance as doctors visited the respective facilities only on certain days of the month.

Participants

As noted earlier, each of the selected two study sub-districts had a well-functioning team of LHWs (Flowchart 1), and most of these LHWs lived in the communities in they worked. In each of these districts, the Provincial Department of Health had contracted a non-governmental organisation (NGO) to recruit, manage and pay LHWs to provide community-based PHC services on behalf of the Department of Health. This form of contracting out is common in the Western Cape Province [35]. The LHWs received non-professional training on various topics related to the services they provide, which include supporting clients whom the health facility assigns to them as well as health promotion activities. The LHWs worked 4.5 h a week and earned between 99 and 122 US$ per month, depending on their level of training. At the time of the study, the NGO in Sub-district 1 had two LHW teams (29 LHWs in total), and the NGO in Sub-district 2 had four teams (64 LHWs in total). Each team was supported by a supervisor, who was a retired nurse. Demographic data for the LHWs and supervisors within each sub-district is detailed in Table 1.

Table 1.

Supervisor and LHW demographics

	Sub-district 1		Sub-district 2
	Supervisors	LHWs	Supervisors	LHWs
Female	2	29	4	62
Male	-	-	-	2
Average age	58	35	58	34
Age range	55 – 63	22 – 59	47 – 68	23 – 60
Average number of years working as supervisor or LHW	8	6	3	5
Years in post – range	4 months – 7 years	1 – 10 years	6 – 10 years	3 – 10 years

No demographic data were collected about the clerks.

Settings in which the study was implemented

The mHealth system

The three key role players involved in the implementation of the mHealth system were the LHWs, their supervisors, and mHealth clerks, henceforth referred to as clerks. Each facility manager appointed one staff member, in most cases from the administrative staff, to act as clerk. Managing the mHealth system was in addition to the clerks’ other duties. LHWs were given smartphones for the project, and clerks and nurses tablets, to manage the system.

The main feature of the system was to enable LHWs to record their routine client visits, that is monitoring how they were doing on treatment, do pill counts, and conducting general health assessments in clients’ households, on project-funded smartphones. The system enabled real-time access for supervisors to these reports. The system also enabled two-way communication between LHWs and clerks. LHWs, supervisors and clerks received 2 days of training on how to use the system, offered by a for-profit mHealth service provider (Mobenzi), who developed the system. Thereafter, the implementation staff had 2 weeks to practice using the system before the system went live. Though there were a paper-based recall and referral system in use before the intervention, it was not standardised. The mHealth system could be considered a completely new system to the participating facilities.

The mHealth recall and referral process were as follows (Figure 1): Firstly, the healthcare professionals at the facility instructed the clerk to ask an LHW to locate and advise a client to return to the facility (hereto referred to as recalls, Figure 2). The clerks issued these requests through the tablet, and these were received by the LHW on their smartphones while working in the community. Real-time communication ensued between clerks and LHWs when they discussed recall progress using the system. The supervisors’ role in the recall process was added in month 5 of the implementation, after this was requested by them. The late addition was due to pre-project consultations suggesting that this functionality was not necessary.

Figure 1.

mHealth recall and referral system

Figure 2.

Example of the recall format of correspondence between the LHW and clerkfacility

Secondly, when LHWs identified a person, who could have been an existing client or someone else in the community with a health problem, for example, headaches or wounds requiring care, they would advise that person to seek care at the facility (hereto referred to as referrals, Figure 3). LHWs sent a notification of these instances to the clerk’s tablet, using their smartphones. The clerks closed recalls and referrals, respectively, as successful, when the person arrived at the facility, or unsuccessful when the person failed to attend at the facility.

Figure 3.

Example of the format for a referral sent by an LHW to the health facility

The digital health interventions included in the mHealth system evaluated in this study targeted healthcare providers and can be classified as follows, using the World Health Organization’s classification of digital health interventions [36]:

Interventions focused on client health records: Longitudinal tracking of clients’ health status and services (classification number 2.2.1)

Interventions focused on healthcare provider decision support: Provide checklist according to protocol (classification number 2.3.2)

Interventions focused on healthcare provider communication: Communication from healthcare provider(s) to supervisor (classification number 2.5.1)

Interventions focused on referral coordination: Manage referrals between points of service within health sector (classification number 2.6.2)

Interventions focused on health worker activity planning and scheduling: Schedule healthcare provider’s activities (classification number 2.7.2)

Data collection

The main question we wanted to answer using quantitative data was whether the mHealth system allowed facility staff and LHWs to, respectively, recall existing clients and refer community members with health concerns to healthcare at the health facilities. All recall and referral data via the mHealth system in the two study sub-districts were stored on a Mobenzi server and exported to Excel by the research team. The data included a date and time stamp, sender and recipient, geographic location, and content of the messages between LHW and clerk. As reasons for recalls and referrals were recorded without predetermined categories, the research team manually coded the recall and referral reasons, categorising them according to most frequent reasons – the codes developed are shown in Table A1. As we collected service indicators that were not in use in the study sites prior to this study, baseline data were not available.

Table A1.

Recall and referral reason categorisation

	Description
Children < 5	Checking that immunisation, deworming, and Vitamin A injections were up-to-date, and monitoring their growth and nutrition status
Diagnostic tests (excluding TB/HIV/AIDS)	Clients to collect test results or have a blood test done for non-communicable diseases
Facility appointment reminders	LHWs reminded clients about appointments or informed them about rescheduled appointments
Finding defaulting clients	Clients who have missed facility appointments, or whom the LHWs were concerned that they are not adhering to their treatment
Male medical circumcision	Encouraging males to have medical circumcision done; applied to referrals only
Medication collection	Reminding clients to collect their medication at the facility
Non-communicable disease care	Facility care for clients on hypertension and/or diabetes treatment
Obstetrics/Gynaecology	Females were referred or recalled for maternal care, such as papsmear tests, routine antenatal visits, and family planning advice
Physical symptoms	People who complained about aches, swollen limbs, skin rash, and similar symptoms; applied to referrals only
TB/HIV/AIDS care	Routine check-up visits
TB/HIV/AIDS testing	Encouraging community members to be tested for TB and HIV/AIDS; applied to referrals only
Other	A variety of issues, including wound care, eye care, having to see the occupational therapist or social worker, and mental healthcare. It also included unspecified reasons, when the recall/referral simply stated that the client needed to seek care at the facility

The qualitative component of the study aimed to provide an understanding of the implementation processes and how the participants perceived the mHealth system. These data were intended to help us contextualise the quantitative findings through incorporating participant perspectives. We used two methods of qualitative data collection: semi-structured individual and group interviews, conducted at the end of the project with all of the participating LHWs, supervisors, clerks, and facility managers; and a fieldwork journal kept during the implementation of the study. WO collected the data. The interview questions included how LHWs, supervisors, and clerks felt about using the mHealth system; whether this system changed their practices; their views regarding barriers and facilitators to implementation; and how the mHealth system compared to the paper-based system. We invited the facility managers to join the interviews with the clerks, as it was important for us to ascertain their perceptions, experiences, and recommendations, too. We include the findings from seven clerk/manager interviews, as these cadres were key to the implementation of the mHealth system within the facility, and had a good overall view of the implementation processes. In total, 12 of the 15 clerks, and three of the eight facility managers participated in the interviews. In some instances, facility managers were responsible for two facilities, and the four participating mobile facilities were managed by some of the ‘fixed facility’ managers. The remaining three facilities and clerks were not available at the time that WO conducted these interviews. The staff of some facilities were interviewed together as this was the most convenient approach for gathering data from staff in remote, neighbouring facilities.

The fieldwork journal detailed the researcher’s reflections and observations during the fieldwork visits. For instance, visiting the LHWs in the most remote areas highlighted the challenges of regular contact with, and reporting to, supervisors based in the main towns in the respective sub-districts.

Analysis

Recalls were categorised as successful if the clerk recorded that the client attended the facility as requested. Failed recalls included the following categories: (i) clients who failed to attend the facility as requested; (ii) unclosed recalls, i.e. where either the clerk or LHW did not respond to the other’s latest correspondence; (iii) LHWs who did not view the recall; and (iv) clerk errors, for example, when the clerk sent the request to the wrong LHW. There were no time limits on keeping recalls open. Referrals were categorised as successful if a person attended the facility for the reason he/she was referred by the LHW. Referrals expired within 14 days of being issued and were then categorised as failed referrals. We collected data from June 2015 until February 2016, the second last month of the study, to ensure that recalls and referrals issued in February could be acted upon by the end of the study in March 2016.

We used Excel and R statistical software (https://www.r-project.org/) for the descriptive statistics, calculating facility- and sub-district level averages of recall and referral numbers and success rates, and to present participant demographics. In order to understand and contextualise our results, we applied a qualitative content analysis approach to the interview data from clerks and facility managers [37]. We used Atlas.ti version 8.1 (https://atlasti.com/product/v8-windows/) to conduct this content analysis. As we were primarily interested in explaining the recall and referral outcomes, we analysed the interview data deductively at the manifest level [37]. We used both the apriori themes from the interview guides, including, e.g. barriers and facilitators to implementation, while allowing additional themes to emerge from the data to explain our quantitative results. SA read and reread the transcripts to familiarise herself with the data and generated condensed meaning units from the data. These meaning units were then further reduced to codes, from which categories were generated that related directly to the quantitative data, see Table 2 for an example of the analysis. The analysis was checked by WO, and differences resolved by discussion. WO referred to the fieldwork journal as the interviews were being analysed, looking for content that could illuminate and further explain the views that participants shared during the interviews. The qualitative and quantitative data are reported in parallel in the results below.

Table 2.

Example of qualitative analysis

Categories	Sub-categories	Example codes	Examples of extracts from the data
Lack of support and time	Need technical support Maintaining dual systems Additional workload After hour recalls	LHWs had difficulties in maintaining two systems	‘Some of them struggled to use paper and the phone. So I either carry on with the paper or I carry on with the phone … ’
Communication and interpersonal challenges	Users struggled adapting to mHealth LHW-facility staff relationships	LHW: difficulties in the beginning because not familiar with smart phone	‘like I’ve said, in the beginning when I first went for this training I was nervous. Because I have never used something like this in my life. The only time I might have used a phone, then it was a phone with buttons. But it’s not these modern phones.’
Closing recalls and referrals	Facility staff communication delay System description Immediacy	NC: mH offered immediacy to LHW-facility communication	‘And when I walk in there the sister would quickly tell me, listen, I’ve passed on this or that to the tablet, or I’ve given this and that to that person. And almost miraculously, when I get to those carers and I’ve already got it from the sister early in the morning, then I get to the CCWs: you know what, I got this and that referral.’
Effects of mHealth	Remote settings Improved communication and reporting Improved recalls Paper vs mHealth	NC: very difficult to get data from remote living LHWs	“ … I got those every second week. I had to arrange it with them because we have to stick to our petrol budget; in other words, I can’t drive to [Place A]and [Place B] every week. I was able to go about once a month, and then I had it delivered every second week at the [Place C], that’s the satellite, so I had an arrangement with that man that I would collect it on a Friday.”

Categories, sub-categories and example codes and quotes (sub-category in bold where codes and quotes are related).

Ethics, consent and permissions

Prior to conducting this study, ethical approval was obtained from the South African Medical Research Council (EC016-11/2014). The approval included the participant information sheet and informed consent form signed by all participants. All interviewees provided signed informed consent, and the interviews took place in a private space in their respective facilities, at a time that was convenient for them. The interviews were conducted in the language preferred by participants, which was predominantly Afrikaans. Interviews were audio-recorded, transcribed and translated into English. Before the study commenced, we provided the LHWs with an information flyer in plain language and asked them to use this when describing the study to their clients. LHWs were instructed during the training to only use the mHealth system after clients were briefed and had given consent to participate.

Results

The quantitative results for recalls of PHC clients and referrals of clients and community members are presented below, according to success rates. The categories emerging from the qualitative data – lack of support and time, communication and interpersonal challenges, closing recalls and referrals, and the effects of mHealth – are reported together with the quantitative data. We first present the recall results and then discuss the referral results.

Recalls

In total, 2,204 client recalls were issued across the two sub-districts, of which 28% (628/2,204) resulted in clients attending at the facility as requested (Table 3). Most recalls were initiated in Sub-district 2 (1642, 74% of total recalls) as could be expected due to the larger sub-district population. However, the recall success rate in Sub-district 1 was 53% (301/562), compared to 20% (372/1,642) in Sub-district 2. The two most common recall categories across the two sub-districts were recalling children under five (23%, n = 514), for example, for growth or nutrition monitoring or vaccinations, and facility appointment reminders (10%, n = 230), that is reminding clients of upcoming appointments, and in many cases, of rescheduled appointments (see Table A1 for further detail).

Table 3.

Recall success rates according to the recall reason

Reason for recall		Sub-district 1N (%)	Sub-district 2N (%)	Total across sub-districtsN (%)a
Children < 5 years (e.g. deworming, Vitamin A, immunisation)	Successful recalls	101 (64%)	75 (21%)	176 (34%)
	Total recalls	158	356	514 (23%)
Diagnostic tests (being tested or receiving results for all conditions excluding TB/HIV and AIDS)	Successful recalls	22 (44%)	11 (9%)	33 (20%)
	Total recalls	50	117	167 (8%)
Finding defaulting clients (including TB/HIV clients)	Successful recalls	4 (44%)	11 (15%)	15 (19%)
	Total recalls	9	72	81 (4%)
Medication collection	Successful recalls	16 (55%)	51 (15%)	67 (17%)
	Total recalls	29	348	377 (17%)
Non-communicable disease care	Successful recalls	5 (83%)	7 (39%)	12 (50%)
	Total recalls	6	18	24 (1%)
Obstetrics/Gynaecology (including family planning)	Successful recalls	4 (27%)	3 (5%)	7 (9%)
	Total recalls	15	62	77 (3%)
Reminding clients about facility appointments	Successful recalls	86 (57%)	9 (11%)	95 (41%)
	Total recalls	150	80	230 (10%)
TB/HIV/AIDS care	Successful recalls	8 (44%)	10 (16%)	18 (21%)
	Total recalls	18	64	82 (4%)
Otherb	Successful recalls	55 (43%)	150 (29%)	205 (31%)
	Total recalls	127	525	652 (30%)
Total	Successful recalls	301 (53%)	327 (20%)	628 (28%)
	Total recalls	562 (26%)	1,642 (74%)	2,204 (100%)

aThe % reported for the total of each recall reason is the proportion of the total recalls across sub-districts.

bIncluded a range of health issues, such as wound care, eye care, having to see the occupational therapist or social worker, and mental healthcare. It also included unspecified reasons, when the recall/referral simply stated that the client needed to seek care at the facility.

Table 3 shows the recall success rates according to the recall reasons. There was a high number of medication collection recalls in Sub-district 2 (21% of the total recalls) because of fewer community-based medication dispensing outlets available for the population than what was available in Sub-district 1. Many Sub-district 2 clients, therefore, had to collect their medication at the nearest health facility and thus became part of the mHealth recall system. The high number of facility appointment reminder recalls in Sub-district 1 (27% of total recalls in that sub-district) were due to frequent doctor appointment rescheduling (personal communication, facility manager, 8 April 2016). Qualitative data suggested that clerks thought that the mHealth system supported this rescheduling well, as it speeded up messages getting to clients via LHWs, adding to the system’s feasibility.

To facilitate ownership of the mHealth system, we decided in advance to allow clerks and LHW teams to adapt the system in any way that made it easier for them to use. The interview data suggested that this happened in relation to the process of issuing recalls, which differed across facilities. In some facilities, recall requests were made through a meeting among the healthcare professionals, while in others, clerks received a stack of folders or were told verbally or through stickers or book notes to recall clients. Facility staff also noted that when the period for facility audits, i.e. reporting on how well the facility performed against service targets, was approaching, certain client categories would be prioritised for recalls, to meet these targets. One clerk noted the following when asked about the improvement in recall numbers:

Audits! … When an audit is coming, then the patients are called in because we are worried that we won’t get it [the targets] right. (Sub-district 1, Facility 1 clerk)

The qualitative data from over half of the participating facilities indicated that interpersonal relationships and communication patterns in the facility and with LHWs could impact substantively on closing recalls, and thus impacted on the feasibility of the mHealth system. As reported below, interpersonal conflicts meant that information about clients arriving was not communicated, hampering system implementation:

The communication between us [clerks and healthcare professionals] was just not right to say that the patient did come back….(Sub-district 2, Facility 1 clerk)

Referrals

A total of 1,085 referrals were recorded across the two sub-districts, of which 45% (485/1,085) were successful (Table 4). Sub-district 1 had 84 referrals (8% of the total referrals), compared to the 1,001 referrals (92% of the total referrals) in Sub-district 2. The success rate in Sub-district 1 was 33% (28/84), compared to the 46% (457/1,001) in Sub-district 2. Table 4 shows the referral success rates according to the referral reasons.

Table 4.

Referral success rates according to the reason for referral

		Sub-district 1N (%)	Sub-district 2N (%)	Total across sub-districtsN (%)
Reason for referral				Total (%)a
Children < 5 years (e.g. deworming, Vitamin A, immunisation)	Successful referrals	8 (22%)	172 (57%)	180 (53%)
	Total referrals	36	303	339 (31%)
Diagnostic tests (being tested or receiving results for all conditions excluding TB/HIV and AIDS)	Successful referrals	2 (100%)	8 (29%)	10 (33%)
	Total referrals	2	28	30 (3%)
Finding defaulting clients (including TB/HIV clients)	Successful referrals	1 (50%)	19 (34%)	20 (34%)
	Total referrals	2	56	58 (5%)
Male medical circumcision	Successful referrals	0	5 (29%)	5 (29%)
	Total referrals	0	17	17 (2%)
Medication collection	Successful referrals	6 (32%)	30 (38%)	36 (37%)
	Total referrals	19	79	98 (9%)
Non-communicable disease care	Successful referrals	2 (50%)	5 (28%)	7 (32%)
	Total referrals	4	18	22 (2%)
Obstetrics/Gynaecology (including family planning)	Successful referrals	0 (0%)	50 (56%)	50 (56%)
	Total referrals	3	90	93 (9%)
Physical symptoms	Successful referrals	2 (33%)	38 (45%)	40 (44%)
	Total referrals	6	85	91 (8%)
Reminding clients about facility appointments	Successful referrals	2 (100%)	5 (31%)	7 (39%)
	Total referrals	2	16	18 (2%)
TB/HIV/AIDS care	Successful referrals	0	30 (65%)	30 (65%)
	Total referrals	0	46	46 (4%)
TB/HIV/AIDS testing	Successful referrals	2 (67%)	60 (43%)	62 (33%)
	Total referrals	3	140	143 (13%)
Otherb	Successful referrals	3 (43%)	35 (28%)	38 (29%)
	Total referrals	7	123	130 (12%)
Total	Successful referrals	28 (33%)	457 (46%)	485 (45%)
	Total referrals	84 (8%)	1,001 (92%)	1,085 (100%

aThe % reported for the total of each referral reason is the proportion of the total referrals across sub-districts.

bIncluded a range of health issues, such as wound care, eye care, having to see the occupational therapist or social worker, and mental healthcare. It also included unspecified reasons, when the recall/referral simply stated that the client needed to seek care at the facility.

Data from the implementation journal suggested that one facility in Sub-district 2 had a higher referral success rate than the others because the nursing staff had prioritised LHW referrals. Clients arriving at this facility were, therefore, more closely followed by clerks. This was corroborated by the interview data, where some clerks indicated that certain referrals and recalls were made and closed based on priorities and targets set by the facility – and sub-district management:

Every month they [management] check … if there are reports that have to be signed off … So she [the clerk] knew that these had to go out. (Sub-district 1, Facility 2 clerk)

The interviews suggested several reasons why success rates for recalls and referrals were low. Closing the recalls and referrals was referred to by clerks as ‘the worst part of the job’. They described a lack of feedback from the healthcare professionals about who had arrived after a recall or referral, and that receiving feedback from the LHWs about whether a client was found, sometimes lacking, too:

The feedback wasn’t always there, especially when you sent out something to the home-based carers [LHWs], you just don’t get feedback. (Sub-district 2, Facility 3 clerk)

This lack of feedback from LHWs meant that to close the recalls and referrals, clerks had to either bother busy healthcare professionals; have in-person confirmation from the recalled or referred clients when seeing the client arriving at the facility; or they had to obtain client folders from the rooms in which they were stored. This impacted on their perceptions of the system itself. Similar to referrals, if interpersonal relationships between clerks and LHWs or between clerks and the healthcare professionals were strained, it hampered the closing of recalls. This was described by clerks in three of the facilities:

In the beginning she [the nurse responsible for checking folders] told me the [recalled] child had arrived … but then there was … [an incident]. Something between me and her … that she wasn’t able to tell me [when recalled clients arrived]…. (Sub-district 2, facility 1, clerk)

To support closing recalls and referrals, most clerks suggested developing a recording system at the reception that would enable identifying recalled or referred clients without depending on memory or personal communication. Poor internet and mobile phone connections, and staff absences, also contributed to unclosed recalls and referrals: when only one person was responsible for the system, it did not function when that person was absent.

The clerks’ description of the challenges with closing recalls was corroborated by data showing that only 8% (n = 129) of recalls across the sub-districts failed because clients did not attend the facility as requested (Table 5). The remaining 92% of failures were due to the following issues: 66% of the failures were ‘unclosed recalls’, with the final correspondence in the system coming either from the clerk or LHW, without the other party responding; in 17% of failed recalls, LHWs did not view the recall; and 9% of failures were due to clerk errors, referring to clerks assigning a recall to an LHW not serving the area in which the client lived, or creating a recall without assigning an LHW to it.

Table 5.

Reasons for failed recalls

	Sub-district 1N (%)	Sub district 2N (%)	TotalN (%)
Client did not attend the facility	43 (16%)	86 (7%)	129 (8%)
Unclosed recall	161 (62%)	871 (66%)	1033 (66%)
LHWs did not view the recall	51 (20%)	215 (16%)	266 (17%)
Clerk error	6 (2%)	142 (11%)	148 (9%)
Total failed recalls	261	1,314	1,575

Referral and recall implementation and feasibility

Despite the low success rate for recalls, which the clerks themselves also lamented, clerks and some managers were generally optimistic that the mHealth system had had a positive impact and was feasible to implement:

Did you notice at all that more people were coming [to the facility] now than in the past?

Yes, definitely … I don’t know if we have the statistics, but all we can say is that the clinic’s headcount has increased, especially at [Facility 1] … our chronic service there has improved a lot. And I think the new system is definitely responsible for that.” (Sub-district 1, Facility 1 manager)

Yes, like I’ve said, it worked very well. It worked very well because we could send people [LHWs to recall clients] out. (Sub-district 1, Facility 3 clerk)

Only one clerk thought that there was no difference in results between the mHealth and paper systems. She was of the view that only the mechanism had changed, and not the work itself, but even she found the mHealth system more convenient than the paper-based system. Others reported benefits such as increased immediacy in issuing recalls and in getting feedback on these, as well as reaching more clients across a wider geographic area.

It was nice to work like that because if you sent out a message, then you immediately got a reply from the home-based carer [LHW]. (Sub-district 1, Facility 2 clerk)

But with the tablet we could address a wider population, and also on their (LHW) routes they could address more people if they had cell phones because you were able to send messages to them. (Sub-district 2, Facility 2 clerk)

While most staff acknowledged that the mHealth system was not perfect, they were generally of the view that it was more reliable and functional than the paper-based system. They also saw the mHealth system as offering accountability, as records were retained of when recalls or referrals were made. The clerks and managers also recommended a number of improvements to the system itself, including using data from existing electronic patient records to populate clients’ personal details in the mHealth system.

Discussion

The mHealth system evaluated in this study aimed to improve continuity of care in rural and semi-rural communities in South Africa. However, the success rates of the intervention modalities were low, at 28% and 45%, respectively, for recalls and referrals.

The qualitative data from our study suggest that while the mHealth system was appreciated by clerks and health professionals at the facilities, there were a number of challenges in implementing the system. The biggest challenge was for clerks to record the outcomes of recalls and referrals. It is, therefore, possible that the true recall and referral success rates were higher than those measured, given that a large proportion of cases (66% across the two sub-districts) were left unclosed by either the clerk or LHW. One approach to overcoming this challenge in future implementations of similar mHealth systems could be to integrate the mHealth functions into facility-level electronic health information systems, where these already exist. For example, a Zambian study similar to ours, but with automated issuing of recalls, achieved a much higher recall success rate (63%) than the 28% in our study [38]. Several studies confirm the need for integration, claiming that it may strengthen health system functioning [39] and promote the sustainability of the mHealth system [40]. As was found in our study, poor integration may frustrate users and lead to additional labour-intensive tasks for them [41,42]. With this in mind, automated entry, as suggested by the clerks, could help to ensure that staff with many other responsibilities do not have to spend time on manually issuing and closing recalls and referrals. In addition, this could lead to more accurate data on success rates.

The feasibility of our system was further challenged by the fact that clerks already had high workloads prior to being tasked with managing the mHealth recalls and referrals. In some facilities, they had no additional support with these tasks, and allocating such support could be difficult in overburdened and understaffed PHC facilities in resource-constraint settings. As in several other studies [42-44], we found that successfully maintaining an mHealth system was a challenge when added to staff’s existing job descriptions. In addition, many clerks reported struggling with the technical aspects of the mHealth system. Given that some clerks had not used a tablet before, this is not surprising. These findings show how health system absorption capacity, in terms of human resources, is key to the success of any mHealth intervention, and provide an important ‘reality check’ to the widespread optimism regarding the impacts of digital health interventions delivered via mobile.

Our results show that success rates and recalls and referrals varied greatly between sub-districts. Direct comparisons between these two settings are difficult, as infrastructure and population size vary greatly. The differences between sub-districts could partly be attributed to facilities, that set priorities based on needs and service performance audits. The qualitative and quantitative data showed that context influenced the number and content of recalls and referrals greatly. This suggests that the system is adaptable to different needs across different settings. Similarly, the success rates for recalls and referrals were strongly influenced by a single, well-performing facility in each sub-district. In the participating hospital in Sub-district 1, the recalls comprised of notifying LHWs when newborns were sent home from the hospital. When LHWs reported that they had contacted the family and had enrolled the child into their care, the clerk closed the recall as successful. Given that this task is much easier than, for example, ensuring that a client who has defaulted from their medication returns to a facility, the hospital recall success rate was much higher (77%) than for any other facility in that sub-district.

The referral and recall process in our study depended on teamwork and communication within the facility, as healthcare professionals needed to share information with the clerk to ensure that referral and recalls were issued and closed appropriately. Therefore, facility-based variation was influenced by interpersonal relationships within the facility, with these relationships being critical to how well a facility functions [45]. Finding recall or referral outcome information from individual client folders was difficult. Another South African project focusing on referrals for tuberculosis case finding [46] gave clients barcodes that allowed referrals to be identified on arrival at the facility. This suggests that to improve the feasibility of interventions such as ours, attention should be paid to completing the feedback loop from facility to community and back to the facility, through a standardised system that relieves pressure from busy professionals. Such approaches may be more complicated and expensive to implement, but may be a useful approach for tracking recalls and referrals more reliably at facilities.

Although this paper does not report on client perspectives regarding the mHealth system (these findings will be reported separately), client-related factors, including how far from the facility they live, and their values and preferences regarding their health, are also likely to impact on the success of recalls and referrals [47]. This makes it difficult to predict how clients will respond to health workers’ encouragement to seek healthcare [48]. Ensuring that clients attend the facility when requested to do so is, therefore, a complex social process in which encouragement plays only a part [49].

Our study suggests that, broadly speaking and taking into account the caveats discussed above, it may be feasible to implement this mHealth system in similar settings, and that it offers advantages, including immediacy and improvement in communication and accountability between LHWs and facility staff. It is also adaptable according to facility needs. Importantly, this mHealth intervention enabled LHWs to receive instructions and report progress while in the community, which could translate to important gains in process timing. These advantages have also been noted in other mHealth studies [9,23,50]. However, the low recall and referral success rates, and the challenges highlighted in the qualitative data, suggest that the implementation of mHealth interventions in the ‘real world’ primary healthcare services is challenging. The capacity of the local health system to implement and sustain such interventions needs to be assessed in advance and monitored closely during implementation.

The system encountered obstacles such as poor internet access, changes in staffing and clerks already burdened with other duties. Further implementation research is needed to address questions such as how mHealth systems can be implemented with minimal disruption to facility staff and service delivery processes, how best to integrate it with existing health information systems and the cost-effectiveness of these approaches. A forthcoming qualitative evidence synthesis will also provide valuable insights regarding the healthcare workers’ perceptions and experiences reported in this paper [51]. This mHealth intervention was implemented within the real world of PHC services in under-resourced, rural and semi-rural communities, with no additional staff employed to manage the mHealth system. This study confirms that it is possible to implement these systems in real-world settings [38]. Based on our evaluation, we present suggestions for similar mHealth interventions in Box 1.

Box 1.

Suggestions for future mHealth interventions

Integrate mHealth systems with existing health information systems in the implementation context. Support the implementation of mHealth systems with training directed to all facility staff. It may be helpful for this training to cover relevant interpersonal communication skills and communication channels. Explore ways of ensuring buy-in from all implementers. Ensure that mHealth applications do not result in significant increases in workload for staff assigned to manage the mHealth system. Explore ways of providing on-site technical support to teams, for example, through having an online support desk.

Our mixed-methods design provided a more nuanced understanding [52,53] of the implementation successes and challenges of the mHealth system than could have been achieved through one method alone, with the qualitative findings being used to understand and corroborate the quantitative findings [28,29,54]. We enhanced the trustworthiness of our qualitative data through including one of the NGO supervisors and research participants [PH] as co-author. In this way, we were able to confirm that our interpretations of the qualitative data resonated with her and her colleagues’ perceptions and experiences. This was a type of ‘member checking’ [55], which can improve the trustworthiness of qualitative findings.

This study has several limitations: first, the mHealth system was discontinued by the Western Cape Government’s Department of Health after the project funding ended. We, therefore,cannot comment on the sustainability and performance of the system over time. Nevertheless, the project provided insights on sustainability considerations for digital health interventions of this kind, for example, in relation to the need for continuous training of staff and the importance of embedding new interventions into current systems (Box 1). Second, the study was conducted only in two, rather different semi-rural South African communities, and therefore generalisability is limited. Third, due to a lack of integration with existing health information systems as well as resource constraints, we were unable to assess the intervention’s impacts on client’s health, for example, whether better recall led to improvements in blood pressure levels for clients living with hypertension. We were also not able to use the existing health information system to automatically populate personal details in the mHealth system, as the two systems were not linked. Given the absence of data on how successful the paper-based system was in recalling and referring clients and community members, we were unable to make a direct comparison between the paper-based and mHealth systems, beyond participants’ reported experiences. Similarly, we did not have data to allow us to compare the costs of the paper-based system to the monthly mHealth costs per device, which was 3 US$ for technical support from the service provider, 3 US$ for data bundles, and 14 US$ per for platform hosting.

Conclusions

Mobile health interventions to improve the continuity of primary healthcare are probably feasible to implement in rural and semi-rural communities in poorer settings, but there remain a number of challenges to their implementation. Our study found that health facility staff appreciated aspects of the system but also experienced challenges and asked for modifications, including automated input of client details from the health information system, as well as automated recording when recalled and referred clients and community members arrived at the facility. Future studies should investigate the absorption capacity of health systems to adopt new mHealth interventions as well as whether such systems can improve clients’ health outcomes.