The effect of an mHealth clinical decision-making support system on neonatal mortality in a low resource setting: A cluster-randomized controlled trial.

BACKGROUND: MHealth interventions promise to bridge gaps in clinical care but documentation of their effectiveness is limited. We evaluated the utilization and effect of an mhealth clinical decision-making support intervention that aimed to improve neonatal mortality in Ghana by providing access to emergency neonatal protocols for frontline health workers.
METHODS: In the Eastern Region of Ghana, sixteen districts were randomized into two study arms (8 intervention and 8 control clusters) in a cluster-randomized controlled trial. Institutional neonatal mortality data were extracted from the District Health Information System-2 during an 18-month intervention period. We performed an intention-to-treat analysis and estimated the effect of the intervention on institutional neonatal mortality (primary outcome measure) using grouped binomial logistic regression with a random intercept per cluster. This trial is registered at ClinicalTrials.gov (NCT02468310 ) and Pan African Clinical Trials Registry (PACTR20151200109073).
FINDINGS: There were 65,831 institutional deliveries and 348 institutional neonatal deaths during the study period. Overall, 47 ∙ 3% of deliveries and 56 ∙ 9% of neonatal deaths occurred in the intervention arm. During the intervention period, neonatal deaths increased from 4 ∙ 5 to 6 ∙ 4 deaths and, from 3 ∙ 9 to 4 ∙ 3 deaths per 1000 deliveries in the intervention arm and control arm respectively. The odds of neonatal death was 2⋅09 (95% CI (1 ∙ 00;4 ∙ 38); p = 0 ∙ 051) times higher in the intervention arm compared to the control arm (adjusted odds ratio). The correlation between the number of protocol requests and the number of deliveries per intervention cluster was 0 ∙ 71 (p = 0 ∙ 05).
INTERPRETATION: The higher risk of institutional neonatal death observed in intervention clusters may be due to problems with birth and death registration, unmeasured and unadjusted confounding, and unintended use of the intervention. The findings underpin the need for careful and rigorous evaluation of mHealth intervention implementation and effects. FUNDING: Netherlands Foundation for Scientific Research - WOTRO, Science for Global Development; Utrecht University.

Research in context

Evidence before this study

In Ghana, as in many low-and-middle income countries, neonatal mortality has not seen the necessary decline to make progress towards achieving the Sustainable Development Goal of ending preventable new-born deaths and reducing neonatal mortality to 12 per 1000 live births by the year 2030. Non-adherence of health workers to guidelines for case management contributes to persistently high neonatal mortality. Training and access to clinical guidelines for health workers is also inadequate. Previous studies have shown that adoption of a clinical decision-making support system that facilitates easy access to maternal and neonatal guidelines for frontline providers in health facilities could improve the quality of maternal and neonatal care in Ghana.

MHealth interventions have been on the ascendency in low-and -middle income countries in recent times as a means to achieve universal health coverage. Many of these mHealth interventions have been small pilots (with a few large studies) targeting improvements in maternal and neonatal healthcare, and management of HIV/AIDS and tuberculosis among others. Few of these studies assessed the impact of the mHealth interventions on health outcomes and where health outcome assessments were done, results of these studies have been mixed. Globally, there is a call to bridge the knowledge gap regarding the paucity of evidence of effectiveness of mHealth interventions on health outcomes in low-resource settings. The implementation of a large cluster-randomized controlled trial in Ghana allowed us to rigorously assess the impact of an mHealth clinical decision-making support intervention on neonatal mortality in Ghana. The intervention facilitated easy access to neonatal healthcare protocols via preformed messages on an unstructured supplementary service data system, while providing access to free phone calls, text messaging and access to the internet for health workers to seek clinical decision-making support from their colleagues. This intervention was designed using a bottom-up approach with frontline health workers in the national capital of Ghana and subsequently tested in a cluster-randomized controlled trial in the neighboring Eastern Region of Ghana.

Added value of this study

A grouped binomial logistic regression with a random intercept per cluster was used to evaluate the effect of the intervention on institutional neonatal mortality. We used Spearman correlation of the number of deliveries and the number of intervention unstructured supplementary service data protocols assessed as a proxy to evaluate utilization of the intervention. We found that the provision of an mHealth clinical decision-making support intervention did not lead to improvement in neonatal mortality in this study. Rather, we observed higher odds of institutional neonatal death in the intervention arm compared to the control arm of this study. This observed effect however, may be partly attributed to the data structure of the national institutional health database of Ghana, which did not allow adequate adjustments for confounding. The free flow of patients in and out of clusters, and inadequate and unintended use of the intervention may have influenced the observed effect of the intervention. We observed a significant correlation between the number of deliveries and the number of protocol requests made which however does not preclude inappropriate use of the intervention.

Implications of all the available evidence: Careful and rigorous evaluations of mHealth interventions are needed if progress towards improvements in neonatal healthcare and universal health coverage is to be made using mHealth interventions in low-and middle-income countries. Such evaluations must incorporate the complexities of contexts in low-and middle-income countries.

Alt-text: Unlabelled Box

Background

Neonatal mortality remains undesirably high in many low- and middle-income countries (LMICs) despite recent improvements in neonatal health outcomes [1], [2]. In 2010, 98 % of the 3.2 million neonatal deaths, occurred in LMICs [3], [4], and the majority of these deaths occurred in Sub-Saharan Africa (SSA). Significant causes of neonatal morbidity and mortality in LMICs include birth asphyxia, infections and prematurity [3]. Although interventions against these and other causes of neonatal mortality exist (e.g., early initiation of breast feeding, hygienic care of the cord and kangaroo-mother care for preterm infants, immediate drying and provision of warmth for newborns, vitamin A supplementation, and intramuscular vitamin K injection) [5], [6], [7], [8], [9], these interventions do not reach those who need them the most [10]. Higher neonatal mortality rates have been projected if interventions are not put in place to stop neonatal deaths [11]. There is therefore an urgent need to focus attention on neonatal interventions in LMICs.

Mobile health (mHealth) interventions hold promise of bridging the gap in improving access to neonatal healthcare services [12], [13], and improved health outcomes in LMICs. There have been many documentations of pilot mHealth studies in LMICs [14], [15], [16]. Although these mHealth interventions are well received by health workers and the community [17], evidence of their effectiveness on patient outcomes, efficiency of health systems or their use by health workers is limited [17], [18], [19]. A shift of mHealth interventions from small pilot studies to larger studies that utilize more robust techniques to assess health outcomes is required to bridge the knowledge gap regarding their effectiveness. One of such large mHealth intervention studies was recently conducted by the Accelerate Project in Ghana [20].

Ghana is a lower-middle-income country with high neonatal mortality rates of 25 deaths per 1000 live births [21]. Higher mortality rates are reported in rural areas of the country [22], [23]. Clinical causes of persistently high neonatal mortality in Ghana include non-adherence of health workers to clinical guidelines [24], [25]. Training and access to these guidelines for providers is inadequate [26]. Clinical decision-making support systems that facilitate easy access to maternal and neonatal guidelines for healthcare providers could improve the quality of maternal and neonatal care in Ghana [27], [28]. To improve access to neonatal health guidelines for health providers, the Accelerate Project designed and implemented an mHealth intervention whose components were based on suggestions for clinical decision-making support gathered in a previous formative study [26]. The intervention aimed to provide quick and easy access to emergency maternal and neonatal health protocols to frontline health workers on the request of the health workers. This mHealth intervention was implemented in a cluster-randomized controlled trial in the Eastern Region of Ghana.

Description of the intervention

The mHealth clinical decision-making support intervention (mCDMSI) consisted of 4 components - phone calls (voice), text messaging (SMS), access to the internet (data) and access to an unstructured supplementary service data (USSD) that provided protocols for management of obstetric and neonatal emergencies in response to selection from a short code drop down menu. Unstructured supplementary service data is a communications protocol that allows two-way exchange of data between phone users and information linked to the pre-designed short codes stored on a remote computer of a telecommunications company. This makes USSD more interactive than text messaging. Each response message linked to a short code is limited to a length of 150 to 182 alpha numeric characters. The messages in this intervention were created by a team of frontline health workers, family physicians, obstetricians and pediatricians in the Greater Accra Region, drawing on the Ghana's Safe Motherhood protocols [29]. All four components of the intervention were part of a single composite intervention delivered on a non-smart mobile phone (Table 1). Access to the USSD was considered to be the main intervention component. Health workers were expected to use the phones primarily to access neonatal and maternal health emergency protocols via the USSD and obtain additional support from colleagues and the internet via the other intervention components. Each project mobile phone had a unique Subscriber Identification Module (SIM) card. All the SIM cards were networked in a Closed User Group (CUG) that allowed free and unlimited access to the USSD. Access to the intervention was however limited to the project SIM cards to avoid contamination.

Table 1

Components of the intervention.

Intervention component	Description
Cell phones	Distribution of the non-smart mobile phones by the research team to health facilities in the intervention clusters (districts) either as a shared-use phone or as individual-use phone. Each midwife was provided an individual-use phone and each health facility had a shared-use phone
Closed User Group (CUG)	A network of SIM cards with unlimited access to make free phone calls to other SIM cards within the network. All intervention users constituted membership of the CUG
Text messaging	Sending of up to 100 free SMS per month to SIM cards in as well as outside the CUG
Data bundle	System that provides up to 25megabytes of free data per month to the project SIM cards
Monthly credit top-up	҂An automated system from the telecommunication company that topped up 2 ∙ 50 cedis (0 ∙ 70 US dollars) worth of Vodafone credit on project SIM cards each month. This top up credit could be used at the discretion of the health worker for making calls, texting or browsing the internet beyond the limits set for text messaging and data bundle aforementioned
Reminders	Monthly reminders sent to the intervention users reminding them of the availability of the USSD protocols
Training	Health workers were trained on how to use the intervention firstly at a group gathering in each intervention district capital before the start of the cluster randomized controlled trial and then at least once during monitoring visits in their individual health facilities during intervention implementation
Unstructured Supplementary Service Data (USSD)	A communications protocol that allows a two-way exchange of data between a phone user and pre-programed information linked to short codes stored on a remote computer of a telecommunication company. This makes it more interactive than text messaging. Each response message linked to a short code is limited to a length of 150 to 182 alpha numeric characters. In the intervention districts it was used for requesting and receiving text-message based standard emergency obstetric and neonatal protocols on the request of a health worker. Access to the USSD was limited to only project SIM cards (CUG members)∙ For CUG members access to the USSD was free and with no limits to the number of times the USSD could be accessed

҂Exchange rate of 1 US dollar = 3 ∙ 56 cedis is based on the Bank of Ghana exchange rate at start of the intervention in August 2015.

Study objectives

In the CRCT whose findings are reported here, we evaluated the utilization and effect of the mCDMSI on institutional neonatal mortality in the Eastern Region of Ghana.

Methods

Study design

A two-arm cluster-randomized controlled trial (CRCT) to evaluate the effect of mCDMSI on neonatal mortality was implemented in 16 districts in the Eastern Region of Ghana [20]. Each of the 16 districts formed one cluster in this study. The intervention period lasted for 18 months.

Study site

The study site was the Eastern Region of Ghana, the third most populous region in Ghana (Fig. 1) [30]. The region is divided into twenty-one [21] geographic local administrative units called districts. At the start of intervention implementation, there were a total of 250 health facilities i.e. Community-based Health Planning and Services (CHPS) compounds and maternity homes, Health centres (HCs), and hospitals in the Eastern Region. At the primary health care level, the CHPS, HCs and maternity homes provide services including neonatal healthcare services to the various communities and refer cases to the hospitals. The Eastern Region ranks fourth in terms of high neonatal mortality rate (NMR) in Ghana [31]. The NMR for the region in 2014 was 30 per 1000 live births [31].

Fig. 1

Clusters participating in randomized controlled trial to evaluate the effect of an mHealth clinical decision-making intervention on neonatal mortality in Ghana.

Cluster selection criteria

The inclusion criteria for cluster selection for the CRCT included the following: i) District located in the Eastern Region of Ghana ii) Expected deliveries of ≥ 1100/year for the year 2014 for a district iii) Both District Health Management Team and the District Hospital Management Team agree to participate in the study iv) Health facilities within the district should have conducted at least one (1) delivery in the year 2014. The exclusion criteria for our study were: i) District location outside the Eastern Region ii) Expected deliveries of < 1100/year for the year 2014 for a district iii) The District Health Management Team and the Hospital Management Team disagreeing to participate in the study iv) Health facilities within the districts not conducting at least one (1) delivery during the year 2014.

The year 2014 was selected as the baseline year as the most current data pertaining to deliveries (births) at the time of commencement of the study was for that year. The protocol for this study has been published previously [20]. As data analyzed in this study was obtained from Ghana's national institutional health database, informed consent of patients for this study was not applicable. Consent to utilize data from the national institutional health database and to conduct this study was obtained from the Regional Health Directorate, Eastern Region, Ghana. The study was approved by the Ghana Health Service Ethics Review Committee (Reference: GHS-ERC: 10/09/14), and was registered at clinicaltrials.gov and Pan African Clinical Trials Registry PACTR20151200109073.

Randomization and masking

Out of the twenty-one eligible districts in the Eastern Region, seventeen districts fulfilled the inclusion and exclusion criteria for the CRCT. The regional capital was excluded from the selection process to avoid selection bias as its regional hospital is the highest referral point in the region. Sixteen clusters were therefore randomized into 8 intervention and 8 control clusters (Fig. 2). Cluster-randomization was preferred over individual randomization to avoid contamination both at the health professional and client levels, which may occur as a result of social interaction. A randomization scheme of permuted blocks was used to randomize the 16 districts equally to the two-armed program (control and intervention). The randomization scheme consisted of a sequence of blocks such that each block contained a pre-specified number of treatment assignments in random order. The purpose of this was so that the randomization scheme was balanced at the completion of each block. Randomization was performed by an independent data analyst in order to achieve comparability and avoid selection bias. Within the randomized clusters, all health facilities that conducted deliveries in the year preceding the start of the intervention (2014) were recruited into this study. Due to the nature of this intervention, masking was not feasible.

Fig. 2

Trial flow-chart of cluster randomized controlled trial to assess the effect of an mHealth clinical decision making tool on neonatal mortality in Ghana.

Sample size calculation

This study was designed as a superiority trial with neonatal mortality as the primary outcome. To detect a 30% decline in neonatal mortality at a power of 80%, a significance level of 0.05 (two-tailed test), with a fixed number of 8 clusters in each arm of the study and intra-cluster correlation coefficient for neonatal mortality of 0.0007256 [32], approximately 1065 patients in each of the 16 clusters was needed [20].

Recruitment of clusters

Participation in this study was at the cluster level. The impact of the intervention was measured by extracting data about deliveries that occurred in health facilities in the clusters recruited in this study.

Data collection

Data was extracted from the district health information management system-2 (DHIMS-2) database. The DHIMS-2 is a data recording, collection, collation and analysis tool that hosts the entire national institutional health data of Ghana [20]. Data in the DHIMS-2 comes from mainly public health facilities and a few private ones. The DHIMS-2 has been shown to provide reliable estimates of measures in some studies [33], [34], however, other studies have reported incomplete entries for certain variables in the database [35].

In the DHIMS-2, data of clients or patients who seek health services in a health facility is captured either in aggregate per health facility (e.g., hospital ‘A' had 20 deliveries), or as individual level data of all patients who were treated in each health facility. Individual level data is however, limited to clients who are seen and treated in hospitals. With regard to this study, data that was available in the DHIMS-2 and captured as aggregate per health facility were the number of neonatal deaths and the number of deliveries. Detailed information regarding each delivery captured in the DHIMS-2 was limited to hospital deliveries, and further limited to peri-partum maternal data (e.g., age, parity, and duration of pregnancy etc.). Thus, detailed information about babies delivered e.g., Apgar scores, weight and gender could not be obtained from the DHIMS-2. For each delivery that occurred in a hospital, there was no data that linked the detailed maternal delivery information to neonatal deaths that occurred in each health facility. Given these limitations with the DHIMS-2, we extracted data regarding incidence of neonatal mortality and deliveries per health facility and individual records of peri-partum characteristics of women who delivered in hospitals in the study clusters for the 18-month intervention period (August 2015 to January 2017), from the database. Fig. 3 illustrates the data structure for this study. Due to technical challenges with data entry and extraction from the DHIMS-2, seven hospitals agreed and captured the individual records of women who delivered in their facilities on excel spreadsheets that were given to the project team for analysis. The data entry in such situations was done by the hospital health information officers responsible for entering that data into the DHIMS-2 and the data was validated by the head of the health information unit in these hospitals. Thus data analyzed in this study is a combination of data already captured in the DHIMS-2 at the time of data analysis and, facility level data that may or may not be presently captured in the DHIMS-2. There were 8 private hospitals in total in this study; only one contributed individual level data into the database for analysis.

Fig. 3

Data sources and structure for cluster randomized controlled trial evaluating the effect of a clinical decision-making intervention in the Eastern Region of Ghana.

The research team collected baseline data regarding the number of doctors and midwives at post in each health facility and the location of health facilities. We classified health facilities into two groups of remote and non-remote areas based on access. Remote facilities were located either more than 30 min' walk, or more that 15 min motor-bike ride from the main district township, and had poor road access (uneven and untarred roads overcrowded with weeds and shrubs) leading to them [36]. Non-remote health facilities were located either within 30 min' walk, or 15 min motor-bike ride from the main district township, and had good road access leading to them.

Data concerning the use of the USSD protocols during intervention implementation was extracted from the database of the telecommunication company that provided support for the intervention (Vodafone Ghana).

Four intervention clusters were of interest in this study for 2 reasons; i. they shared boundaries with non-study clusters that did not have hospitals and/or, ii. they recorded high neonatal mortalities. In Ghana, address systems are not well established. To enable us analyze the addresses of women who delivered in hospitals within these clusters, the district health management team (DHMT) in each cluster was tasked to identify addresses within and outside their district from a list of addresses captured as addresses in their district in the DHIMS-2. The DHMT run the day-to-day health activities within a district, traveling to every corner of their districts; they are therefore a good resource with regard to identification of names of locations within a district that may not be formally documented.

Outcome measures

The primary outcome measure estimated in this study was institutional neonatal mortality which included deaths of babies admitted from birth and those (re)admitted from home. Utilization of the mCDMSI for clinical decision-making was estimated as a secondary outcome. For this study neonatal mortality was defined as death of a new-born occurring from birth up to the 28th day of life [37]. In Ghana, the expulsion of a product of conception before 28 completed weeks of gestation is considered an abortion. We therefore limited our analysis to pregnancies of gestation 28 completed weeks or more.

Statistical analysis

We performed an intention-to-treat analysis at cluster level. We assessed the peri-partum characteristics of the women who delivered in hospitals during the intervention period to identify possible imbalance in characteristics of these women and their pregnancies in the study arms. We limited our analysis of peri-partum characteristics of women delivering in health facilities to pregnancies of women in the reproductive age group of 15 to 44 years [38] as the excluded ages formed < 1% of available data. Potential sources of imbalance in the study arms i.e., age, parity, duration of pregnancy were summarized and expressed as means or medians, while insurance status and education level of women were expressed as numbers and percentages (Table 3). Differences in distributions of these potential confounders between the intervention and control arms where assessed using t-tests or Wilcoxon rank-sum tests and chi-square tests where appropriate. We calculated the proportion of remotely located health facilities, the number of deliveries per midwife and, number of deliveries per doctor per cluster to assess cluster level imbalance in the study arms.

Table 3

Characteristics of women delivering in hospitals in CRCT clusters during the intervention period.

aCluster	Age of women (years) (N = 39,803)	Parity of women (N = 39,636)	Gestation (weeks) (N = 37,407)	҂Total deliveriesb (N = 39,803)	Type of delivery (N = 39,803)			Number of multiple gestation deliveries (N = 38,938)	Education level of women (N = 20,918)				Number of women insured (N = 37,633)
	Mean (SD)	Median (IQR)	Mean (SD)	n (%)	Spontaneous vaginal delivery n (%)	Cesarean n (%)	cAssisted n (%)	n (%)	None n (%)	Primary n (%)	Secondary n (%)	Tertiary n (%)	n (%)
Intervention (n = 18,091)
A	28 ∙ 37 (5 ∙ 89)	1 (0–2)	38 ∙ 29 (2 ∙ 25)	1904 (10 ∙ 52)	1239 (65 ∙ 97)	638 (33 ∙ 97)	1 (0 ∙ 05)	40 (2 ∙ 10)	129 (6 ∙ 78)	210 (11 ∙ 04)	1168 (61 ∙ 38)	396 (20 ∙ 81)	1895 (99 ∙ 53)
B	27 ∙ 09 (6 ∙ 43)	2 (1–2)	37 ∙ 15 (1 ∙ 54)	3116 (17 ∙ 22)	1944 (64 ∙ 09)	1037 (34 ∙ 19)	52 (1 ∙ 71)	41 (1 ∙ 35)	505 (16 ∙ 24)	1119 (35 ∙ 98)	1390 (44 ∙ 69)	96 (3 ∙ 09)	3075 (98 ∙ 68)
C	27 ∙ 53 (6 ∙ 38)	2 (1–3)	38 ∙ 31 (2 ∙ 51)	2305 (12 ∙ 74)	1490 (64 ∙ 75)	796 (34 ∙ 59)	15 (0 ∙ 65)	55 (2 ∙ 67)	0 (0 ∙ 00)	29 (7 ∙ 61)	281 (73 ∙ 75)	71 (18 ∙ 64)	702 (99 ∙ 86)
D	26 ∙ 70 (6 ∙ 58)	2 (1–4)	37 ∙ 62 (1 ∙ 81)	1915 (10 ∙ 59)	1501 (78 ∙ 38)	413 (21 ∙ 57)	1 (0 ∙ 05)	39 (2 ∙ 04)	11 (0 ∙ 57)	205 (10 ∙ 71)	1611 (84 ∙ 17)	87 (4 ∙ 55)	1910 (99 ∙ 74)
E	26 ∙ 27 (6 ∙ 57)	2 (0–3)	39 ∙ 05 (1 ∙ 98)	824 (4 ∙ 55)	639 (77 ∙ 55)	184 (22 ∙ 33)	1 (0 ∙ 12)	20 (2 ∙ 44)	252 (31 ∙ 66)	129 (16 ∙ 21)	377 (47 ∙ 36)	38 (4 ∙ 77)	668 (83 ∙ 19)
F	27 ∙ 02 (6 ∙ 33)	1 (0–3)	35 ∙ 87 (0 ∙ 72)	4252 (23 ∙ 50)	3599 (85 ∙ 16)	609 (14 ∙ 41)	18 (0 ∙ 43)	53 (1 ∙ 25)	337 (7 ∙ 94)	605 (14 ∙ 25)	3071 (72 ∙ 31)	234 (5 ∙ 51)	4062 (95 ∙ 53)
G	26 ∙ 75 (6 ∙ 37)	1 (0–2)	38 ∙ 09 (2 ∙ 02)	1854 (10 ∙ 25)	1341 (72 ∙ 33)	407 (21 ∙ 95)	106 (5 ∙ 72)	25 (1 ∙ 35)	299 (17 ∙ 13)	256 (14 ∙ 67)	1109 (63 ∙ 55)	81 (4 ∙ 64)	1849 (99 ∙ 78)
H	26 ∙ 50 (6 ∙ 58)	1 (0–2)	38 ∙ 48 (2 ∙ 21)	1921 (10 ∙ 62)	1618 (84 ∙ 23)	303 (15 ∙ 77)	0 (0 ∙ 00)	30 (1 ∙ 56)	1465 (95 ∙ 88)	11 (0 ∙ 72)	49 (3 ∙ 21)	3 (0 ∙ 20)	1921 (100 ∙ 00)
dTotal	27 ∙ 09 (6 ∙ 40)	1 (1–3)	37 ∙ 39 (2 ∙ 02)	18,091 (100 ∙ 00)	13,371 (74 ∙ 48)	4387 (24 ∙ 44)	194 (1 ∙ 08)	303 (1 ∙ 71)	2998 (19 ∙ 19)	2564 (16 ∙ 41)	9056 (57 ∙ 96)	1006 (6 ∙ 44)	16,082 (97 ∙ 66)
Control (n = 21,712)
I	27 ∙ 40 (6 ∙ 27)	1 (0–2)	36 ∙ 97 (1 ∙ 96)	2186 (10 ∙ 07)	1519(69 ∙ 49)	454 (20 ∙ 77)	213 (9 ∙ 74)	51 (2 ∙ 33)	126 (5 ∙ 78)	225 (10 ∙ 32)	1618 (74 ∙ 22)	211 (9 ∙ 68)	2176 (99 ∙ 54)
J	26 ∙ 46 (6 ∙ 64)	2 (1–3)	36 ∙ 58 (1 ∙ 72)	961 (4 ∙ 43)	736 (76 ∙ 91)	187 (19 ∙ 54)	34 (3 ∙ 55)	17 (1 ∙ 77)	648 (67 ∙ 43)	13 (1 ∙ 35)	266 (27 ∙ 68)	34 (3 ∙ 54)	961 (100 ∙ 00)
K	26 ∙ 44 (6 ∙ 66)	2 (1–4)	38 ∙ 99 (2 ∙ 29)	1138 (5 ∙ 24)	875 (77 ∙ 85)	249 (22 ∙ 15)	0 (0 ∙ 00)	18 (1 ∙ 58)	121 (10 ∙ 64)	297 (26 ∙ 12)	668 (58 ∙ 75)	51 (4 ∙ 49)	1133 (99 ∙ 56)
L	26 ∙ 52 (6 ∙ 56)	2 (1–3)	36 ∙ 72 (1 ∙ 63)	2556 (11 ∙ 77)	2023 (79 ∙ 18)	512 (20 ∙ 04)	20 (0 ∙ 78)	39 (1 ∙ 55)	105 (4 ∙ 64)	257 (11 ∙ 37)	1758 (77 ∙ 75)	141 (6 ∙ 24)	2526 (98 ∙ 83)
M	26 ∙ 63 (7 ∙ 07)	2 (1–3)	39 ∙ 09 (2 ∙ 68)	762 (3 ∙ 51)	571 (75 ∙ 43)	181 (23 ∙ 91)	5 (0 ∙ 66)	2 (0 ∙ 78)	190 (25 ∙ 13)	43 (5 ∙ 69)	500 (66 ∙ 14)	23 (3 ∙ 04)	737 (96 ∙ 72)
N	26 ∙ 98 (6 ∙ 45)	1 (0–2)	37 ∙ 09 (2 ∙ 15)	2244 (10 ∙ 34)	1708 (76 ∙ 15)	531 (23 ∙ 67)	4 (0 ∙ 18)	36 (1 ∙ 60)	145 (6 ∙ 47)	223 (9 ∙ 96)	1595 (71 ∙ 21)	277 (12 ∙ 37)	2231 (99 ∙ 42)
O	27 ∙ 93 (5 ∙ 96)	1 (0–3)	39 ∙ 22 (2 ∙ 11)	9070 (41 ∙ 77)	6101 (67 ∙ 29)	2249 (24 ∙ 80)	717 (7 ∙ 91)	165 (1 ∙ 82)	74 (0 ∙ 82)	52 (0 ∙ 57)	8484 (93 ∙ 58)	456 (5 ∙ 03)	9003 (99 ∙ 26)
P	27 ∙ 02 (6 ∙ 35)	1 (0–2)	38 ∙ 46 (1 ∙ 86)	2795 (12 ∙ 87)	2181 (78 ∙ 12)	555 (19 ∙ 88)	56 (2 ∙ 01)	28 (1 ∙ 00)	185 (6 ∙ 62)	154 (5 ∙ 51)	2288 (81 ∙ 92)	166 (5 ∙ 94)	2784 (99 ∙ 61)
dTotal	27 ∙ 31 (6 ∙ 30)	1 (0–3)	38 ∙ 23 (2 ∙ 30)	21,712 (100 ∙ 00)	15,714 (72 ∙ 48)	4918 (22 ∙ 68)	1049 (4 ∙ 84)	356 (1 ∙ 68)	1594 (7 ∙ 45)	1264 (5 ∙ 91)	17,177 (80 ∙ 29)	1359 (6 ∙ 35)	21,551 (99 ∙ 26)

Clusters have been anonmyized A-P.

Column percentages are presented.

Includes vacuum, forceps and vaginal deliveries with episiotomy.

Total per CRCT arm.

We defined our denominator for neonatal mortality rate as ‘number of deliveries’ as we could only obtain information regarding peri-partum conditions of pregnancies that resulted in deliveries from the DHIMS-2. We estimated neonatal mortality as the number of neonatal deaths per the number of deliveries occurring in each cluster. We estimated the neonatal mortality per cluster during the one year proceeding the intervention period (prior risk of neonatal mortality) and analyzed the trend in neonatal mortality in the clusters during the intervention period. We estimated the effect of the intervention using a grouped binomial logistic regression with a random intercept per cluster specifying the Laplacian approximation to correct for the clustered design and estimated the intra-cluster correlation. We adjusted for the prior risk of neonatal mortality per cluster in analysis. The effect of the intervention compared with the control group was expressed with odds ratios (with 95% CI and p-values), which, given the low risk of the outcome, may be interpreted as relative risks.

Additional analysis of addresses of women who delivered in hospitals in four intervention clusters (clusters B, C, F and H) was performed to assess the proportion of deliveries within a cluster that were actually deliveries by women who lived within a specified cluster. We further analyzed the correlations between the number of USSD requests (maternal and neonatal requests combined) and the number of deliveries per cluster; the number of neonatal USSD requests and the number of neonatal deaths using Spearman correlation as a proxy for the extent to which the intervention was utilized in decision-making.

All analyses were two-tailed with a significance level 0 ∙ 05, and were performed in Stata version 13 [39].

Findings

Overall, 176 health facilities participated in this study: 74 of the health facilities were in the intervention arm of the CRCT, the rest (102) were in the control arm. Each cluster had at least one district hospital and a varying mix of health facilities, CHPS and maternity homes. The intervention arm had a higher proportion of remotely located health facilities compared to the control arm. The ratios of the number of deliveries to the number of doctors and midwives were comparable in both study arms at baseline. Table 2 describes the baseline characteristics of each cluster.

Table 2

Characteristics of study clusters.

aCluster	Number of health facilities				Proportion of remotely located health facilities	bNumber of deliveries per midwife	bNumber of deliveries per doctor	Prior risk of neonatal mortality per 1000 deliveries
	CHPS	Health Centre	Hospital	Total
	n (%)	n (%)	n (%)	n (%)	n (%)
Intervention
A	3 (27 ∙ 27)	7 (63 ∙ 64)	1 (9 ∙ 09)	11 (100 ∙ 00)	6 (54 ∙ 55)	110	350	2 ∙ 39
B	10 (66 ∙ 67)	1 (6 ∙ 67)	4 (26 ∙ 67)	15 (100 ∙ 00)	9 (60 ∙ 00)	98	459	3 ∙ 63
C	3 (37 ∙ 50)	3 (37 ∙ 50)	2 (25 ∙ 00)	8 (100 ∙ 00)	5 (62 ∙ 50)	85	c312	18 ∙ 53
D	2 (33 ∙ 33)	3 (50 ∙ 00)	1 (16 ∙ 67)	6 (100 ∙ 00)	3 (50 ∙ 00)	103	1035	1 ∙ 45
E	7 (77 ∙ 78)	1 (11 ∙ 11)	1 (11 ∙ 11)	9 (100 ∙ 00)	7 (77 ∙ 78)	77	1072	0 ∙ 93
F	2 (18 ∙ 18)	8 (72 ∙ 73)	1 (9 ∙ 09)	11 (100 ∙ 00)	4 (36 ∙ 36)	103	436	1 ∙ 53
G	1 (16 ∙ 67)	2 (33 ∙ 33)	3 (50 ∙ 00)	6 (100 ∙ 00)	1 (16 ∙ 47)	74	348	2 ∙ 88
H	4 (50 ∙ 00)	3 (37 ∙ 50)	1 (12 ∙ 50)	8 (100 ∙ 00)	4 (50 ∙ 00)	93	771	2 ∙ 60
Total	32 (43 ∙ 24)	28 (37 ∙ 84)	14 (18 ∙ 92)	74 (100 ∙ 00)	39 (52 ∙ 70)	92	534	4 ∙ 48
Control
I	11 (57 ∙ 89)	6(31 ∙ 58)	2(10 ∙ 53)	19(100 ∙ 00)	8 (42 ∙ 11)	84	490	0 ∙ 34
J	5 (45 ∙ 45)	5 (45 ∙ 45)	1 (9 ∙ 09)	11 (100 ∙ 00)	6 (54 ∙ 55)	99	2186	2 ∙ 74
K	7 (50 ∙ 00)	6 (42 ∙ 86)	1 (7 ∙ 14)	14 (100 ∙ 00)	10 (71 ∙ 43)	99	690	7 ∙ 25
L	8 (53 ∙ 33)	4 (26 ∙ 67)	3 (20 ∙ 00)	15 (100 ∙ 00)	5 (33 ∙ 33)	63	370	2 ∙ 70
M	6 (60 ∙ 00)	3 (30 ∙ 00)	1 (10 ∙ 00)	10 (100 ∙ 00)	6 (60 ∙ 00)	82	825	6 ∙ 67
N	6 (42 ∙ 86)	7 (50 ∙ 00)	1 (7 ∙ 14)	14 (100 ∙ 00)	8 (57 ∙ 14)	87	676	2 ∙ 59
O	1 (10 ∙ 00)	6 (60 ∙ 00)	3 (30 ∙ 00)	10 (100 ∙ 00)	3 (30 ∙ 00)	124	436	7 ∙ 25
P	3 (33 ∙ 33)	3 (33 ∙ 33)	3 (33 ∙ 33)	9 (100 ∙ 00)	5 (55 ∙ 56)	116	539	0 ∙ 78
Total	47 (46 ∙ 08)	40 (39 ∙ 22)	15 (14 ∙ 71)	102 (100 ∙ 00)	51 (50 ∙ 00)	93	527	3 ∙ 92

Clusters have been anonmyized A-P.

Data represents the one year preceeding the start of the intervention.

Excludes data from one hospital whose hospital management did not provide baseline data during data collection.

There were 65,831 deliveries during the intervention period. Of these deliveries, 31,155 (47 ∙ 3%) were in intervention clusters and the rest were in the control clusters. The median number of deliveries per cluster in the intervention arm was 3665 (range 1580 - 6319); in the control arm, median number of deliveries was 3750 (range 2076 - 10,473). In both study arms, most deliveries occurred in hospitals (intervention arm- 26,303 (84 ∙ 4%); control arm- 25,780 (74 ∙ 4%)). During the intervention period, there were 348 neonatal deaths; 198 (56 ∙ 9%) of these deaths occurred in the intervention arm and 150 (43 ∙ 1%) occurred in the control arm (ignoring clustering, the crude odds ratio of neonatal death in the intervention arm compared to the control arm was 1 ∙ 47 (95% CI (1 ∙ 19;1 ∙ 82); p < 0 ∙ 001). Neonatal deaths ranged from 4 to 80 (median = 16) in intervention clusters, and 0 to 86 (median = 9) in control clusters. All but 1 neonatal death occurred in hospitals; this neonatal death occurred in a HC in a control cluster.

Characteristics of women delivering in hospitals in the study clusters

Due to data availability, detailed information of women who delivered in the study clusters was analyzed for 39,803 deliveries (representing 76 ∙ 4% of hospital deliveries). Of this number, 45 ∙ 5% were from intervention clusters and 54 ∙ 6% were from control clusters. The women delivering in the study hospitals were on average aged 27 ∙ 1 (SD = 6 ∙ 4) and 27 ∙ 3 (SD = 6 ∙ 3) years in the intervention and control arms respectively (p < 0 ∙ 001). Seventy-five percent (75%) of the women in this study had experienced at least one previous childbirth. Women in the intervention arm delivered at a slightly earlier gestation (median gestation was 37 weeks) while most women in the control arm delivered at 38 weeks (p < 0 ∙ 001) (Table 3). Spontaneous vaginal delivery was the main mode of childbirth in both study arms (over 70%), followed by cesarean sections (24 ∙ 4% and 22 ∙ 7% in intervention and control arms respectively). The control arm recorded a higher proportion of assisted deliveries (4 ∙ 8% representing 1049 deliveries) compared to intervention arm (1 ∙ 1% representing 194 deliveries, (p < 0 ∙ 001)). The proportion of twin deliveries (1 ∙ 7%) was the same in both study arms. More than twice the number of women delivering in the intervention arm (35 ∙ 6%) had no form of formal education or had only attained primary education as compared to women in the control arm (13 ∙ 4%) (p < 0 ∙ 001). Both study arms had the same proportion of tertiary level educated women (6 ∙ 4% for both intervention arm and control arm). Nearly all women delivering in the hospitals held a form of health insurance. The proportion of health insured women was however, slightly lower in the intervention arm (97 ∙ 7%) compared to the control arm (99 ∙ 3%) (p < 0 ∙ 001).

Effect of the mHealth intervention on neonatal mortality

During the 18-month intervention period, institutional neonatal mortality in the intervention arm increased from 4 ∙ 5 to 6 ∙ 4 deaths per 1000 deliveries and in the control arm from to 3 ∙ 9 to 4 ∙ 3 deaths per 1000 deliveries. At cluster level, six intervention clusters and three control clusters recorded higher neonatal mortality during the 18-month intervention period (Fig. 4). The remaining clusters recorded lower or same incidence of neonatal deaths during the intervention period compared to the pre-intervention period. Intention to treat analysis, accounting for variation in the clusters showed non-significant higher odds of neonatal death in the intervention arm compared to the control arm (odds ratio = 2 ∙ 10 (95% CI (0 ∙ 77;5 ∙ 77); p = 0 ∙ 15)) and the intra-cluster correlation coefficient was 0⋅22 (95% CI (0 ∙ 10;0 ∙ 41)) (Table 4). Adjusting for the pre-intervention risk of neonatal mortality in the clusters, the odds of neonatal death was 2 ∙ 09 times higher (95% CI (1 ∙ 0;4 ∙ 38), p = 0 ∙ 051) in the intervention arm compared to the control arm.

Fig. 4

Institutional neonatal mortality per 1000 deliveries in intervention and control clusters one year before the start of the intervention and during the intervention period.

Table 4

Odds ratios of neonatal death during the 18-month intervention period

Variable	Number of neonatal deaths	Crude analysis			aAdjusted analysis
		Odds ratio (95% CI)	p-value	bICC (95% CI)	Odds ratio (95% CI)	p-value	ICC (95% CI)
Arm
Intervention (n = 31,155)	198	2 ∙ 10 (0 ∙ 77-5 ∙ 77)	0 ∙ 149	0 ∙ 22 (0 ∙ 10-0 ∙ 41)	2 ∙ 09 (1 ∙ 00-4 ∙ 38)	0 ∙ 051	0 ∙ 12 (0 ∙ 05-0 ∙ 26)
Control (n = 34,676)	150	1		1
Prior risk of neonatal death	-	-	-	-	2 ∙ 16 (1 ∙ 42- 3 ∙ 30)	< 0 ∙ 001

Analysis was performed using grouped binomial logistic regression with a random intercept per cluster and specifying the Laplacian approximation

Adjusted for prior risk of neonatal mortality in clusters during the one year preceding implementation of the intervention

ICC- intra-cluster correlation coefficient

Analysis of addresses of women delivering in key intervention clusters

Cluster C recorded the highest neonatal mortality in the intervention arm. In this cluster, 98 ∙ 5% (2217) of all addresses captured in the DHIMS-2 as being located in district C were identified by the DHMT staff. Of the addresses identified, 49% were within the cluster, 44 ∙ 5% were from other four intervention clusters and, 6 ∙ 5% were from control and non-CRCT clusters. In cluster F (district with second highest neonatal deaths in the intervention arm), 91 ∙ 5% of 4251 addresses were identified. Of the addresses identified, 73 ∙ 1% were addresses within the district, 22 ∙ 2% were addresses in control clusters, 4 ∙ 1% were addresses in non-CRCT clusters, and < 1% were addresses from other intervention clusters. For cluster B, 96 ∙ 4% (2994) of the addresses of women who delivered in the district hospital were identified by the DHMT staff. Of the addresses identified, 67 ∙ 7% were within cluster B, while 29 ∙ 7% were from neighboring non-CRCT clusters that did not have hospitals and 2 ∙ 4% from other intervention clusters. In cluster H, 96% of 1892 addresses were identified, of which 73 ∙ 5% were from the cluster H while 23 ∙ 6% were from non-CRCT clusters, 2 ∙ 8% were from other intervention clusters, and < 1% were addresses in control clusters.

Utilization analysis

There were 5329 requests made to the USSD from all clusters during the intervention period; the number of requests per intervention cluster ranged from 403 to 1167. The correlation between the number of USSD requests (maternal and neonatal requests combined) and the total number of deliveries in the intervention clusters was 0 ∙ 71 (p = 0 ∙ 05). The correlation between the number of USSD neonatal requests and the number of neonatal deaths was 0 ∙ 48 (p = 0 ∙ 23).

Discussion

The results of this cluster-randomized trial of the effects of perinatal mHealth support show that overall the risk of institutional neonatal mortality was higher in the intervention arm compared to the control arm. Lack of use of an intervention would be expected to leave mortality risks unaffected. In the text that follows we highlight possible explanations for the unexpected observed results.

Problems with registration of births and deaths

Births and deaths are captured in the DHIMS-2 according to the location these events occur irrespective of the primary residence of patients. Patient flow in and out of the clusters could have therefore influenced the observed effect. Four of the intervention clusters shared boundaries with non-study clusters that had no hospitals. These non-study clusters referred cases to intervention clusters as shown in the analysis of addresses. Frequent referral of cases from HCs, CHPS and even district hospitals to other hospitals is not uncommon [40] and could overburden referral hospitals, thereby hampering the quality of neonatal care services referral hospitals provide [41]. High risk deliveries are usually the ones that also get referred [40], [42]; thus the prognosis for these cases by the time they reach referral hospitals in settings similar to the study context tends to be poor. Several of the control clusters were close to the regional capital (Koforidua in the New Juabeng Municipal), where the regional hospital (the main centre for referrals) in the Eastern Region is located (see Fig. 1). Three control clusters were close to the national capital (Greater Accra Region) that has the largest density of better resourced health facilities and the largest referral centre in Ghana. Patients from these control clusters are often referred to the regional hospital or to Greater Accra Region for treatment. Patient flow out of the control clusters might explain further, the lower neonatal mortality rates observed in the control clusters. The DHIMS-2 at the time of data extraction did not capture detailed information of maternal or neonatal referrals to enable further analysis regarding patient flow in and out of clusters and how that may have contributed to the observed effect.

Confounding not adjusted in analysis or unmeasured confounding

By chance, there was variability in known prognostic factors of neonatal mortality (education status of women, age, delivery type, pregnancy gestation, number and proportion of rural located health facilities and prior risk of neonatal mortality per cluster) [43], [44], [45], [46], [47] between the study arms. However, in the DHIMS-2 database, individual level maternal data that provide the details of the aforementioned prognostic factors is not linked to neonatal data; neither are detailed characteristics of newborns captured in the database. This limitation in the data structure did not permit the correction for (potential) confounders in the analysis. Attempts to correct for confounding with propensity score methods [48] summarized at cluster level gave similar results possibly due to an ecological fallacy [49]. The correction of the aforementioned baseline imbalances in the study arms as well as other unmeasured confounders known to impact neonatal mortality (e.g., sex of neonate, APGAR scores, birth weight, multiple gestation) [18], [22] may have given different results. Stratification of key prognostic factors during randomization (in order to adjust for these prognostic factors in analysis) was not considered in the design phase of this study as any baseline imbalance observed was expected to be due to chance.

Inadequate use of the intervention

To understand how and why the intervention was used (or not) to help us interpret the results of this trial, a study was undertaken. Data collection involved key informant interviews and focus group discussions with intervention users and the data was manually analyzed for themes. The study showed that the phones were predominantly used for voice calls (64%), followed by data (28%), SMS (5%) and USSD to access protocols (2%) respectively [36]. Over time, use of all intervention components declined. Individual health worker factors (demographics, personal and work-related needs, perceived timeliness of intervention, tacit knowledge), organizational factors (resource availability, information flow, availability, phone ownership), technological factors (loss of phones, network quality) and client perception of health worker intervention usage explained the pattern of intervention use observed [36]. In this study we report significant correlation between the number of deliveries and use of the USSD, however, this does not preclude inappropriate use of the intervention protocols. Although unintended use of mHealth interventions is not uncommon [50], [51], and strategies to improve appropriate use of mHealth interventions (such as reward schemes and reminders) are well documented in literature [52], [53], [54], overall, our findings suggest to carefully consider whether this kind of mHealth intervention is the most appropriate in the study context.

Fig. 3 summarizes the limitations in the data structure of this study that led to the inability to measure and (or) adjust for differences in prognostic factors between the study arms to improve the quantification of effect size. Analyses, e.g. post-hoc and baseline comparability analysis, which are not conventionally performed as per the CONSORT guidelines were undertaken to in the context of this study to gain insight into possible explanations for the observed intervention effect. This study did not measure observed use (practical application of protocols in case management) or non-use of the intervention in this evaluation. Details of voice, data and SMS components of the intervention could not be ascertained to have been used by health workers to obtain clinical decision-making support [36] thus in the correlation analysis, we analyzed only the USSD component of the intervention. The rise in institutional neonatal mortality observed in both study arms could not be explained by the methodology used in this study but warrants urgent attention. Concurrent neonatal health improvement interventions that may have been on-going in the Eastern Region particularly in the control clusters that could have influenced the findings of this study could not be accounted for. Despite these limitations, this large study provides valuable information about the impact of an mHealth intervention on health outcomes in a low resource setting. Previous documentation of mHealth interventions in low resource settings have been mainly small pilots with a focus on utilization of interventions [13]. The few mHealth interventions that have measured outcomes have shown mixed results and could have possibly overestimated intervention effect size due to the relatively small study sample size [55], [56].

Conclusion

This study showed that providing access to an mCDMSI to frontline health workers to facilitate clinical decision-making in a low-resource setting did not lead to an improvement in institutional neonatal mortality. The point estimate of the adjusted analysis even suggests an increased risk in the intervention group. We discussed various factors that could have influenced the results, though the exact impact of these factors remains uncertain. Our study highlights that technological innovation alone is not enough to affect health outcomes. It is important to understand the mechanisms influencing outcomes in context as shown in our linked study and to design and implement interventions that address the combined effect. As the paradigm of mHealth interventions shift from small pilots to larger studies in LMICs, careful evaluations to assess their impact on health outcomes and not merely their uptake are needed. Such large studies will require improvements in available databases leading to better data quality. Furthermore, lessons learnt from this study could inform design and evaluations of mHealth interventions in similar settings.

Declaration of Competing Interest

We declare no competing interests.