The NeoTree application: developing an integrated mHealth solution to improve quality of newborn care and survival in a district hospital in Malawi.

More than two-thirds of newborn lives could be saved worldwide if evidence-based interventions were successfully implemented. We developed the NeoTree application to improve quality of newborn care in resource-poor countries. The NeoTree is a fully integrated digital health intervention that combines immediate data capture, entered by healthcare workers (HCW) on admission, while simultaneously providing them with evidence-based clinical decision support and newborn care education. We conducted a mixed-methods intervention development study, codeveloping and testing the NeoTree prototype with HCWs in a district hospital in Malawi. Focus groups explored the acceptability and feasibility of digital health solutions before and after implementation of the NeoTree in the clinical setting. One-to-one theoretical usability workshops and a 1-month clinical usability study informed iterative changes, gathered process and clinical data, System Usability Scale (SUS) and perceived improvements in quality of care. HCWs perceived the NeoTree to be acceptable and feasible. Mean SUS before and after the clinical usability study were high at 80.4 and 86.1, respectively (above average is >68). HCWs reported high-perceived improvements in quality of newborn care after using the NeoTree on the ward. They described improved confidence in clinical decision-making, clinical skills, critical thinking and standardisation of care. Identified factors for successful implementation included a technical support worker. Coproduction, mixed-methods approaches and user-focused iterative development were key to the development of the NeoTree prototype, which was shown to be an agile, acceptable, feasible and highly usable tool with the potential to improve the quality of newborn care in resource-poor settings.

More than two-thirds of newborn lives could be saved worldwide if evidence-based interventions were successfully implemented.

Reliable data sources and health information systems for counting births, stillbirths, neonatal deaths and delineating the causes of death are lacking.

With the fast-moving digital revolution in low-income countries mobile health applications are being increasingly designed but these have not yet focused on facility-based newborn care.

Here we present the NeoTree application, focusing on newborn care in low-income facilities, combining data collection by healthcare workers themselves, with interactive decision support and education for improving quality of care.

We report iterative codevelopment with healthcare workers to create a highly usable interactive admission platform, which provides teaching and training, improving the perceived quality of care delivered by healthcare workers while admitting the baby.

Introduction

Every year, 2.9 million neonatal deaths (NND) occur worldwide accounting for 44% of deaths in children under-five.1 In sub-Saharan Africa, reduction in neonatal mortality rate (NMR) has been slow, at 1.5% per year from 2000 to 2010.2 An estimated 70% of these deaths could be avoided through the implementation of evidence-based, simple interventions especially on the first day of life.3 Accurate and timely data capture is key to improved outcomes for newborns. Only through counting each newborn and each newborn death can we better understand, develop and prioritise strategies to tackle critical areas for improvement. Only 17 priority countries (11 of which are from sub-Saharan Africa, Mexico, Azerbaijan, Uzbekistan, Iraq, Bangladesh and Indonesia) have a policy for reporting and reviewing stillbirths and NNDs4 and at least 15 000 babies die each year globally without a formal record of their existence.1

In February 2017, the WHO maternal and newborn quality of care network committed to halving preventable newborn deaths in the next 5 years, aiming ‘to develop evidence-based, context-specific strategies to manage quality improvement (QI)’, and ‘to collect data about what is working’.5 Malawi joined this quality of care network and has made good progress in reducing NMR (3.5% per year over last decade) but this reduction is slower than for children aged 1–59 months (7.1% per year over last decade).1 Malawi has one of the poorest quality healthcare systems (ranking 185th out of 190th globally),6 an NMR of 23 per 1000 live births7 and the highest preterm birth rate in the world.8 Leaders in newborn health have drawn attention to gaps in quality of care, especially at birth in medical facilities (where 89% of women in Malawi deliver).9 Key deficits are paucity of trained newborn healthcare workers (HCW) and a lack of facility-based information systems for data collection, monitoring and evaluation.

Mobile health (mHealth) is a fast-developing global health tool. In low-income countries (LIC) most people now own a mobile phone10 and mHealth clinical decision support tools are changing practice, for example, in the management of emerging outbreaks11 and fever in under 5 year-olds.12 Newborn mHealth solutions designed for LICs, such as the ‘Safe delivery app’,13 mainly focus on maternal health rather than newborn health.

Developing the NeoTree application

In partnership with key stakeholders and HCWs in Malawi, we are developing the NeoTree application (app),14 a fully integrated QI tool for hospital-based sick and vulnerable newborns. It combines immediate digital data capture, evidence-based algorithmic clinical decision support and newborn education on one platform. This mHealth app is used on low-cost android tablets or phones at the hospital bedside by HCWs (primarily nurses) supporting their care and treatment of sick newborns. The NeoTree works off-line to synchronise with a network when available, is a not-for-profit venture with open-source code15 and can be preconfigured to adapt to available resources within the index facility.

Developing NeoTree-Alpha

We have followed the usual software development and Medical Research Council complex intervention development cycles in developing the NeoTree (figure 1). In phase 1, a literature review of HCW-led newborn interventions in LICs informed conceptualisation of the NeoTree. A prototype was found to be acceptable to HCWs in Bangladesh. In the UK (phase 2), an editor platform was designed (using Fire-base by Google and Java code (Java is a common computer programming language for general purpose in software development)) to be configurable without software development skills thus allowing a clinician to build the app.

Figure 1

NeoTree research programme overview showing software development cycle linked with research plan and detail of phase 3 methods. FGD, focus group discussion; MVP, minimum viable product; SUS, system usability study.

Functional components of the NeoTree

Immediate digital data capture

NeoTree-Alpha was initially configured as an electronic admission form whose screen-flow exactly replicated the Malawi Ministry of Health (MOH) paper form (figure 2). Field validation has been set so the user cannot progress to the next page until key information has been entered correctly. Fields on the form exist as multiple or single-choice questions, drop-down menus or free-text fields.

Figure 2

Example pages of NeoTree app with example functional content. F1, function 1: data collection; F2, function 2: decision support; F3, function 3: education in a=emergencies, b=non emergencies.

Clinical decision support

Emergency decision support

Emergency prompts and cues are interwoven with the emergency triage section of the form so that emergency management screens appear in real time if abnormal data are entered. Abnormal data prompt evidence-based clinical actions by the HCW to stabilise the baby, according to WHO and Malawi guidelines (Care of the Infant and Newborn (COIN)).16

Non-emergency decision support

The functional ability for the NeoTree to generate a machine diagnosis, by processing all the information entered in a clinical algorithm embedded within the app, has been conceptualised and will be implemented in future phases of development. For this phase (phase 3), following data entry, the HCW is asked to choose a differential diagnosis(es) from a list of common diagnoses provided by the app. Management pages, created as succinct summaries of COIN and WHO guidelines, appear to the HCW according to their chosen diagnosis(es). In the future these will instead be linked to the clinical algorithm diagnosis.

Educational content

Brief educational material has been added (text and pictorial) to support all management pages. Figure 2 shows an example image describing the mechanisms of heat loss on the hypothermia page. Video material is not in the scope of this phase of intervention development.

Data transfer

Data transfer from each device via an application program interface to a national database and to a local database linked with a local newborn ‘data dash-board’ in each facility, as easily digestible graphs and charts, has also been conceptualised, but not developed in this phase.

Development of NeoTree-Beta

NeoTree-Beta was subsequently developed from NeoTree-Alpha within Zomba Central Hospital (ZCH) in Malawi (phase 3) with iterations launched using the app ‘Beta’ from Crashlytics.17 The aims were to refine the order, layout and design of the digital form, to create a user-friendly, efficient and smooth-flowing digital admission process and to refine the content of the management pages and the educational material with the end users in the Malawi setting.

Setting

Healthcare in Malawi is provided through the government (free at point of care), the Christian Health Association of Malawi (small user fee) and private companies (with each sector delivering 62%, 37% and 1% of provision, respectively). Government provision consists of central, district and rural hospitals. ZCH is one of the four central hospitals but operates as a district-level facility. The neonatal unit has three wards, and at the time of the study, nine permanent staff (with one to four nurses working each shift) and an average daily admission rate of 7.5 babies. The most recent 2016 clinical audit prior to this study revealed a case fatality rate of 160 per 1000 admissions.18 HCWs have access to intranasal oxygen, bubble continuous positive airway pressure (CPAP), blood sugar monitoring, overhead warmers, phototherapy and basic laboratory tests. Approximately two-thirds of admissions are born in the hospital. There are no paediatric or neonatal doctors with the majority of care delivered by nurses and student-nurses.

Implementation activities

The NeoTree was introduced and tested in ZCH in four stages: (A) feasibility and acceptability focus group discussions (FGD); (B) theoretical usability workshops; (C) a clinical ward usability study; and (D) final FGDs. Iterative app configuration by the onsite researcher via the editor continued throughout the study.

Feasibility and acceptability focus group discussions

We conducted two FGDs, exploring HCWs perceptions of digital aids in healthcare and the NeoTree concept. The hospital director, matrons and all paediatric or neonatal clinicians and nurses were invited to attend. The director and matrons did not attend and approximately half of invited clinical staff participated. The topic guide was developed by the University College London research team, a Malawian neonatologist and a bilingual Chichewa/English-speaking FGD coordinator (Chichewa is the local language). Guides were developed in English, translated to Chichewa and then back translated to ensure local applicability and consistency. Part 1 prompted discussion around perceptions of facility-based quality newborn care, barriers and facilitators to its delivery and the use of digital aids in healthcare. Part 2 encouraged discussion regarding the acceptability and feasibility of the NeoTree concept (preceded by a short introductory presentation of the NeoTree). The FGDs were facilitated by a Chichewa-speaking coordinator and note taker in the absence of the research team. Written consent was obtained, and participant demographic information recorded using a unique participant identification number. Audio recordings were taken, transcribed and translated into English. Manual thematic analysis was conducted on the transcripts by four researchers (EK/CC/MH/FGD coordinator). Each researcher independently listed emerging codes and themes, which were collated until saturation point was reached.

Theoretical usability experience workshops

All newborn HCWs working at ZCH (n=20) were invited to attend one-to-one, 60 min, theoretical usability workshops. After consenting, video and audio recordings were taken as HCWs completed the NeoTree-Alpha for the first time according to a clinical scenario. Behaviours, reactions and body language were recorded. Common themes from the audio were identified and cross-checked with video recordings. Finally, each HCW completed the System Usability Scale (SUS),19 a validated 10-item rating of a product’s usability,20 applicable to many types of technology.

Clinical usability study

A 1-month, ward-based clinical usability study was conducted. After an hour-long group training session HCWs were supervised by the researcher using the NeoTree on one new admission each. Hardware included three Nexus-7 tablets stored at the nursing station (protected with wipe clean covers), an extension lead, chargers and a locally purchased wireless printer.

HCWs were asked to admit a baby using both the usual paper admission form and the NeoTree. The usual paper process involves infants presenting to the ward with an MOH form already containing maternal information completed by HCWs in the labour ward or the referring facility. The newborn HCW subsequently completes the newborn section of the form, documenting their admission diagnosis and management plan. In comparison, during a digital NeoTree admission, as the HCW progresses they are prompted to respond to any abnormal data entered and stabilise the baby. Once the NeoTree form is complete, the HCW can wirelessly print a summary, sign and attach it to the patient record.

The NeoTree was introduced to the ward gradually, increasing from weekday use only, to include weekend days, nights and, finally, 24 hours use without supervision. A researcher was present from 09:00 to 17:00 to troubleshoot technical issues. Daily details of completed NeoTree episodes were recorded including: infant name, NeoTree unique identification number (UID), admission date, HCW identification code, whether or not the admission was supervised (by a researcher) and start and finish time of the session. These data were cross-checked against the ward report book (a paper record of admissions) to calculate percentage daily admissions captured by the NeoTree. Session start and finish times were automatically recorded by the app unless the HCW neglected to press save, in which case session duration was unavailable for analysis. Clinical outcomes at discharge, namely survived, died (NND) or absconded, were manually entered by the researchers on a ‘NeoDischarge’ form within the app and linked to the admission data via UID.

Final focus group discussions

One week after completion of the clinical usability study, two FGDs were conducted following the same procedures as the previous FGDs, with the exception that participants were grouped by clinical seniority into two groups: (1) student-nurses and (2) nursing staff. Qualitative acceptability, feasibility and usability data were collected. Quantitative data were also collected at the end of these discussions including SUS (end-line) and a Perceived Quality Improvement (PQI) score. End-line SUS was only collected from HCWs who had not previously completed one at baseline. For HCWs who could not attend the FGDs, due to rotation out of the department, SUS was collected at the end of the clinical usability study.

The PQI questionnaire was devised by the research team guided by observations of the NeoTree, HCW feedback and their clinical neonatal experience. Fifteen areas of basic clinical care that NeoTree could potentially impact were identified. Eleven items best suited to Likert scoring were chosen to be in the questionnaire.

Preliminary evaluation of the NeoTree

Overall, 46 HCWs participated in this development study, representing all five cadres of newborn HCWs at ZCH (table 1). Participants were predominantly female (71%), aged 19–65 years with a wide range of neonatal experience (0–19 years, median 1).

Table 1

Participant characteristics

Characteristic		Stage A		Stage B	Stage C	Stage D
		FGD1 (n=6)	FGD2 (n=9)	Theoretical usability study and baseline SUS (n=13)	Clinical usability study (n=31)	FGD3 (n=9)	FGD4 (n=4)	SUS*(n=13)	PQI† (n=13)
Gender, female (n (%))		3 (50)	7 (78)	11 (84)	22 (73)	6 (67)	4 (100)	9 (64)	10 (77)
Age, years (mean (range))		34 (26–51)	35 (25–65)	34 (24–65)	30 (19–65)	23 (19–28)	43 (28–65)	23 (19–27)	29 (19–65)
Cadre (%)	NMT	33	44	61	24	0	75	0	23
	Nursing officer	0	0	15	18	0	25	0	8
	Nurse	0	11	0	4	0	0	0	0
	Student-nurse	0	0	15	53	100	0	100	69
	Clinical officer‡	67	44	8	0	0	0	0	0
Years of neonatal experience (mean (range))		13.5 (1–19)	4 (1–8)	2.5 (0–8)	1 (0–8)	0	4.5 (1–8)	0	1.4 (0–8)
Previously used a tablet device (n (%))		2 (33)	4 (44)	3 (23)	15(50)	8 (89)	1 (25)	3 (23)	9 (69)
Regularly used a tablet device (n (%))		1 (16)	2 (22)	2 (15)	4 (13)	2 (22)	1 (25)	2 (15)	3 (23)
Received COIN training (n (%))		4 (66)	7 (78)	6 (46)	5 (16)	0 (0)	2 (50)	0 (0)	2 (15)
Received HBB training (n (%))		0 (0)	0 (0)	2 (15)	22 (71)	8 (89)	1 (25)	13 (100)	9 (69)
Total NTs completed (mean (range))		NA	NA	1 (1–1)	4 (1–12)	3.5 (1–11)	7.3 (5–11)	3.5 (1–11)	4.7 (1–11)

*The 13 participants completing end-line SUS included eight students from FGD3 and five other students from the ward study who could not attend the FGDs.

†Participants who attended FGD3 and FGD4 also completed the PQI survey.

‡Clinical officer is a clinician who has received 4 years of medical training: 2 years theoretical, 2 years clinical.

COIN, Care of the Infant and Newborn; FGD, focus group discussion; HBB, Helping Babies Breathe; NA, not applicable; NMT, nurse midwife technician; NT, NeoTree; PQI, Perceived Quality Improvement; SUS, System Usability Scale.

Quantitative evaluation

The main quantitative outcome measures were SUS and PQI score (table 2), both of which use Likert scales of 1–5, where 1=strongly disagree and 5=strongly agree. SUS was collected before and after the clinical usability study (termed baseline and end-line, respectively) while PQI was only measured at end-line.

Table 2

System Usability Scale (SUS) and Perceived Quality Improvement (PQI) score*

SUS*	Baseline		End-line
	Mean (SD)	Converted†	Mean (SD)	Converted†
1. I think I would like to use this system frequently.	4.8 (0.4)	3.8	4.6 (1.1)	3.6
2. I found the system unnecessarily complex.	1.8 (1.3)	3.2	1.1 (0.3)	3.9
3. I thought the system was easy to use.	4.2 (0.6)	3.2	4.4 (1.1)	3.4
4. I think that I would need the support of a technical person to be able to use this system.	3.1 (1.5)	1.9	1.6 (1.3)	3.4
5. I found the various functions were well integrated.	4.8 (0.4)	3.8	4.8 (0.4)	3.8
6. I thought there was too much inconsistency in this system.	1.5 (0.7)	3.5	1.8 (1.3)	3.2
7. I would imagine that most people would learn to use this system.	4.5 (0.7)	3.5	4.2 (1.2)	3.2
8. I found the system very cumbersome to use.	1.9 (1.3)	3.1	2.3 (1.3)	2.7
9. I felt very confident using the system.	4.5 (0.9)	3.5	5.0 (0.0)	4.0
10. I needed to learn a lot of things before I could get going with this system.	2.3 (1.4)	2.7	1.8 (1.2)	3.2
Calculated SUS (total converted mean scores × 2.5)		80.8		86.1

*For both scores 1=strongly disagree, 5=strongly agree.

†Converted score out of 4; calculated by the formula x−1 for odd numbered questions and 5−x for even numbered questions where x=mean score.

CRT, capillary refill time; TRY-CPAP, Tone, Respiratory distress, Yes heart rate is >100, continuous positive airway pressure.

SUS was high at baseline (80.8) (table 2), where a score of 68 or higher indicates above average usability,19 demonstrating inherent usability of NeoTree-Alpha even after one single use under theoretical conditions. This rose to 86.1 at end-line which may represent improved usability following iterative app development and cocreation, or user characteristics. HCWs completing baseline SUS were mainly qualified nurses with lower reported computer literacy than participants completing end-line SUS who were all student-nurses. However, levels of previous regular tablet exposure in the two groups were similar at 22% and 25%, respectively.

PQI mean scores were high (more that 3.9 out of a maximum of 5) in all questions. HCWs perceived that the NeoTree improved a range of neonatal skills (table 2) including recording and measuring vital signs and using the TRY-CPAP algorithm (an experimental algorithm to optimise CPAP allocation where T=Tone is good, R=Respiratory distress, Y=Yes heart rate is >100).18 Greatest perceived improvements were in checking admission blood sugar and history taking and examination, where all respondents gave a maximum score of 5 (table 2; questions 4 and 9 of PQI score, respectively). This provides preliminary evidence of quality of care improvement, particularly for student-nurses (the future nursing workforce) who are often unsupervised.

NeoTree coverage

During the clinical study month, 70% of new patients were admitted on the NeoTree (134 of 191 admissions) (mean 7.4 per day). Thirty-one nursing staff; aged 19–65 years, predominantly female (73%), representing four different cadres of HCW (53% were student-nurses) completed NeoTree admissions. Fifty per cent had previously used a tablet device. HCWs used the NeoTree for admitting a baby between 1 and 12 times during the clinical usability study. A fifth of NeoTree admissions were supervised by the NeoTree researcher. Time taken to complete a NeoTree session data were available for 84 of the 134 admissions (48 cases with missing end-time, 1 case with duration of 14 hours and 1 repeat entry were excluded). Mean NeoTree completion time was 37.0 min (range 18.0–59.0 min), which compares to 10–15 min taken to fill in the paper form. Mean NeoTree completion time reduced from 37.6 to 31.4 min from week 1 to week 4, suggesting a learning affect. Importantly, all NeoTree admissions were completed within 60 min, the ‘golden hour’, a current worldwide standard emphasising evidence-based specialised neonatal care within the first hour of life.21

Digital audit

Of 134 digital admissions, 129 were linked anonymously with outcome data (four incomplete sessions and one repeat entry were excluded). Data were instantly exported by HCWs from the devices to an Excel spreadsheet on the researcher’s laptop and presented as graphs and charts to the department in an outcome audit. Results demonstrated an overall case fatality rate of 93 per 1000 babies admitted using the NeoTree (n=129). The most common admission diagnoses were sepsis (in 40% of admissions), prematurity (19%) and birth asphyxia (18%) with respective case fatality rates of 40, 40 and 260 per 1000 NeoTree admissions. These preliminary data support our hypothesis that the NeoTree could provide perinatal death audit data and potentially influence data-driven QI behaviours in HCWs via local dash-boards.

Qualitative evaluation

Acceptability and feasibility focus groups

Participants comprised 44% nursing staff and 55% clinical officers (table 1). Five key themes were identified: (1) barriers and (2) facilitators to the delivery of quality newborn care; (3) experience of digital aids in healthcare; (4) perceptions of digital aids in healthcare; and (5) impressions of the NeoTree concept. Key barriers to quality newborn care identified included inadequate, outdated training and poor access to updated guidelines. Potential facilitators included improved data systems and the use of clinical guidelines.

Perceptions of digital aids in healthcare

Participants reported limited experience of digital aids in healthcare, examples given included the use of library computers, the Baobab (Baobab is an electronic medical record system developed for tracking the treatment of HIV/AIDs in hospitals in Malawi using touchscreen computers and numbered labels—https://www.cdc.gov/globalaids/success-stories/innovativemalawi.html) system and tablet data collection within projects such as ‘Learner Treatment Kit’ (Learner Treatment Kit project is a project where teachers used a first aid kit including rapid diagnostic tests to diagnose and treat malaria in schools—file:///Users/carolinecrehan/Downloads/Learner_Treatment_Kits.pdf). Table 3 shows subthemes which were both positive (feasible, saves information, eases workflow, aids data processes) and negative (not timely, person dependent, inflexible, unsustainable, rigid, slow, needs backup, creates dependency, difficult for locums, fear, lack of autonomy, maintenance and difficulties with parallel systems). Participants identified factors that might improve implementation of digital aids in healthcare (online supplementary appendix table A). Discussions included app-related factors, such as responsiveness and timeliness of processes, and wider system factors, such as equitable access, sustainability, fast network connection, secure storage and backup, technical support and local ownership.

Table 3

Perceptions of the use of digital aids in healthcare and example quotes

Subthemes	Example quotes
3A: Positive perceptions
Saves information	‘I think it's good for the sake of keeping information’(N12)‘a helpful data collection tool.’ (CO8)
Ease workflow	‘Electronic devices maybe can improve data collection…ease workflow.’(CO8)
Robust	‘With the system we are using now, files, you find that2days…[later] other pages are not in the file, or maybe the file has been soaked with urine…So I think it [digital system] will help us.’(CO8)
Fast, time saving	‘It's very fast…You are able to send information immediately’(CO7)‘it is time saving.’ (N13)
Clinically supportive	‘I think technology is helping us a lot.’(CO8)
Retrievable	‘Easy retrieval of information.’(N7)

CO, clinical officer; N, nurse.

Perceptions of the NeoTree concept

HCW discussion of the NeoTree concept (online supplementary appendix table B) was largely positive; that it could potentially inspire confidence, guide clinical management and provide a platform for training and data collection. They also anticipated the value of the NeoTree in health centres (HC) where HCWs receive limited newborn training and from where ZCH receives many referrals. The consensus was that the NeoTree would be acceptable, feasible and ultimately successful. Key success factors included positive staff attitude, backup power, training of temporary staff, strong leadership, teamwork and staff engagement.

Theoretical usability workshop feedback

Thirteen of 20 newborn HCWs working in ZCH participated in the workshops (table 1). Eighty-five per cent reported regular use of Android smartphones and 23% reported previous tablet use. Online supplementary appendix table C summarises the qualitative usability feedback obtained, and the corresponding app adjustments made.

Nine of the 15 student-nurses and four of the nine staff in the clinical usability study attended (table 1). Eight key themes emerged as follows (quotes are labelled: nurses: N and student-nurses: SN).

Theme 1: improved quality of care and outcomes

Subthemes included perceived improvements in clinical skill, critical thinking, confidence, standardisation and improved quality of care, the comprehensive, holistic nature of the NeoTree, task-shifting and improved newborn survival. HCWs felt the NeoTree ‘…added some [clinical] experiences [skills] we have never used, for example using the stethoscope’ (N1), and ‘assess[ing] how many weeks gestation was the baby’ (SN7). They felt it helped them to prioritise; ‘when using the NeoTree, you first solve the important part’ (SN8) and ‘think critically’ (SN2). They felt it improved diagnosis and management saying it was ‘like a guide’ (N3) and ‘it provides a standard management’ (SN6), ‘[it provided clinical guidance], in cases where you forget’ (SN1). All participants felt the NeoTree was comprehensive in collecting information; ‘using the NeoTree you can't skip’ (N2), ‘it makes sure that all parameters of the baby holistically have been touched’ (SN6). Resultant positive attitudes such as empowerment, confidence and ‘credibility’ (SN7) were also reported; ‘whenever you are holding a tablet you are so confident’ (SN4) and ‘to me it was just pride to be using the tablet’ (N1). Student-nurses reported improved efficiency; ‘it (The NeoTree) made our lives easier,’ task shifting; ‘it helps you…when doctors are away’ (SN7) and improved outcomes ‘it is a life-saving system’ (SN2).

Theme 2: drawbacks

Subthemes included concerns regarding potential time delays, resource waste, data loss and system rigidity leading to false data entry. HCWs were concerned the NeoTree is ‘time-consuming’ (N1) and (N3) and that ‘… when you are the only one on duty, it is difficult to use’ (N3). One HCW described potential delay in management of the index baby and other babies ‘you will not attend the other clients quickly’ (N1). Specific resources wasted were gloves (due to multiple glove changes) and paper for printing. A false perception was voiced that a pause during a NeoTree session would result in data loss. Rigidity of the system was a concern for student-nurses: ‘… sometimes it says no to going on if you skip a space, hence we lie’ (SN3) and ‘if we don't have gluco-sticks, we end up writing false figures’ (SN5).

Theme 3: data systems

Subthemes identified included completeness of data, ‘positively it has helped us get more information’ (SN5), flexibility of the system ‘you can change it and correct it’ (N2), easy to capture, ‘it saves (the) history’ (SN7) and that the information is ‘retrievable’ (N4). Potential audit capacities were recognised: ‘one day [NeoTree] will be able to give us how many patients…have you discharged? How many have died?’ (N3).

Theme 4: perceived parental/guardian view

HCWs reported that parents ‘say it… [the NeoTree] takes a lot of time’ (N4) and ‘we ask unimportant questions’ (N4). Reportedly it was difficult for guardians to answer NeoTree questions on behalf of the parent and some parents were ‘afraid they think you are taking the information someplace else’ (N3). The student-nurses reported that parents/guardians thought the tablets were a distraction and that ‘we are just on social networks’ (SN6), while others viewed it positively ‘with gladness’ (SN6) finding it ‘a privilege’ (SN6).

Theme 5: requested improvements

Constructive feedback included app-related issues, process issues, suggestions for the printout, improved manpower and educating parents/guardians—the vast majority of which have been responded to with iterative app development. Two key suggestions yet to be implemented were to ‘get rid of the paper form’ (SN3) to avoid repetition of work and ‘Have a NeoTree ambassador to be stating advantages, to the ones using it and the mothers and guardians’ (SN5), and to help complete the NeoTree during busy periods.

Theme 6: education and training

Student-nurses thought the NeoTree ‘will play a big role in teaching and education’; ‘in class, you can't learn everything’ (SN3). Indeed, the NeoTree has been designed as an ‘aide memoire’ presenting guidelines as static information (eg, the Helping Babies Breathe algorithm) and addressing the problem of knowledge decay, which can occur after simulation training.22 Previous mHealth approaches for emergency care have included the digital presentation of guidelines23; however, the NeoTree’s interactive, emergency decision support screens go one step further by enhancing learning through pattern recognition after multiple uses. ‘After using the app regularly … you maybe sort of memorise, even in a situation without the app, you will remember’ (R1).

Theme 7: attitude

The importance of the HCW adopting a positive attitude towards the NeoTree was stressed particularly by student-nurses who may have experienced negativity from older staff; ‘old[er] staff should have a positive attitude’ (SN7) ‘we [student-nurses] should have a positive attitude towards it so that they [older staff] should be attracted’ (SN7).

Theme 8: usability

Usability appeared to be perceived differently by age. A more experienced HCW said, ‘it is easier for the girls than for old folks like me’ (N1). Student-nurses by contrast reported higher levels of usability, for example, describing the NeoTree as ‘a simple way of collecting information’ (SN6). One HCW suggested the NeoTree could be helpful in HCs stating, ‘most of the trainings they [HCWs in HCs] are attending they are not dealing with neonates.’

Quality improvement by the NeoTree

The qualitative feedback suggests the NeoTree could mitigate some major barriers to quality newborn care; for example, lack of nurse specialisation, lack of access to guidelines, poor training and frequent staff rotation (without choice) contributing to a general lack in confidence in handling newborns.

To our knowledge, the NeoTree is the only such intervention in development combining data collection, clinical decision support and education to improve quality of care for newborns. Decision-tree apps based on clinical algorithms have been trialled in Malawi for triage of sick children and in village clinics (D-tree International)24 and for disease-specific management of neonatal sepsis in Tanzania,25 but not for the assessment and care of newborns admitted to hospital.

Although the NeoTree was perceived by some to have delayed the provision of less emergent treatments such as antibiotics, this could reflect inadequate HCW understanding of the importance of emergency stabilisation before considering other treatments such as antibiotics. Arguably, 5–10 min of thorough assessment is necessary to improve quality of care overall and may improve efficiency, for example, by rationalising antibiotics and mitigating antibiotic overuse and resistance, a current major global challenge.26

The highest case fatality rates in the digital audit were for asphyxia and prematurity, consistent with other data sources.18 The potential beneficial effects of the NeoTree on prematurity and mild birth asphyxia are clear, for example, in guiding the management of respiratory complications. Benefits are less obvious for severe birth asphyxia and extreme prematurity as this would require greater resource from within the health facility itself and/or greater emphasis on prenatal and perinatal factors. The NeoTree is not yet configured to address maternal factors within the labour ward, but this is a focus for future work.

Data-driven quality improvement

There is a global call for prioritising immediate, meaningful data collection to measure newborn outcomes in low-income settings. The NeoTree allows accurate data collection by the HCWs themselves—thus allowing local and national investigation of newborn health, for example, the epidemiology of birth asphyxia. However, by going beyond data collection to provide bedside clinical decision support, education and training there is an immediate feedback loop to HCWs to encourage and ensure quality data entry. Replacing a paper-based admission process for a sick neonate with a touchscreen interface has emphasised patient safety, practical feasibility and clinical integration through multiple iterations (n=43 versions) towards a successful implementation strategy.

Generalisability

We have shown that NeoTree is usable in a busy district hospital with high admission to staff ratios. Our results are therefore generalisable to similar or better staffed facilities responding to concerns regarding lack of newborn training in these settings. The NeoTree is preconfigurable by a ward manager, according to equipment available (eg, CPAP) and skill mix in each facility and thus is generalisable to all levels of health facilities and adaptable within individual facilities as equipment and skill mix change. Our results demonstrate strong usability and acceptability among student-nurses, therefore are particularly generalisable to facilities where admissions are completed by student-nurses.

Strengths and limitations

The strengths of our study are severalfold. Our development strategy has used contemporary agile software and an editor platform which enabled iterative ‘build-measure-learn’ loops keeping the NeoTree in line with rapidly moving technology23 and allowing a truly integrated clinical and software design approach. We have upheld an important principle of mHealth: partnership and collaboration,23 through cocreation with HCWs, parents and guardians and collaboration with our clinical and digital partners including the Malawi MOH, working towards national QI priorities. Finally, we have taken a holistic integrational approach to QI in resource-poor settings including the recommended elements of systems thinking, stakeholders’ participation, accountability, evidence-based interventions and innovative evaluation.27

Limitations include that only four qualified nurses could attend the final FGD due to poor staff availability which may have affected the depth and richness of these data. The PQI score is not validated and does not adequately capture the WHO newborn quality of care initiatives; hence serves more for hypothesis generation than testing. The presence of the researcher in the clinical usability study may have biased NeoTree uptake and feedback.

Next steps

Future work includes ongoing software development, guided by behaviour change theory, including data transfer to local dash-boards and national systems. Clinical validation of the diagnostic algorithm for non-emergency newborn conditions (F2b, figure 2), and pilot testing of the resultant version, wherein the NeoTree completely replaces the paper admission form and a ‘NeoTree Ambassador’ provides technical support. Finally, we propose a large-scale clinical, cost-effectiveness and implementation evaluation of the NeoTree to reduce newborn case fatality rate.

Conclusion

The NeoTree app is a low-cost, acceptable, feasible and highly usable tool that could improve morbidity and mortality for sick newborns in LICs such as Malawi. The NeoTree has considerable potential to address a number of global and Malawian policy directives, including: ‘to strengthen and invest in care on the first day and week of life and to ensure that every newborn is counted.’9 Successful implementation is not straightforward nor primarily a software development task and studies of impact are needed to adopt mHealth strategies into current child health policy and programme.28 The coproductive iterative approach of our intervention development study, closely involving the end users, has produced an agile digital QI tool suitable for a larger impact study.