Perceptions of a mobile health intervention for Community Case Management in Malawi: Opportunities and challenges for Health Surveillance Assistants in a community setting.

Background: Improved availability of mobile phones in low- and middle-income countries (LMICs) offer an opportunity to improve delivery of Community Case Management (CCM). Despite enthusiasm for introducing mHealth into healthcare across LMICs, end-user attitudes towards mHealth solutions for CCM are limited. We aimed to explore Health Surveillance Assistants' (HSAs) perceptions of the Supporting LIFE electronic CCM Application (SL eCCM App) and their experiences incorporating it as part of their clinical practice.
Methods: This exploratory qualitative study was part of a mixed methods feasibility study investigating whether children under-5 presenting to village clinics could be followed-up to collect patient outcome data. The convenience sample of 12 HSAs enrolled into the feasibility study participated in semi-structured interviews, which were conducted at village clinics after HSAs had field-tested the SL eCCM App over a 10-day period. Interviews explored HSAs perceptions of the SL eCCM App and their experiences in using the App in addition to paper CCM to assess and treat acutely unwell children. Open coding was used to label emerging concepts, which were iteratively defined and developed into six key themes.
Results: HSAs' perceived enhanced clinical decision-making, quality of CCM delivery, and work efficiency as opportunities associated with using the SL eCCM App. HSAs believed the inability to retrieve patient records,, cumbersome duplicate assessments/data entry study procedures, and inconsistencies between the SL eCCM App and paper-based CCM guidelines as challenges to implementation. Adding features to the App, such as, permitting communication between colleagues/supervisors, drug stock-out reporting, and community assessments, were identified as potentially supporting HSAs' many roles in the community.
Conclusion: This study identified opportunities and challenges associated with using the SL eCCM App in Malawi. This information can be used to inform future development and evaluation of the SL eCCM App, and similar mHealth solutions for CCM in Malawi and other developing countries.

Introduction

Increased availability of low-cost mobile solutions, coupled with expansion of cellular and data networks in low- and middle-income countries (LMICs)1 , offer an opportunity to improve delivery of paper-based Integrated Management of Childhood Illness (IMCI) and Community Case Management (CCM) strategies2. Both IMCI and CCM represent paper-based clinical decision aids that facilitate identification and management of mild, moderate and severe illness among children under-5 years3. Delivered by frontline community health workers (CHWs), constraints such as poor training opportunities and supervision4 have prevented these strategies from achieving the desired geographic coverage and impact5.

Despite proliferation of mobile health (mHealth) solutions across sub-Saharan Africa, there are few published studies describing end-user attitudes towards mobile solutions of either of these major childhood survival strategies6–9. Existing literature suggests perceived benefits amongst CHWs include beliefs that digitised versions of these strategies encourage better adherence to the guidelines, are faster and simpler to use9, and could ease workload burdens7. However, there may also be potential challenges to use, such as difficulty typing in data8, incomplete cellular network coverage and low information technology (IT) literacy7. Since CCM is deployed by the largest cadre of CHW10, known as Health Surveillance Assistants (HSAs) in Malawi, determining the future role mobile solutions might have in the delivery of CCM first requires a clearer understanding of the opportunities and challenges associated with use. To the authors' knowledge, this has yet to be explored in Malawi.

As part of a feasibility study we sought to explore HSAs' perceptions and experiences of a purpose-developed mHealth decision support application for CCM. The central objectives were to improve understanding of the opportunities and challenges associated with using the Supporting LIFE (Low cost Intervention For disease control) electronic Community Case Management Application (SL eCCM App), to inform its development and evaluation in a clinical trial11. We also hoped our findings could be used to guide future evaluations of mHealth solutions for CCM in other LMICs. This paper presents the qualitative findings from the feasibility study.

Methods

Setting and participants

A convenience sample of 12 HSAs from village clinics (1 HSA per clinic) accessible to the study team in Mzimba North, Northern Malawi, were recruited in the feasibility study between July and September 2015, and all participated in semi-structured interviews. Our sample size was determined a priori by extant methodological research conducted in sub- Saharan Africa, that identified that data saturation had occurred within the first twelve semi-structured interviews12. As we were interested in understanding HSAs' perceptions and experiences of the SL eCCM App to inform subsequent software development of the App and design of a larger clinical trial, we believed our sample size would be adequate to satisfy our research objectives. HSAs were identified from village clinics listed on the national District Health Information System 2 (DHIS 2) database. To be eligible to participate, HSAs were required to attend a full-day training workshop to familiarise participants with the SL eCCM App, smartphone (i.e. HTC Desire 526G), and study procedures; those who declined attendance were excluded from participation. HSAs were initially approached in the community by a member of the study team to determine eligibility, introduce the study, and gauge interest in participation. Within 2-weeks of initial contact, HSAs were re-approached (via telephone or in-person) and were given verbal and written information about the study, either in English (depending on fluency), or Tumbuka. Written informed consent was obtained from each HSA prior to commencing the training workshop.

mHealth Application

The SL eCCM App (full details of the App's development is provided elsewhere)(13) replicated the logic of existing paper-based CCM guidelines adapted for use in Malawi (i.e. Sick Child Form) to manage children aged ≥2 months up to 5 years with acute illness (Figure 1), and captured patient details (including name, age and gender), presence and duration of ‘sick child’, ‘danger’ signs, and vital sign measurements (i.e. breathing rate), to categorise illness severity. The SL eCCM App included a tap screen breath count feature, using a 60-second countdown timer (imitating the stopwatch currently used as part of standard care), during which time the user taps the + bar with every observed inhalation (Figure 2). HSAs were given the option to use this feature to measure breathing rate during field-testing. Validation was switched on, forcing HSAs to populate mandatory fields with data within clinically valid parameters. Patient data could only be entered into the App. Therefore, HSAs were unable to retrieve previously entered patient records to review. The SL eCCM App was tested in Malawi on an ongoing basis from 2014 onwards, and updated versions were released iteratively over an 18-month period. Feedback was solicited from a separate group of HSAs at each stage to guide development. Version 5.0 was tested during the feasibility study.

Figure 1

Sample SL eCCM App Assessment screen shots

Figure 2

Sample SL eCCM App Breath Counter screen shot

Study Procedures

A total of three training workshops, each with a maximum of four HSAs were conducted. Within a week of completing training, HSAs were issued with the SL eCCM App to field test. The SL eCCM App was used in addition to paper CCM (standard care) over a 10-day period to assess and treat children ≥2 months and <5 years presenting consecutively to village clinics with any acute illness. HSAs typically operate village clinics a maximum of 3 days per week14. Due to human resource constraints we introduced the App to village clinics in phases over the course of two weeks. The 10-day time period was chosen to ensure HSAs were exposed to the SL eCCM App for at least 3-days. Each child was first assessed using standard care then assessed immediately afterward using the SL eCCM App. Treatment was issued according to recommendations of standard care. As the SL eCCM App was not clinically validated, double assessment and data entry into both the village clinic register and App were undertaken to maintain standards of care. Since double assessment/data entry was anticipated to lengthen consultations, participants could discontinue use of the SL eCCM App at any time, and were encouraged to do so if standard care deemed a child to be severely unwell. HSAs were advised to sync the data entered into the App with the secure cloud-based study database at the end of each day, or as soon as they had internet/data connectivity. Data was securely stored locally on the mobile device until records were successfully synced via the SL eCCM app, when they were automatically deleted from the mobile device.

Data Collection

An exploratory approach was chosen due to limited prior knowledge of the phenomenon, to gain insights into the opportunities and challenges of using the SL eCCM App as part of HSAs' practice, for later investigation in a clinical trial. Semi-structured interviews were conducted in private with all 12 participating HSAs at their village clinic within a week of field-testing. Topic guides consisting of open-ended questions with prompts were used to facilitate discussions. One-to-one interviews were chosen to encourage an open dialogue with the interviewer.. Topic guides developed by the research team were derived from existing research exploring end-user perceptions as well as the broader literature related to implementing mobile solutions of IMCI and CCM (Appendix 1)9,16, and were translated into Tumbuka. Topic guides were not pilot tested, but were reviewed for appropriateness by members of the Malawi-based research team. Interviews were conducted in Tumbuka, transcribed and translated into English by two translators and then proof read by one author (GBC) who is a lecturer and qualified clinician in Malawi, and who also conducted the interviews. Interviews were digitally recorded and lasted between 20 and 30 minutes. This nested study was approved by the University of Washington's Human Subjects Division (49117) and the College of Medicine Research Ethics Committee in Malawi (P.03/15/1701).

Data Analysis

Data analysis began once interviews had been completed and transcripts translated into English, checked for accuracy, and identifiers removed. Open coding was used to examine the meaning in each word or groups of words, which were labelled (or coded) forming seed themes. Seed themes were identified inductively and modified in an iterative cycle, before finally being grouped into key themes17. Initial labelling of the data and identification of seed themes was undertaken by one author (GBC), reviewed and refined by another (CH) and then discussed collaboratively with a third author (YOC). This process continued iteratively until a consensus was reached regarding the final themes. MS Excel was used to organise the data and facilitate analysis.

Results

Six key themes were identified: 1) overall experiences of the SL eCCM App, 2) experiences and perceptions of breath rate counter, 3) similarity of the SL eCCM App to standard care, 4) impact on perceived quality of CCM delivery and 5) impact of double assessment and data entry, and 6) other uses of mobile phones.

Overall experiences of SL eCCM App

HSAs' overall experiences of the SL eCCM App were positive. Most reported that they enjoyed using it as part of the management of sick children, and rarely mentioned technical issues preventing them from using the App:

“Although it's kind of new to us but it's working quite alright.” (HSA3)

They reflected that the few problems that did occur were user issues related to HSAs inexperience in using more advanced mobile technology:

“I think I tampered with the settings. You know this phone is advanced we are not used to having it.” (HSA8)

They liked that the App automatically guided them through the CCM assessment and provided treatment recommendations, removing some of the pressure to remember the next steps in the CCM sequence:

“The phone is more direct and it even makes the decision of what you are supposed to do after entering the data.” (HSA3)

HSAs viewed the availability of necessary clinical information contained within the App as facilitating work efficiency:

“You just press on the phone. Everything is already there. While as in the register you have to write down everything, name, problem and it takes time.” (HSA8)

“The phone makes the work load easier…it helps us work faster…” (HSA5)

However, some mentioned they had encountered problems or experienced uncertainty and required additional support from the study team to use some of the App's features. Notably, they were consistently unsure about how to sync records to the database:

“…problems were there especially on how to send data. But I was assisted accordingly. I was told to take a paper and write down steps to follow in case I got stuck again while they [the study team] were gone. And I followed those steps and didn't find any problems.” (HSA8)

“Just that at first, on sending data, I thought that I was doing it wrong. So I ask […] and she assured me that I was doing it correctly.” (HSA11)

HSAs also expressed concerns about the future implications of data not being retained within the App once it is uploaded onto the study database, identifying that if use of the village clinic register was to be stopped, lack of access to patients' records would prevent them from reviewing, aggregating and reporting illness case rates to the district health office:

“But I still can't tell the number of children I have assessed from the phone because once I have synced the data, then that's the end of it… I think it would be better if the phone was able to keep records for us to see.” (HSA6)

“The phone should be able to keep the data after we have sent it, in case we will stop using the registers […] it means we will not have any data at the end of the month.” (HSA5)

HSAs also mentioned this as a perceived concern amongst parents/caregivers presenting to village clinics with a sick child during the study:

“Some caregivers were asking about their information. They were asking that if we stop using the register, and if they come with a patient and we don't write anywhere and we just use the phone, is the phone able to keep records?” (HSA 2)

Experiences and perceptions of breath count feature

Many HSAs stated that they did not use the breath count feature within the SL eCCM App:

“No, I can't lie, I never used it. I was using my timer instead…” (HSA6)

Only a small number of HSAs attempted to use this feature, and only reported using it for a short period at the beginning of field testing:

“I used it at first, just to see how it works… after I had used it I asked my friends if it's ok to just use the timer and they said it's the same as using the phone.” (HSA2)

Reasons why HSAs failed to use the breath count feature to measure a child's breathing rate (and relied instead upon their usual method) included viewing the feature as complex, and forgetting it was available to use:

“The tapping was a bit confusing to me so I just used the timer provided [to] enter the breaths.” (HSA6)

“I can say I used to forget since [I] am used with this one, so I didn't remember to use the phone, unfortunately, when it came to counting breaths per minute…” (HSA2)

Similarity of the SL eCCM App logic to standard care

HSAs held mixed views regarding the similarity of the logic between the SL eCCM App and standard care. Some HSAs believed the App replicated paper-based CCM recommendations:

“There is no problem because the information that is in the phone is the same as the one in the register.” (HSA4)

However, others reported that the SL eCCM App did not replicate some of the standard treatment recommendations in Malawi. For example, they noted that the antibiotics listed in the SL eCCM App differed to the antibiotics they were currently expected to administer for patients presenting with cough and fast breathing:

“There's a slight difference. Because when the child is having… fast breathing which is pneumonia we are supposed to give him amoxicillin which is recommended in Malawi. While in the phone, there is cotrimoxazole.” (HSA2)

Some HSAs noted that some items of CCM were missing from the App altogether. In particular, malaria rapid diagnostic testing (mRDT) for children presenting with fever was not integrated into the SL eCCM App and indicated that the SL eCCM App reflected previous (and now outdated) CCM guidelines for Malawi, where anti-malarial medication was presumptively administered for children presenting with fever:

“There is no mRDT system in the phone they should include the new system of how to do mRDT. When it's negative it means no treatment is given, we just give advice. This phone uses the old system where we used to give treatment to any child who has got fever.” (HSA12)

One other HSA noted that the App did not have the capacity to record children's immunization history:

“Just that there is nothing about immunization in the phone.” (HSA8)

Impact on perceived quality of CCM delivery

Improved quality of delivery of the CCM guidelines was highlighted as a perceived benefit of the SL eCCM App. HSAs viewed the App as preventing data entry errors and treatment reporting mistakes:

“Let's say a child has diarrhoea, the phone tells you the exact treatment of it unlike in the register where you have to choose and then tick. Sometimes it may happen that you are too tired and you may end up ticking wrong treatment. But the phone automatically gives you the treatment.” (HSA11)

They attributed this to the fact that the App prevented them from missing required fields necessary for the logic to determine illness classification and treatment, as well as automatically prompting them on what to do next, which they perceived as an advantage over standard care:

“In fact, we are very careless with the register than the phones. So I think if we could be using the phones all the time, it could be better.” (HSA9)

“…Sometimes you may make a mistake [using the village clinic register] but you cannot make such mistakes when using the phone (HSA3)”

HSAs also believed using the App encourage them to give more thorough explanations regarding the assessments they were conducting, which they perceived parents/caregivers viewed as improving the accuracy of clinical management:

“Another good thing about the phone is that when a patient arrives, we explain to the caregiver how the phone works and we take her through the steps up to treatment. This makes them satisfied that they are indeed getting accurate treatment instead of just telling them [the treatment] like we used to do in the past.” (HSA7)

Conversely, other HSAs experienced difficulties with the lack of flexibility when entering clinical data into the SL eCCM App, which they reported as cumbersome as it required them to return to the missing field, which sometimes resulted in complete loss of the record, and repeat assessment with the App:

“When it comes to submit patient records, if you want to submit that patient's record, you go to treatment side, and the phone will tell you that you have not completed the data.” (HSA2)

“However, it confused me somehow, in that if you miss one step, let's say you haven't submitted, and you press another button that is not in line with that, it meant that you were supposed to start all over again with that patient.” (HSA4)

Impact of double assessment and data entry

Some HSAs believed the simplicity of SL eCCM App could facilitate faster data capture of sick child visits compared to standard manual procedures:

“It's faster because you just ask the caregiver and then you simply touch on the phone, without writing…while when using register you are supposed to write down the names. When you use a phone though, you just tick. You tick, and then the information is recorded.” (HSA2)

However, the majority of HSAs expressed doubts about the beneficial impact of double assessment and data entry on sick child visits, which they viewed as time-consuming, extending visits, adding to their workload, and were worried about being perceived by parents as inefficient:

“It takes a lot of time on one child. When you are done on the sick child recording form, you have to come to the phone, that's when you get child.” (HSA1)

“I wish all the information that is in the register was in the phone too so that we just use the phone instead of using them both… because it may happen that 20 children have come that day which means it will take you a lot of time to assess all of them.” (HSA5)

In contrast, other HSAs believed the double assessment/ data collection procedures could be managed by capitalising on available time:

“Yes it's possible. Let's say you are doing RDT and you have written in the village clinic register and as you wait for 20 minutes for the results, you enter the data in the phone. There is no problem.” (HSA3)

And that without access to entered patient records, it would be necessary to use both modalities so HSAs could refer to previous patients' visits to ensure continuity of care:

“I think it's necessary to do so because after you have closed the clinic in the evening and you have sent the data, it doesn't remain in the phone… but you still have your back up data in the village register. You have all the records that a certain child came on such a day and was referred to the hospital with such a problem and whether the child was admitted or not and so on. You know all this because you have the records in the village clinic register. So it's important to use both of them.” (HSA8)

Other uses of mobile phones

HSAs were enthusiastic when considering the potential opportunities for smartphone technology. They identified scenarios where mHealth could be used to further support their many duties as a HSA, in particular, communicating health information including drug stock outs with colleagues/ supervisors as scope for future use:

“It can help me communicate with my colleagues… or to get information from my colleagues. That way it can help me besides CCM.” (HSA1)

“Because I walk around with it so if I want to communicate something with you or I have a shortage of drugs, I can simply use the phone wherever I am. But I cannot walk around with the register.” (HSA5)

HSAs believed the functionality and portability of the phone could facilitate the recording of community sanitation assessments currently recorded in the village clinic register, which they are required to carry around with them:

“We check things like the environment, sanitation, so if the phone has a camera, you can take pictures of such things and use them in your report.” (HSA4)

“Apart from that, we keep data of catchment population, number of household and so on. But we keep this information in our notebooks. But it would be easier if we kept this data in the phone so that we know that exact number of people or under 5 children in my catchment.” (HSA6)

Discussion

Main findings

Overall, HSAs' perceptions and experiences of the SL eCCM App were positive. They liked that the App automatically reminded them of the next steps in the assessment and treatment sequence, which they believed could improve work efficiency and overall quality of CCM delivery. There were mixed views as to whether inbuilt validation rules which forced relevant fields to be completed, contributed to this belief. For HSAs who were unconvinced by the helpfulness of the validation prompts, it was unclear as to whether they were frustrated because they had made a data entry error or omission and were directed back to complete the field(s), or the inflexibility of the App's decision rule. Whether due to the novelty of the SL eCCM App or aspects of the interface encouraging user engagement, HSAs reported verbalizing each of the steps of CCM, which they thought enhanced parents/caregivers' perceptions of the quality of care they were receiving. HSAs' viewed duplicate assessment and data entry procedures using the SL eCCM App and paper CCM as possible but time-consuming. Impediments to use of the SL eCCM App (either in conjunction with paper-CCM, or as a standalone device) included the inability to retrieve patient records to review, and inconsistencies in the logic/treatment recommendations between the SL eCCM App and paper CCM.

Our findings share commonalities with existing research. Previous research found that similar mHealth solutions for IMCI and Community Case Management of malaria (CCMm) were perceived to enhance clinical decision-making amongst CHWs. This was also attributed to the accessibility of clinical information within such devices which reduced the reliance on memory, thus simplifying their work8,9,18. These solutions were believed to be viewed more favourably amongst parents/caregivers8, and by CHWs as potentially beneficial to connect with colleagues/supervisors for professional support (including problem-solving), and to communicate drug stock outs6,7, which is also consistent with our findings. HSAs also expressed similar concerns about imposed double data entry procedures in a programmatic report of an evaluation of a mobile solution of CCM in Malawi19. Different interfaces, system architectures and features amongst existing mHealth solutions for CCM/IMCI will have16,20 each different strengths and weaknesses,influencing end-user experiences. However, the consistent belief in the benefits of the SL eCCM App and other similar solutions, suggests a pervasive need for methods to reduce pressures on frontline CHWs from the multiple tasks they are expected to perform. Contrary to what we observed, introduction of a software-based breath rate feature under IMCI was perceived in one study as easy to use. CHW were confident after 1–2 days training21 or experience with patients that they would use it regularly, indicating that our population may have benefited from a longer training period dedicated to use of this feature and/or direct field support.

Strengths and Limitations

To the authors' knowledge this is the first published study exploring end-user perceptions of a mobile solution of CCM in Malawi. By pursuing an exploratory approach, we gained a variety of insights into some of the challenges and opportunities associated with using a mHealth app for CCM. These insights can be used to guide future evaluations of similar solutions in Malawi, and other LMICs.

However, our study has several limitations. Our non-probabilistic convenience sampling strategy may have introduced some bias. By recruiting participants who were most accessible to the study team, we may not have captured issues associated with using the App in more remote locations. Furthermore, as we did not collect information about participants' characteristics (e.g. age, education, prior exposure to smartphones etc.), we have limited insight into the homogeneity/heterogeneity of our sample. Despite this, we observed no new themes emerging upon review of the final transcript, suggesting our sample was homogenous and/or our research question narrow enough to achieve saturation. Additionally, non-probabilistic sampling is non-representative, thus it is questionable how transferable our findings are. HSAs have multiple responsibilities in communities and few operate village clinics more than 3-days per week. This may have limited exposure to the variability of issues experienced had they been issued the SL eCCM App for longer than 10 days. Interviews were conducted by a Malawian lecturer/clinician. Given that participants may have been aware the interviewer was part of a research programme, HSAs might have reported what they believed we wanted to hear. We did not solicit an independent review of translated topic guides by a researcher with qualitative experience in Malawi, and so there is a risk questions may have become leading/judgemental during translation. Finally, some authors who undertook data analysis, were also involved with the development of the SL eCCM App (YOC, CH), meaning data interpretation could have been positively influenced by underlying favourable beliefs. To try to minimize this bias we were mindful to thoroughly present negative as well as positive perceptions of the App, where they existed, and transcripts were reviewed a posteriori by a fourth author (VH) to check for agreement and balance in interpretation.

Implications for future research

We only explored perception of the SL eCCM App amongst a single group of stakeholders. Since the decision to adopt solutions is contingent on national-level support, further research exploring the acceptability of similar types of mHealth solutions for CCM should include: parents/caregivers, HSA supervisors, district health officials, and decision-makers within Malawi's Ministry of Health. mHealth solutions for IMCI-related algorithms have been evaluated as a standalone device in other LMICs22. Therefore, duplicating assessment and data entry procedures may indicate a lack of interoperability of the solutions being tested with health information systems in Malawi, or insufficient readiness of local infrastructure to accommodate integration of mHealth, which may limit future scalability of interventions. This warrants further attention. Mobile applications with built-in decision-support restrict autonomy by automating decision-making. Therefore it would be interesting to examine their long-term impact on health workers' retention and application of clinical knowledge without support from the App, and relationship with patients. HSAs' concerns regarding inconsistencies in the decision rule between the App and paper CCM, should be addressed prior to further evaluation of the SL eCCM App. Finally, acceptability of the App and similar technologies should be evaluated after an extended period of use.

Conclusion

Perceived opportunities associated with use included supporting decision-making, improved efficiency and quality of CCM delivery. Challenges primarily related to cumbersome duplicate assessments, data entry procedures, and logic discrepancies between the SL eCCM App and standard care. Integrating features promoting communication of health information with other HSAs or supervisors and facilitating drug stock-out reporting, were viewed as potentially supporting HSAs additional duties in the community. Prior to further deployment and evaluation, challenges related to delivery in this study should be addressed, and considered by other organisations seeking to develop and evaluate similar solutions of CCM.