Health worker acceptability of an mHealth platform to facilitate the prevention of mother-to-child transmission of HIV in Tanzania.

OBJECTIVES: Health workers (HWs) are increasingly using mobile health (mHealth) technologies in low-resource settings. Understanding HW acceptability of mHealth is critical to increasing the scale of mHealth solutions. We examined pre- and post-pilot clinical knowledge and acceptability of a tablet-based platform, the Tanzania Health Information System (T-HIT), targeting HWs delivering prevention of mother-to-child transmission (PMTCT) of HIV services in seven health facilities in Misungwi District, Tanzania.
METHODS: We developed a survey based on the diffusion of innovation theory and administered it to 27 HWs before and after a 3-month pilot of T-HIT. Using a Wilcoxon signed-rank test, we analyzed changes in acceptability defined as attitudes towards and self-efficacy for system use comparing pre- and post-test assessment scores and changes in knowledge of clinical care. Using analysis of variance, we explored these changes, stratifying health facilities by level of care and by distance from the district hospital.
RESULTS: Post-pilot scores showed statistically significant improvement from pre-test for the total survey (Z = -2.67, p < 0.008) and for questions concerning system attitude (Z = -2.63, p < 0.008). HWs in hospitals and health centers exhibited a lower initial level of system acceptability in attitude than those in dispensaries and a significant improvement in overall mean acceptability over the pilot (95% CI 0.004-0.0187). HWs working more than 20 km from the hospital had a lower initial level of both system knowledge and acceptability than their less remote counterparts, but demonstrated larger improvements in knowledge and acceptability over time, although this change was not statistically significant.
CONCLUSIONS: The pilot demonstrates that HWs in PMTCT in Misungwi have a high acceptability of mHealth solutions. Using an mHealth solution can facilitate HW delivery of PMTCT care in rural and remote settings. Consideration of acceptability is important for fostering mHealth scale and program sustainability.

Background

The global proliferation of mobile information and communication technologies (ICT) and widespread network coverage in low-resource settings has created potential for mobile health interventions to improve healthcare delivery and health outcomes. The broad term mobile health, or mHealth, has been used to describe a number of innovative interventions using mobile phones, tablets, and hand-held computers with the aim to improve health outcomes, healthcare services, and health research into care delivery in low-resource settings.[1] Many of these mHealth programs have focused on the health worker (HW) as the point of intervention for new technology. HWs in remote clinical settings, such as health posts and dispensaries, are integral to improving access to care where infrastructure obstacles, like distance and difficulty of transportation, may otherwise prevent access to consistent healthcare.[1]

A 2016 review of HWs’ mHealth utilization by White et al. synthesized research and demonstrated the potential for the use of programs delivered via cell phone, smartphone or tablet in diverse low-resource settings targeting HWs,[1] whose education and training ranged from limited to advanced and who generally worked in more remote community locales. Other research on HWs’ use of technology for care and delivery by Källander et al. suggests a widespread acceptance of, and enthusiasm for, mHealth solutions.[2] Such solutions are appreciated for their potential to reach large numbers of individuals with low cost, easy to implement strategies for care delivery. Medhanyie et al. have demonstrated both the effectiveness of HWs in maternal care services and the utility of mHealth technologies in health service delivery by HWs in the East Africa setting.[3,4] As the body of evidence for mHealth interventions targeting HWs in low-resource settings grows and shows increasing promise for impact and potentially large return on investment, it is critical to now consider how best to facilitate HW mHealth adoption and system implementation to ensure these programs can be scaled and sustained over time.[5] Diffusion of innovations (DI) theory is a useful framework to use for considering scale and sustainability. System compatibility, observability, and relative advantage are all critical aspects of diffusion of an innovation.[6]

One specific element of compatibility identified in DI – acceptability – is a concept that has been applied in multiple health studies of mHealth innovations targeting HWs. While widely used, the measurement of the concept has been highly variable, ranging from “positive acceptance,”[7-14] to “improved morale” and “work satisfaction,”[15] to preference over a non-digital tool and “comfort.”[11,16,17] Other aspects of acceptability of mHealth solutions include improved logistics, decreased travel, and increased work speed.[9,11,15,16,18-22] Finally, the literature highlights a socio-cultural aspect to acceptability with the use of mobile devices in health care delivery. Social respect, “confidence” and “empowerment” are identified as important aspects of acceptability,[11,15-17,23] as are perceived improvement in job knowledge and capability enhancement – aspects of self-efficacy.[11,13,20,22,24]

Given the lack of consistency in measuring acceptability, the DI framework was utilized to guide the measures of acceptability for this study. Sekhon et al. stress the importance of theorizing the construct of acceptability,[25] and in the case of acceptability for use in mHealth interventions, the DI framework was considered to be a useful theoretical model. Based on this framework, we considered acceptability to incorporate: (1) how well the user likes the mHealth system, and (2) what the user identifies about the technology that is useful (together these comprise attitudes toward the system). Additionally, we considered the concept of self-efficacy, given the wide acceptance of this construct to support appropriate behaviors and measures to the extent of one’s belief in one’s own ability to complete tasks and reach goals.[26] Lastly, we considered knowledge of PMTCT care provision guidelines as an indicator that information in the form of reminders and prompts was effective in improving job knowledge.

The Tanzania Health Information Technology (T-HIT) system was developed by researchers as a pilot tablet-based, near real-time surveillance and health communication system for use in rural health facilities that did not have other mechanisms for capturing digital individual patient data.[27] It was designed as an integrated mHealth solution that linked geographic information systems (GIS), tablet-based surveillance, HW educational messages, patient reminders through text messaging, and a decision-maker dashboard summarizing testing into a single robust and responsive system to facilitate prevention of mother-to-child transmission (PMTCT) of HIV care within the context of antenatal, delivery, and postnatal patient visits.[28] HWs used T-HIT during a pilot study from March 1, 2015, through June 7, 2015, in Misungwi District, Mwanza Region, in northwestern Tanzania. Thomas et al. provide more descriptive information about the system design and function.[28]

Methods

Similar to many sub-Saharan countries, the Tanzanian Health System is organized in a pyramid structure with dispensaries as the entry point and then health centers, district hospitals, and regional hospitals offering respectively higher levels of care. The Misungwi District where this study took place has approximately 351,607 people,[29] is comprised of 20 wards, and is part of the Mwanza Region. The Misungwi District Hospital is located centrally and in the more urban center. Residents of the district typically go to the district hospital in the district where they reside when they are referred to higher level care, and as such distance from the district hospital is a reasonable proxy for remoteness in respect to access to higher level health care.

In Misungwi District in 2015 there were 42 total health facilities: 36 dispensaries, four health centers, and two hospitals, one of these a district hospital. Tablets were introduced for digital data entry to HWs at seven health facilities in Misungwi District: Misasi Health Centre, Mbarika Health Center, Mondo Dispensary, Nguge Dispensary, Mwawile Dispensary, Gambajiga Dispensary, and Misungwi District Hospital. In addition to data entry, health care reminders/prompts were built into the system (e.g., reminding women to come back for additional prenatal visits and encouragement to deliver at the hospital), as well as weekly educational texts to the HW about PMTCT care.

Feasibility was assessed in research conducted in 2014 and again immediately prior to this study by research assistants from the Catholic University of Health and Allied Sciences in Mwanza. Access to electricity, cell service and data connection was determined and the T-HIT system was designed as a tablet base system with simplicity of data entry to account for vision limitations in the HW workforce. A technical support number was provided and research assistants checked in with the HWs at periodic site visits. No tablets were lost over the course of the pilot and one was replaced for damage.

At the health centers and dispensaries, all HWs trained and working in PMTCT were invited to participate in the pilot study and all opted to do so. Eleven HWs from dispensaries and nine HWs from health centers were included in study. At the hospital, five of the nine HWs working in PMTCT were invited to and agreed to participate and two additionally opted in. The participating 27 HWs were introduced to T-HIT and the tablet technology through rigorous training in early February 2015. Preparation for the launch of the tablet intervention started with an intensive day of training, followed by a week of practice data entry, and concluded with a second intensive day-long training session.

With the goal of providing a pilot for a test of acceptability that could be used in many different mHealth contexts, we opted for a quantitative survey to assess acceptability, while acknowledging that aspects only available in qualitative assessment might be missed. The survey was pilot-tested in Mwanza District, a separate area from this study, with HWs at the regional health center and HWs in similar roles in provision of PMTCT. The survey was revised slightly based on feedback.

Participating individuals were given the same 19-question survey immediately after the initial training and again after the 3-month pilot using the T-HIT tablet application for PMTCT. The pre-test was given after the training, but before the pilot in order to minimize potential for basic anxiety about the tablet technology. Training was conducted by research assistants from the Catholic University of Health and Allied Sciences in Mwanza using a manual developed for T-HIT users. The training focused on the use of the interface, data entry and navigation of the system, but did not include educational messages or other training specific to PMTCT. Of the 27 HWs, 25 completed both a pre- and post-test for use in the paired analysis. Two HWs from the hospital were not available for the post-test and were not counted in the study.

Measures

Pre- and post-tests were developed to assess three domains of acceptability of the T-HIT system: compatibility, relative advantage, and observability (see Table 1).

Table 1.

Descriptive table of measures.

Measure	Question topics Included	Relevant DI construct
Attitudes towards the system	Comfort with system, confidence that the use of the T-HIT system will constitute an improvement, desire to use the T-HIT system	Compatibility
Self-efficacy	Ability to better perform job and improve patient outcomes	Relative advantage
Knowledge	Specific PMTCT care provision guidelines	Observability

Seven survey questions documented system attitude, six addressed self-efficacy, and six concerned knowledge – the knowledge questions were developed to document specific awareness of care delivery protocols. The pre-test was delivered at the conclusion of the training at the beginning of the pilot and the post-test was administered at the conclusion of the pilot. Questions/statements related to attitudes and self-efficacy offered three response options: yes (scored as “1”), somewhat/maybe (scored as “2”), and no (scored as “3”). Knowledge questions were scored as correct (yes, scored as “1”) or incorrect (no, scored as “3”), with one exception for the item “How many pre-natal visits should a pregnant mother ideally have before delivery?” with response options 1, 2, 3, or 4. This question was re-coded to 1 indicating a correct response if the participant indicated four prenatal visits as ideal and 3 if they indicated any other response. Each individual survey was given a total mean score for each domain by averaging the scores for all the questions, with a mean closer to 1 indicating greater acceptability of the system and a mean closer to 3 indicating lesser acceptability. Table 1 shows the question topics included in each domain and how they relate to the DI constructs that guided the development measures.

Analysis

Due to the non-parametric distribution of differences in pre- and post-survey means, the Wilcoxon signed-rank test was used to compare paired pre- and post-survey scores using SPSS®. A significance level of p < 0.05 was used to determine if there was a statistically significant difference between scores for each domain of questions and for the total survey before and after the pilot. Hypothesis tests were two-sided.

The level of care and distance from Misungwi Hospital were independently evaluated for changes in scores on attitudes, self-efficacy or knowledge between pre- and post-tests. Level of care was divided into two groups: (1) hospitals/health centers (Mbarika and Misasi Health Centers and Misungwi Hospital) and (2) dispensaries (Mondo, Mwawile, Nguge, and Gambajiga). Hospitals and health centers generally see more patients and offer a higher level of care (e.g., capacity for C-sections) compared to dispensaries. The distance from the district hospital was also considered by comparing sites less than 20 km from Misungwi District Hospital, including the hospital (Misasi, Misungwi, Mondo, and Nguge) against those located more than 20 km away (Gambajiga, Mbarika, and Mwawile). The pre- and post-test means were compared using analysis of variance Tukey’s Student’s test using SAS®.

Results

Table 2 is descriptive and shows an improvement in mean scores from pre- to post-test in attitudes towards T-HIT, self-efficacy for using the system, and knowledge of specific PMTCT care provision guidelines. Mean scores closer to 1 indicated a positive acceptance. Knowledge questions had scores only of 1 or 3 (right or wrong) and demonstrated less variability in scores.

Table 2.

Mean survey scores by acceptability measure.

N = 25	Pre-pilot mean score	Post-pilot mean score	Difference
Total	1.23	1.14	0.09
System attitude	1.09	1.03	0.06
Self-efficacy	1.19	1.07	0.12
Knowledge	1.5	1.4	0.1

A Wilcoxon signed-rank test indicated that post test scores showed significant improvement in the total survey (Z = −0.267, p < 0.008) and in questions related to system attitude (Z = −2.63, p < 0.008). For self-efficacy the improvement was not statistically significant and for knowledge the pre- and post-test means were frequently the same leading to a small N.

Level of care

Table 3 shows results from the Tukey’s range test comparing the survey means by level of care (hospitals/health centers to dispensaries) between pre- and post- pilot surveys. This test compares all 16 possible pairs of means and minimizes Type 1 error. The only finding of statistical significance was between pre- and post-test means for the total survey in HWs from hospitals/health centers. These HWs, surprisingly, had lower levels of acceptability in the area of attitude on the pre-tests than HWs from dispensaries.

Table 3.

Analysis of means comparison by level of care and pre- and post-pilot surveys.

	Difference between means	Simultaneous95% confidence limits
Test comparison (total survey)
PreHC – PreD	0.0282	−0.0699	0.12633
PostHC – PostD	0.0101	−0.0908	0.1110
PreHC − PostHC	0.0955	0.0038**	0.1871**
PreD – PostD	0.0773	−0.0295	0.18406
Test comparison (system attitude)
PreHC – PreD	−0.0043	−0.1178	0.1093
PostHC – PostD	0.0035	−0.1133	0.1203
PreHC − PostHC	0.0604	−0.0458	0.1665
PreD – PostD	0.0682	−0.0555	0.1918
Test comparison (self-efficacy)
PreHC – PreD	0.0581	−0.1381	0.2543
PostHC – PostD	−0.0316	−0.2335	0.1702
PreHC − PostHC	0.1488	−0.0345	0.3322
PreD – PostD	0.0591	−0.1545	0.2727
Test comparison (knowledge)
PreHC – PreD	0.0523	−0.1487	0.2532
PostHC – PostD	0.0686	−0.1439	0.2810
PreHC − PostHC	0.0964	−0.0914	0.2842
PreD – PostD	0.1127	−0.1115	0.33692

Pre indicates pre-pilot survey means, Post post-pilot survey means.

HC indicates hospitals and health centers, D dispensaries.

**Significant p < 0.05, Pr > F is the two-tailed significance probability.

In every means comparison post-test acceptability was higher than pre-test showing that use of the T-HIT system over the 3-month period resulted in improved acceptability. No statistically significant difference in acceptability between HWs from hospitals/health centers and those from dispensaries in either pre- or post-tests is promising in that it demonstrates that acceptability is fairly high among both groups despite differences in levels of care of the institutions where they work.

Distance

Table 4 shows the Tukey’s range test comparing the survey means by distance and pre- and post-pilot surveys. The reasoning for the use of this test is the same as above. Except for system attitude, which was equal, all the means were higher in the pre-pilot test in the health facilities that were considered “far” – greater than 20 km from the district hospital – indicating lower initial “acceptability” of the T-HIT system.

Table 4.

Analysis of means comparison by distance and pre- and post-pilot surveys.

	Difference between means	Simultaneous95% confidence limits
Test comparison (total survey)
PreF – PreN	0.0611	−0.0389	0.1611
PostF – PostN	0.0173	−0.0878	0.1223
PreF − PostF	0.1174	−0.0017	0.2364
PreN – PostN	0.0735	−0.0093	0.1564
Test comparison (system attitude)
PreF – PreN	0.0361	−0.0814	0.1536
PostF – PostN	−0.0138	−0.1371	0.1096
PreF − PostF	0.0971	−0.0427	0.2369
PreN – PostN	0.0472	−0.0501	0.1445
Test comparison (self-efficacy)
PreF – PreN	0.00	−0.2042	0.2042
PostF – PostN	−0.0197	−0.2443	0.2050
PreF − PostF	0.1359	−0.1162	0.3880
PreN – PostN	0.1162	−0.0530	0.2854
Test comparison (knowledge)
PreF – PreN	0.1778	−0.0188	0.3743
PostF – PostN	0.0735	−0.1329	0.2799
PreF − PostF	0.1722	−0.0617	0.4061
PreN – PostN	0.0680	−0.0948	0.2308

Pre indicates pre-pilot survey means, Post post-pilot survey means.

N indicates near (less than 20 km from the district hospital), F far (more than 20 km from the district hospital).

Across all categories, HWs in facilities more than 20 km from the hospital exhibited descriptively greater improvement between pre- and post-pilot tests for the total survey and for the questions relating to HWs’ knowledge. Although the findings were not statistically significant, this trend is promising and indicates HWs may improve knowledge when educational reminders and prompts are embedded within a mHealth solution and the use of technology is viewed as more acceptable over time with use.

Discussion

In this study, we evaluated acceptance of a tablet-based system used for the PMTCT of HIV with a 19-question survey administered before and after a 3-month pilot. The results showed significantly improved acceptability overall and for system attitude after use of the T-HIT system, as well as a trend of improvement in each of the other two sub-categories of self-efficacy and knowledge among HWs. Secondarily, we analyzed whether level of care or distance from the district hospital yielded variable changes in mean survey scores. Despite a significant difference in pre-pilot means versus post-pilot means in one category, comparing post-pilot results showed no significant differences between facility level of care or distance in affecting acceptability, system attitude, self-efficacy, or knowledge after a 3-month trial. When considering distance from the district hospital, the results suggest some evidence for the potential for the system to help remote HW improve their knowledge of PMTCT protocols.

Chang et al. found that, despite a growing interest by researchers in mHealth,[15] there is “limited evidence” on the acceptability and impact of mHealth interventions. Our study is unique in separating out acceptability as a primary measure for this paper and in comparing the same survey made up of questions about acceptability and knowledge given to the same health care workers before and after a pilot session. Acceptability is often neglected, not thoroughly evaluated and/or inconsistently measured, although it is crucial to sustainability and scalability of mHealth interventions. Agarwal et al. demonstrate in a systematic review that adoption of technology for healthcare delivery in low-resource settings requires a “multidimensional evaluation approach” that includes examining the effects of a program on HWs.[30]

We found the DI theoretical framework useful for measuring and considering acceptability. For future work it may also be relevant to include a more focused perspective that is specific to healthcare delivery, inasmuch as there are issues and factors unique to healthcare that warrant attention in the process of diffusion. To this end, we suggest Chib et al.’s ICT for healthcare development model,[20] which considers opportunity production, capabilities enhancement, social enabling, knowledge generation, and barriers as providing additional indicators to consider in measuring acceptability.

Limitations

This work was based on findings from a small sample of HWs from seven facilities, and thus cannot be generalized to HWs throughout Tanzania or East Africa. Due to the small number of HWs serving populations from the dispensaries and health centers and high interest in participation, it was not possible in the scope of the study to have a control group. Although the survey was tested and refined prior to this study, it was not validated. Additionally, the pilot study was conducted in a single 3-month period and data may not reflect potential variations in acceptability of the T-HIT system if it were implemented over a longer period of time. Statistically, the small sample sizes when analyzing HWs separated by level of care and distance categories, limited the power of the study for these research questions. Clustering was not accounted for due to the ratio of the sample size to the number of facilities, which was considered to be too small for analysis.[31]

Conclusion

This paper provides an important step in theorizing the construct of acceptability using the DI model and provides evidence of key acceptability domains by HWs of the T-HIT tablet-based system for PMTCT of HIV. The T-HIT pilot intervention was designed with the intention of transitioning to a larger scale efficacy trial for improvement in PMTCT and the potential scale up of the model in Tanzania and in other East African settings. Acceptability of technology used by HWs is a critical factor in sustainability and scalability of any mHealth system. Measuring and establishing acceptability is a critical part of evaluation from the inception of any mHealth pilot project that has ambitions to scale and sustainability.