mHealth-community health worker telemedicine intervention for surgical site infection diagnosis: a prospective study among women delivering via caesarean section in rural Rwanda.

BACKGROUND: Surgical site infections (SSIs) cause a significant global public health burden in low and middle-income countries. Most SSIs develop after patient discharge and may go undetected. We assessed the feasibility and diagnostic accuracy of an mHealth-community health worker (CHW) home-based telemedicine intervention to diagnose SSIs in women who delivered via caesarean section in rural Rwanda.
METHODS: This prospective cohort study included women who underwent a caesarean section at Kirehe District Hospital between September 2019 and March 2020. At postoperative day 10 (±3 days), a trained CHW visited the woman at home, provided wound care and transmitted a photo of the wound to a remote general practitioner (GP) via WhatsApp. The GP reviewed the photo and made an SSI diagnosis. The next day, the woman returned to the hospital for physical examination by an independent GP, whose SSI diagnosis was considered the gold standard for our analysis. We describe the intervention process indicators and report the sensitivity and specificity of the telemedicine-based diagnosis.
RESULTS: Of 787 women included in the study, 91.4% (n=719) were located at their home by the CHW and all of them (n=719, 100%) accepted the intervention. The full intervention was completed, including receipt of GP telemedicine diagnosis within 1 hour, for 79.0% (n=623). The GPs diagnosed 30 SSIs (4.2%) through telemedicine and 38 SSIs (5.4%) through physical examination. The telemedicine sensitivity was 36.8% and specificity was 97.6%. The negative predictive value was 96.4%.
CONCLUSIONS: Implementation of an mHealth-CHW home-based intervention in rural Rwanda and similar settings is feasible. Patients' acceptance of the intervention was key to its success. The telemedicine-based SSI diagnosis had a high negative predictive value but a low sensitivity. Further studies must explore strategies to improve accuracy, such as accompanying wound images with clinical data or developing algorithms using machine learning. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Telemedicine applications have been effective in the follow-up of surgical patients in high-income countries.

Telemedicine interventions have been implemented successfully in low and middle-income countries, but their use for postsurgical patients’ follow-up in those countries, particularly in rural Africa, has not been explored.

In a prospective study of 787 women, 719 were located at their home and all accepted the telemedicine intervention implemented by trained community health workers with remote support of a general practitioner.

The full telemedicine intervention including receipt of the surgical site infection (SSI) diagnosis by a general practitioner within 1 hour was completed for nearly four out of five women.

While the telemedicine-based specificity for diagnosis of SSI was high, its sensitivity was low.

Home-based follow-up of surgical patients to detect SSI using telemedicine applications is feasible in rural Africa and can be leveraged to improve access to care for these patients.

Telemedicine-based follow-up can help reduce unnecessary clinic visits by surgical patients through remotely ruling out SSI-negative cases, particularly in this context of an overwhelmed health system and the COVID-19 pandemic.

Further studies are needed to improve telemedicine sensitivity through accompanying wound photos with clinical data or through computer-assisted algorithms.

Introduction

Caesarean section (c-section) is the most commonly performed surgical procedure in sub-Saharan Africa (SSA) and accounts for the majority of the surgical volume at district hospitals.1 2 C-section rates vary widely in the region, ranging from 1.0% to 41.9% but averaging 12.4% of total births.3 Thus, concerns about limited access to and potential overuse of c-sections remain. While access to c-sections is critical for reducing maternal and neonatal mortality, increases in c-section rate correlate with rising cases of surgical site infections (SSIs).4 5

In SSA, SSI rates after c-section range from 7.3% to 48.2%.3 6 SSIs are also significant contributors to maternal mortality7 and result in high costs to patients, families and healthcare facilities.8 Furthermore, SSI rates are likely underestimated because most SSIs develop after patient discharge from the hospital.9 Due to the lack of active surveillance and standard postoperative follow-up, these infections may go undetected and thus under-reported.10 11

Various factors prevent women from returning to follow-up care after discharge. These include geographical and financial barriers,12 13 low household income and education level, and dependence of women on their husbands.14 15 However, developments in technology may help reduce these barriers. In high-income countries, mobile health (mHealth) applications have helped decentralise postoperative follow-up care16–18 and minimise geographical and financial hurdles.19 20 Telemedicine, which is the use of telephone-based follow-up, has been proven to be especially effective for simple abdominal surgeries,21 including SSI surveillance after obstetric patient discharge.22 However, challenges in implementing telemedicine in rural Africa have been documented, including insufficient technical and financial support, inadequate clinical adoption, poor staffing and limited or unstable communication infrastructure.23

Similar to the rest of SSA, Rwanda has a high postdischarge SSI rate, previously estimated at 10.9%.24 Women who have delivered via c-section have reported that geographical distances and the related cost of transportation limit access to postoperative care,25 and these factors are associated with worse outcomes for neonates,26 27 and the development of SSIs.24 Recent advances in telecommunication infrastructure and increased access to mobile phones have created opportunities to implement mHealth interventions in Rwanda. According to national statistics, in 2017, 71% of households reported owning a cell phone and 99% had access to mobile networks.28 Another resource to leverage in Rwanda is the widely used and highly effective community health worker (CHW) network that has played a critical role in maternal and child care during the prenatal period and following vaginal delivery.29 Current CHW training, however, does not include post-c-section follow-up care.

Overall, there is conducive environment to implementing mHealth strategies to support CHW-based postoperative follow-up in Rwanda and the region more broadly. This project assesses the feasibility and accuracy of an mHealth-CHW intervention that uses telemedicine to diagnose SSI in rural Rwanda. If proved effective, this intervention would reduce financial and geographical barriers to postdischarge follow-up care and could improve SSI surveillance.

Methods

Study design and population

This prospective cohort study included all women who were permanent residents in Kirehe District, underwent c-section surgery at Kirehe District Hospital (KDH) between 22 September 2019 and 16 March 2020 and were discharged before postoperative day 10 (POD10). Patients from the Mahama Refugee Camp, those who were still hospitalised on POD10 or those who had been readmitted by the time of their scheduled study follow-up visits were excluded.

Study setting

KDH is a 233-bed facility under management by the Rwandan Ministry of Health and with technical and financial support from Partners In Health/Inshuti Mu Buzima. The hospital is located in the rural Eastern Province of Rwanda and serves a catchment area of 364 000 people,28 including the 50 000 residents of the Mahama Refugee Camp.30

In Rwanda, women in labour present at their nearest health centre. Those with emergency complications are referred to district hospitals to be assessed and managed by a general practitioner (GP), a non-specialist primary care physician, who performs a c-section if necessary. Few district hospitals, such as KDH, have an obstetrician to manage complicated obstetric cases and mentor GPs. After a c-section, patients are admitted to the postoperative ward for monitoring and medication. On average, patients remain admitted for 3 days before being discharged by a GP. On discharge, a patient is instructed to seek follow-up care with a nurse at her nearest health facility for a wound check and dressing change. If an SSI is detected at the health facility, a patient can be treated as an outpatient at the health centre or, for complicated cases, referred to a district hospital for management by a GP. In the case of a deep SSI, involving deep soft tissues such as fascial or muscle layers, patients are transferred to a referral hospital for management by a specialist.

The population of Rwanda is classified into four socioeconomic ‘Ubudehe’ categories that serve as a guide for social protection programmes, such as community-based health insurance (CBHI) and other socioeconomic services.31 Ubudehe category 1 represents the poorest group, whose medical costs are fully subsidised by the government, while category 4 represents individuals in the highest wealth category, who are expected to cover the majority of their medical expenses either out of pocket or via health insurance.31 CBHI is a public insurance scheme implemented by the government of Rwanda across the country which uses premium contribution by community members based on Ubudehe categories as a mean for solidarity for affordable access to healthcare.32

Participant enrolment

Trained study data collectors approached eligible patients after c-section delivery and prior to discharge. All eligible patients who delivered between 22 September 2019 and 16 March 2020, and who consented to participate in the study were enrolled. However, due to the COVID-19 pandemic, the study was stopped on 17 March 2020, and all subsequent home visits and clinics were cancelled.

Study intervention and follow-up

In Rwanda, CHWs are community volunteers with primary education. They are elected by community members to serve as liaisons between the community and the health system and to provide home-based primary healthcare to residents in their community. For the mHealth-CHW intervention, we used study-specific CHWs (sCHWs); this was at the request of colleagues at the Ministry of Health to avoid disrupting the activities of existing CHWs during the course of the study. The sCHWs were elected through the same procedure as other CHWs and selected to have the same characteristics as typical CHWs. They were all females because of the gender-sensitive duty they were expected to perform. We also hired GPs who were based in Kigali and incentivised two GPs in KDH to help with SSI diagnosis.

The sCHWs visited participants at home at POD10 (±3 days). We chose POD10 because the majority of SSIs develop between POD5 and POD1033 and due to the importance of timely identification and referral of SSIs. To facilitate the home visit, sCHWs met with the village-based CHW who guided her to the study participant’s home. If the local CHW was not available, the sCHW attempted to contact the participant directly via phone when a contact number was available. On the day before the POD10 home visit, study staff would call the participant and village CHW to remind them of the upcoming home visit. If the participant was not accessible by phone, the village CHW was asked to convey the reminder.

During the home visit, the sCHW photographed the incision site using the Wound Screener application on a Samsung Galaxy J8 smartphone. This phone was chosen for its ability to take high-quality photographs in low light and for its relatively low cost. The Wound Screener application was developed specifically for this study,34 with the goal of standardising image capture and quality.

The sCHW transmitted the photograph to a Kigali-based GP using WhatsApp (WhatsApp, Mountain View, California), an end-to-end encrypted messaging platform. On receipt, the GP made a diagnosis of presence or absence of an SSI and texted the result to the sCHW. The sCHW did not communicate the SSI diagnosis to the participant to avoid bias in the subsequent in-hospital assessment. All participants were asked to attend a special study visit at KDH the following day (approximately POD11 ±3 days) and a monetary voucher was provided by the study staff to cover transportation costs.

Comparability assessment

We referred to the available literature that supports the effectiveness of using mHealth in diagnosing SSI. Gunter and colleagues proved that digital application images were of sufficient quality and deemed usable for SSI diagnosis by physicians in more than 80% of cases.18 In addition to saving time, cost and travels, telemedicine-based and usual follow-up care were proven to yield comparable outcomes.18 However, in our context, we expect the utility of image-based diagnoses to be limited to visible superficial or deep SSI, and so will be ineffective in diagnosing organ/space SSI.

Data collection and variables

The study used data collectors with advanced diplomas in nursing and with experience working in the hospital setting. They received training on the study procedures and materials, and on research ethics before the launch of the study. Study data collectors approached eligible study participants and explained the study aim and procedures, and the intervention’s design, benefits and risks. They informed them of their duties and rights, including their right to withdraw consent at any time, and invited them to participate in the study. Informed consent was obtained before enrolling any study participant. At the time of enrolment, data collectors administered structured questionnaires to collect demographic and socioeconomic data. These sociodemographic data included age, marital status, education, occupation, insurance, income level, and phone number of the patient, phone number of a family member or, in case the patient does not have personal phone, phone number of a neighbour.

Study staff also extracted clinical data from patients’ medical files using data extraction grid designed in REDCap (V.8.10.20), a secure web application certified for medical research studies.35 Clinical data included medical history such as comorbidity, defined as having any documented underlying disease conditions prior to c-section, namely diabetes, HIV/AIDS, hypertension and other cardiovascular disorders; intraoperative data (antibiotic therapy, intraoperative complications); and postoperative care. The postoperative length of stay was calculated by subtracting date of surgery from date of discharge from hospital.

During the POD10 home visit, the sCHW documented process indicators using paper-based questionnaires. These indicators detailed the feasibility of implementing the intervention, including the ability to find patients’ homes, patients’ acceptance of wound examination and wound image capture, patients’ willingness to share electronically transmitted data with a GP, successful transmission of wound images to a GP, receipt and timeliness of a GP diagnosis and any challenges encountered. These process indicators were then transcribed by the study staff into REDCap. At the POD11 hospital visit, a Kirehe-based GP asked SSI screening questions, performed a physical examination on the patient and determined whether the patient had an SSI as per the Centers for Disease Control and Prevention definition. SSI refers to an infection that occurs after surgery in the part of the body where the surgery took place.36 This included superficial, deep or organ/space SSI. This GP diagnosis, based on their patient interview and physical examination, was considered the gold standard for the purposes of this study, as this is the current standard of the field for SSI diagnosis in Rwanda. The responses to the GP interview and physical examination were directly recorded in REDCap.

Data management and protection

All data were collected, managed and stored using REDCap. Images shared through WhatsApp had end-to-end encryption and no personal identifiers were transmitted. The photos were deleted from the memory of the phone after being uploaded to a password-protected study server and a password-protected study computer. The Kigali GPs were also instructed to delete all patient photos from their phones after review. Participant data were deidentified using study IDs; we maintained a separate password-protected file matching study ID and participant identifiers.

Data analysis

We described the demographic and clinical characteristics of participants with frequencies and proportions and assessed the feasibility of the mHealth-CHW intervention by reporting the percentage of process indicators successfully completed. The target was for GPs to transmit a diagnosis within 1 hour; the full telemedicine intervention was considered unsuccessful if the response was received more than 1 hour after the image was sent. We reported the sensitivity and specificity of the GP telemedicine diagnosis-based images texted by the sCHW to the GP compared with the diagnosis based on a physical examination conducted by a GP at the hospital. All descriptive analyses, sensitivity and specificity, and predictive value analyses were completed in Stata V.15 (College Station, Texas: StataCorp).

Patient and public involvement

The patients contributed to identifying the issue which inspired the research question and outcome measures. During previous qualitative research in the same setting, patients expressed concerns regarding long distances and financially prohibitive travels to access health facilities and inappropriate postdischarge service. They also signalled their appreciation for services rendered to them by CHWs. The intervention that was implemented in this study was informed by the current literature showcasing advancement in mHealth in high-income countries. We believed that we could leverage the telecommunication advances and well-established CHWs’ network available in Rwanda to improve postdischarge follow-up care for postoperative patients, especially women after c-section. We did not involve patients in the design of this study, but they were strong partners in the recruitment and the conduct of the study. The study results were disseminated to the hospital leadership and to CHWs.

Results

Of the 787 women enrolled in the study, the majority were aged 21–30 years (54.5%, n=429), had primary education as their highest level of education (67.3%, n=529), were farmers (84.8%, n=667), were in Ubudehe category 2 (51.7%, n=406) and used CBHI (94.3%, n=742). Nearly all women had no comorbidities (98.3%, n=773) and 76.9% (n=605) were discharged from the hospital by POD3 (table 1).

Table 1

Demographic and clinical characteristics of study participants (n=787)

Variables	Frequency	%
Age (n=787)
≤20 years old	126	16.0
21–30 years old	429	54.5
>30 years old	232	29.5
Marital status
Single or separated (divorced or widowed)	137	17.4
Married	296	37.6
Cohabiting (no legal marriage)	354	45.0
Education level (n=786)
No education or less than primary education	79	10.1
Primary education	529	67.3
Secondary education or higher	178	22.7
Occupation
Farmer	667	84.8
Employed/trader	87	11.1
Housewives	33	4.2
Type of insurance
No insurance	6	0.8
Community-based health insurance (CBHI)	742	94.3
Private insurance	39	5.0
Ubudehe category* (n=785)
Category 1	89	11.3
Category 2	406	51.7
Categories 3 and 4	290	36.9
Mode of communication for POD10 visit reminder
Reached on her phone/household phone	354	45.0
Reached through a neighbour’s phone	55	7.0
Reached through a CHW	230	29.2
Not reached	148	18.8
Comorbidity†
No	773	98.2
Yes	14	1.8
Postoperative length of stay
Within 3 days	605	76.9
More than 3 days and less than 10 days	182	23.1

*Ubudehe category 1 is the socioeconomic category of the poorest while category 4 is the wealthiest.

†Comorbidity includes underlaying diseases such as diabetes, HIV/AIDS and cardiovascular diseases.

CHW, community health worker; POD10, postoperative day 10.

During the intervention implementation, 91.4% (n=719) of women’s homes were located, mostly (89.9%, n=684) through the support of a village CHW (figure 1). All visited women (100%, n=719) agreed to have a physical examination conducted by the sCHW and to have the wound photo taken and shared with the Kigali-based GP via WhatsApp. Of the 719 images captured, 686 (95.4%) were successfully sent to the GP, and 623 (86.7%) had the GP’s SSI diagnosis returned within 1 hour. For these, the mean duration between the time the photo was sent and the time of receipt of the SSI diagnosis was 11 min (IQR: 2–29 min). The SSI diagnosis from the GP was received after 1 hour for 92 images (12.8%), and not received at all for four photos (0.6%). The full mHealth-CHW intervention was successfully completed for all steps, including receipt of diagnosis within 1 hour, for 79.2% (n=623) of participants (figure 1).

Figure 1

Flow chart of study participants into the study and implementation process indicators. Mean duration in minutes from when the photo was sent to the time of surgical site infection (SSI) diagnosis reception: 11 min (2–29). SSI diagnoses received from general practitioners (GPs) more than 1 hour after their photos being sent were still considered for telemedicine accuracy analyses, though recorded in process indicators by study-specific community health workers (sCHWs) as not successfully sent.

Of the 715 women for whom the telemedicine SSI diagnosis was received from a GP, 30 SSIs (4.2%) were identified. Of the 707 women screened at the POD11 study visit, the GP diagnosed 38 SSIs (5.4%). For the 694 women who had both POD10 telemedicine diagnoses and POD11 physical examination diagnoses, the sensitivity was 36.8% (95% CI 22.6% to 53.8%) and specificity was 97.6% (95% CI 96.1% to 98.5%). Of those diagnosed as SSI positive based on telemedicine, 46.7% (n=14) were SSI positive according to the GP examination, and of those diagnosed as negative, 96.4% (n=640) were not subsequently diagnosed as having an SSI (table 2).

Table 2

Accuracy of the telemedicine-based SSI diagnosis

	Physical examination-based SSI diagnosis
		Positive	Negative	Total
Telemedicine-based SSI diagnosis	Positive	14	16	30	PPV46.67%
	Negative	24	640	664	NPV96.4%
	Total	38	656	694*
		Sensitivity36.8% (22.6–53.8)	Specificity97.6% (96.1–98.5)

*Only analysed those with both telemedicine and physical examination-based SSI diagnosis.

NPV, negative predictive value; PPV, positive predictive value; SSI, surgical site infection.

Discussion

This mHealth-CHW intervention was successfully implemented in rural Rwanda, and while the intervention was effective for ruling out SSIs in postdischarge c-section patients, the overall sensitivity was low. Although telemedicine interventions for postoperative follow-up have demonstrated promise in high-income countries,16–18 they have not been widely used in SSA or other low and middle-income settings. Our study findings confirmed that these types of interventions could be applied in rural Rwanda and in similar settings, but scalability in different settings is contingent on an established CHW network, national and local health facility leadership support, reliable communications infrastructure and sufficient mobile phone penetration in rural areas. In our study, the challenge of unreliable internet and cell phone coverage was manageable but resulted in delayed GP diagnosis via telemedicine. These challenges have been highlighted elsewhere23 37 38 and should be taken into account when planning mHealth interventions.

Most patients were accepting of all steps of the mHealth-CHW intervention. This is consistent with other studies demonstrating high acceptability of telemedicine interventions,39 and is likely influenced by how integrated CHW home visits are in Rwanda’s health system. CHWs are locally elected by the community members, respected and strongly committed to providing village-level care. CHWs link the community to the health system40 41 and therefore are highly trusted, even when implementing new activities.

The telemedicine SSI diagnosis had a low sensitivity but a high specificity, which was similar to findings when evaluating the wound photography for remote postoperative SSI diagnosis in high-income settings.42 This low sensitivity can be attributed to poor quality of wound image or the ability and confidence of the GP to consistently evaluate the wound status based on images alone, which can change from one to another and from time to time.43 44 While the sensitivity is low, the high negative predictive value could help reduce unnecessary postoperative clinic visits. This could relieve financial and travel hardships for these vulnerable patients and workload for clinicians, as has been shown in other studies.18 20 45 This could also reduce the number of unsolicited health facility visits, which is of critical importance now given COVID-19 infection risks.46

Although we strived to obtain the best of quality and standardised wound images, the forementioned challenges might have impaired the ability of GP to accurately diagnose SSI. In response to these observed challenges, we upgraded the lighting and quality image camera as part of our iterative quality improvement process. Therefore, prior to scaling such an intervention, we strongly recommend identifying strategies to increase sensitivity. One potential approach would be to include clinical data along with the wound photo.44 Other strategies we are exploring are including wound images taken at different times after surgery and tracking wound change over time. This would allow comparisons between wound images from POD3 to the POD10 so that the GP can assess how the wound has progressed over time rather than at a static time point. A third strategy is to develop algorithms using artificial intelligence, particularly machine learning.47 This would require developing large libraries of images and diagnoses to train and evaluate algorithms, but if successful, these algorithms could improve accuracy and remove the person-to-person variability of diagnoses based on image review.44

Limitations

We encountered several limitations during this study’s implementation that should be considered while interpreting our study findings. First, we did not assess the GP’s perception of image quality or the GP’s confidence in making diagnoses based on an image alone. The questions to the Kigali-based GP only included a closed question to indicate ‘SSI positive’ or ‘SSI negative’, and therefore we did not receive further insights or clarifications that could help improve the mHealth-CHW intervention. Second, the observed SSI rate at KDH declined from 10.9%24 to 5.4%, and as such, we had fewer than expected cases of SSI. This led to lower than desired precision in our sensitivity estimates.

A third limitation is that the gold standard for SSI diagnosis is a GP physical examination, without isolating pathological organisms through swab or fluid culture. Due to lack of infrastructure for pathology confirmations, this is the main means of SSI diagnosis in this rural hospital, similar to many other rural SSA settings.48 49 Since we are interested in identifying strategies that imitate a facility visit without the burden of travel, the GP diagnosis is a suitable gold standard in this context. Finally, for logistical reasons, the telemedicine screening and physical examination were 1 day apart. It is possible that during that period, an SSI could have developed, or the wound could have changed considerably. Nevertheless, we assume the likelihood of a change in SSI status or wound appearance to be minimal because all follow-up physical examinations by a GP were performed within 1 day of the telemedicine diagnosis. We also suspect that the effectiveness of the telemedicine-based diagnosis was limited to visible signs of superficial and deep SSI, and thus was unable to detect organ/space SSI.

Conclusions

This study showed that implementation of an mHealth-CHW postoperative follow-up intervention in rural Rwanda and similar settings is feasible. Health system and community infrastructure, including telephone network coverage, a robust CHW network and buy-in from providers and the community, are prerequisites for its success. The intervention’s high acceptability among patients suggests potential scalability in similar settings. However, the telemedicine-based SSI diagnosis had a low sensitivity that will need to be addressed by further studies.