SMS messaging to improve retention and viral suppression in prevention of mother-to-child HIV transmission (PMTCT) programs in Kenya: A 3-arm randomized clinical trial.

BACKGROUND: Pregnant and postpartum women living with HIV (WLWH) need support for HIV and maternal child health (MCH) care, which could be provided using short message service (SMS). METHODS AND
FINDINGS: We compared 2-way (interactive) and 1-way SMS messaging to no SMS in a 3-arm randomized trial in 6 MCH clinics in Kenya. Messages were developed using the Health Belief Model and Social Cognitive Theory; HIV messages were integrated into an existing MCH SMS platform. Intervention participants received visit reminders and prespecified weekly SMS on antiretroviral therapy (ART) adherence and MCH, tailored to their characteristics and timing. Two-way participants could message nurses as needed. Clinic attendance, viral load (VL), and infant HIV results were abstracted from program records. Primary outcomes were viral nonsuppression (VL ≥1,000 c/ml), on-time clinic attendance, loss to follow-up from clinical care, and infant HIV-free survival. Among 824 pregnant women randomized between November 2015 and May 2017, median age was 27 years, gestational age was 24.3 weeks, and time since initiation of ART was 1.0 year. During follow-up to 2 years postpartum, 9.8% of 3,150 VL assessments and 19.6% of women were ever nonsuppressed, with no significant difference in 1-way versus control (11.2% versus 9.6%, adjusted risk ratio (aRR) 1.02 [95% confidence interval (CI) 0.67 to 1.54], p = 0.94) or 2-way versus control (8.5% versus 9.6%, aRR 0.80 [95% CI 0.52 to 1.23], p = 0.31). Median ART adherence and incident ART resistance did not significantly differ by arm. Overall, 88.9% (95% CI 76.5 to 95.7) of visits were on time, with no significant differences between arms (88.2% in control versus 88.6% in 1-way and 88.8% in 2-way). Incidence of infant HIV or death was 3.01/100 person-years (py), with no significant difference between arms; risk of infant HIV infection was 0.94%. Time to postpartum contraception was significantly shorter in the 2-way arm than control. Study limitations include limited ability to detect improvement due to high viral suppression and visit attendance and imperfect synchronization of SMS reminders to clinic visits.
CONCLUSIONS: Integrated HIV/MCH messaging did not improve HIV outcomes but was associated with improved initiation of postpartum contraception. In programs where most women are virally suppressed, targeted SMS informed by VL data may improve effectiveness. Rigorous evaluation remains important to optimize mobile health (mHealth) interventions. TRIAL REGISTRATION: ClinicalTrials.gov number NCT02400671.

RCT Entities:   Population Interventions Outcomes

Introduction

Mobile health (mHealth) interventions are increasingly deployed to facilitate engagement in care, retention, and adherence to treatment in HIV programs. mHealth systems are attractive as they extend the reach of the clinic, have efficiencies that can support overextended healthcare workers, and can be implemented at relatively low cost. There is promising, but mixed, evidence that short message service (SMS) may improve retention in care and antiretroviral therapy (ART) adherence and viral suppression. A 2020 systematic review identified 27 studies evaluating the effect of SMS, interactive voice response (IVR), and phone calls on HIV treatment in low- and middle-income countries [1]. Of these studies, 41% reported significant positive effect on ART adherence and 21% on retention in care; the authors identified short follow-up time and small sample sizes as limitations. Similarly, a 2019 systematic review and meta-analysis of phone interventions to improve ART adherence identified 9 trials evaluating text message interventions [2]. Meta-analysis identified moderate or nonsignificant effect of text messages on ART adherence, depending on the measure used. Studies with interactive interventions reported stronger effects, as has been reported in other health conditions [3]. mHealth interventions could be especially useful in prevention of mother-to-child HIV transmission (PMTCT) programs [4]. During the peripartum period, women living with HIV (WLWH) need to optimize both maternal child health (MCH) and HIV care. Several studies have found waning ART adherence postpartum and suboptimal viral suppression in cohorts of peripartum women [5-7]. Qualitative studies note that women desire peripartum support and encouragement, both for MCH and for HIV care [8-10].

We previously developed an open-source human–computer hybrid interactive SMS platform (Mobile Solutions of Women and Children’s Health (Mobile WACh)) to improve MCH outcomes during pregnancy and early postpartum [11]. Mobile WACh significantly reduced time to uptake of effective contraception (in both a 1-way arm, where participants received SMS but could not respond, and a 2-way arm, where participants could additionally respond and converse with nurses) and prolonged exclusive breastfeeding (EBF) (in the 2-way arm) in previous randomized clinical trials (RCTs) among peripartum women in Kenya [12,13]. We hypothesized that integrating HIV and MCH messages within a single SMS platform would improve both HIV and MCH outcomes among mother–infant pairs in PMTCT programs. Therefore, we developed a new SMS platform (Mobile WACh-X), which added HIV adherence reminder and support messaging to the existing Mobile WACh platform and conducted an RCT to determine the effect of 2-way or 1-way SMS on viral nonsuppression, programmatic retention in PMTCT, ART adherence, and infant HIV-free survival [14].

Methods

Study design

This was an individually randomized nonblinded 3-armed trial, conducted at 6 public MCH clinics in Kenya: 2 in peri-urban Nairobi County and 4 in rural Siaya, Kisumu, and Homa Bay Counties. Ethical approval was obtained from the University of Washington and the Kenyatta National Hospital–University of Nairobi. The trial protocol was previously published and registered at ClinicalTrials.gov as NCT02400671 [14].

Participants

Participants were recruited from antenatal care (ANC) clinics. Eligible participants were pregnant WLWH, ≥14 years old, had daily access to a mobile phone (own or shared) with a Safaricom SIM card, were willing to receive SMS, planned to reside in the area for 2 years postpartum, planned to receive both MCH and HIV care at the facility, and were not enrolled in other studies. During the first 6 months of enrollment, an additional eligibility criterion of ≤36 weeks gestational age was used; this criterion was removed to expedite recruitment. To maximize generalizability, women who were illiterate but had another person to help them read and write SMS were eligible.

Study staff obtained verbal consent to assess eligibility using a tablet-based questionnaire, using Open Data Kit (ODK). Eligible women provided written informed consent to participate. Consent materials were provided in English, Kiswahili, and Dholuo. In accordance with Kenyan regulations, adolescent participants age 14 to 17 years were considered emancipated by pregnancy and provided consent without the need for parental permission. The target sample size for the RCT was 825 participants. The target was reached, but 1 individual was later found to have been enrolled twice. It was determined that this participant was not pregnant at her first enrollment; thus, only data from the second enrollment were retained.

Randomization and masking

Participants were individually randomized by study nurses 1:1:1 to 1 of 3 arms: control (no SMS), 1-way SMS (participants received SMS but were unable to send messages to the study), and 2-way SMS (participants received SMS and could message the study). Randomization was stratified by site with no more than 399 women from any 1 site. A randomization list was generated by a statistician not involved with the study, using variable block sizes (blocks of 5 different sizes ranging from 3 to 15 each were randomly selected) using Stata 12.1 ralloc. ado v.3.5.2. Allocation codes were placed in sequentially numbered, sealed, opaque envelopes by site, which were sequentially distributed by study nurses to participants and opened by participants. Study investigators were blinded to block number, size, and sequence. Participants and study investigators were not masked to group assignments.

Procedures

The study intervention has been previously described [10,14]. Participants randomized to 1-way and 2-way SMS arms received weekly, automated messages from enrollment to 2 years after delivery. Additionally, participants received SMS clinic visit reminders. SMS were sent at the participant’s preferred time of day (8 AM, 2 PM, or 8 PM), day of week (Monday to Friday), and language (English, Kiswahili, or Dholuo). Two-way automated messages ended with a question related to the message topic to promote engagement. Participants in the 2-way arm could respond to these or initiate a message to the study nurse at any time. Participant messages were answered by a study nurse within 1 business day.

Formative interviews were conducted with peripartum WLWH, healthcare workers, and partners to inform development of the SMS intervention [8,10,15]. SMS content addressed varied topics, including ART adherence, infant HIV prophylaxis, pregnancy education, birth preparedness, pregnancy and delivery complications, infant health, family planning, and clinic appointment reminders. For every 3 messages related to HIV, 1 message was sent related to non-HIV MCH topics (automated SMS bank available on request) [14]. SMS messages were composed based on the Health Belief Model and Social Cognitive Theory [16,17], designed to provide tailored and actionable education, support, and reminder messages to reinforce health behaviors such as clinic attendance and ART adherence. SMS topics were scheduled according to antenatal/postnatal timing and ART experience, with tailored messaging tracks for adolescents, participants newly initiating ART, and participants who experienced pregnancy loss or infant death [14,18]. Participants who had disclosed their HIV status or had their own phone were given the option to receive SMS containing overt HIV-related language; all other participants received SMS with covert references to HIV [10]. Clinic appointment schedules were abstracted from clinical records and entered into the SMS system by the study team. Visit reminder SMS were sent 3 days before the scheduled appointment date, and congratulatory SMS were sent when visits were attended. SMS were sent 3 and 6 days after scheduled visits were missed. All SMS were delivered free of charge through a reverse-billed short code and managed by study nurses through a custom, semiautomated, open-source web application [11,14]. The system was hosted on a password-protected virtual private server.

Data collection occurred at in-person study visits at enrollment in pregnancy, 6 weeks postpartum, and 6, 12, 18, and 24 months postpartum. At each study visit, questionnaire data were collected using ODK, blood samples were collected and archived, and infant dried blood spots (DBSs) were collected. Study staff abstracted viral load (VL) results, appointments, deliveries, clinic visits, and medication refills from clinical records. All clinical care was provided by clinic staff. Since retention in care was a study outcome, study visit attendance was not optimized by the study team, except at the exit visit. If participants did not attend a study visit between 21 and 25.5 months postpartum, study staff traced the participant by phone and home visit. If participants did not wish to attend their exit visit at the clinic, study staff completed the visit at participants’ homes.

At visits where no programmatic HIV VL data were available, the study performed VL assays on archived plasma samples. These VL assays were performed at the Kenya Medical Research Institute (KEMRI)/Centers for Disease Control and Prevention (CDC) in Kisumu or Nairobi, Kenya, using the Roche COBAS TaqMan Analyzer or COBAS TaqMan Version 2.0 (CAP/CTMv2.0) platform. Infant HIV DNA polymerase chain reaction (PCR) testing were abstracted from routine PMTCT program at 6 weeks, 6 months, and 12 months. The study performed an additional infant test at study exit using fourth-generation rapid or fourth-generation ELISA tests using DBS. Maternal plasma ART resistance was assessed using an oligonucleotide ligation assay (OLA) on archived enrollment plasma samples and subsequent samples where VL was nonsuppressed. [19] OLA is designed to detect mutations at HIV-1 pol reverse transcriptase codons 103, 181, 184, and 190 that can confer resistance to non-nucleoside reverse transcriptase inhibitors (NNRTIs) and nucleoside/tide reverse transcriptase inhibitors (NRTIs). If samples yielded indeterminate results by OLA, the amplicon from RNA also underwent consensus sequencing.

Outcomes

Primary trial outcomes were maternal virologic nonsuppression, on-time visit attendance, loss to follow-up, and infant HIV infection or death. Secondary outcomes were maternal ART adherence and ART resistance. MCH outcomes previously noted to be improved by the Mobile WACh platform (timing of postpartum contraception and EBF duration) were also assessed. Qualitative evaluation of participant experiences will be reported elsewhere. Virologic nonsuppression was defined as plasma VL ≥1,000 copies/mL. A secondary analysis evaluated a cutoff of the assay limit of detection (20 copies/mL for plasma, 839 copies/mL, or 550 copies/mL for DBS depending on assay type). In order to probe potential differences in effect in particular groups of women, subgroup analyses for nonsuppression based on participant age, phone sharing, pregnancy history, employment, HIV status disclosure, education, VL, and ART resistance at baseline were conducted. On-time attendance was defined as attending clinic within 2 weeks of (before or after) a scheduled appointment. Loss to follow-up was defined as not attending any clinic visits for at least 6 months. A combined outcome of infant HIV infection or death was based on clinic records, HIV testing, and participant reports of death. ART adherence was defined as the proportion of days “covered” by ART between pharmacy refills. ART resistance was defined as detection of any resistance mutations by OLA at an abundance of 10% or more.

Adverse events, including pregnancy loss, hospitalization, death, and social harms such as HIV status disclosure, intimate partner violence, or loss of housing, were monitored. Serious adverse events (maternal or infant hospitalization or death) were reported to the Kenyatta National Hospital/University of Nairobi ethics review committee. All events were reported to the University of Washington institutional review board annually and to the trial Data Safety Monitoring Board (DSMB) at annual meetings.

Statistical analysis

Our analysis compared outcomes between each intervention arm and the control arm and between the combination of either intervention arm and control. Trial sample size was calculated for 80% power to detect a hazard ratio (HR) of <0.65 for either viral nonsuppression or loss to follow-up, assuming a 25/100 py incidence of each outcome during 2-year follow-up [14]. This sample size was estimated to have power to detect an HR of <0.55 for drug resistance (assuming incidence of 15/100 py) and HR >2.0 for HIV-free survival (assuming incidence of 10/100 py). The initial analysis approach (time to event) only utilized data until first viral nonsuppression and required initial viral suppression at baseline. Because WLWH may have episodic nonadherence and viral nonsuppression and because we had serial VL data, generalized estimating equations (GEE) that incorporate all VL data increased study power to detect effect of the intervention. We adapted our prespecified analysis approach (NCT02400671) accordingly. Using repeated measures approaches, the study would have power to detect a change from baseline of 70% to 85% prevalence of viral suppression to 95% suppression in intervention arms with <168 women per arm, with fewer women per arm depending on the number of repeated measures per woman. All analyses were intent to treat, adjusted for baseline differences between arms in participant characteristics (primigravida and employment status).

Viral nonsuppression

Prevalence of viral nonsuppression during follow-up was defined as the proportion of VL measurements that were nonsuppressed at any time in follow-up and compared using GEE clustered by participant, with log-binomial link, exchangeable correlation structure, and robust standard errors. Data were used after enrollment and ≥4 months since ART initiation, to include only data after VL could have been suppressed by ART. Visits with nonsuppressed VL within 30 days of a previous nonsuppressed visit were excluded. Secondary analyses were conducted comparing the incidence of first virologic nonsuppression by Cox regression, comparing the incidence of virologic suppression as a recurrent event by Andersen–Gill regression with robust error estimation clustered by participant, and comparing the cumulative incidence of nonsuppression among women with any post-enrollment VL data by delivery, 6, 12, and 24 months postpartum by log-binomial regression. Time of event was calculated as the midpoint between the last suppressed and the first unsuppressed VL. The prespecified primary analysis was modified between publication of the trial protocol [14] and conducting the analysis: GEE was used as the primary analysis in place of Cox regression, as explained above. Additional exploratory analyses included comparisons of the 2-way versus 1-way arm and analyses using different VL thresholds (undetectable versus detectable using limit of detection).

Retention and engagement in care

On-time clinic visit attendance during follow-up to 12 and 24 months postpartum was defined as the proportion of scheduled clinic visits attended on time and compared using GEE clustered by participant, with Poisson link, robust standard errors, and exchangeable correlation structure. The proportion of women lost to follow-up at 12 and 24 months postpartum were compared between arms using log-binomial regression. Time to loss to follow-up was compared between arms using Cox regression for the 0 to 365 day and 0 to 730 day intervals. An exploratory analysis was conducted among intervention participants, comparing on-time attendance of visits that successfully received an SMS visit reminder within the preceding 2 weeks versus visits that did not due to system errors.

Infant HIV-free survival

Time to infant HIV infection or death after live birth in the first 850 days postpartum was compared using Cox regression.

ART adherence

We classified ART adherence as high (≥95%) versus low (<95%) based on 2 approaches: pharmacy refills and self-report. Pharmacy refill data were abstracted from records of ART doses administered during HIV care visits. Longitudinal ART adherence was calculated as the proportion of days between visits that were covered by doses dispensed. Self-reported data were based on the number of doses missed in the last 30 days at study visits. For each method of ascertainment, GEE log-binomial regression with exchangeable correlation structure and robust standard error was used to compare the prevalence of high (≥95%) versus low (<95%) adherence between randomization arms.

ART resistance

Time to development of ART resistance was compared between randomization arms using Cox regression on data from enrollment until 850 days postpartum. Plasma samples with VL <1,000 copies/mL were not tested for resistance and were assumed to not have resistance mutations.

MCH outcomes

Since Mobile WACh-X messaging was derived from Mobile WACh messaging, which focused on MCH topics and was shown to prolong EBF and increase contraception uptake in HIV–negative women [12,13], we explored whether efficacy of MCH messaging persisted with addition of HIV messages. Time to contraception initiation by 6, 12, and 24 months postpartum and time to introduction of complementary foods by 6 months postpartum were compared between arms by Cox regression.

Results

Between November 22, 2015 and May 4, 2017, 9,697 women attended ANC at study sites, of whom 1,280 (13.4%) were living with HIV and were referred for eligibility screening (Fig 1). Of 900 eligible women, 824 (91.6%) were enrolled and randomized, 271 to 1-way SMS, 276 to 2-way SMS, and 277 to the control arm. All participants received their allocated intervention. Twenty-six participants discontinued the SMS intervention (12 in 1-way and 14 in 2-way); 4 participants withdrew (2 in 1-way and 2 in 2-way); and 71 (8.6%) participants did not attend their exit visit (Fig 1). The trial was carried out to completion.

Fig 1

Participant flowchart. MCH, maternal child health; SMS, short message service; VL, viral load.

Baseline participant characteristics

Enrollment characteristics are summarized in Table 1. Each site contributed 7.6% to 27.7% of the study population. Median age was 27 years (interquartile range (IQR) 23 to 31), and median gestational age was 24.3 weeks (IQR 18.3 to 29.6). Median monthly household income was 8,000 KES or 80 USD (IQR 40 to 150), most women (84.3%) were married or cohabiting, 19.7% had <8 years of education, 50.9% were employed, and 14.0% were primigravida. Phone sharing was reported by 29.7% of women, and 94.1% reported they could read and write SMS unassisted. Median time since HIV diagnosis was 2 years (IQR 0.08 to 5.00), and 92.7% of women were on ART at enrollment with a median time since ART initiation of 1.00 year (IQR 0.02 to 3.21). Almost all women were on NNRTI regimens, the majority of which were efavirenz (EFZ) based (73.4%), and most (89.2%) received tenofovir (TDF)-containing regimens. At enrollment, 29.6% of women had VLs ≥1,000 copies/mL: 51.6% of women who initiated ART within the past 4 months and 13.0% of women who had been on ART for at least 4 months. The median CD4 count was 473 cells/mL (IQR 333 to 632). Characteristics were balanced by randomization arm, with the exception of employment (higher in control than intervention arms) and primigravida (lower in control than intervention arms).

Table 1

Participant baseline characteristics.

	Overall		Control		One-way		Two-way
n (%) or median (IQR)
Sociodemographic
Age (years)	824	27 (23–31)	277	27 (23–31)	271	28 (23–30)	276	27 (23–31)
<8 years of education	822	162 (19.7%)	277	54 (19.5%)	269	54 (20.1%)	276	54 (19.6%)
Monthly household income (KES)	527	8,000 (4,000–15,000)	177	7,000 (4,000–15,000)	171	7,000 (3,200–15,000)	179	9,000 (3,000–15,000)
Married/cohabiting	823	694 (84.2%)	276	233 (84.4%)	271	230 (84.9%)	276	231 (83.7%)
Employed	822	418 (50.7%)	276	159 (57.4%)	271	126 (46.5%)	275	133 (48.2%)
Shares phone	824	245 (29.7%)	277	87 (31.4%)	271	77 (28.4%)	276	81 (29.3%)
Can read SMS unassisted	824	808 (98.1%)	277	273 (98.6%)	271	264 (97.8%)	276	271 (97.8%)
Can write SMS unassisted	824	775 (94.1%)	277	260 (93.9%)	271	261 (96.3%)	276	254 (92.0%)
Obstetric
Primigravida	824	115 (14.0%)	277	26 (9.4%)	271	40 (14.8%)	276	49 (17.8%)
Pregnancy intended	821	452 (55.1%)	275	149 (54.2%)	271	145 (53.5%)	275	158 (57.5%)
Gestational age (weeks)	817	24.3 (18.3–29.6)	274	24.7 (19.8–30.3)	271	24.6 (18.1–28.9)	272	23.3 (17.9–29.6)
HIV/ART
Time since HIV diagnosis (years)	820	2.00 (0.08–5.00)	242	2.00 (0.08–5.00)	271	2.00 (0.08–5.00)	275	2.00 (0.08–5.00)
On ART	824	764 (92.7%)	277	258 (93.1%)	271	249 (91.9%)	276	257 (93.1%)
Time since ART start (years)	819	1.00 (0.02–3.21)	274	0.62 (0.01–2.64)	270	1.24 (0.04–3.36)	275	1.10 (0.01–3.42)
On ART ≥4 months	824	470 (57.0%)	277	149 (53.8%)	271	161 (59.4%)	276	160 (58.0%)
HIV status disclosed to partner	806	662 (81.1%)	271	218 (80.4%)	264	224 (84.8%)	271	220 (81.2%)
ART regimen	771		251		259		261
AZT + 3TC + NVP		45 (5.8%)		12 (4.8%)		15 (5.8%)		18 (6.9%)
AZT + 3TC + EFV		4 (0.5%)		1 (0.4%)		3 (1.2%)		0 (0.0%)
TDF + 3TC + LPV/r		12 (1.6%)		4 (1.6%)		1 (0.4%)		7 (2.7%)
TDF + 3TC + NVP		107 (13.9%)		35 (13.9%)		35 (13.5%)		37 (14.2%)
TDF + 3TC + EFV		566 (73.4%)		190 (75.7%)		190 (73.4%)		186 (71.3%)
TDF + FTC + EFV		3 (0.4%)		1 (0.4%)		2 (0.8%)		0 (0.0%)
Other		34 (4.4%)		8 (3.2%)		13 (5.0%)		13 (5.0%)
VL ≥1,000 at enrollment
Total	824	244 (29.6%)	277	81 (29.2%)	271	75 (27.7%)	276	88 (31.9%)
Established ART (≥4 months)	470	61 (13.0%)	149	15 (10.1%)	161	21 (13.0%)	160	25 (15.6%)
New ART (<4 months)	349	180 (51.6%)	125	64 (51.2%)	109	53 (48.6%)	115	63 (54.8%)
CD4 at enrollment	697	473 (333–632)	235	454 (334–601)	229	495 (350–662)	233	456 (306–612)
Adherence characteristics
IMB score	551	77.3 (73.3–82.7)	188	77.3 (73.3–82.7)	180	77.3 (72.0–82.7)	183	77.3 (73.3–82.7)
Information (ART literacy)	695	85.0 (75.0–95.0)	234	85.0 (75.0–95.0)	226	80.0 (75.0–95.0)	235	85.0 (80.0–95.0)
Motivation (to take ART)	695	60.0 (40.0–80.0)	232	60.0 (45.0–80.0)	227	60.0 (40.0–80.0)	236	60.0 (45.0–80.0)
Behavior skills (for adherence)	584	80.0 (77.1–91.4)	202	80.0 (77.1–90.7)	191	82.9 (77.1–94.3)	191	80.0 (75.7–88.6)

ART, antiretroviral therapy; AZT, Zidovudine; EFV, Efavirenz, IMB, information–motivation–behavior skills; IQR, interquartile range; LPV/r, Lopinavir/Ritonavir; NVP, Nevirapine; SMS, short message service; TDF, tenofovir; VL, viral load.

Outcomes of RCT

Overall, 727 participants had ≥1 VL available at ≥4 months after ART initiation, for a total of 3,150 VL measurements included in analyses. Number of VL measurements per participant did not differ significantly by arm (median 6 [IQR 4 to 7] in all 3 arms). The overall proportion of VL measurements that were nonsuppressed at any time during follow-up was 9.78%: 9.62% in the control arm, 11.20% in the 1-way arm, and 8.53% in the 2-way arm (Table 2). There were no significant differences in the frequency of nonsuppression over all follow-up time in the 1-way versus control arm (adjusted risk ratio (aRR): 1.02 [0.67 to 1.54], p = 0.94) or 2-way versus control arm (aRR: 0.80 [0.52 to 1.23], p = 0.31). By delivery, 6 months, 12 months, and 2 years postpartum, 11.1%, 12.1%, 16.0%, and 19.6% of women had had virologic nonsuppression, respectively. Cumulative incidence of virologic nonsuppression by 24 months postpartum did not differ between intervention and control: The control arm 19.9% of control, 23.3% of 1-way, and 15.6% of 2-way participants experienced nonsuppression (1-way adjusted Hazard Ratio (aHR): 1.17 [0.83 to 1.65], p = 0.38; 2-way aRR: 0.78 [0.53 to 1.15], p = 0.21) (Table 2). Incidence of first or recurrent nonsuppression did not differ between intervention and control (1-way aHR for initial nonsuppression: 1.17 (0.82 to 1.68), p = 0.39; 0.85 (0.58 to 1.24), p = 0.39) (Table 2, Fig 2A, and S1 Table). In exploratory analysis, there was a lower cumulative incidence of viral nonsuppression at 6 and 24 months postpartum in 2-way versus 1-way (24 month aRR: 0.67 [0.45 to 0.99], p = 0.04) (Table 2). Secondary analyses using different VL cutoffs yielded similar findings.

Table 2

Effect of Mobile WACh-X on primary outcomes.

	Overall (N = 824)	Control (N = 277)	One-way (N = 271)	Two-way (N = 276)	One-way vs. control	Two-way vs. control	Either arm vs. control	Two-way vs. one-waya
Virologic nonsuppression1
GEE model	Frequency of post-enrollment VL NS n unsuppressed/n VL measurements (%)				RR (95% CI), p-value
N participants	727	243	235	249
VL NS at any time	308/3,150 (9.78)	99/1,029 (9.62)	119/1,066 (11.20)	90/1,055 (8.53)	cRR: 1.09 (0.73–1.65), p = 0.67aRR: 1.02 (0.67–1.54), p = 0.94	cRR: 0.87 (0.56–1.34), p = 0.53aRR: 0.80 (0.52–1.23), p = 0.31	cRR: 0.98 (0.68–1.41), p = 0.91aRR: 0.91 (0.63–1.31), p = 0.60	cRR: 0.79 (0.52–1.22), p = 0.29aRR: 0.79 (0.52–1.21), p = 0.28
Log-binomial model	Cumulative incidence of VL NS n ever unsuppressed/n womenb (%)
Participants with VL NS by 6 months postpartum	62/495 (12.5)	20/159 (12.6)	29/170 (17.1)	13/166 (7.8)	cRR: 1.36 (0.81–2.26); p = 0.24aRR: 1.36 (0.82–2.26); p = 0.23	cRR: 0.62 (0.32–1.20); p = 0.16aRR: 0.63 (0.33–1.20); p = 0.16	cRR: 0.99 (0.62–1.59); p = 0.98aRR: 1.00 (0.63–1.58); p = 1.00	cRR: 0.46 (0.25–0.86); p = 0.01aRR: 0.46 (0.25–0.86); p = 0.01
Participants with VL NS by 12 months postpartum	101/633 (16.0)	32/208 (15.4)	42/216 (19.4)	27/209 (12.9)	cRR: 1.26 (0.83–1.93); p = 0.28aRR: 1.26 (0.83–1.92); p = 0.28	cRR: 0.84 (0.52–1.35); p = 0.47aRR: 0.84 (0.52–1.35); p = 0.47	cRR: 1.06 (0.72–1.55); p = 0.78aRR: 1.05 (0.72–1.55); p = 0.79	cRR: 0.66 (0.43–1.04); p = 0.07aRR: 0.67 (0.43–1.04); p = 0.07
Participants with VL NS by 24 months postpartum	134/683 (19.6)	46/231 (19.9)	53/227 (23.3)	35/225 (15.6)	cRR: 1.17 (0.83–1.65); p = 0.36aRR: 1.17 (0.83–1.65); p = 0.38	cRR: 0.78 (0.53–1.15); p = 0.21aRR: 0.78 (0.53–1.15); p = 0.21	cRR: 0.98 (0.72–1.33); p = 0.89aRR: 0.97 (0.72–1.33); p = 0.87	cRR: 0.67 (0.45–0.98); p = 0.04aRR: 0.67 (0.45–0.99); p = 0.04
Cox model					HR (95% CI), p-value
n eventsn pyIR/100 py	1711,596.210.7	57547.010.4	63490.712.8	51558.49.1	cHR: 1.21 (0.84–1.73); p = 0.31aHR: 1.17 (0.82–1.68); p = 0.39	cHR: 0.86 (0.59–1.26); p = 0.45aHR: 0.85 (0.58–1.24); p = 0.39	cHR: 1.02 (0.75–1.41); p = 0.88aHR: 1.00 (0.73–1.38); p = 1.00	cHR: 0.72 (0.49–1.04); p = 0.08aHR: 0.72 (0.50–1.05); p = 0.09
Timely attendance2
	Proportion of scheduled PMTCT program appointments attended within 2 weeks n attended/n scheduled (%)				RR (95% CI), p-value
N participants	794	268	264	262
Attendance by 12 months postpartum	6,885/7,776 (88.5)	2,306/2,614 (88.2)	2,297/2,592 (88.6)	2,282/2,570 (88.8)	cRR: 1.00 (0.98–1.03); p = 0.85aRR: 1.01 (0.98–1.04); p = 0.52	cRR: 1.00 (0.98–1.03); p = 0.84aRR: 1.01 (0.98–1.04); p = 0.55	cRR: 1.00 (0.98–1.03); p = 0.82aRR: 1.01 (0.99–1.03); p = 0.47	cRR: 1.00 (0.97–1.03); p = 0.98aRR: 1.00 (0.97–1.03); p = 0.99
Attendance by 24 months postpartum	11,006/12,437 (88.5)	3,709/4,199 (88.3)	3,633/4,124 (88.1)	3,664/4,114 (89.1)	cRR: 1.00 (0.97–1.02); p = 0.78aRR: 1.00 (0.98–1.03); p = 0.81	cRR: 1.00 (0.98–1.03); p = 0.70aRR: 1.01 (0.99–1.04); p = 0.36	cRR: 1.00 (0.98–1.02); p = 0.95aRR: 1.01 (0.99–1.03); p = 0.50	cRR: 1.01 (0.98–1.03); p = 0.50aRR: 1.01 (0.98–1.03); p = 0.49
Loss to follow-up3
					HR (95% CI), p-value
N participants	813	276	270	267
n eventsn pyIR/100 py	1831,349.013.6	58450.712.9	62447.813.8	63450.514.0	cHR: 1.08 (0.75–1.54); p = 0.68aHR: 0.99 (0.69–1.42); p = 0.97	cHR: 1.11 (0.77–1.58); p = 0.58aHR: 1.02 (0.71–1.47); p = 0.90	cHR: 1.09 (0.80–1.49); p = 0.58aHR: 1.01 (0.74–1.38); p = 0.96	cHR: 1.03 (0.72–1.46); p = 0.89aHR: 1.03 (0.73–1.47); p = 0.86
Infant HIV acquisition or death4
					HR (95% CI), p-value
N with live births and outcome data	744	252	240	252	cHR: 0.89 (0.40–1.98); p = 0.77aHR: 0.83 (0.37–1.87); p = 0.66	cHR: 1.57 (0.78–3.15); p = 0.20aHR: 1.44 (0.71–2.92); p = 0.31	cHR: 1.23 (0.65–2.36); p = 0.53aHR: 1.14 (0.60–2.20); p = 0.69	cHR: 1.76 (0.84–3.68); p = 0.13aHR: 1.73 (0.83–3.62); p = 0.15
n eventsn pyIR/100 py	441,460.73.0	13501.82.6	11475.82.3	20483.04.1
n HIV+n pyIR/100 py	71,013.60.7	1349.40.3	1326.40.3	5337.91.5
n deathsn pyIR/100 py	381,492.22.6	12509.32.4	10486.42.1	16496.63.2

1Frequency of VL NS was compared by GEE log-binomial regression. VL NS detected multiple times within 30 days is only counted once.

2Proportion of on-time attendance was compared by GEE Poisson regression with exchangeable correlation structure. Only data abstracted from paper records were used, since the intervention would not be expected to affect on-time attendance in a system where the visit schedule was not accessed in real time by study nurses. Scheduled appointments upto 351 or 716 days postpartum were included in the 1-year and 2-year analyses, respectively.

3Incidence of LTFU compared by Cox proportional hazards regression. Both paper and EMRs data were included.

4Events until 850 days postpartum were included. Analyzed using Cox proportional hazards regression. Infection time = midpoint between date of testing and last negative or delivery (if no previous negative). Death time = exact death date. Event time = earlier of infection time and death time.

aExploratory analysis.

bDenominator is cumulative number of women with any post-enrollment VL data within the specified time period.

aHR, hazard ratio adjusted for primigravida and employment; aRR, risk ratio adjusted for primigravida and employment; cHR, crude hazard ratio; CI, confidence interval; cRR, crude relative risk; EMR, electronic medical record; GEE, generalized estimating equations; HR, hazard ratio; LTFU, lost to follow-up; PMTCT, prevention of mother-to-child HIV transmission; py, person-years; RR, relative risk; VL, viral load; VL NS, virologic nonsuppression; IR, incidence rate.

Fig 2

Effect of Mobile WACh-X on viral nonsuppression, loss to follow-up, infant HIV-free survival, and ART resistance.

aHR, adjusted hazard ratio; ART, antiretroviral therapy; SMS, short message service.

A total of 794 participants had ≥1 clinic appointment scheduled during study follow-up, with 12,437 scheduled visits included in analysis. Number of scheduled visits per participant did not differ significantly by arm (median 17 [IQR 11 to 23], 18 [IQR 11 to 23], and 18 [IQR 10 to 23] in control, 1-way, and 2-way arms, respectively). During follow-up to 12 months postpartum, 88.5% of visits were attended within 2 weeks of scheduled dates, and the same proportion up to 24 months postpartum. There was no significant difference between arms in on-time attendance up to 24 months postpartum (1-way aRR 1.00 [0.98 to 1.03], p = 0.81; 2-way aRR 1.01 [0.99 to 1.04], p = 0.36). In explanatory analysis to determine whether lack of effect was due to SMS system failure, among 3,232 scheduled visits in the intervention arms which had a system-sent reminder SMS within 14 days of visit, 2,914 (90.2%) were attended on time, which was significantly higher than among scheduled visits that did not have an SMS sent within 14 days of visit (4,732/5,458, 86.7%, p < 0.001). Incidence of loss to follow-up was 13.6/100 py, with no significant difference between control (12.9/100 py), 1-way (13.8/100 py, aHR: 0.99 [0.69 to 1.42], p = 0.97), and 2-way (14.0/100 py, aHR: 1.02 [0.71 to 1.47], p = 0.90) (Table 2 and Fig 2B).

A total of 744 participants had live births with known infant survival status at exit, and 705 had at least 1 infant HIV result. Of 744 live-born infants, 7 infants (0.94%) acquired HIV, and 38 infants (5.11%) died by 24 months postpartum. Incidence of infant HIV or death was 3.0/100 py: 2.6/100 py in the control arm, 2.3/100 py in the 1-way arm, and 4.1/100 py in the 2-way arm. There were no significant differences in infant HIV acquisition or death between intervention versus control (1-way aHR: 0.83 [0.37 to 1.87], p = 0.66; 2-way aHR: 1.44 [0.71 to 2.92], p = 0.31) (Table 2 and Fig 2C).

A total of 759 participants had ≥1 ART pharmacy refill visit recorded, for a total of 12,935 visits (Table 3). The median overall estimated adherence by pharmacy refill was 100% (IQR 97.2 to 105.0). Among all clinic visits, 8,762 visits (70.7%) were attended soon enough after the preceding visit that ≥95% of days in the interval were covered by ART doses dispensed at the preceding visit. There were no significant differences between intervention and control (1-way aRR of high adherence: 1.00 [0.96 to 1.04], p = 0.98; 2-way aRR: 1.01 [0.97 to 1.05], p = 0.69). A total of 737 participants had self-reported ART adherence data from 2,335 study visits. Median ART self-reported adherence was 100% (IQR 100 to 100), and self-reported ART adherence ≥95% was reported at a total of 2,240 visits (95.9%), with no significant differences by arm (Table 3).

Table 3

Effect of Mobile WACh-X on secondary outcomes.

	Overall (N = 824)	Control (N = 277)	One-way (N = 271)	Two-way (N = 276)	One-way vs. control	Two-way vs. control	Either arm vs. control	Two-way vs. one-waya
	n (%)				RR (95% CI), p-value
Maternal ART adherence: pharmacy refills
	Proportion of refill visits with ≥95% adherencen adherent/n visits (%)
N participants	759	256	250	252
Median adherence	100 (97.2–105)	100 (98.2–107)	100 (98.4–105)	100 (96.8–104)
Visits with ≥95% adherence	8,762/12,395 (70.7)	2,956/4,191 (70.5)	2,881/4,096 (70.3)	2,925/4,108 (71.2)	cRR: 0.99 (0.95–1.04); p = 0.85aRR: 1.00 (0.96–1.04); p = 0.98	cRR: 1.00 (0.96–1.05); p = 0.91aRR: 1.01 (0.97–1.05); p = 0.69	cRR: 1.00 (0.96–1.04); p = 0.96aRR: 1.00 (0.97–1.04); p = 0.82	cRR: 1.01 (0.97–1.05); p = 0.76aRR: 1.01 (0.97–1.05); p = 0.67
Maternal ART adherence: self-reported doses missed in the last 30 days
	Proportion of visits with ≥95% self-reported adherencen adherent/n visits (%)
N participants	737	248	244	245
Mean adherence	100 (100–100)	100 (100–100)	100 (100–100)	100 (100–100)
Visits with ≥95% adherence	2,240/2,335 (95.9)	727/758 (95.9)	755/785 (96.2)	758/792 (95.7)	cRR: 1.00 (0.98–1.03); p = 0.83aRR: 1.00 (0.98–1.03); p = 0.84	cRR: 1.00 (0.97–1.02); p = 0.79aRR: 1.00 (0.97–1.02); p = 0.74	cRR: 1.00 (0.98–1.02); p = 0.97aRR: 1.00 (0.98–1.02); p = 0.93	cRR: 0.99 (0.97–1.02); p = 0.64aRR: 0.99 (0.97–1.02); p = 0.60
Maternal ART resistance
					HR (95% CI), p-value
N participants	748*	254	248	246
Any mutation post-enrollment	24	9	11	4	cHR: 1.28 (0.53–3.08); p = 0.59aHR: 1.18 (0.49–2.87); p = 0.71	cHR: 0.46 (0.14–1.49); p = 0.20aHR: 0.42 (0.13–1.37); p = 0.15	cHR: 0.87 (0.38–1.98); p = 0.73aHR: 0.80 (0.35–1.83); p = 0.59	cHR: 0.36 (0.11–1.13); p = 0.08aHR: 0.35 (0.11–1.11); p = 0.08
Py	1,788.42	608.29	585.33	594.80
Incidence rate/100 py	1.34	1.48	1.88	0.67

Adjusted models include primigravida and employment at baseline.

*Resistance was assessed post-enrollment, and women with baseline resistance were excluded. Women with suppressed viral load were assumed to have no resistance.

aHR, adjusted hazard ratio; aRR, adjusted relative risk; ART, antiretroviral therapy; CI, confidence interval; cHR, crude hazard ratio; CI, confidence interval; cRR, crude relative risk; HR, hazard ratio; py, person-years.

Of 748 participants without ART resistance at enrollment, 282 (37.7%) qualified for resistance testing. Among these participants, 24 developed an ART resistance mutation, for an incidence of 1.34/100 py. There were no significant differences between intervention versus control (1-way aHR: 1.19 [0.49 to 2.89], p = 0.70; 2-way aHR: 0.42 [0.13 to 1.37], p = 0.15).

Viral suppression results were similar in subgroup analyses stratified by age, phone sharing, pregnancy history, employment, HIV status disclosure, education, baseline VL, and baseline ART resistance (S1 Fig).

In exploratory analysis of MCH outcomes, we found significantly lower time to postpartum contraceptive initiation in 2-way versus control: 53.3% of participants in the control arm had initiated contraception by 6 months postpartum, compared with 58.0% of 1-way participants and 58.9% of 2-way participants (1-way aHR 1.09 [0.84 to 1.42], p = 0.51; 2-way aHR 1.31 [1.01 to 1.70], p = 0.04). No significant difference was found in time to cessation of EBF between arms; 80.2% of participants reported EBF to 6 months postpartum (79.3% of control participants, 81.6% of 1-way participants, and 79.6% of 2-way participants).

At exit, participants reported very high satisfaction with the intervention; 91.3% of 1-way participants and 94.2% of 2-way participants strongly agreed that they would recommend Mobile WACh-X to a friend. Participants reported finding the intervention helpful for HIV care (95.7% 1-way and 94.6% 2-way participants strongly agreed) and pregnancy care (97.0% 1-way and 95.4% 2-way participants strongly agreed) with >90% in both arms strongly agreeing that the intervention was helpful for providing new information, emotional support, and helping them take ART.

Overall, 121 adverse events were reported among 112 participants (13.5% of participants). These included maternal hospitalizations (n = 9), infant hospitalizations (n = 14), miscarriages (n = 10), stillbirths (n = 33), infant deaths (n = 38), maternal deaths (n = 12), and social harm (n = 3). The frequency did not differ significantly by arm. One event was determined to be potentially related to the intervention: an incident of intimate partner violence in which the participant’s partner learned she was attending ANC.

Discussion

In this 3-arm RCT, we found that during pregnancy to 2 years postpartum in PMTCT programs, 1-way and 2-way SMS did not significantly change maternal clinic engagement, retention, viral nonsuppression, or infant HIV-free survival. The previously published effect of this SMS platform on maternal postpartum contraceptive initiation was maintained. In exploratory analyses, 2-way SMS was associated with significantly increased viral suppression versus 1-way, consistent with evidence that bidirectional messages are more effective than unidirectional messages, though neither arm displayed higher viral suppression than control, and this finding should be interpreted with caution given the number of comparisons conducted [3]. Although WLWH and MCH providers endorse and enjoy SMS and telehealth services as noted in this and other studies [8,10], and numerous SMS platforms are being deployed for both HIV and MCH care [20], there are few RCTs rigorously assessing SMS effectiveness in improving clinical outcomes, and very few in the context of PMTCT [4]. To our knowledge, ours is the first study of a PMTCT SMS intervention to comprehensively assess maternal visit attendance, VL, resistance, and infant HIV-free survival with follow-up to 2 years postpartum. Our study demonstrates that mHealth systems engaging all mothers may not improve outcomes when on-time clinic attendance and viral suppression are already high in PMTCT programs.

Previous literature has demonstrated efficacy of some SMS interventions in improving HIV outcomes [1,2,21-23], Odeny and colleagues found that messages focused on visit attendance led to increased attendance and infant HIV testing in the first 8 weeks postpartum [24], though the effect was attenuated in a pragmatic trial [25]. Lester and colleagues found that open-ended “check-in” SMS with phone call triage of participant challenges improved viral suppression, but not retention, in adult HIV care [21,26]. Pop-Eleches and colleagues reported that weekly messaging is effective for ART adherence [27]. However, several other studies did not find significant impact on ART adherence [28,29] or viral suppression [30]. Differences in outcomes may be explained by differences in intervention content between studies, as well as challenges detecting treatment effects in populations with high levels of adherence and viral suppression. We hypothesized that a platform which integrated HIV and MCH messaging would be effective in improving both HIV and MCH outcomes. The Mobile WACh MCH platform previously demonstrated significant benefit in prompt initiation of postpartum contraception and increased duration of EBF in HIV-uninfected women [12,13]. Based on input from formative interviews with mothers [8], and informed by the Health Belief Model and Social Cognitive Theory, Mobile WACh-X integrated ART adherence messages with messages on other priorities for WLWH, including facility delivery, EBF, infant immunization, and family planning. Mobile WACh-X also included options for covert or overt HIV-related messages per participant preference [10]. Our intervention had more HIV than MCH SMS (3:1), yet we found limited effect on HIV outcomes, while retaining effectiveness on contraception uptake. In the broader context of HIV care, it is unclear what messages or approaches reliably improve adherence and viral suppression, how to evaluate intervention mechanisms, or how to separate different intervention components [31-33]. Mobile WACh-X’s lack of impact on engagement in HIV care may indicate that our messaging approach did not adequately target the salient constructs shaping this behavior in WLWH. We evaluated participants’ ART-related information–motivation–behavior skills (IMB) using the LifeWindows IMB tool [34]. Women in this cohort had high IMB scores in all 3 domains; however, lower behavior skills scores were associated with viral nonsuppression at baseline [35]. Perhaps SMS grounded in the IMB skills theory and directly addressing behavior skills, or employing motivational interviewing approaches, would be more effective [36,37]. Additionally, it is likely that interest in repetitive adherence messaging declined over time, particularly when compared to MCH messages that addressed major changes in the peripartum/postpartum period with content that was fresh and relevant based on timing (pregnancy planning, delivery, and breastfeeding initiation of postpartum contraception).

In this study, overall retention in care, ART adherence, and viral suppression was high throughout the study, suggesting that most women, most of the time, did not have need for the intervention. It is therefore possible that the intervention would be more effective if it could facilitate timely identification and tailoring to the minority of women or the minority of times with low ART adherence behavioral skills, nonadherence, or unsuppressed VL [31]. There is increasing recognition that behavioral intervention efficacy is enhanced when interventions adapt dynamically to changes in participants’ needs and contexts [38]. Mobile WACh-X messages were tailored for different “tracks” of women (primigravid or multigravid, adolescent or adult, and ART naïve or experienced) and times in the pregnancy and postpartum period, and 2-way nurse interactions were responsive to participants’ questions. However, ART adherence message content was prespecified and delivered with minor adaptations over the pregnancy and 2-year postpartum period and did not incorporate links to VL data or ecological assessments of participant challenges. Our exploratory analysis comparing attendance of visits with and without a reminder SMS suggests better linkage between medical records and the SMS system may improve intervention efficacy, by delivering messages to the participants who need them at the times they need them. Among scheduled visits in the intervention arms, those that had a system-sent reminder SMS were attended more frequently than those who did not (90.2% versus 86.7%, p < 0.001), confirming that visit reminders were effective for those who received them. Linkage of SMS reminders with scheduled clinic visits required nurses to manually enter visit dates from medical records into the SMS system to trigger reminder messages. Challenges in locating paper records with the next appointment dates resulted in the system not always sending messages before the visit. This highlights challenges in administering SMS systems that are unlinked to medical records and the need to develop integrated systems to facilitate targeted interventions. Our study was implemented prior to wide use of electronic medical records (EMRs) in MCH in Kenya. SMS systems integrated within EMRs are likely to improve visit attendance and provide additional data for precise message tailoring.

Few infants acquired HIV infection in this cohort, which is a testament to the success of PMTCT programs in Kenya. Overall, the MTCT rate was <1% during the 2-year follow-up, which is lower than anticipated but consistent with what is possible with PMTCT Option B+ regimens [39,40]. Low MTCT occurred despite occasional maternal viral nonsuppression and resistance and may reflect infant prophylaxis in addition to generally good maternal ART adherence. Five percent of infants died during 2-year follow-up, underscoring continued high mortality in HIV-exposed infants and in the general population.

Strengths of our study include the study design, a multisite longitudinal RCT that included 2-year postpartum follow-up with detailed assessment of multiple important programmatic outcomes. We intentionally designed the study to evaluate programmatic outcomes to get a “real-world” perspective on the effectiveness of the intervention. While the landscape of mobile messaging is fast changing, and internet-based messaging and social media has grown exponentially in the last 5 years, we used SMS, a message format that is accessible on ubiquitous basic phones. Use of SMS means our study can be generalized to low-income and remote communities, and our findings are applicable across messaging platforms. Limitations included that abstraction of programmatic clinic and VL data posed challenges in completeness and data quality. During the study, there were changes in VL testing detection limits and in clinic medical records (transitioning in some cases from paper to electronic but with lapses back to paper at some sites due to power or internet outages). Visit reminder SMS required nurses to enter appointment dates from clinic records, which were not always available. Finally, high attendance, adherence, viral suppression, and infant HIV-free survival limited statistical power for some comparisons: our observed RRs from 0.80 to 1.36, with large confidence intervals (CIs), suggesting that effects may have been detected in a larger study.

In conclusion, this study found no significant impact of an integrated HIV/MCH SMS system on HIV outcomes, which may be explained by ineffective ART adherence messaging strategy or better than anticipated baseline PMTCT outcomes. There were few infant HIV infections despite maternal nonsuppression, and clinic attendance and ART adherence were high demonstrating excellent services in public sector Kenyan PMTCT programs during this period. Future SMS platforms for PMTCT may be more effective by targeting messaging to women at risk for nonadherence (based on behavioral skills or other indicators) or with demonstrated viral nonsuppression. Adaptive SMS platforms that integrate medical record and VL data can better tailor messaging that retains relevance to women in PMTCT. It remains important to rigorously evaluate SMS systems for effectiveness in improving clinical outcomes to ensure their effectiveness and optimally steward HIV care resources.

CONSORT-recommended reporting of trial conduct.

(DOC)

Click here for additional data file.

Incidence rate of virologic nonsuppression by arm.

(DOCX)

Click here for additional data file.

Subgroup analysis of effect of Mobile WAChX on viral nonsuppression.

ART, antiretroviral therapy; CI, confidence interval; GEE, generalized estimating equations; IMB, information–motivation–behavior skills; RR, relative risk; VL, viral load.

(TIF)

Click here for additional data file.

18 Feb 2021

Dear Dr John-Stewart,

Thank you for submitting your manuscript entitled "SMS messaging to improve retention and viral suppression in prevention of mother-to-child HIV transmission (PMTCT) programs: a 3-arm randomized clinical trial" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff and I am writing to let you know that we would like to send your submission out for external assessment.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by .

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for external assessment.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

7 Apr 2021

Dear Dr. Ronen,

Thank you very much for submitting your manuscript "SMS messaging to improve retention and viral suppression in prevention of mother-to-child HIV transmission (PMTCT) programs: a 3-arm randomized clinical trial" (PMEDICINE-D-21-00822R1) for consideration at PLOS Medicine.

Your paper was discussed among the editors and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to invite you to submit a revised version that addresses the reviewers' and editors' comments fully. You will appreciate that we cannot make a decision about publication until we have seen the revised manuscript and your response, and we expect to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We hope to receive your revised manuscript by May 3. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

Please let me know if you have any questions, and we look forward to receiving your revised manuscript.

Sincerely,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

-----------------------------------------------------------

Requests from the editors:

In the article metadata, it appears that the information under "competing interests" should be moved to the "funding" section.

Please add "in Kenya" to the title, preceding the colon.

Please quote study dates in your abstract.

Please combine the "Methods" and "Findings" subsections of your abstract; and add a new final sentence to the restructured combined subsection, which should begin "Study limitations include ..." or similar and quote 2-3 of the study's main limitations.

Please quote the trial registration number with your abstract, but remove information on funding.

After the abstract, please add a new and accessible "Author summary" section in non-identical prose. You may find it helpful to consult one or two recent research papers in PLOS Medicine to get a sense of the preferred style.

Please remove trademarks, e.g., in your Methods section.

Please remove "data not shown" from your Results, and incorporate the relevant data in supplementary files.

Please remove information on competing interests and study funding (inc. grant numbers) from the end of the main text. In the event of publication, this information will appear in the article metadata, via entries in the submission form.

Throughout the text, please style reference call-outs as follows: "... in Kenya [11,12]. We hypothesized ..." (noting the absence of spaces within the square brackets).

In the reference list, please convert all boldface text and italics into plain text. Where appropriate, 6 author names should be listed rather than 3, followed by "et al.".

Does reference 18 lack a journal name?

Please rename figure 1 "Participant flowchart" or similar.

We understand that CONSORT discourages statistical comparisons of baseline characteristics, and ask you to remove this element from table 1.

Please add a completed CONSORT checklist with your revision, labelled "S1_CONSORT_Checklist" or similar and referred to as such in your Methods section. In the checklist, please refer to individual items by section (e.g., "Methods") and paragraph number rather than by line or page numbers, as the latter generally change in the event of publication.

Comments from the reviewers:

*** Reviewer #1:

Statistical review

This paper reports a three arm randomised controlled trial testing SMS-based interventions for improving treatment of pregnant women with HIV. The trial was delivered successfully and was sizeable.

I had some comments on the statistical methods and reporting, listed below.

1. Abstract: I realise that if the abstract is required to be 300 words (I was not aware PLOS medicine had this as a hard limit), then it may be difficult to address some of the following issues as more words would be required.

- The authors mention an outcome being 'loss to follow up' - is this from the clinic or the trial?

- I would change 'no difference' to 'no significant difference' throughout. Since the trial was powered to testing the primary outcome it would be appropriate to add p-values in addition to the confidence intervals.

- "Median ART adherence (100%) and incident ART resistance (1.48/100 py) did not differ by arm." - does this mean that incident ART resistance was exactly 1.48/100 py in each arm? Otherwise I would drop the figure and put 'significantly differ' instead of differ.

2. Introduction, paragraph 2 - '1-way and 2-way arms' are used without much introduction (although it is briefly mentioned in the abstract).

3. Methods, randomisation and masking - this is well reported; can I just ask for the possible block sides to be added?

4. Methods, outcomes - I'd recommend the pre-specified subgroups are added here.

5. Methods, statistical analysis - when I first read this, I did not follow why the trial was powered on a composite criteria rather than the primary outcome itself. I think it is clearer once I read the protocol paper that this is separately the case for both outcomes. I would rewrite this a bit to ensure it is unambiguous.

6. Methods, statistical analysis - overall there seemed to be some differences in the planned analyses between the protocol paper and this paper. The biggest difference is that the primary outcome appeared to be described as a time-to-event outcome in the protocol paper (with the sample size calculation framing it in terms of a hazard ratio). I would recommend the authors describe differences from the protocol and reasons.

7. Methods, statistical analysis - was any adjustment for multiple testing, due to the three arms, made?

8. Methods, statistical analysis - what baseline characteristics were adjusted for?

9. Methods, statistical analysis - I didn't follow here (or perhaps it could be made clearer in the outcomes) whether the analysis was the proportion of eligible visits where the participant was non-suppressed, or the proportion of participants who were non-suppressed at any point. If the former I would recommend ensuring it is clear whether the arm assignment influenced the number of visits per participant.

10. Methods, statistical analysis - I didn't follow what the actual data for retention and engagement in care was. Is it a count variable of number of scheduled clinic visits kept? Were number of scheduled visits consistent across arms?

11. Results, table 1 - generally it is not recommended to test for significant differences in baseline characteristics. Here the number of significant variables is as expected given the number of characteristics considered.

12. Results, outcomes of RCT - it's not too clear what the quoted adjusted risk ratios refer to (see comment 9).

13. Results/discussion - I note the significant 2-way vs 1-way result is highlighted in the discussion. I would ensure it is clear that there were a lot of significance tests reported and this (marginally) significant result could be a false positive due to multiple testing.

14. Results, for the adherence data, when risk ratios are provided I'd clarify that this is referring to proportion of 'high adherence' periods.

15. Abstract/Methods/results - were there any serious adverse events/harms reported in the study?

James Wason

*** Reviewer #2:

This is a very well written and conducted evaluation of SMS for a particularly important HIV risk group, pregnant and post-partum women. The study is well reported and clearly well conducted and the findings are important. SMS is an increasingly popular method of communication with patient groups in Africa and is administered usually in a non-critical way. I think that this paper adds to the dialogue about recognizing both its utility and limitations. I have only a few important points, that I list below:

1) Perhaps in the introduction an again in the discussion there should be some recognition that in populations with high levels of the outcome of interest (say VL suppression), detecting treatment effects is difficult.

2) This was non-blinded rather than unblinded.

3) Who actually administered the allocation to groups? What I mean is, when a patient has agreed to be in the study, how does the enrolling staff member get from consent to allocation?

4) I was a little surprised to see that the figures and tables were in the body of the manuscript. Did experienced writers see the final version?

5) In the discussion, I think you should add a section on limitations of this RCT and also limitations of our knowledge of how to evaluate communication tools with patients. The SMS literature is, in general, not that clear in whether it is a useful tool or not and yet it is widely used among health programs across many settings in Africa. I happen to believe it is useful (but that is just a belief not necessarily driven by the evidence). Could you discuss some the challenges of evaluating digital technologies in such a rapidly changing digital world? For example, you used SMS. But surely by the time this study was completed, people were using Whatsapp as their major form of communication? That's not a limitation, per se, but a challenge in such a rapidly moving environment.

*** Reviewer #3:

Overall, this was a good paper that was generally clearly written, and with a thoughtful discussion. However, the study size was not large, and greater critique of the assumptions made for the study power calculation would be helpful.

Some specific comments, by section, are given below.

Abstract

1. Time on ART 1 year - does this mean follow-up from starting to 1 year after? Or on ART for 1 year at time of recruitment? It becomes clear in the main text, but needs to be clearer in the abstract.

2. Should report in the abstract the absolute values of the outcomes (percentages), and always by trial arm. For the purpose of the study, it is not so helpful to present figures pooled across trial arms and then say there was no difference by trial arm but not showing this.

3. Related to the issue of study power, for which more detail should be provided in the methods of the main paper: the RR of 0.8 comparing 2-way SMS vs control for "ever virally non-suppressed" has a wide confidence interval, and it would be better to say "no evidence of a difference" rather than "no difference" compared to the control arm. I think this result does not correspond to the study power calculation, which was for the cumulative incidence of viral non-suppression by 24 months? It is important to present, as the first result, the one that corresponded to the primary study question for which the study was powered.

4. Need to quote figures on the time to postpartum contraception, especially as the conclusion says the SMS made a difference -but not quantified in the results.

5. Not sure how the conclusion about targeted SMS follows from the results that are presented, the logic for this conclusion could be clarified in relation to the results presented.

Introduction

A bit more detail on the evidence to date on SMS to support adherence to ART post-partum would be helpful, beyond that there is "promising, but mixed evidence" and two papers cited. However, both the papers are systematic reviews - the introduction could mention that there are 2 systematic reviews, and then give a bit more information on what they found and whether there were patterns to the heterogeneity found e.g. did the SMS work better in particular settings, for particular groups of women, when implemented in particular ways?

Methods

1. Study power is very important here, given the study size is not large for a 3-arm trial and given the absence of evidence for effect on the primary outcomes. It seems from the methods text that the study was powered to be able to show a difference between 75% viral suppression in the control arm and 85% or more in the intervention arm, for a comparison of 2-year postpartum virologic non-suppression or retention. It could help to express this also as a risk ratio, given the analysis is done with risk (or rate) ratios, e.g. 15/25 = 0.6 so the study was powered to show a risk ratio of 0.6 (and then the risk ratios observed, were more of the order of 0.8 or 1/0.8 or closer to 1).

Please clarify if this outcome was "virologic non-suppression at the 2 year time point" or "any virologic suppression during 2 years of follow-up". This matters, for which results correspond directly to the study power calculation. Please clarify exactly which analysis in the results corresponds to this power calculation (make sure the language in the results aligns precisely with the language for the study power calculation).

2. Page 14 - says qualitative evaluation of participant experiences will be reported elsewhere, but something would be useful in this paper to explain lack of evidence of effect of the SMS. Please consider including some reference to what was learnt from the qualitative evaluation, in the discussion even if not in the results.

3. Page 15 - Please explain a bit more on "Anderson-Gill regression". Cumulative incidence could have been estimated using Kaplan-Meier methods and using Cox regression to compare trial arms; please explain how "Anderson-Gill regression" is different for a "time to event" analysis and why it was used instead of Kaplan-Meier estimates of the percentage with the outcome by different time points and Cox regression to estimate rate ratios. There is a footnote/caption to Figure 2 which explains why Anderson-Gill regression was used, and that it gave similar hazard ratios to Cox regression of "time to first event", but this information should be in the Methods.

Results

1. Please present first the results that correspond directly to the study power calculation. It is reported in the first paragraph of the "outcomes of RCT" paragraph that overall 20% were ever non-suppressed by 2 years - if this is the primary outcome, please report it in the text separately by trial arm. But, in Table 2 the focus is on (1) viral non-suppression at any time point (regardless of time point) and (2) viral suppression by particular time points i.e. 6, 12, 24 months post-partum.

Please consider some re-ordering of the presentation of results, to first report on the primary outcomes defined in the study power calculation, and then to present other results that are informative.

2. Please clarify how the numerators and denominators are counted for outcome (2) [viral suppression by particular time points] in Table 2, as the model used is a log-binomial model; is the denominator for the 24-month time point all individuals who were followed up at 24 months?

3. Confidence intervals are in general wide, e.g. aRR 0.80 [0.52-1.23], p=0.31 for comparing 2-way SMS with control for non-suppression over time (page 20). The wide confidence intervals, and how they relate to study power, could be given more attention in the discussion.

4. Page 20 - Outcomes of RCT. Please report in the text the percentage with cumulative incidence of virologic suppression by 24 months, as well as the risk ratios. There are numerators and denominators for the log-binomial model used to estimate cumulative incidence of viral non-suppression in Table 2, but it is not completely clear how these numbers are counted - who contributes to the denominator, and who contributes to the numerator? - it is important that is clear in both Table 2 (e.g. with footnotes for more detail) and in the text.

5. Was it considered to use the Kaplan-Meier method, rather than a log-binomial model, for a simple and transparent unadjusted analysis of the comparison of cumulative incidence of viral non-suppression by trial arm that allows for loss to follow-up? And then to use Cox regression to estimate hazard ratios to compare trial arms? It would help to understand the choice of the log-binomial model, if the numerators and denominators were precisely defined for each time point (who contributes to the analysis), see comment also above.

Figure 2A shows "time to first non-suppression", I think from a Kaplan-Meier analysis, but then presents hazard ratios from Anderson-Gill regression that allow for repeated instances of viral non-suppression. I think it would be better for Figure 2A to show the hazard ratios from Cox regression, that directly correspond to the graph. The findings from the Anderson-Gill regression could be included in a table instead - they should be included as part of Table 2, and I think they correspond closely to the primary analysis defined for the study power calculation (cumulative incidence of viral suppression by 24 months) but are not exactly the same as they allow for repeated events (of viral non-suppression).

6. In Table 2, please write out what RR stands for in the table column heading, as it matters for interpretation whether is a risk ratio (which I think it is) or a rate ratio and it needs to be clear for the reader and correspond to the analysis that was done. I think from the log-binomial models, it is a risk ratio, but for example if Cox regression were done it would be a rate ratio / hazard ratio.

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

Submitted filename: PloS Med review - SMS for post-partum ART viral suppression April 2021.docx

Click here for additional data file.

12 Apr 2021

Submitted filename: ResponseToReviewers_final.docx

Click here for additional data file.

6 May 2021

Dear Dr. Ronen,

Thank you very much for re-submitting your manuscript "SMS messaging to improve retention and viral suppression in prevention of mother-to-child HIV transmission (PMTCT) programs in Kenya: a 3-arm randomized clinical trial" (PMEDICINE-D-21-00822R2) for consideration at PLOS Medicine.

I have discussed the paper with editorial colleagues and it was also seen again by two reviewers. I am pleased to tell you that, provided the remaining editorial and production issues are dealt with, we expect to be able to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We hope to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Please note, when your manuscript is accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you've already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosmedicine@plos.org.

Please let me know if you have any questions, and we look forward to receiving the revised manuscript shortly.

Sincerely,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

------------------------------------------------------------

Requests from Editors:

At line 19 (abstract), please make that "infant HIV infection".

At line 22, please adapt the text to "... but was associated with improved initiation of postpartum contraception" or similar, as this result is apparently from an exploratory analysis.

At line 23, we suggest "women are virally suppressed".

At line 27 (author summary), please make that "previously reported apparent effect" or similar, as noted above.

Please make randomized/randomised and derivatives consistent through the paper.

Please remove the competing interest statement from the end of the main text.

Please make that "PLoS ONE" in reference 29.

Please adapt the attached CONSORT checklist so that individual items are referred to by section (e.g., "Methods") and paragraph number. Page and line numbers should not be used, as these generally change in the published paper.

Comments from Reviewers:

*** Reviewer #1:

Thank you to the authors for addressing all my previous comments well. I have no further issues to raise.

*** Reviewer #2:

The authors have done a very good job addressing all of the reviewer's comments and suggestions. They should be commended for the hard work necessary to conduct and successfully complete this trial.

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

6 May 2021

Submitted filename: ResponseToReviewers2_final.docx

Click here for additional data file.

9 May 2021

Dear Dr Ronen,

On behalf of my colleagues and the Academic Editor, Dr Mills, I am pleased to inform you that we have agreed to publish your manuscript "SMS messaging to improve retention and viral suppression in prevention of mother-to-child HIV transmission (PMTCT) programs in Kenya: a 3-arm randomized clinical trial" (PMEDICINE-D-21-00822R3) in PLOS Medicine.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

Prior to final acceptance, please:

Make that "... through health record integration" in the author summary, and

Noting our submission guidelines (https://journals.plos.org/plosmedicine/s/submission-guidelines), please remove "(Chohan, submitted)" from your Discussion section, and if necessary remove the relevant sentence.

In the meantime, please log into Editorial Manager at http://www.editorialmanager.com/pmedicine/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with medicinepress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for submitting to PLOS Medicine. We look forward to publishing your paper.

Sincerely,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org