Do electronic medication monitors improve tuberculosis treatment outcomes? Programmatic experience from China.

BACKGROUND: In China, an indigenously developed electronic medication monitor (EMM) was used. EMM recorded each time the device was opened (no real time data), offering an indirect measure of tuberculosis treatment adherence. Previous study in China showed that the EMM uptake was satisfactory, missing adherence data were common in the information management system (25%) and shift to directly observed therapy (DOT) based on poor adherence documented by EMMs were seldom.
OBJECTIVES: Among people with tuberculosis notified in 30 counties (July-December 2018) where EMM supported self-administered therapy (SAT) was suggested to all eligible (no communication impairment, ambulatory), we assessed the relative differences in unfavourable outcomes and deaths among those started on EMM at baseline (within first month of diagnosis) when compared to SAT alone.
METHODS: This was a cohort study using secondary data. We employed an intention to treat analysis, and used modified Poisson regression with robust variance estimates to assess the association.
RESULTS: Of 1810 eligible people, 1047 used EMM at baseline and of them, 216 (20.1%) stopped using EMM midway. Of 763 people who did not use EMM at baseline, 267 (35.0%) started using EMM later during the treatment. Among those who started using EMM at baseline, 6.3% [95% CI: 4.9, 8.0] had unfavourable outcomes compared to 6.7% [95% CI: 5.1, 8.8] among those who did not (p = 0.746). Lesser deaths were observed in people who started EMM at baseline when compared to those who did not: 2.5% [95% CI: 1.7, 3.7] versus 3.5% [95% CI: 2.4, 5.2], p = 0.191. The lack of association remained after adjusting for potential confounders (occupation, TB classification and TB category).
CONCLUSION: Under programmatic settings, we did not find significant differences in the outcomes. Optimization of EMMs by shifting to DOT when indicated, addressing the issue of missing data and ensuring continuous use is required.

Introduction

Globally, 85% of people with new and relapse tuberculosis (TB) successfully completed treatment while it was 61% among previously treated people (2017 cohort) [1]. Adherence to TB treatment is essential to attain treatment success. To ensure treatment adherence, World Health Organization recommends community-based or home-based directly observed treatment (DOT) over health facility based DOT or self-administered therapy (SAT). DOT administered by trained lay providers or health-care workers is recommended over DOT administered by family members or SAT [2].

SAT has lower treatment success rate when compared to DOT alone, but with no significant difference in mortality [3]. TB treatment outcomes improved with the use of adherence interventions when compared to DOT or SAT alone [3]. Hence, if used, SAT has to be implemented with certain adherence support mechanisms. The adherence support includes one or more of the following: patient education, staff education, material support, psychological support, incentives, reminders, tracers and digital support technologies [2].

Three digital technologies are recommended by World Health Organization as adherence support: short message service, video observed treatment and electronic medication monitors (EMM) [4]. There is a very modest effect of short message service on outcomes and the quality of evidence is low [5-7]. In India, the strategy of the patient giving a missed call after every dose intake showed suboptimal accuracy for monitoring treatment compliance and also was not associated with improved outcomes. Non-receipt of missed call was not followed up with appropriate action by the healthcare providers [8]. Additionally, video observed treatment had no evidence of a difference in rates of treatment completion and mortality when compared to DOT [9, 10].

EMMs are automated electronic devices that monitor and store adherence records combined with audible reminder alarms [11]. The EMMs can function with or without mobile broadband internet coverage and the use of EMMs reduces health workers to engage with TB patients on daily basis for DOT. Thus, the EMMs are looked as accessible and affordable solutions for adherence monitoring in resource-limited settings [4]. In rural Morocco, medication event monitoring system (real time data shared by monitoring the weight of drugs in the medication monitor) with SAT increased treatment success and decreased the loss to follow-up when compared to SAT alone [12].

Universal implementation of DOT in a country like China is impossible. A systematic review in China found that only 50% of people with TB received DOT (from a health care worker or a family member) and 50% were on SAT without any adherence support [13]. Hence, the Chinese Center for Disease Control and Prevention (China CDC) explored the use of an indigenously developed EMM to support SAT from 2009 [14]. These EMMs are automated electronic devices without mobile broadband internet coverage that record (within the device) each time the device is opened, offering an indirect measure of retrospective drug consumption assessment to the health care provider during the hospital visits.

Under trial settings in China, these EMMs reduced poor medication adherence by 40–50% compared to the standard of care in China’s National Tuberculosis Control Program [6]. Previous studies have also shown a high level of acceptability and satisfaction among the people with TB and health care workers [14, 15]. Under programme settings, EMM uptake was satisfactory, but instances of missing EMM data in the information management system (EMM-IMS) were common (25%). Furthermore, nearly in four-fifth instances, people were not shifted to DOT although objective evidence of non-adherence was available [16].

However, the evidence on the impact of EMMs on TB treatment outcomes is still limited [17], especially of those EMMs that do not provide real-time data. In this paper we assessed the relative differences in unfavourable TB treatment outcomes and deaths among those started on EMM when compared to SAT alone.

Methods

Study setting

China has the world’s second highest tuberculosis burden, with an estimated 866,000 cases in 2018 [1]. The treatment success rate among people with new and relapse TB is 93% and among people with previously treated TB is 83% (2017 cohort) [1].

A total of 30 counties were involved in this study, which come from three provinces: nine from Zhejiang province (eastern region), sixteen from Jilin province (middle region) and five from Ningxia Autonomous Region (west region). The economic development is higher in eastern regions and lower in western region. The prevalence of TB, by contrast, is higher in western region and lower in eastern region.

Study design and population

This was a cohort study using secondary data (routinely collected in national TB Programme) in 30 pilot counties of China. We used an intention to treat design. We classified eligible people into two groups based on EMM use at baseline (within first month of diagnosis). Those who did not use EMM at baseline included those who later started using EMM after first month or never used EMM. Those who used EMM at baseline also included those who stopped using EMM midway. We decided against stratifying based on ‘ever EMM use’ because of the possibility of survival bias among people who started using EMM later during the treatment. This could possibly result in a biased association of ‘ever EMM use’ with more favourable outcomes. For the same reason, we did not classify the people based on ‘duration of using EMM’.

People having no communication impairment (mental, visual, auditory, or speech) and not requiring admission at notification were identified as ‘eligible’ to use EMM. All the ‘eligible’ people with TB routinely notified in web-based TB information management system (TBIMS) between July and December, 2018 were included and suggested EMM supported SAT by the doctors from TB designated hospitals (county level, TB basic management unit starting at county level). We excluded people known to be rifampicin-resistant or multidrug-resistant, and people transferred into a county.

The EMM was designed to monitor treatment adherence throughout a one-month fixed-dose regimen. The structure of the EMM and quality control protocol has been described elsewhere [16].

Management of TB with and without EMM

All the people with TB received daily fixed-dose combination treatment over six to 12 months (see Table 1). People with TB would mark daily intake of medication in their treatment cards. The village doctors (village level licensed general practitioners) would visit every ten days during the intensive phase followed by once a month to check on their health status and treatment adherence. People on treatment visited the TB designated hospital on a monthly basis where the doctor would assess adherence based on their treatment card. If <20% of doses were missed, the person was counselled on importance of treatment adherence. If 20–49% of doses were missed, the frequency of home visits by village doctors was increased to once every seven days for rest of the treatment. If there was continued instance of missing 20–49% of doses or a single instance of missing ≥50% of doses, people were shifted to DOT administered by village doctors [14].

Table 1

Treatment regimens for drug-susceptible TB used in the national TB programme, China (2018–19).

Type	Regimen
New TB	2HRZE/4HR
Previously treated TB	2HRZES/6HRE or 3HRZE/6HRE
Pleurisy	2HRZE/7HRE or 2HRZE/10HRE

TB—tuberculosis; H—isoniazid; R—rifampicin; Z—pyrazinamide; E—ethambutol; S–streptomycin.

For EMM eligible people, the verbal consent of using EMM was obtained at the TB designated hospitals. The trained doctors programmed the EMMs before starting on outpatient treatment. During the monthly visits to the TB designated hospital, doctors generated the adherence report by connecting the EMM to offline software in their computers and took appropriate action as described before. The data from the offline system were uploaded to the EMM-IMS by the TB designated hospital staff.

Data variables and sources of data

We extracted data variables from TBIMS (diagnosis date, treatment start date, sex, age, occupation, migrant status, category of TB, classification of TB, treatment outcomes), EMM-IMS (date of starting EMM) and paper-based patient records at county (to assess edibility for EMM). We derived the following variables: treatment initiation delay (in days) and EMM use at baseline (yes/no). We classified cure and treatment completion as favourable treatment outcomes and loss to follow up, death, treatment failure, transfer to drug-resistant TB treatment and not evaluated as unfavourable outcomes (see Table 2) [18].

Table 2

Operational definition of TB treatment outcomes used in the study, China (2018).

Outcome	Definition
Cured	A pulmonary TB patient with bacteriologically confirmed TB at the beginning of treatment who was smear- or culture-negative in the last month of treatment and on at least one previous occasion.
Treatment completed	People with TB who completed treatment without evidence of failure but with no record to show that sputum smear or culture results in the last month of treatment and on at least one previous occasion were negative, either because tests were not done or because results are unavailable.
Treatment failed	People with TB whose sputum smear or culture is positive at month 5 or later during treatment.
Died	People with TB who dies for any reason before starting or during the course of treatment.
Lost to follow-up	People with TB who did not start treatment or whose treatment was interrupted for 2 consecutive months or more.
Not evaluated	People with TB for whom no treatment outcome is assigned. This includes people “transferred out” to another treatment unit as well as people for whom the treatment outcome is unknown to the reporting unit.
Transferred to DR-TB*	People notified as drug susceptible TB and then drug-resistant TB is detected during treatment
Treatment success	The sum of cured and treatment completed.
Unfavourable outcome	All outcomes other than cured and treatment completed

TB—tuberculosis; DR-TB—drug resistant TB.

* If transferred to DR-TB care during TB treatment and there is evidence of registering in the prefecture level DR-TB center, the person will be excluded from this cohort. If there is no evidence of registering in the prefecture level DR-TB center, the person will be included in this cohort and reported under unfavorable outcomes.

Data analysis

We used STATA (version 12.1, copyright 1985–2011 Stata Corp LP USA) for analysis. Separate intention to treat analysis was done for unfavourable outcomes and death (outcomes). We summarized them as proportions with 95% confidence intervals (CI). We identified potential confounders for the association between EMM use at baseline (yes–exposure of interest) and the outcomes using the following criteria: associated with the exposure of interest (p<0.2) and associated with the outcome (unfavourable outcome / death—p<0.20). Age and sex were included irrespective of their p values as they are universal confounders. Association between exposure of interest and outcome was assessed using crude and adjusted relative risks (95% CI). Adjusted analysis was done using modified Poisson regression with robust variance estimates.

Ethics

The ethics committees of the China CDC (number 201909 dated 18 April 2019) and the Ethics Advisory Group of the International Union Against Tuberculosis and Lung Disease (The Union), Paris, France (EAG no 15/19 dated 01 April 2019) approved the study. As the study involved use of secondary programme data, we sought waiver for written informed consent and this was approved by the ethics committees.

Results

There were 2294 people notified with TB and 2227 (97.1%) had EMM eligibility related data. Of 2227 people, 417 (18.7%) were not eligible for using EMM.

Of 1810 ‘EMM eligible’ people, 1047 used EMM at baseline and of them, 216 (20.1%) stopped using EMM midway. Of 763 people who did not use EMM at baseline, 267 (35.0%) started using EMM later during the treatment.

The baseline characteristics of 1810 people, stratified by EMM use from baseline, are depicted in Table 3. The characteristics were not significantly different but for occupation, TB classification and category. These three variables were also associated with unfavourable outcomes as well as death (p<0.20, data not shown) and were therefore considered potential confounders (along with age and sex–potential confounders).

Table 3

Socio-demographic, clinical and treatment accessibility related characteristics of notified people with TB in the 30 selected counties of China between July and December, 2018, stratified by EMM use at baseline*.

Characteristics	EMM use at baseline*
	Yes		No		p value@
	N	(%)	N	(%)
Total	1047	(100)	763	(100)
Age in years
<15	3	(0.3)	5	(0.7)	0.131
15–44	377	(36.0)	252	(33.0)
45–64	393	(37.5)	274	(35.9)
> = 65	274	(26.2)	232	(30.4)
Sex
Male	716	(68.4)	505	(66.2)	0.335
Female	331	(31.6)	258	(33.8)
Occupation
Farmers and herdsmen	507	(48.4)	309	(40.5)	0.006**
Semi-skilled employee	97	(9.3)	71	(9.3)
Salary employee	101	(9.6)	93	(12.2)
Unemployed	233	(22.3)	217	(28.4)
Studying	62	(5.9)	38	(5.0)
Others	47	(4.5)	35	(4.6)
Migrant^
No	997	(95.2)	736	(96.5)	0.238
Yes	50	(4.8)	27	(3.5)
Classification
Bacteriologically confirmed PTB	584	(55.8)	396	(51.9)	0.001**
Clinically diagnosed PTB	394	(37.6)	278	(36.4)
Pleurisy	69	(6.6)	89	(11.7)
Category
New	1008	(96.3)	708	(92.8)	0.001**
Previously treated	39	(3.7)	55	(7.2)
Time interval from diagnosis to treatment (in days)
Zero	1002	(95.7)	729	(95.5)	0.907
≥ One	45	(4.3)	34	(4.5)

Column percentages.

TB—tuberculosis; PTB—pulmonary TB; EMM- electronic medication monitor.

*Of 1810 notified people with TB who were eligible to use EMM, 1047 used EMM within first month of diagnosis (baseline). The remaining 763 who did not use EMM at baseline.

^ Migrant defined as person staying in the same prefecture for less than six months.

@ Chi squared test.

**p<0.05.

The treatment outcomes, stratified by EMM use at baseline, are depicted in Table 4. Among those using EMM at baseline, 6.3% [95% CI: 4.9, 8.0] had unfavourable outcomes compared to 6.7% [95% CI: 5.1, 8.8] among those not using EMM at baseline (p = 0.746). Lesser deaths were observed in people who started EMM at baseline when compared to those who did not: 2.5% [95% CI: 1.7, 3.7] versus 3.5% [95% CI: 2.4, 5.2], p = 0.191. The lack of association remained after adjusting for potential confounders (see Table 5).

Table 4

Treatment outcomes of notified people with TB in the 30 selected counties of China between July and December, 2018, stratified by EMM use at baseline*.

Treatment outcomes	EMM use at baseline*
	Yes		No
	n	(%)	n	(%)
Total	1047	(100.0)	763	(100.0)
Favourable
Cured	493	(47.1)	346	(45.3)
Treatment completed	488	(46.6)	366	(48.0)
Unfavourable
Treatment Failure	10	(1.0)	3	(0.4)
Died	26	(2.5)	27	(3.5)
Not evaluated	30	(2.8)	21	(2.8)

Column percentages.

TB—tuberculosis; EMM- Electronic Medication Monitor.

*Of 1810 notified people with TB who were eligible to use EMM, 1047 used EMM within first month of diagnosis (baseline). The remaining 763 who did not use EMM at baseline.

Table 5

Association between EMM use at baseline* and TB treatment outcomes in the 30 selected counties of China between July and December, 2018.

Treatment outcomes	EMM use at baseline*		RR^(0.95 CI)	aRR^ (0.95 CI)**
	Yes	No
	% (outcome/total)	% (outcome/total)
Unfavourable	6.3 (66/1047)	6.7 (51/763)	1.00 (0.98, 1.03)	1.00 (0.98, 1.03)
Death	2.5 (26/1047)	3.5 (27/763)	0.70 (0.41, 1.19)	0.74 (0.44, 1.26)

TB—tuberculosis; EMM—Electronic Medication Monitor; RR—relative risk; aRR—adjusted RR; CI—confidence interval.

*Of 1810 notified people with TB who were eligible to use EMM, 1047 used EMM within first month of diagnosis (baseline). The remaining 763 who did not use EMM at baseline.

^EMM use at baseline (yes) was the exposure of interest.

**adjusted for occupation, TB classification (bacteriologically confirmed pulmonary TB / clinically diagnosed pulmonary TB / pleurisy) and TB category (new / previously treated) using modified Poisson regression with robust variance estimates.

Discussion

Globally, this is the first study assessing the association between TB treatment outcomes and the use of an EMM that does not provide real-time data under programmatic conditions. TB treatment outcomes did not change overall, but there were 33% reductions in deaths, though not statistically significant.

Our findings are consistent with the findings in trial setting in China using the same EMM. However, the primary study endpoint of the trial was treatment adherence, the sample size was not sufficiently powered to look for differences in treatment outcomes [6]. Meanwhile, our findings are in contrast to the study from Morocco and South African. In rural Morocco, a medication event monitoring system with SAT increased treatment success and decreased the loss to follow-up among people with new smear positive pulmonary TB when compared to SAT alone, but only age, sex and health facility were adjusted for while deriving the adjusted measures of association [12]. Another study in South Africa, using SIMpill system (real time data communication with a web-based application by SMS every time the patient opens the bottle) also found that TB cure rates improved compared with control group, but the study did not describe the management approach of control group, and only 24 people were involved in the EMM group [19]. In addition, availability of mobile broadband internet may be a limitation for use in certain resource limited settings. Absence of mobile connectivity has been reported as a barrier in India [8].

There are four reasons for our observation. First, our previous study showed that the doctors from designated hospital did not shift people to DOT in nearly four-fifth instances when objective evidence of non-adherence was available. Besides missing EMM data in the EMM-IMS were common, and the guidelines did not provide clear instructions to the doctor regarding what steps need to be taken in case of missing EMM data during a month [16]. We didn’t know the exact reasons for these results, but the ineffective implementation could reduce the effect of the intervention. A study from south India even showed TB treatment outcomes conversely worsened after using 99DOTS portal (the portal monitored the free missed call by the patient after taking every dose which indicated medication compliance) because of poor implementation [8].

Second, the intention to treat analysis gives conservative estimates while at the same time provides information on what would happen in a real life scenario. A significant number of people (20.6%) that used EMM at baseline stopped using EMM later during the treatment and many started using EMM after the first month (35% of those who did not use EMM from baseline).

Third, the piloting of EMM in programme settings in 30 counties of China was one of the many interventions of the comprehensive TB control model, designed by National Health Commission of the PRC—Bill & Melinda Gates Foundation Tuberculosis Prevention and Control Project (China-Gates Foundation TB Project). Other interventions that were applicable to all people with TB irrespective of EMM use may also impact the TB treatment outcomes: strengthening collaboration between centers for disease control and prevention and designated hospitals, establish new financing model for reduction of the medical burden for people with TB, integration of information systems and provision of online training for health care providers.

Finally, statistically, the baseline percentage of unfavourable outcomes and deaths were very low. Therefore, the sample size was powered enough only to detect a minimum 50% relative reduction in unfavourable outcomes (assuming baseline of 6.7%) and a minimum 67% relative reduction in deaths (assuming baseline of 3.5%). Future studies may consider assessing the effect among people with previously treated TB and other sub-groups that have a higher baseline percentage of unfavourable outcomes in China (17% with previously treated TB for 2017 cohort).

There is a need for further implementation research (including systematic qualitative enquiry) to optimize the utilization of the EMM. This may include steps to reducing missing adherence data during monthly visit to TB designated hospital or assuming non-adherence and shifting patient to DOT in the event of missing adherence data. The programme should ensure continuous use of EMM by reducing the number of people stopping EMM midway. The programme should also sensitize the doctors to take action based on EMM data which includes shifting people to DOT administered by village doctor as and when indicated [16]. Effect on TB treatment outcomes needs to be reassessed in the future after implementation of these recommendations.

Our limitation was that we relied on secondary programme data. Recording errors and residual confounding cannot be ruled out. Severity of disease and co-morbidities like diabetes, smoking and tobacco use at baseline were not adjusted for.

Conclusion

There is limited information on the effect of EMM on TB treatment outcomes. We therefore assessed an indigenously developed EMM in China that locally recorded (within the device) information every time the box was opened to take the drugs, providing a surrogate indicator for medication adherence (no real time data). Despite, improving adherence in trial conditions, under programmatic settings, the EMM did not result in improved TB treatment outcomes when compared to SAT alone. Actions are required for better optimization by taking action on shift to DOT when indicated, addressing the issue of missing data and ensuring continuous use once started on medication monitor.

Data set including the codebook.

(XLSX)

Click here for additional data file.

27 Aug 2020

PONE-D-20-13035

Whether electronic medication monitors improve tuberculosis outcomes? Programmatic experience from China

PLOS ONE

Dear Dr. Huang,

Thank you for submitting your manuscript to PLOS ONE, and many thanks for your understanding with the delay experienced in securing reviewers for your paper.

After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In particular, the authors were interested in some methodological aspects, including an analysis of the results by stratified levels (EMM duration use, etc), some clarification about the population eligible to enrol in the study, some crucial queries regarding the ethical approval obtained for this study, and the justification of statistical significance with consistency. Additionally, I agree with the reviewer that a more robust framing of this study in the existing literature should be carried out in the Introduction, as well as the Discussion. Finally, please consider the formatting issues about references

Please submit your revised manuscript by Oct 11 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Enrique Castro-Sánchez

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have included the phrase “data not shown” in your manuscript. Unfortunately, this does not meet our data sharing requirements. PLOS does not permit references to inaccessible data. We require that authors provide all relevant data within the paper, Supporting Information files, or in an acceptable, public repository. Please add a citation to support this phrase or upload the data that corresponds with these findings to a stable repository (such as Figshare or Dryad) and provide and URLs, DOIs, or accession numbers that may be used to access these data. Or, if the data are not a core part of the research being presented in your study, we ask that you remove the phrase that refers to these data.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The study “Whether electronic medication monitors improve tuberculosis outcomes? Programmatic experience from China” is dealt with TB adherence technology and traced its comparative unfavorable outcomes with self-administered therapy. It is a retrospective cohort in design and reflects a relevant subject. However, I have some major concerns about the methodology and how the paper is written.

Title:

1. Is a bit peculiar; good to revise. And revise the abstract once you have gone through the comments in the main text.

Introduction

2. This section seems a skewed one as it relies on limited literature including a copy-past of single evidence (paragraph 2, ref 3). Authors need to consider more literature for a reasonable judgment of existing evidence.

3. It helps if the authors highlight outcomes of previous SAT studies in India and nationally approved adherence modalities. And I suggest if some descriptions under “study setting” are pooled here, mainly in paragraph 6, for a strong justification of the need of this study.

Methods

4. The authors should rewrite this section with clear sub-headings for inclusive information and potential replications elsewhere.

5. Study setting: could benefit from the 1st paragraph in “study design and population” (provinces, counties …) and this will simplify understanding.

6. Study design and population. Line 116 says “This was a cohort study design using secondary data…”. Is this to mean a retrospective cohort study?

7. Population/eligibility should be well defined.

8. Ethics. Does it mean China CDC and The Union ethics committees gave the use of secondary data a waiver of informed consent or they approved the primary study? This should not contradict with the data collection date to ensure credibility. Page 6 line 119, study design and population section, states that all the eligible people with TB during July-December 2018 were included. Additionally, the authors should include some brief information on consent to participate.

9. Tables 1 & 2 are not in their places.

10. Data analysis: Not clear why the authors preferred to use two p values (p<.05 and <.2)

Result

11. It is not clear how level of significance is interpreted across.

12. Authors need to check consistency of data in the text descriptions and tables.

Discussion

13. Comparison of findings with more literature would benefit the study.

Conclusion

14. Recheck the conclusion following reviews of results

Acknowledgment

15. This is a very exaggerated one.

References

16. This needs revision across for consistency. Some are with full names of authors, some lack the journal’s name and the like.

Reviewer #2: General comments:

This publication seeks to assess the relative differences in unfavourable outcomes and deaths among those started on EMM at baseline (within first month of diagnosis) when compared to SAT alone. The paper was well written, and the analysis well conducted. Some additions/clarification would improve the analysis interpretation.

However, the paper in its current form lacks further stratifications beyond EMM use at baseline, to fully capture the outcomes. Authors have restricted themselves to EMM use at baseline which does not tell the full story of the EMM group during treatment. The background and methods are well written, and the paper would be more informative should the results and discussion include further stratifications, at the very least stratification by duration of EMM use or stratification by those who Shift to DOT, this is available from a prior publication by the same authors (https://doi.org/10.1371/journal.pone.0232337.g003) (this is the same cohort).

General recommendation:

Revise/resubmit with revisions (especially the results)

Specific comments:

Title:

The wording of the title should be changed to frame it as a question if that is goal of the authors, otherwise it should not end with a question mark.

Background:

Line94: add (s) to solution “most accessible and affordable solution”

Line 95: rephrase “… studies have also shown a high level of acceptability…..among people”

Methods:

Line 120: replace “during” with between

Results:

The authors did not address (1) duration or (2) shift to DOT for the EMM group in a stratified analysis. This is key in assessing treatments outcomes when digital technologies are employed to enhance adherence. The authors could expand Table 4 and 5 with these results and variables “DuraEMM2” and “DuraEMM3” from the previous publication could serve as duration and “shifdot” would be used for shift to DOT. Is it possible to perform these analyses (i.e. stratified analysis by duration and/or shift to DOT)? (I understand that due to limited data this might not be possible but where necessary, please address as a limitation the small numbers in some strata. At the very least, stratification warrants a paragraph or two in the discussion because it is key to interpreting the outcomes.

Could you also look at favourable outcomes in table 5? These would further shed light on the impact of EMM during TB treatment.

Line 209: you can remove “as”

Discussion:

Most of the feedback is detailed in the results section above. You can also add more pill boxes references in the discussion as there are various studies looking at the use of these adherence technologies.

Line 257: correct setting(s) in “use in certain resource limited setting.”

Conclusion:

Line 310: correct “providing an surrogate”

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Tsegahun Manyazewal

Reviewer #2: Yes: Ntwali Placide

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

25 Sep 2020

Reviewer #1:

The study “Whether electronic medication monitors improve tuberculosis outcomes? Programmatic experience from China” is dealt with TB adherence technology and traced its comparative unfavorable outcomes with self-administered therapy. It is a retrospective cohort in design and reflects a relevant subject. However, I have some major concerns about the methodology and how the paper is written.

REVIEWER COMMENTS

1. Title

Is a bit peculiar; good to revise. And revise the abstract once you have gone through the comments in the main text.

AUTHOR RESPONSE

Thank you very much for this comment. We have revised the title, please see lines 1-2 of revised manuscript with track changes. Currently the revised title reads as “Do electronic medication monitors improve tuberculosis treatment outcomes? Programmatic experience from China”.

REVIEWER COMMENTS

2. Introduction

This section seems a skewed one as it relies on limited literature including a copy-past of single evidence (paragraph 2, ref 3). Authors need to consider more literature for a reasonable judgment of existing evidence.

AUTHOR RESPONSE

Thank you. We agree with your suggestion, we have further searched and supplemented with relative literature. Besides, we have carefully gone through the manuscript and revised the introduction as you suggested. (please see lines 75-136 of revised manuscript with track changes)

REVIEWER COMMENTS

3. Introduction

It helps if the authors highlight outcomes of previous SAT studies in India and nationally approved adherence modalities. And I suggest if some descriptions under “study setting” are pooled here, mainly in paragraph 6, for a strong justification of the need of this study.

AUTHOR RESPONSE

Thanks for the suggestion. We have revised introduction section based on your inputs. Please refer to line 85-103 of revised manuscript with track changes.

REVIEWER COMMENTS

4. Methods

The authors should rewrite this section with clear sub-headings for inclusive information and potential replications elsewhere.

AUTHOR RESPONSE

Thank you. We have rewritten this section and let it more clear under the sub-headings (please see lines 139-187 of revised manuscript with track changes)

REVIEWER COMMENTS

5. Methods

Study setting: could benefit from the 1st paragraph in “study design and population” (provinces, counties …) and this will simplify understanding.

AUTHOR RESPONSE

Thank you. We have revised this sentence. (please see lines 144-149 of revised manuscript with track changes)

REVIEWER COMMENTS

6. Methods

Study design and population. Line 116 says “This was a cohort study design using secondary data…”. Is this to mean a retrospective cohort study?

AUTHOR RESPONSE

Thank you. This is a retrospective cohort study design using secondary data. EMM supported SAT was offered to all the eligible people at the beginning of outpatient treatment under the consent of patients. We extracted data variables from TBIMS and EMM-IMS, which were regularly online, reported and quality controlled. However, the STROBE guidelines do not recommend the use of retrospective or prospective in the study design. Hence, we have not used the word ‘retrospective’ in the study design. We hope this is fine.

REVIEWER COMMENTS

7. Methods

Population/eligibility should be well defined.

AUTHOR RESPONSE

Thank you. We have revised this sentence, clearly defined the ‘eligible’ people. (please see lines 175-177 of revised manuscript with track changes)

REVIEWER COMMENTS

8. Methods

Ethics. Does it mean China CDC and The Union ethics committees gave the use of secondary data a waiver of informed consent or they approved the primary study? This should not contradict with the data collection date to ensure credibility. Page 6 line 119, study design and population section, states that all the eligible people with TB during July-December 2018 were included. Additionally, the authors should include some brief information on consent to participate.

AUTHOR RESPONSE

Thank you very much for this comment. Yes, China CDC and The Union ethics committees approved this primary study and the data collected from routine online system (secondary data). As the implementation was done in routine programmatic settings, the experience which we are sharing in this paper and the data used involved secondary data routinely captured in the programme, both the ethics committees waived the need for written informed consent. We have included information on consent to participate. We hope this is fine. (please see lines 211-212 of revised manuscript with track changes)

REVIEWER COMMENTS

9. Methods

Tables 1 & 2 are not in their places.

AUTHOR RESPONSE

Thank you. We have revised it. (please see lines 220-223 and 236-242 of revised manuscript with track changes)

REVIEWER COMMENTS

10. Methods

Data analysis: Not clear why the authors preferred to use two p values (p<.05 and <.2)

AUTHOR RESPONSE

We have revised this. To determine potential confounders, both p value cut offs have been used as p<0.2. In addition age and sex were included as universal confounders. There has been no change in the interpretation of results as a result of this minor change. (please see lines 251 of revised manuscript with track changes)

REVIEWER COMMENTS

11. Result

It is not clear how level of significance is interpreted across.

AUTHOR RESPONSE

If 95% CIs cross each other (for two proportions) or the 95% CIs cross the null value (RR=1), then the results are considered to be statistically insignificant (this corresponds to p>/=0.05).

If 95% CIs do not cross each other (for two proportions) or the 95% CIs do not cross the null value (RR=1), then the results are considered to be statistically significant (this corresponds to p<0.05).

We hope this is fine.

REVIEWER COMMENTS

12. Result

Authors need to check consistency of data in the text descriptions and tables.

AUTHOR RESPONSE

Thank you. We have checked this and made corrections. (please see lines 318-327 of revised manuscript with track changes)

REVIEWER COMMENTS

13. Discussion

Comparison of findings with more literature would benefit the study.

AUTHOR RESPONSE

Thank you. We have added some more references and revised the discussion part. Please refer to line 334-356 of revised manuscript with track changes.

REVIEWER COMMENTS

14. Conclusion

Recheck the conclusion following reviews of results.

AUTHOR RESPONSE

Thank you for the comment. There has been no change in results or its interpretation after the review of results.

REVIEWER COMMENTS

15. Acknowledgment

This is a very exaggerated one.

AUTHOR RESPONSE

This study was supported by SORT IT, a training model organized by The Union and other stakeholders. SORT IT is an operational research training initiative endorsed by WHO and this is a standard requirement. We hope this is fine.

REVIEWER COMMENTS

16. References

This needs revision across for consistency. Some are with full names of authors, some lack the journal’s name and the like.

AUTHOR RESPONSE

Thank you very much for pointing this out. Our sincere apologies for this. We have revised this part. (please see lines 473-526 of revised manuscript with track changes)

Reviewer #2:

REVIEWER COMMENTS

1. General comments

This publication seeks to assess the relative differences in unfavourable outcomes and deaths among those started on EMM at baseline (within first month of diagnosis) when compared to SAT alone. The paper was well written, and the analysis well conducted. Some additions/clarification would improve the analysis interpretation.

However, the paper in its current form lacks further stratifications beyond EMM use at baseline, to fully capture the outcomes. Authors have restricted themselves to EMM use at baseline which does not tell the full story of the EMM group during treatment. The background and methods are well written, and the paper would be more informative should the results and discussion include further stratifications, at the very least stratification by duration of EMM use or stratification by those who Shift to DOT, this is available from a prior publication by the same authors (https://doi.org/10.1371/journal.pone.0232337.g003) (this is the same cohort).AUTHOR RESPONSE

Thank you for the comment. We discussed this stratification issues among the authors and decided to restrict our analysis into two groups. We think under programmatic setting, intention to treat analysis is the most feasible, so we stratified those eligible people into two groups: i) started EMM in first months ii) not started EMM in first month. We decided against dividing those who were eligible for EMM by‘duration’or by ‘ever EMM use’ because that will induce a selection bias, those who partially used EMM (involved starting EMM after first month) will have better outcomes than those who used EMM starting from baseline. We have clarified this further in lines xxx of revised manuscript with track changes. The description of EMM use, missing data, shift to DOT based on EMM data has been mentioned in detail in our previous paper (mentioned by you). We have also summarized the findings of this previous paper in the lines 107-111 (of the introduction) in the revised manuscript with track changes.

REVIEWER COMMENTS

2. Title

The wording of the title should be changed to frame it as a question if that is goal of the authors, otherwise it should not end with a question mark.

AUTHOR RESPONSE

Thanks for the suggestion. We have revised it. Please refer to lines 1-2 of revised manuscript with track changes. The revised title is as follows

Do electronic medication monitors improve tuberculosis treatment outcomes? Programmatic experience from China.

REVIEWER COMMENTS

3. Background

Line94: add (s) to solution “most accessible and affordable solution”

AUTHOR RESPONSE

Thank you. We have revised it. Please refer to lines 89 of revised manuscript with track changes.

REVIEWER COMMENTS

4. Background

Line 95: rephrase “… studies have also shown a high level of acceptability…..among people”

AUTHOR RESPONSE

Thank you. We have revised it. Please refer to lines 106 of revised manuscript with track changes.

REVIEWER COMMENTS

5. Methods

Line 120: replace “during” with between

AUTHOR RESPONSE

Thank you. We have revised it. Please refer to lines 179 of revised manuscript with track changes.

REVIEWER COMMENTS

6. Results

The authors did not address (1) duration or (2) shift to DOT for the EMM group in a stratified analysis. This is key in assessing treatments outcomes when digital technologies are employed to enhance adherence. The authors could expand Table 4 and 5 with these results and variables “DuraEMM2” and “DuraEMM3” from the previous publication could serve as duration and “shifdot” would be used for shift to DOT. Is it possible to perform these analyses (i.e. stratified analysis by duration and/or shift to DOT)? (I understand that due to limited data this might not be possible but where necessary, please address as a limitation the small numbers in some strata. At the very least, stratification warrants a paragraph or two in the discussion because it is key to interpreting the outcomes.

AUTHOR RESPONSE

Thank you for the comment. We have stuck to the intention to treat analysis based on EMM use among eligible patients at baseline. There are multiple subgroups among whom stratified analysis may be provided and crude rates of unfavourable outcomes or deaths in these sub-group could be calculated. However these crude rates are prone to be mis-understood and mis-interpreted on their face value. For example the unfavourable outcomes among those who started EMM later (after one month) and in those who used EMM from baseline. The unfavourable outcomes in the former are lower than the latter and this is due to selection bias (survival benefit for those who start EMM after one month). Hence, in lines 160-171 of revised manuscript with track changes, we have explained why we strict to divide the patients into two groups based on baseline EMM use. We hope this is fine.

REVIEWER COMMENTS

7. Results

Could you also look at favourable outcomes in table 5? These would further shed light on the impact of EMM during TB treatment.

AUTHOR RESPONSE

Thank you. The impact of EMM on TB outcomes will not change irrespective of whether we have unfavourable or favourable outcomes as our outcome of interest. If there is no association between EMM use (yes) and unfavourable outcome, that means there is no association between EMM and outcomes. This association will not change irrespective of which outcome we use as our outcome of interest.

For example, if we take favourable outcome as the outcome of interest, then the aRR for favourable outcome is also one. If we take ‘no death’ as the outcome of interest then the aRR will be around 1.4 and this will not be statistically significant also.

REVIEWER COMMENTS

8. Results

Line 209: you can remove “as”

AUTHOR RESPONSE

Thank you. We have revised it. Please refer to lines 290 of revised manuscript with track changes.

REVIEWER COMMENTS

9. Discussion

Most of the feedback is detailed in the results section above. You can also add more pill boxes references in the discussion as there are various studies looking at the use of these adherence technologies.

AUTHOR RESPONSE

Thanks for the suggestion. We have add several references and revised the discussion part. Please refer to line 334-356 of revised manuscript with track changes.

REVIEWER COMMENTS

10. Discussion

Line 257: correct setting(s) in “use in certain resource limited setting.”

AUTHOR RESPONSE

Thank you. We have revised it. Please refer to lines 355 of revised manuscript with track changes.

REVIEWER COMMENTS

11. Conclusion

Line 310: correct “providing an surrogate”

AUTHOR RESPONSE

Thank you for the comment. We have replaced ‘an’ with ‘a’. Please refer to lines 409 of revised manuscript with track changes.

Submitted filename: response to reviewers_v2.docx

Click here for additional data file.

27 Oct 2020

Do electronic medication monitors improve tuberculosis treatment outcomes? Programmatic experience from China

PONE-D-20-13035R1

Dear Dr. Huang,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Enrique Castro-Sánchez

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have addressed my comments and revised the manuscript accordingly. I have no additional comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Tsegahun Manyazewal

29 Oct 2020

PONE-D-20-13035R1

Do electronic medication monitors improve tuberculosis treatment outcomes? Programmatic experience from China

Dear Dr. Huang:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Enrique Castro-Sánchez

Academic Editor

PLOS ONE