Literature DB >> 26889786

Factors Associated with Loss to Follow-up during Treatment for Multidrug-Resistant Tuberculosis, the Philippines, 2012-2014.

Thelma E Tupasi, Anna Marie Celina G Garfin, Ekaterina V Kurbatova, Joan M Mangan, Ruth Orillaza-Chi, Leilani C Naval, Glenn I Balane, Ramon Basilio, Alexander Golubkov, Evelyn S Joson, Woo-Jin Lew, Vivian Lofranco, Mariquita Mantala, Stuart Pancho, Jesus N Sarol.   

Abstract

To identify factors associated with loss to follow-up during treatment for multidrug-resistant (MDR) tuberculosis (TB) in the Philippines, we conducted a case-control study of adult patients who began receiving treatment for rifampin-resistant TB during July 1-December 31, 2012. Among 91 case-patients (those lost to follow-up) and 182 control-patients (those who adhered to treatment), independent factors associated with loss to follow-up included patients' higher self-rating of the severity of vomiting as an adverse drug reaction and alcohol abuse. Protective factors included receiving any type of assistance from the TB program, better TB knowledge, and higher levels of trust in and support from physicians and nurses. These results provide insights for designing interventions aimed at reducing patient loss to follow-up during treatment for MDR TB.

Entities:  

Keywords:  MDR TB; TB; follow-up; multidrug-resistant tuberculosis; patient compliance; risk factors; side effects; the Philippines; treatment; tuberculosis and other mycobacteria

Mesh:

Substances:

Year:  2016        PMID: 26889786      PMCID: PMC4766881          DOI: 10.3201/eid2203.151788

Source DB:  PubMed          Journal:  Emerg Infect Dis        ISSN: 1080-6040            Impact factor:   6.883


The Philippines is 1 of 22 countries considered to have a high burden of tuberculosis (TB) (), including multidrug-resistant (MDR) TB (resistant to isoniazid and rifampin) (). Compared with treatment for drug-susceptible TB, treatment for MDR TB is longer, more expensive, and less effective, and it causes more medication side effects (–). Resistance to anti-TB drugs has been detected in all regions of the Philippines; an estimated 8,500 MDR TB cases occurred in 2013 (). Programmatic Management of Drug-resistant Tuberculosis (PMDT) was jointly initiated in the Philippines in October 2000 by the private Makati Medical Center in Metro Manila and the Tropical Disease Foundation, Inc., in collaboration with the National TB Control Program and the local government unit, as the first directly observed therapy (DOTS)–plus pilot project for the management of MDR TB approved by the Green Light Committee (). In 2003, a grant proposal from the Philippines for Round 2 of the Global Fund to Fight AIDS, Tuberculosis and Malaria included treatment for 500 patients with MDR TB (National Tuberculosis Control Program, the Philippines, 2013 Aug 25–Sep 6. Report of the Joint Program Review; 2013 Sep 30, unpub. data). This funding was approved and subsequently expanded to 2,500 MDR TB patients approved to receive treatment according to a Round 5 proposal. In 2010, a new coordination team for PMDT was established by the National TB Control Program/Department of Health; the Lung Center of the Philippines was the implementing arm for PMDT. As of September 2014, a total of 44 PMDT health facilities were located in 16 of 17 regions. The annual number of patients with drug-resistant TB who began receiving treatment under PMDT increased from 191 in the 2005 cohort to 2,056 in the 2012 cohort. Despite substantial progress made by PMDT in the Philippines, the proportion of patients for whom treatment was successful decreased from 73% in the 2005 cohort to 46% in the 2010 cohort, while the proportion of loss to follow-up increased from 13% to 38%, respectively (). Even with recent efforts to improve retention of patients receiving treatment for TB (e.g., providing transportation allowance, financial incentives at treatment milestones, food baskets, and halfway houses for patients from remote areas), the proportion of patients lost to follow-up remains substantial. An effective approach to reducing loss to follow-up during treatment for MDR TB is needed (National Tuberculosis Control Program, the Philippines, 2013 Aug 25–Sep 6. Report of the Joint Program Review; 2013 Sep 30, unpub. data), especially in light of data from a prospective multinational study demonstrating that among MDR TB patients lost to follow-up, almost a third had extensively drug-resistant or pre–extensively drug-resistant TB when treatment was started; drug resistance was acquired during treatment by an additional 12% (,). In addition, a third of those lost to follow-up remained culture-positive at last contact, enabling community transmission of strains with more extensive resistance (). However, most studies of loss to follow-up were done retrospectively, through medical record reviews (–), and lacked a theoretical framework. Specific reasons why patients in the Philippines are lost to follow-up during MDR TB treatment are limited and based primarily on the views of healthcare providers. We aimed to determine which individual, diagnosis and treatment, interpersonal, healthcare setting, and social factors were significantly associated with patient loss to follow-up during MDR TB treatment in the Philippines.

Methods

Study Design and Patient Population

We conducted a case–control study among adult patients (>18 years of age) with confirmed MDR or rifampin-resistant TB for whom treatment was initiated during July 1–December 31, 2012, at PMDT treatment facilities in the Philippines. We excluded from study inmates, children <18 years of age, patients enrolled in pharmaceutical clinical trials, and patients who had a major psychiatric disorder or were physically incapacitated. The study was approved by the institutional review board of the Tropical Disease Foundation, Inc., the Lung Center of the Philippines–Ethics Review Committee, and the Ethics Research Committee of the Philippine Tuberculosis Society, Inc. The US Centers for Disease Control and Prevention (CDC) determined that CDC staff involvement did not constitute engagement in human subject research and that submission for CDC institutional review board review was not required. In the Philippines, the standardized treatment regimen for MDR TB is pyrazinamide, kanamycin, levofloxacin, prothionamide, and cycloserine; the intensive phase lasts >6 months and the continuation phase an additional >12 months. For this study, case-patients were defined as patients who were lost to follow-up from MDR TB treatment (i.e., patients whose treatment was interrupted for >2 consecutive months) (). Those who later returned (after being considered lost to follow-up) at the time of interview were eligible for inclusion in the study as case-patients. Control-patients were defined as patients who had continued treatment for MDR TB for >12 months or who had a documented treatment outcome of cured, completed, or failed (). Data collection and interviews were conducted from April 14 through July 31, 2014; thus, all control-patients were receiving MDR TB treatment for >15 months. Case-patients were identified by review of MDR TB registers. The number of patients who were lost to follow-up per treatment facility was assessed; centers with >3 patients with drug-resistant TB who had been lost to follow-up by January 1, 2014, and who were not known to have died, were selected for logistical reasons. Field study staff attempted to find all eligible patients who were lost to follow-up and invite them to participate in the study. Two control-patients were randomly selected for each enrolled case-patient from the same PMDT treatment facilities at which treatment was initiated for case-patients. Of 986 eligible patients, a total of 273 were enrolled: 91 case-patients and 182 control-patients (Figure 1).
Figure 1

Selection of participants for study of loss to follow-up during treatment for multidrug resistant tuberculosis (MDR TB) in the Philippines, 2012–2014. *Study exclusion criteria were incarceration, age <18 years, enrollment in pharmaceutical clinical trials, and major psychiatric disorder or physical incapacitation. †Control-patients who did not give consent for the study were replaced by other randomly selected eligible patients.

Selection of participants for study of loss to follow-up during treatment for multidrug resistant tuberculosis (MDR TB) in the Philippines, 2012–2014. *Study exclusion criteria were incarceration, age <18 years, enrollment in pharmaceutical clinical trials, and major psychiatric disorder or physical incapacitation. †Control-patients who did not give consent for the study were replaced by other randomly selected eligible patients. To characterize factors associated with loss to follow-up during MDR TB treatment, we followed a 5-level social ecologic model (,) that focuses on individual and environmental factors that affect health outcomes: 1) individual factors; 2) interpersonal factors; 3) healthcare setting factors, such as individual experiences with services and relationships within the setting; 4) diagnosis and treatment factors; and 5) social factors (Figure 2). To operationalize each category of factors and develop data collection forms, investigators reviewed TB literature and a 2013 Joint Program Review of the National Tuberculosis Control Program in the Philippines, which was led by the World Health Organization (WHO), and solicited input from experts within the country.
Figure 2

Social ecologic model used to identify factors influencing loss to follow-up during treatment for multidrug resistant tuberculosis in the Philippines, 2012–2014. Boldface indicates data collected through patient interviews and medical record abstractions. DOTS, directly observed therapy.

Social ecologic model used to identify factors influencing loss to follow-up during treatment for multidrug resistant tuberculosis in the Philippines, 2012–2014. Boldface indicates data collected through patient interviews and medical record abstractions. DOTS, directly observed therapy.

Data Collection

Clinical and laboratory data were abstracted from participants’ medical records by using standard data collection forms. In-depth interviews with a series of closed- and open-ended questions were conducted to collect information about the following: demographics, social history, adverse drug reaction experiences, TB knowledge, perceived barriers to treatment completion, self-efficacy to adhere to treatment (), values and expectancies associated with treatment; psychosocial factors (e.g., stigma, sources of emotional support, and family dynamics), financial support, perceptions of the healthcare setting, their diagnosis, their prescribed treatment, impressions of and feedback for the PMDT program regarding the current TB enabler package and patient-centered activities, and interventions under consideration. Cumulative scores were calculated from items focused on 1) patient perceptions of disease severity and TB knowledge; 2) expected outcomes, treatment self-efficacy; 3) patient-reported social support from family and friends; 4) trust in, rapport with, and support from health center staff; and 5) stigma (Table 1). Case-patients were asked to report their primary reason for stopping treatment. (Interview and scoring instruments are available from T.E.T.)
Table 1

Calculation of summary scores in study of loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

Score typeScore calculationScore interpretation
General TB knowledge, including understanding of severity of the disease and susceptibility to the disease
Participants were asked 15 questions that focused on 1) the severity of the TB problem in their community, 2) TB transmission and morbidity/mortality, and 3) TB treatment. Each item answered correctly was awarded 1 point. Incorrect answers or “Not sure” responses received 0 points. The summary score was extrapolated onto a scale of 100 and reported as a percentage by using the following formula: General TB knowledge score = (total points earned/15) × 100 (i.e., score is calculated on a scale of 0–100%).
A higher score may indicate greater TB knowledge and greater perceived severity and susceptibility to the disease.
Expectations related to TB and its treatment
Participants were asked 5 interview questions aimed at determining their concerns for passing TB to loved ones, relapsing, and developing worsening drug resistance. Possible range of scores 5–15.
A higher score may indicate greater concerns or an expectation that TB could cause problems in the future. In addition to factors such as knowledge, attitudes, and beliefs, expected outcomes can determine a person’s actions. These expectations may be derived from 1) previous experiences, 2) observing or hearing about others in similar situations, 3) persuasive conversations, or 4) emotional or physical responses.
Self-efficacy (or confidence) to adhere to treatment at the time treatment was about to begin
Eight interview questions were included in the self-efficacy questionnaire. “Very confident” = 3 points, “A little confident” = 2 points, “unsure” = 1 point, and “I knew I could not do this” = 0 points. The score for each item would be added together to calculate a cumulative self-efficacy score. Possible range of scores 0–24 points.
A higher score may indicate a high degree of self-reported self-efficacy for adhering to treatment regimen, coping with the treatment, and meeting with DOT staff when about to start treatment.
Social support from family and friends
Score was based on responses to 3 interview questions with possible range of scores 3–15.
Lower scores may indicate less support.
Trust in, rapport with, and support from physicians and nursing staff
An overall score was based on 22 items grouped together. Possible range of scores 22–110.
A higher score may indicate a greater level of trust, rapport, and perceived support. Items were separated by topic, and separate scores were also calculated for participants’ 1) trust in, and rapport with physicians (13 questions); 2) trust in, and rapport with nurses (5 questions); and 3) perceived support from health center staff (4 questions).
Patient self-stigmatizationA cumulative score for stigma was based on 2 interview questions. Possible range of scores 1–10.A higher score may indicate less stigma.

*DOTS, directly observed therapy; TB, tuberculosis.

*DOTS, directly observed therapy; TB, tuberculosis.

Data Analysis

Data were entered in a Microsoft (Redmond, WA, USA) Access electronic database. Statistical analyses were performed by using SAS software version 9.3 (SAS Institute Inc., Cary, NC, USA). Thematic qualitative analysis was conducted by using Microsoft Excel software. We assessed associations between individual, interpersonal, healthcare setting, diagnosis and treatment, and social factor data and an outcome of lost to follow-up. For continuous variables, we compared means (SDs) or medians (percentiles) or both, depending on the underlying distributions. The proportions of patients with characteristics of interest were compared between case-patients and control-patients by χ2 or Fisher exact tests, as appropriate. We calculated odds ratios with corresponding 95% CIs. To identify independent factors associated with loss to follow-up, we performed multivariable logistic regression analyses. The initial multivariable model included covariates with epidemiologic, biological, or statistical associations with the dependent variable. We evaluated effect modification and confounding in the full model, and then we performed backward elimination to improve precision of the estimates (). All tests were 2-sided, and a p value of <0.05 was considered statistically significant.

Results

Mean ± SD age of the 273 study participants at start of MDR TB treatment was 39 ± 13 (median 40, range 16–68) years; 164 (60.1%) were male. An HIV test result was recorded for 56 (20.5%) of the 273 patients; 2 (3.5%) were HIV positive. All patients had pulmonary TB, and 35% had cavitary TB. Most (70 [77.8%]) of the 90 case-patients for whom information on length of treatment was available were lost to follow-up during the intensive phase of treatment. Mean ± SD time receiving MDR TB treatment for case-patients was 7.8 ± 3.4 months (median 7 months; 25th percentile 4 months; 75th percentile 11 months) and for control-patients was 19.8 ± 1.7 months (median 20 months; 25th percentile 19 months; 75th percentile 21 months). Most (121 [66.5%]) of the 182 control-patients were still receiving treatment at the time of interview. Among 61 control-patients for whom treatment outcome was available, 35 (57.4%) experienced cure, 24 (39.3%) completed treatment, and 2 (3.3%) experienced treatment failure. Univariate analysis indicated that individual factors significantly associated with loss to follow-up included older age (mean ± SD age 42 ± 13 years for case-patients vs. 38 ± 12 years for control-patients; p = 0.028); tobacco smoking (odds ratio [OR] 2.86, 95% CI 1.65–4.97); alcohol abuse (OR 1.93, 95% CI 1.09–3.40); and residence in an urban slum (OR 0.52, 95% CI 0.29–0.91) (Table 2). General knowledge of TB was significantly lower among case-patients than among control-patients; knowledge included understanding of the severity of and susceptibility to the disease (mean ± SD score 67.8 ± 16.3 vs. 74.3 ± 13.8, respectively; p<0.001), but recall for self-confidence for adhering to treatment at the time of treatment start was significantly higher among case-patients (4.9 ± 5.3 vs. 2.44 ± 3.8, respectively, p<0.001) ().
Table 2

Univariate analysis of Individual factors associated with loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

FactorTotal†Case-patients‡Control-patients‡Odds ratio (95% CI)p value
Data from review of medical records
Demographics
Sex
M16460 (65.9)104 (57.1)1.45 (0.86–2.45)0.16
F10931 (34.1)78 (42.9)1.00
Age27341.6 (13.2)§38.0 (12.5)§1.02 (1.00–1.04)¶ 0.028
Social history
Tobacco smoking
Current, past15365 (73)88 (48.6)2.86 (1.65–4.97) <0.001
Never11724 (27)93 (51.4)1.00
Alcohol abuse
Current, past17566 (75)109 (60.9)1.93 (1.09–3.4) 0.02
Never9222 (25)70 (39.1)1.00
Drug abuse
Current, past5219 (22.4)33 (18.6)1.26 (0.67–2.37)0.48
Never21066 (77.6)144 (81.4)1.00
Clinical information
No. previous TB episodes2731.71 (0.90)§1.62 (1.07)§1.10 (0.86–1.40)¶0.449
BMI
<18.514651 (56)95 (52.2)1.17 (0.7–1.94)0.55
>18.512740 (44)87 (47.8)1.00
Cavitary TB disease
Yes9731 (44.3)66 (41)1.14 (0.65–2.02)0.64
No13439 (55.7)95 (59)1.00
Smear-positive at treatment start
Yes21970 (82.4)149 (85.1)0.81 (0.41–1.63)0.56
No
41
15 (17.6)
26 (14.9)
1.00

Data from patient interviews
Total no. persons residing in household2725.12 (2.79)§5.53 (2.95)§1.94 (0.63–5.99)¶0.27
Residence Comparison 1
Rural area4015 (23.4)25 (24.5)0.94 (0.45–1.96)0.88
Urban slum12649 (76.6)77 (75.5)1.00
Residence Comparison 2
Urban area10526 (34.7)79 (50.6)0.52 (0.29–0.91) 0.02
Urban slum12649 (65.3)77 (49.4)1.00
Paid employment before starting treatment
Yes12649 (54.4)77 (42.3)1.63 (0.98–2.71)0.06
No14641 (45.6)105 (57.7)1.00
Employed before starting treatment but had to quit#
Yes9035 (79.5)55 (83.3)0.78 (0.29–2.07)0.61
No209 (20.5)11 (16.7)1.00
Employed before starting treatment but fired/asked to take leave of absence#
Yes105 (35.7)5 (31.3)1.22 (0.27–5.59)0.80**
No209 (64.3)11 (68.8)1.00
Family sold belongings or household items (assets) to help pay expenses during TB treatment
Yes9328 (31.8)65 (35.7)0.84 (0.49–1.44)0.53
No17760 (68.2)117 (64.3)1.00
Family borrowed money to cover costs due to TB illness
Yes18160 (74.1)121 (70.8)1.18 (0.65–2.14)0.58
No7121 (25.9)50 (29.2)1.00
General TB knowledge†† 27267.81 (16.31)§74.25 (13.78)§0.97 (0.95–0.99)¶ <0.001
Expectations related to TB and TB treatment27211.01 (1.87)§10.76 (1.55)§1.10 (0.94–1.28)¶ 0.28
Self-efficacy (or confidence) to adhere to treatment at the time treatment was about to begin2724.91 (5.28)§2.44 (3.77)§1.13 (1.06–1.19)¶ <0.001

*Boldface indicates significance. BMI, body mass index; TB, tuberculosis. 
†Total reflects number of patients for whom data or responses for each respective category were available. 
‡No. (%) unless noted otherwise. 
§Mean (SD). 
¶Odds ratio is per 1 unit increase.
#Of 126 patients who had paid employment before starting treatment, 90 reported that they subsequently “had to quit” and 10 reported that they had subsequently been “fired/asked to take leave of absence.”
**Fisher exact test.
††Such as understanding of disease severity, susceptibility, scale 0%–100%).

*Boldface indicates significance. BMI, body mass index; TB, tuberculosis. 
†Total reflects number of patients for whom data or responses for each respective category were available. 
‡No. (%) unless noted otherwise. 
§Mean (SD). 
¶Odds ratio is per 1 unit increase.
#Of 126 patients who had paid employment before starting treatment, 90 reported that they subsequently “had to quit” and 10 reported that they had subsequently been “fired/asked to take leave of absence.”
**Fisher exact test.
††Such as understanding of disease severity, susceptibility, scale 0%–100%). Univariate analysis also indicated that the only interpersonal factor significantly associated with loss to follow-up was social support from family and friends. Scores were lower among case-patients than among control-patients (mean ± SD score 12.1 ± 3.4 vs. 12.9 ± 3.0, respectively, p = 0.047) (Table 3).
Table 3

Univariate analysis of interpersonal factors associated with loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

Characteristic, from data from patient interviewsTotalCase-patients†Control-patients†Odds ratio (95% CI)p value
Head of household‡
Yes9137 (41.1)54 (29.7)1.65 (0.98–2.8)0.06
No
181
53 (58.9)
128 (70.3)
1.00

In charge of household budget
Yes9632 (35.6)64 (35.2)1.02 (0.6–1.72)0.95
No
176
58 (64.4)
118 (64.8)
1.00

Social support from family and friends27112.07 (3.35)§12.87 (2.99)§0.92 (0.85–1.00)¶ 0.047

*Boldface indicates significance. TB, tuberculosis.
†No (%) unless noted otherwise. 
‡Provided more than half the cost of keeping up a home the year before becoming sick with TB. 
§Mean (SD).
¶Odds ratio is per 1 unit increase.

*Boldface indicates significance. TB, tuberculosis.
†No (%) unless noted otherwise. 
‡Provided more than half the cost of keeping up a home the year before becoming sick with TB. 
§Mean (SD).
¶Odds ratio is per 1 unit increase. Among healthcare setting factors, case-patients reported having received any type of assistance from the TB program significantly less often than control-patients: overall (OR 0.11, 95% CI 0.04–0.29), food (OR 0.46, 95% CI 0.27–0.79), free medications for treatment of adverse drug reactions (OR 0.28, 95% CI 0.16–0.48), or money for transportation (OR 0.19, 95% CI 0.1–0.37) (Table 4). Scores for trust in, rapport with, and support from physicians and nursing staff were significantly lower among case-patients than among control-patients (mean ± SD score 81.9 ± 15.6 vs. 90.7 ± 7.8, respectively; p<0.001), as were scores for trust in and rapport with physicians (56.0 ± 11.5 vs. 62.2 ± 5.4, respectively; p<0.001), trust in and rapport with nurses (21.8 ± 3.7 vs. 23.8 ± 2.4, respectively; p<0.001), and information and support received from healthcare center staff (7.5 ± 1.7 vs. 8.4 ± 1.2, respectively; p<0.001).
Table 4

Univariate analysis of healthcare setting factors associated with loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

Characteristic, from data from patient interviewsTotalCase-patients†Control-patients†Odds ratio (95% CI)p value
Treatment center provided financial assistance or other items to facilitate treatment adherence
Yes24569 (76.7)176 (96.7)0.11 (0.04–0.29) <0.001
No
27
21 (23.3)
6 (3.3)
1.00

Types of assistance provided
Food products
Yes11427 (29.7)87 (47.8)0.46 (0.27–0.79) 0.004
No15964 (70.3)95 (52.2)1.00
Housing assistance
Yes156 (6.6)9 (4.9)1.36 (0.47–3.94)0.57
No25885 (93.4)173 (95.1)1.00
Free medications to treat side effects from anti-TB drugs
Yes19649 (53.8)147 (80.8)0.28 (0.16–0.48) <0.001
No7742 (46.2)35 (19.2)1.00
Money for transportation
Yes22459 (64.8)165 (90.7)0.19 (0.1–0.37) <0.001
No
49
32 (35.2)
17 (9.3)
1.00

Trust in, rapport with, and support from physicians and nursing staff
273
81.88 (15.56)‡
90.73 (7.79)‡
0.93 (0.91–0.96)§
<0.001
Information and support from healthcare center staff2727.53 (1.68)‡8.43 (1.22)‡0.64 (0.52–0.78)§ <0.001

*Boldface indicates significance. TB, tuberculosis.
†No (%) unless noted otherwise. 
‡Mean (SD). 
§Odds ratio is per 1 point increase in cumulative score.

*Boldface indicates significance. TB, tuberculosis.
†No (%) unless noted otherwise. 
‡Mean (SD). 
§Odds ratio is per 1 point increase in cumulative score. Among diagnosis and treatment factors, frequency of certain adverse drug reactions reported by patients did not differ significantly, except for less frequently reported diarrhea among case-patients (OR 0.49, 95% CI 0.29–0.85). However, scores for self-reported severity of adverse drug reactions were significantly higher among case-patients; reactions included vomiting (mean ± SD 5.23 ± 3.72 vs. 4.02 ± 3.46; p = 0.008), dizziness (5.48 ± 3.51 vs. 4.55 ± 3.14; p = 0.029), and fatigue or extreme tiredness (5.18 ± 3.56 vs. 4.14 ± 3.25; p = 0.017) (Tables 5, 6). Reported cost of travel to the treatment center during the intensive phase of treatment was significantly higher among case-patients than among control-patients (mean ± SD 98 ± 104 pesos vs. 71 ± 57 pesos, respectively, p = 0.035).
Table 5

Univariate analysis of diagnosis and treatment factors associated with loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

Characteristic, data from patient interviewsTotalCase-patients†Control-patients†Odds ratio (95% CI)p value
Side effects during MDR TB treatment‡
Nausea
Yes23074 (81.3)156 (85.7)0.73 (0.37–1.42)0.35
No4317 (18.7)26 (14.3)1.00
Vomiting
Yes21074 (81.3)136 (74.7)1.47 (0.79–2.75)0.22
No6317 (18.7)46 (25.3)1.00
Diarrhea
Yes19556 (61.5)139 (76.4)0.49 (0.29–0.85) 0.01
No7835 (38.5)43 (23.6)1.00
Headache
Yes21268 (75.6)144 (79.1)0.82 (0.45–1.48)0.50
No6022 (24.4)38 (20.9)1.00
Sleep disturbances
Yes23175 (83.3)156 (85.7)0.83 (0.42–1.67)0.61
No4115 (16.7)26 (14.3)1.00
Tingling/pain in hands or feet
Yes18755 (61.1)132 (72.5)0.6 (0.35–1.02)0.06
No8535 (38.9)50 (27.5)1.00
Hearing problems
Yes20268 (75.6)134 (73.6)1.11 (0.62–1.98)0.73
No7022 (24.4)48 (26.4)1.00
Dizziness
Yes23175 (83.3)156 (85.7)0.83 (0.42–1.67)0.61
No4115 (16.7)26 (14.3)1.00
Nervousness/anxiety
Yes16051 (56.7)109 (59.9)0.88 (0.53–1.46)0.61
No11239 (43.3)73 (40.1)1.00
Pain in joints
Yes22168 (75.6)153 (84.1)0.59 (0.31–1.09)0.09
No5122 (24.4)29 (15.9)1.00
Vision problems
Yes15553 (58.9)102 (56)1.12 (0.67–1.87)0.66
No11737 (41.1)80 (44)1.00
Fatigue/extreme tiredness
Yes20870 (77.8)138 (75.8)1.12 (0.61–2.04)0.72
No
64
20 (22.2)
44 (24.2)
1.00

Participant travel expenses
Cost to travel to treatment center, intensive phase of treatment23997.58 (103.64)§70.83 (57.24)§1.00 (1.00–1.01¶ 0.035
Cost to travel to treatment center, continuation phase of treatment13867.93 (65.24)§52.46 (39.37)§1.01 (0.00–1.02)¶0.383
Source of funds to travel to/from treatment center, intensive phase of treatment
Own/personal money
Yes20368 (74.7)135 (74.2)1.03 (0.58–1.83)0.92
No7023 (25.3)47 (25.8)1.00
TB program funds
Yes18348 (52.7)135 (74.2)0.39 (0.23–0.66) <0.001
No9043 (47.3)47 (25.8)1.00
Borrowed money
Yes9938 (41.8)61 (33.5)1.42 (0.85–2.39)0.18
No17453 (58.2)121 (66.5)1.00

*Boldface indicates significance. TB, tuberculosis.
†No. (%) unless noted otherwise.
‡Participants were asked to rate severity of medication side effects on a scale of 1–10 (0 = absence of symptoms; 1 = very mild, 10 = extremely severe).
§Mean (SD). 
¶Odds ratio is per 1 point increase in cost unit.

Table 6

Rating of the severity of medication side effects experienced during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

Side effect
Total
Case-patients, mean (SD) score
Control-patients, mean (SD) score
Odds ratio (95% CI)†
p value
Nausea2735.05 (3.51)4.42 (3.18)1.06 (0.98–1.14)0.136
Vomiting2735.23 (3.72)4.02 (3.46)1.10 (1.02–1.18) 0.008
Diarrhea2733.34 (3.41)3.74 (3.18)0.96 (0.89–1.04)0.345
Headache2724.37 (3.36)4.20 (3.23)1.02 (0.94–1.10)0.689
Sleep disturbances2725.23 (3.59)5.31 (3.26)0.99 (0.92–1.07)0.854
Tingling/pain in hands or feet2723.58 (3.53) 4.14 (3.56)0.96 (0.89–1.03)0.218
Hearing problems2724.49 (3.47)4.13 (3.46)1.03 (0.96–1.11)0.417
Dizziness2725.48 (3.51)4.55 (3.14)1.09 (1.01–1.18) 0.029
Nervousness/anxiety2723.26 (3.34)3.01 (3.16)1.02 (0.95–1.11)0.548
Skin problems or rash2722.70 (3.26)2.69 (3.22)1.00 (0.93–1.08)0.975
Joint pain2725.14 (3.57)5.49 (3.34)0.97 (0.90–1.04)0.427
Vision problems2723.10 (3.07)2.80 (3.05)1.03 (0.95–1.12)0.45
Fatigue/extreme tiredness2725.18 (3.56)4.14 (3.25)1.10 (1.02–1.18) 0.017

*Boldface indicates significance. TB, tuberculosis.
†Odds ratio is per 1 point increase in severity rating score.

*Boldface indicates significance. TB, tuberculosis.
†No. (%) unless noted otherwise.
‡Participants were asked to rate severity of medication side effects on a scale of 1–10 (0 = absence of symptoms; 1 = very mild, 10 = extremely severe).
§Mean (SD). 
¶Odds ratio is per 1 point increase in cost unit. *Boldface indicates significance. TB, tuberculosis.
†Odds ratio is per 1 point increase in severity rating score. Univariate analysis indicated that social factors significantly associated with loss to follow-up were self-reported lack of time to go to the treatment facility (OR 2.23, 95% CI 1.1–4.54) and absence of someone to accompany the patient to the treatment facility during the intensive phase of treatment (OR 1.97, 95% CI 1.06–3.65). Scores reflecting patient self-stigmatization among case-patients and control-patients did not differ significantly (p = 0.10) (Table 7).
Table 7

Univariate analysis of social factors associated with loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

CategoryTotalCase-patients†Control-patients†Odds ratio (95% CI)p value
Distance between participant's home and treatment center, intensive phase of treatment
Comparison 1
0 to <1 km237 (23.3)16 (31.4)0.67 (0.24–1.87)0.44
1 to <5 km (referent)5823 (76.7)35 (68.6)1.00
Comparison 2
5 to <10 km4613 (36.1)33 (48.5)0.6 (0.26–1.37)0.23
1 to <5 km (referent)5823 (63.9)35 (51.5)1.00
Comparison 3
>10 km10438 (62.3)66 (65.3)0.88 (0.45–1.7)0.69
1 to <5 km (referent)5823 (37.7)35 (34.7)1.00
Comparison 4
Not sure/don't know4210 (30.3)32 (47.8)0.48 (0.2–1.15)0.10
1 to <5 km (referent)
58
23 (69.7)
35 (52.2)
1.00

Usual mode of transportation/transportation used to cover the greatest distance traveling to the treatment center, intensive phase of treatment
Walk
Yes3719 (20.9)18 (9.9)2.4 (1.19–4.85) 0.01
No23672 (79.1)164 (90.1)1.00
Public transportation
Yes23979 (86.8)160 (87.9)0.91 (0.43–1.92)0.80
No3412 (13.2)22 (12.1)1.00
Personal vehicle
Yes166 (6.6)10 (5.5)1.21 (0.43–3.45)0.72
No
257
85 (93.4)
172 (94.5)
1.00

Major challenges when traveling to the treatment center, intensive phase of treatment
The center was far away
Yes14152 (57.8)89 (48.9)1.43 (0.86–2.38)0.17
No13138 (42.2)93 (51.1)1.00
Did not always have money for transportation
Yes19371 (78)122 (67.4)1.72 (0.96–3.08)0.07
No7920 (22)59 (32.6)1.00
Did not have the time to go for treatment
Yes3618 (19.8)18 (9.9)2.23 (1.1–4.54) 0.02
No23673 (80.2)163 (90.1)1.00
Going for treatment caused problems with work or school
Yes8133 (36.3)48 (26.5)1.58 (0.92–2.71)0.10
No19158 (63.7)133 (73.5)1.00
Did not have anyone to go with
Yes5224 (26.4)28 (15.4)1.97 (1.06–3.65) 0.03
No22167 (73.6)154 (84.6)1.00
The center’s hours were not convenient
Yes2311 (12.1)12 (6.6)1.95 (0.82–4.6)0.12
No
250
80 (87.9)
170 (93.4)
1.00

Minutes to travel from home to treatment center, intensive phase of treatment27251.00 (43.56)‡54.16 (45.03)‡1.00 (0.99–1.00)§0.583
Minutes to travel from home to treatment center, continuation phase of treatment19822.25 (15.93)‡31.07 (36.94)‡0.99 (0.97–1.01)§0.057
Patient self-stigmatization2726.20 (2.76)‡5.66 (2.44)‡1.09 (0.98–1.20)§0.104

*Boldface indicates significance. TB, tuberculosis.
†No. (%) unless noted otherwise.
‡Mean (SD). 
§Odds ratio is per 1 point increase in unit.

*Boldface indicates significance. TB, tuberculosis.
†No. (%) unless noted otherwise.
‡Mean (SD). 
§Odds ratio is per 1 point increase in unit. Independent factors positively associated with loss to follow-up were alcohol abuse (OR 2.84, 95% CI 1.39–5.80) and patient higher self-rating of vomiting severity (OR 1.10, 95% CI 1.01–1.21, per 1 point in severity rating score). Factors protective against loss to follow-up were receipt of any type of assistance from the TB program (OR 0.09, 95% CI 0.03–0.25); better general TB knowledge (OR 0.97, 95% CI 0.95–0.99, per 1 point in cumulative score); and higher levels of trust in, rapport with, and support from physicians and nursing staff (OR 0.93, 95% CI 0.90–0.96, per 1 point in cumulative score) (Table 8).
Table 8

Multivariable analysis of factors associated with loss to follow-up during treatment for multidrug-resistant TB, the Philippines, 2012–2014*

Social ecologic model level, factorOdds ratio (95% CI)p value
Personal factors
Score TB knowledge0.97 (0.95–0.99)†0.003
Alcohol abuse
2.84 (1.39–5.80)
0.004
Healthcare setting factors
Received assistance from TB program0.09 (0.03–0.25)<0.001
Score trust/rapport with healthcare worker
0.93 (0.90–0.96)†
<0.001
Diagnosis and treatment factors
Self-rated severity of vomiting as adverse drug reaction1.10 (1.01–1.21)‡0.03

*TB, tuberculosis. 
†Odds ratio is per 1 point increase in cumulative score.
‡Odds ratio is per 1 point increase in severity rating score.

*TB, tuberculosis. 
†Odds ratio is per 1 point increase in cumulative score.
‡Odds ratio is per 1 point increase in severity rating score. The primary reason for stopping treatment, most commonly reported by case-patients, was medication side effects or the fear of side effects, reported by 52 (58%) of 89 case-patients who responded to this question. The 2 other most commonly self-reported reasons for loss to follow-up were need to work and financial problems, reported by 25 (28%) of 89 patients, and lack of money for transportation to the treatment facility, reported by 18 (20%) of 89 patients.

Discussion

This large study, guided by a 5-level theoretical social ecologic model, demonstrated that loss to follow-up from MDR TB treatment in the Philippines was independently associated with 2 individual factors (general TB knowledge and alcohol abuse), 2 healthcare setting factors (receiving any type of assistance from the TB program and levels of trust in, rapport with, and support from physicians, nursing staff and other healthcare workers in the treatment facilities), and 1 diagnosis and treatment factor (higher self-rated severity of vomiting as an adverse drug reaction). Multivariable analysis did not identify any interpersonal or social factors associated with loss to follow-up. The most commonly reported primary reason for loss to follow-up was medication side effects or fear of side effects. This study demonstrated that general TB knowledge was significantly lower among case-patients than among control-patients. Although nonadherence to treatment rarely results from patient apathy, patients’ lack of knowledge of their medical condition and its treatment is associated with poor health outcomes (). For this reason, patient education is a valuable component of TB control. A recent systematic review and meta-analysis demonstrated that provision of patient education was a strategy associated with lower rates of loss to follow-up (). Among TB patients, interventions to improve general TB knowledge are significantly associated with better outcomes (). Knowledge of the bacterial causation of TB, mode of transmission, diagnostic testing, meaning of test results, rationale for prolonged treatment, and effect of treatment interruptions should be clearly explained to patients and their loved ones. It is also crucial to educate patients about expected adverse events before starting treatment. Patient education that addresses commonly held misperceptions about TB may also discourage patients from consulting traditional healers, thereby avoiding delayed diagnosis and treatment (). Alcohol abuse was recorded in medical records at a significantly higher frequency for case-patients than for control-patients. The association between alcohol abuse or alcohol use disorders and loss to follow-up during MDR TB treatment has been demonstrated in multiple studies (–,); 1 small randomized clinical trial demonstrated improved TB outcomes for patients in groups randomly assigned to receive naltrexone or behavioral counseling integrated into TB care (). This finding, when combined with similar findings in other studies, calls for additional studies to assess the effect of using standard assessment tools to screen for alcohol dependence and of managing alcohol use as part of TB clinical services. Receiving assistance from the TB program, including such measures as covering the cost of transportation, food, and housing, was associated with improved treatment adherence. However, if the process of applying for financial assistance is long and difficult and if this assistance is not given in a timely manner or regularly, patients may abandon their efforts to adhere to treatment. Decentralization of treatment (i.e., the transfer of care from a centralized MDR TB treatment center to a community DOTS facility) was protective; odds of being lost to follow-up decreased by 10 times (). This finding suggests that the decentralization of care into multiple treatment facilities closer to patients’ homes, which was used as part of the PMDT scale-up scheme in the Philippines, is a valid strategy for improving TB treatment patient retention. In addition to decentralization, the National TB Control Program began piloting the Community-Based PMDT Care Initiative in 2014 to provide a more accessible venue for management of MDR TB in the patient’s home. The effectiveness of this initiative in improving treatment adherence should be rapidly evaluated, and the evaluation results should be used for future program planning. All modalities for addressing patient barriers should be considered. Adherence to MDR TB treatment might be improved by providing sufficient and timely financial assistance to patients (especially for transportation) by augmenting current enablers and providing livelihood programs during and after MDR TB treatment through strategic multisectoral partnership. We found that levels of trust in, rapport with, and support from physicians, nursing staff, and other caregivers in the treatment facilities were significantly lower among patients lost to follow-up than among control-patients. A higher degree of trust, good rapport, and support from providers has been shown to be associated with patients’ adherence to medical recommendations and with improvements to self-reported and objective measures of health (–). In a study by Holtz et al. in South Africa, the strongest individual risk factor for nonadherence to MDR TB treatment was having an unsatisfactory opinion about the attitude of the healthcare workers (). Interventions focused on enhancing provider–patient mutual trust and respect are needed. Provider training with regard to active listening, health literacy, message-framing, motivational interviewing, communication skills for trust-building and sensitivity should be considered. We found that patients’ higher rating of the severity of their vomiting was independently associated with loss to follow-up. Moreover, the most commonly reported reason for stopping treatment (58%) was medication side effects. However, frequency of adverse drug reactions reported by patients in interviews did not differ significantly between case-patients and control-patients (except that diarrhea symptoms were reported significantly less often by case-patients than by control-patients). Still, case-patients reported significantly higher subjectively perceived severity of vomiting, dizziness, and fatigue as medication side effects. These symptoms affect quality of life and interfere with the capacity to work and the ability to engage in activities of daily living. A study of 583 MDR TB patients in the Philippines who had undergone treatment during 1999–2006 showed that taking >5 drugs was significantly associated with loss to follow-up compared with taking 2–3 drugs (). The authors interpreted the association between a higher drug burden and loss to follow-up as being related to more extensive drug resistance and competing risk for death among those patients. However, it also might be related to experiencing more side effects by the patients who received a higher number of toxic second-line drugs, which more likely led to stopping treatment, than by patients who received a lower number of drugs. Our study suggests that strict monitoring for, and appropriate treatment of, adverse drug reactions may help improve treatment adherence. Ancillary drugs must be included in procurement plans and made widely available at treatment facilities. Initial and refresher trainings for healthcare providers about management of adverse drug reactions and patient education about expected adverse drug reactions before treatment initiation may also help improve treatment adherence. Patients should understand the need for strict monitoring for adverse drug reactions and the availability of effective and free treatment for those reactions, especially reactions that are subjectively difficult for patients (e.g., nausea, vomiting, dizziness, and extreme fatigue). Our study is subject to several limitations. As with any case–control study, it provides relatively weak empirical evidence. Because patient interviews were part of the protocol, deceased patients were excluded, which might have introduced survivor biases into our results. Almost a third of selected control-patients could not be located, which could bias the results in favor of selecting control-patients with a stronger relationship to the clinics. The retrospective nature of interviews is also subject to recall and exposure misclassification biases. Interview responses may reflect socially desirable answers rather than true thoughts and experiences. Some patients who were adherent to treatment at the time of the interview may be lost to follow-up at a later time (in this study, 66% of control-patients were still receiving treatment at the time of the interview); thus, outcome misclassification is possible. Identification of alcohol abuse was based on the records in medical charts without standardized assessment for each patient (such as the Alcohol Use Disorder Identification Test) (). Previous studies have demonstrated that loss to follow-up from treatment was less in smaller cohorts () and was more with program scale-up (). Thus, increased loss to follow-up may be related to healthcare setting structural factors such as insufficient number of facilities and providers or limited experience with management of MDR TB, but our study did not address those factors. Despite these limitations, our study provides useful data. The interviews captured patients’ perspectives and provided nuances that retrospective cohort studies lack (,,–). These data, along with medical record data, afford program leaders greater insight for improving services and designing patient-centered interventions to reduce loss to follow-up during MDR TB treatment in the Philippines. A revision of the PMDT strategy should address identified barriers to completing MDR TB treatment and implement actions that support patients’ adherence to treatment.

Technical Appendix

Multivariable analysis used to assess factors associated with loss to follow-up during treatment for multidrug-resistant tuberculosis, the Philippines, 2012–2014.
  25 in total

Review 1.  Social ecological approaches to individuals and their contexts: twenty years of health education & behavior health promotion interventions.

Authors:  Shelley D Golden; Jo Anne L Earp
Journal:  Health Educ Behav       Date:  2012-01-20

2.  Risk factors associated with default among new pulmonary TB patients and social support in six Russian regions.

Authors:  W M Jakubowiak; E M Bogorodskaya; S E Borisov; E S Borisov; I D Danilova; D I Danilova; E V Kourbatova; E K Kourbatova
Journal:  Int J Tuberc Lung Dis       Date:  2007-01       Impact factor: 2.373

3.  Predictors and mortality associated with treatment default in pulmonary tuberculosis.

Authors:  K Kliiman; A Altraja
Journal:  Int J Tuberc Lung Dis       Date:  2010-04       Impact factor: 2.373

4.  Self-efficacy: toward a unifying theory of behavioral change.

Authors:  A Bandura
Journal:  Psychol Rev       Date:  1977-03       Impact factor: 8.934

5.  Barriers to successful tuberculosis treatment in Tomsk, Russian Federation: non-adherence, default and the acquisition of multidrug resistance.

Authors:  I Y Gelmanova; S Keshavjee; V T Golubchikova; V I Berezina; A K Strelis; G V Yanova; S Atwood; M Murray
Journal:  Bull World Health Organ       Date:  2007-09       Impact factor: 9.408

6.  DOTS-Plus for multidrug-resistant tuberculosis in the Philippines: global assistance urgently needed.

Authors:  T E Tupasi; M I D Quelapio; R B Orillaza; C Alcantara; N R C Mira; M R Abeleda; V T Belen; N M Arnisto; A B Rivera; E R Grimaldo; J O Derilo; W Dimarucut; M Arabit; D Urboda
Journal:  Tuberculosis (Edinb)       Date:  2003       Impact factor: 3.131

Review 7.  Treatment outcomes among patients with multidrug-resistant tuberculosis: systematic review and meta-analysis.

Authors:  Evan W Orenstein; Sanjay Basu; N Sarita Shah; Jason R Andrews; Gerald H Friedland; Anthony P Moll; Neel R Gandhi; Alison P Galvani
Journal:  Lancet Infect Dis       Date:  2009-03       Impact factor: 25.071

8.  Impact of patient and program factors on default during treatment of multidrug-resistant tuberculosis.

Authors:  M T Gler; L J Podewils; N Munez; M Galipot; M I D Quelapio; T E Tupasi
Journal:  Int J Tuberc Lung Dis       Date:  2012-05-07       Impact factor: 2.373

9.  Loss-To-Follow-Up on Multidrug Resistant Tuberculosis Treatment in Gujarat, India: The WHEN and WHO of It.

Authors:  Kalpita S Shringarpure; Petros Isaakidis; Karuna D Sagili; R K Baxi
Journal:  PLoS One       Date:  2015-07-13       Impact factor: 3.240

10.  Tuberculosis patients' knowledge and beliefs about tuberculosis: a mixed methods study from the Pacific Island nation of Vanuatu.

Authors:  Kerri A Viney; Penelope Johnson; Markleen Tagaro; Saen Fanai; Nguyen N Linh; Paul Kelly; David Harley; Adrian Sleigh
Journal:  BMC Public Health       Date:  2014-05-17       Impact factor: 3.295

View more
  27 in total

1.  Outcomes of multidrug-resistant tuberculosis in Zambia: a cohort analysis.

Authors:  Nathan Kapata; Martin P Grobusch; Gershom Chongwe; Pascalina Chanda-Kapata; William Ngosa; Mathias Tembo; Shebba Musonda; Patrick Katemangwe; Matthew Bates; Peter Mwaba; Alimuddin Zumla; Frank Cobelens
Journal:  Infection       Date:  2017-08-04       Impact factor: 3.553

2.  Treatment Outcomes and Adverse Events from a Standardized Multidrug-Resistant Tuberculosis Regimen in a Rural Setting in Angola.

Authors:  María Luisa Aznar; Ariadna Rando Segura; María Milagros Moreno; Mateu Espasa; Elena Sulleiro; Cristina Bocanegra; Eva Gil Olivas; Arlete Nindia Eugénio; Adriano Zacarias; Domingos Katimba; Estevao Gabriel; Jacobo Mendioroz; Maria Teresa López García; Tomas Pumarola; María Teresa Tórtola; Israel Molina
Journal:  Am J Trop Med Hyg       Date:  2019-09       Impact factor: 2.345

3.  Risk factors for poor engagement in drug-resistant TB care in South Africa: a systematic review.

Authors:  K C McNabb; A Bergman; J E Farley
Journal:  Public Health Action       Date:  2021-09-21

4.  Factors contributing to the high prevalence of multidrug-resistance/Rifampicin-resistance in patients with tuberculosis: an epidemiological cross sectional and qualitative study from Khabarovsk krai region of Russia.

Authors:  Ilia Bykov; Olga Dyachenko; Pavel Ratmanov; Huan Liu; Libo Liang; Qunhong Wu
Journal:  BMC Infect Dis       Date:  2022-07-13       Impact factor: 3.667

5.  Strengthening the response to drug-resistant TB in Pakistan: a practice theory-informed approach.

Authors:  S Abbas; M Kermode; S Kane
Journal:  Public Health Action       Date:  2020-12-21

6.  Predictors of mortality and loss to follow-up among drug resistant tuberculosis patients in Oromia Hospitals, Ethiopia: A retrospective follow-up study.

Authors:  Demelash Woldeyohannes; Yohannes Tekalegn; Biniyam Sahiledengle; Tesfaye Assefa; Rameto Aman; Zeleke Hailemariam; Lillian Mwanri; Alemu Girma
Journal:  PLoS One       Date:  2021-05-06       Impact factor: 3.240

7.  Multidrug-resistant tuberculosis patients' views of interventions to reduce treatment loss to follow-up.

Authors:  T Tupasi; A M C G Garfin; J M Mangan; R Orillaza-Chi; L C Naval; G I Balane; R Basilio; A Golubkov; E S Joson; W-J Lew; V Lofranco; M Mantala; S Pancho; J N Sarol; A Blumberg; D Burt; E V Kurbatova
Journal:  Int J Tuberc Lung Dis       Date:  2017-01-01       Impact factor: 2.373

8.  Multidrug-resistant tuberculosis patients lost to follow-up: self-reported readiness to restart treatment.

Authors:  J M Mangan; T E Tupasi; A M C G Garfin; V Lofranco; R Orillaza-Chi; R Basilio; L C Naval; G I Balane; E S Joson; D Burt; W-J Lew; M Mantala; S Pancho; J N Sarol; A Golubkov; E V Kurbatova
Journal:  Int J Tuberc Lung Dis       Date:  2016-09       Impact factor: 2.373

9.  Factors Associated with Medical Follow-Up Adherence for Patients on All-Oral Regimen for Multidrug-Resistant Tuberculosis in Shenzhen, China.

Authors:  Hui Li; Hailin Zhang; Juan Xiong; Yi Wang; Weiyu Wang; Jingjing Wang; Yi Lin; Peize Zhang
Journal:  Patient Prefer Adherence       Date:  2021-07-05       Impact factor: 2.711

10.  Patient- and provider-related factors in the success of multidrug-resistant tuberculosis treatment in Colombia.

Authors:  Gloria Mercedes Puerto Castro; Fernando Nicolás Montes Zuluaga; Jacqueline Elizabeth Alcalde-Rabanal; Freddy Pérez
Journal:  Rev Panam Salud Publica       Date:  2021-06-21
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.