Evaluating the sustained health impact of household chlorination of drinking water in rural Haiti.

The Jolivert Safe Water for Families program has sold sodium hypochlorite solution (chlorine) and conducted household visits in rural Haiti since 2002. To assess the impact of the program on diarrheal disease, in 2010 we conducted a survey and water quality testing in 201 program participants and 425 control households selected at random. Fifty-six percent of participants (versus 10% of controls) had free chlorine residuals between 0.2 and 2.0 mg/L, indicating correct water treatment. Using intention-to-treat analysis, we found that significantly fewer children < 5 in participant households had an episode of diarrhea in the previous 48 hours (32% versus 52%; P < 0.001) with 59% reduced odds (odds ratio = 0.41, 95% confidence interval = 0.21-0.79). Treatment-on-treated estimates of the odds of diarrhea indicated larger program effects for participants who met more stringent verifications of participation. Diarrheal disease reduction in this long-term program was comparable with that seen in short-term randomized, controlled interventions, suggesting that household chlorination can be an effective long-term water treatment strategy.

Introduction

Haiti has the lowest rates of access to safe drinking water and sanitation in the western hemisphere.1 As described in the 2005–2006 Demographic and Health Surveys, 11% of households have access to a private sanitation facility that safely separates fecal waste from the environment whereas 50% of the population defecates in the open.2 Poor sanitation leads to contamination of drinking water sources. In rural Haiti, 50% of households use unimproved drinking water sources such as dug wells, unprotected springs, and surface water.2 Although 21% of rural households self-report adding bleach or chlorine to disinfect their water and 3% report boiling or other treatment methods, 76% report that they do not treat their water.2 The World Health Organization states that diarrheal disease is the second leading cause of death in children < 5 years of age in Haiti.3 In rural households, 25% of children < 5 years of age were reported to have had diarrhea in the past two weeks.2

The Safe Water System (SWS) was developed by the Centers for Disease Control and Prevention (CDC) and the Pan American Health Organization as a method to safely treat drinking water in the home. The SWS consists of three components: 1) water treatment with sodium hypochlorite solution (chlorine); 2) safe storage of household drinking water; and 3) education and behavior change messages to encourage safe household water, sanitation, and hygiene practices.4 The sodium hypochlorite solution is packaged in a bottle with directions instructing users to add one bottle cap of the solution to clear water in a 20-liter storage container, agitate, and wait 30 minutes before drinking. The dosage is calculated to inactivate most diarrheal disease-causing pathogens and provides residual protection from recontamination.5 In six randomized, controlled trials, the SWS was shown to reduce diarrhea by 22–84% in children and adults.6–11 Arnold and Colford found in a meta-analysis that household chlorination reduced the risk of diarrhea in children < 5 years of age by 29% (risk ratio = 0.71, 95% confidence interval [CI] = 0.58–0.87).12

In September 2002, an SWS program was established at the Missions of Love (MOL) health clinic in the rural, northwestern community of Jolivert, Haiti (Figure 1). The non-governmental organization Deep Springs International (DSI) (Léogâne, Haiti) currently operates the program and trains Haitian technicians to 1) manufacture quality-controlled chlorine solution (Gadyen Dlo) (“Water Guardian” in Haitian Kreyòl); 2) enroll participating families through the sale of safe storage containers consisting of modified buckets with lids and taps (Figure 2); 3) sell Gadyen Dlo to participating families; 4) maintain sales records for each participating family; and 5) conduct regular household visits to monitor Gadyen Dlo use and provide ongoing education. As of May 2010, the Jolivert Safe Water for Families (JSWF) program had reached 4,253 participants since program inception through a distribution network of technicians and resellers. Approximately 48,000 chlorine refills, enough to treat 12,000,000 liters of water or approximately 11 240-mL bottles per family on average, have been sold by the JSWF program (Turbes A, unpublished data). Currently, the JSWF program is financially sustainable, with chlorine sales funding chlorine production, supplies, and staff costs. However, financial inputs into the program were invested to purchase the electrolytic generator and increase access by subsidizing buckets and program entry. In addition, all evaluations, including site visits to the program, are externally funded.

Figure 1.

Map of survey households for whom global positioning system coordinates were recorded.

Figure 2.

Jolivert Safe Water for Families safe storage container and sodium hypochlorite bottle (Source: Michael Ritter, Deep Springs International, Léogâne, Haiti).

The extensive existing research documenting the significant health impact of household chlorination is based primarily on randomized, controlled intervention trials lasting no more than a year. In a programmatic setting, the effectiveness of household chlorination is less well understood. Critics claim that in the long-term, chlorine uptake decreases and health impact becomes negligible.13,14 Another critique is that household chlorination programs are ineffective unless promoted alongside sanitation improvements.15 Consequently, some researchers have argued that widespread promotion of household water treatment to reduce diarrhea is premature until further high-quality studies are conducted.16

This study is an impact evaluation that assesses consistency of use and diarrheal disease reduction in the JSWF long-term household chlorination program in rural Haiti, and addresses the following questions. First, do households enrolled in the program have significantly higher chlorine use than non-enrolled control households? Second, do households enrolled in the program have significantly reduced point prevalence of diarrhea than non-enrolled control households, while adjusting for household characteristics that may confound the relationship between program enrollment and diarrheal disease?

In contrast to randomized, controlled trials, which randomly allocate participants and controls to their respective groups and typically provide all intervention components, this study was an intention-to-treat analysis of the health impact of an existing cost-recovery program. Intention-to-treat analyses are based on the initial treatment intent and do not adjust for differences in actual treatment status within the intended treatment group, which can arise when not all persons who are offered a program choose to adopt or maintain program activities. In this study, households that were entered in program records were considered to be participants for the purposes of the intention-to-treat analysis, regardless of whether they were actually treating their water. Participant health outcomes were compared with those of non-participants while adjusting for potential confounders. A treatment-on-treated analysis was also conducted, which considered only participants who were actually treating their water (as confirmed by positive free chlorine residual in the household's stored drinking water). These analyses provided valuable insight into the actual health impact of the program for all intended program members and for those who were confirmed program users.

Methods

Setting.

This study was conducted during May–June 2010 in the Nord-Ouest (North-West) and Artibonite Departments of Haiti. This study was conducted after the earthquake of January 12, 2010, in departments not significantly directly affected by the earthquake, but before the onset of cholera in the area in October 2010.

Population and sample.

The target population was households living less than three hours by motorbike from the MOL clinic, in which the JSWF program is based. Since program inception in September 2002 up to May 2010, 4,253 households were enrolled. The total population in the administrative communes where the JSWF program operates is 385,106, which consists of 63,857 rural households.17 Program coverage in this area was therefore approximately 6.7%. The JSWF technicians maintain paper records of bucket purchases, chlorine purchases, and the results of free chlorine residual tests during routine household visits. Households with a recorded bucket or chlorine purchase or household visit were considered enrolled in the program (participants). The participant household list was entered from paper records into Microsoft (Redmond, WA) Excel 2007 to develop the sampling frame. Duplicate entries were identified in Excel using pivot tables and removed.

Records were excluded from the sampling frame if 1) the record referred to a group, such as a school, church, police station, or clinic; 2) the household was more than three hours away from the MOL clinic; 3) the household was missing necessary information for an enumerator to locate them (i.e. family name or community name); or 4) the household joined the program in the previous three months. After a random sample was drawn, consultations with DSI staff led to the exclusion of an additional 31 households because the program coordinator could not provide clear directions to the enumerators on how to get to a particular village, or the household was in a larger city such as Port-de-Paix, where the recorded information was insufficient to locate the person. In these cases, the randomly sampled household was replaced with the subsequent household on the randomly sorted list. Enumerators visited each household on the list until reaching the target sample size of 200 participant households.

The first control household was selected by walking three houses to the right of the participant household's dwelling when facing away from the doorstep; the second control was selected by walking three additional houses in the same direction. For all control households, if a household member was not home, the enumerator selected the next closest household whose dwelling appeared to be made of similar construction materials (i.e., a proxy for similar socioeconomic status).

Survey procedures.

The survey instrument contained 47 questions and took 10–15 minutes to complete. Informed consent was obtained from all participants. Approval was requested and an exemption was granted by the institutional review board (IRB) of Emory University. The IRB of CDC deferred to the Emory IRB decision. The survey was translated into Kreyòl and back-translated to English to verify translation accuracy. Enumerators were instructed to state in general terms that they were conducting research about drinking water, with no mention of the program except in the survey questions. Topics included household demographics, assets, water sources, water treatment methods, Gadyen Dlo knowledge and use, treatment of diarrhea, and latrine use. Using a list of the ages and sexes of all household members, we asked survey respondents whether each person had had an episode of diarrhea, which was defined as ≥ 3 loose or watery stools in a 24-hour period, during the last 48 hours. As an indicator of dysentery, enumerators also asked whether blood was present in the stools of any person who was reported to have had experienced an episode of diarrhea.

If the respondent reported purchasing Gadyen Dlo, the enumerator measured the amount of Gadyen Dlo remaining in the bottle using a graduated cylinder. In each home, regardless of reported chlorine use, drinking water was tested for free chlorine residual using a Hach® Color Wheel Chlorine Test Kit (Hach Company, Loveland, CO). Global positioning system coordinates were recorded at the entrance of each home using a Garmin® eTrex Vista 360° (Garmin International, Inc., Olathe, KS).

Data were entered into Excel at the conclusion of each day of survey collection. Written-in responses were translated to English by the researchers with the assistance of the research coordinator/translator. Quality of data entry was verified by cross-checking implausible responses in the database with the original written surveys. Mistakes were corrected in the database if the intended responses could be determined from the context.

Analysis of survey data.

The survey data were cleaned and analyzed in Stata version 11.1 (StataCorp LP, College Station, TX). An intention-to-treat analysis was initially used in the analysis; i.e., all households with records indicating enrollment in the JSWF program (participants) and all corresponding controls were included in the analysis regardless of whether they reported using Gadyen Dlo at the time of the survey or whether they actually had a positive free chlorine residual in their drinking water. An asset index was derived using a principal component analysis for variables related to socioeconomic status.18

Because diarrhea-related questions were asked for each person in every household, analysis of health outcomes treated each person as an observation rather than each household. Thus, if a household was a participant, all persons in that household were considered participants with identical household characteristics but with unique responses to age, sex, diarrhea, and the presence of blood in feces. The prevalences of diarrhea and bloody stools were calculated among all persons and among children < 5 years of age.

Odds ratios (ORs) with 95% CIs were calculated to determine the effect of being a program participant on having diarrhea and bloody diarrhea. Because our outcome measures were binary variables, we used multivariate logistic regression models to adjust for potential confounders in the relationship between diarrheal disease and program participation. We accounted for clustering at the household level using the cluster option in Stata version 11.1, which enables intracluster correlation but maintains the assumption that clusters are independent of one another. We estimated a basic model in which diarrhea was regressed on program participation for the entire sample, and a full model including all potential confounders available in our dataset. These models were constructed for persons of all ages and for children < 5 years of age by using diarrhea and bloody diarrhea as outcome measures. Covariates in the full model included respondent sex, asset index, time to collect water, turbidity in the household drinking water supply, presence of soap in the household, reported use of an improved drinking water source, use of a safe storage container, and reported use of a latrine by children in the household.

We also tested whether the effects of program enrollment were modified by household characteristics. These characteristics included soap in the household, reported use of an improved drinking water source, safe storage of drinking water in the home, confirmed positive chlorine residual in current drinking water, and reported use of a latrine by children in the household.

We then refined our estimates of the health effects of program participation to account for the fact that not all households categorized as participants actually had treated water at the time of the unannounced survey visit. This approach, commonly referred to as the effects of the treatment on the treated, honed in on the health benefits experienced by those who actually adopted the program. Despite the benefits of this approach, selection biases were a larger concern because there could have been other unobservable characteristics that were correlated with the decision to chlorinate, which might have also affected health outcomes.

We verified participation using multiple indicators including 1) self-reports of always treating the household's water with Gadyen Dlo, 2) record of a household visit by a technician, 3) record of a chlorine purchase within the past year, 4) chlorine in the household at the unannounced survey visit, and 5) positive free chlorine residual in household stored water on the day of the unannounced survey. For each of these indicators, we compared participants who met the criteria to all controls, including controls who might have also met the criteria. This feature was the most conservative estimate of the effects of treatment on treated persons, and did not account for the possibility that the program activities could also have inadvertently increased chlorination among the control group.

Lastly, because more than just JSWF participants practiced chlorination, we examined the odds of diarrhea among all persons (both participant and control households) with positive free chlorine residuals in their stored household drinking water. We also examined differences in the proportion of households with diarrhea by asset quintile and having chlorinated water.

Results

Sample selection, demographics, and water behaviors.

Sample selection.

Between program inception in September 2002 and May 2010, 4,253 households were enrolled from 182 communities. After applying the exclusion criteria explained above, 2,670 households were included in the sampling frame. The target sample size was 200 participant households with 400 control households. Sixty-six participant household surveys were not completed because the household members had recently moved away or died (35 households), they could not be found by the enumerators after two attempts on different days (29), or they refused to complete the survey (2). A total of 201 participant and 425 matched control households were surveyed (Figure 1), consisting of 3,122 persons. These households lived in 72 communities, with between 1 and 24 households surveyed per community. Based on data from the program records, surveyed participant households were similar to the rest of the entries in the database despite the exclusion criteria (Supplemental Table 1). Participant households had been enrolled in the program for an average of 53 months at the time of the survey, compared with 54 months for non-surveyed records in the database.

Demographics.

Demographics between participant and control households were similar (Table 1). On average, households had 5–6 members and either one or no children < 5 years of age. However, there were a few minor but statistically significant differences between participants and controls in the directions that would be expected: among participants, more respondents attended school and more male household heads reported they could read, although female participant household heads were no less likely to be literate. Three times as many controls as participants practiced voodoo, although these were small shares of both groups. The asset index had similar medians and ranges for participants and controls, so although the t-test for the difference in means approached significance, there did not appear to be meaningful differences in socioeconomic status between the groups. Most respondents in both groups were female, though this proportion was significantly higher among controls than among participants (74% versus 64%). In summary, of 12 demographic characteristics considered, not including the 29 variables that were used to derive the asset index, 4 variables were significantly different between participants and controls. The proportion of households with specific asset indicators that made up the asset quintile are included in Supplemental Table 2.

Table 1

Household demographics and water collection and storage among program participants and controls, Haiti

Variable	Participants	Controls	P
No. (%) female respondents	128/201 (63.7)	316/425 (74.4)	0.006
Mean (SD) respondent age, years	40.5 (14.3)	39.8 (15.6)	0.602
No. (%) respondents who attended school	157/201 (78.1)	295/422 (69.9)	0.032
No. (%) male household heads who can read	145/190 (76.3)	267/394 (67.8)	0.034
No. (%) female household heads who can read	133/199 (66.8)	259/421 (61.5)	0.200
Mean (SD) household size	5.7 (2.3)	5.6 (2.2)	0.585
Mean (SD) number of children < 5 in household	0.54 (0.75)	0.57 (0.80)	0.712
Median (range) asset index	0.4 (–1.8 to 2.1)	0.3 (–2.1 to 2.1)	0.086
No. (%) respondents who practice voodoo	9/199 (4.5)	61/409 (14.9)	< 0.001
No. (%) using improved drinking water source	69/201 (34.3)	133/423 (31.4)	0.471
Collect water times per day, mean (SD)	2.2 (1.0)	2.5 (1.4)	0.010
Time to collect water and return, mean (SD)	25.1 (26.1)	28.4 (31.4)	0.194
No. (%) with soap available in household	109/190 (57.4)	209/386 (54.2)	0.465
No. (%) using a latrine (respondent and children)	162/201 (80.6)	315/424 (74.3)	0.083
No. (%) who believe their water is safe to drink	161/172 (93.6)	184/303 (60.7)	< 0.001
No. (%) using a safe storage container	53/201 (26.4)	28/425 (6.6)	< 0.001

Water collection and storage.

Use of an improved drinking water source did not significantly differ between participants and controls (Table 1). On average, respondents collected water 2–3 times per day, and slightly but significantly more trips were made by control households. Significantly more participants (94% versus 61%; P < 0.001) believed that their drinking water was safe to drink. The most commonly volunteered reason why they believed their water was safe to drink is that it was free from bacteria. A larger but not significantly different percentage of controls incorrectly believed that because their water was clear it was safe to drink.

Gadyen Dlo use.

Respondents were asked to self-report all water treatment methods that they had heard of and how often they used each of these methods. The most frequently reported methods were boiling, Aquatabs, citrus, commercial bleach products such as Clorox and Jif, and Gadyen Dlo (Supplemental Figure 1). Significantly more controls than participants (45% versus 22%; P < 0.001) reported having heard of a commercial bleach product other than Gadyen Dlo being used to treat water (Supplemental Table 3). There were significant differences in the reported frequency of use of Gadyen Dlo between participants and controls, but there were no significant differences in the reported frequency of use for the other water treatment methods among households who voluntarily identified having heard of the methods. Nearly all participants (97%) volunteered that they had heard of Gadyen Dlo without explicitly asking, and 96% of participants reported having heard of Gadyen Dlo when explicitly asked (Table 2). However, eight participants (4%) had not heard of Gadyen Dlo when explicitly asked, indicating that they were not actually participants or that the enumerators coded the response incorrectly.

Conditional on voluntarily reporting having heard of Gadyen Dlo, 34% of participants reported using it every day, compared with 11% of controls (P < 0.001), although a large (19%) proportion of participants reported only using Gadyen Dlo once. Overall, 93% of participants reported having ever used Gadyen Dlo (compared with 15% of controls), 75% reported that they were currently using Gadyen Dlo (compared with 10% of controls), and 46% reported having used Gadyen Dlo in the past 24 hours (compared with 5% of controls) (P < 0.001 for all comparisons).

Seventy-nine percent of participants compared with 32% of controls (P < 0.001) self-reported treating their drinking water using any method (Table 2); 56% of participants and 10% of controls had positive chlorine residuals in the acceptable range of 0.2–2.0 mg/L (P < 0.001 for both comparisons). Seven participants had chlorine residuals greater than 2.0 mg/L, indicating that they added too much chlorine. Of those participants who had positive chlorine residuals in the acceptable range, 94% had a bottle of Gadyen Dlo in their home. Thirty-two percent of participants without positive chlorine residuals had a bottle of Gadyen Dlo with chlorine remaining in the bottle, which could indicate either that they treated their water too long ago for the free chlorine residual to remain, or that they did not consistently treat their water.

Table 2

Gadyen Dlo reported and actual use among program participants and controls, Haiti

Variable	Participants	Controls	P
No. (%) who reported having heard of Gadyen Dlo when asked to list all known water treatment methods	194/201 (96.5)	170/424 (40.1)	< 0.001
No. (%) who reported using Gadyen Dlo, conditional on voluntarily reporting having heard of it
Every day	65/191 (34.0)	18/168 (10.7)	< 0.001
Once a week	71/191 (37.2)	13/168 (7.7)	< 0.001
Sometimes	15/191 (7.9)	26/168 (15.5)	0.023
Once	36/191 (18.8)	8/168 (4.8)	< 0.001
Never	4/191 (2.1)	103/168 (61.3)	< 0.001
No. (%) who reported having heard of Gadyen Dlo when explicitly asked	192/200 (96.0)	84/390 (21.5)	< 0.001
No. (%) who reported ever using Gadyen Dlo	184/197 (93.4)	57/389 (14.7)	< 0.001
No. (%) who reported now using Gadyen Dlo	149/200 (74.5)	39/389 (10.0)	< 0.001
No. (%) who reported treating their drinking water with Gadyen Dlo in the past 24 hours	92/200 (46.0)	21/390 (5.4)	< 0.001
No. (%) who reported treating their current drinking water, using any method	153/195 (78.5)	129/404 (31.9)	< 0.001
No. (%) who had a positive chlorine residual in their current drinking water between 0.2 and 2.0 mg/L	98/176 (55.7)	25/258 (9.7)	< 0.001

Health outcomes: intention-to-treat estimates.

We first present a conservative estimate of the health effects of program participation using an intention-to-treat analysis. This approach compared all participants, including those who were not currently treating their water, with controls.

Point prevalence.

Because respondents were asked to report health outcomes for every person living in their household, the health results treated each person rather than each household as an observation. The proportions of respondents with diarrheal disease among persons of all ages and among children < 5 years of age for males and females were significantly different between participants and controls (Table 3). Fourteen percent of participants had an episode of diarrhea in the past 48 hours compared with 21% of controls (P < 0.001). These proportions were similar for males and females. Among children < 5 years of age, 32% of participants had an episode of diarrhea in the past 48 hours compared with 52% of controls (P < 0.001). More female participants < 5 years of age (36%) had diarrhea than male participants < 5 years of age (29%), although this difference was not statistically significant (P = 0.464). This finding was also true for children < 5 years of age living in control households. In addition, significantly fewer participants than controls with diarrhea had blood in feces (2% versus 4%; P < 0.001). Thus, we found that for all participants and for children < 5 years of age, significantly fewer participants than controls had diarrhea and bloody stools, and there were no significant differences between males and females.

Table 3

Proportion of participants and controls reported to have had diarrhea and bloody stools in the past 48 hours, for all ages and for children < 5 year of age, Haiti

Variable	Participants	Controls	P
No. (%) of all ages reported to have had diarrhea	140/1,010 (13.9)	449/2,138 (21.0)	< 0.001
Males	67/468 (14.3)	201/965 (20.8)	0.003
Females	73/540 (13.5)	248/1,161 (21.4)	< 0.001
P value for equality of males and females among participants or controls	0.715	0.765	–
No. (%) of all ages reported to have had bloody stools	17/993 (1.7)	88/2,096 (4.2)	< 0.001
No. (%) of children < 5 years of age reported to have had diarrhea	33/104 (31.7)	117/224 (52.2)	< 0.001
Males	17/59 (28.8)	52/106 (49.1)	0.011
Females	16/45 (35.6)	65/116 (56.0)	0.020
P value for equality of males and females among participants or controls	0.464	0.692	–
No. (%) of children < 5 years of age reported to have had bloody stools	5/101 (5.0)	28/219 (12.8)	0.032

Multivariate logistic regression models.

Regression models were developed for diarrhea and for dysentery in household members of all ages and for children < 5 years of age (Table 4). In the basic model, the odds of diarrhea was 39% less for participants than for controls (OR = 0.61, 95% CI = 0.45–0.82). When we controlled for potential confounders (including sex, asset index, time to collect water, turbidity in the household drinking water supply, presence of soap, reported use of an improved drinking water source, use of a safe storage container, and reported use of a latrine by the respondent and children in the household), the odds of diarrhea was 26% less, which was still significant at the alpha level of 0.1 (OR = 0.74, 95% CI = 0.52–1.05). Thus, being a participant in the JSWF program significantly reduced the odds of having diarrhea, even when adjusting for characteristics that may confound the relationship between participants and controls.

Table 4

Multivariate model assessing the association of program enrollment with diarrhea and bloody stools in the past 48 hours, accounting for clustering and adjusting for potential confounders, Haiti*

Variable	Basic model				Full model†
	OR	95% CI	P	N‡	OR	95% CI	P	N‡
Dependent variable: diarrhea in past 48 hours
All ages	0.61	0.45–0.82	0.001	3,148 (614)	0.74	0.52–1.05	0.092	2,221 (406)
Children < 5 years of age	0.43	0.26–0.70	0.001	328 (240)	0.41	0.21–0.79	0.008	229 (166)
Dependent variable: bloody stools in past 48 hours
All ages	0.40	0.23–0.70	0.001	3,089 (611)	0.37	0.20–0.67	0.001	2,178 (404)
Children < 5 years of age	0.36	0.13–0.95	0.040	320 (235)	0.40	0.15–1.06	0.065	225 (164)

Standard errors were adjusted for clustering at the household level. OR = odds ratio; CI = confidence interval.

Adjusted for sex, asset index, time to collect water, turbidity in the household drinking water supply, presence of soap, reported use of an improved drinking water source, use of a safe storage container, and reported use of a latrine by the respondent and children in the household.

Individuals (households).

Presence of household soap and use of a safe storage container were significant in the full model, showing that handwashing and safe water storage were important preventative behaviors, which is consistent with results of other studies (Supplemental Table 4). We also considered diarrhea with blood in the feces as an outcome measure, and we also considered a sample limited to children < 5 for both diarrhea and bloody feces. In all three of these alternative regressions, we found significantly reduced odds of the outcome in question among participant households by using the basic model, and the results incorporating the other characteristics in the full models were similar. A series of restricted models using the same specifications as the basic models, but with data on only those households included in the full models, yielded similar results to the basic models for all four outcomes (Supplemental Table 5). This finding provided a strong indication that the results of the full model were not driven by selection bias caused by some households being excluded from the full model because of missing data on covariates. Program participants thus had significantly reduced odds of diarrhea and odds of bloody feces for all ages and for children < 5 years of age, even when we adjusted for potential confounders. There was no evidence that presence of soap in the home, use of an improved drinking water source, safe storage of drinking water, or reported use of a latrine modified the effect of program participation on diarrheal disease (Supplemental Tables 6–9).

Health outcomes: treatment-on-treated estimates.

Program participation.

We now compare subgroups of participants to controls to hone in on the effects of the program for those who were most likely to benefit on the basis of more stringent definitions of adoption, using the basic model without adjustments. These subgroups of participants were compared with all controls, regardless of whether the controls also met the same characteristics as the subgroups of participants. The odds of diarrhea for participants who self-reported always treating their household water with Gadyen Dlo was 67% less (OR = 0.33, 95% CI = 0.21–0.50) than for controls (Table 5), compared with a 39% reduction in the odds of diarrhea from the intention-to-treat analysis. The odds of diarrhea for children < 5 years of age in these participant households was 70% less (OR = 0.30, 95% CI = 0.15–0.60) than for children < 5 years of age in control households, which was also greater than the 57% reduction in the odds of diarrhea from the intention-to-treat analysis for children < 5 years of age. Therefore, we found that when we limited our analysis to only those participants in the JSWF program who reported treating their water, the reduction in the odds of diarrhea was even greater than when we included all participants.

Table 5

Treatment on treated analysis of the odds of diarrhea in the past 48 hours among sub-groups of program participants compared with controls, Haiti*

Variable	All ages				Children < 5 years of age
Program participant sub-group	OR	95% CI	P	N†	OR	95% CI	P	N†
All participants (repeated from Table 3)	0.61	0.45–0.82	0.001	3,148 (614)	0.43	0.26–0.70	0.001	328 (240)
Self-report always chlorinating	0.33	0.21–0.50	< 0.001	2,462 (481)	0.30	0.15–0.60	0.001	268 (195)
Record of technician visit	0.68	0.47–0.99	0.043	2,620 (512)	0.44	0.22–0.89	0.021	270 (199)
Record of chlorine purchase in past year	0.52	0.26–1.06	0.071	2,228 (433)	0.46	0.12–1.75	0.252	233 (172)
Presence of bottled chlorine in household	0.48	0.32–0.72	< 0.001	2,731 (534)	0.29	0.15–0.57	< 0.001	286 (207)
Presence of free chlorine in water	0.50	0.31–0.79	0.003	2,619 (511)	0.30	0.14–0.64	0.002	268 (196)

Standard errors were adjusted for clustering at the household level. OR = odds ratio. CI = confidence interval.

Individuals (households),

For participants with a record of a technician visit or a chlorine purchase in the past year, the odds of diarrhea were significantly less than for controls, and CIs were similar with those of the intention-to-treat estimate, with the exception that the CI for the < 5 years of age subset of participants with record of a chlorine purchase was substantially wider, such that the reduction in diarrhea among participants < 5 years of age was not statistically significant.

In participant households with bottled chlorine in the household at the time of the unannounced survey, the odds of diarrhea was significantly less than in control households for all ages and among children < 5 years of age. The same finding was true of the odds of diarrhea in participant households with positive free chlorine residuals in the drinking water at the time of the unannounced survey.

Although not all participants had treated water at the time of the unannounced visit, the treatment-on-treated analysis indicated that the odds of diarrhea was still significantly less in participant households than in control households across a multitude of indicators when subgroups of program participants were analyzed. When compared with the reduction in the odds of diarrhea among all participants for all ages and for children < 5 years of age, those who self-reported that their drinking water was always treated, purchased chlorine in the past year (all ages only), had a bottle of chlorine in the home, or had positive free chlorine residuals in their drinking water had an even greater reduction in the odds of diarrhea.

Chlorination.

Whereas the analysis so far has concentrated on the effects of program participation, we now address a more general question: what is the effect of chlorination on diarrhea, recognizing that water treatment is practiced by more than just JSWF participants? The odds of diarrhea among all households with positive free chlorine residuals in their drinking water was 61% less (OR = 0.39, 95% CI = 0.26–0.57) than among households without positive chlorine residuals. Although there is more likely to be selection bias here because those who treat their water may differ from those who do not on unobservable household characteristics, these findings suggest that household chlorination significantly reduced the odds of diarrhea, whether or not a household participated in the JSWF program.

Program effects and socioeconomic status.

In all except the highest asset quintile, program participants were more likely to have chlorine residual in their stored household drinking water than non-participants (Table 6). A higher proportion of persons in lower quintiles had diarrhea, regardless of whether they had positive chlorine residuals (Table 6). In households with positive chlorine residuals, diarrhea ranged from 17.0% of persons in the lowest asset quintile to 2.9% in the highest asset quintile. In households without positive residuals, diarrhea ranged from 32.7% of persons in the lowest asset quintile to 12.2% in the highest asset quintile. Thus, the proportion of persons with diarrhea was inversely proportional to socioeconomic status, but having a positive chlorine residual reduced the magnitude of the disparity.

Table 6

Proportion of participants and controls with positive chlorine residuals by asset quintile; proportion of individuals with diarrhea in the past 48 hours by chlorine residual test and asset quintile, Haiti

Variable	Positive chlorine residuals, no. (%)
	Participants	Controls	P
Asset quintile
First (lowest)	20/27 (74.1)	2/46 (4.4)	< 0.001
Second	15/25 (60.0)	7/43 (16.3)	< 0.001
Third	15/28 (53.6)	5/31 (16.1)	0.002
Fourth	28/38 (73.7)	2/40 (5.0)	< 0.001
Fifth (highest)	11/32 (34.4)	8/41 (19.5)	0.151
P for overall Pearson's chi-square test	0.006	0.096	–
	Reported to have had diarrhea, no. (%)
	With positive chlorine residual	Without positive chlorine residual	P
Asset quintile
First (lowest)	19/112 (17.0)	96/294 (32.7)	0.002
Second	18/128 (14.1)	53/247 (21.5)	0.083
Third	9/85 (10.6)	33/175 (18.9)	0.089
Fourth	12/136 (8.8)	87/265 (32.8)	< 0.001
Fifth (highest)	3/102 (2.9)	36/294 (12.2)	0.007
P for overall Pearson's chi-square test	0.012	< 0.001	–

Discussion

This study was an evaluation of the long-term health impact of household chlorination in the JSWF program in the Northwest and Artibonite Provinces of Haiti. Program records indicate that over the course of nearly eight years, positive chlorine residuals were detected for 70% of unannounced household visits (Turbes A, unpublished data). In this study, 56% of participants and 10% of controls had confirmed positive free chlorine residuals. It was expected that the proportion of participants with positive residuals would be higher than that of controls because they receive regular household visits and education. The reason for the lower percentage of participants whose water was positive for chlorine residual in the survey than in the program records can be partially explained by inconsistent use of chlorine by some participants or by selection bias in the program records if program technicians chose known frequent users for the unannounced programmatic visits to households.

In the intention-to-treat analysis, program participants had a 26% reduced odds of diarrhea compared with controls, and children < 5 years of age had a 59% reduced odds when we controlled for potential confounders. The effect of the program was likely stronger for children < 5 years of age because they are more susceptible to illness from waterborne organisms. Because participants who were not consistently using chlorine were included in the intention-to-treat analysis, this was a conservative estimate of the long-term health effects of this program. The treatment-on-treated analysis indicated larger diarrheal disease reductions when restricting the analysis to only the subset of participants who met more stringent criteria for use. The magnitude of the reduction in the odds of diarrhea was stronger for children < 5 years of age in participant households with positive chlorine residuals (treatment-on-treated OR = 0.30, 95% CI = 0.14–0.64) than for all children < 5 years of age in participant households (intention-to-treat OR = 0.43, 95% CI = 0.26–0.70).

In summary, in the intention-to-treat and treatment-on-treated analyses, program participation was strongly associated with reduced diarrheal disease, especially in children < 5 years of age. These findings indicate that the JSWF program has achieved long-term behavior change among program participants and resulted in improved health.

In this study, we found 10% of controls with positive free chlorine residuals in household stored drinking water. This finding is consistent with that of Rosa and Clasen,19 but could also be partially attributable to the community education being undertaken by the JSWF program, including posters, radio advertisements, and church announcements, which reach program participants and controls. When considering chlorination practiced by either participants or controls, rather than focusing on the health effects of the JSWF program specifically, we found that the odds of diarrhea among households with positive free chlorine residuals was 61% less (OR = 0.39, 95% CI = 0.26–0.57) than among households without positive chlorine residuals. Program participation did not have a significant effect (P = 0.332 for all ages and P = 0.202 for children < 5 years of age) when the analysis was limited to only households with positive free chlorine residuals, indicating that household chlorination resulted in improved health regardless of participation in the JSWF program, although this result could have been affected by selection bias.

The diarrheal prevalence reported in this study was approximately twice as high as in the most recent Demographic and Health Surveys. We hypothesize that the reasons for this are 1) the study area is the most remote, mountainous region of the Department and 2) displacement after the earthquake might have increased the number of family members living together in study households. Although reporting bias might have led to inflated diarrhea prevalence, we found a meaningful and statistically significant reduction in diarrhea among participant households.

The JSWF program has several aspects not found in many other household chlorination programs that potentially help make the program more effective. First, technicians conduct regular household visits and maintain records of purchase and household chlorine residuals, which can be used to establish whether households are consistent users. Second, the staff members running the program are all Haitian. Third, the components of the program, including the safe storage containers and the sodium hypochlorite, are produced locally. Fourth, the chlorine is sold at a slight profit margin so that program staff are fully paid with program income.4

One of the strengths of this evaluation was the incorporation of multiple indicators of program adoption, including self-reported use, program records, verified available chlorine supply in the household, and verified treatment based on free chlorine residual in household stored water. Our research design enabled us to complete a spectrum of analyses ranging from intention-to-treat, which provided a conservative result, to treatment-on-treated, which was less likely to underestimate the impact because it focused on participants who actually practiced chlorination, but potentially had more selection bias. Further research should have more stringent definitions of program participation so that controls who are using Gadyen Dlo are not included, and analyze subgroups of controls using the same categories as the subgroups of program participants in the treatment-on-treated analysis.

Lastly, the authors would like to place these results within the larger context of water treatment in Haiti after the earthquake and cholera emergencies. This research was planned before the earthquake occurred and the onset of cholera, although it was conducted between the two emergencies. The sole reason this location was selected for this research was because the JSWF program is the longest-running chlorination program with continuous monitoring and records known to the researchers. We did not modify our research protocol after the January 2010 earthquake because the research area was outside the affected area, although as mentioned previously, internally displaced persons did move into the area. However, one of the authors did conduct independent research on water treatment options distributed within the first eight weeks after the earthquake.20 DSI was the largest immediate water treatment responder to the earthquake; as of February 16, 2010, and DSI had reached 2,880 new families with safe storage containers and Aquatabs brand sodium dichloroisocyanurate chlorine tablets through a network of 165 CHW distributors in the Léogâne area, near the epicenter of the earthquake. This program built upon a replicate of the JSWF project in Léogâne, Haiti that existed before the earthquake. The DSI program in Léogâne had the highest effective use (56%) of all the water treatment options distributed in all four acute emergencies, including filters and chlorine options in Haiti. Effective use was measured as the percentage of targeted families using the distributed product to improve the microbiologic quality of their stored household drinking water to meet international guidelines. The lessons learned in the JSWF project, and replicated in the Léogâne project before the earthquake, directly led to successful emergency implementation because unlike other programs, DSI was able to target households with contaminated water and provide them with an effective water treatment option that they were familiar with in a timely manner.

Additionally, between the time of this evaluation and the subsequent cholera outbreak in Haiti in October 2010, the program has undergone a four-fold expansion to approximately 16,000 households. Further research will be needed to assess whether the program is able to maintain the characteristics that have made it so effective in its original form as it expands to a much larger population as part of emergency response efforts.

There were four main limitations in this study. First, the JSWF program members consciously made a decision to enroll in the program, and thus enrollment was not random. Therefore, there may have been unobservable differences between participants and controls not accounted for in the regression analysis that were inherent to why JSWF participants enrolled, and which could have contributed to differences in diarrhea rates between the two groups. Second, 39 controls (10.0%) stated at survey outset they were not program participants, but contradicted themselves later in the survey by self-reporting that they currently used Gadyen Dlo. In the future, evaluations should attempt to more strictly confirm non-participation, although this can be difficult in the field. Third, there was an abnormally high rate of missing data for the free chlorine residual testing (12.5% of participants and 38.4% of controls), which we suspect was caused by enumerator neglect and not participant refusal, because one of the authors has conducted more than 1,000 free chlorine residual tests in household surveys in rural Haiti and showed a refusal rate < 1% rate for free chlorine residual testing. This might have led to selection bias. Fourth, we relied on self-reported diarrhea as our outcome measure. A subset of households might have over-reported water treatment and under-reported or over-reported diarrhea because of social desirability or courtesy bias, which would lead to spurious evidence of program health effects. However, these concerns are mitigated, but not fully alleviated, by the facts that: 1) JSWF program members closely resembled control households on most observable characteristics for which we have data; 2) enumerators were not associated with the JSWF program; 3) the JSWF program was not identified in the survey; and 4) strong correlations were observed between diarrheal disease reductions and more stringent indicators of use, including bottle sales and chlorine residual presence, based on independent program records.

This study is one of the first to examine the long-term health impact of household chlorination programs. The health impact of the JSWF program among households who had participated in the program for an average of more than four years is consistent with other efficacy studies of household chlorination for shorter time periods. A meta-analysis of household chlorination studies, in which only four studies had a duration of at least one year, found that the risk of diarrhea in children < 5 years of age was reduced by 40%.19 Another meta-analysis in which the longest study period considered was 87 weeks (1.7 years) but the median length was 20 weeks, found a 29% reduction in the risk of childhood diarrhea.12 The JSWF program, which had a 59% reduced odds of diarrhea in children < 5 years of age after more than four years of participation demonstrates that the health impact of household chlorination programs does not necessarily decrease over time when consistent chlorine use is maintained.

After nearly eight years of operation, the JSWF program has achieved long-term behavior change and significant diarrheal disease reduction among program participants who live within three hours of the clinic. Using both conservative intention-to-treat analysis that assessed JSWF programmatic impact among all enrolled households and treatment-on-treated analysis restricted to participants with stricter indicators of adoption, we found that household chlorination can be an effective long-term water treatment strategy. In the JSWF program, participants continue to make small investments to improve their water quality in the home and have better health outcomes as a result. The findings may help inform the development of other programs by demonstrating a working household chlorination model in which chlorine sales and use have been consistently recorded and diarrheal disease reduction has been evaluated.

Click here for additional data file.