Insights from community case management data in six sub-Saharan African countries.

There is strong research evidence that community case management (CCM) programs can significantly reduce mortality. There is less evidence, however, on how to implement CCM effectively either from research or regular program data. We analyzed monitoring data from CCM programs supported by the International Rescue Committee (IRC), covering over 2 million treatments provided from 2004 to 2011 in six countries by 12,181 community health workers (CHWs). Our analysis yielded several findings of direct relevance to planners and managers. CCM programs seem to increase access to treatment, although diarrhea coverage remains low. In one country, the size of the catchment area was correlated with use, and increased supervision was temporally and strongly associated with improved quality. Planners should use routine data to guide CCM program planning. Programs should treat all three conditions from the outset. Other priorities should include use of diarrhea treatment and insurance of adequate supervision.

Background

In most high-mortality countries, many families do not have adequate access to treatment of fatal childhood diseases. Community case management (CCM) is a strategy used to increase access to such treatment in countries with inadequate access to curative services by empowering community health workers (CHWs) to identify and treat children with life-threatening illnesses. Multicountry evidence reviews have shown that CHWs provided with adequate training, supervision, tools, and logistics support can identify and appropriately treat children with diarrhea, pneumonia, and malaria.1,2 Typically, communities select CHWs, who are then trained in a simplified version of the Integrated Management of Childhood Illness (IMCI) curriculum to counsel parents, identify, and treat sick children under 5 years of age or refer them if they have danger signs. The World Health Organization (WHO), United Nations Children's Fund (UNICEF), and major donors are promoting CCM as a key strategy to meet Millennium Development Goal 4 of reducing under 5 years mortality from 1990 levels by two-thirds by 2015, and an increasing number of countries have incorporated CCM in their national strategies.3

However, nearly all the evidence for the impact of CCM is based on single disease models. Meta-analyses of studies conducted in the 1980s and 1990s found that management and treatment of pneumonia in the community could lead to significant reductions in pneumonia-specific and overall mortality among children under 5 years of age.4,5 Presumptive treatment of fever with effective antimalarial drugs in the community and the home has been shown to increase the number of patients receiving treatment,6,7 decrease malaria morbidity and parasitological indices,6 and reduce overall and malaria-specific mortality.8 The impact of use of oral rehydration salts/therapy in the home on child mortality and incidence of severe diarrhea has been well-documented,9 and a community-based trial showed that zinc for diarrhea management can reduce diarrhea morbidity, antibiotic use, and overall mortality.10,11

There is also a large body of literature that examines operational components of programs based on CHWs, including selection and training, program supervision, health information systems, sustainability, and scalability.12 However, much of this literature comes from Asia and Latin America and focuses on single disease management, and this information is merely descriptive. There are only a handful of studies that assess the effect of operational choices on program results in a quantitative fashion. One systematic review of intervention models involving CHWs recommended integrated multiple disease case management in sub-Saharan Africa.13

More specific evidence on the effect of different implementation strategies for CCM is scarce. A few studies conducted in Africa have formally investigated operational aspects of programs, in which CHWs used integrated guidelines to manage children sick with multiple illnesses at the community level.14–16 In Siaya district, Kenya, CHWs used a modified IMCI algorithm to classify and treat malaria, pneumonia, and diarrhea/dehydration concurrently. An evaluation showed that CHWs adequately treated 90.5% of malaria cases but had difficulties in classifying and treating sick children with pneumonia and severe illness.15 In Sudan, an evaluation of a CHW program found that CHW classification rates were consistent with facility-based IMCI evaluation studies.14 A cluster randomized controlled trial in Zambia showed the feasibility and effectiveness of using CHWs to provide integrated management of pneumonia and malaria at the community level.16 Except the latter, these studies were not comparative and do not provide evidence to decision-makers about which CCM operational strategies are most effective in improving access to treatment, use, quality, and mortality reduction. This gap is problematic given that implementation choices in areas such as CHW selection, training, and supervision are often the difference between success and failure.

Randomized trials would be expensive and impractical. In contrast, monitoring data, collected on a regular basis—usually monthly—in the course of program implementation, is a rich source of learning about the most effective ways to implement CCM. In this paper, we present findings and lessons learned from monitoring data collected by the International Rescue Committee (IRC) in its CCM programs.

The IRC Global CCM Program

The IRC has been implementing CCM in sub-Saharan Africa since December of 2004. It started in Rwanda and progressively added programs in South Sudan, Sierra Leone, Uganda, Ethiopia, and Ivory Coast. By June of 2011, the IRC was supporting CCM in 17 underserved, rural districts in those countries, covering a population of over 3.6 million people, including close to 646,000 children under the age of 5 years who are served by a network of 12,181 CHWs referring to 304 health facilities. IRC-supported CHWs have provided over 2 million treatments.

The IRC's methodology in Rwanda, developed in collaboration with the Ministry of Health (MoH), has been scaled up nationally, and the IRC's CCM program is also being scaled up nationally in Sierra Leone in collaboration with the Ministry of Health and Sanitation, UNICEF, and other non-governmental organizations (NGOs). Some aspects of the program methodology are similar in all countries, whereas there is variation between and within country programs for other parameters. In all six countries, communities select CHWs according to criteria set by the MoHs. It is important to note that CHWs in Ethiopia are a different cadre than Health Extension Workers. Initial training of CHWs lasts from 5 to 7 days, consists of variations of the WHO's CHW IMCI treatment manual, and focuses on management of malaria, diarrhea, and pneumonia. The training curriculum occasionally includes other skills, such as nutrition screening in Rwanda and newborn care in Uganda, depending on ministry policy. Literacy is an MoH criterion for CHW selection in Rwanda, Ethiopia, and Ivory Coast but not Sierra Leone and South Sudan. Literacy is a requirement in Uganda, except in some areas with extremely low literacy levels. A large proportion of the IRC-supported CHW network in Sierra Leone, South Sudan, and Uganda is composed of illiterate CHWs. Except in Ethiopia, supplies to CHWs are channeled through the health facilities to which CHWs refer. Replenishment of CHWs' drug stocks usually occurs one time per month and is facilitated by a supervisor that acts as a link between communities and health facilities. CHWs are supported through non-financial incentives, with the exception of Rwanda. In Rwanda, CHWs are unsalaried but receive financial support through a contract between the government and CHW cooperatives. CHWs treat fever presumptively with antimalarials with the exception of CHWs in Rwanda, where rapid diagnostic tests were introduced in 2010. CHWs in all six countries treat diarrhea with zinc and low osmolarity oral rehydration salts (ORSs) and pneumonia with amoxicillin or cotrimoxazole.

Materials and Methods

Data collection.

For all six countries, CHWs record the children visited and treatments given in patient and drug registers (patient register in Figure 1 ). These registers provide information on the name, age, and sex of the child, village where the child lives, classification of the condition of the child, treatment given, and whether the child was referred. The registers also contain information about each CHW's drug stock, including drugs received, used, and remaining for any given 1 month. On a monthly basis, a peer supervisor compiles all the information from the patient and drug registers used by the CHW in his/her catchment area into a CHW compiled report (data flow in Figure 2 ).

Figure 1.

CHW patient register in Sierra Leone.

Figure 2.

Data flow in IRC-supported CCM programs.

Peer supervisors are responsible for conducting a home visit to each CHW under their catchment area each month. During this visit, peer supervisors review the patient and drug registers to identify possible errors, check availability of supplies and storage conditions, assess how CHWs manage drugs, and assess whether CHWs can count breathing rates correctly. In addition, the supervisor and the CHW pay a visit to one of the children who recently sought care from the CHW. During this visit, the peer supervisors ask the caregiver why the child was taken to the CHW, what treatment, if any, was provided by the CHW, and how much of each treatment the caregiver actually gave to the sick child. The peer supervisor then compares the information obtained from the caregiver with what was recorded by the CHW in the patient and drug registers. At the end of the patient visit, the peer supervisor provides feedback to the CHW on any performance issues identified during the supervision and documents the findings in a supervision checklist. Information from the supervision checklist is included in the CHW compiled report.

In turn, field officers used by the IRC compile the CHWs' data in collaboration with health center and district staff, and they also collect data about peer supervision. Each IRC officer supports several health facilities and peer supervisors and is based in the field. An IRC officer or the health facility in charge will collect the totals from the CHW compiled report and the numbers of treatments for fever, diarrhea, and pneumonia in children under 5 years at the health facility and prepare a CCM health facility report. This information is then entered into a database at the district level. For the purposes of the project, private clinics are not classified as health facilities.

The database is an Excel worksheet with built in data validation and completeness checks. An IRC manager checks district data for completeness and quality before submitting it to a monitoring and evaluation officer/manager in charge of compiling all district data into a national database. The IRC coordinator in the country will check the national database for completeness and quality before submission to the IRC headquarters technical unit for inclusion into IRC's global database.

The Excel database holds updated information on population, treatments given, and stock levels. For treatments, the database also holds data from health facilities. A full list of database elements is included in Table 1. If a CHW experiences a stock out, the standard procedure involves referring the child to the health facility and documenting that referral in the patient register. A full course of treatment is never given with a referral, and therefore, no information would be recorded for a referral case in the drug register.

Table 1

Main data elements in the Excel database

Reference information	Community data	Health facility data
Country	Number of active CHWs	Number of malaria treatments
Year	Number of reporting CHWs	Number of diarrhea treatments
Month	Number of children visited	Number of pneumonia treatments
District	Number of malaria treatments
Health facility	Number of diarrhea treatments
Total population	Number of pneumonia treatments
Under 5 years population	Number of referred children
	Number of CHWs with antimalarials stock outs
	Number of CHWs with ORS and zinc stock outs
	Number of CHWs with antibiotics stock outs
	Number of CHWs supervised

At the time of this analysis, the database contained information from October of 2004 to June of 2011 for 2,023,984 CHW treatments in 304 health facility catchment areas in supported districts from six sub-Saharan countries.

Analysis.

The analysis presented in this paper was done in Excel. Numbers were aggregated and stratified using pivot tables, and simple correlation coefficients were calculated. Table 2 lists the main outcome and stratification variables used in the analysis.

Table 2

Main outcome and stratification variables used in the analysis

Indicator	Formula	Units	Notes
Use	12 × number of treatments for a particular disease/(number of children < 5 years in the area × number of months covered by the data)	Treatments per child per year	A proxy for coverage compared over time among different areas or to an expected incidence. It gives an indication of the proportion of children in need receiving treatment.
Treatment ratio	Use/expected incidence	Unit or percentage	The expected incidence is an estimate, the precision of which varies by disease. For pneumonia, there is small but solid amount of literature on expected incidence rates. There is some evidence about rates for diarrhea. Malaria treatment is the most difficult to evaluate in this way, because malaria incidence varies widely across time and geography.
Treatment mix	Number of treatments for each condition/total treatments	Percentage	As with the treatment ratio, this number would be expected to vary according to local epidemiology. As with use and treatment ratio, however, in practice, major variations are associated with program issues rather than epidemiological variation.
Size of catchment area	Number of under 5 year children in the area/number of CHWs	Number of households per CHW	This information can be calculated for a program globally or individual CHWs depending on the analysis needs. The number of under 5 years of age children is estimated using a percentage fixed in national statistics of the total population.
Supervision intensity	Supervisions done each month/number of CHWs in the area	Supervisions per CHW per month	Although it is theoretically possible for one CHW to receive multiple supervisions while another CHW might receive none, this case is rarely a problem.

Our main outcome variable was use, which was expressed as the number of treatments per child per year. Population figures are obtained from either the district-level health office in country or the health facility, because they are aware of the population covered by their catchment area. Use can be used as a proxy for coverage by following it across time, comparing different areas, or comparing the number to an expected incidence (in which case, it is referred to as the treatment ratio).17 This indicator assumes that, in the intervention areas where IRC is implementing CCM, all children have equal access to CHWs. In addition, use focuses on treatments and not encounters, and it is possible for a child to receive more than one treatment (e.g., if the child had malaria and diarrhea) during a single encounter. The treatment ratio can also be used as a proxy for quality, particularly for pneumonia, for which the expected incidence is reasonably consistent and backed by solid data. The treatment ratio provides an average number of treatments that each child receives per year and includes children who may have repeated episodes of a certain condition throughout the year. High or low treatment ratios can indicate poor quality, although such a determination can only be made with other contextual information. The incidence of childhood clinical pneumonia has been obtained from estimates published in 2008 in developing countries.18 The most recent estimates of diarrhea morbidity among children under 5 years in Africa are based on reviews of five published studies conducted over 20 years ago, with limitations because of the small number of data points and the lack of representativeness.19 However, a structured literature review of 27 studies looking into diarrhea morbidity by prospective surveillance in the work by Kosek and others20 estimated a global median incidence of diarrhea to be 3.2 episodes per child per year in the year 2000, which is similar to those results found in previous reviews by Snyder and Merson21 and Bern and others.22 It is not practical to use a similar benchmark for fever treatments, because malaria incidence varies widely across time and place.

Results

The analysis of these data, covering over 6 years of treatments in six very different countries, has yielded several findings. First, the data show that CHWs provide many more treatments than health facilities; Figure 3 presents the number of treatments per child per year for malaria, pneumonia, and diarrhea. The extent to which this finding is true varies across countries from Uganda, where community workers provide 11% more treatments than health facilities, to South Sudan, where community workers provide 10 times as many treatments as health facilities. There are good indications that community workers are providing new treatments rather than replacing treatments previously provided by health facilities. As Figure 4 illustrates for Sierra Leone, the increase in treatments by the introduction of CCM was far greater than any decrease in health facility treatments. The data also suggest that the CCM treatments are needed rather than excessive; Figure 3 shows that total use rates with CCM remain at or below what would be expected in areas with a high incidence of malaria, diarrhea, and pneumonia.

Figure 3.

Health facility and community use rates by country.

Figure 4.

Health facility versus community treatments in Sierra Leone before and after the introduction of CCM.

The data also indicate, however, that, despite relatively high overall use rates, fewer children are receiving treatment for diarrhea than would be expected. Treatment rates for diarrhea ranged from 0.1 treatments per child per year in Rwanda to 0.8 treatments per child per year in South Sudan, whereas young children in sub-Saharan Africa would be expected to have more than three bouts of diarrhea per 1 year.20

As Figure 3 indicates, use varies considerably between countries along with the balance between different treatments, which we refer to as the treatment mix (Figure 5). Sierra Leone and South Sudan started their programs treating all three conditions—fever, diarrhea, and pneumonia—at the outset, and they have continued to show a balanced treatment mix, with relatively good use of pneumonia treatment in particular. In Rwanda, the program started off providing only fever treatment. Although treatments of diarrhea and pneumonia were introduced within a couple of years after the launch in 2004, the imbalance in treatment has persisted, with treatments for fever representing more than two-thirds of the total. In Ethiopia and Ivory Coast, pneumonia treatment was initially not allowed at community level. This imbalance has persisted even after pneumonia treatment was accepted and implemented, with much lower treatment rates for pneumonia.

Figure 5.

Treatment mix.

Another finding is that there is a strong and negative correlation between the number of children in a CHW's catchment area and use, which is shown in Figure 6. This finding is based only on data from Sierra Leone, which has the greatest variability between different CHWs in the size of the catchment area. The data show a correlation across the entire range, with CHWs who have above 50 children in their care in particular showing se rates well below what would be expected, suggesting low coverage.

Figure 6.

CHW use versus under 5 years population served by CHW in Sierra Leone.

Finally, using data from Sierra Leone, where supervision was reintroduced after a period of neglect, we found that a sharp increase in supervision, close to one supervision visit per CHW per month, was associated with a decreased and more regular use of pneumonia treatment. Pneumonia treatments were initially variable, with peaks at more than three times the expected incidence; they went down rapidly to almost exactly the expected incidence shortly after supervisions were reintroduced, which is shown in Figure 7. The decrease in variability and decrease to expected levels suggest a marked improvement in quality of screening.

Figure 7.

Pneumonia treatment and supervision intensity in Kono District (Sierra Leone).

Discussion

This analysis of routine data generated by a long-standing CCM program generated several findings of direct relevance for international policy advocates, national decision-makers, and field-level implementers.

These findings should be applied with an awareness of the limitations of the programmatic data. Although the IRC has focused on data quality, dedicating resources at both field and headquarters level to audit and otherwise check the validity of information, these resources cannot match the resources of research programs. Population data, in particular, are usually derived from official estimates rather than from an individual census, and they may, in some cases, be significantly inaccurate, particularly in countries with limited infrastructure. However, in South Sudan, where there were strong indications of major errors in the population figures, the IRC did count the number of households in each village using an estimated average household size to translate this count into population estimates. Another limitation of program data is that data collection methods varied from country to country according to MoH policy or simply program evolution. The data are observational and influenced by many factors outside of the IRC's control. The IRC does not collect data regarding the private sector and cannot estimate its contribution in terms of coverage. Conclusions about causation can only be made tentatively and with relevant contextual information from other sources.

The data show that community providers have higher use rates than health facilities. This finding is consistent with published research on CCM, but it is noteworthy that it seems to be confirmed in the setting of large-scale programs. The data suggest that the increased treatments are filling a treatment gap, in part at least, between the low number of treatments being provided by health facilities in these low-resource settings and the high need given the burden of these conditions. Unfortunately, treatments for diarrhea remain low, although CCM represents an improvement over facility-only use, whereas for pneumonia, they are low in some places and have been at times excessive in other settings. There are a number of possible policy and program actions that can be taken to address this issue, ranging from improving the packaging and supply of drugs to increasing education and other behavior change activities for conditions such as diarrhea, which may be perceived as routine rather than a real threat to life. There is also evidence that broader roles for CHWs, including curative treatment of malnutrition, acute respiratory infection, and diarrhea, improve use of CHW services.23

The data in Sierra Leone also show a strong correlation between use and the size of the catchment area the CHWs cover. It is hard to rule out, with observational data, that this correlation might be attributable to a confounding factor, such as having more CHWs put in areas with expected higher incidence of disease. A closer look at the program showed that, in general, larger villages had fewer CHWs per children under 5 years than smaller villages. However, it also suggested an alternative explanation: drugs were supplied in quantities that were fixed per CHW rather than according to the size of their catchment area. Thus, CHWs with larger catchment areas had fewer drugs available per child. This finding would suggest that the proper program or policy response would not necessarily be to increase the density of CHWs —although it may help—but rather, to ensure that drugs supplies are determined on the basis of catchment area and need. This step has been taken in Sierra Leone. In any case, this finding suggests that national and local planners must be aware of CHW catchment areas and ensure that any variability does not result in variable access to treatment. Indeed, national planners may want to try different catchment areas to determine if they affect use. Currently, the size of CHW catchment areas in many country CCM programs is based on arbitrary criteria rather than evidence about the optimal balance between proximity to treatment and program cost.

The finding that supervision is strongly and temporally associated with improved quality is consistent with other research showing that regular supervision is associated with better project outcomes24 and more accurate classification and treatment of childhood illness by CHWs.25 Our data suggest that one supervision per CHW per month is a necessary and feasible standard; currently, many, if not most, national CCM programs fall far short of this standard.

Conclusions

This analysis of routine CCM program data in several countries has yielded several findings with implications for policy and practice. The analysis supports, although it does not prove, that integrated CCM dramatically increases the number of children treated for fever, diarrhea, and pneumonia. These findings are consistent with research showing the benefits of CCM in reducing mortality, and they should be considered by national planners deciding whether to adopt or scale-up CCM as well as donors looking to make the most of their investment in saving lives. National as well as local planners should also analyze use rates by disease and act on their analysis given the finding in all six countries that treatment rates for diarrhea fall well short of expected incidence.

Our analysis also suggests that planners should give preference to integrating CCM programs from the outset rather than introducing treatment of each condition at a separate time. We found that countries that started programs without treatment of all three conditions suffered from persistent imbalances in the treatment mix, even after the three conditions were eventually introduced.

Our analysis showed a strong correlation between the size of CHW catchment areas and use of treatment. The reasons for this correlation may be complex, and our analysis did not yield a specific policy recommendation. At the least, however, national and local planners should follow this parameter and be aware of the possibility that the size of CHW catchment areas may significantly influence access to treatment.

Finally, our analysis strongly suggested that regular supervision—at a more intense level than most current CCM programs—significantly improves quality. Planners should track the number of supervisions given and ensure that they are of good quality, and we propose that they aim for a standard of one supervision for CHW per month.