A health systems approach to integrated community case management of childhood illness: methods and tools.

Integrated community case management (iCCM) of childhood illness is an increasingly popular strategy to expand life-saving health services to underserved communities. However, community health approaches vary widely across countries and do not always distribute resources evenly across local health systems. We present a harmonized framework, developed through interagency consultation and review, which supports the design of CCM by using a systems approach. To verify that the framework produces results, we also suggest a list of complementary indicators, including nine global metrics, and a menu of 39 country-specific measures. When used by program managers and evaluators, we propose that the framework and indicators can facilitate the design, implementation, and evaluation of community case management.

Background

Integrated community case management (iCCM) enjoys broad-based policy support at the global level from a range of development partners and donor agencies. Across the developing world, countries are increasingly scaling up this strategy in efforts to meet the fourth Millennium Development Goal. Joint policy documents released by the World Health Organization (WHO) and United Nations' Children's Fund (UNICEF) support home-based management of fever, as well as community management of pneumonia, diarrhea, severe and acute malnutrition.1–4 The U.S. Government supports iCCM through the Global Health Initiative and the President's Malaria Initiative (PMI)5, the Bill and Melinda Gates Foundation (BMGF) and the Canadian International Development Agency6 support iCCM research and scale-up, and many Ministries of Health (MOHs) and non-governmental organizations directly implement iCCM.

Although published evaluations of iCCM operating at scale are forthcoming, a body of peer-reviewed literature from CCM pilots documents a number of factors that contribute to programmatic success. Key studies by Kidane and Morrow,7 Winch and others,8 and Barat and Schubert9 reference the importance of robust quality assurance schemes, appropriate training, and retention of human resources, and uninterrupted drug supply. Conversely, studies by Kelly and others10 and Nsungwa-Sabiti and others11 refer to the challenges associated with timely and quality supervision and insufficient community sensitization and dialogue, which have undermined the impact of some programs. Recent systematic literature reviews also show that community health worker (CHW) programs may face political obstacles in environments where CHWs are still seen as a second rate option for service delivery or where mechanisms for their remuneration may cause controversy.12,13 At the same time, the factors that have facilitated iCCM policy change and scale-up have not been well documented.14 The evidence suggests that with appropriate support and training, CCM can improve child health outcomes, but that program planners require support to design iCCM programs that are scalable and politically supportable.15

In light of existing evidence, we propose that community case management must be designed from a health systems perspective to be successful. Without careful attention to financing, human resources, supply chain management, quality assurance, and other inputs, iCCM programs risk uneven roll out and disappointing results. Creating a new or revitalizing an existing cadre of CHWs, who may or may not become routine health system expenditures, also exposes iCCM to political vulnerability, and program planners require frameworks that justify designing iCCM from an evidence-based perspective.16 Alternatively, where iCCM is built on top of an existing CHW network, a health systems approach to community-case management can offer insights on how to develop comprehensive service delivery. Given the proliferation of MOHs and development partners implementing iCCM, a shared framework can also improve coordination, communication, and roll out.

To assist in effective design and implementation, we present an interagency framework, in the form of a benchmarks matrix, to ensure key components are addressed throughout the life of an iCCM program. To verify that the achievement of benchmarks produces positive child health outcomes, we also propose a two-tiered set of indicators, including nine global indicators, and 39 country-specific measures to assist in monitoring and evaluation of iCCM.

Methods

Development of a programmatic tool: iCCM benchmarks.

In mid-2008, the United States Agency for International Development (USAID) initiated a consultative process to review policies surrounding and support of iCCM programs. USAID hosted stakeholder meetings to discuss implementation of iCCM and evidence that had been gathered to date on this strategy, at which participants recommended that a group of experts codify and share a list of key components of successful programs. USAID led the initial effort to specify the components, which resulted in a matrix of iCCM benchmarks. Colleagues from UNICEF, Save the Children, and other members of the iCCM Interagency Task Force17 finalized the framework's contents and shared them for review with in-country partners.

Proposed benchmarks, grouped into three phases of program evolution: advocacy/planning, pilot/early implementation, and expansion/scale-up are shown in Table 1. We identify eight health systems components: coordination and policy setting; costing and financing; human resources; supply chain management; service delivery and referral; communication and social mobilization; supervision and performance quality assurance; and monitoring and evaluation. These components mirror those of the WHO health systems' building blocks18 (leadership and governance; health financing; health workforce, quality health services; and drugs, vaccines, and technologies), with the addition of communication/social mobilization and supervision/performance quality assurance. Within each component, we define key activities for each phase of implementation.

Table 1

Integrated community case management benchmarks matrix*

Component	Advocacy and Planning	Pilot and Early Implementation	Expansion/Scale-up
1. Coordination and Policy Setting	Mapping of CCM partners conducted	MOH leadership to manage unified CCM established	MOH leadership institutionalized to ensure sustainability
	Technical advisory group established including community leaders, CCM champion, and CHW representation
	Needs assessment and situation analysis for package of services conducted
	Stakeholder meetings to define roles and discuss current policies held	Discussions regarding ongoing policy change (where necessary) completed	Routine stakeholders meetings held to ensure coordination of CCM partners
	National policies and guidelines reviewed
2. Costing and Financing	CCM costing estimates based on all service delivery requirements undertaken	Financing gap analysis completed	Long-term strategy for sustainability and financial viability developed
	Finances for CCM medicines, supplies, and all program costs secured	MOH funding in CCM program invested	MOH investment in CCM sustained
3. Human Resources	Roles of CHWs, communities and referral service providers defined by communities and the MOH	Role and expectations of CHW made clear to community and referral service providers	Process for update and discussion of role/expectations for CHW in place
	Criteria for CHW recruitment defined by communities and the MOH	Training of CHWs with community and facility participation	Ongoing training provided to update CHW on new skills, reinforce initial training
	Training plan for comprehensive CHW training and refresher training developed including modules, training of trainers, and monitoring and evaluation plan
	CHW retention strategies, incentive/motivation plan developed	CHW retention strategies, incentive/motivation plan implemented and made clear to CHW; community plays a role in providing rewards, MOH provides support	CHW retention strategies reviewed and revised as necessary.
			Advancement, promotion, and retirement offered to CHWs who express desire
4. Supply Chain Management	Appropriate CCM medicines and supplies are consistent with national policies and included in the essential drug list	CCM medicines and supplies procured consistent with national policies and plan	Stocks of medicines and supplies at all levels of the system monitored (through routine information system and/or supervision)
	Quantifications for CCM medicines and supplies completed
	Procurement plan for medicines and supplies developed
	Inventory control and resupply logistic system for CCM and standard operating procedures developed	Logistics system to maintain quantity and quality of products for CCM implemented	Inventory control and resupply logistics system for CCM implemented and adapted based on results of pilot with no substantial stockout periods
5. Service Delivery and Referral	Plan for rational use of medicines (and RDTs where appropriate) by CHWs and patients developed	Assessment, diagnosis and treatment of sick children by CHWs with rational use of medicines and diagnostics	Timely receipt of appropriate diagnosis and treatment by CHWs made routine
	Guidelines for clinical assessment, diagnosis, management, and referral developed	Review and modify guidelines based on pilot	Regular review of guidelines and modifications as needed
	Referral and counter referral system developed	Referral and counter referral system implemented: community information on where referral facility is made clear, health personnel also clear on their referral roles	CHWs routinely referring and counter referring with patient compliance, information flow from referral facility back to CHW with returned referral slips
6. Communication and Social Mobilization	Communication strategies developed, including prevention and management of community illness for policy makers, local leaders, CHWs, communities, and other target groups	Communication and social mobilization plan implemented	Communication and social mobilization plan and implementation reviewed and refined based on monitoring and evaluation
	Development of CSM content for CHWs on CCM and other messages (training materials, job aids)	Materials and messages to aide CHWs in place
	Materials and messages for CCM defined, targeting the community and other groups	CHWs dialogue with parents and community members about CCM and other messages
7. Supervision and Performance Quality Assurance	Appropriate supervision checklists and other tools developed, including those for the use of diagnostics	Supervision visit every 1–3 months, includes reviewing of reports, monitoring of data	CHWs routinely supervised for quality assurance and performance
	Supervision plan, including number of visits, supportive supervision roles, self-supervision established	Supervisor visits community, makes home visits, provides skills coaching to CHWs	Data from reports and community feed-back used for problem solving and coaching
	Supervisors trained in supervision and provided access to appropriate supervision tools.	CCM supervision included as part of the CHW supervisor's performance review	Yearly evaluation that includes individual performance and evaluation of coverage or monitoring data
8. M and E and Health Information Systems	Monitoring framework for all components of CCM developed and sources of information identified	Monitoring framework tested and modified accordingly	Monitoring and evaluation through HMIS data performed to sustain program impact
	Standardized registers and reporting documents developed	Registers and reporting documents reviewed	OR and external evaluations of CCM performed as necessary to inform scale-up and sustainability
	Indicators and standards for HMIS and CCM surveys defined
	Research agenda for CCM documented and circulated	CHWs, supervisors and M&E staff trained on the new framework, its components, and use of data

CCM; community case management; MOH = Ministry of Health; CHW = community health worker; RDTs = rapid diagnostic tests; CSM = communication and social mobilization; M and E; monitoring and evaluation; HMIS = health management information system; OR = operations research.

In principle, all benchmarks in each phase should be completed before initiating activities in the next column (i.e. the development of a functional logistics and resupply system, which falls under advocacy/planning, should take place before CHWs are actually trained, which falls under pilot/early implementation). In cases where curative interventions are being proposed within the context of existing CHWs, benchmarks in the advocacy/planning phase stress that iCCM costing estimates be based on all service delivery requirements and that CHW training be performed comprehensively. These pre-implementation planning exercises, especially in the context of existing community-based activities, provide the information necessary to decide whether a given setting is appropriate for iCCM implementation.

In addition to recommending a phased approach, the framework aims to mitigate the potential imbalance of budgetary and managerial resources across iCCM. Several aspects of integrated CCM, such as the training and deployment of human resources, typically receive substantial attention from program managers and Ministries of Health, and others, such as supply chain management and supervision, reflect areas where we believe partners should place greater prioritization. In the case of supply chain management, we advocate that iCCM products be included in the national essential medicines list, but also emphasize the establishment of a functional system for the resupply of CHW commodities. Recent research projects, such as the Bill and Melinda Gates Foundation–funded Improving Supply Chains for Community Case Management of Pneumonia and Other Common Diseases of Childhood, have documented limited availability of essential iCCM products at the community level, and only 35–50% of CHWs surveyed having all key drugs in stock on the day of the visit in Malawi, Rwanda, and Ethiopia.19–21 The benchmarks stress that there is no program without a product, and advocate for investing the resources needed to establish a functional supply chain.

Similarly, in the case of supervision and performance quality assurance, we emphasize that CHWs must be linked to higher levels of the health system by means of designated supervisors. Our framework recommends that CHWs receive clinical supervision once every 1–3 months by a trained supervisor and that supervisory visits be used as mechanisms for quality improvement through coaching, clinical observation and on-the-spot-training. We stress that programs include training in supervision, given that some CCM pilots have found that low competence among CHWs is associated with poor supervisors.22 Where iCCM has been added to a package of existing community health services, the greatest efforts possible should be made to streamline both training and supervision processes across CHW responsibilities.

Taken together, the benchmarks matrix and its components offer a systematic framework for designing and monitoring CCM. When used as a tool by managers and stakeholders, the benchmarks may improve and simplify the design and roll out of iCCM, offering a globally vetted framework that distributes attention appropriately across community-based health systems.

Development of iCCM benchmarks indicators.

A framework may improve planning and implementation, but to verify the effects and impact of iCCM across countries, harmonized indicators are required. The need for a collaborative process to develop CCM indicators based on program experience was first identified in a country exchange meeting in the Democratic Republic of Congo in 2009. Subsequently, the iCCM Task Force supported an interagency effort to develop an accompanying list of benchmark indicators, building upon the work of the CCM Operations Research Group and Save the Children's CCM results framework.23 The Task Force also used the interagency countdown to 2015 indicators, which provide a measurement approach for benchmarks as achieved on a yes, no, or partial scale.24 In November 2010, indicators pertaining to the quality of care were refined in a meeting convened by WHO and in June 2012 the full list of indicators was reviewed and finalized. The resulting compendium has nine global indicators for cross-national comparisons, and a list of 39 country-level indicators. The indicators list incorporates input, process, and output and outcome measures, applicable across the life of a program.

In this report, we present and discuss the nine global indicators listed in Table 2, of which there is one or two per benchmark component. A web annex presents a complete list of indicators, including the 39 country-specific measures.25 Some proposed indicators are currently being used by CCM programs, and other aspects of the framework have required the establishment of new metrics, for which field testing is ongoing. New indicators have been included to draw attention to particular aspects of programs believed to be important. However, if incorporated into national plans, countries will need to invest in systems through which they can be tracked. Country-specific indicators are intended to be incorporated into routine monitoring on an as needed basis, depending on the scale and location of the program. Global indicators are intended to be used by all countries implementing CCM, such that progress can be tracked internationally.

Table 2

Integrated community case management benchmarks global indicators list*

Component	Indicator	Definition	Metric
Coordination and Policy Setting	CCM policy	CCM is incorporated into national MNCH policy/guideline(s) to allow CHWs to give: low osmolarity ORS and zinc supplements for diarrhea; antibiotics for pneumonia ACTs (and RDTs, where appropriate) for fever/malaria in malaria-endemic countries	Yes: National policy guidelines allow CHWs to provide treatment in line with WHO recommendations for all relevant conditions
			Partial: National policy guidelines allow CHWs to provide treatment in line with WHO recommendations for at least one but not all relevant conditions
			No: No national policy guidelines exist that support CCM in line with WHO recommendations
Costing and Financing	Annual CCM costed operational plan	A costed operational plan for CCM exists and is updated annually	Yes: A costed operational plan/work plan for CCM for all relevant conditions† exists (or is part of a broader health operational plan) and is updated annually
			Partial: a) A costed CCM plan exists including at least one relevant health condition OR b) A costed CCM work plan exists but is not updated on an annual basis
			No: No costed plans for CCM are available for any relevant health condition
Human Resources	Targeted CHWs providing CCM	Proportion of CHWs targeted for CCM that are trained and providing CCM	Numerator: Number of CHWs targeted for CCM‡ that are trained and can provide evidence of providing CCM services in the last three months
			Denominator: Total number of CHWs targeted for CCM
Supply Chain Management	Medicine and diagnostic availability	Proportion of CCM sites with all key CCM medicines/diagnostics in stock§	Numerator: Number of CCM sites with all key CCM medicines/diagnostics in stock
			Denominator: Total number of CCM sites either visited or reporting on their stocks
Service Delivery and Referral Service Delivery and Referral	Treatment coverage	Proportion of sick children who receive timely and appropriate treatment¶	Numerator: Number of children under five with a CCM condition∥ that received timely and appropriate treatment
			Denominator: Number of children under five with a CCM condition
Communication and Social Mobilization	Caregiver knowledge of illness signs	Proportion of caregivers who know two or more signs of childhood illness that require immediate assessment/treatment	Numerator: Number of caregivers who can correctly state two or more signs of childhood illness that require immediate assessment/treatment
			Denominator: Total number of caregivers interviewed
Supervision and Performance QA	Routine supervision coverage	Proportion of CHWs who received at least one administrative supervisory contact** in the last three months	Numerator: Number of CHWs who received at least one supervisory contact in the last three months
			Denominator: Total number of CHWs trained in and deployed for CCM††
	Correct case management (knowledge)	Proportion of CHWs who demonstrate correct case management knowledge‡‡	Numerator: Number of CHWs who correctly manage sick child case scenarios
			Denominator: Total number of CHWs assessed
Monitoring & Evaluation and Health Information Systems	National monitoring and evaluation plan for CCM	Existence of a comprehensive, integrated monitoring and evaluation (M&E) plan for CCM	Yes: An M&E plan for CCM has all critical components and covers all relevant CCM conditions. Critical components of an M&E plan include: program goals and objectives, indicators to be measured, data collection methodologies and frequency, and mechanisms for dissemination/use of information
			Partial: M&E plan exists but has only some critical components or does not cover all CCM conditions
			No: Either a CCM M&E plan exits, but has no critical components, or no written M&E plan that covers CCM exists

CCM = community case management; MNCH = maternal, newborn, and child survival; CHW = community health worker; ORS = oral rehydration salts; ACTs = artemisinin-based combination therapies; RDTs = rapid diagnostic test; WHO = World Health Organization; QA = quality assurance; HMIS = health management information system.

Relevant conditions specified by country policy or implementation status.

The number of targeted CHWs should be specified in the country's national CCM plan.

Key products defined by country policy; this indicator can be measured as on the day of assessment visit or on the last day of the reporting period.

This indicator is measured through maternal recall in household surveys and usually specifies a time period in the last two weeks.

CCM conditions include diarrhea, suspected pneumonia, or malaria in malaria-endemic areas.

An administrative supervisory contact should include registers and/or reports being reviewed.

If this indicator is measured through a survey rather than administrative records, the denominator should be amended to represent the total number of CHWs interviewed.

This can be measured by either individual CHWs or by CHW service delivery point.

The proposed global metrics used several data collection methods. Medicine and diagnostic availability, routine supervision coverage, and the inclusion of iCCM indicators in the health management information system can be gathered through supervision records, CHW surveys or review of HMIS documents. Other global indicators, such as caregiver knowledge of illness signs and treatment coverage, necessitate household surveys. Given that CCM-specific questions are just beginning to be integrated into the Demographic and Health Surveys and Multiple-Indicator Cluster Surveys, we propose that evaluators work with partners to devise an appropriate mixture of data collection methods that work best on a country-by-country basis. As the level of community-based data available increases, the iCCM Task Force will refine data collection methodologies and revise the list accordingly.

Results

Beta versions of the benchmarks and indicators have been piloted in a variety of countries as part of both the design and monitoring and evaluation of iCCM. Thus far, these tools have accompanied the presence of external technical assistance, although we believe that both could be translated and used locally as part of routine program management. The benchmarks have been used in development of the USAID/President's Malaria Initiative-supported iCCM program in Mali, as well as the framework for USAID/President's Malaria Initiative iCCM documentations in Senegal, the Democratic Republic of Congo, and Malawi. A select number of benchmarks indicators have also been used in a study of the quality of care of iCCM in Malawi by Johns Hopkins University, the Malawian Ministry of Health, WHO, and UNICEF, as well as in Save the Children projects in Ethiopia and Zambia.

In the USAID Senegal documentation, a review of iCCM benchmarks performance highlighted many success factors, but also showed the need to stress routine supervision and improve functionality of the community-based supply chain. In the case of Malawi, a review guided by the framework indicated a need for action to be taken in the areas of financing and human resources. The USAID-funded Child Survival and Health Grants Program is also undertaking a review of 17 CCM projects in 12 countries (Sudan, Benin, Niger, Nepal, Uganda, Zambia, Burundi, Liberia, Afghanistan, India, Ethiopia, and Rwanda), which is likely to produce a significant amount of data on benchmark performance across countries and provide useful feedback on the framework itself.

The iCCM indicators are also now being used across agencies and implementing partners. In a Save the Children program in Ethiopia, many indicators are being tracked at the project level and have received high ratings; however, indicator performance in service delivery has shown room for improvement. Similarly, in the case of the Senegal, high marks were received in government commitment to iCCM and financing from donor agencies, but weaknesses in quality and service delivery were observed, particularly in CHWs' knowledge regarding the correct management of diarrheal disease.26 Indicators included in the Johns Hopkins University quality of care evaluation showed that CHWs in Malawi also showed relatively low levels of correct counseling on the treatment of diarrheal disease. However, other skills in Malawi were better developed, such as the correct prescription of artemisinin-based combination therapies for the treatment of malaria.27

Results of implementation of benchmarks and indicators have been consistent with our ex ante predictions; namely, that there has been a great deal of progress made in the financial and political commitments to CCM, but there is room for improvement in supervision and quality assurance, as well as supply chain management. These asymmetrical outcomes highlight the importance of examining iCCM from a systems perspective, and caution against evaluations that examine only one or two program components, which may miss the broader context and fail to offer comprehensive recommendations.

Discussion

Because of the relatively recent phenomenon of iCCM operating at a national scale, benchmarks and indicators will continue to be tested and refined on an ongoing basis. The full list of iCCM indicators are being formally assessed in Mali and Malawi as part of the USAID Translating Research into Action Project, which should offer additional information on the extent to which they can be integrated into national monitoring plans. In the meantime, we propose that these tools and metrics be incorporated into the planning and evaluation of CCM in a manner that is financially feasible and context-specific, while providing room for flexibility, adjustment, and continuous feedback from the field.

Despite the need for ongoing research, we propose that a systems approach is an effective method for designing and evaluating iCCM. Evaluations of CCM pilots have documented the importance of various health systems inputs in facilitating success, and recent evaluations of iCCM operating at scale emphasize the need to review programs holistically.28 The iCCM benchmarks framework provides such an approach, outlining components that span the health system, and codifying various steps that managers should follow throughout the course of design and implementation. In addition, the iCCM indicators offer a mechanism for verifying that the achievement of benchmarks results in positive health outcomes.

In a global environment in which CCM programs are proliferating within and across countries, a coordinated approach to iCCM and how to measure it has the potential to facilitate global implementation. Designed with input from a wide variety of donor agencies and implementing partners, we believe that the iCCM benchmarks and indicators can assist in the effective design, implementation, and monitoring and evaluation of CCM. As more data from evaluations of CCM becomes available, we plan to revise the framework and indicators as necessary, with the goal of providing state of the art guidance to program managers and evaluators alike.

Click here for additional data file.