Cost-effectiveness of community health systems strengthening: quality improvement interventions at community level to realise maternal and child health gains in Kenya.

INTRODUCTION: Improvements in maternal and infant health outcomes are policy priorities in Kenya. Achieving these outcomes depends on early identification of pregnancy and quality of primary healthcare. Quality improvement interventions have been shown to contribute to increases in identification, referral and follow-up of pregnant women by community health workers. In this study, we evaluate the cost-effectiveness of using quality improvement at community level to reduce maternal and infant mortality in Kenya.
METHODS: We estimated the cost-effectiveness of quality improvement compared with standard of care treatment for antenatal and delivering mothers using a decision tree model and taking a health system perspective. We used both process (antenatal initiation in first trimester and skilled delivery) and health outcomes (maternal and infant deaths averted, as well as disability-adjusted life years (DALYs)) as our effectiveness measures and actual implementation costs, discounting costs only. We conducted deterministic and probabilistic sensitivity analyses.
RESULTS: We found that the community quality improvement intervention was more cost-effective compared with standard community healthcare, with incremental cost per DALY averted of $249 under the deterministic analysis and 76% likelihood of cost-effectiveness under the probabilistic sensitivity analysis using a standard threshold. The deterministic estimate of incremental cost per additional skilled delivery was US$10, per additional early antenatal care presentation US$155, per maternal death averted US$5654 and per infant death averted US$37 536 (2017 dollars).
CONCLUSIONS: This analysis shows that the community quality improvement intervention was cost-effective compared with the standard community healthcare in Kenya due to improvements in antenatal care uptake and skilled delivery. It is likely that quality improvement interventions are a good investment and may also yield benefits in other health areas. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

What is already known?

Maternal and infant outcomes are improved when pregnancies are identified early, quality of antenatal care is high and women have a skilled delivery, but these are not being achieved consistently in Kenya.

Quality improvement is feasible and low cost to implement at community level, even in resource-limited health systems.

What are the new findings?

Investment in community quality improvement can yield quantifiable benefits in both maternal and child health outcomes.

The cost per disability-adjusted life year averted by the intervention is US$249 in Kenya.

Quality improvement at community level is cost-effective in the Kenyan healthcare system.

What do the new findings imply?

Investment in quality of community healthcare is an important component of the commitment to maternal and child health in Kenya.

Measuring the benefits of health system strengthening interventions to determine cost-effectiveness can be done if a minimum threshold of benefits is accrued within a single health area.

Introduction

Improvements in maternal and newborn health are major policy priorities in Kenya. To reduce maternal mortality, Kenya has provided free maternity care since 2013.1 2 The policy and implementation priorities for achieving these health outcomes are simple, proven interventions: early, focused and frequent antenatal care (ANC), and attendance at delivery by skilled birth attendants. Yet maternal mortality has remained high in Kenya and inequities persist between regions, with worse outcomes among younger and poorer mothers.3–5

Community health volunteers (CHVs) in Kenya are front-line health workers who focus on maternal and child health outreach, forming a key component of primary healthcare. These CHVs are expected to identify pregnant women in their communities who have not yet attended ANC or have defaulted on their scheduled visits. They also counsel pregnant women, informing them and their families on the benefits of ANC, testing in pregnancy and skilled birth attendance (SBA), supporting them with individual birth planning. After birth, CHVs follow up newly delivered mothers with postnatal home visits, nutrition support and immunisation checks. The CHVs work with, and are supervised by, a salaried community health extension worker (a nationally recognised cadre tasked with supervisory responsibilities) from a primary care facility to assist with referring and following up individuals to improve maternal and neonatal health outcomes. Community health units, which include both the CHVs and extension workers, form the lowest level of the four-tiered Kenyan health system.6 7

As Kenya (and other countries) grapples with defining and achieving Universal Health Coverage (UHC), the importance of quality at all levels of the healthcare system is widely recognised, but community health remains marginalised in the devolved Kenyan system,7–9 despite its potential to contribute to health outcomes.10 Defining and measuring quality at community level in low-resource settings are a challenging, but essential, precursor to understanding coverage and performance of services and to identifying areas for improvement.11–16 One approach to doing this is through quality improvement (QI)—a structured, cyclical health system strengthening process intervention. QI is often characterised by the Plan–Do–Study–Act (PDSA) cycle, although this is by no means the only approach.17–19 In health, the PDSA approach to QI has been successfully applied to identifying and addressing quality problems in many disease areas and contexts in health facilities.20–23 At the community level in low-income and middle-income countries (LMICs), it has been used in a limited but growing number of cases to address health areas such as HIV, maternal health and child health.24–27 The types of quality problems that can be addressed by community QI teams vary by the expected responsibilities of the community health workers as well as context and programme design.28 29

While the costs of a QI programme can be calculated fairly easily,30 estimating the benefits of QI interventions at community level is a challenge. Outcomes are likely to be improved across a wide set of health areas or conditions. Without information on the benefits, it is not possible to estimate cost-effectiveness, make evidence-informed decisions about investing in QI for community healthcare or advocate for funding. To our knowledge, evaluations to date of community-level QI in LMIC settings have focused on process evaluation and feasibility31 32 but not on attribution of health outcomes.

The aim of this paper is to evaluate the cost-effectiveness of community-level QI in Kenya to support policy and financing decision-making. As QI is intended to improve the quality of care and patient experience in any health area to which it is applied, we selected ANC and SBA as the focus health areas in which to assess the costs and outcomes. These were chosen because (1) maternal health is a policy priority in Kenya; (2) maternal health is a health area covered by community health workers both in and beyond the Kenyan context; and (3) ANC and maternal health have been shown to be affected by the QI intervention.33

Methods

The study is an ex post economic evaluation of an intervention designed to improve quality of healthcare in Kenya’s community health system. An economic evaluation involves the assessment of the costs and consequences of at least two alternative activities; in this case, we compared the community-level QI intervention (described in the following section) with standard community health delivery in Kenya. This was done using a decision tree, an analytic model that applies probabilities to different pathways to weigh the associated costs and outcomes.

The study takes a health system perspective on cost-effectiveness and uses a discount rate of 3% on future costs; in line with the Global Burden of Disease Study 2010 we do not discount disability-adjusted life years (DALYs).34 35 We report against the Consolidated Health Economic Evaluation Reporting Standards or CHEERS checklist,36 details of which are in online supplemental file 1.

The community-level QI health system strengthening intervention

The health system strengthening approach to community QI studied was a capacity development intervention delivered to two levels (community health unit and subcounty) of the Kenyan health system. At each level, QI teams made up of community and facility stakeholders were established as part of the REACHOUT (a 5-year, eight-country implementation research programme focused on measuring and improving efficiency, effectiveness and equity of community healthcare) and USAID SQALE (a 3-year implementation research programme that built on the work on REACHOUT in Kenya, improving on it based on the findings and expanding it to additional counties and subcounties) implementation research programmes in three counties (Kitui, Migori, Nairobi) from 2016 to 2019.37–39 Local QI teams identified and intervened to address locally relevant quality problems in community healthcare. The community-led nature of QI made it impossible to collect data across communities on the same quality problem or health area. For example, quality problems from intervention communities in 2018 included the poor uptake of ANC, checking of immunisation status and data quality, among others (see Box 1).

‘In Ribakia Community Unit, only 52% of pregnant women completed four ANC visits between 1st April - 30th Sept 2018’ (Nairobi County).

‘In Embakasi West Sub-County, 66% of community health volunteers do not check Mother and Child Booklets during household visits (Nairobi County).

‘In Mwingi North Sub-County, 100% of community health extension workers do not submit reporting forms (Ministry of Health form 515) to the Information Officer by the 5th of every month’ (Kitui County).

ANC, antenatal care.

Source: USAID SQALE programme data, 2018, unpublished.

Decision tree model development and structure

We developed a decision tree model of the patient pathway for pregnant women in Kenya, following them along the ANC pathway to delivery. Model development was an iterative process. First, we conducted a scoping review of economic models for ANC, with priority given to those from Kenya, followed by sub-Saharan Africa and then LMICs more generally.40–42 Second, we developed preliminary structures, on which we sought input and feedback from healthcare professionals (managers, doctors and researchers) working in the Kenyan healthcare system, several of whom are included as coauthors. Model verification exercises involved this cycle of feedback and refinement of the model.43

Figure 1A and B show the final decision tree structures for infants and pregnant women, respectively. Due to the complexity of the tree structures, maternal and infant outcomes were analysed separately. In figure 1A, we assess the impact of early (before 16 weeks) ANC initiation on outcomes related to maternal HIV, maternal anaemia and maternal syphilis infection (ie, ‘sick mother’). Specifically, those outcomes are congenital infections (syphilis and HIV), low birth weight (<2500 g) and infant mortality. The same tree structure is replicated for the standard of care or comparator arm of the decision tree (not shown). In figure 1B, we examine the impact of SBA on maternal mortality with and without the QI intervention.

Figure 1

Decision tree model schematics for ANC influenced by community health workers in Kenya. (A) A schematic intervention arm of the decision tree for the infant outcomes; the same tree is repeated for the status quo arm in the decision model. Here, ‘sick mother’ refers to mothers with HIV, syphilis or anaemia; ‘sick baby’ refers to either low birth weight, congenital syphilis or HIV-infected plus infant outcomes; these are each delineated separately in the model. (B) A schematic of the decision tree for the maternal outcome of interest. ANC, antenatal care; QI, quality improvement; Dx, diagnosis; Tx, treatment; inapp., inappropriate.

Technical details of the model structure and contents, costs and effectiveness measures, and deterministic and probabilistic sensitivity analyses are presented in online supplemental file 2. Data on the likelihood of each outcome at the chance nodes are shown in table A.1 in online supplemental file 2.

Evaluating cost-effectiveness

We estimated several incremental cost-effectiveness ratios (ICERs) comparing current community healthcare and the QI intervention, where the ICER gives the additional cost required to achieve an additional outcome. The numerator for all ICERs was cost, determined by the estimated aggregate incremental cost per subcounty of providing the intervention in addition to the cost of routine care (detailed in online supplemental file 2). The denominators or outcomes used included the following:

DALY averted (combines life years lost due to premature death with life years lost due to reduced quality of life, weighted by disability and Kenyan life expectancy).

Priority policy outcomes of the following:

Additional pregnant women attending early ANC.

Additional skilled delivery.

Priority health outcomes of the following:

Infant death averted.

Maternal death averted.

The denominator or effectiveness in each ICER was calculated using the impact of the QI intervention on a reference target population of 12 208 pregnant women annually per subcounty. (To estimate the population of pregnant women per subcounty annually, we took the average of two values: the first takes the average population of the three costing subcounties (in Nairobi, densely populated)30; the second of the values was obtained from national data: the national population from 2019 census divided by the number of subcounties nationally. Each of these values was then multiplied by the percentage of the Kenyan population that is female, age distribution and fertility rate44–46; detailed in online supplemental file 2.) A detailed table of parameters (table A.1 in online supplemental file 2) provides the data that inform the model described in figure 1.

We conducted two types of sensitivity analyses to examine the impact of uncertainty on the input parameters: deterministic and probabilistic. In the deterministic analysis, we manually change point estimates for key parameters that differ between the two trees (those directly observed for early ANC uptake and skilled delivery in outcomes and costs of the intervention). These were changed to represent the extremes of the possible range. In the probabilistic sensitivity analysis, we assessed the simultaneous change of multiple parameters within the range and according to the probability distribution of values for each. For details on the distributions used, see table A.1 in online supplemental file 2.

Clinical assumptions

Assumptions were made about the clinical conditions to simplify the decision model. However, as clinical or facility-based quality was held constant across the two arms of the study, we expect limited impact of clinical nuance on the findings. We have summarised the assumptions and their potential impact on our findings in table 1.

Table 1

Clinical assumptions in decision model

Assumption	Likely effect on estimated outcomes	Generalisability
No change in the clinical quality of care at health facility level (ie, ANC visit quality) due to the community-level quality intervention.	Would be more likely to improve.Conservative assumption; underestimates benefit.	Largely reflected in real-life field observations that no improvement happened without additional inputs at facility level.
Patients adhere to treatment as prescribed.	If adherence is poor, some outcomes will be worse in both arms (possibility 1), so no effect.If adherence is higher in the intervention arm, then our assumption would mean we are underestimating the benefits (possibility 2).	A patient’s relationship with community health worker may impact adherence.In real life:Syphilis treatment stock-outs frequent.HIV treatment well taken.Iron not taken well—people stop this.
TPHA assumed perfect sensitivity/specificity.	Some people with previously treated syphilis will still have positive TPHA on a rapid test, resulting in (low) overestimate of prevalence, therefore slightly overestimating benefit.	Some people who do not have active syphilis may get unnecessary treatment.
No confirmed HIV diagnosis (single test only).	A few people with a first positive result will have a false positive (<4/1000). Overestimate of prevalence may result in slight overestimate of benefit.	Unlikely to influence results or generalisability to other contexts.
No interactions between diseases/comorbidities.	Likelihood of infection with each disease was treated as independent variable. This overestimates the number who benefits from intervention but underestimates the size of the benefit because of increased severity.	Treatment selection may vary by comorbidity (we have used data on the first-line treatment rates for uncomplicated single infections).
Prematurity overlaps with low birth weight.	Gestational age is difficult to measure.	In the model we have not considered gestational age as an outcome given this is violation of independence. Association of prematurity with different diseases considered in the model is less clear, but we recognise this as an important infant outcome that also influences mortality.

ANC, antenatal care; TPHA, Treponema pallidum particle agglutination assay.

Patient and public involvement

Patients and the public were not directly involved in the research question development or analysis. However, the research questions were informed by evidentiary needs of subnational health system managers involved in investment decisions for community and other health programmes expressed as part of the intervention. One of those is a coauthor of this paper (EM).

Results

Incremental costs and outcomes

In table 2, we report the full results of the deterministic analysis of the costs, outcomes and cost-effectiveness for different outcomes of interest. In the middle column, we show the costs and outcomes at the subcounty level. For the incremental cost of $34 133 per subcounty, the intervention averts 126 DALYs in each subcounty. Of these, some are from decreased mortality (reduction in infant deaths (0.9 averted per subcounty) and reduction in maternal deaths (2.4 averted per subcounty)) and some from decreased morbidity. There are also increases per subcounty in both annual numbers of skilled births (1441) and early initiation of ANC in the first trimester (195), as shown in table 2.

Table 2

Deterministic predictions of incremental health impact and incremental cost-effectiveness of QI for community health systems intervention44–46

Incremental cost of the intervention	Per subcounty	At national scale*
Detailed costing breakdown presented in Kumar et al 30	$34 133	$2 564 859
Estimated annual number of:	Per subcounty	At national scale
DALYs averted	126	13 930
Clinical outcomes
Infant deaths averted	0.9	93
Maternal deaths averted	2.4	272
Policy targets
Skilled births	1441	160 636
Early ANC initiations	195	21 781
ICERs: incremental cost (2017 US dollars) per:
DALY averted	$249
Clinical outcomes
Infant deaths averted	$37 536
Maternal deaths averted	$5654
Policy targets
Skilled births	$10
Early ANC initiations	$155

*In the Kumar et al paper, the authors provide the per capita cost of the intervention. To estimate the cost at a national scale here, we have multiplied that by the population of Kenya as determined by the 2019 census.

ANC, antenatal care; DALYs, disability-adjusted life years; ICERs, incremental cost-effectiveness ratios; QI, quality improvement.

If taken to a national scale, at the current efficacy we estimate that the intervention would avert nearly 14 000 DALYs per year of implementation and would increase the annual number of skilled births by over 160 000 in Kenya, averting 93 infant deaths and 272 maternal deaths annually out of an estimated 1 361 326 pregnancies per year.44 45 These deterministic estimates are shown in table 2 in the far right column, with details in online supplemental file 2.

Incremental cost-effectiveness

The intervention requires an incremental investment of US$249 for each DALY averted. For the policy priority outcomes, additional skilled births cost approximately $10 each and additional early ANC initiations about $155. In the probabilistic analysis, we found 76.4% of the 1000 runs under the average of the threshold range values. Under the least strict threshold or high end of the threshold range (US$621), 93% were cost-effective. The full results of the deterministic and probabilistic sensitivity analyses are found in Figures A.1, A.2 in online supplemental file 2 and online supplemental file 3.

Discussion

Our model has shown that QI for community health is cost-effective compared with the current standard of care in community health in Kenya. The benefits of QI as a health system strengthening intervention can be examined through the lenses of different clinical conditions; here we have selected maternal health as both a national priority and a target of community health worker efforts. There are quantifiable benefits of community QI on policy priorities of increasing SBA and early ANC initiation, and these are the drivers of the impact in this decision tree model. The model shows the cost per DALY averted to be $249 and cost-effectiveness in over 75% of cases per the cost-effectiveness threshold selected.

Building a case for investing in QI at community level

Our finding that the QI intervention at community level is cost-effective is in keeping with other economic evaluations of community-level health systems strengthening efforts. Every intervention assessed by Nkonki et al 47 in their systematic review of community health interventions was cost-effective. Despite this value for money, community health has been chronically underfunded by domestic financing in most LMICs.48 This is in part due to focus on a curative approach to healthcare driving funding for treatment over prevention efforts.47 It is also because community health is a service delivery platform rather than an intervention—it is easy to conceptualise buying more chemotherapy drugs for a hospital, but less immediate to invest in capacity building or QI. This is especially true for those investment decision-makers who are subject to electability considerations and may prioritise ‘visible’ hardware investments in infrastructure over health system strengthening.8 49 A series of investment cases at global and national levels48 50 51 underscore this point, yet recent research by Lu et al 52 suggests that this has not been successful in increasing financing for community health.

Even once a decision to finance community health programmes has been reached, investing in quality of care at community level suffers in comparison with expansion of services (either through additional staff or new disease-focused programmes). In 2018, three major reports on quality were published that highlighted the importance of quality care at facility level,53–55 but the quality of community health programmes was not included in a meaningful way.10 It is here that our work links the limited community-level quality of care literature with the limited economic evaluations of community health work to move towards investing to improve practice—meaningful coverage that leads to improved health.

Economic evaluation can help address the disconnect between evidence around quality and investment in QI only if it addresses decision-makers’ needs. At global, national and subnational levels, decision-makers often cite concerns about whether existing economic research can be generalised to their setting and describe structural and capacity barriers to economic evidence commissioning and use.56 Koon et al 57 described the process embedding health systems research in decision-making in LMICs, emphasising both health system or contextual factors as well as factors about the evidence-generating organisation that increase trust in the evidence produced. This work focuses on evidence generated in a given context for the same context. More recently, Vanyoro et al 58 have explored the barriers to health systems research uptake in LMICs, emphasising ‘ownership’ as an important intermediate step between evidence generation and use in decisions, policy and practice. Both of these differ in a fundamental way from much of the economic evidence in community health for LMICs, which is often generalised from another setting and/or uses externally defined thresholds for cost-effectiveness, making it less trusted and thus less easily ‘owned’ by national decision-makers. Thus the prioritisation of what to evaluate economically incorporates assumptions about the decision-maker, their values and their investment priorities. In Kenya, both national and county decision-makers have focused on quality of maternal and newborn care to reduce mortality as a priority health indicator for change. The policy outcomes examined in this analysis are evidently valued by decision-makers, given their prominence in the Kenyan national strategy.59 By elucidating decision-maker priorities for evidence, and the likely structures or mechanisms for using it, evidence uptake may be improved.

Economic evaluation of complex service delivery interventions

A conceptual framework for the causal pathway showing how community QI strengthens the health system to yield downstream benefits is shown in figure 2. This places the QI cycle at the centre of the structures, processes and decisions that underpin the functioning health system, in which community health plays an integral role. Examination of this causal pathway illustrates clearly why a full economic evaluation of community-level QI is challenging: it is a complex intervention and is operating in a complex system.60–63 Complex interventions elicit three specific challenges for cost-effectiveness analysis: diverse or heterogenous outcomes; complex and indirect links between intervention and desired outcomes; and violation of the assumption that the outcomes of the intervention can be isolated from the healthcare system context.64 Figure 2 shows how these challenges were directly experienced in our evaluation: the non-linear causal links between intervention and intended outcomes; impacts at multiple levels in the healthcare system, on multiple stakeholders and across multiple health areas; and the high degree of flexibility in the intervention through selection of QI problems as a behavioural or service delivery intervention.65–67 Ongoing discussions with CHVs and county leadership have helped us to understand the intended and unintended consequences at each step of implementation and build up a picture of the complexity. This also gives us confidence in the fact that the benefits of the intervention are likely to extend beyond the narrow area from which we have explicitly derived them.68 For a decision-maker, this means an investment in community QI is likely to return more benefits than what was estimated here.

Figure 2

System map of plausible impact of community quality improvement (QI) health outcomes for priority health issues, including antenatal care.

In considering the impact of community QI on health outcomes in figure 2, improvement at the community level is mediated by the quality of care obtained in primary care facilities (held constant in the two arms of the decision tree). Facility-based QI has been shown in many locations in the region to yield positive effects in maternal and newborn healthcare outcomes, like those evaluated here.69–72 However, rates of early presentation and diagnosis/treatment without community involvement remain persistently low due to cultural reasons, poor access and costs.73–75 The converse is also true: without good facility-level care and treatment, community-level QI does not yield health impact and communities lose trust in community health workers that referred them. In future, we would propose joint community–facility primary healthcare QI teams to collaborate at different points in the continuum of care. It is only when such linkages and continuum of care can be created and sustained in the system that referrals can function optimally, integration between vertical programmes can be achieved, national policy revised and systems sustained, ensuring real progress towards UHC.

Generalisability and limitations

We have likely underestimated the cost-effectiveness of community QI because we have underestimated the benefits in two ways. The first is through limiting the measurement of benefits of this health system strengthening intervention to a single technical area: maternal health. We cannot assume what QI problem a team will select; indeed, we expect them to address different problems over time as their capacity is built in these transferable skills and they work iteratively through the PDSA cycles. By selecting just one condition or health area for which to measure outcomes, we are purposefully underestimating the benefits of a QI intervention at community level. Having shown the cost-effectiveness ratio is less than the selected threshold (see online supplemental file 2 for details), then a full quantification of all the benefits across health areas is unnecessary. The second reason for the underestimate is the choice of impact data33: the improvement in outcomes is driven by indicators (in the infant outcomes tree in figure 1A, this is the rate of early ANC initiation; in the maternal outcomes tree in figure 1B, this is the rate of skilled delivery) that were not the target of the local QI interventions in study sites. Rather, they were related policy priority areas that showed improvement after the intervention. As such, we expect a focus of local QI interventions on these areas might yield additional benefits.

Within Kenya, the intervention effectiveness data on increases in early ANC attendance and SBA came from Migori County and were measured through lot quality assurance sampling76; thus, we have reasonable confidence that the study data represent Migori County. In Migori, there are lower than national average rates of early ANC. Poor performance on health indicators is counterbalanced by strong leadership and a positive funding environment for maternal and community health. In selection of parameter data for the other incidence and outcome parameters, we have prioritised nationally representative data and therefore suggest that these findings could be generalisable to Kenya nationally. However, as county governments are the healthcare fundholders in Kenya, a county-level analysis would be recommended to define specific funding requirements and should include more granular detail on policies, disease and population.

There are two main factors that may influence the generalisability of the study findings beyond the Kenyan context to other LMICs that use community health workers as part of their primary healthcare system. First, in the selection and comparability of study site, as rates of SBA and early ANC are lower in many countries than at baseline in Migori County, we would expect that the model may underestimate potential benefits of the intervention in other contexts. As such, we consider the determination of ‘cost-effective’ robust for generalisation. Second, in the selection of the QI priority issue (ANC/SBA) vis-à-vis responsibilities of community health workers, community health workers in almost all countries deal with maternal health and conduct health promotion with pregnant women (usually among other tasks). By selecting maternal and newborn health as the priority areas for which to assess benefits, this helps make the case for generalisability beyond Kenya.

Conclusions

In conclusion, this health system strengthening intervention to build capacity in community QI was shown to be cost-effective, with impacts derived from improvements in maternal health. Investment in quality of community healthcare can drive Kenya to achieve improvements in maternal and child health. The impact of QI in primary healthcare settings could be increased through leadership and coordination between teams at community and facility levels. Functional primary-level QI teams could improve referral, treatment, adherence and outcomes across multiple health areas in a more equitable way.