Unraveling PBF effects beyond impact evaluation: results from a qualitative study in Cameroon.

INTRODUCTION: Performance-based financing (PBF) has acquired increased prominence as a means of reforming health system purchasing structures in low-income and middle-income countries. A number of impact evaluations have noted that PBF often produces mixed and heterogeneous effects. Still, little systematic effort has been channelled towards understanding what causes such heterogeneity, including looking more closely at implementation processes.
METHODS: Our qualitative study aimed at closing this gap in knowledge by attempting to unpack the mixed and heterogeneous effects detected by the PBF impact evaluation in Cameroon to inform further implementation as the country scales up the PBF approach. We collected data at all levels of the health system (national, district, facility) and at the community level, using a mixture of in-depth interviews and focus group discussions. We combined deductive and inductive analytical techniques and applied analyst triangulation.
RESULTS: Our findings indicate that heterogeneity in effects across facilities could be explained by pre-existing infrastructural weaknesses coupled with rigid administrative processes and implementation challenges, while heterogeneity across indicators could be explained by providers' practices, privileging services where demand-side barriers were less substantive.
CONCLUSION: In light of the country's commitment to scaling up PBF, it follows that substantial efforts (particularly entrusting facilities with more financial autonomy) should be made to overcome infrastructural and demand-side barriers and to smooth implementation processes, thus, enabling healthcare providers to use PBF resources and management models to a fuller potential.

A number of impact evaluations of performance-based financing (PBF) programs have pointed to the fact that PBF often produces mixed and heterogeneous effects.

Relatively little scientific effort has sought to understand how implementation processes affect the efficacy of an intervention and can explain the observed outcomes.

This study represents one of the first attempts to use qualitative research to explicitly explain heterogeneity in the effects detected by the quantitative impact evaluation of a PBF program.

Heterogeneity in the outcomes produced by PBF could be attributed both to health systems factors hampering the potential of the intervention to achieve its set objectives and to the ways individual healthcare providers responded to the intervention.

Based on our findings, we propose three, broad recommendations for policymakers: (1) pay closer attention to implementation challenges and bottlenecks linked to pre-existing health system structures, (2) consider demand-side barriers and monitor the potential for PBF to be effectively pro-poor and (3) invest in measures to ensure greater equity across all districts and facilities.

Background

Performance-based financing (PBF) is a health financing reform that entails a shift in purchasing modalities by introducing a transfer of resources to providers based on a contract, which defines a bonus payment according to a fee-for-service schedule for a selected list of services, conditional on a measure of quality.1 In the last decade, PBF programs have proliferated across many low-income and middle-income countries, particularly in sub-Saharan Africa.1 Empirical research has accompanied this growth and focused primarily on quantitatively assessing PBF’s impact on: health outputs and outcomes, patient satisfaction, and the quality of service delivery.2–8 Some of these studies have shown a positive association between PBF and health service coverage or quality, but most have uncovered mixed results. Recently, as part of an effort to understand these mixed results, a number of qualitative and mixed-methods studies have emerged exploring PBF’s implementation processes9 10; providers’ and communities’ perceptions of PBF11 12; providers’ incentives, motivation and performance13–15 and implementation perspectives among national stakeholders16 or in a cross-country analyses.17

Gaps remain in understanding why and through which mechanisms PBF has (or has not) attained expected results or has achieved mixed or heterogeneous results, that is, positive results on some indicators or for some facilities and areas, yet no impact on or in others. PBF proponents and detractors are grappling with questions about whether the reform’s successes or setbacks lie in design or rather in implementation; whether key assumptions inherent to PBF (ie, linking payments and results, independently verifying results, fostering managerial autonomy of facilities and so on) are playing out as expected; how contextual factors contribute to outcomes and whether adjustments could be made to improve programmes.18 19

In this paper, we present results of a qualitative study that seeks to address a gap in the literature by looking specifically at experiences of PBF in Cameroon. Building on findings from a quantitative impact evaluation,8 this study investigates factors that may explain heterogeneity in performance across PBF indicators and facilities. The study also explores how providers, managers and communities perceived the design and rollout of PBF. As several low-income and middle-income countries move to scale with PBF, evidence and lessons learnt herein will be useful to inform implementation well beyond our study setting.

Methods

Study setting

In Cameroon, socioeconomic conditions have deteriorated in recent decades and the relative rank of the country fell from the 97th place in the 1980 Human Development Index to 153rd in 2014.20 Several key millennium development goals were not achieved. Under-five mortality, maternal mortality and general adult mortality (15–60 years) were listed, respectively, at 88/1000, 508/100 000 and 403/1000 in 201521; skilled birth attendance has settled at around 60%–65% since the 1990s,22 and HIV prevalence (4.5% in 2015) continues to be substantially higher than in neighbouring countries.23

With the objective of increasing utilisation and improving the quality of health service delivery, the Ministry of Health (with financial support from the World Bank) in 2011 launched a PBF pilot in the country’s Littoral region, followed by a scale-up in 2012 to North-West, South-West and East regions, covering approximately 500 public, private and faith-based facilities (including primary and secondary facilities) across the four regions.24 25 The introduction of PBF was facilitated by a shift in political discourse favouring the implementation of more accountable health financing purchasing structures.25 Per the design, PBF bonuses were awarded for a list of services with a focus on maternal and childcare (see online Supplementary additional file 1). Bonuses were adjusted for quality of service delivery as determined based on a quality checklist. PBF payments were further adjusted adding an ‘equity bonus’ (a percentage of the basic PBF bonus, which varied between regions) to address contextual differences (such as remoteness, rural location or difficulty in access), which could affect service provision. Furthermore, facilities were to provide free services to the very poor identified in the communities and would receive a higher bonus payment from the PBF programme for services provided to the very poor to account for forgone user fees. PBF facilities filled in a monthly activity report, later verified by the ‘fund-holding agencies’ in collaboration with District Health Management teams, based on which payment was made monthly. PBF bonuses were paid to facilities, with a proportion allocated to health workers (shared using an ‘indice tool’) and the rest, alongside other incomes from user fees and/or budget allocations, used to cover facility expenditures (running costs, small investments, etc) as decided autonomously and outlined in a quarterly business plan prepared jointly by the facility staff.26

An impact evaluation (IE) was carried out between mid-2012 and mid-2015 in 14 of 22 districts across three of the regions (excluding Littoral). Under the IE, all facilities were randomised to receive PBF or one of three alternative control interventions, which provided varied levels of financial support and supervision not conditional on performance.8

Study design

We engaged in an explanatory-exploratory cross-sectional qualitative study,27 which sought explanations for patterns observed in the quantitative IE (explanatory), but also allowed an open investigation into the way PBF was experienced across a range of stakeholders (exploratory). During the qualitative study’s preparation, quantitative results as well as emerging interpretations and possible directions for qualitative research were discussed in depth with stakeholders in Yaoundé. Based on these discussions, the team finalised the study design, sampling, data collection strategy and tools and identified key themes to be explored qualitatively.

As highlighted earlier, preliminary results from the quantitative IE informed our qualitative work. Similar to other contexts, the IE in Cameroon showed that PBF produced overall positive changes in service coverage for some indicators (eg, maternal and child immunisation, family planning and HIV testing), but not for others (eg, antenatal care, assisted deliveries and child curative consultations) and recognised a limited capacity to reach the very poor. Looking at quality of care, the IE showed that PBF improved the availability of essential inputs and equipment, but not service delivery processes, in spite of an increased presence of qualified health workers and an increased patient satisfaction.8 The programme also induced heterogeneity in impact across and within districts, with some facilities performing better than others.

Data collection

The study team conducted in-depth interviews (IDIs) and focus group discussions (FGDs) in order to capture perspectives on both the supply (providers and managers) and demand side (service users) and to include respondents across health system levels (communities, facilities, districts, regional and central levels). Table 1 illustrates the sample, data collection instrument and themes addressed in each set of interviews. In each region, we interviewed health workers in three purposively selected primary facilities (a high performer, a delayed performer and a low performer) and in one secondary facility to ensure representativeness across districts and across public and non-public facilities. District medical officers (DMOs) were interviewed in the same districts and women (service users) in the catchment areas of the same sampled facilities.

Table 1

Summary of respondents by level, type, number and instrument used

Level	Number and type of respondents	Instrument type	Description of data collected and topics covered
National and Regional	Ministry of Health officials: five Regional delegates: three Fund-holding agency personnel: three Total=11	IDI	Implementation challenges and solutions devised Role of PBF in shifting operating processes within the health system PBF and facility autonomy Targeting of the very poor— knowledge, applications, challenges
District	District Medical Officers (DMOs)→ three in each region Total=9	IDI	Changes in work schedule induced by PBF PBF ability to produce change in service coverage PBF and facility autonomy Mitigation strategies for challenges induced by PBF implementation PBF and health worker retention Targeting of the very poor—knowledge, applications, challenges
Facility	Hospital-based providers: one hospital per region→ three staff per hospital (family planning, maternity ward, head of facility) (total nine) Health centre-based providers: three health centres per region→ two staff per health centre (family planning, maternal and child services) (total 18) Total=27	IDI	Changes in work schedule induced by PBF PBF ability to produce change in service provision PBF and facility autonomy Mitigation strategies for challenges induced by PBF implementation Satisfaction with operational aspects of PBF implementation Mitigation strategies for challenges induced by PBF implementation
Community	Three FGDs per region with female patients and service users Total=9 Three IDIs per region with women who delivered at home (total nine) Five IDIs per region with women who use/do not use FP methods (total 15) Total=24	IDI and FGD	Knowledge, beliefs and practices on specific illnesses/services PBF and care-seeking pathways Community PBF engagement PBF and access to care for the very poor
Overall totals	IDIs=71 FGDs=9

FGD, focus group discussions; IDI, in-depth interviews; PBF, performance-based financing.

We completed data collection between December 2016 and January 2017 with support from trained research assistants in collaboration with Centre d’Etudes et de Recherches Appliquées en Sciences Sociales (CERASS), using semi-structured interview guides, either in English or French depending on the region.

Data analysis

FGDs and IDIs were recorded, verbatim transcribed in the original language (either French or English)and coded without further translation. Coding was performed in the original language of the interview. French quotes included in this paper have been translated, while those in English are reported verbatim with no adjustment. Coding of the transcripts followed a thematic approach, using a mixture of deductive and inductive coding techniques, which is aligned with the explanatory-exploratory study design and allowed us to remain faithful to protocol and research questions, while remaining open to unexpected or unforeseen findings. We applied analyst triangulation (two researchers independently coded the material), data triangulation (multiple sources of data) and method triangulation (different coding techniques). The final interpretation of findings was discussed among all authors and with selected key informants closely related to PBF implementation in Cameroon.

Results

Key findings are displayed to represent (1) views and perspectives on PBF, changes and challenges, (2) factors that explain heterogeneity in results across selected PBF indicators and (3) factors that explain heterogeneity in performance across PBF facilities.

Perspectives on PBF

Providers and managers interviewed held positive views of PBF and the changes it introduced. Many mentioned that PBF ‘encourages us’, it ‘motivates us’, it ‘improves our situation’, it ‘helps the facility’, it ‘makes us work hard’, and it ‘rewards good work’. The main reasons provided to explain the positive accounts concerned the substantial change in the mentality of health workers and in their attitude towards work. Health workers reported that the requirements of PBF and the standards and tools defined in the contracts helped them become more rigorous in their work and compelled them to strive for continued, collective improvements—as an interviewee said, PBF has brought ‘the spirit of getting better’ (DMO). Others stressed how PBF gave health workers a sense of direction and purpose.

“Now they have improved. If you work well you are going to be motivated and it also gives us a sense of direction that is very important (…). PBF has made us change our attitude: it is good to give your clients good services, which PBF can also buy” (Provider, District Hospital).

Individual financial incentives played a key role in the narratives of health workers on PBF. Financial incentives for health workers were often referred to simply as ‘motivation’ (both in English and French) during interviews and were seen as a fundamental positive element of PBF given incentives’ role in improving the livelihoods of health workers, and in particular those who had no other formal source of income. Moreover, individual PBF bonuses reduced the need for financial coping strategies, such as parallel sale of drugs.

To a lesser extent, improvements in the work environment (such as improvements in infrastructure and equipment, as well as the possibility to recruit additional personnel using PBF funds), further bolstered positive views. However, it was difficult, and at times impossible for many facilities, to address key structural challenges (eg, availability of running water and electricity) using PBF funds. While such a possibility was foreseen in principle, administrative barriers to autonomous decision-making (see the description of later findings hereafter) coupled with limited funds for low-performing facilities hindered capacity to promote substantial infrastructural changes. Therefore, structural features remained uneven across facilities and districts. Additionally, staff improvements were relatively short-lived because delays in PBF payments negatively affected the retention of those recruited on PBF funds.

Indeed, while the majority of those interviewed agreed that overall PBF had enhanced staff satisfaction and motivation and led to better quality of work, many noted that these positive effects have been diminishing with time, due to challenges related to programme implementation. One key concern was delays in PBF payment, which became starker after the IE period due to a change in the payment processes during the transition from pilot to national programmes.28 The prospect of this transition also introduced additional worries, as health workers feared the changes in the price attached to the single indicators that would follow. Workers and managers mentioned how delays demotivated workers, introduced the risk of reverting to illegal income-generating practices and created problems with staff retention. Moreover, without resources, many respondents felt that business plans became an empty exercise, while facilities were deprived of the means to cover expenditures for services provided to the very poor. The enactment of an actual autonomy for facilities was another often-mentioned implementation issue. Most of the providers interviewed agreed that autonomy existed in theory but in practice ‘it’s not respected’, ‘it’s limited’, ‘it’s only partial’, mostly because of the lack of funds (also linked to delays in payment) to realise decisions and make autonomy effective. As one respondent described,

“The main thing is that as long as we don’t have funds, we cannot manage (…). Well, we start to have a number of business plans, but then we cannot follow-up on those because we don’t have the money for it” (Provider, District Hospital).

Additionally, respondents reported that the autonomy in financial management envisaged by PBF largely clashed with existing laws regulating financial management at the facility level. Specifically, respondents reported that facilities are normally required to return part of their revenues to the central level. Facilities under PBF should in principle have been exempted from complying with this regulation, allowing them autonomous decision-making in financial management. In practice, however, respondents reported not always being able to work according to the PBF model, due to pressure to comply with pre-existing laws. On many occasions, due to pressure to comply with national regulations, PBF providers abdicated their autonomy in financial decision-making while on other occasions, they reported acting autonomously, but ‘with a degree of fear’.

Heterogeneity in results across indicators

Our analysis also allows to explore the heterogeneity in results across PBF indicators,8 in particular concerning the increase in uptake of vaccination and family planning and the absence of change in assisted deliveries. Two main, interrelated elements emerged that help explain this pattern.

Both providers’ and women’s narratives highlighted a change over time with providers becoming more proactive in the delivery of some services, in order to increase utilisation and maximise income. Both groups indicated that outreach and sensitisation activities (eg, community and school visits, establishment of outreach posts or family planning sensitisation integrated into antenatal, postnatal or child vaccination visits) had become more common in recent years. Some said,

“Especially because it is the ‘child completely vaccinated’ which are paid for (by PBF), we have to keep doing it, we have to increase our outreach activities” (Provider, District Hospital).

“In fact, yes (there are more children being vaccinated). (…) Now the nurses come to sensitize us, they even knock at our doors, ‘mami we are to tchouk (give) vaccines ooo’” (woman, South-West region).

This change of practice was focused on services, such as family planning and vaccination, which were already free and where an increase in coverage could be obtained via outreach and sensitisation, while other services (such as assisted deliveries) were less of a focus because strategies to increase uptake would have been more difficult to implement, especially given demand-side barriers (assisted deliveries and antenatal visits are charged a fee). Indeed, demand-side barriers emerged as a key factor to influence uptake of certain services, as both providers and women explained,

“Those who deliver at home do so because of financial reasons. It is a monetary issue” (Provider, Health Center, South-West region).

“I don’t know anybody that prefers to deliver at home. But there are some parents that deliver at home because of lack of finance” (woman, North-West region).

Respondents described demand-side barriers to include geographical (distance from the facility) barriers as well as financial ones, related to both direct costs (fees) and indirect costs (transport). Demand-side barriers were not explicitly addressed in the PBF programme and they would have been more difficult to overcome by providers alone. Although a system was in place to ensure and improve access to care for the very poor, this was seen as ineffective by providers, managers and women. Interviews showed that the mechanism was not well known by providers, especially lower cadres in primary facilities. Additionally, its design and the way the very poor were identified varied by region and was described as ‘anarchic’ by one respondent, who emphasised that identifying the exceptionally poor was difficult in the absence of structured procedures to do so. Finally, delays in the payment of PBF bonuses to facilities made it impossible to recover the costs of care provided for free to the poor.

Heterogeneity in performance between facilities

We explored the issue of heterogeneity in performance between PBF facilities formulating hypotheses and potential explanations and found that factors influencing facility performance extend across three levels: contextual, facility and individual.

First, contextual factors related to the position of the facility (eg, rural/urban, ease of access and supply, densely populated vs large and sparsely populated area, serving a wealthy or poor or mobile population, competition with other providers and so on), as well as historical factors (length of facility’s presence, trust of communities) seemed to influence the performance of the facility. For example, a low performing facility was operating on an abandoned industrial site which had a dwindling and poor population. In those cases, as one of the respondent stressed, it was difficult to achieve high performance, while it was easier to do so in more heavily populated areas:

“The principal reason is the population in the catchment area. The greater the population, the better the performance” (DMO).

While the equity bonus in place should have helped facilities overcome some of the context-related issues, its implementation which was partial and uneven across regions may have contributed to the variation in facilities’ performance.

Second, the availability of infrastructure (decent buildings, access to water and electricity and so on), materials and equipment and the presence of qualified staff, predating the introduction of PBF, were often mentioned to explain a facilities’ performance and constraints. Many noted that PBF started without considering the level of preparedness of facilities in terms of staff, equipment and infrastructure and this led insufficiently equipped facilities to struggle. As one district manager said,

“When PBF came, they did not put a level ground. They just started, they took health center equal to health center. Whether you have one staff, whether you have twenty staff, whether you have a microscope, whether you have equipment… ” (DMO).

Finally, factors related to individual health workers’ (and in particular facility in-charges’) skills, such as their seniority, leadership, management capacity, entrepreneurial spirit and their understanding of PBF based on PBF-specific training and previous experience in a PBF facility, were reported to have an impact on the facility’s performance. Heterogeneity in performance was often explained in relation to the capacity of single managers to make use of the increased, although limited, autonomy granted by the programme, to identify effective changes in practice, by overcoming the fear to make mistakes. As a DMO said,

“(There are) some health facilities in the district that have very sharp leadership and with that sharp leadership they can put in place strategies that will lead to better productivity. There are others that their leadership is a bit problematic. We have so many health centers that have nurse aides as chief of center because the state registered nurses are not there. We have centers with one staff” (DMO).

Discussion

This qualitative study brings together narratives of actors from both the health system’s demand and supply side that were affected by the introduction of the PBF pilot programme in Cameroon. The key findings resonate with those from other settings, as they show that overall providers and managers held positive views of PBF because it changed attitudes towards work, bolstered a sense of direction and purpose and rewarded those involved with financial incentives.9 11 13 15 29–31 However, a series of challenges hampered the capacity to produce change to an extent that would have been possible and may explain some of the mixed results found in the IE. A challenge concerned the limits to facility autonomy and the discrepancy between autonomy de jure under PBF and de facto.10 Similar to other contexts,9 10 13 15 32 delays in PBF payments (although starker after the IE period) were a source of frustration and ultimately led to demotivation of health workers and may have had negative effects on the programme’s results.

Importantly, this study adds to the existing literature on PBF as the analysis of providers, managers and communities’ perspectives builds on a quantitative IE and thus sheds light on the factors explaining the impact of PBF (as found in the IE) and outlining causal pathways to change. Heterogeneity in results across indicators may be explained by the changes in providers’ practice (eg, more outreach, greater focus on quality of service provision) that focused on services where a substantial change in coverage could be obtained at a low cost for providers and demand-side barriers were less important. This echoes findings from Benin30 and Burundi3 and suggests that changes across a wider range of services can only be produced if changes in purchasing structures are accompanied by measures to remove demand-side barriers. Fewer changes were reported in the quality of care (eg, women complained about bad reception and lack of sufficient explanation of family planning’s side effects), which is more complex to obtain. This result is in line with the Cameroon IE8 and other studies,7 33 which expectedly found that PBF improved structural elements of quality of service delivery (eg, equipment and drug availability) but not process ones (eg, quality of clinical encounter). Furthermore, our findings indicate that heterogeneity in facility performance was largely attributable to the capacity of single managers to executing leadership skills, making use of the limited resources available and the increased, although constrained, autonomy granted to PBF facilities. Additionally, also in line with prior evidence from Benin, Tanzania and Nigeria,30 34 35 our analysis confirmed that differences in performance between facilities may be due to factors at contextual, facility and individual levels predating the launch of PBF. This seems to suggest that facilities which were advantaged for what concerns those factors (and therefore likely to be already performing well before the introduction of PBF) may have thrived under PBF, while worse-off facilities kept struggling and their performance improved relatively less, if at all. One can also postulate that worse-off facilities are also often the ones caring for the poorest populations; hence, they may face an additional barrier in improving their performance, given the additional expectation to provide care for free to the very poor. This emerging hypothesis warrants further quantitative research in Cameroon and other contexts.

Methodological considerations

From a methodological perspective, our study adopts a novel approach, which builds on an open dialogue with concerned policymakers and implementers and looks at explaining and exploring meaning behind the effects detected by the quantitative IE. While many studies have previously used qualitative methods to explore issues pertaining to PBF implementation,9 17 32 34 they have not, to our knowledge, been explicitly set to explain observed heterogeneity in effects. However, our qualitative study was not planned ex ante as part of a comprehensive mixed-methods study.27 This limited our methodological choices in terms of design and data collection and we might have missed out on specific opportunities, such as engaging in a longitudinal design or better integrating quantitative and qualitative data collection and analysis. Furthermore, the delay between the quantitative and the qualitative data collection inevitably affected respondents’ recall. While we were interested only in events pertaining to the IE phase, respondents at times struggled to differentiate events having taken place before, during or after the IE (eg, delays in payment, changes in vaccination practices). In addition, in spite of all due precautions we took to this regard (clear semistructured interview guide, trained local interviewers with no direct relation to the PBF programme, confidential interview setting), we cannot exclude the risk, inherent to all social research, which respondents at times provided socially appropriate answers to fulfil normative expectations.

Policy recommendations

As PBF is further scaled up in Cameroon and as it continues to garner considerable buy-in from the Ministry of Public Health, this study not only generates policy-relevant knowledge that can be used to inform PBF rollout in coming years in Cameroon but also highlights lessons that are generalisable to PBF in other low-income and middle-income contexts. We advance here a few policy recommendations. First, there is a need to pay more attention and effectively address implementation challenges, such as those causing delays in payment and limiting autonomy, which undercut facilities’ capacity to generate change and hinder PBF from functioning at its full potential.10 17 32 Second, demand-side barriers (such as geographical distance and payments at point of use) remain a powerful constraint in increasing the uptake of certain services. It would be interesting to test whether combining PBF with demand-side incentives, following the example of other countries, such as Burundi where PBF is implemented in the context of targeted free healthcare5 or Malawi where PBF has been coupled with conditional cash transfers,36 can produce effective changes on a wider range of indicators. This would also allow investigating whether PBF has a multiplicative effect when coupled with demand-side interventions as some suggested.5 Similarly, in line with what documented in prior studies,4 37 offering higher reimbursements for the care of the very poor does not seem to be enough to effectively reach disadvantaged populations and increase care among the very poor. Direct investments in targeting and in demand-side interventions are needed. Third, in order to ensure performance improvements across districts and facilities, a better functioning equity bonus, streamlined across all areas and including the different elements hampering facilities’ performance (including size of catchment area, poverty levels as well as remoteness) should be in place and it may be also worthwhile to provide additional resources to facilities at the start of the PBF programme for initial infrastructural and personnel upgrades, since in poorly equipped or otherwise disadvantaged facilities, PBF does not seem to be able to generate change to the extent it does in better-off facilities.