Introducing a sector-wide pooled fund in a fragile context: mixed-methods evaluation of the health transition fund in Zimbabwe.

INTRODUCTION: Aid effectiveness and improving its impact is a central policy matter for donors and international organisations. Pooled funding is a mechanism, whereby donors provide financial contributions towards a common set of broad objectives by channelling finance through one instrument. The results of pooled funds as an aid mechanism are mixed, and there is limited data on both methodology for, and results of, assessment of effectiveness of pooled funding.
METHODS: This study adapted a conceptual framework incorporating the Paris Principles of Aid Effectiveness and qualitative methods to assess the performance of the Health Transition Fund (HTF) Zimbabwe. 30 key informant interviews, and 20 focus group discussions were conducted with informants drawn from village to national level. Descriptive secondary data analysis of Demographic Health Surveys, Health Management Information Systems (HMIS) and policy reports complemented the study.
RESULTS: The HTF combined the most optimal option to channel external aid to the health sector in Zimbabwe during a period of socioeconomic and political crisis. It produced results quickly and at scale and enhanced coordination and ownership at the national and subnational level. Flexibility in using the funds was a strong feature of the HTF. However, the initiative compromised on the investment in local capacity and systems, since the primary focus was on restoring essential services within a nearly collapsed healthcare system, rather than building long-term capacity. Significant changes in maternal and newborn health outcomes were observed during the HTF implementation in Zimbabwe.
CONCLUSION: A framework which can be used to assess pooled funds was adapted and applied. Future assessments could use this or another framework to provide new evidence regarding effectiveness of pool donor funds although the frameworks should be properly tested and adapted in different contexts. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The study has a strong potential for influencing practice, since it presents a solid benchmark to evaluate in future the capacity of other pooled funds implemented in the health sector.

This is the first study which used a conceptual framework to assess performance and effectiveness of a pooled fund in Zimbabwe.

There is little in the existing peer-reviewed literature analysing the effects of multi-donor trust funds which makes it difficult to compare our findings with earlier studies.

There is no universally agreed or recommended framework to assess the pooled funds in place. We adapted Coppin’s (2012) framework which will require fine tuning before wider use.

Transforming this broad and multidimensional Paris Principles into quantifiable indicators do not allow for a robust assessment. However, we have used qualitative methods to capture key principles of aid effectiveness in depth.

Introduction

Improving the impact of aid, particularly in fragile and conflict affected states, has become a central policy matter for donors and international organisations.1 The theme of aid effectiveness emerged in the global development agenda in the early 2000, becoming central to the achievement of Millennium Development Goals and culminated in the Paris Declaration on Aid Effectiveness in 2005, which set the key principles of ownership, alignment, harmonisation, managing for results and mutual accountability as the pillars of effective and transparent aid processes.2 In 2008, the Accra Agenda for Action introduced the additional concepts of predictability of aid, conditionality and use of country systems.3

In 2011, the New Deal for Engagement in Fragile States emphasised key principles of aid for fragile states, identifying factors such as mutual trust, transparency, risk sharing, use of country systems, strengthening of national capacity and predictability of aid as additional principles necessary for effective aid in countries facing or emerging from crises.

The principles set by the Paris Declaration and the subsequent above-referred statements have triggered important dynamics in the flows of aid. ‘Multi–bi’ aid mechanisms, that is, a combination of bilateral and multilateral aid, have progressively flourished and have found their most typical instrument in pooled funds.4

Pooled funding mechanisms are to be understood as ‘arrangements where donors provide financial contributions towards a common set of broad objectives, and where allocations for specific activities are decided by a joint governing mechanism’.5 A typical pooled funding mechanism involves an administrative agent.5 Hence, pooled funding is a financing mechanism which aims to harmonise aid flows and reduce the transaction costs of aid for recipients by channelling finance from multiple donors through one instrument.6 It is known as a multi-donor trust fund, a pooled fund or a basket fund.6 In this paper, we will use the term ‘pooled fund’.

Since 2005, the use of pooled funds by donors has progressively increased, reaching nearly 20% of the total allocation of aid funding from the OECD Development Assistance Committee countries in 2012.4 Concurrently, Arregoces et al document that basket funding for maternal, newborn and child health (MNCH) consistently grew in terms of the overall official development assistance (ODA) support for MNCH reaching 6% of it in 2009, and then declined, accounting for only 1.4% of the ODA to MNCH in 2012.7 Such decrease was offset by a tendency from donors in investing instead in project-based initiatives.

Although pooled funds rarely dominate the aid flows at the country level, they assume a catalytic role that makes them of high importance, since the mechanisms and processes associated with pooled funds often trigger collaboration among donors and with governments. Examples of such collaboration include pooled funds implemented in Liberia, Central African Republic, Ethiopia, Nepal and Southern Sudan6

Recently, much has been written about this aid modality, and yet clear evidence on the performance and effectiveness of these funds is still relatively scarce. The 2011 World Development Report indicates that the ‘performance of multi-donor trust funds is mixed, with criticisms ranging from slowness to a lack of expectation management and mixed in working through national systems’. Similarly, a recent review from Barakat shows a mixed picture regarding the ability of pooled funds according to the principles of the Paris Declaration, pointing towards the need of generating further evidence on this aid modality.1

Furthermore, Coppin highlights that there is a paucity of comparable data on pooled funds’ performance limiting the ability of donors, recipients and implementers to transparently assess the effectiveness of this mechanism in enhancing aid effectiveness.8 Most importantly, there is no agreed framework or tool which can be used to assess the pooled funds.9

This paper intends to contribute to filling the research gap on this aid modality through the assessment of the Health Transition Fund (HTF) in Zimbabwe, a multi-donor pooled fund implemented from 2012 to 2015 countrywide to support the health sector. Measuring the effectiveness and the contributions of pooled funding to aid in fragile settings is essential, given the evidence that donors are reverting to project-based initiatives instead, at least for MNCH. Although pooled funds are not the golden standard in terms of aid harmonisation and alignment, they are a key intermediate measure in such direction—at least in fragile settings—to overcome the incoherence observed in ODA between the Paris Principles and the actual current practice of fragmentation and verticalisation of many development initiatives.10

The paper begins by describing the study setting to present the context of the study. It then describes the conceptual framework (Coppin’s framework) and the methods used for this assessment. This is followed by the results section which presents the assessment of HTF in Zimbabwe using the proposed conceptual framework. The paper concludes with a discussion of the framework relating to the results of HTF and pooled funds elsewhere, its limitations and potential applicability.8

Study setting

In Zimbabwe, the health system was severely weakened by a political and economic crisis that affected the country between 2000 and 2008. Such crisis resulted in a near collapse of the economy, that in turn caused a humanitarian crisis and massive migration to neighbouring countries. The impact of this crisis was further exacerbated by the severe HIV/AIDS epidemic that hit the country during the same period. Life expectancy dropped, and maternal and child mortality increased because of reduced access to goods and services.

In this social, economic and political context, the HTF was designed as a multi-donor pooled fund aiming to support the government in inverting trends in maternal and child health for all Zimbabweans.11 The Fund was implemented from January 2012 to December 2015, with an allocated investment of US$235 million. Department for International Development (DFID), the European Union, the Governments of Ireland, Sweden, Norway and Canada were the major contributors to the fund. Unicef was the fund manager. The initiative was designed to target the entire country. The main purpose of the Fund was to improve maternal, newborn and child health (MNCH) by strengthening the health system and scaling up the implementation of high-impact interventions for MNCH and nutrition. To achieve this goal, the funding mechanism was designed to support critical interventions such as: retention allowances to support the health workforce (nurses and doctors) in rural areas; procurement of medicines, vaccines and commodities for maternal and child health; introduction of a results-based financing (RBF) mechanism in 42 of the 60 districts of Zimbabwe to support the operations of facilities at secondary level and below and support to community-based interventions at scale.

Methodology

Conceptual framework

This paper has adapted Coppin’s framework to assess the pooled fund, HTF.8 For each of the Paris Principles of Aid effectiveness, Coppin identifies a range of indicators that are designed to capture the fit of the pooled fund under analysis with each of the five principles as below.

Ownership

A good pooled fund promotes ownership by aligning itself with the national strategies and plans and by engaging key players at the national level to ensure that local capacity and systems are strengthened. Four dimensions of ownership were explored in this study: the relationship of the fund with the government strategy; the representation of the government on fund committees; ministerial accountability for expenditure to the legislature and the location of the pooled fund financial management.

Alignment

The principle of alignment claims for development aid to invest as much as possible on country systems, so to build the capacity of national governments. The framework used in this study explores multiple indicators to assess alignment (table 1). They can be grouped in five major conceptual categories: the extent to which the fund is ‘on budget’; the flexibility of the fund; the alignment of the fund management to country systems and procedures; the existence of salary top-up mechanisms and transparency.

Table 1

Summary score of the key principles of the conceptual framework

Principle	Indicator	Scoring of HTF Zimbabwe
Ownership	Pooled fund relationship to relevant government strategy	1
	National government is represented on committees	1
	Ministerial accountability for expenditure to legislature	0.5
	Location of pooled fund financial management in ministry	0
		63%
Alignment	CABRI on budget indicators
	… on plan	0.5
	… on budget	0.5
	… on treasury	0
	… on parliament	0
	… on procurement	0
	… on accounting	0
	… on audit	n/a
	… on report	1
	Low proportion of funds that can be earmarked	1
	Flexibility of technical assistance to work beyond pooled fund	1
	Business conducted in national currency	1
	Business conducted in sync with the national financial year	1
	Business conducted using national budget classifications	0
	Salary top-ups go beyond management unit staff	1
	Pooled fund documentation and reports made publicly available	1
	Project preparation and approval guidelines available	0
		53%
Harmonisation	Regular interface with wide group of donors	1
	Experience and competence of fund administrator	1
	Protocol for misuse of funds includes national accountability processes	0.5
		83%
Delivery of results	Financing: commitments to projects/financing received	0.54
	Financing: actual spending/planned spending	0.88
	Flexibility to reallocate funds to different priorities within year	1
	No requirement of counterpart funding	1
		86%
Mutual accountability	Monitoring of pooled fund includes government processes	0.5
	Timeliness of pooled fund reports	1
	Independent (not joint) reviews	1
		84%

CABRI, Collaborative Africa Budget Reform Initiative; HTF, Health Transition Fund.

Harmonisation

The principle of harmonisation concerns with coordinated, transparent and collectively effective donor support. Three key dimensions of harmonisation were assessed: the interfacing of the fund with the wide group of donors; the experience and competence of the fund administrator and the existence of protocols for misuse of funds.

Delivery for results

The principle ‘delivery for results’ considers the ability of aid to produce outputs and to contribute to outcomes. The analysis reviews four dimensions of delivery for results: the fulfilment of financing commitments; the ability of the fund to spend allocated resources; the flexibility of the fund in reallocating resources and the requirement of counterpart funding.

Mutual accountability

To sustain mutual accountability, a good pooled fund mechanism is expected to set in place, transparent monitoring mechanisms, ideally built on national systems, as well as timely and regular reporting and independent evaluations.

Online supplementary table 1 presents the indicators proposed by Coppin for each of the principles; each indicator is assigned a score ranging from 0 to 1. Two independent reviewers (who did not directly involve with the design and implementation of the fund) were selected on the basis of their knowledge of the pooled fund design and features. The reviewers scored each of the indicators, using primary and secondary data available via HTF evaluation. Any discrepancy in scoring was solved by discussions among reviewers to reach consensus. A summary score was then calculated per each of the principles, using unweighted averages of all the indicators calculated for each principle. For each principle, the score ranges from 0% to 100%.

Methods

This study used a mixed-methods approach to evaluate the HTF against the Coppin’s framework.

Qualitative data

Primary qualitative evidence was collected through the HTF evaluation, using key informant interviews (KIIs) and focus group discussions (FGDs). Purposive sampling was used to identify and recruit potential participants for FGDs and KIIs. The evaluation conducted a total of 30 KIIs with national level policy-makers (n=5), members of district health executives (n=15), managers and healthcare providers from health facilities (n=13) within the districts of Matabeleland South, Mashonaland East and Masvingo provinces. The provinces and districts were chosen due to the difference in socioeconomic and health characteristics; in particular, the selection criteria included: geographical coverage of the country; different levels of health indicators across provinces and poverty levels were used to stratify and select districts. We also interviewed national level informants (n=7) from multilateral and bilateral organisations. A total of 20 FGDs were also conducted, comprising 124 community men and women and 73 volunteer health workers (VHWs). For interviews, questions were focused on effectiveness, relevance, efficiency and sustainability of the pooled fund. For FGDs, questions were focused on the MNCH situation and use of MNCH health services, experience in accessing those services and barriers and enablers to use services. All transcribed FGD and KII material were analysed thematically guided by the Coppin’s framework. The analysis was guided by a qualitative content analysis of Graneheim and Lundman which focuses on the key areas of consensus and disagreement,12 and where relevant, triangulating KIIs with FGDs.

Secondary data analysis

We also used available reports and data sets that were relevant to both the Zimbabwe health system and the HTF. Routine data from HMIS were obtained via the Ministry of Health and Child Care (MoHCC). Other reports were either accessed through development partners in Zimbabwe or via literature search conducted by the Liverpool School of Tropical Medicine team. The main secondary data sources used are included in the online supplementary table 2.

Ethics

Participation in the study was voluntary, and no incentives were given to participants. In line with the consent agreement, data for this study are not publicly available, and anonymity of study participants was maintained during data collection, data management and analysis.

Patient and public involvement

Patients were not involved in the design of this study.

Results

The results of the evaluation are presented against the Coppin’s principles and using the conceptual framework adapted for this study. Table 1 presents the final score of the HTF against the five principles, as per indicators adapted from the proposed conceptual framework.

Ownership

A review of national documents, complemented by KIIs, strongly suggests that the HTF was fully aligned with national policies and strategies as it was fully designed and implemented in line with the objectives and key priorities of the Zimbabwe National Health Strategy 2009–2013. It was also informed by the Maternal and Neonatal Roadmap 2007–2015 and by the National Child Survival Strategy 2010–2015.

The government was highly represented on committees, thanks for a well-defined management and governance structure (figure 1). The pillar of this structure was a steering committee, appointed as the core body overseeing the governance and decision-making processes of HTF. A fund manager (Unicef) was tasked to ensure the implementation on time and within the budget of all the initiatives approved by the steering committee. Day-to-day activities were overseen and coordinated by an HTF Secretariat, hosted by MoHCC. The steering committee was composed of the MOHCC and representatives from the civil society and multilateral and bilateral organisations. A review of available records showed that steering committee meetings included on average 50 invitees. It was chaired by the MoHCC, in the person of the permanent secretary and co-chaired by an HTF donor. Evidence from KIIs confirms a highly participatory approach of the committee (box 1, quote 1). Most interview respondents reported that by engaging a diverse range of stakeholders and high-level policy-makers, the Fund promoted ownership and leadership role of the MoHCC in the design and implementation of activities. Respondents believed that the structure of HTF governance was inclusive as it allowed key health system players to participate in planning and implementation. This promoted accountability and transparency (box 1, quote 2). Of note, some respondents perceived that the steering committee was too big to be effective in decision making and hence, having subcommittees would have been more effective.

Figure 1

Governance and management structure of the HTF. HTF, Health Transition Fund; LSTM, Liverpool School of Tropical Medicine; MoHCC, Ministry of Health and Child Care; RBF, results-based financing; VHMASS, Vital Medicines Availability and Health Services Survey.

"…there has been a lot of transparency, people know exactly who is bringing what and what it’s being used for and therefore I wouldn’t change anything in terms of the governance structure. I think that has worked very, very well and it also has brought confidence …in the whole HTF mechanism. Because everybody knows what is happening.” (A national level respondent from a multilateral organisation)

“I like the way that we engage very closely with the government, for ownership and capacity building. I like the way that was very transparent with donors…. it is a very transparent way of dealing with things and it makes it very easy to manage these funds because any decision that needs to be taken is escalated to the steering committee then we are clear.” (A national level respondent from a multilateral organisation)

Regarding financial accountability, data confirmed that the HTF was referred to in the annual budget statements of Zimbabwe on a yearly basis. Yet, the Government was not formally accountable for reporting the HTF expenditure to the Parliament. More importantly, all financial transactions were accounted for by the Unicef. The fund manager bore the financial risk of managing the Fund on behalf of the donors and used its own financial systems rather than government systems to account for the expenses incurred by the Fund.

Therefore, HTF Zimbabwe scored 63% with regard to the ownership principle (table 1).

Alignment

The Collaborative Africa Budget Reform Initiative criteria were used to assess whether HTF was ‘on budget’. The HTF relied on a thorough process of annual planning, well integrated with national structures and meetings which were approved by the steering committee. However, as Unicef managed the fund, national systems of budgeting, reporting, accounting and auditing were not used for the fund management. The procurement and supply management implementation arrangements entailed a single-source procurement modality via the Unicef Supply Division. Unicef directly delivered procurement items to the National Pharmaceutical Company (NatPharm) warehouses, and the NatPharm was responsible for distribution under financial support of the HTF. The HTF procurement system presented a high level of efficiency and fulfilled all the requirements expressed in annual MoHCC procurement plans at competitive price values. Yet, it relied on a parallel supply mechanism, and it did not invest in addressing the structural weaknesses of the national procurement system (box 2, quote 1). The assessment provides a picture of the HTF being largely an ‘off budget’ funding mechanism.

“You also need to start building the capacity of NatPharm to take over the role that these agencies are doing, in procuring and distribution. In terms of capacity building in terms of distribution has been put in place. But mainly for example the procurement aspect of it. Not a lot has been done… even in the distribution making sure that the systems are functioning properly. Capacitating NatPharm to be able to do it independently. That hasn’t been done. Everybody talks about capitalise NatPharm…but they don’t have the capacity to procure, they don’t have the capacity to distribute.” (A national level respondent from a multilateral organisation)

“…the program is being implemented to large extent through the Ministry of Health structure. The coordination is also led by the Ministry, the PS is the chair of the steering committee as donors chair with him. So, it has been flexible…just great flexibility….so that flexibility also I think has helped it be what it is as compared to other processes that are sort of rigid and…we can’t change anything.” (A national level respondent from a multilateral organisation)

Regarding flexibility of funds, a clear process was in place, whereby annual work plans and budgets were planned and approved by the steering committee. Mechanisms and processes allowed for continuous review of activities and related expenditure and guaranteed flexibility in managing the funds as per emerging needs and priorities (box 2, quote 2). The fund also provided flexibility in using HTF funded technical assistance for broader purposes than the fund oversight only.

Regarding the fund alignment with systems, the review of HTF records and reports confirmed that the business was entirely conducted using the national currency and synchronising with national financial years. However, the national budget classification was not used as the fund was managed outside of the MoHCC.

Salary top-up mechanisms was a rather central component of the fund. The fund supported retention and critical postallowances at the national, provincial, district and facility levels. According to 2015 HR staffing data, the scheme had contributed to reducing the vacancy rate since the implementation of the retention scheme in 2009. The overall vacancy rate for the key health professional cadres had fallen from 42% in 2009 to 30% in 2015; the number of doctors increased by 346 over the period, and vacancy rates for the nursing cadres fell by half from 14% in 2009 to 7% in 2015.

Despite those achievements, health workers perceived the scheme to be divisive causing conflict and resentment in some facilities as the rationale for who was benefiting was not always clearly understood. One of the common complaints over the years was the issue of the untimeliness and unpredictability of disbursements leading to confusion and frustration of the lengthy payment processes. Respondents also expressed dissatisfaction with the declining amount of the incentives and reduction in the allowances over the years, especially when there has not been a corresponding increase in government. Some indicated that nurses and midwives were staying in their jobs only because they had no other alternative nor did not have sufficient resources to leave. Strong mechanisms were found to be in place to ensure transparency. Project documentation including the initial proposal, annual reports, monitoring reports and thematic briefs were available and widely disseminated in the country.

The assessment of the alignment principle presents a total score of 50% (table 1).

Harmonisation

In Zimbabwe, in 2015 official development assistance accounted for more than 50% of the overall annual envelope for the health sector, and the HTF alone accounted for approximately 7% of that envelope. Other major donors to health included United States Agency for International Development (USAID), the Global Fund, the World Bank and the Global Alliance for Vaccine and Immunization (GAVI). Document review and qualitative data showed that the HTF was designed and implemented to complement and support other donors’ initiatives rather than duplicating efforts (box 3, quote 1). As an example, the RBF mechanism was scaled up via HTF, drawing on the model piloted with World Bank funding. Also, the health retention scheme supported via the HTF complemented and progressively absorbed the support provided to healthcare workers with Global Fund resources.

“For me the legacy of HTF is with good corporate governance, if there is good corporate governance UNICEF coordinating and resources are there, and if resources are channelled in a way which is holistic which covers everything.” (A national level respondent from the MOHCC)

“…I think there was also improved coordination at national level by bringing these funding mechanisms together in one basket. That was one thing that was a very huge positive in terms of focusing resources on the priority area. For me that was the driving forces of the success of that program.” (A national level respondent from the MOHCC)

More importantly, the HTF Steering Committee formed a platform where non-HTF donors could participate, which offered a unique opportunity for all partners to discuss policies and to ensure coordination between the various funding initiatives designed and implemented in Zimbabwe (box 3, quote 2).

Regarding experience and competence of the Fund Manager, the results of interviews and document review demonstrated that Unicef presented a unique and advantageous blend of consolidated and efficient fund management expertise and of technical capacity in assisting the MoHCC in designing and implementing health strategies. In addition to fulfilling the function of the fund manager, Unicef also played an active role in providing technical assistance to the MoHCC and in directly implementing some of the HTF activities (eg, procurement).

Regarding protocols for misuse of funds, this component largely relied on the internal financial and quality assurance mechanisms and procedures of Unicef.

The HTF scored 83% for the principle ‘harmonisation’ (table 1).

Delivery for results

The initial HTF proposal document presented an overall budget estimate of US$435 million. A total of US$235 million were allocated by donors, corresponding to 54% of the initial estimates. Of those, US$207 million (88%) were used. Overall, financial use analysis indicates a good degree of flexibility from the HTF in adjusting the distribution of planned allocation to needs emerging during implementation, oriented by the Steering Committee. Both in FY 2014 and 2015, the budget forecasts were readjusted to allocate additional resources to thematic area 3 by reallocating resources from other budget lines as shown in figure 2. Many central level stakeholders who participated in the interviews perceived flexibility as a key asset of the HTF. Finally, the fund had no requirement of counterpart cofunding.

Figure 2

HTF expenditure versus allocated budget, cumulative 2012–2015 (in US$).

The assessment highlighted that HTF was a well-established mechanism that supported the achievement of documented results in Zimbabwe throughout its implementation. The HTF log–frame analysis highlighted that 96% of the HTF output level indicators had shown progress during the implementation period (table 2). This suggests that the Fund strategies and management arrangements were successful in achieving the intended results of the initiative. As a result, the availability and readiness of MNCH services at primary and secondary level was largely restored across the country due to a mix of interventions including the provision of equipment and supplies, availability of trained and qualified staff, RBF, improved planning and supervision. Despite those achievements, HTF had challenges in achieving results in some of its supported programmes. This included restoration of services at the community level (box 4, quote 1)and the health workers’ retention schemes (box 4, quotes 2, 3).

Table 2

Average annual rate of change for tracer HTF outcome indicators

Interventions along the continuum of care	Tracer indicators	Pre-HTF	Baseline	Endline	AAR 2005/2006–2010/2011	AAR 2010/2011–2015
		DHS 2005/6 (%)	DHS 2010/11 (%)	DHS 2015 key findings (%)	(%)	(%)
Maternal and newborn health	Antenatal care visits (4+)	71	64.8	75.7	−1.3	3.1
	Skilled attendance at delivery	68.50	66.20	78.1	−0.5	2.4
	Postnatal care visits	30.30	27.1	51.1	−0.6	4.8
Immunisation coverage	Full immunisation (12–23 months)	52.60	55.60	73	0.6	3.5
Infant and young child nutrition	Exclusive breastfeeding 0–5 months	22.20	31.40	47.8	1.8	3.3
	Vitamin A supplementation	47.10	65.60	32.3*	3.7	−8.3
Care seeking and access to treatment for sick children	Diarrhoea treatment (ORS or RHF)	61.6	63.30	72.5*	0.3	2.3
	Children with pneumonia treated with antibiotics	7.9	31	34.3*	4.6	0.8

*From MICS 2014 as these indicators were not available in DHS Key Findings 2015.

AAR, Average annual rate; DHS, Demographic Health Survey; HTF, Health Transition Fund; MICS, Multiple Indicator Cluster Survey; ORS, Oral rehydration salts; RHF, recommended homemade fluids.

“As Village Workers, we should be equipped with kits to assist people (…). We also need improvement in communication. What I mean is that, we do not afford to communicate by mobile because of lack of financial resources. We expect some allowances.” (A community health worker, FGD)

“The divide and rule policy the Ministry of Health has is very dangerous. Someone working in labour ward gets an allowance, someone has got midwifery and working in outpatient is not getting it, but now when they happen to have a problem in maternity, which they know I can assist them, if they call me I don’t go, because I am not getting an allowance.” (A health worker from a district hospital)

“You sometimes go for long periods without anything and then when it comes you are not sure whether you have received what you are supposed to be receiving because it comes as a lump sum. Like now I see something on my phone that money is coming but I am not sure where it is coming from.” (A health worker from a district hospital)

In summary, regarding the principle ‘delivery for results’, HTF scored 86% (table 1).

Mutual accountability

Regarding monitoring, a complex logical framework was set in place since the inception of the HTF, with indicators at output, outcome and impact level defined. The log–frame entailed tracking a baseline value and then annual progress against a final target. The assessment highlighted some design gaps in the monitoring framework due to lack of specifications in the sources of information, definition of indicators and frequency of data collection.

Also, many indicators relied primarily on population-based surveys or on facility assessments. For instance, the Vital Medicines Availability and Health Services Survey was essential for the HTF to monitor performance. The mechanism was managed directly by Unicef with data collection and analysis subcontracted to a local agent. Ultimately, only a handful of all the indicators identified by the HTF to monitor performance relied on routine Health Management Information Systems (HMIS) data. This approach to monitoring generated continuous demand for ad hoc surveys, instead of stimulating investment in the quality and use of routine information and data.

Reporting was solid and regular as Unicef produced annual activity reports and shared it with the Steering Committee and donors. The HTF had an independent evaluation embedded in its design. The evaluator, Liverpool School of Tropical Medicine, was selected via a competitive bidding process and contracted by Unicef. The final evaluation report is publicly available on the Unicef global evaluation database.

The HTF scored 86% with regard to mutual accountability (table 1).

Discussion

Evidence and analysis of pooled funds are abundantly available via reviews and evaluations performed in various contexts.13–17 Yet, pooled funds are still an emerging mechanism,18 and the peer-reviewed literature on this aid modality and on its effectiveness is still insufficient. A common framework for analysis of pooled funds is lacking.

Based on a framework proposed by Coppin, this paper provides an assessment of the performance of the HTF during its implementation in Zimbabwe, from 2012 to 2015, evaluating the pooled fund against the Paris Principles of aid effectiveness (figure 3).8 This pooled fund was an optimal option to channel external aid to the health sector in Zimbabwe during a period of social, political and economic crisis. There is evidence that: the HTF worked well; it produced results at scale and quickly; it enhanced coordination and ownership at national and subnational level. However, and to some extent, the initiative did not always succeed in strengthening local capacity and systems. In fact, if on one hand the HTF investment in human resources for health was critical for creating long-term capacity, on the other hand for other interventions such as RBF or procurement, long-term conditions were not created during the life of the programme. This is understandable, given the situation in Zimbabwe at the time of its implementation, where the focus was on restoring essential services within a nearly collapsed system rather than on building long-term capacity.

Figure 3

Summary score: performance of the Health Transition Fund.

Ownership emerges as a key feature of the HTF. Our assessment provides substantial evidence of a high degree of ownership of the HTF at all levels of the health system, although the fund was not fully administered via national systems. Barakat documents that in many cases governmental involvement in pooled funds is virtual or minimal and that donors control the real decision-making process.19 To the contrary, the HTF showed an effective partnership and coordination mechanism within the health sector in Zimbabwe and strengthened corporate governance through the steering committee, where donors and the MoHCC met regularly, ensuring ownership and leadership. Also, the HTF proved to be a clear and well-understood mechanism throughout the health system; it enhanced planning at the district level and reinforced accountability systems at the facility level. This is in line with findings from Scanteam, which documents that the pooled fund is an important vehicle in terms of coordination and harmonisation.20

Regarding alignment, HTF presented mixed results. As reported by Graham, most pooled funds fall into the category of earmarked funds.21 Funding allocated to HTF was not earmarked by donors, although bound to annual work plans and budgets approved by the Steering Committee. Positive features of the HTF were the flexibility in using the funds for activities that were designed to fulfil government strategies and plans. Also, the HTF relied on annual plans of work and was not ‘projects based’ as other pooled funds are; this reduced ad hoc decision making and micromanagement and increased efficiency.14 22 Alignment with national systems was poor in comparison to other pooled funds.2 3 14 20 This was due to risk management considerations, during a period where systems were barely functional in Zimbabwe and where some donors had pulled out of the country for political reasons. Similar observation was reported by Martinez-Alvarez and colleagues9 that pooled funding was declined over their study period.9 Despite this weakness, the investment in lower levels structures and mechanisms of accountability and governance sustained via HTF formed a solid platform to enhance the capacity of the system to fulfil its mandate at the frontline. This was not observed in other pooled funds.22

Harmonisation is the basis of the aid effectiveness pyramid; our assessment concludes that the HTF performed well regarding harmonisation.23 As reported by Salama et al, before HTF, there was no forum in Zimbabwe for systematic sectoral level engagement between donors, UN agencies and the Ministry of Health.15 Martinez-Alvarez et al 9 also presented fragmentation of funding at national level in their case study in Tanzania9 as a key barrier to harmonisation and hence to aid effectiveness. As well documented by Dijkstra, donors’ divergence in administrative requirements, and more importantly in political agenda, can be a strong barrier to harmonisation and hinder the effectiveness of aid initiatives. In the case of the HTF, the Steering Committee functioned as a catalytic force, stimulating dialogue and coordination among HTF and non-HTF partners which proved to be useful in restoring the health sector in the country. Although other donors to Zimbabwe were not part of the HTF (World Bank; GAVI; Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM); USAID), there was a clear effort of streamlining high-level decisions under the Steering Committee. Progressively some of the key donors to the country were approached to join the HTF (eg, GAVI) or for technical coordination (eg, World Bank). Also, other UN technical agencies were involved in HTF-related technical dialogue. Prospectively, as the country embarked in the design of a second phase of the fund, the HDF, this approach helped to reduce transaction costs related to multiple and often parallel committees set up by various vertical initiatives or donors as observed in other pooled funds.14 In its review of transition funds as a model of innovative financing in various sectors in Zimbabwe, Salama et al identify accountability mechanisms as a key feature of the transition fund.15 The evaluation findings confirm this assessment. Whereas monitoring mechanisms for other pooled funds in Zimbabwe were set for ad hoc donor measurement, in the case of the HTF, despite the limited use of HMIS data, the monitoring mechanism became central to measure progress for the entire health sector.22 Also, all reviews and evaluations were strongly owned by the Steering Committee and used for evidence-based decisions.

As noted by Pearson,24 joint agreed strategies, expected results and goals between donors, government and implementing agents are essential to ensure that aid has a strong focus on results. In that, delivery of results was central to the HTF design and implementation. Targets were established at outcome and output levels that were aligned to and supportive of the national health strategy, and significant changes in maternal, newborn and child health were observed during the HTF implementation in Zimbabwe. While the under-5 mortality rate was stagnant during the 5 years preceding the HTF inception, data suggest that Zimbabwe inverted trends and reported significant reduction in child mortality since 2010, which decreased from 84 deaths per 1000 live births in 2010/2011 to 69 deaths per 1000 live births in 2015. During the same period, maternal mortality decreased from 960 per 100 000 live births to 651 per 100 000 live births.25 26 Our assessment highlights that HTF was instrumental in supporting this improvement as a consistent progress in coverage of key Maternal, Newborn, Child Health & Nutrition (MNCHN) interventions along the continuum of care was observed during the period 2009–2015. This was mainly possible due to a streamlined approach to governance and fund management that catalysed resources against a clear and shared plan of action. Bigsten and Tegstam27 suggest that improved coordination has potentially strong effects on harmonisation, ownership and alignment of aid and as a result on aid effectiveness. On the contrary, Martinez-Alavarez et al highlight that volatility of funding may affect the governments’ ability to plan and delivery for results.9 In Zimbabwe, while progress in achieving improved outcomes at country level cannot be attributed to HTF only, the findings of the evaluation consistently pointed towards the importance of the HTF in playing a central role of ensuring service delivery on one hand, and of aggregation, coordination, dialogue and decision making on the other, that benefited the entire health sector. In synthesis, the strong capacity of the HTF to deliver results was largely a function of good governance and coordination between donors, the fund management and the government and the capacity of donors to guarantee a predictable allocation of funds against set targets. Ball and Van Beijnum,5 in reviewing the experience of 15 pooled funds in transition situation, argue that leadership, coordination and participation of donors and government to the governance of pooled fund is essential to enhance effective fund management, and yet this is not often consistently achieved when designing and implementing pooled funds. In that, the HTF was instead a successful example.

At times where the political and administrative system was weak, the Fund served as an essential risk management vehicle that permitted to channel donors’ resources to the health sector reducing risks and transaction costs and permitting an effective delivery of results across various pillars of the health system. Commins et al argue that a critical trade-off embedded in the logic and design of pooled funds is that between the speed of service delivery and fiduciary risk versus capacity development and building of government systems.6 In fact, the strong focus of the HTF on delivering for results led to political and programmatic choices of investing in quick-win strategies and approaches aimed to yield returns in the short term. Consequently, the investment in local capacity and systems was not sufficient to ensure sustainability. A transition towards full implementation capacity from the Government lacked during the implementation phase.

Limitations of the study

This study has used qualitative methods complemented by document review to assess a pooled fund mechanism in Zimbabwe. Besides, we have used purposive sampling of specific groups of participants who were familiar with the pooled fund mechanisms and procedures in Zimbabwe. Hence, the results of this study may have limited generalisability beyond its immediate study sites as it included a limited number of stakeholders at a specific point in time. The content analysis guided by the conceptual framework impose an informed bias to the study.28 However, during analysis, we employed both inductive and deductive (within the predetermined themes) coding. The trustworthiness of the findings was ensured by the collective experience of the authors complemented by triangulating the information from secondary sources.

There is no universally agreed or recommended framework to assess pooled funds in place.9 However, we adapted Coppin’s (2012) framework to assess HTF in Zimbabwe.7 8 We conclude that the tool will require fine tuning before wider use; as an example, none of the indicators for the principle of delivery of results provide an assessment of whether the pooled fund produced its intended results or not. The guidance to score each of the 30 proposed indicators and to produce summary scores need further refinement as well. In applying the scoring to the HTF, the authors agreed on an adapted approach to produce a score in those cases where the answer was not necessarily a yes or no (0; 1).

Furthermore, the framework proposed by Coppin has not been widely implemented with lack of available benchmarks to qualify the scoring against agreed criteria or golden standards. This limits the credibility and reliability of the framework, particularly due to the absence of clear guidance on how to measure each of the indicators. Each principle from the Coppin’s framework originates from the broad Paris Principles for Aid Effectiveness.2 3 9 Transforming this multidimensional Paris Principles into quantifiable indicators do not allow for a robust assessment. Similarly, Martinez-Alvarez et al 9 suggested qualitative assessment of some of the principles.9 Additionally, the ability of the indicators of the tool ‘as is’ to capture the dimensions that they are aimed to measure is limited. For instance, under the principle ‘Delivery for result’, there is no indicator in Coppin’s framework attempting to capture the capacity of the Fund to achieve results against initial targets or plans.

Implications for policy and practice

This paper provides a methodological contribution to development literature through (1) adapting one of the well-known aid effectiveness frameworks to assess a pooled fund mechanism and (2) assessing a pooled fund mechanism in a fragile context, Zimbabwe. At the end of the HTF, an extension of it, named the HDF, was designed and launched in Zimbabwe, following the same logic of the HTF and broadening its scope of work to more programmatic areas. This confirms some key conclusions regarding the HTF: first, that it was a win–win solution for both the government and donors, which therefore decided to replicate it; second, that the risk of investing directly in the government was still too high for donors at the end of the transition phase due to the uncertainty of the context and third, that the local systems and capacity were not yet at a stage where they could deliver some of the activities without the support of a fund manager.

Evaluating whether the model of the transition fund evolves during the development stage to addresses its weaknesses rather than doing more of the same will be central to conclude whether the modality adopted in various sectors in Zimbabwe via transition funds is eventually sustainable and effective.