Measuring effectiveness, efficiency and equity in an experimental Payments for Ecosystem Services trial.

There is currently a considerable effort to evaluate the performance of Payments for Ecosystem Services as an environmental management tool. The research presented here contributes to this work by using an experimental design to evaluate Payments for Ecosystem Services as a tool for supporting biodiversity conservation in the context of an African protected area. The trial employed a 'before and after' and 'with and without' design. We present the results of social and ecological surveys to investigate the impacts of the trial in terms of its effectiveness, efficiency and equity. We find the scheme to be effective at bringing about additional conservation outcomes. However, we also found that increased monitoring is similarly effective in the short term, at lower cost. The major difference - and arguably the significant contribution of the Payments for Ecosystem Services - was that it changed the motives for protecting the park and improved local perceptions both of the park and its authority. We discuss the implications of these results for conservation efficiency, arguing that efficiency should not be defined in terms of short-term cost-effectiveness, but also in terms of the sustainability of behavioral motives in the long term. This insight helps us to resolve the apparent trade-off between goals of equity and efficiency in Payments for Ecosystem Services.

Introduction

Early proponents of Payments for Ecosystem Services (PES) argued that they offered a potential improvement upon Integrated Conservation and Development Projects (ICDPs). ICDPs were considered to be ineffective due to overly ambitious combinations of environmental and social goals, and a lack of causal linkage between delivery of social benefits and desired conservation outcomes (Ferraro and Simpson, 2005, Ferraro, 2001, Ferraro and Kiss, 2002). Whilst recent studies identify some positive outcomes from ICDPs in some cases (Blomley et al., 2010, Morgan-Brown et al., 2010), advocates of PES propose that making benefits directly contingent on provision of outcomes would, in some circumstances, be more effective and that enabling competition among possible service providers would also lead to efficiency gains. There are still very few rigorous empirical studies that test this proposition (Miteva et al., 2012) and this paper contributes to this knowledge gap.

Such ‘direct payments for conservation’ were well known in Europe, where paying farmers for conservation-oriented practices was already an important part of the Common Agricultural Policy's environment pillar (Gomez-Baggethun et al., 2010). There were some concerns about the equity of these schemes, as farmers with the largest land holdings were able to provide the most services and received the greater payments (Wilson, 1997), but this was offset by the political expedience of finding an eligible way to package farm subsidies. However, concerns about the distributional outcomes of PES schemes have become more salient because PES is now more widespread in developing countries, in contexts where poverty alleviation is often a desired co-benefit of conservation interventions (Pagiola et al., 2005). Thus the question of whether PES is more effective in achieving desired outcomes and more cost-efficient than its alternatives must be considered alongside questions about its impacts on poverty and, more generally, equity. Put simply, will efficient modes of providing environmental goods bypass the poorest and be considered inequitable? We think this question will become increasingly important because (a) those living near to forests and terrestrial protected areas in the tropics are likely to be poorer than their compatriots (Ferraro et al., 2011, Sunderlin et al., 2008), (b) these locations where poverty and biodiversity coincide will be increasingly targeted by PES schemes such as REDD+, for example due to increasing confidence in the evidence of synergies between biodiversity conservation and carbon storage (Parrotta et al., 2012), and (c) because the poor sell cheap due to lower opportunity costs and therefore PES is more likely to target their lands (Kosoy and Corbera, 2010). In this paper, we assess the effectiveness, efficiency and equity of a PES scheme in Rwanda. As the argument proceeds, we ask whether equity, rather than efficiency, might in fact be the greatest advantage arising from our PES and, following-on, whether efficiency and equity can really be considered to be independent outcomes.

Theory

We define effectiveness as the achievement of stated objectives additional to what would have been achieved in the absence of the PES intervention. We provisionally define efficiency in utilitarian, economic terms as maximizing total welfare (Adger et al., 2003); it typically implies a ‘value for money’ characteristic whereby human wellbeing outcomes are achieved at least cost so that we can afford more of them.

Equity is not so amenable to a universal definition because it is typically understood through reference to both objective and contextual assessments (Miller, 2013). In pursuit of a more objective and universal benchmark of equity, conservation practitioners have tended to consider ‘equity’ to mean ‘pro-poor’ (Pascual et al., 2010). This more often refers to impacts on the distribution of material costs and benefits, but can also refer to procedural inclusion, ways of interacting, and impacts on authority and control. One way of operationalizing ‘pro-poor’ is to think about the impact of an intervention on the difference between rich and poor – does it increase or decrease the gap? By this reading, a ‘weak’ rendering of ‘pro-poor’ is that environmental management interventions should ‘do no harm’ to the poor (Barrett et al., 2011). A slightly stronger, Rawlsian egalitarian reading, is that any deviation from equal treatment should be positively in favor of the poor and, most particularly, the poorest (Rawls, 1971). In practice, this tends to be interpreted as a call for positive discrimination, with a ‘good’ outcome being a narrowing of the gap between rich and poor (Brock, 2009).

A second way of operationalizing ‘pro-poor’ that is common in the conservation literature is to associate it with a commitment to uphold human rights (Greiber et al., 2009, Kashwan, 2013). For a rights-based approach, pro-poor outcomes can be assessed from the perspective of thresholds. For human capabilities and rights thinkers, equity outcomes are at least partly judged by whether states and other agents are striving to secure minimum material and social thresholds – supporting those capabilities considered minimally necessary to live a valued and dignified life (Brock, 2009, Doyal and Gough, 1991, Gough, 2004, Nussbaum, 2011).

Given that our focal country, Rwanda, is a low income country with high levels of rural poverty, we consider it appropriate to consider a pro-poor conception of equity, especially given previous evidence that the poor have often found it difficult to participate in PES schemes due to high transaction costs or structural barriers (Corbera et al., 2007a, Corbera et al., 2007b, Engel et al., 2008). However, it is important to recognize that what constitutes an equitable distribution can vary from situation to situation as well as from person to person: for example an ‘equal share’ might be considered the equitable way to distribute votes, whereas ‘need’ might be seen as a more equitable basis for allocation of aid, and merit the best way for allocating employment (Deutsch, 2000, Martin et al., 2014). Furthermore, different individuals and groups, perhaps with affinities to different political philosophies, may view each situation differently, giving rise to plural, rational ideas about the right thing to do (Sen, 2009). Moral valuation is therefore contextual, determined by individual and social characteristics across time and space, and not reducible to singular notions of the ‘good’, to simple aggregation, or to individualistic analysis (Sandel, 1998). Thus we reject the usefulness of looking for a singular, universal definition of the good, such as that presented in utilitarian political philosophies (Nussbaum, 2011, Sen, 2009). Our interest in this paper is the place-bound conceptions of equity that are used by different groups to construct claims about the fairness of conservation interventions and that we ultimately consider to be related to conservation outcomes.

It is important to note that the acceptance of a pro-poor conception of equity, as well as an inclination toward a pluralist conceptions of the ends of justice, puts us in opposition to utilitarianism (Rawls, 1971, Sen, 2009). Whilst utilitarianism is equitable toward the poor in the sense that enhancement to all people's utility is equally valued, an ultimate concern with aggregate utility is problematic. Firstly, to define the ends of justice as the maximization of aggregate utility (the greater good) is potentially consistent with allowing the poor to become poorer or for minimum capability thresholds to be transgressed. Secondly, the use of a single metric (utility) assumes that diverse values are ultimately commensurate and suited to aggregation. We flag this point at the outset because it creates a tension between our conceptions of efficiency and equity. Efficiency is concerned with total welfare irrespective of its distribution, and so can be at odds with equity goals. For example, through our efficiency lens we might not be concerned if the poor sell their environmental services ‘cheap’, whereas through our equity lens we might be.

The apparent trade-off between common working definitions of efficiency (utilitarian, aggregate welfare gains) and equity (egalitarian) cuts to the heart of many environmental dilemmas, presenting a tension between ‘pursuit of the greatest happiness for the greatest number, and the assertion of individual, local or ethnic rights that ought not to be violated, even at the expense of the aggregate good’ (Rayner and Malone, 2000). Such tension has recently come to the fore in academic debate about PES (Corbera, 2012, Gross-Camp et al., 2012a, Kinzig et al., 2011, Pascual et al., 2010). Whilst nobody denies that equity is important, there is disagreement as to whether it should be a first order objective of PES. Whilst there is a spectrum of thinking, we think it is reasonable to say that the dominant view of PES holds that, in situations where PES is appropriate, utilitarian efficiency is its principle advantage over alternatives, and should be the overriding objective for the PES instrument itself (Ferraro and Simpson, 2005, Kinzig et al., 2011, Wunder, 2007). This paper questions this view, asking whether equity might in fact be so deeply linked to efficiency that the latter cannot be prioritized or, alternatively, that equity rather than efficiency might sometimes prove to be the primary advantage of PES over some alternative conservation practices such as a focus on rule enforcement alone.

The prioritization of effectiveness and efficiency poses some problems for PES in practice. Firstly, there is the problem of measuring the additional good provided by the intervention due to scientific uncertainty about the link between land management practices, ecosystem functions and service provision (Fisher et al., 2009). Where this problem can be addressed, it will normally require greatly increasing the transaction costs of monitoring. Secondly, there is a problem of legitimacy, because not everyone agrees that efficiency concerns should prevail over equity ones in PES design (Pascual et al., 2010) and because humankind's deontological attachment to fairness is considered to be a widely distributed human characteristic (Brock, 2009, Rawls, 1971, Sen, 2009) that might even be rooted in our neural processing (Hsu et al., 2008). The fact that humans hold equity so dear may help to explain why environmental conflicts often arise from competing visions of fairness (Harvey, 1996, Redpath et al., 2013, Schlosberg, 2004, Whiteman, 2009) and indeed why ‘real’ PES schemes cannot avoid meeting legitimacy requirements and therefore seldom resemble their imaginary, ‘efficient’ form (Milne and Adams, 2012, Muradian et al., 2010). We contribute to this debate by asking whether our experimental PES scheme was effective, efficient and equitable, and by considering the relationship between these values.

Study site and PES features

The Nyungwe National Park (1013 km2) is part of the biologically rich Albertine Rift (Plumptre et al., 2002) and one of the largest remaining blocks of high altitude rain forest on the continent. The park was initially gazetted as a forest reserve in the early 1930s but had little rule enforcement up until the late 1980s when the management authority, Rwanda Development Board (RDB), was first mandated to oversee activities in the reserve. Although clearing of the forest for agriculture was prohibited, local rights to collect wood were recognized and upheld in the early 1970s with the initiation of a buffer zone of fast-growing exotic trees (Weber, 1989). The reserve transitioned to national park in 2004 and is now a strictly protected area with no human inhabitants or permitted use. There are a total of 52 cells (an administrative unit consisting of a cluster of 3–6 villages, and population of 1500–6000 people) abutting the park. Our project was based in a total of eight of these cells – four experimental and four controls (Fig. 1).

Fig. 1

Location of study sites.

Currently, there are only a small number of published PES evaluations that seek to rigorously establish causal impact through use of controls to construct counterfactuals (Pattanayak et al., 2010). Furthermore, these are all in Latin America and all involve PES schemes that incentivize management of household farm or forest plots (Miteva et al., 2012). The site for this study is therefore unusual, both for being in Africa, as well as for using PES for the management of public- rather than privately-managed lands. This is a critical site characteristic because it is well established that tenure, amongst other institutional factors, is important for understanding the outcomes of PES schemes (Tacconi et al., 2012).

There is considerable debate in the literature about what can rightfully be described as a PES, and what should be considered instead as ‘PES-like’ (Engel et al., 2008, Sommerville et al., 2009). The defining features of a PES scheme are widely cited as being (a) that it is voluntary, (b) that a defined environmental (?) service is bought by one party from another party, and (c) payment is conditional on provision of that defined service (Wunder, 2005). In our own PES trial, (a) communities enter into contracts voluntarily, (b) we (a research partnership backed by EU funding) pay communities to provide specified conservation services, and (c) payments are conditional based on measured performance of those services. Indeed, our PES was designed with these very criteria in mind and on the face of it, there is a clear match. But as with most schemes labeled as PES, there are gray areas. First, some might question whether participation is voluntary for each individual, given that entry is a collective decision. Second, the service we value is biodiversity conservation but what we actually pay for and monitor are proxy measures that we think will lead to this (reduced tree cutting, hunting and mining, and increased tree planting). Third, some might argue that some of these things are covered by rules of the park and a PES should not pay for rule abeyance. Fourth, one might also argue that PES transactions should involve market exchange, or purchase by private sector service buyers – although such criteria would make ‘real’ PES vanishingly rare (Milne and Adams, 2012). We consider the key criteria to be direct linkage, conditionality and the expectation of additionality. Regarding linkage, what we pay for can reasonably be expected to have a direct impact on the target service – biodiversity. Regarding conditionality (?), our monitoring of performance involved systematic monitoring of levels of provision, with payments scaled accordingly. Concerning additionality, the use of controls was designed to test whether any measured effects were above and beyond those brought about by pre-existing rules and enforcement. On balance we think it helpful to describe this as a PES.

We provided financial incentives to communities to reduce activities that were widely considered to be detrimental to biodiversity conservation, predominantly hunting with snares, cutting trees, and mining. At the same time, we incentivized those activities outside of the park that were considered to support reduction of these in-park activities in the long-term, mainly private tree and bamboo planting. Performance was measured once a year and based upon the collective actions of the community. Specific performance indicators and targets were negotiated through community meetings in each of the four participating cells. Performance targets were also informed by, and measured against, baseline assessments. Methods of payment were partly negotiated with communities. In particular, each participating cell decided the proportion of total payment to be retained for communal activities and the proportion to be distributed at the household level (Gross-Camp et al., 2012a, Gross-Camp et al., 2012b). All payments were cash rather than in-kind, with communities on average deciding to distribute 68% of total payments in equal shares to each household, and to retain the remainder for investment in community activities. Both parts were discussed at public meetings and were both presented as part of the PES payment.

Communities had complete discretion over how to spend the collective part of the payment, spending it on, for example, provision of tree saplings and purchase of goats for poorer families. The level of payment was up to 25 USD per household annually, determined to be the average opportunity cost for reduced park access (using Masozera and Alavalpati, 2004). It was determined through consultation with park authorities and community representatives that the actual payment (conditionally awarded and scaled according to performance monitoring) would be paid as a flat, undifferentiated rate to each household, regardless of household-level opportunity cost. Elsewhere, we present evidence that this equal distribution was the most widely held to be equitable at local level, whilst distribution by opportunity cost was the least popular conception of equity (Martin et al., 2014). As has been argued in Section 2, conceptions of what is fair and equitable are determined in context and this is a potential challenge for any pre-determined view that the study of opportunity cost can help establish payment levels that are fair and efficient. We should also note that, whilst it would have been possible to use our baseline household surveys to differentiate household opportunity costs, doing this rigorously would have added considerably to project transaction costs. Furthermore, measuring performance at household level is impractical due to the public nature of the good – a National Park. Thus, there was in practice something of a pragmatic alignment between what communities wanted, and what was possible. In terms of guiding objectives, there was also pragmatic alignment between effectiveness and equity goals as the design of the PES were adapted to local norms. For example, for the PES to be successfully launched it was all but essential to align with local stakeholders’ views about what constituted fair distribution of payments.

The counterfactual was established through (1) control communities with no PES but with similar additional monitoring effort to the experimental units (e.g. similar surveying of human activity along transects in the forest) and (2) the rest of the park which had no PES and no additional monitoring. The counterfactual that we constructed would clearly not satisfy the most stringent standards for Randomized Controlled Trials (RCTs). Perhaps most importantly, it was not considered practical to employ double- or even single-blinding into the design, and thus members of both experimental and control cells were potentially aware of their roles. As has been noted, the challenge of measuring the effectiveness of PES typically requires a significant and costly monitoring effort. In this case, monitoring could not be conducted remotely and required on-the-ground observation of forest-based activities through regular visits by monitoring teams.

As a research-oriented trial, the short duration of the project meant that payments only lasted for two years, which is a shorter commitment period than most PES schemes. We consulted at length with local authorities how best to handle this in order to avoid any potential negative impacts when payments finished. The resultant approach was one of open communication – to stress in all of our dealings with communities and other stakeholders that this was a temporary trial. For communities, this was hardly unusual, in the sense that it conforms to their experience of project interventions.

Methods

The basic design of the study was (a) ‘with and without’, through selection and monitoring of both experimental and control locations and (b) ‘before and after’ through baseline social and ecological surveys in 2009 and subsequent surveys to monitor change. We established an experimental PES in four selected cells adjacent to the Nyungwe National Park. Three cells were randomly selected with the fourth (Uwumusebeya) being intentionally included based on the expressed interest of our project partners and park authority. Uwumusebeya is adjacent to the largest bamboo ecosystem in the park (3174 ha) which is home to a rare and endemic primate, the owl-faced monkey (Cercopithecus hamlynii). The bamboo has multiple uses, including construction, household and agricultural instruments such as baskets, and making snares for trapping animals. The four control cells were located adjacent to experimental cells in an effort to improve matching and potentially capture problems of leakage of human activities from experimental cells. We considered the potential benefits of placing controls further from the experimental cells (i.e. to try to reduce contamination) but decided that this would add significantly to our monitoring costs and still not eliminate the likelihood that our presence (forest monitoring and household surveys) would influence people's behavior. Distant controls would equally have been less well matched.

Human activity monitoring by the PES team (ReDirect). Human activity in the park was measured by walking the existing trail system within a defined sub-section of the park abutting each cell and geo-referencing defined human activities as they were found. The existing trail system was geo-referenced in 2009 when baseline information was collected and followed during subsequent surveys. A total of 7 surveys were conducted in each of the 8 cells over the course of the study. The monitored area was defined by drawing perpendicular lines 2.5 km from the edge of a cell's boundary into the park using ArcGIS v. 9.3. This distance was chosen based on previous research that found human activity tends to occur within 1.5 km of protected area boundaries and sharply declines at a distance greater than 5 km (Waas, 1995). We examined whether human activities differed significantly by cell type (control versus experimental) through time using a generalized linear model on the mean number of human activities (snares and wood cutting) using R version 2.15.1 and the MASS 7.3-18 package. We utilized a log-link function and negative binomial model based on overdispersion of the data set in the Poisson GLM.

Human activity monitoring by the park authority. Ranger-based monitoring (RBM) data were collected by the park authority, Rwanda Development Board, our partners on this project. The RBM is collected by rangers that extensively patrol the park's existing trail system on an approximately daily basis, geo-referencing a broad number of observations of human impact. We utilized the RBM data overlapping with the period of our PES (2009–2012) to examine the park-wide human activities (i.e. limited to snares and wood cutting) as well as activities within areas where the PES was active. Patrols and their associated findings were isolated using ArcGIS v. 9.3 and the shapefiles of our monitored areas. Similar to the data collected by ReDirect, we utilized a negative binomial GLM to determine whether human activity differed significantly through time in areas where the PES was active (control and experimental cells) from that of park-wide trends. To test for trends over time, Mann–Kendall tests were run using the Kendall 2.2 package.

Livelihood survey data. We collected extensive livelihood information in all eight cells, experimental and control, in 2009 and 2012. Forty-eight households in each cell were randomly selected for interviews (N = 376, 8 households were lost in 2012 due to death and movement). The baseline interviews were conducted in September 2009 and the same households re-interviewed at the end of the study in early 2012. These interviews enabled us to calculate household consumption as a measurement of wealth and included the market value of items consumed from the park, food items, durable goods (e.g. bicycles, radios), social events, and educational expenses. Mean market values were established through a series of price surveys in markets close to the cells where interviews occurred coinciding with the livelihood interviews in 2009 and 2012. All analyses of the survey data were done in SPSS.

Results

Effectiveness

A generalized linear model (GLM) showed significantly fewer human activities in the experimental cells (Cell type (exp) −0.98) than control cells with a small downward trend for both cell types over the course of the study period (Month −0.04; Table 1). However, there was no significant interaction between cell type and month (data for more complex models with interaction terms not shown, model selection carried out via likelihood ratio tests) indicating that this downward trend was of a similar nature in both experimental and control cells (Fig. 2). In other words although the model suggests that experimental cells have a lower rate of human activity than controls, the rate of change through time is similar suggesting that something is influencing human activity in both cell types.

Table 1

A summary of the selected generalized linear model (via likelihood ratio tests for nested models) using a log-link function and negative binomial distribution to investigate change in human activity through time in control and experimental cells as collected by the PES team (ReDirect). Under this model, logs of expected monthly activity counts in experimental cells are expected to be almost one unit lower than those of control cells. (0 ‘***’, 0.001 ‘**’, and 0.01 ‘*’).

	GLM human activity (PES)Snare & tree cuttings
	Estimate	z value
Intercept	2.05	6.75***
Number of patrols	0.37	6.16***
Month	−0.04	−2.30*
Cell type (exp)	−0.98	−3.45***
Log-likelihood	−298.65
AIC	308.65
Sample size	20

Fig. 2

The mean number of human activities or threats per patrol through time in control and experimental cells using data collected by the PES team (N = 20 sampling months).

We generated alternative hypotheses to explain this finding. Our null hypothesis proposed that the PES intervention was ineffective insomuch as human activity was changing as a park-wide process and that the trends observed in the model above (Table 1) were merely a reflection of that process. We considered this specifically in relation to the service of biodiversity gains from reduced human activity within the park. To test this we utilized ranger-based monitoring (RBM) data collected by the park authority (RDB) to compare park-wide human activity with our experimental and control cells. The selected RBM data model (again chosen via likelihood ratio tests of nested models) indicated significant differences in the level of human activity by cell type (Table 2). Human activity in the experimental cells was significantly less than that of the control cells (Cell type (exp) −2.661), whereas the activity throughout the park was higher but not significantly so (Cell type (park-wide) 0.779). Furthermore, an examination of model terms including ‘Month’ showed that trends over time were different in all three cell types, with a slight increase in observed human activity throughout the park over time as well as in experimental cells, and a slight decrease in control cells (Month: Cell type interactions). However, the net effect in experimental cells (the estimate value for Cell type (exp) is the single greatest value of the model) suggests that the level of human activity in these areas is significantly lower from that in both the control and parkwide, i.e. something is strongly suppressing human activity in these cells.

Table 2

A summary of the selected generalized linear model (via likelihood ratio tests of nested models) using a log-link function and negative binomial distribution to investigate change in human activity in areas where the PES was active (control and experimental cells) as well as park-wide. These data were collected by the park authority (RDB) ranger-based monitoring program. (0 ‘***’, 0.001 ‘**’, and 0.01 ‘*’).

	GLM human activity (RBM)Snares & tree cuttings
	Estimate	z value
Intercept	1.884	3.359***
Number of patrols	0.020	0.583
Month	−0.070	−2.714**
Cell type (exp)	−2.661	−3.424***
Cell type (park-wide)	0.779	1.004
Number of patrols: Month	0.004	2.209*
Number of patrols: Cell type (exp)	0.228	3.952***
Number of patrols: Cell type (park-wide)	−0.009	−0.274
Month: Cell type (exp)	0.095	2.668**
Month: Cell type (park-wide)	0.078	2.457*
Number of patrols: Month: Cell type (exp)	−0.009	−3.108**
Number of patrols: Month: Cell type (park wide)	−0.004	−2.170*
Log-likelihood	−665.961
AIC	691.96
Sample size	36

Additional Mann–Kendall trend tests corroborated the finding that park-wide activities are increasing slightly, but lacked significance for a downward trend in control cells, presumably due in part to the weakness of the trend (park-wide: tau = 0.23, p = 0.05, control: tau = −0.15, p = 0.20; Fig. 3a and b). Experimental cells similarly lacked significance for a change in human activity (experimental: tau = −0.12, p = 0.32; Fig. 3c) yet activity appears to be decreasing after the first initial months of the project, suggesting that this change may be due in part to the PES intervention (Fig. 3d; an observation supported by a subsequent Mann–Kendall test, experimental truncated: tau = −0.30, p = 0.02). We therefore reject the null hypothesis that the trends in the first model (Table 1) are merely a reflection of park-wide trends and conclude that the PES intervention has had a significant impact on levels of human activity in experimental (and potentially neighboring control) cells, but with experimental cells showing the lowest activity levels of all.

Fig. 3

The mean number of human activities or threats per patrol over the course of the PES with a lowess smoothing line to depict the trend of activity (ranger-based monitoring data collected by the management authority, RDB).

Efficiency

The second alternative hypothesis concerning the lack of a significant difference between the rate of change in human activity levels through time between control and experimental cells (of the ReDirect data set Table 1) is due to contamination of controls by the additional monitoring effort – i.e. they are not true counterfactuals. The difficulty of providing adequate controls in conservation research is well known (Soule and Orians, 2001). One reason for this is that it is both impractical and/or unethical to apply a double-blinded or even a single-blinded experimental design. Resulting bias can include the Hawthorne effect, whereby participants change behavior in response to being monitored, rather than or in addition to responding to the intervention itself; and the John Henry effect, whereby controls change their behavior in response to knowing that they are being compared to an experimental group (Fuente and Whittington, 2012, Zwane et al., 2011). John Henry was a steel worker who worked himself to death in the 1870s having learned that his performance was being compared to that of a steam-powered drill (Irving and Holden, 2013). In our case there were at least two unavoidable sources of contamination. Firstly, it was not practical to create controls that we could be certain to be blind to their role. Indeed, it was necessary to consult with local leaders to ensure that they were comfortable with our activities. Secondly, and most pertinent to our hypothesis, becoming a control involved being monitored by teams on the ground, which may itself lead to behavioral change.

Regarding the possibility of a John Henry effect, we undertook a small survey in control cells and found that 23% (n = 17 of 73) of respondents could correctly answer three questions about the PES scheme, confirming that communication about the trial was indeed taking place and that the control was not fully blind. Furthermore, 44% of respondents from control cells stated that reduced illicit activities in the park were partly due to expectations of future inclusion in a PES scheme (Figure B1).

However, our research shows that the second type of contamination (the effect of our additional monitoring in the controls) was the most important. When asked to identify the single most important reason for reduced illicit activities in their cell, no respondent cited expected PES inclusion whilst 50% (n = 29 of 58) cited the presence of park guards (Figure B1). PES monitoring activities in the park occurred every four months and involved the presence of a team of 6 people that spent approximately one week in the area. Whilst the monitoring team are not in fact ‘park guards’ we learned that local people tended to associate them with the management authority (RDB). This is confirmed by findings that households in both experimental and control cells reported a significant increase in interaction with park authority staff between 2009 and 2012 (paired t-test: control t(175) = −11.51, p < 0.0001 and participating t(180) = −20.61, p < 0.0001; Figure B2).

These results suggest that the PES trial has not been cost-effective, because a similar outcome has been achieved in controls (in the short term at least), as a result of enhanced monitoring alone, at lower intervention cost. The cost of ecological and social monitoring was on average 3068 and 6793 USD per cell respectively, the same for participating and control cells. The cost of payments, for participating cells only, was 23,110 USD per cell per year. Lack of cost-effectiveness is likely here to reduce efficiency, in the sense that we assume the conservation outcome to bring welfare benefits and that a more cost-effective intervention could produce a greater total welfare gain. However, as our discussion of equity will reveal, there is yet a crucial distinction to be made between the impact of the project on experimental versus control cells.

Equity

Because conceptions of equity tend to be context specific we investigated whether an egalitarian perspective resonated with local norms. We tested a total of four alternative principles for payment distribution and found that respondents selected egalitarian (38%) as their first choice more often than expected by chance (N = 78, Chi-square = 14.4, df = 3, p = 0.002). The alternative principles of ‘effort’, ‘need’ and, ‘opportunity costs’ were ranked first by 29%, 23% and 9% of respondents, respectively. Similarly respondents’ least preferred method of payment distribution was opportunity costs, with 54% ranking this last (Chi-square = 35.3, df = 3, p = <0.001). Whilst this may be a favored system of distribution for utilitarian efficiency, it was reported to us to be morally perverse (and unpalatable to our RDB partners) because it is seen as rewarding the worst offenders.

The size of annual household PES payments was calculated to offset costs rather than enhance income. As expected, we found no significant change in household consumption (as an indication of wealth) across all cells as well as between cells, control and experimental (Wilcoxon signed-rank: z = −0.98, p = 0.32 and z = −0.83 and −0.48, p > 0.05, respectively) based on panel data collected in 2009 and 2012 (N = 357 households). We are more interested in the distribution of any income effects and how this is locally perceived. Comparing change of consumption per household, we find some evidence that the lowest income quartiles are doing relatively well in that their consumption growth is favorable compared to other quartiles (Fig. 4). Given a range of possible confounding factors, we cannot confidently attribute any such change to the project, but this at least supports the view that the project did no obvious harm to the material wellbeing of the poorest.

Fig. 4

Quartiles of household consumption with standard error bars in control and experimental cells (N = 176 and 181 for control and experimental cells, respectively; 1 USD = 600 RWF).

We further assessed equity by testing for elite capture of benefits – given that less than 100% of households actually received the payments they were due1 (Figure B3), and considerably less gained other benefits, was there evidence of discrimination against the poor? We examined the relationship between wealth and the likelihood of receiving (a) each of the three PES payments made over the course of the project, (b) employment as local monitors, (c) receipt of small livestock and/or tree saplings, and (d) involvement in anti-crop raiding programs. None of these relationships were significant suggesting that distribution conformed to an egalitarian conception of equity (Table A1 and Figure B4). The exception to this was the observation that a small number (n = 6 of 29) of households receiving free livestock were among the highest income quintile that contrasts with our stated objective to target the poor.

In addition to distribution of material benefits, we also considered the equity of social impacts. We were particularly interested in whether PES might lead to injustices arising from failures of recognition, including insensitivity to local culture and the undermining of non-monetary ways of valuing interactions with the environment. Qualitative interviews revealed hardly any concern with cultural loss or domination arising from the PES. The attachment of monetary value to particular practices was not articulated as dominating other ways of valuing livelihood practices or relationships with nature. We also found very little concern about the fact that the incentive structure of the PES led to a deepening of antagonism toward traditional forest uses such as hunting. We want to stress that these findings are highly context-specific and reflective of a particular Rwandan historical process of modernization that tends to construct notions of progress in contrast to traditional cultural practices associated with a violent past (Martin et al., 2014).

A related concern in critical PES literature is that extrinsic incentives introduced through PES might lead to a ‘crowding out’ of intrinsic motives for conservation (Corbera, 2012, Fisher, 2012, Sommerville et al., 2009). There is strong empirical evidence that material interests and moral sentiments are inseparable and that crowding-out (and in) occurs because economic incentives affect individuals’ moral framing of a situation (Bowles, 2008, Frey and Jegen, 2001), as can the institutional features of these incentives (Falk and Szech, 2013). Of particular relevance to equity, the psychological conditions that foster crowding-out include those where an intervention is perceived to be unfair, such as where it is domineering and undermines self-determination (Frey and Jegen, 2001) or where it allows free-riders to prosper (Bowles, 2008). Moreover, the correlate may also be true, that intrinsic motivations might be crowded-in where an intervention is perceived to support local control and self-determination (Frey and Jegen, 2001).

We provide here evidence that the PES did indeed change the motives underlying observed behavioral change. The key finding is that whilst the project as a whole led to similar conservation outcomes in both experimental and control cells, there was an important difference in how people perceived their behavioral change leading to such outcomes. To link this difference to the theory of crowding-in and -out is admittedly rather more speculative at this moment, but we think promising enough to warrant some discussion. Respondents in experimental cells increasingly agreed that participation in decision-making was the main reason for reducing illegal activities in the park (Table 3). In contrast control cells have become less likely to attribute the reduction in illegal activities to participation and environmental education and more likely to attribute it to law enforcement.

Table 3

Mean values (SE) are based on a Likert scale where 1 equals strongly agree and 5 strongly disagree. (0 ‘***’, 0.001 ‘**’, and 0.01 ‘*’).

	2009		2012		Repeated measures
The main factor that stops people from entering the park is:	Experimental	Control	Experimental	Control	Between years	Between years × cell type
	Mean (SE)		Mean (SE)
a. Law enforcement (RDB)	2.23 (0.08)	2.13 (0.09)	2.14 (0.06)	1.68 (0.06)	F1,332 = 5.52*	F1,332 = 15.31***
b. Involvement in park management (participation)	2.28 (0.08)	2.31 (0.09)	2.16 (0.06)	2.60 (0.09)	F1,343 = 0.67	F1,343 = 4.84*
c. Environmental education that they receive	1.82 (0.05)	2.31 (0.09)	1.94 (0.06)	2.55 (0.10)	F1,342 = 0.39	F1,342 = 10.7***

We used qualitative data to further explore how the PES changed motives. We coded 357 responses to an open question about why human activity in the park had reduced (Table A2). Three answers dominated, with 65% (n = 481 of 744) of coded responses stating (i) enforcement by park management, (ii) education about the park's ecological importance and rules, or (iii) the PES scheme. We then selected out only those responses that were premised by words to the effect that ‘this is the main reason for such behavioral change’. This yielded 86 observations, where 66% (n = 29 of 44) of control respondents identified enforcement as the main reason, compared to 21% (n = 9 of 42) of participant respondents (Table A3). Fifty-five percent of the respondents from experimental cells cited the PES scheme as the main cause of change with their statements suggesting that the non-pecuniary aspects of the PES (participation, education) are just as important as the money. What we take with confidence from these findings is that the PES changed local perceptions of the motivations for park protection, away from fear of rule enforcement and toward the PES scheme itself, which is associated not only with monetary benefits but also increased participation (Table 3, question b).

These differences in perception are hardly trivial in terms of the experience of local people and the implications for longer-term conservation practice. In control cells, respondents are very clear that they are changing behavior in response to fears that park staff will fine and beat them, and even shoot them. Examples of statements made during interviews give a sense of how serious this difference is:

‘The core reason why human activities have decreased in our cell is the reinforcement of the law along with heavy punishments. Three people have been killed when caught in the park.’ (Rukore cell)

‘Illegal activities in the park have reduced because ORTPN [RDB] increased its effort in protecting the park and sensitising the population. Also, there have been people who were shot and died, and this made people fear.’ (Kiyabo cell)

We have found the contrast with statements made in participating cells to be notable, for example:

‘The illegal activities in the park reduced because we were sensitised and given some money from ReDirect which we used to sustain our households.’ (Shaba cell)

‘The community was sensitised by ReDirect on the conservation of Nyungwe and given payments, which many people bought rabbits with—one male and one female. Instead of going into the park for meat, they could eat their own rabbits.’ (Gahurizo cell)

More generally, our surveys found that those in cells with PES are more likely to perceive benefits of living adjacent to the park, to say that the park authorities are sensitive to their needs and opinions, and that they benefit from the presence of park authorities. We also see significantly more tree-planting in PES cells, as a longer term way to find substitutes for park-based resources.

In terms of crowding in and out, what we think we are seeing is that the PES introduces new motives for caring for the forest (based on economic benefits and opportunities for better relationships with authorities), whilst not crowding out older, intrinsic motives. Thinking about this from the perspective of motivation crowding theory (Frey and Jegen, 2001), we propose that the taking on of new, positive motives is aided by perceptions of the equitable design of this PES scheme (participatory, egalitarian), i.e. that perceptions of equity, such as perceived progress toward self-determination, have served as antecedents of developing new motives. The absence of crowding out is largely concluded because respondents have never highlighted to us prior intrinsic motivations for park protection. We have explored this absence through focus groups and it seems that such motivations were crowded out much earlier through progressive fear of authority since Nyungwe was first gazetted in the 1930s (Martin et al., 2014).

Discussion

We have found that it is possible to design an effective PES scheme to support protection of a park. However, whilst the PES was effective at reducing threats to the park's biodiversity, compared to changes in the park as a whole, it was not more effective than our controls. Because controls were not blinded, they in fact turned out not to be true controls but an alternative treatment based on additional monitoring activity in the absence of financial incentives. We found that this alternative treatment had as much of an impact on our measures of human activity in the park as did the PES. The equity dimension was evaluated in relation to pro-poor distribution of material benefits, finding that poorer groups were as likely to benefit as the wealthier ones, and little evidence of the distributional inequity observed in some other PES schemes (Corbera, 2012, Mahanty et al., 2013), though not all (Li et al., 2011). Equity was also evaluated in social terms, considering whether the PES is considered to dominate and exclude non-economic values and, connected to this, whether it crowds-out non-economic motivation for park protection. We found that crowding-out is a moot point in this part of Rwanda, because the erosion of culturally particular ways of valuing nature was already far advanced. We did however find some evidence to propose that crowding-in might be a more relevant phenomenon, with some evidence that perceptions of procedural equity in particular might help to foster the psychological conditions conducive to the development of non-fear based conservation motives. This suggested role for perceived procedural equity broadly concurs with what is already known about the contexts that foster crowding-in over crowding-out (d’Adda, 2011, Frey and Jegen, 2001).

If one removes equity as a high order objective, the PES was not efficient (in the short term) because the environmental objective could have been achieved at lower economic cost, bringing greater aggregate expected welfare gains to current and future people. However, we think that equity might actually have been the most important impact of this particular PES. There is a normative reason for thinking this – quite simply, we think that projects that are perceived as equitable are preferable to those that are not. But there is also an instrumental reason for thinking this – that inequity can undermine efficiency in the long run due to conflict (Collier and Hoeffler, 2004, Esteban and Ray, 2011, Ibarra et al., 2011, Redpath et al., 2013, Robbins et al., 2009, To et al., 2012), and more positively, that equity might serve to ‘crowd-in’ more sustainable motives for conservation, such as the positive benefits that can flow from a forest, that will serve its efficiency in the long term. The latter remains quite speculative and requires further research effort.

In trying to evaluate the evidence, the key fact before us is that both experimental and control cells achieved broadly the same quantitative conservation outcome but for qualitatively different reasons. In control cells behavioral change was motivated by fear of park guards but in experimental cells there was a greater shift toward additional motives based on enhanced education about the environment, and a greater sense of involvement in park management which is tied in with perceptions of equity. This poses a challenge to our working definition of efficiency because, if we stick with this definition, our control cells (a ‘more guards’ intervention) appear to be the more efficient. And yet, motives and feelings about management legitimacy are likely to determine how well behavioral change is sustained, and more generally, the level of cooperation with park authorities.

We are implying here that inequity might undermine efficiency in the longer run, and thus that the two are fundamentally linked. One mechanism suggested for this is that perceptions of inequity undermine cooperative behavior and foster conflictive behavior (Deutsch, 2000, Martin et al., 2011). Whilst competitive relations can foster efficiency in the context of markets, PES schemes rarely resemble markets (Milne and Adams, 2012). Most PES schemes, including our case study, are more about public goods provision, and thus require cooperation. To prioritize competition over cooperation would be a myopic vision of efficiency (Jennings, 2005, Marglin, 2010). By contrast, if we look at efficiency with a longer time horizon, we are able to see that non-cost factors such as equity effect longer-term outcomes (Adhikari and Boag, 2013, Fehr and Schmidt, 1999, Wilkinson and Pickett, 2010), and thus should be constitutive of our vision of both effectiveness and efficiency.

In summary, we are offering preliminary theory and evidence to propose that observed efficiency/equity trade-offs might be resolved by expanding the time horizon and disciplinary scope of our evaluative framework. We cannot be certain that inequitable PES design will lead to conflict, or even the extent to which any such conflict would inflate transaction costs. This is in part because conflicts can occur early on and these schemes simply never get off the ground, making it hard to study cases of failure (Corbera, 2012, Wunder et al., 2008). But it is also because conflict can take time to become apparent and there is growing case study evidence from some of the longer running PES schemes that perceived inequities in procedure or distribution are linked to conflicts (Corbera et al., 2007b, Corbera et al., 2009, Garcia-Amado et al., 2011, Locatelli et al., 2008).

As a final word, we should reiterate that what passes for equitable cannot be universally prescribed. For example, in this particular context of access to a public good, we found that opportunity cost was not considered an equitable basis for determining payment distribution. But in other contexts, notably use of private farmlands, it may well be. The point for policymakers then, is that it is important to understand equity-in-context.