Producing valuable information from hydrologic models of nature-based solutions for water.

Nature-based solutions (NBS) are an increasingly popular approach to water resources management, with a growing number of projects designed to take advantage of landscape effects on water flow. As NBS for water are developed, producing hydrologic information to inform decisions often requires substantial investment in data acquisition and modeling; for this effort to be worthwhile, the information generated must be useful and used. We apply an evaluation framework of salience (type of information), credibility (quality of information), and legitimacy (trustworthiness of information) to assess how hydrologic modeling outputs have been used in NBS projects by three types of decision makers: advocates, implementers, and analysts. Our findings, based on documents and interviews with watershed management programs in South America currently implementing NBS, consider how hydrologic modeling supports two types of decisions for NBS projects: quantifying the hydrologic impact of potential and existing NBS and prioritizing where NBS might be sited within a watershed. To help inform future modeling studies, we identify several problematic assumptions that analysts may make about the credibility of modeled outputs for NBS when advocates and implementers are not effectively engaged. We find that salient, credible, and legitimate results in applications evaluating NBS for water are not always generated in the absence of clear communication and engagement. Integr Environ Assess Manag 2022;18:135-147.

INTRODUCTION

Interest in nature‐based solutions (NBS) to help ensure safe, sufficient, and consistent water supplies has begun to enter the mainstream (WWAP & UN Water, 2018), and investments in watershed protection and management are growing (Bennett & Ruef, 2016; Salzman et al., 2018). NBS are defined by the International Union for the Conservation of Nature (IUCN) as “actions to protect, sustainably manage and restore natural and modified ecosystems in ways that address societal challenges effectively and adaptively, to provide both human well‐being and biodiversity benefits” (IUCN, 2020, p. 1). Most NBS for water aim to improve water supply through a combination of conservation, restoration, and management of existing natural and seminatural landscapes, including forests, floodplains, grasslands, wetlands, and certain types of agriculture, which, as a group, are sometimes referred to as natural infrastructure; interventions are often sited close to waterways or in critical source areas (Brauman et al., 2007; Cohen‐Shacham et al., 2016; UNEP‐DHI et al., 2014). NBS also include land contouring, infiltration ditches, and a variety of other management approaches, many of which date back centuries (Ochoa‐Tocachi et al., 2019). As reflected in the IUCN definition, NBS for water have a strong focus on cobenefits, particularly biodiversity conservation, carbon sequestration, and diversified livelihoods (Bennett & Ruef, 2016; Pascual et al., 2014).

NBS can be implemented through a variety of mechanisms, including regulation and direct investment by governments. One major mechanism through which NBS are currently implemented is voluntary transaction programs, sometimes called Investments in Watershed Services, Payments for Watershed Services, Payments for Ecosystem Services, or Water Funds (subsequently “watershed investment programs”). These programs take a variety of institutional forms but are generally organized around compensation to private or communal landholders in a watershed in exchange for actions that improve downstream water supplies (Brauman et al., 2019). The voluntary nature of watershed investment programs has the potential to make this type of management scheme more appealing to participants, thus theoretically making them more feasible, effective, and equitable than unenforced legal requirements or outright purchase and management of watershed lands (Goldman‐Benner et al., 2012; Salzman et al., 2018). In South America, for example, the popularity of voluntary transaction programs has grown from the recognition that watershed management needs to support diverse communities and account for local livelihoods (Bremer, Auerbach, et al., 2016).

As existing NBS programs have matured and additional programs have been launched, calls to evaluate and quantify their biophysical impacts have become more common (e.g., Naeem et al., 2015). Efforts to evaluate social outcomes are also increasing (Blundo‐Canto et al., 2018; Bremer et al., 2018). Framing watershed protection as NBS or as natural infrastructure interventions has affected how these projects are conceptualized as well as the type and quality of data that certain implementing institutions and funders demand (Nelson et al., 2020). This has led to post hoc studies of effectiveness, including return‐on‐investment analyses (e.g., Kroeger et al., 2019), and increased emphasis on quantitative evaluation during project planning (Bremer, Vogl, et al., 2016). In the context of NBS for water, this has translated to increased interest in hydrologic modeling. However, the desire for sophisticated hydrologic models of NBS has collided with limitations in understanding ecohydrological processes, data scarcity, and the functionality of landscape hydrologic models (Hamel, Riveros‐Iregui, et al., 2017). Hydrologic science and associated models were largely developed in the temperate regions of the United States and Europe, limiting transferability to tropical and mountain regions where NBS projects are increasingly popular (Hamel, Riveros‐Iregui, et al., 2017; Ponette‐González et al., 2014).

To better understand the modeling approaches and types of information that could more successfully inform the development of NBS, as well as other watershed management projects, we analyze the way information generated by existing models is currently used in decision making using the concepts of salience, credibility, and legitimacy. We define these concepts in the conceptual framework section and then identify target audiences among decision makers, presenting our findings of how those audiences use information with the aim of helping analysts rapidly assess their own audiences. We use the concepts of salience, credibility, and legitimacy to address the way hydrologic information is used by these audiences in two key decision‐making contexts identified by the authors in previous research (Bremer et al., 2020): (1) quantification of the hydrologic impact of NBS and (2) decisions about where NBS should be implemented. Building on work evaluating the use of models and measurements in watershed investment programs in Brazil (Bremer et al., 2020), this analysis expands to Andean programs and derives streamlined lessons on how hydrologic information is used by different audiences and might be usefully developed further.

CONCEPTUAL FRAMEWORK

We frame our assessment of the information generated by hydrologic models using the concepts of salience, credibility, and legitimacy (Cash et al., 2003), illustrated in Figure 1. Introduced by Cash et al. (2003), these concepts were proposed to assess the effectiveness of scientific information for decision making in sustainable development projects generally. In the context of hydrologic modeling for NBS, salience, credibility, and legitimacy delimit attributes useful for evaluating whether a hydrologic model and the information it generates are likely to influence decisions and, therefore, whether investing in generating hydrologic information is warranted.

Figure 1

Salience, credibility, and legitimacy describe three aspects of useful information. The definition of each from Cash et al. (2003) is provided, as well as a pertinent question relevant to hydrologic modeling

Salience describes the type of information generated. In the context of NBS for water, salience characterizes how relevant the information generated by a hydrologic model is for the decision at hand. Accurate but irrelevant information generated through hydrologic modeling will not be used in decision making, so addressing appropriate questions and selecting a model equipped to answer those questions are critical (Bremer et al., 2020).

Credibility describes the quality of the information generated and is likely the concept that modelers are most familiar with (Cash et al., 2003). Credibility can be thought of as an assessment of how well a model matches observations, the theoretical basis of the model, the quality of input data, and the adequacy of the treatment of uncertainty (Hamilton et al., 2019). Increased credibility indicates the improved quality of information, but an increase in credibility may not affect a decision, so an appropriate level of credibility must be defined by the users of information, given the decision at hand.

Legitimacy describes the way users perceive the trustworthiness of information. Although this is, in part, a byproduct of the salience and credibility of the hydrologic model, it is strongly influenced by the process by which decision makers engage in the practice of modeling and assessment (Hamel & Bryant, 2017; Hamilton et al., 2019; Posner et al., 2016). The process of generating agreement and understanding about the questions asked and the assumptions made in modeling are critical components of increased legitimacy.

METHODS

Study sites

We focus on three Brazilian and three Andean (from Colombia, Ecuador, and Perú) watershed investment programs (Figure 2 and Table 1). The Brazilian programs referred to as Water Producer Projects, use direct payments to landowners in compensation for reforestation and protection of Atlantic forest in areas prioritized by Brazil's Forest Code. In the Andes, the programs, known as Water Funds, are financial and governance mechanisms funded by a mix of private and public sources and use a range of in‐kind compensation rather than direct payments; they focus on the protection and restoration of highland Andean forest and grasslands (páramo and puna) as well as on sustainable agricultural and ranching projects (e.g., agroforestry and silvopasture) (Brauman et al., 2019; Bremer, Auerbach, et al., 2016; Joslin, 2020). We selected watershed investment programs for interviews based on their focus on and advancement in hydrologic monitoring, as evidenced by their participation in a hydrologic monitoring collaboration through the Natural Capital Project, The Nature Conservancy, and the Latin American Water Funds Partnership (Bremer, Vogl, et al., 2016).

Figure 2

Locations of watershed investment programs analyzed in this study

Table 1

Watershed investment programs analyzed in this study

Program	Location	Downstream governance board interviews	Upstream interviews/focus groups
Conservador das Águas	Extrema, Brazil (Sao Paulo)	5	4 focus groups; 3 interviews
Produtores de Água e Floresta	Guandu, Brazil (Rio de Janeiro)	7	4 focus groups; 3 interviews
Projeto Produtor de Água da Bacia do Rio Camboriú	Camboriú, Brazil	8	3 focus groups; 3 interviews
Fondo Agua por La Vida y La Sostenibilidad	Cauca Valley, Colombia	11	13 interviews; focus group with association leaders
Fondo para la protección del Agua (FONAG)	Quito, Ecuador	3	Previous interviews (Leisher et al., 2012)
AquaFondo (Fondo de Agua para Lima y Callao)	Lima, Peru	2	Previous social impact assessment (Bremer, Gammie, et al., 2016)
Chancay–Lambayeque watershed assessment	Chancay–Lambayeque watershed, Peru	Key documents produced by analysts	1 focus group

Data collection

To understand how hydrologic information is currently used and would ideally be used in decision making for NBS projects, we use data from interviews and published documents. We conducted semistructured interviews (interviews with a mix of open‐ and close‐ended guiding questions; Creswell & Creswell, 2017) with board members of the project management units of the three Brazilian and three Andean watershed investment programs (Table 1; see Bremer et al., 2020) for a description of interviews in Brazil). In Brazil and Colombia, we also use data from focus groups (facilitated group discussions) and interviews with program participants (Meza Prado, 2018; Nelson et al., 2020; Wilburn et al., 2017). Individual interviews focus on the perspective of one person and help to ensure that their voice is heard, whereas focus groups provide opportunities for deliberation of ideas among group members; both methods are regularly used and combined in social–ecological research (Biggs et al., 2021). In total, we use data from 36 semistructured interviews with key informants who were either project managers or on the boards that govern the programs, primarily composed of public agencies, civil society, and downstream water users; 12 focus groups with upstream participants; and 22 semistructured interviews with upstream participants (Table 1). Semistructured interviews with project managers and board members (downstream) addressed a range of topics including program objectives, goals and activities, and social and hydrologic monitoring. Interviews included a specific section addressing the types of hydrologic information, modeling, and tools currently being used and the types of information, modeling, and tools desired for decision making (see Bremer et al., 2020). Focus groups and semistructured interviews with program participants (upstream) included goals for participation, perceived benefits of participation, and ideas for greater involvement of participants in program planning and design, including during hydrologic modeling and monitoring efforts (Meza Prado, 2018; Wilburn et al., 2017).

In addition to these interviews, our analysis is informed by a study of the potential for NBS for water in the Chancay–Lambayeque watershed in northern Peru and key documents produced in that analysis. Reflecting concern about future water supply, in 2019, the Chancay–Lambayeque watershed council collaborated with the National Water Authority of Peru, the World Bank, and the Natural Infrastructure for Water Security project to evaluate a range of investments to improve water security. Modeling and analysis were carried out by a consortium of Deltares and FutureWater (Taner et al., 2019). RTI International set up and ran the watershed hydrology model HydroBID to account for water inputs to the watershed distribution system (Corrales et al., 2019). Our analysis is also informed by documents produced in conjunction with sites in Brazil, Ecuador, and Perú (Bremer, Gammie, et al., 2016; Hamel, Bremer, et al., 2020; Kroeger et al., 2019; Leisher et al., 2012; Ochoa‐Tocachi et al., 2020; Ozment et al., 2018).

Analysis

We used deductive coding (Creswell & Creswell, 2017), in which interviews with board members as well as program‐related documents were coded into categories based on research questions focused on the types of decision makers and their information needs. These codes were based on research by Cash et al. (2003) on salience, credibility, and legitimacy and on research by Bremer et al. (2020), in which we classified decision makers in the Brazilian projects, identifying five different decision contexts and, within those, individuals in up to seven distinct roles: champion, funder, program manager, planner, regulator, land manager, and researcher. For this analysis, we simplified these roles and contexts based on research about the information needs of different types of decision makers (Lavis et al., 2003; McKenzie et al., 2014; Rosenthal et al., 2014). For example, in describing how different target audiences for knowledge transfer make different decisions and different information needs, Lavis et al. (2003) identified four general audience types for healthcare information: service recipients, service providers, organization managers, and elected policy makers (Lavis et al., 2003). We followed this approach, integrated information from the Andean sites, and, based on their responses about the salience, credibility, and legitimacy of information, collapsed the seven roles and five decision contexts identified in the Brazilian study into three audience types with distinct information needs for two key questions.

We categorized interview responses by their relevance into salience, credibility, or legitimacy and in the context of either quantification of hydrologic impact or siting of NBS. We then used these responses to inform general insights about the role of salience, credibility, and legitimacy for each hydrologic question by each type of audience. Our findings reflect the overall weight of interview responses as well as the range of responses in each category.

RESULTS AND DISCUSSION

Identifying NBS decision makers

We identified three key target audiences—implementers, advocates, and analysts—with distinct information needs, illustrated in Figure 3. The literature focused on effective transfer of research insight into practice highlights the crucial need to identify a target audience and communicate based on their decisions and associated information needs (Lavis et al., 2003; Rosenthal et al., 2014). However, evaluation of environmental research explicitly undertaken to influence policy‐relevant decisions often finds that the target audience is not well defined (Rosenthal et al., 2014) and that the way information will be used by decision makers is poorly understood (McKenzie et al., 2014; Posner et al., 2016). The diversity of decisions and decision makers could explain these observations: in Bremer et al. (2020), we identified five decision contexts and seven roles within the relatively small and focused group of interview subjects in the project management units of Brazilian watershed investment programs.

Figure 3

Major roles in nature‐based solutions (NBS) projects and NBS modeling for advocates, implementers, and analysts. These actors may be present at the same or different institutions

The three types of target audience identified here—implementers, advocates, and analysts—are all decision makers, each with distinct needs and objectives. For example, analysts are a type of decision maker because the decisions made when setting up and running hydrologic models profoundly affect the information generated. In addition to responsibility for different types of decisions, the three audiences may have distinct training and, therefore, different assumptions and approaches to evaluating NBS. Individuals of each audience type frequently coexist within the same organization, and individuals may be part of different target audiences at various points in project development; audiences are not necessarily aligned in a formal group. Instead, we developed this typology of three audiences to represent collections of people with similar decision criteria, which we describe in the following paragraphs.

Implementers encompass a diverse array of decision makers, ranging from national‐level water agencies with control over funds to pay for NBS projects to local water management agencies and NGOs who must decide whether to assign staff and effort to a particular NBS project or how and where to implement NBS in collaboration with local farmers. In some cases, implementers lack in‐depth familiarity or technical expertise with NBS; for example, many have training in built infrastructure approaches to water management. Depending on their position, implementers have a range of decision criteria for projects of any type, including the cost and effectiveness of proposed projects, and they are not always predisposed toward NBS. One national‐scale implementer noted that “we are different from [advocates] in that they care about green infrastructure whereas we care about the production of water.” Implementing organizations include watershed committees in Brazil that are mandated to spend money on water resources in their watersheds. These committees have historically focused on sanitation and leakage, but are increasingly interested in NBS as part of a larger portfolio and thus need comparative information about the benefits of NBS. In other cases, such as in Peru, legal mandates require expenditures on NBS; in these cases, water management organizations need information to prioritize NBS siting.

Advocates are those actively working to promote NBS. Advocates, including those referred to as project champions, may work within an implementing organization or be external to it. In general, whether directly involved with the implementing organization or not, the role of advocates is to convince an implementing audience to invest in and undertake an NBS project. An example of an advocate is a former intern within the Camboriú municipal water company, which could be thought of as an implementing organization, who identified NBS as a promising option for the water company's legal requirement to invest in some type of watershed activity. Prior to this internal advocacy, which involved engaging The Nature Conservancy and the Brazilian National Water Agency, the water company was under‐spending and primarily focused on litter cleanup and similar activities. This particular advocate eventually played an important implementing role. However, she first had to advocate for higher‐level implementers within the water company to agree to NBS projects. We found that advocates often commission or organize a hydrologic study to concretize a proposed NBS project or to provide evidence of NBS performance. In the Chancay–Lambayeque watershed study, for example, the Natural Infrastructure for Water Security project advocated to include NBS as one of the interventions considered in the water security assessment (Corrales et al., 2019; Taner et al., 2019).

Analysts run hydrologic and other environmental models. Analysts may be employed by an implementing or advocacy organization, or they may be third‐party consultants or university researchers with technical expertise in hydrologic modeling. At the Camboriú project in Brazil, analysts came from a public research agency who collaborated with The Nature Conservancy, the water company, and universities to frame and fund the analyses (Kroeger et al., 2019). Critically, because NBS projects are relatively new and infrequent, many analysts were trained to use hydrologic models in ways distinct from the approaches needed to evaluate the hydrologic performance of NBS. For example, analysts are generally trained to use models to predict the impact of changing rainfall, given existing landcover, rather than to evaluate the impact of changing landcover, given a particular rainfall input.

Quantifying the hydrologic impact of NBS

For the two types of modeling presented here—quantifying the hydrologic impact of NBS and prioritizing siting of NBS—we first discuss credibility, the type of information assessment with which hydrologic modelers may be most familiar. We then focus on salience and legitimacy, which, similar to Cash et al. (2003) and Posner et al. (2016), our interviews suggest may be of paramount importance to the use of hydrologic information.

We found that quantifying the hydrologic impact of NBS is of interest to all three audience types, but for different reasons. Advocates often look to analysts to produce hydrologic information demonstrating to implementers the hydrologic benefits of NBS. This may occur as part of project planning, providing insight to those setting up an NBS project about expected impacts, and it may also occur after a project has begun, as part of project evaluation.

Credibility

The level of accuracy and precision required of hydrologic information varies among audiences and decisions. For example, in the programs analyzed in this study, advocates frequently focused on producing detailed financial return‐on‐investment information to convince potential implementers, particularly funders, that investment in NBS would be worthwhile (e.g., Ozment et al., 2018). Assessing whether to implement an NBS project based on return‐on‐investment requires precise modeling if the return is close to the amount invested. In assessing the Camboriú water fund, for example, analysts Fisher et al. (2018) found that the accuracy of land‐use maps, and by extension, the accuracy of development and NBS scenarios, shifted the return‐on‐investment from positive to negative. However, depending on the decision context, hydrologic information need not be highly accurate to be credible. Fisher et al. (2018) also noted that their findings would have been unlikely to change the decision to invest in NBS despite the change in return‐on‐investment because they found a positive hydrologic impact of the NBS project regardless of the data source. In this, as in many NBS cases, hydrologic data need only be accurate enough to robustly demonstrate direction and magnitude to be credible.

At all scales, implementers voiced interest in the credibility provided by hydrologic models assumed to be most representative of landscape hydrologic processes. Possibly in response to this, we found that analysts often focused on improving credibility by demonstrating the match between modeled and observed data (e.g., Corrales et al., 2019). However, increased model sophistication frequently increases reliance on a large number of unknown parameter values and on calibration, which in turn, decreases credibility if the uncertainty they introduce is not appropriately characterized (Hamilton et al., 2019; Saltelli et al., 2020). The need for many parameter values is common in hydrologic models; yet, whether the number of parameters is useful or problematic depends on the data available to inform parameter selection and for model calibration. For example, the Soil and Water Assessment Tool (SWAT; Arnold et al., 2012), among the most commonly used models in Brazil (de Almeida Bressiani et al., 2015), can have up to 30 parameters whose sensitivity levels rank differently depending on the calibration method and sensitivity analysis used in the same modeling exercise (Khorashadi Zadeh et al., 2017). Uncertainty about key parameter values may translate to far less reliable outcomes than those provided by simple models with clear bounds of uncertainty (Saltelli et al., 2020), and the preciseness of the results produced by multiparameter models can mask substantial uncertainty (Guillaume et al., 2019; van Liew et al., 2005).

The credibility of hydrologic information about NBS is also constrained by the models and modeling approaches used to generate it. In most cases, NBS interventions are modeled by changing parameters associated with different landcover types in an attempt to leverage currently existing hydrological modeling tools (Vigerstol & Aukema, 2011). This can be problematic, as most watershed models are designed to assess how changing precipitation, the forcing condition, will affect water flow (Garen & Moore, 2005). To evaluate the effects of NBS on flow, the analyst must make assumptions about how changes in NBS affect parameters within the model and then use the model to translate those parameter changes to changes in flow. For example, when simulating the effect of a reforestation NBS designed to increase infiltration, if a modeler selects a higher infiltration parameter for forest, this tests the impact of increased infiltration on modeled flow, but not whether reforestation actually changes the infiltration parameter value as modeled (Kirchner, 2006). In lumped or semidistributed models, the use of average parameter values over large areas can make the translation from changed parameter value to flow inaccurate because NBS are often small in scale and strategically sited relative to the size of a watershed. Scenarios, often developed using geostatistics or land‐use change models, introduce another type of uncertainty. In addition, models may not effectively represent the particular NBS issue or decision at hand. In practice, NBS such as check dams, repair of dirt roads, and riparian restoration are often coarsely or ill‐represented in hydrologic modeling (Guswa et al., 2014). As a result, the current nature of hydrologic modeling itself may undermine credibility (Shin et al., 2013), again suggesting that simple models with clear bounds of uncertainty may be more credible in many cases (Saltelli et al., 2020).

We found that implementers often recognize the uncertainty introduced by the choice of model and need for parametrization. For example, the National Water Agency in Brazil pointed to the data limitations for soil and other critical model inputs, with one interviewee noting that “our soil maps are inadequate for [watershed scale] modeling… if we don't know what kind of soil we have in our watersheds, we can't use [the models]… when I did my masters, I used SWAT, but it is deceptive to apply it.”

Implementers also expressed concern about translating models to the montane tropics. In the Andes, for example, implementers expressed deep skepticism about modeling steep slopes and unique Andean ecosystems like the páramo with generalized hydrologic models such as SWAT developed in temperate environments. In Brazil, a recent review found over 100 published studies applying SWAT across a wide range of tropical landscapes (de Almeida Bressiani et al., 2015). Implementers expressed similar concern about generalized hydrologic methods such as the SCS Curve Number, which most applications of SWAT (Arnold et al., 2012), as well as the HydroBID model developed specifically for Latin America (Moreda et al., 2014), rely on and for which most applications use values from published literature, not direct measurement (Malone et al., 2015).

Reflecting the challenges of modeling in low‐data environments, a number of advocates and implementers, in collaboration with analysts, have developed their own models for site‐level assessment. The Natural Infrastructure for Water Security project in Peru developed the CUBHIC (Cuantificación de Beneficios Hídricos de Intervenciones en Cuencas) toolbox to provide a quick assessment of potential benefits of NBS interventions (Foster et al., 2020) using simple formulas implemented in Microsoft Excel, a software that is widely available and used by analysts in the Andes, to compute a water balance equation at a daily timestep. The equations can be seen and modified directly in the spreadsheet if needed and require very limited input data that can be gleaned from the literature or defined a priori using expert criteria. Though the CUBHIC models have limited spatial representation and physical complexity, they offer decision‐relevant information that is simple, transparent, and practical. The Ecuadorian water fund FONAG, the oldest Andean water fund and the program with the most developed monitoring system commissioned analysts to develop a simplified but fully distributed site‐specific model, which they found more credible in the Andean landscape than SWAT (Ochoa‐Tocachi et al., 2020). This was then used to carry out a return‐on‐investment analysis in close collaboration with the water company. The demand for simple models useable in low‐data environments in other regions also drove the development of several InVEST models used in ecosystem service assessments, including a model of seasonal water yield that uses a monthly to seasonal temporal resolution (Hamel, Valencia, et al., 2020; Tallis et al., 2020).

Salience

Regardless of the accuracy and precision of the model used to generate results, the usefulness and usability of hydrologic modeling studies lie first in the relevance, or salience, of the reported outputs. The types of information desired by implementers were varied. Some existing implementers (e.g., Quito's water company) were interested in information to support return‐on‐investment analyses to leverage additional funding or justify existing funding. However, aligning with previous research in which the authors found that actors working at different scales (e.g., local vs. national) often made different types of decisions (Bremer et al., 2020), we found that implementers frequently focused on watershed protection more broadly. These implementers, who often considered NBS as part of a legal mandate, were more interested in a range of co‐benefits than on return‐on‐investment and, thus, needed general hydrologic information about the direction and magnitude of impact to reaffirm their commitment to investing in NBS (Bremer et al., 2020). In other cases (e.g., the Municipality of Extrema, which supported watershed payments for over 15 years), implementers emphasized that they were “already convinced” themselves and were interested in hydrologic information to justify and convince others to increase their impact within and beyond their particular area.

Although, in theory, the process through which an NBS watershed project is generated will articulate the outcomes to be modeled, we found that projects often have multiple goals that are frequently not articulated as modelable hydrologic variables. In three Brazilian NBS projects, for example, in Bremer et al. (2020), we identified general improvements in water quality and water quantity as desired outcomes, but rarely with a specific definition of acceptable levels of pollutants or flow. Andean funds generally focused on maintaining or improving dry season flow, sometimes identifying sediment retention as a secondary objective, and these programs often had clear social objectives related to rural livelihoods in the watershed (Bremer, Auerbach, et al., 2016). As a result of multiple and general objectives, we found that analysts often select the hydrologic model that is easiest to run or for which parameterization or forcing data is most readily available. Outside of São Paulo, for example, although implementers are primarily concerned about the quantity of water available, an advocate report on NBS focused on sedimentation, rather than water availability, because sediment modeling results were more robust (Ozment et al., 2018). By contrast, the Chancay–Lambayeque project in Peru had elements of a more traditional water infrastructure assessment and very clear performance metrics were identified (Taner et al., 2019). However, because projected future water demand substantially exceeded supply in all scenarios, instead of quantifying the total contribution of NBS, analysts focused on ranking the cost‐effectiveness of NBS by watershed (Corrales et al., 2019).

A different challenge to salience is that in many cases, advocates and analysts do not link landscape hydrologic effects to water extraction and water use. A measure of change in groundwater recharge, for example, is insufficient for most decision makers; linking this to a change in water supply or required depth of groundwater wells is necessary for water managers to assess impact (Brauman et al., 2015). In many cases, this requires that a salient assessment include or at least link to a water infrastructure and distribution model. The FONAG model developed with and for implementers explicitly considers water abstraction and return points (Ochoa‐Tocachi et al., 2020) and the Chancay–Lambayeque assessment was designed to link to a water resources management model (Taner et al., 2019).

Models provide key insight into how NBS modify the flow of water across the landscape because they provide information about counterfactual scenarios. Water flows regardless of whether NBS are in place, so measuring total flow provides little insight into NBS performance. Instead, quantifying impact requires comparing water flow with and without NBS in place (Brauman, 2015). As a result, understanding and agreement among advocates, implementers, and analysts about the scenarios that will be compared is critical to the salience of the information generated. In the Chancay–Lambayeque project in Peru, project reports frequently compare scenarios including NBS to flow from the current landscape (Corrales et al., 2019). However, a primary focus of the potential NBS to be evaluated was conservation of forest, given the expectation that without investment, the forest would be converted into agriculture. The salient scenarios for comparison in this case would, thus, be the NBS scenario and an expected degradation scenario, sometimes called “business as usual.” FONAG's return‐on‐investment compared flow from expected degradation with the flow, given sustainable‐ecosystem management; these scenarios were developed by the water fund and the water utility company (Ochoa‐Tocachi et al., 2020).

Legitimacy

The legitimacy of the information generated by hydrologic modeling is strongly connected to the process by which the salience of the information was mutually determined and how well analysts communicated their assumptions and process to advocates and implementers. FONAG's return‐on‐investment study, for example, has largely been viewed as legitimate by the water company implementer because it was carried out in close collaboration with them and using their data. We found that transparent communication among advocates and implementers and the analysts undertaking modeling was key. The general need for advocates and analysts to engage implementers early in the process and communicate clearly about question selection and model setup has been well documented (Cash et al., 2003; Posner et al., 2016; Rosenthal et al., 2014). We found that distinct to NBS projects, advocates have additional communication challenges with both implementers and analysts. Water engineers in both implementation and analysis traditionally think of infrastructure interventions as costly, active investments. Though there is often concern about the cost of NBS, in practice, they are often low cost compared to built infrastructure because they focus on conservation or passive restoration of existing natural infrastructure. For NBS, the benefits of protection or restoration should be conceptualized in comparison to losses of hydrological services in business‐as‐usual degradation scenarios. This type of potentially passive intervention was unfamiliar in both concept and language in the Chancay–Lambayeque watershed assessment, and it proved challenging for advocates to get analysts to consider NBS and business‐as‐usual degradation as part of their portfolio of intervention options. In addition, because the impact of conservation NBS is measured in comparison to expected degradation and cost is heavily dominated by opportunity cost, assessment requires different analytic approaches from those used to evaluate new dam or pipe projects (Corrales et al., 2019). Advocates found that working with both implementers and analysts so that they understood the need for a plausible expected degradation scenario was critical to validating the project.

Prioritizing siting of NBS

Distinct from quantifying the hydrologic impact of a given set of NBS, the process of selecting where and the extent to which NBS will be implemented is a critical element of these projects. From an analytical perspective, this type of decision is an optimization evaluating combinations of NBS in particular locations that will have the largest hydrologic impact for the lowest cost, area, or effort, making it a cost‐effectiveness problem (Guswa et al., 2014; Hamel et al., 2019). Similar evaluations have been undertaken for conventional water infrastructure projects, such as determining the preferred arrangement of dams in a large river system, given competing desires for hydropower and fish passage (O'Hanley et al., 2020). The most rigorous way to undertake an analysis of this type is to repeatedly run a watershed model evaluating the hydrologic impact of many possible scenarios of spatially distributed NBS (e.g., Antolini et al., 2020; Kennedy et al., 2016). This would provide detailed information with which to compare and select among scenarios; however, doing so requires substantial effort to generate scenarios as well as computing power for repeated model runs. Depending on the needs of advocates and implementers, it may be equally informative to use either a spatially distributed hydrologic model to identify areas of high contribution or “hot spots” across a watershed or to forgo process modeling and instead identify sites with characteristics making them likely to be areas of high contribution. The most productive areas are then selected until the constraint of cost, area, or effort is met.

Credibility is a major challenge when siting NBS, as different models often select different priority areas and little data are available to validate the outputs (Guswa et al., 2014). In Hamel, Bremer, et al. (2020), for example, site prioritization generated by running two different models of sediment generation and identifying sites of high contribution produced two distinct maps. Outputs were validated based on how well the models matched the total sediment output from the watershed, a related but distinct measure from validating areas of high contribution. Most hydrologic models have been developed to replicate watershed outputs—a hydrograph or sediment concentration at the watershed outlet, for example—because this is relatively easy to measure and calibrate against (Guillaume et al., 2019). It is easy to assume that if a model is able to accurately replicate watershed outputs, it must be doing so by accurately representing hydrologic processes within the watershed and, thereby, accurately identifying locations in the watershed where processes of interest are most active. However, models could be getting the watershed outputs “right, for the wrong reasons” (Kirchner, 2006). Models with highly simplified assumptions about hydrologic processes that explicitly consider the processes of interest to NBS analysis may, as a result, provide more credible information than more complex process‐based models.

Even if a model is selected that can accurately identify locations of high contribution to water yield or sediment generation, the results may lack salience because these locations may not be the sites in which NBS will have the largest impact. Areas of heavy rainfall or steep slopes may make large contributions to water yield or sediment, but NBS may have little impact modifying flow and constituents (Brauman, 2015). This is a major drawback of simpler approaches to siting such as a single model run across a watershed or selection based on candidate characteristics. Identifying areas that will have the biggest change when NBS are implemented requires comparison runs with and without NBS. Alternatively, simple, site‐scale models to quantify the effect of NBS may provide the required additional information. In the Chancay–Lambayeque watershed in Peru, for example, advocates undertook a complimentary assessment following the completion of the watershed‐scale study using a site‐scale model specifically developed for that landscape (Román et al., 2020).

The ability to integrate non‐biophysical considerations when prioritizing NBS siting is also critical to the salience of analysis. Many NBS projects, and certainly the watershed investment programs that we evaluated, are sociohydrologic programs with strong social and political constraints and opportunities on the location of potential NBS investments (Bremer, Auerbach, et al., 2016). Incorporating local livelihoods, cultural values and preferences, and political conditions into siting processes can be critical to the success and equity of projects (Kolinjivadi et al., 2014). Project sites in Colombia, for example, were highly limited (and continue to be limited) by safety in the context of civil war and activity of armed groups, as well as deep distrust of institutions due to an unjust and colonial history (Nelson et al., 2020; Santos de Lima et al., 2017). The sites where the water fund works are largely determined by where local river user associations have developed trust with communities; in some cases, the trust‐building process has taken 20 years (Meza Prado, 2018). Implementers working directly with communities find a wide range of motivation and constraints on engagement with NBS projects, many of which outweighed the hydrologic benefits and implementation cost so carefully modeled (Bremer et al., 2018). Other implementers, particularly those focused on return‐on‐investment, prioritize locations most like to undergo land‐use change (e.g., Farley et al., 2011). For conservation NBS projects, in particular, substantial effort has been focused on identifying threatened areas to increase the “additionality” and impact of interventions (Goldman‐Benner et al., 2012).

For siting prioritization, the legitimacy of modeling outputs is highly contingent on the credibility and salience of the information generated, but we found that communication and trust with communities in which NBS will be implemented was the most important factor in whether projects were actually implemented. In Colombia, for example, the NBS that have been most successful are those developed in communities that had already organized to protect their own resources, but who welcomed the support of the water fund, once trust was built, to help with financial and technical components. In one watershed where a river user association leader had worked for decades, through deep civil unrest, several indigenous communities noted that it was only because they were confident that this community leader respected them and would act to support their decisions and interests that they agreed to participate in the program. Overall, we found that the most salient, credible, and legitimate outputs of hydrologic models were those that helped identify these places to focus on to build a conversation with stakeholders about where NBS might be appropriate and fair.

CONCLUSIONS

The criteria of credibility, salience, and legitimacy in the context of NBS for water provide insight into the type and sophistication of models necessary to make NBS projects actually happen. NBS projects do require hydrologic information for both impact quantification and siting of NBS. However, to make sure that information is useful and used, it is critical to recognize that decision makers fall into distinct audience types—which we term implementers, advocates, and analysts—with different information needs and facing different constraints. As a result, transparency and mutual understanding among advocates, implementers, and analysts are critical. To generate salient outcomes, deliver an appropriate level of credibility, and do so in a legitimate way, there must be a mutual understanding of the aims of the project to ensure that the actual variable of interest is identified and modeled. For siting projects, it is critical to identify nonhydrologic constraints.

The constraints that potential NBS projects face, as well as the full suite of program goals, inform the type and rigor of modeling that is necessary to support them. In many cases, simple models may be preferable both because of decision makers' information needs and because of the challenges inherent in modeling NBS. When decision‐makers need only general hydrologic information about the direction and magnitude of hydrologic impact, either to reaffirm their commitment to investing in NBS or to convince others, simple models with well‐understood limits to accuracy often suffice. Given the challenges to modeling in low‐data environments and the associated assumptions necessary to model NBS, simple models with clear bounds of uncertainty are often more transparent, practical, and, thus, decision‐relevant.

A with many NBS in the form of Investments in Watershed Services, the subset of NBS projects that we evaluated are interested in a suite of nonhydrologic benefits ranging from biodiversity conservation to alternative livelihoods and they are implemented as part of complex social systems in which participants value a wide range of returns beyond the financial (Arriagada et al., 2015; Bremer et al., 2018; Grillos, 2017). As a result, using purely hydrologic criteria to assess the value of the information provided by hydrologic models is insufficient to indicate whether that information will be useful and used. Indeed, a narrow focus on hydrologic model performance, ignoring the larger social, political, and environmental context of the project and decisions that the models support, can severely disadvantage already disadvantaged communities (Hamilton et al., 2019) and reduce the success and legitimacy of projects (Kolinjivadi et al., 2014). Increasingly, programs have emphasized the importance of social equity in program design and the need for programs to be responsive to myriad reasons for participation (Bremer et al., 2018; Corbera & Pascual, 2012; Lliso et al., 2021; Pascual et al., 2014). These additional considerations make the process of producing useful information less straightforward than a hydrologic study, thus making a focus on salience and legitimacy ever more important. Simpler models that gain legitimacy because they are more easily explained often provide sufficiently credible information to take action in NBS projects. These findings may well be valid beyond NBS and apply to broader watershed management modeling projects.

CONFLICT OF INTEREST

The authors declare that there are no conflicts of interest.