Quantifying ecosystem quality by modeling multi-attribute expert opinion.

The evaluation of ecosystem quality is inherently subjective, requiring decisions about which variables to notice or measure, and how these variables are integrated into a coherent evaluation. Despite the central role of human judgment, few evaluation methods address the subjectivity that is inherent in their design. There are, however, advantages to directly using opinion to create an expert system where the metric is constructed around opinion data. These advantages include stakeholder inclusion and the encouragement of a dialogue of data-driven criticism rather than subjective counter-opinion. We create an expert system to express the quality of a grassland ecosystem in Australia. We use an ensemble of bagged regression trees trained on calibrated expert preference data, to model the perceived quality of this grassland using a set of eight site variables as inputs. The model provides useful predictions of grassland quality, producing predictions similar to real expert evaluations of independent synthetic test sites not used to train the model. We apply the model to real grassland sites ranging from pristine to highly degraded, and confirm that our model orders the sites according to their degree of modification. We demonstrate that the use of too few experts produces relatively poor results, and show that for our problem the use of data from over twenty experts is appropriate. The scaling approach we used to calibrate between-expert data is shown to be an appropriate mechanism for aggregating the opinions of multiple experts. The resultant model will be useful in many contexts, and can be used by managers as a tool to evaluate real sites. It can also be integrated into ecological models of change as a means of evaluating predicted changes, for example, as a measure of utility when combined with cost estimates. The basic approach demonstrated here is applicable to any ecosystem, and we discuss the opportunities and limitations of its wider use.