What do you mean by false positive.

Misunderstandings regarding the term "false positive" present a significant hurdle to broad adoption of eDNA monitoring methods. Here, we identify three challenges to clear communication of false-positive error between scientists, managers, and the public. The first arises from a failure to distinguish between false-positive eDNA detection at the sample level and false-positive inference of taxa presence at the site level. The second is based on the large proportion of false positives that may occur when true-positive detections are likely to be rare, even when rates of contamination or other error are low. And the third misunderstanding occurs when conventional species detection approaches, often based on direct capture, are used to confirm eDNA approaches without acknowledging or quantifying the conventional approach's detection probability. The solutions to these issues include careful and consistent communication of error definitions, managing expectations of error rates, and providing a balanced discussion not only of alternative sources of species DNA, but also of the detection limitations of conventional methods. We argue that the benefit of addressing these misunderstandings will be increased confidence in the utility of eDNA methods and, ultimately, improved resource management using eDNA approaches. The term false positive is often misused in eDNA research and natural resource management. There are issues of scale of inference, the base rate fallacy, and confirmation errors using conventional methods of detection. We offer a perspective to guide discussions of errors in species detection.

THE PROBLEM

The fear of false-positive detections is commonly cited by end-users as a primary hurdle to adopting eDNA-based approaches for species surveillance and monitoring (Jerde, 2021). Unfortunately, this fear is compounded by misunderstandings of what is meant by the term “false positive” and unrealistic expectations regarding eDNA detection error rates and sources of error. Estimating the probability of false-positive error is, of course, an important aim of eDNA research and has received considerable attention in the literature, leading to the development of sophisticated statistical approaches to error estimation (e.g., Griffin et al., 2020; Tingley et al., 2021). However, despite these advances, there still persist challenges to communicating the meaning of these errors when they exist and to establishing terminology that clarifies rather than confuses. Achieving common understanding of these issues should be an immediate goal of eDNA practitioners, and doing so will facilitate the development and appropriate application of eDNA methods in a variety of operational contexts.

At least three areas of misunderstanding persist around the term “false positive.” The first of these is semantic. Chambert et al. (2015) lays out a useful distinction between false-positive samples (i.e., errant detection in an individual sample) and false-positive sites (i.e., errant detection at an unoccupied site) in biodiversity monitoring. This distinction is particularly useful for eDNA studies, since eDNA methods test for DNA presence in a sample and it is entirely possible to have a true-positive sample when the organism itself is not present at the site (e.g., sampling location or water body). Unfortunately, it is common for both circumstances to be described as “false positive” eDNA detections. We believe this definition reflects a fundamental confusion between sample-and site-level inferences. Whereas any individual eDNA detection can address only the sample level, substantially more advanced approaches (e.g., occupancy modeling, error analysis, or other statistical methods based on extensive sampling programs) are required to achieve site-level inferences. If target DNA is present at a location at the time of collection and detected in a sample even though the organism is not observed at that location, it is highly misleading and inappropriate to use the term “false positive” to imply that the eDNA test is erroneous when, in fact, it is doing exactly what it is designed to do— that is to say, detect the presence of target DNA in a sample, regardless of its spatiotemporal origin. The error in such cases lies not with the test, but rather with the unrealistic expectation that the test ought to be doing something more than it is designed to do or capable of doing— specifically, definitively identifying whether or not the organism is present at the site at the time the samples were collected.

This semantic clarification is of particular importance when considering the spatiotemporal dynamics of eDNA and species movements. eDNA can be horizontally transported by current, vertically transported by gravitational settling, trapped by stratified water layers, and inadvertently transferred by humans and wildlife (Andruszkiewicz et al., 2019; Deiner et al., 2016; Lafferty et al., 2021; Littlefair et al., 2020; Merkes et al., 2014). Depending on the ecology of the eDNA (Barnes & Turner, 2016), it may degrade and disappear quickly or, if it degrades slowly, it may persist and be transported long distances, retained for prolonged periods of time, or resus-pended into the water column at later times. Clearly, environmental conditions affect the detection pattern, and positive eDNA detections are possible outside the spatial or temporal range of the target species’ presence (Barnes et al., 2014; Lafferty et al., 2021). Adding to this complexity is that many species targeted for eDNA surveillance are highly mobile and temporal or geographic presence is transient compared with the persistence of eDNA (Littlefair et al., 2020). Taken together, eDNA transport and species movement patterns can make the DNA of some species or entire communities detectable, even when the species is not present inside the area in which water samples are collected— yet the DNA is present and the assay is performing correctly.

Calling this result a false-positive site distracts from identifying potential false-positives samples, which may be due to contamination, failures of primer specificity, or other technical issues which are recognizable and tractable and can be addressed with appropriate quality assurances and statistical models (Darling & Mahon, 2011; Goldberg et al., 2016; Tingley et al., 2021). More importantly, using the inaccurate language of “false positives” to generalize outcomes erodes confidence in the method from end users. If we set false expectations for a novel method, the failure may be ours, but the temptation is to place all blame on the method and subsequently reject its application to the detriment of scientific enquiry and effective natural resource management (Jerde, 2021). The targeted species assay approach to eDNA surveillance is easier to conceptualize, but the problem is similar for the metagenetic approach (Lacoursière-Roussel & Deiner, 2021), where groups of species may have DNA found within a sample, but not be physically present in the water body sampled (allochthonous eDNA). Similarly conceptualized, these are not false-positive samples, but rather a potential overreaching of expectation about the eDNA metabarcoding approach.

Adding to the confusion is a second misconception emerging from the common application of eDNA approaches to inferring presence of rare species. The “base rate fallacy” is a well understood, and yet widely overlooked, problem in the detection of rare events (Bar-Hillel, 1980). When the likelihood of a true-positive detection (the “base rate”) is extremely low because the target of surveillance is rare, even very specific tests may yield higher percentages of false-positive detections than desired. A method with an expected false-positive rate of 1 out of 100 tests will yield 50% false positives if the base rate of true-positive detections is only 1 out of 100 samples. In fact, empirical evidence suggests that false-positive rates for eDNA detection methods may sometimes be much higher than this (e.g., Sepulveda, Hutchins, et al., 2020; Smith & Goldberg, 2020). Communicating this expectation about the rate of false positives for rare species detections to end users is critically important because it emphasizes just how essential it is to have a tolerance for false-positive sample detections when looking for something that is likely to be rare (Sepulveda, Nelson, et al., 2020). The base rate is often low for invasive species monitoring, species eradication monitoring, and monitoring of threatened and endangered species. Even extremely specific tests will deliver higher than desired percentages of false positives if the base rate is very low, and end-users who fear false-positive detections need to understand that sample-level false positives are likely unavoidable in any realistic surveillance context.

A third uncertainty is associated with the challenge of confirming site-level inferences based on sample-level positive detections. How does one determine if there has been a site-level false inference, that is, a positive eDNA detection in a sample despite the absence of the target from the water body at the time of sample collection? One of the primary motivations for exploring eDNA surveillance is its potential for greatly increased sensitivity over available alternatives. This means there is some expected target-species density at which the likelihood of eDNA detection is far higher than the likelihood of direct detection by conventional, non-molecular approaches (McElroy et al., 2020; Wilcox et al., 2016). Unfortunately, it also means that conventional approaches may be insufficient or unable to provide timely confirmation of species presence based on positive eDNA sample detections (Hunter et al., 2019; Jerde, 2021). If positive eDNA sample detections are followed by negative site detections using conventional methods that are known or suspected to be less sensitive, the end-user could interpret this as a site-level false positive for the eDNA method, or a site-level false negative for the alternative method depending on the efficacy of the methods.

Knowing the difference in site-level errors could be extremely challenging for resource managers faced with making costly decisions. It should be noted that recently developed statistical models are well-suited to address this issue, providing quantitative estimates of the probability of site-level presence or absence of the target given a particular pattern of eDNA detections and ancillary data relevant to assessing site occupancy (Griffin et al., 2020; Lahoz-Monfort et al., 2016; Tingley et al., 2021). Unfortunately, the default seems to be to trust the conventional tools and saddle the molecular method with false-positive error until conventional approaches ultimately confirm eDNA-based inferences, even in the absence of more rigorous attempts to assess the likelihood of site occupancy (Jerde, 2021). This delayed action may come with added management costs of preventing invasions (Finnoff et al., 2007) or localized extirpation of rare species of concern. More to our point, this approach undermines confidence in eDNA methods by potentially conflating sample-and site-level false-positive error. Even in cases where statistical evidence suggests that site-level target-species presence is unlikely despite one or more positive eDNA tests, this does not resolve the issue of whether or not there has been sample-level false-positive error. We maintain that in these contexts it is misleading to generally label these eDNA detections as false positives.

SOLUTIONS

As a community of scientists, we are still grappling with how to achieve common understanding of something as fundamentally important as false positives. In the absence of greater clarity in the way, we discuss eDNA false positives there will be continued misunderstanding and, potentially, distrust. Practitioners should be very clear about the distinction between sample-and site-level false positives, should recognize that every presumptive false positive is weighed against evidence with a non-zero false-negative rate, and should strive to educate end users about how the base rate fallacy impacts their understanding of the false-positive rate.

But what can be done to improve scientific and science communication with expert and non-expert audiences (Mosher et al., 2020)? First, eDNA experts must self-police to ensure correct and consistent use of technical terms and to ensure that these terms are always associated with clear definitions that can be easily understood by non-experts. Indeed, other practitioners have made a similar plea (e.g., Lacoursière-Roussel & Deiner, 2021) and recent publications now provide explicit, detailed definitions for technical terms like “limit of detection” and “limit of quantification” (e.g., Klymus et al., 2019).

To reduce the semantic ambiguity of the term false positive, it may be necessary to introduce additional terms that reflect important nuances in eDNA monitoring. We recommend that the term “false positive test” be reserved for those cases in which sample-level error can be clearly demonstrated— for example, those cases where detections are attributed to non-target amplification or associated with contaminated negative controls. For cases in which species presence at the site level is inferred from a pattern of eDNA detections, we recommend the term “presumed positive site” to describe such sites in the absence of non-eDNA confirmation. For presumed positive sites in which non-eDNA methods have demonstrated, to an acceptable level of confidence, that the target species is not present, we reserve the term “false positive inference.” We note that these latter cases are likely to be rare, depending on what end users consider to be “acceptable level of confidence” in the assertion that a site is truly negative for the species.

Determining the mechanisms leading to site level, false-positive inferences is also critical, and it is important to recognize that such errors will not always, or even often, be due to sample-level, false-positive tests. As previously discussed, spatiotemporal dynamics of eDNA and species movements and inadvertent transfer of DNA by humans and wildlife can cause mismatches between the presence of target-species eDNA in a site and the presence of the target organism at the site when the sample was collected. It is crucially important to modulate expectations in these cases where the area of reasonable inference does not align with our understating of the fate and transport of DNA more broadly (Jerde & Mahon, 2015). For these reasons, it is worth working with science communicators and managers to reinforce that eDNA methods detect the presence of target eDNA regardless of the state of the DNA source and that eDNA methods are not the same as direct species observations. However, these concerns should also be presented with a balanced discussion about confidence or apprehension in the target organism being present, releasing DNA, but remaining undetected by potentially less sensitive conventional survey approaches.

Integrating fate and transport of eDNA is a recognized need for improved eDNA occupancy modeling, largely to account for positive eDNA detections when the target organism is presumably absent (Jerde & Mahon, 2015) or to project the upstream distribution of a species or community (Carraro et al., 2018). In many aquatic systems (lentic, lotic, and marine), there is some understanding of how conservative tracers move (Cao et al., 2020) and existing studies may be informative to eDNA transport. However, in small streams the conservative tracer analog may not hold due to substrate effects (Jerde et al., 2016) and the retention and resuspension of variably sized eDNA particles (Shogren et al., 2017). Yet with some reasonable assumptions, Andruszkiewicz et al. (2019) were able to use Lagrangian particle tracking in a marine system to delimit the source location for eDNA of Northern anchovy (Engraulis mordax). The robustness of fate and transport eDNA model parameters is an active area of research and may need to be context specific, requiring some baseline information about water flow and eDNA degradation rate (Barnes & Turner, 2016). Minimally, any species list derived from eDNA detection with unusual detections (such as marine fish DNA found in freshwater systems) should be reported, scrutinized, and put in context of transport and/or contamination (Sepulveda, Nelson, et al., 2020).

eDNA experts must continue to develop and follow best practices for reducing potential for false-positive tests resulting from contamination, non-specific amplification, or errors in genomic reference databases. General considerations for field sampling and laboratory practices were provided in Goldberg et al. (2016) and have been improved on by Minamoto et al. (2021), Schenekar et al. (2020), and Thalinger et al. (2021), but finer guidance is still needed to improve confidence in positive results. For example, most targeted and metagenetic eDNA studies now process negative controls to assess for contamination during laboratory analyses; however, far fewer studies have included negative controls to assess for contamination in the field (Sepulveda, Hutchins, et al., 2020). Unfortunately, due to unique needs, there is little consistency across studies in the types (e.g., field, travel, equipment, and PCR) of negative controls collected, the water source for negative controls (e.g., field water from a site where the target species are presumed absent vs. deionized water), the proportion of negative controls to field samples, and how to proceed when negative controls have unexpected amplification (Sepulveda, Hutchins, et al., 2020). This diversity of quality assurance measures likely reflects study-specific nuances, but may appear as chaos and incompetence to a non-expert since quality assurance protocols in other fields adhere to standardized protocols. Developing eDNA quality assurance methods that meet auditable standards should provide non-experts with increased confidence that positive field samples were not due to contamination.

Finally, non-expert decision makers must become part of the eDNA study design process rather than just recipients of the results. Involvement of decision makers is the fundamental requirement of decision analysis (Keeney, 2020). This integration will help to ensure clear and effective science communication, regardless of how the study results are ultimately shared. Decision makers and scientists should jointly develop an operations and communications plan that includes jointly defined terms like false positives, identifies critical quality control and assurance measures, and criteria for what constitutes actionable, positive eDNA results. Decision trees can form the basis for communicating results across broad audiences and stakeholders (see Sepulveda, Nelson, et al., 2020 for an example with invasive species).

BENEFITS

The use of targeted species and metagenetic eDNA approaches for natural resource management is a game changing tool, but as exemplified by other indirect methods such as remote sensing (He et al., 2015; Kerr & Ostrovsky, 2003), it comes with added complexity of errors and inference that must be acknowledged. We advocate for improved clarity in the communication of these errors as part of any eDNA-based program with importance on par with best practices in field sample collection and laboratory techniques. This includes efforts to quantify error rates, application of probabilistic models to estimate detection given these error rates, delimitation of the area over which the inference is being made, acknowledging our limitations in delimiting species from conventional methods, and listing legitimate, alternative sources of species DNA. But it also entails semantic clarity and adoption of common terminology for fundamental concepts critical to communicating the results of surveillance and monitoring efforts. If these types of solutions are in place, managers will be more confident when using eDNA sampling as another tool to help detect biological hazards when they are at low abundances or cryptic.