Describing communication during a forensic investigation using the Pebbles on a Scale metaphor.

During the investigation of a crime, evidence is collected, analyzed, interpreted, and discussed by various stakeholders. This article examines the communication that may occur between two of these stakeholders: detectives and forensic analysts, and how their interaction influences the interpretation of evidence as the investigation proceeds and the theory of the case evolves. Such communication can be understood as sets of actions that are inter-dependent: for example, a request for a specific analysis by a detective leads to analyses and conclusions that the analyst shares with the detective, which leads to an assessment of these conclusions relative to the theory of the case, which leads to further analysis requests, and so forth. We present the Pebbles on a Scale metaphor, which describes how communication and the understanding of evidence takes place between the detective and analysts, and the different ways in which they consider the information as a function of their roles in the investigation. Using a hypothetical case for illustration, we discuss communicative challenges, the evolving theory of the case, the language that is used by analysts to discuss "yes", "no" and "I don't know" conclusions, and how those conclusions are used by detectives during the progression of the investigation.

Forensic disciplines specialize in the processing of specific types of evidence, such as DNA, latent prints, footwear, firearms, bloodstain patterns, trace, fire debris, seized drugs, questioned documents, and digital/video imaging. Although each discipline has developed its own processes for collection and analysis, all disciplines operate within a fundamental framework by which information is communicated back and forth between stakeholders, including crime scene investigators, law enforcement officers, detectives, forensic scientists, lawyers, judges and juries. The focus of this article is on the communication between two specific stakeholders: detectives and forensic scientists. Our goal is to examine how this interaction influences the way in which evidence is interpreted as the investigation proceeds and as the theory of the case evolves.

The foundational idea that communication is a joint action between two interlocuters to accomplish a goal [1] is critical to an understanding of the role communication can play in shaping and supporting the evolving theory of a case. Embedded in this approach is the idea of common ground – during communication each party makes assumptions about the overlapping knowledge that the parties share and therefore does not necessarily need to be stated (e.g., [2,3]. For example, when a detective issues a request for a specific analysis, or when a forensic scientist issues a report of those analyses, there may be unspoken but assumed shared understanding of what a given test entails and the scope of its conclusions. Conversely, there may be disparities between how the forensic scientist and detective view the same result based upon their differing goals, perspectives, and access to broader case information. Throughout this article, occasions for such unstated inferences are identified and the assumptions unpacked so as to better reveal the dynamic and interactive nature of this communication and the liabilities that may occur when these remain latent.

In examining this communication, we adopt a task decomposition approach (e.g. [4,5], that emphasizes the value of breaking a task down into underlying components and assessing the goals and required knowledge and skills at each stage for the overall completion of the task. As an example, consider Fig. 1, which provides a stage-process diagram illustrating a typical sequence of interactions that involves the communication of information between detectives and analysts. Note that multiple detectives and analysts may be involved at each stage; as such we intend “detective” and “analyst” to be interpreted as category labels rather than single individuals (e.g., analyst may refer to crime scene investigators and to forensic scientists, depending upon the stage of investigation).

Fig. 1

Stage-process diagram of actions and moments of communication between detectives (blue boxes, left side) and analysts (yellow boxes, right side). This article presents the Pebbles on a Scale metaphor to represent communicative actions within the box in the middle of the figure: requests and assessments by the detective, and analyses and conclusions by the analysts. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

During the investigation of a suspected crime, the detective is involved in a series of communicative actions, summarized in the green boxes on the left-hand side of Fig. 1: they receive notification of a crime; they request analyses; they assess the conclusions from these analyses in light of their hypotheses of what happened; and they seek new analyses as necessary. The analyst is also involved in a series of communicative actions, summarized in the yellow boxes on the right-hand side of Fig. 1: they collect and document evidence; conduct analyses of the evidence as requested; and issue conclusions with respect to these analyses. The arrows in Fig. 1 indicate moments of communication when there is a flow of information between the detective and the analysts during these sequences.

These communicative actions are inter-dependent, and understanding these moments of communication requires not only an understanding of the complexity of each one but also understanding how they all fit together.

In this article we describe the Pebbles on a Scale metaphor, which we have developed with the intent to graphically represent the centrally framed sequence of communicative actions between the detective and analyst in Fig. 1. The value of a metaphor is that it takes something that is abstract and reveals its key properties in more concrete terms. This symbolic use is helpful for understanding how complex systems work, but it is important that they are understood only as an approximation and not as an explanatory model. Our intention is to describe how the communication and understanding of evidence takes place between the detective and analyst, and the different ways in which they consider this information as a function of their role during the investigation.

The basic elements of the metaphor are shown in Fig. 2 and consist of a bag of pebbles and a balancing scale. The analyst's selection of a pebble is triggered by a query from the detective to the analyst about a piece of evidence, and the conclusion drawn by the analyst in response to that query. The color of the pebble represents the analyst's answer to the question (yes, no, I don't know) and the size represents the analyst's confidence in that conclusion. The placement of the pebble on the scale is made by the detective, who assesses the conclusion's relevance to the theory of the case. The pebble's location indicates whether or not the pebble provides support for the evolving theory– either in the “yes” pan; the “no” pan; or at the base, which indicates “I don't know,” or a lack of relevance or clarity as to how the evidence affects the theory. Thus, the balance of scales reflects the degree of support for the detective's evolving theory of the case at a given moment in time. The Pebbles on a Scale metaphor is a metaphor for how scientific information is communicated between forensic scientist and detective and represents how this information is interpreted by the detective in the context of their current understanding of the greater case. As such, any pebble placed on the scale could be moved or resized during the investigation as the theory of the case continues to evolve.

Fig. 2

The basic Pebbles on a Scale metaphor in which the analyst selects a colored pebble of a particular size to convey the conclusion of a given analysis, and passes it to the detective who places it on a scale that represents the evolving theory of the case. The red, yellow, and green ball colors correspond to the analyst’s conclusions of “no”, “I don’t know”, and “yes”, respectively and their size indicates the analyst’s confidence in that conclusion. The placement of the balls on the “Yes”, “No”, or “I don’t know” areas of the figure is done by the detective to indicate their interpretation of the support the evidence provides to the evolving theory of the case. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

The organization of the remainder of the article is as follows. Following the time course of Fig. 1, Section 1 sets the stage for a discussion of the Pebbles on a Scale metaphor by describing possible communicative challenges and recommended best practices during the notification of a crime and the collection and documentation of evidence, steps that precede the processes of central interest in this article. This section also presents a hypothetical case for illustrative purposes that will be further developed in each section of the article. Section 2 focuses on the “request” action by the detective and delves into the representation of the theory of the case as scales whose balance represents the likelihood of the theory being true within the context of that evidence. Section 3 examines the “analyzes and concludes” actions of the analyst, and discusses the representation of a conclusion of an analysis for a given piece of data as a pebble. Section 4 turns to the detective's assessment of the conclusions drawn by the analyst, which takes place in the context of an evolving theory, and is conceptualized in our metaphor as the detective placing the pebble at the appropriate location on the scales. In Section 5, we offer concluding reflections.

Section 1: Communicative challenges related to notification and evidence collection and documentation

Notification

Law enforcement has arrived at the scene and starts observing and forming questions to be explored. A detective contacts the crime scene unit to come collect forensic evidence. The communication channel is typically a phone call. Prior to arrival, the crime scene unit is provided with preliminary information that includes location of scene (apartment address and an indication that the crime scene is indoors), number of possible victims (1), possible types of injuries (gunshot wound), and any special circumstances (apartment is on 10 floor and elevator is broken). The communication focus here is to ensure that the forensic experts know how to find the location and what to expect so that they can anticipate how to process the scene.

A key communicative challenge is to ensure that the crime scene unit does not receive any potentially biasing information that could influence the way in which the crime investigators process the scene. This challenge is quite difficult to address. Extensive research has shown that people automatically draw inferences during comprehension [6], with a bias to imagine the situation being described, and generate inferences that explain that situation [7]. Moreover, particular words can bias the way in which these inferences are drawn. Classic work by Loftus and Palmer [8]; for a review see [9] illustrates this point dramatically. Participants were shown a video of a car accident involving two cars. After watching the video, participants were asked to estimate the speed that the cars were travelling, using the prompt “about how fast were the cars going when they each other?” was replaced by one of five verbs: smashed, collided, bumped, hit, or contacted. The key finding was that speed estimates increased as a function of the intensity of the impact as conveyed by the verb. This bias also affected the likelihood that participants reported that there was broken glass in the video (there was never any broken glass). Such findings have been replicated extensively and this work well-establishes the automaticity of generating inferences which form the basis for making sense of the world.

The difficulty is that descriptions often reflect such inferences. With respect to our case example, a problematic example of a notification from law enforcement to the crime scene investigator would be: The female victim lives in a 10 floor apartment at 101 W. Main St. She must have been in good shape because the elevator is broken. Suspected homicide from a gunshot wound. This is a problematic example because it invites the following unsubstantiated inferences: victim is used to walking up the stairs; she was killed (as opposed to suicide); and the cause of death was gunshot. Note that it is not that these inferences are not valuable – they are, and they all need to be verified by the detectives as they build up an understanding of the case. It is just that they are not relevant and are potentially biasing for forensic experts (see Spellman et al., [10]). To be of most assistance to a crime scene investigator, law enforcement needs to minimize as much as possible the communication of such inferences by focusing exclusively on what can be directly observed and is verifiable. A better example would be: The incident is at 101 W. Main St, inside a 10 floor apartment. The elevator is broken, so use the stairs. There is one possible victim, and injuries include an apparent gunshot wound.

Best Practice: We recommend the development of a template for conveying the basic information about the crime scene necessary for notification, such as where, count of victims, types of evidence to anticipate, and any direct observations that may assist the crime scene investigator's arrival onto the scene.

Evidence collection and documentation

The crime scene investigator is given a briefing of the situation from the detective or on-scene officer that represents the most current understanding of the incident based on observations and any initial interviews with witnesses. It is important at this step to separate out observations from interpretations of what happened. For example, task-relevant information that should be conveyed to the crime scene investigator relates to facts, such as “there is a cartridge case by the couch,” or any information related to changes to the scene, such as “we turned the lights on and the radio off.” In contrast, task-irrelevant information that reflects the in-the-moment interpretation, such as “this looks like a drug deal gone bad” should be avoided because it may affect decisions about which evidence to collect. Naturally, the crime scene investigator needs basic information on the type of incident and evidence items of interest to focus their search, but care should be taken to avoid giving biasing information that can color their interpretation of the information they observe, or keep them from collecting information that could later prove to be relevant. It is inevitable that the investigators even at this early point are constructing theories of what happened. This is a good thing. The point here is that these theories should not be shared with the crime scene investigators because it may influence their scene treatment and assessment.

The crime scene unit begins to document the evidence, with the goal of creating an objective recording and a fair and accurate representation of the scene. Note that an accurate representation may only be partially attainable based upon the methods and technology utilized to record the scene (e.g., digital photography versus by 3-D laser scanning techniques). It may be impossible to record the entire scene, providing the potential for bias in the selection of what to record and overall size of the scene. However, there are tools available to minimize the human bias influence and increase objectivity. For example, utilizing 3-D laser scanning technology to record and measure crime scenes can provide accurate documentation in less time and an unbiased viewpoint. Note this technology is expensive and may not be available to all laboratories or jurisdictions.

The decision about what to collect as evidence has two sources: the crime scene investigators and the detectives. The crime scene investigators have the jurisdiction for collecting any evidence that they deem forensically relevant - evidence that may be of use for answering a potential question in the case. This task is particularly challenging because these investigators should be operating without an unfolding theory of the case. For example, imagine that two sets of wine glasses had been taken out of the cabinet - one set is found on the table in front of the couch where the victim is found, and another set is in the drying rack by the sink. Without a sense of forensic relevance, there would be no way to guide the crime scene investigator to be sure to take pictures of the glasses on the coffee table but not necessarily in the drying rack (or maybe both). As such, crime scene investigators often use heuristics to determine what to collect, such as objects out of place, the scene around the victim, and possible identifying evidence. For example, a blood droplet may be collected to determine whether it belongs to the victim or someone else, which would place an additional bleeding person at the scene.

In addition to the crime scene investigator working from heuristics, the detective can also direct collection of evidence based on his/her evolving interpretation of the scene. However, even here, this is only an initial guess. As such, a productive partnership is particularly important at this stage because there is only one chance to record the scene. At this point, who decided to collect a given piece of evidence is not relevant; the evidence is simply added into the case record for possible testing because its potential value with respect to determining what happened is not yet established, and many possible scenarios need to be considered.

During or after collection, the crime scene investigator provides a briefing to detectives about what they have observed, documented, and collected. As with the opening briefing, this needs to be factual, and any evolving theory of the crime that the crime scene investigator has developed should not bias the information provided in their briefing, either through emphasis or exclusion.

An additional communicative challenge is that the recording of any evidence collected is itself subject to bias. For example, the size of a given object or trace evidence may be impossible to assess from a close-up photo without the inclusion of a reference object. In addition, the perspective from which photographs are taken may dictate how information is prioritized. For example, a photograph of the front of a couch (its normal interactive side) may discourage further exploration of evidence that may have fallen on the sides of the couch that are less used, such as a stain on the back side. Without a photo of all sides of the object, it may not be possible to spatially locate the evidence relative to the scene, and this could compromise the evolving theory of the case (for example, explanations of how and why there was a stain on the back of the couch).

With respect to our example, at the end of the scene processing phase there is now a case record that includes the following:

Physical evidence from the scene:

two cartridge cases on the ground

green leafy and white powder substances

latent print cards

10 swabs of apparent blood

Documentation of the scene:

photographs of scene and evidence location (e.g., cartridge cases in front of the couch)

sketches and diagrams of the scene

narrative report from the crime scene investigator that details the evidence collected

Physical evidence from the morgue:

two projectiles recovered from the body

sexual assault samples (vaginal, rectal, and oral swabs)

post-mortem samples (blood, urine)

victim's blood standard

Best Practices: We recommend the assignment of primary and secondary roles at the crime scene. The primary would serve as the decision maker for what to record and how to record it at the scene. The secondary would control the flow of information to the primary. This would include filtering requests and information received from law enforcement investigators at the scene to eliminate any information that would reflect an emerging theory of the incident.

Section 2: The “request for an analysis” action by the detective

We now move into the processes represented by the Pebbles on a Scale metaphor, starting with a request by a detective to an analyst to conduct specific analyses on the collected evidence. Such requests are issued in the form of questions about the evidence. For example, a request to analyze the two cartridge cases found at the scene in our sample case may be presented with this query: Were the cartridge cases from the same gun? The questions that the detective asks of the evidence are often based on an initial set of hypotheses about what happened, and the results of the tests and the conclusions drawn by the analyst are taken to support or contradict the evolving theory of the case. For example, if the theory is that there was one shooter, then evidence that the cartridge cases came from the same gun would be deemed to support the theory; evidence that they came from different guns would not clearly support either the theory or an alternative theory more strongly (one shooter could have two guns, or there could be two shooters, each with a gun). Thus, the detective views the results of a given test within the context of the evolving theory. Fig. 2 illustrates this idea using a scale, with one pan holding evidence in support of a given theory of the case at that moment in time, and one pan holding evidence that supports some alternative theory of the case. If the conclusion of the analyst is supportive of the theory of the case, the detective would place a pebble corresponding to that test in the “yes” pan; if it was disconfirming or in support of an alternative, it would be placed in the “no” pan. If it supported neither hypothesis clearly, it would be placed at the base. As tests on the evidence accumulate, more pebbles are added to the scale, with the overall balance of the pans representing support for the current theory of the case or for an alternative theory.

As the detective considers how a given piece of evidence fits into the evolving theory of the case, it is critical that time is spent considering which analyses are being requested as well as the form and content of the request.

Selecting analyses. When selecting which evidence to process and the underlying question being posed with the request, it is important that the detective guard against confirmation bias [11], the human tendency to look for evidence in support of a theory rather than evidence to refute a theory. The results of a test that confirm a theory are necessary for establishing that the theory is a plausible explanation. However, such tests by themselves are not sufficient, because confirmation in support of one theory does not rule out the possibility that the same result may also support any number of other additional theories. Thus, the detective needs to also assess the evolving theory of the case by posing tests on the evidence that have the potential to disconfirm the theory. If the theory passes those tests in addition to being supported by confirming evidence, the detective can have more confidence in the evolving theory. For example, if the evolving theory of the case is that there was one shooter, conclusions of analyses on the cartridges indicating that they came from the same gun would be supportive. However, if the blood sample swabs indicated the presence of two people at the scene in addition to the victim, then that would raise the possibility of the gun being used by more than one shooter.

Communicating the request. In many laboratories, requests are made through a central portal that includes a narrative specifying the question to be answered and the evidence to be tested. As with crime scene processing, only task-relevant information should be included in the narrative request. It is critical that communication is clear at this point because testing the appropriate evidence items initially saves both time and resources, and ensures a steadier flow of case processing through the system to best serve all the cases being investigated.

Nevertheless, there are a number of communicative challenges that may occur at this stage. First, the narrative requests may be too vague. For example, consider possible narrative requests related to the two cartridge cases found at our scene. A narrative request that is vague such as “ballistics testing” is problematic because the analyst must seek clarification – for example, is the request whether the cartridges cases came from the same gun? Is the request to determine the type of gun that fired the cartridge cases? The analyst is not able to significantly alter the request without notification, and both the original request and any approved changes are entered into the case history. These additional processes cause delays in the release of results, which prevent the detective from receiving the necessary information in a timely manner in conjunction with other lab results to form a more complete picture of the narrative of the crime and potentially unduly influence the decision scale. Second, there is often a misunderstanding of what tests can be done on the collected evidence. For example, if the narrative request is “were the cartridge cases and projectiles fired from the same gun?“, unless a gun was submitted for comparison, the test would not be possible. Third, the scope of the narrative request may not fit within resources and time constraints. For example, in our case, assume that subsequent to scene processing, a suspect was developed and a warrant was issued for the suspect's home where 20 firearms were discovered and collected. It may not make sense to request “test all guns”. Instead, it may be more fruitful for analysts to use their knowledge and tools such as the National Integrated Ballistics Information Network (NIBIN) to determine which guns could have been used with those cartridge cases, and thus triage the group of guns to test. Note that this selection is not based on any theory of the case and thus is not subject to selection bias; rather it is based on the forensic scientist's expertise. Fourth, there is an inevitable delay between the submission of the test request and the analysis being done. In this delay, the theory of the crime will likely evolve as additional detective work is performed. This means that in many cases, there is a revision of the test request that updates the question that is being asked. Both the original request and the modification are recorded in the case history. Depending upon the length of the delay, at the point at which the lab is to begin testing, there may be a subsequent consultation to ensure that the requestor understands the tests available, to receive a verification of which tests are to be done, and to ensure that the questions those tests are intended to answer are still the ones that the requestor wants to pursue.

Finally, an additional communication issue with the narrative requests is prioritization. For example, consider in our case the analysis of biological evidence. Assume that the narrative request is “perform DNA analysis to develop suspect information and enter into CODIS” and that the items selected for testing are all 10 swabs of apparent blood and the sexual assault kit which includes 3 samples. Rather than test for everything, there needs to be a prioritization of what to test and when, based on the underlying questions being asked. For example, if the question is “is there male DNA in the sexual assault kit?” that would dictate prioritization of those samples. In contrast, if the question was “is any of the blood foreign to the victim?” a portion of the blood swabs would be tested first. If the crime scene investigators and the analysts doing the processing are different people, prioritization may also emerge from their communication, either in person or by the lab consulting the crime scene photos and notations. For example, the lab might want to start with a swab that could be indicative of the suspect based on an analysis of the crime scene – for example, if the swab was taken at a location away from the victim and there was no evidence in favor of the victim changing location. Note that this prioritization does not necessarily involve a theory of the case, but is compatible with the crime scene investigator's designation of evidence that may have forensic value.

All together, these communicative challenges reveal that the request for tests is not a simple one-way communication, but that there must be interaction between the detective and the laboratory that maintains the withholding of the theory of the case from the lab. The requestor must have a basic understanding of how each requested discipline can potentially help answer certain investigative questions as well as those questions that cannot be answered with forensic testing. This understanding can only be achieved through education and training of the detectives and analysts.

It is also important to emphasize that the analyst does not know how the evidence conclusions will be understood or utilized as they do not typically know, nor should they know, the (entirety of the) working theory of the case. This does not require analysts to work in complete isolation of the case information, but it does highlight the need to provide the appropriate case information to analysts for logical and balanced evidence evaluation [12] while also ensuring that the order of the release of information through procedures such as linear sequential unmasking protects the analyst from cognitive bias and contamination as much as possible (e.g. [[13], [14], [15]]).

Best Practices: We recommend providing training for detectives who submit analysis request forms to develop their skills in identifying the questions they are trying to answer with any given evidence. These training efforts should include explanations of the types of analyses that are available for given evidence and the possible questions that could be answered for any given analysis. It is also important to recognize that as analysts prioritize which tests to run, they are likely working from their own theory of the case. This makes it critical that the requests for analysis be received in the form of a question that the analyst is trying to answer that is supplied by the detective, and not by the analyst according to his/her own theory.

We also recommend including a case manager for complex cases with high volumes of evidence and requests across forensic disciplines. The case manager would interface with the detective and be privy to the evolving theory of the case. The case manager would be able to advise on different types of tests that could answer different types of questions given the set of evidence. This would be particularly useful for large cases for prioritizing the analyses that may lead to the most probative conclusions. The case manager could also serve as a shield to protect the case analysts from unnecessary task-irrelevant and biasing information (see Spellman et al., [10]).

Section 3: The “analyze, conclude and report” actions by the analyst

The actions of “analyze, conclude and report” explored in this section are inextricably linked to the nature of the question in the formulation of the analysis request, as discussed in Section 2. These actions culminate in the translation of the result, often numeric, into a conclusion, often linguistic. This translation is complicated, as it involves converting continuous information, such as probabilities that represent a range of possible numeric outcomes, into linguistic labels that represent categorical information. Adding to this complication is the fact that different forensic disciplines use different linguistic expressions to convey their conclusion, as we further discuss below.

Analysis

Analysis encompasses testing, verification (discipline-specific), and administrative/technical review. The focus in this article is not on how the analyst reaches a conclusion, but on the communications that occur throughout these processes with the laboratory.

Testing. Quality manuals and sectional standard operating procedures within each laboratory outline the standards of documentation required during analysis, including specifications for photographs, charts, and notes. As a general rule, the documentation should be of a quality such that another analyst could review the case file and evidence, understand which decisions were made and the reasoning behind those decisions, and be able to reproduce the steps that were followed.

The outcome of the analysis is a decision about the evidence with respect to the question being asked. Across disciplines, at the most general level, the decision categories are “yes,” “no,” and “I don't know,” as captured within the Pebbles on a Scale metaphor. However, the central communicative challenge is in how these decision categories get translated into very specific terms based on the underlying science in a given discipline. Table 1 represents the way in which these categories are translated for a sample of disciplines. It can be confusing to the recipient of laboratory reports when reports from different disciplines use different language to convey the same essential message. In addition, some disciplines convey statistical or other supplemental information along a gradient or continuum that would change the size of the pebble in the Pebbles on a Scale metaphor, whereas others convey only categorical information that is represented by the color of the pebble.

Table 1

The articulation language used to convey “yes,“, “no,” and “I don't know” for various forensic disciplines. This list is not meant to be exhaustive but rather to demonstrate the potential variation of terms between and within forensic disciplines.

	Results and articulation language
Discipline	Yes	No	I don't know
Seized drugs	Present, confirmation, or determined to contain	Not present or does not contain	Inconclusive
DNA analysis (multiple rows = variation across laboratories)	Included	Excluded	Inconclusive or uninformative
	Included	Cannot be included
	Cannot be excluded	Excluded
Firearms	Identified	Eliminated	Inconclusive
Latent print (multiple rows = variation across laboratories)	Identified	Excluded	Inconclusive
	Associated	Excluded
Bloodstain pattern analysis/pattern classification	Yes (could be)	No (eliminated)	Undetermined
Fire investigation (ignition and source)	Included	Excluded	Undetermined

To illustrate the entries in Table 1, we next offer a discussion of the typical questions associated with these disciplines and how the conclusion categories of “yes” “no” and “I don't know” are defined and translated into linguistic labels.

Seized Drugs. The most common underlying question is whether a substance is present and the decision is made relative to some threshold (defined by State and/or Federal statutes). Thus the “yes” decision category includes “confirmation” that a substance was present or contained in the item examined, whereas the “no” decision category includes “no controlled substance was identified.” With respect to our example, if the test request was “what is the white powder?” and the white powder is the evidence to be tested, the decision may be “determined to contain Cocaine.” If in testing there is consumption of the substance such that it can no longer be tested, that is noted. Though inconclusive (“I don't know”) determinations are possible, they are not typical in seized drug analyses.

DNA. The most basic DNA question is “whose DNA is present?” More specific questions are whether the DNA is foreign to the victim or whether the DNA is from the suspect. The category of “yes” decisions is variously expressed across laboratories and often encompasses, but is not limited to, the expressions “included” or “cannot be excluded.” Similarly, the category of “no” decisions is variously expressed as “excluded” and “cannot be included.” The “I don't know” category is expressed as being “inconclusive” or “uninformative.” The differences between these categorical terms correspond to laboratory preferences and may be tied to the technology utilized. This is understandable, but in terms of communication with detectives, such variability may be particularly confusing, especially as they are integrating across multiple sets of results and evidence. Oftentimes for further expression of results, a statistical analysis method (e.g., a likelihood ratio), is utilized that can be interpreted as indicating the strength of support that the evidence provides for a “yes” decision (person of interest is a contributor) versus a “no” decision (unknown person is a contributor). Further confusion can arise from “yes” decisions with the variable application of verbal equivalent terms (e.g., findings provide very strong support for a proposition over the alternative for LR ≥ 1 million).

Firearms. The most common questions are whether the evidence was fired from the same firearm, and whether the evidence was from a specific firearm. The category of “yes” is expressed as “identification” whereas the “no” category is expressed as “elimination.” The “I don't know” category utilizes the term “inconclusive.”

Latent Prints. The most common underlying question is to determine the identity of the donor of an unknown scene impression. The category of “yes” decisions is expressed as “identified” or “associated,” with the term determined by laboratory preference. The “no” category is typically “excluded” and the “I don't know” category is “inconclusive.”

Bloodstain Pattern Analysis. The most common underlying question is what mechanism(s) could have created the observed pattern. The category of “yes” decisions is taken to mean “could be.” The category of “no” decisions is taken to mean “eliminated.” The “I don't know” category is expressed as being “undetermined.” A further expression of results would be to provide a list of potential mechanisms for pattern creation.

Fire Investigation. Fire investigation analysis can be broken down into questions about the ignition location and the source. Of most interest here are requests related to the source. The most common underlying question is whether a material could be the source of the ignition. The category of “yes” decisions are expressed as “included.” The category of “no” decisions are expressed as “excluded.” The “I don't know” category is expressed as being “undetermined.” A further expression of results would be the addition of qualifiers such as “this item is probably the source of the ignition."

It is clear from Table 1 that even though all disciplines can reduce their analyses to a conclusion in “yes,” “no,” or “I don't know” categories, the conclusions are expressed differently by the different disciplines. For parties who work with the reports that present these conclusions (e.g., detective or later lawyer, judge, jury), this change in terminology across type of evidence and report can be very confusing, and may unintentionally be taken as an index of strength or confidence [[16], [17]]. Moreover, if the Pebbles on a Scale metaphor illustrated in Fig. 2 is an approximation to how the detective or downstream users consider the conclusions, they will ultimately need to translate these back into the “yes,” “no,” and “I don't know” categories in order to combine this evidence with other evidence in the course of assessing a question related to the theory of the case.

Verification. Errors may arise during a forensic analysis. To reduce the chance that such errors affect how a conclusion is determined, in some disciplines such as firearms and latent prints, a verification process is used, in which some portion of the tests that were run are repeated. Here there is variability as well, with some labs completing 100% verification of all results, whereas others only complete a 100% verification of identifications, and still others complete 100% of identifications and an additional percentage of elimination and inconclusive results. Some labs also have a percentage of cases that are completely reworked as part of the quality system. Regardless of this variation, for verification to reduce errors, both the primary analyst and the verifier need to view the process as essential to quality control, and they must communicate professionally and efficiently to resolve any questions or concerns.

Within the comparative (pattern evidence) analysis field, verification may proceed in a more interactive fashion. For example, with firearms examination, it is not uncommon for a verifier to approach the primary analyst with an area of agreement between two pieces of evidence previously determined to be inconclusive, and ask the primary to reexamine a particular area. The opposite scenario is also possible, in which the verifier does not see the agreement that the primary analyst saw between two items of evidence, and may ask for the primary analyst to show the verifier the area, or adjust the lighting on the microscope. The verifier should be careful to not use any real or perceived influence over the other examiner to influence them. Two possible areas of bias may emerge through a more experienced examiner influencing a newer examiner, or a more experienced examiner under-estimating a new examiner. The laboratory needs to foster an environment of inclusion and respect, regardless of experience level. The culture in each laboratory sets the tone for how disagreements are handled and influences the quality of the work product.

Best Practices. We recommend including blind verification where the identities of the analysts are not shared (see also Spellman et al., [10]). Each analyst provides full documentation for their opinion. This ensures that each receives equal weight. Any differences in outcome are then bridged during administrative and technical review. We also recommend that management provides consistent messaging about the need to acknowledge bias, and the importance of continuous training and development in communication and understanding bias to combat these pitfalls. Protections should be put in place during verification that ensure collaborative and unbiased communication between primary analysts and verifiers, particularly when these parties have different levels of experience. It is also important to ensure that each independent analysis, if competently done, is given equal weight, independent of the personal characteristics of the analyst. Labs should consider instituting a double-blind verification process, such that the identities of the two analysts are not known to either party. It is also helpful to make sure that the laboratory culture recognizes that error is inevitable given that analyses are being done by humans. Strengthening the understanding of the value of verification, and using disagreements as an opportunity to further assess processes will reduce the likelihood of error to the extent possible. We further recommend establishing a climate in which errors can serve as feedback thereby enhancing learning. Indeed (Eldridge et al., [18]), illustrates the power of learning from negative examples.

Administrative and Technical Review. After verification the case moves forward to administrative and technical review. Laboratories vary in how this review is accomplished. For example, one may have three qualified analysts on every case (one primary, one verifier, and one conducting the administrative and technical review). Another may not have the staffing to achieve this, and the verification and technical review may both be done by the verifier, and the administrative review may be done by an administrative assistant or manager.

The administrative and technical review, when conducted by a qualified analyst, is an additional opportunity to ensure that any errors that may have occurred are caught before the report is finalized. In addition to reviewing the report for the complete chain of custody as well as for correct grammar and spelling, a qualified analyst is expected to inform the primary analyst and verifier of any technical issues that are identified while reviewing the case file. The communication that occurs during administrative and technical review includes: ensuring that standard operating procedures were followed, that documentation is sufficient to support conclusions and opinions, that any required math has been rechecked, that processes that were followed are clearly described, and that the conclusions reached between the primary analyst and the verifier are in agreement (if applicable).

Best Practices. We recommend that units offer training as needed on conflict resolution and emotional intelligence, both of which are critical for open communication among analysts, verifiers, and reviewers, and hold frequent section meetings to discuss potential issues, present case studies, or discuss complex cases so examiners can learn from each other.

Reporting

The results are initially provided in a report that typically includes the following information, although specific contents may vary across laboratories:

Items examined

Results or interpretations of testing

Conclusions regarding the evidence

Methodology section

Evidence disposition

The focus here is on the information included in the report that conveys the conclusions drawn from each analysis. The detective uses these conclusions in light of other evidence and ensuing conclusions to further develop or refine the theory of the case, as discussed further in Section 4.

Best Practices: We recommend consistency among examiners in how work is reported – especially in multi-laboratory or large lab systems. It is important to make it easy for detectives to find the conclusions that are drawn with respect to a particular question about a given piece of evidence. Consider creating a cover sheet, attaching appendices that provide in depth descriptions of conclusions, and/or utilizing a “Summary” section at the beginning of the report that better enable the detective to assess how a given piece of evidence and the conclusions that are offered fit into the evolving theory of the case. Additionally, laboratories may consider soliciting feedback about how detectives understand the reporting outcomes and providing consistent training to detectives for how to read and understand forensic reports.

Section 4: The “assess conclusions” action by the detective

Forensic analyses in the laboratory produce conclusions with respect to specific questions asked about specific pieces of evidence. These conclusions are then reported to the detective who must assess the implications with respect to the evolving theory of the case. This is illustrated in the expanded version of the Pebbles on a Scale metaphor shown in Fig. 3. The analyst selects a color (green, red, yellow) to correspond to their conclusion (yes, no, I don't know) on the basis of the results of the given analysis. The analyst also selects a size to correspond to the strength of the conclusion. This pebble then gets passed as the answer to a given question to the detective.

Fig. 3

A further refined version of the Pebbles on a Scale metaphor that illustrates the significance of the size of the pebble, as determined by the cultural perception of the discipline and the probative value with respect to the evolving theory of the case. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

As shown in Fig. 3, the size of the pebble may be further altered by the detective. For instance, the detective may have a preconception or understanding of the discipline that reflects the societal confidence in a decision based on evidence from a particular discipline. For example, a pebble for a DNA analysis might be larger than a pebble for a bitemark analysis, indicating a higher degree of confidence in the decision that is drawn on the basis of these analyses. The detective may further adjust the size of the pebble depending upon the evolving theory of the case. For example, if decisions drawn from bitemark evidence are central to the theory of the case, the pebble size may be increased to reflect its probative value. The detective must then decide which pan to place the pebble in – “yes”, “no”, or “I don't know” at the base.

This metaphor can also be thought of as fluid and changing as the detective updates the evolving theory of the case, and at different moments in time consults the conclusions of the forensic analyses with respect to particular questions – for example, should I pursue suspect X? Is this incident a homicide or a suicide? Were drugs involved in the incident? and so on. The accumulation of pebbles reflects the accumulation of evidence, and as the pebbles cluster on a given side (pan), the scale tips, suggesting support for the evolving theory of the case or support for an alternative theory.

The impact of the weight of the evidence, which corresponds to the size of the pebble, will also depend on the prior position of the pans [19]. The structure of the Pebbles on a Scale metaphor highlights the way that information is integrated across different sources of evidence. Within the metaphor, each pebble is assumed to be independent and is used to update the theory of the case. However, not all evidence is independent, and not all evidence is presented in a way that can easily be integrated with the rest of the facts of the case. Likelihood ratios within DNA analyses are an example, because the assigned likelihood ratio expresses the relative comparison of the probability of observations given two hypotheses. This numerical ratio effectively provides the amount of updating (or “support”) that the evidence provides to one scenario versus an alternative that can contribute to the overall theory of the case.

To illustrate, consider Table 2, which applies particular questions with respect to particular evidence from our example case and illustrates how each conclusion may be used by the detective. Each row in Table 2 corresponds to a question that the detective may ask about a given piece of evidence. The columns represent the type of evidence, the underlying question, the decision arrived at by the analyst and stated in the report, how that decision is interpreted with respect to the question being asked, and the inferences that can then be drawn and applied to a given theory of the case. The last three columns illustrate these processes as applied to the pebbles within the Pebbles on a Scale metaphor. Specifically, the first of these columns shows the size based on the type of evidence and the discipline of the analysis; the second shows the size with respect to its relevance for the theory of the case, and the third indicates where the pebble would be placed on the scales (on the yes side, on the no side, or at the base).

Table 2

The application of particular questions with respect to particular evidence from our example case and an illustration of the conclusions that may be used by detective. The red, yellow, and green ball colors correspond to the analyst’s conclusions of “no”, “I don’t know”, and “yes”, respectively. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

The Pebbles on a Scale metaphor reflects the evidence in relation to a given theory of the case at a specific moment in time. However, often a detective is pursuing several possible theories of the case, and these change as new evidence comes to light. This can be accommodated if one thinks about the Pebbles on a Scale metaphor dynamically over time. For example, if across all the evidence there is weaker support for the evolving theory and stronger support for an alternative theory, then the detective would effectively take all of the pebbles off of the scale, and start over again with a different theory of the case, placing each pebble anew on the scales with respect to the new theory.

To illustrate, a green pebble means that the analyst concluded that the evidence provided an affirmative answer to the question being posed, such as “yes” to the question “is the green leafy substance found at the scene marijuana ?” For the detective, this conclusion may not have much bearing on the overarching investigative question – it may only establish that there is marijuana at the scene, but not possession nor whether this substance was involved in the incident. Thus, it would be placed at the base. However, this evidence may be relevant for a different theory of the case – and if so, in conjunction with a different question, it may subsequently be placed on the “yes” or “no” pan of the scale.

Section 5: Closing thoughts

Crime laboratories vary along many dimensions, including staff size, the types of services that are provided, funding sources, the various systems in use, whether they are public or private, and governance structure (linked to a law enforcement agency or independent). The commonality among them is that they provide results and conclusions/opinions about evidence with respect to particular questions to law enforcement, lawyers, and judges working within the criminal justice system which are utilized to make decisions about cases. Improved efficiency in communications within a laboratory and between laboratories and detectives may reduce the burden on laboratories, particularly those that are limited in funding and need additional staff. The partnership between forensic scientists and detectives is a critical component of crime investigation.

In support of this collaboration and with an eye toward improving these communications, this article has offered the Pebbles on a Scale metaphor that describes how information is communicated through the time course of an investigation, focusing on processes involving the interaction between analysts and detectives around requests for analyses, reporting conclusions, and assessing these conclusions with respect to an evolving theory of the case. This metaphor has provided a graphical way of understanding how scientific information is shared and understood at the current time; it is our hope that it will also serve to illuminate areas where improvements can be made in the future. As with all metaphors, the Pebbles on a Scale metaphor is a concrete representation of an abstract process that is necessarily incomplete. We do not intend the metaphor to be interpreted as an explanatory model. Rather, we hope that it serves to highlight some of the idiosyncrasies of communication between detectives and forensic analysts and to reveal how the information is treated differently by the parties. For example, although the same results are shared between the two, how the evidence is considered by the forensic analyst is different from how it is considered by the detective, due to differences in their perspectives, goals, and the information that they each have available. Additionally, each pebble is an independent event for the forensic scientist, whereas it is part of an integrated whole for the detective (the totality of information available on the case to shape their evolving theory of the case). The reporting language itself can present challenges as the currently prevalent practice of presenting conclusions as categorical (“Yes”, “No”, or “I don't know”) both loses nuance that may be vital to decision-making and can be confusing to the detective when these three concepts are reported using different language from discipline to discipline.

Communication of scientific findings is not trivial, particularly when decisions regarding a person's liberty need to be made in light of those findings. The Pebbles on a Scale metaphor can help to illuminate how this information is shared, and how it perceived and used differently by the different parties who interact with it. It may also suggest new approaches, such as teams of analysts and detectives who work together to better understand how findings across different pieces of evidence should be integrated. Right now, this is left solely up to the detectives, using coarse categorical labels that strip away important nuance related to confidence in the conclusion, confidence in the forensic discipline, and the probative value – as reflected in the color and size of the pebbles. Given that such teamwork would operate after each analysis has been completed, biasing information will not be introduced to the scientists. Rather, bringing the expertise of the forensic scientists to bear as detectives think about the volume of evidence and how and whether it fits together in support of a theory may be a critical stage to add to investigations.

Funding statement

This work was supported, in part, by the Forensic Technology Center of Excellence (Award 2016-MU-BX-K110), awarded by the , , U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the author(s) and do not necessarily reflect those of the Department of Justice.