Cognitive bias in forensic pathology decisions.

Forensic pathologists' decisions are critical in police investigations and court proceedings as they determine whether an unnatural death of a young child was an accident or homicide. Does cognitive bias affect forensic pathologists' decision-making? To address this question, we examined all death certificates issued during a 10-year period in the State of Nevada in the United States for children under the age of six. We also conducted an experiment with 133 forensic pathologists in which we tested whether knowledge of irrelevant non-medical information that should have no bearing on forensic pathologists' decisions influenced their manner of death determinations. The dataset of death certificates indicated that forensic pathologists were more likely to rule "homicide" rather than "accident" for deaths of Black children relative to White children. This may arise because the base-rate expectation creates an a priori cognitive bias to rule that Black children died as a result of homicide, which then perpetuates itself. Corroborating this explanation, the experimental data with the 133 forensic pathologists exhibited biased decisions when given identical medical information but different irrelevant non-medical information about the race of the child and who was the caregiver who brought them to the hospital. These findings together demonstrate how extraneous information can result in cognitive bias in forensic pathology decision-making.

Two data sets revealing cognitive bias in forensic pathologists’ decisions about manner of death.

Death certificate data show racial disparity in judging child deaths as homicide vs. accidental.

Experimental data reveal forensic pathology contextual bias by irrelevant non‐medical information.

Both data sets show extraneous information, for example, race, cognitively biasing forensic pathologists.

Cognitively informed training and policies must be in place to minimize forensic pathology biases.

INTRODUCTION

The biases that may have led to unjustified deaths by police actions have received much attention and have been extensively discussed (e.g., [1, 2, 3]), but are the deceased also subject to bias after their death by scientists examining their bodies? Are scientists biased by race or other irrelevant contextual information [4]?

Forensic pathologists play a critical part in administering justice because of their role in criminal investigations and court proceedings, as they determine whether the manner of death was homicide vs. something else (e.g., accidental or suicide). Despite bias plaguing many forensic domains [5], forensic scientists often deny that bias can impact their decisions (the bias blind spot [6], and the fallacies of expert immunity and illusion of control [7]).

Especially acute has been the resistance to adopt policies that minimize bias, which has "been met with stern resistance from the forensic pathology community" ([8] p. 261). However, cognitive bias in forensic science decisions has led to erroneous conclusions with devastating consequences (e.g., [9, 10]).

Cognitive biases are not intentional discriminatory biases [11], and they can emerge from various sources [7]. They can emerge from the specific case at hand and how it was examined, from the specific person conducting the examination and organizational factors, as well as from human nature ‐‐see sources of bias in Figure 1. Indeed, research has demonstrated the forensic confirmation bias ("the class of effects through which an individual's pre‐existing beliefs, expectations, motives, and situational context influence the collection, perception, and interpretation of evidence during the course of a criminal case" [12] p. 45). Such cognitive bias has been shown to influence DNA mixture interpretations, fingerprint comparisons, toxicology, and other forensic science judgments [7, 13].

FIGURE 1

Eight sources of bias that may impact observations and conclusions, even by expert scientists. They are organized in a taxonomy with three categories: starting at the top, with sources relating to the specific case at hand (Category A); sources relating to the specific person conducting the analysis (Category B); and at the bottom, sources that relate to human nature (Category C). Taken from [7] [Color figure can be viewed at wileyonlinelibrary.com]

Documented biases emerge, for example, from contextual influences, such as ‘an eyewitness identified the suspect’, or ‘the suspect confessed.’ Past research has revealed racial bias in medical diagnosis and treatment of living patients (e.g., [14, 15, 16]), including children (e.g., [17]) —but no research has explored biases in manner of death determinations among forensic pathologists, the principle medical arm of the death investigation system.

To examine bias in manner of death decisions, the obvious data are death certificates, which document the decisions made in real deaths by medically or legally qualified experts (depending on the case and jurisdiction, the manner of death may be determined by medical examiners, forensic pathologists, or coroners who frequently rely on the opinion of the forensic pathologist in determining the manner of death —in this paper we use them interchangeably).

We therefore examined all death certificates of children under the age of six issued in the State of Nevada over a ten year period, comparing the recorded manners of death of "accident" vs. "homicide" across White and Black children.

Although death certificates reflect real death cases, each case is different, and the ground truth of the real manner of death is unknown. Hence, to complement these archival data, we also conducted an experiment in which we presented 133 forensic pathologists with identical medical information about a child's death, but randomly assigned them with differing medically irrelevant contextual information, and asked them to determine the manner of death. It is important to note that non‐medical contextual information may not only be considered when determining the manner of death, but in fact, must be considered. By its very nature of being a circumstantial explanation, the manner of death is dependent on the investigation, which provides the circumstances (and as necessary, the broader background, e.g., medical history) surrounding the death.

The issue of cognitive bias is not trying to ignore or deny the need for circumstances, that is, contextual information, to determine a manner of death, but rather identifying and recognizing what contextual information is irrelevant to that decision‐making process. The specific contextual information chosen in the experimental study was purposely (and properly) designed to be always irrelevant to determining the manner of death, allowing the data to reflect whether cognitive bias impacts the decisions and conclusions of the forensic pathologists.

DEATH CERTIFICATE DATA

We examined all death certificates of children under the age of six from a dataset of all death certificates issued in the state of Nevada between 2009 and 2019. Our focus was on Black and White children in which a qualified expert concluded that the death was unnatural, that is, neither "natural" nor "undetermined" ("suicide" was inapplicable to young children), thus focusing on "accident" and "homicide" determinations (N = 1024).

Overall, the percentage of unnatural child deaths (i.e., that medical examiners categorized as "accident" or "homicide," combined) was comparable across White and Black children: 23.3% vs. 23.5%, respectively. However, when separating "homicide" vs. "accident," a very different picture emerged: coroners more often attributed death to homicide when the child was Black (8.5%) vs. White (5.6%) and more often ruled the death accidental when the child was White (17.7%) vs. Black (15.0%). See Figure 2, X 2(1) = 4.02, p < .05, OR = 1.81 [95% CI: 1.01, 3.25].

FIGURE 2

Forensic pathologists were more likely to attribute the deaths of Black children to homicide, relative to White children; whereas the deaths of White children, relative to black, were more likely deemed accidental [Color figure can be viewed at wileyonlinelibrary.com]

Stated otherwise, the data revealed that forensic pathologists ruled a White child's unnatural death as "homicide" 24% of the time, vs. as "accident" 76% of the time, yielding an approximate ratio of 1:3. In contrast, forensic pathologists ruled a Black child's unnatural death as "homicide" 36% of the time, vs. as "accident" 64% of the time, resulting in a ratio of about 1:2.

We must be careful in drawing conclusions about bias from these archival data, especially given that the ground truth of how these children actually died is unknown. For example, it is possible that Black children die from homicide more often than White children.

EXPERIMENTAL DATA

To complement the death certificate data, we conducted an experiment with a sample of qualified forensic pathologists, who examined a hypothetical death case of a young child, with identical medical information but different extraneous medically irrelevant contextual information (the child was either Black and the mother's boyfriend was the caretaker, or the child was White and the grandmother was the caretaker).

METHOD

Participants

All participants (N = 133) were American Board of Pathology Board‐certified members of the National Association of Medical Examiners (NAME, their premier professional organization, founded in 1966). The participants consisted of 50 females and 79 males (4 left this information blank), and 10 were under the age of 35 years, 30 were between the ages of 35–45, 39 were between 46–55, 28 between 56–65, 17 between 66–75, and 6 over 75 (3 left age information blank). The participants were recruited through an email sent to the NAME mailing list of 713 pathologists (18.6% response rate).

Materials

Participants read a vignette describing a not straightforward or simple case in which a 3.5‐year‐old child was presented to an Emergency Department with diminished vital signs and who died shortly after arrival. In the vignette, the caretaker described finding the toddler unresponsive on the floor of a living room. Postmortem examination determined that the toddler had a skull fracture and subarachnoid hemorrhage of the brain. The vignette further briefly described scene and ancillary investigation findings, as well as the results of the medical examination (details about bruising on the head, neck and extremities, as well as the fractures and brain injury).

By random assignment, each pathologist read one of two vignettes, which were identical apart from two pieces of information: some were told that the child was African‐American and that the caretaker was the mother's boyfriend (henceforth referred to as the "Black condition", n = 65), whereas the other pathologists were told that that child was White and that the caretaker was the child's grandmother (henceforth referred to as the "White condition", n = 68). To be consistent with typical medical information, the race of the child was stated, but the race of the caretaker was not explicitly stated (the caretakers explicitly differed only in their relations to the child as well as their sex; implicitly they may have differed in their age and race).

Procedure

Medical examiners who responded to the recruitment email were randomly assigned to either the Black or White condition. They were given the information and asked to examine the case and to determine the manner of death—they were not directed to do it differently than they do ordinarily. The standard death certificate format mandates that the manner of death be determined as either "natural," "accident," "suicide," "homicide," or "undetermined." Because of the autopsy findings and circumstances described in the case, "natural" and "suicide" were not viable options, leaving realistically only the options of "accident" or "homicide", or that the manner of death was "undetermined."

RESULTS

None of the participants determined the manner of death as "natural" or "suicide," and 78 participants ruled the manner of death as "undetermined." The "undetermined" decisions were comparable across both groups of contextual information (38 vs. 40, for the Black vs. White conditions, respectively).

Of interest were the remaining 55 participants who reached a conclusive determination about the manner of death, with 23 ruling it an "accident" and 32 ruling it a "homicide." The medically irrelevant information had a significant impact on manner of death determinations, revealing a clear contextual effect. In the Black condition, the pathologists were about five times more likely to rule the death as a "homicide" rather than an "accident" (35.4% vs. 6.2%), but in the White condition, the results were the opposite: The pathologists ruled the death as a "homicide" only about half as often as they ruled it an "accident" (13.2% vs. 27.9%). See Figure 3, X 2(1) = 15.89, p < .0001, OR = 12.14 [95% CI: 3.23, 45.68].

FIGURE 3

With identical medical findings, the proportion of forensic pathologists determining that the manner of death was an "accident" (left panel) vs. a "homicide" (right panel). White bars are for White children with the grandmother as a caretaker; Black bars are for Black children with the mother's boyfriend as a caretaker. When the forensic pathologists could not reach a decision (an "accident" or "homicide") with confidence, they concluded that the manner of death was "undetermined" [Color figure can be viewed at wileyonlinelibrary.com]

As presented in Figure 3, with all medical data identical, forensic pathologists were biased in their decisions. The data do not allow us to ascertain whether they were biased by the race of the child or/and characteristics of the caretaker. The important finding is that their decisions were noticeably affected by medically irrelevant contextual information (information that should not have any bearing on the decision). Moreover, their decisions were made with confidence, given that pathologists also had the option to decide that the manner of death was undetermined (indeed, 78 participants did not reach a conclusive manner of death).

Most important is the phenomenon identified in this study, namely demonstrating that biases by medically irrelevant contextual information do affect the conclusions reached by medical examiners. The degree and the detailed nature of these biasing effects require further research, but establishing biases in forensic pathology decision‐making—the first study to do so—is not diminished by the potential limitation of not knowing which specific irrelevant information biased them (the race of the child, or/and the nature of the caretaker). Also, one must remember that the experimental study is complemented and corroborated by the data from the death certificates. These, together, are discussed below.

DISCUSSION

A dataset of death certificates revealed that Black children, relative to White children, were more often judged as victims of homicides rather than accidents. We then complemented these archival data with an experiment that directly manipulated medically irrelevant contextual information, and found that forensic pathologists’ decisions were dramatically influenced and biased by such information.

Cognitive bias is not about denying the need for certain relevant contextual information, but rather about the need to identify and recognize what contextual information is relevant to the decision‐making process. The race of the child and who was the caretaker were chosen because they are irrelevant to determining the manner of death in our experimental study, thus allowing the experimental data to reveal whether or not cognitive bias impacts forensic pathologists.

The experimental data, along with the death certificate data, taken together, show that even highly trained professional scientists can be biased in their decisions. When considering the possibility of bias, it is important to consider its possible source. Cognitive bias can emerge from factors related to the particular case itself (see Figure 1, Category A)—for example, the race of the child in a given case activates implicit racial bias. However, the bias may not emerge from racial bias per se, but instead originate from the second category of sources of bias, namely base rate (see Figure 1, Category B). That is to say, the forensic pathologists may “learn” over time, regardless of accuracy, that more Black children than White children die as a result of homicide. As a result, the forensic pathologists develop an a priori expectation, high prior odds, that a Black child has died as a result of homicide rather than accident.

With time, the pathologists are thus exposed to more rulings that Black children die from homicide, which strengthens the base‐rate bias and prompts even more "homicide" findings—thereby creating a bias that perpetuates itself, resulting in bias cascade and bias snowball effects [5, 7]. It may even be that Black children were in the past indeed more likely to die from homicide, and that is no longer the case (now, or in the future), but the bias has already established itself, feeding and reinforcing itself—a self‐fulfilling prophecy.

This source of cognitive bias relates to the medical examiner's experience and other factors connected to their work environment (Category B) and is not mutually exclusive to implicit racial bias per se (Category A). These biases—Categories, A, B, and C—can unconsciously impact experts, and therefore, we do not insinuate that forensic pathologists intentionally discriminate and knowingly conduct their work in a racially biased manner.

As per the limitations of our studies, given that the death certificates were from the state of Nevada in the United States, it does not allow us to necessarily generalize our findings to other states in the United States or to other countries. Also, the experimental contextual manipulation included both the race of the child and the nature of the caregiver, not allowing to determine which (or if both) contributed to the biasing effect. Furthermore, the caretakers were not only explicitly different in their relation to the child, but also in their sex (as well as perhaps implied differences about their age and race).

Further research can compare the Nevada death certificates to those in other locations, and examine what factors may impact the level of racial bias in determining the manner of death of children, as well as explore other factors that may bias manner of death decisions. Also, further experimental research can elucidate and tease apart the different cognitive biases at play in forensic pathology (e.g., what we found in our experimental study, the race of the child vs. nature of caregiver, as well as other contextual information). Nevertheless, the critical finding from our experimental study is that it is the first study to examine and demonstrate that medically irrelevant contextual information does bias forensic pathology decision‐making.

Our two data sets are the first step in examining biases in postmortem decision‐making, and they do not answer all the questions. For example, were the forensic pathologists aware that the medically irrelevant contextual information impacted their decisions? And, what, if any, contextual information is required for their medical decision‐making? These are important questions that need further research. Even with answers to the above research questions, there is a need to examine if such cognitive bias impacts how experts perceive information, their testing strategies, or/and how they interpret the findings [7, 18].

The data presented in our study is the first to establish cognitive bias in forensic pathology decisions and it has important implications to policies. A critical step is to develop and implement policies that mitigate bias. In many police and forensic domains efforts are made to have policies to minimize bias, however, such policies have yet to be accepted or implemented in the forensic pathology domain [8].

Hence, our study is critical in demonstrating cognitive bias in forensic pathology and should drive discussions, further research and policies in forensic pathology. Rather than denying the existence of bias, there needs to be a cognitively informed discussion on what (as well as how and when) contextual information should be appropriately used in forensic pathology decision‐making [19]. Our concerns about cognitive bias in forensic pathology decision‐making do not call for removing all contextual information, nor do we claim that context cannot be (and is often) important and relevant.

Furthermore, contextual information is not always a simple dichotomy of relevant vs. non‐relevant, but more of a continuum [8]. In addition, an added complexity is that the level of relevance (or lack thereof) can be decision and case dependent, thus changing from case to case and between different types of decisions. The relevance continuum starts with the most relevant, the body of evidence itself—the deceased. Then there is a variety of contextual information that can play a role in interpreting that body of evidence. Moving along the continuum of relevance is the medical history, then findings from the death scene, and moving to social history and various other leads and findings from the police investigation [8]. Further down along the continuum, moving further and further away from the actual body of evidence, is the wider circumstantial context, such as who brought the deceased to the hospital and the socio‐economic status of the deceased.

Given this continuum:

Forensic pathology decisions and conclusions about the cause of death should be based solely on medically relevant information. The manner of death, being a broader circumstantial explanation for how the cause of death occurred, often relies on some investigative, that is, non‐medical, information. However, that does not mean that all contextual information is task relevant to the assessment and determination of the manner of death. Therefore, it must be recognized that some types of circumstantial information should not be considered in that decision‐making process. The two factors that were manipulated in our experimental study were task irrelevant to the decision on the manner of death, and thus should not have affected the decisions, yet they clearly did affect the conclusions, demonstrating cognitive bias in forensic pathology decision‐making.

Since there is no clear cutoff place along the relevance continuum which applies to all cases, there needs to be a discussion and debate about the (mis)use of contextual information by forensic pathologists. Such a discussion should take into account research and findings from cognitive science, especially those about the nature of decision‐making, sources of bias and contextual influences (e.g., [7, 11]). The forensic pathology community should not deny the existence of cognitive bias and potential effects of contextual information and should consider and explore ways to mitigate and minimize such biases. What is clear is that at some point along the relevant‐irrelevant continuum, the context is so far removed and is so task irrelevant, that it should never be used. The forensic pathology community needs to consider and explore the (mis)use of contextual information in light of our data showing that such extraneous information can cognitively bias forensic pathologists. Hence, the need for debate and to take action regarding bias in forensic pathology.

Medical relevant information should be the primary driver of pathology decisions, supplemented by the less medically relevant when needed and justified. To achieve this, the forensic pathology community must explore and adopt procedures that minimize bias. Procedures such as Linear Sequential Unmasking (LSU), minimize bias in forensic DNA and fingerprinting by optimizing the sequence and timing of exposure to potentially biasing information and making sure to document their potential effects. The forensic pathology community should consider LSU approaches for context management (as well as compartmentalization and case managers) in forensic pathology, which will optimize the use of contextual information in a way that minimizes bias. These context management policies will be there to ensure that the medical information, rather than context, drives the forensic pathology decision‐making [5, 7].

It is ideal when the medical pathology decisions are based primarily (if not totally) on the medical relevant information. As the decision is more and more supplemented by, and dependent on, contextual information that is less and less medical, the medical decision weakens. It is warranted and we fully support when non‐medical information independently corroborates medical decisions (that strengthens the medical decision), but not when the non‐medical information is the major (and implicitly hidden) driver of the conclusions.

There is a difference when non‐medical context and investigative findings are used to suggest seeking medical information, such as which toxicology tests to run, vs. when they implicitly influence and bias a medical decision. Consider, for example, a decision whether a death is suicide or homicide. Does the pathologist need to know that next to the body was an empty bottle of pills? Yes, and they should also know which pills they were, that is not disputed. However, this context is not to be used to determine the manner of death per se, but more to trigger which toxicology tests to run, which can help determine the manner of death. Non‐medical context can be used to trigger such tests, or theories, as well as requests for additional information—that is part of the legitimate, not necessarily biased, pathology inquiry. However, when the non‐medical contextual information and investigative findings cognitively bias the pathologists’ decisions, and do so implicitly without transparency about the factors that actually underpin the decision, that is a major concern.

There should be transparency [20] about if and what contextual information was relied upon, that is, that the pathologists’ reports explicitly state what non‐medical contextual information was known and used in their decisions, and how it has impacted their conclusions. It is important to distinguish and make very clear when decisions are medical (and based on medial information) and when decisions are non‐medical (and are based on contextual and non‐medical information). Most dangerous is when juries and other fact finders take the non‐medical decisions as being medical and medically based decisions. To avoid that non‐medical decisions be unintentionally disguised as medical, the forensic pathology reports and testimony must make it explicitly clear what is medical and what is not medical. The cognitive challenge and problem is that when one is exposed to non‐medical irrelevant context, it can unconsciously impact the medical decisions. Given that there is no awareness of such cognitive bias, pathologists cannot account and report what factors actually influenced their decisions. The point is that there needs to be a discussion about how to deal with these issues, rather than just falling into the bias blind spot [6] and dismissing the entire issue under the false belief and pretense that bias does not exist. Two options to consider are adopting LSU kind of context management tools (that blind some information and optimize the exposure and timing of other information), and detailing, documenting and reporting what (as well as when and by whom) information was given to the forensic pathologist.

It is also important to note that in many domains it is relatively easy to have policies that blind scientists to irrelevant contextual information, but in forensic pathology this is not always feasible (e.g., during an autopsy, the race of the deceased is present). However, context management policies, such as LSU, compartmentalization and proper blind reviews [5, 7, 21] can be developed and considered in forensic pathology (e.g., race‐blind peer reviews in complex and not straightforward cases).

“Acknowledging that bias can influence forensic science experts would be a substantial step toward implementing countermeasures that could greatly improve forensic evidence and the fair administration of justice” [5]. Thus, policies that mandate proper cognitive education and training in forensic pathology are important, so they properly understand that no one is immune to cognitive bias nor that it can be controlled by mere willpower [7], and therefore mitigating actions are required.

The findings about bias in forensic pathology have implications that go beyond police investigations, court proceedings, training, and best practices, as they touch upon wider issues, such as stereotyping and prejudice, unconscious processes, motivation, cognitive and perceptual confirmation biases, and a whole host of psychology and law, and socially important issues [22].