Stressors in forensic organizations: Risks and solutions.

Stressors of many types occur in forensic laboratories, with detrimental effects for individuals, laboratory systems, and casework outcomes. These stressors may be general, affecting the entire laboratory or all cases, or specific, affecting individual examiners or single cases. Stressors affecting individual examiners include: vicarious trauma associated with details of worked cases, nonstandard working hours, fatigue, the monotony of repetitious tasks, fear of errors, and severe backlogs. Policies and laboratory cultures can be put in place to minimize the effects of stressors; however, current forensic organizational responses to these stressors may vary from punitive to collaborative approaches. This article presents several models and case studies that can help inform the creation of positive laboratory policies. A system of discipline-wide centralized error reporting, similar to systems used to reduce fatal mistakes in medicine and aviation, could have the potential to identify areas of concern within forensic science practices.

Introduction

Forensic laboratories operate with a difficult set of constraints. The examiner or analyst makes observations and communicates the results of these observations to a stakeholder such as detective, prosecutor, defense attorney, or jury. In many cases, the examiner will also come to conclusions, such as ‘I identified this impression to the suspect’. In other disciplines such as DNA, the report or testimony will present a likelihood ratio or random match probability, and let the jury integrate this information with the rest of the facts of the case rather than making a posterior conclusion as often happens in the pattern comparison disciplines. In either case, the ground truth is rarely known, and in many disciplines the criterion for different decisions is often ill-defined. This leaves the forensic practitioner in the difficult situation of making decisions or observations without clear guidelines.

Additional stressors include vicsarious trauma from association with violent crime, backlogs, loss of motivation, and ambiguity with respect to appropriate decision thresholds. Included in this discussion are concerns about fitness-for-duty factors such as fatigue and substance abuse. Finally, we consider the evolving trends on how laboratories respond to errors and identify the individual strengths of different examiners through proficiency tests and other measures.

Overview of decision-related stressors

When making decisions in a forensic discipline, how much is enough? We have encountered this question from examiners numerous times, and the question itself reveals the ambiguous nature of the decision structure in many forensic disciplines. Much of the evaluation of forensic evidence relies on human perceptual and decision-making skills. In many cases, this evaluation requires a subjective element, such as the degree of perceived detail in agreement between two fingerprints. Even mixture DNA has elements of subjectivity: is that weak allele real or just stutter? This subjectivity can create conflict within the laboratory environment, in part because of the variations in human expertise and the often ill-defined threshold for pattern-comparison discipline decisions [1]. Examiners receive training in the methodology of evidence evaluation for a specific discipline. However, many of the important acts of threshold setting require an assessment of both the strength of the evidence (i.e. the degree of similarity between two impressions and one source or two different sources), and how likely it is that evidence originated from a different source (i.e. base rates of different features). As we discuss in the section on Ambiguous Decision Thresholds, examiners in many pattern comparison disciplines make decisions, and as a result an examiner must also consider both the prior likelihood of a mated pair being submitted to them, as well as the utility of the different outcomes. For example, how much more important is it to keep an innocent person out of jail verses setting a guilty person free, known as Blackstone's Ratio [2]? Each of these factors is ill-defined in the sense of having relatively little guidance from management or policy makers. Thus, the examiner is asked to set a threshold based on unknown and potentially unknowable sources of information. Because of this, it is often important for examiners to clarify the strength of the evidence so that it is interpreted properly by the consumer such as detective or jury, and for management to continuously discuss whether current threshold placements are appropriate.

If an error is made (and given the constraints outlined above, errors are likely and should be expected by the justice system), how an agency responds to that error plays a critical role in the ability of the examiner to continue their work. Responses to an error include termination, retraining, or an open discussion of the error amongst laboratory personnel. We discuss the tradeoffs of different approaches to errors and suggest some best practices based on current research. One approach might be to shift away from a focus on error rates and eliminating errors, and instead focus on risk management, which is an approach taken by the Federal Aviation Administration (Risk Management Handbook (FAA-H-8083-2)).

Examiners typically work in budget-constrained governmental agencies, and often face backlogs that are solved by offering overtime to employees. This in turn can lead to fatigue, which has been shown to affect both the information acquisition and decision-making of latent print examiners [[3], [4], [5], [6], [7]].

Finally, the subjectivity inherent in the decision thresholds for many of the pattern evidence disciplines, combined with the enormous consequences of an error and the stress of upholding an impossible record of zero errors, understandably contribute to a poor work environment. We offer a set of solutions for consideration that address these concerns.

High reliability organizations

There are several different approaches and characteristics of organizations that can provide lessons for forensic organizations. Within the domains of healthcare, nuclear power, and aviation, a number of organizational characteristics have been identified with highly reliable operations and strong safety records. These High Reliability Organizations (HROs) share a commitment to improvement and error reduction, in part because failure can result in catastrophic damage [8,9]. Below we summarize these characteristics, and suggest how they can be applied to forensic laboratories.

Preoccupation with failure

Personnel in HROs are preoccupied with errors. They are constantly on the alert for problems, and question data and outcomes. Expertise is shared, including ways of anticipating potential problems. Within a forensic environment, ‘close calls’ (errors that were caught but could have been bad) should be shared rather than covered up. The organization should place incentives on recognizing the costs of failures and identifying solutions. The key to successful risk management lies in recognizing the potential impacts of various outcomes and taking a balanced approach to considering the consequences of those outcomes against their likelihood [10].

Reluctance to simplify interpretations

HROs accept that work is complex, with the potential to fail in new and unexpected ways. When an error occurs, some agencies simply place the blame on individual analysts and consider the problem resolved. However, a common characteristic of HROs is to evaluate the source of an error from a systems perspective, carefully considering how different components of a process or system may have contributed to the error. Was there a problem with quality assurance that let an error occur? The manager sets the tone and operations within a laboratory. An organization should respond holistically to an error, even if the temptation is to place blame at the most proximal source. ISO/IEC 17025 accredited laboratories are required to conduct a root cause analysis when completing a corrective action for a nonconformity with laboratory policy or procedure. Root cause analysis should be used to avoid recurrence and mitigate risk to the organization. In a forensic setting, this would require a team approach including a quality specialist, subject matter expert, and a manager [11]. Upon the completion of the root cause analysis, often the blame is not assigned to the individual but rather to the process followed.

Sensitivity to operations

HRO's have a heightened awareness of the state of relevant systems and processes. Forensic laboratories often exist in a political environment. They must fight for funding, especially those whose parent organizations are law enforcement agencies. The decision to spend $150,000 to purchase one analytical instrument for the laboratory is weighed against the multiple police vehicle, service weapons, or other equipment items needed by the same agency for their officers. Problems in the laboratory affect political appointments in the chain above. One example is the 2012 Massachusetts laboratory scandal that arose when it was determined that forensic drug chemist Annie Dookhan was dry-labbing by reporting results without conducting tests on drug samples. At least five officials resigned or were fired over the incident including the Department of Public Health Commissioner taking the blame for the chaos created in the court system with over 20,000 drug charges dismissed or overturned due to the actions of a single analyst in his organization. The stress of such possible failures is mitigated by an HRO's awareness of operations at all levels of the organization and the impact of one section upon another.

Effecting change in a laboratory requires knowing how changes to an organization might impact outcomes, and how change occurs in complex organizational structures. For example, a department may change its policies on which latent impressions are submitted to AFIS, such that only latent prints judged to be of fairly high quality are submitted. This might decrease backlog and reduce the chances of an erroneous identification, but may also leave many cases unsolved. What kind of data can be collected that would help guide this decision? How can we predict an outcome of a change in policies? Who needs to know when such a change is implemented both internally and externally? An HRO should have a clear vision of how it fits into the larger operational and political context in which it operates and have functional relationships in place with the larger organizational structure that can be appealed to quickly when changes occur or are needed.

Commitment to resilience

Organizations show resilience in response to a difficulty or deformity. Resilience is reactive, not predictive. Thus, it is not the kind of capacity that is based on a careful analysis of potential faults, with mitigating solutions pre-positioned to cope. In fact, the resilient organization will invent solutions to unexpected problems on the fly. The second feature is that when an unexpected problem occurs, the elastic—resilient—organization will continue to function normally. It continues to produce desired outcomes despite the problem. Recovering from a laboratory crisis requires pre-planning and a culture that includes a business continuity plan. Forensic Laboratories must prioritize training for many unlikely, but possible, system failures. Solid quality management programs in forensic laboratories provide workflows for when there is a significant failure or nonconformity with logical, objective, and clear steps for reporting, evaluating, and responsibilities clearly spelled out. This could include an emergency, such as a fire in the laboratory, or could involve intentional bad acts by an analyst, such as the dry-labbing discussed previously. No laboratory member should be irreplaceable, and periodic updates to the succession plan should be made. Recovering from an institutional crisis such as fraud or mismanagement is even more difficult, as the laboratory may face an existential threat from policymakers. Redundancy, leadership development, and regular updates to plans are key to long-term resilience [12]. Cross-monitoring and cumulative assessment are essential to create a common understanding of mission.

Deference to expertise

Blindly deferring to expertise has led to many accidents in aviation, such as running out of fuel while troubleshooting a gear problem when the navigator failed to properly warn the Pilot in Command [13]. Many forensic disciplines have subjective decision criteria (thresholds) for conclusions such as Identification, and these thresholds are often passed from one examiner to another via a mentorship training model. This can lead to an asymmetric relationship where the mentee continues to defer to the mentor even once training is complete. Management should be sensitive to power-differential relationships, which can result in an expertise-based groupthink. Within aviation, such deference to captains/pilots by navigators has resulted in planes running out of fuel in an otherwise-salvageable emergency situation. Within forensics, a second concern is confirmation bias. Confirmation bias, a type of cognitive bias, was a factor in the FBI's misidentification of Brandon Mayfield's fingerprint in the 2004 Madrid terrorist train bombing investigation. The analysts assigned to verify the identification made by the initial examiner knew of the previous examiner's conclusion and assumed the initial identification was correct. The initial examiner was a highly respected supervisor with many years of experience [14]. Care should be taken to avoid undue deference to senior laboratory members.

Just Culture

A separate approach to management is the Just Culture organizational principle [15,16], which has a goal of reducing errors in a non-punitive way. As Fig. 1 illustrates, there are three types of behaviors that lead to error, but the institutional response differs depending on the type of behavior.

Fig. 1

Just Culture approaches to errors. This figure is adapted from the work of David Marx and Colleagues (www.JustCulture.org).

Human errors, that are not due to malice or fraud, are to be expected. Such errors should be caught through redundancy and cross-pollination of work products. These errors may result from the person, the processes, or the environment. If errors continue, changes are required. However, the initial response is to console the affected employee, and view the error as resulting from systemic structures in the organization, rather than the individual.

At-risk behaviors result from a choice for behavior that is beyond the tolerated organizational risk. This is addressed by removing incentives for risky behavior (e.g., unrealistic productivity expectations could lead to dry-labbing, or taking shortcuts with procedures or examinations). Additional coaching or training may be necessary to modulate behavior.

Reckless behaviors require a more severe institutional response, either through remedial or disciplinary actions. These are conscious disregard for procedures and lead to unjustifiable risk.

Proactive response to errors

Data collection and analysis are central parts of a proactive response to errors. Organizations should create procedures that track all relevant output of the laboratory, including non-conformity and errors. Management should create an environment where errors are not punitive. During training, the employee should be given opportunities to fail and be provided with feedback that helps them improve and calibrate decisions [17].

Stressors in forensic disciplines

Forensic practitioners face stressors that other disciplines may not share. In this section we review the various stressors, and in a later section discuss ways in which a laboratory management can respond to address or mitigate these stressors. Recent studies have also documented the views of forensic practitioners with respect to feedback and workplace stress [18] and found that examiners often felt strong implicit feedback about expected conclusions. McKay-Davis et al. [19] surveyed sworn and civilian forensic practitioners and found that perceived psychological stress was higher for civilian than sworn personnel.

Vicarious trauma

Exposure to violent crime scenes can lead to coping mechanisms that are mobilized to deal with the trauma, but can lead to a lapse in empathy as a result. In addition, in large-scale incidents such as the 2017 Las Vegas shooting or small-town school shootings, the crime scene investigators are also community members and must deal with the personal cost of the incident to friends or colleagues [20,21]. Although some disciplines are somewhat shielded from primary trauma that crime scene and sexual assault personnel experience, secondary trauma may be experienced through exposure to facts of a given case.

Although forensic examiners may not experience the intensity of exposure that police officers undergo, they are still vulnerable to the negative health outcomes experienced by sworn officers. These include hypervigilance, PTSD, increased risk of suicide, and poor job satisfaction (for review, see Violanti, Charles, et al. [22]). This form of occupational stress has demonstrable physiological effects, including an increase in waking cortisol [23]), which is a stress hormone and could lead to depression, anxiety, or headaches.

The response to such stressors could mirror the recommendations from policing. The literature remains somewhat non-conclusive, but the general suggestions include peer support especially for women and minorities, and a sophisticated approach that relies on humor but still respects the severity and seriousness of the occupation [22]).

Brondolo et al. [24] created a set of mechanisms and strategies for preventing post-traumatic stress syndrome after mass casualty events. This approach addresses the predictability and controllability of the event and the employees' reactions, and addresses the threat to both the individual and the agency. They emphasize the need to anticipate, through planning sessions, a set of responses that can be activated for a major casualty event. In some cases, these stressors are simply part of the job, and Raunick et al. [25] suggest making it clear to first responders that this should be anticipated as part of the employee's duties.

NIJ has produced an Organizational Readiness Toolkit for Vicarious Trauma, which can be found here:

https://vtt.ovc.ojp.gov/what-is-the-vt-org.

The FTCoE has also produced a webinar series that addresses topics in vicarious trauma:

https://forensiccoe.org/webinar/vicarious-trauma-workplace-stress-series/

Testimony/defense of the discipline

Court testimony is a stressor for many practitioners, and can include a difficult or even malicious cross examination from opposing counsel. In addition, defendants can make threats at trial that contribute to a stressful work environment. The American legal system is adversarial in nature, and forensic practitioners have traditionally been associated with the law enforcement/prosecution side. This may have led some examiners to view themselves as protectors of their discipline. In some cases this leads to outcomes that are likely positive in nature; for example, a laboratory manager might refuse to perform gunshot residue on ‘shooter's hands', because the evidence might be misrepresented by the consumer since gunshot residue can disperse and land on a bystander.

Backlogs and loss of motivation

Backlogs, unless extreme, should be viewed as typical in most laboratories. Central to the job of a laboratory manager is determining whether all personnel are contributing to the overall mission of the laboratory. For example, should there be a laboratory-wide backlog or an individual backlog for each examiner? How should backlog measures be used in the pursuit of productivity and culture? One issue is that direct metrics or quotas are counterproductive [26]. Goodhart's law suggests that “When a measure becomes a target, it ceases to be a good measure” [27]. When faced with quotas, examiners will find ways to tune their behavior to the metric, at the cost of a true metric such as providing accurate and timely results to the consumers. It is out of this situation that dry-labbing and other corner-cutting techniques emerge, often out of pure self-preservation. Instead of explicit metrics, managers should determine those factors that motivate each member of the laboratory, which may involve a dialog with the laboratory to determine those factors that lead to large backlogs or dissatisfaction [28]. have discussed how to match laboratory resources with job demands, and what happens when the two are in conflict.

Managers might consider a number of approaches to address productivity in the absence of explicit metrics. Of primary concern should be the service to the consumers and the larger community that depends on impartial applications of justice principles. To affect this, managers should instill a scientific mindset in the laboratory, with an emphasis on failure as detailed in the high reliability organization section of this article. A scientific approach requires constant questioning and experimentation, and laboratory personnel should be given the opportunity to prepare personal presentations that discuss interesting cases or close calls they have had. These presentations can also include the results of experiments such as test firing of similar ammunition, or distortion experiments with fingerprints.

There are other techniques that might help motivate employees. While some personnel are content to come in and work their shift, other employees are looking for upward growth. If there are limited managerial slots available, employees should be offered the possibility of cross-training. For example, some DNA analysts also do crime scene work, which can be a very different environment and a welcome change from the laboratory.

One study investigated the levels of stress and job satisfaction reported by forensic scientists in an effort to aid in the development of policies to reduce work stressors and improve general job performance [29]. Their study showed that scientists who reported higher stress were females who worked more hours than the standard 40 h workweek and had a poor relationship with court officials/actors, minimal management support, and role ambiguities at work. In contrast, those with greater job satisfaction were unmarried and highly educated. They had positive attitudes toward their work, great support from management and few problems regarding their roles in the workplace. Scientists who testified more often were likely to report job satisfaction. Testifying provides opportunities for forensic scientists to share knowledge and professional opinions. They play a direct and public role in a case, which may enhance a scientist's perception of the value of their work.

More information about job satisfaction can be found in a recent NIJ grant final report [21]. This includes information about how female and minority employees have difficulty finding support networks, and how managers might interact with employees to address the responses to vicarious trauma.

Feeling of being watched via audit trails

One particular challenge of forensic examiners is the fact that every action and decision must be documented and is discoverable. This can lead to a feeling of being watched, and can lead to decision fatigue and other responses to stress.

Laboratory personnel should be introduced to these accountability constraints early in training [30]. However, management can take steps to mitigate this stress. First, the laboratory should have explicit policies that acknowledge that errors are the responsibility of the entire laboratory, and that if an error makes it out the door it is a failure of the procedures of the laboratory, not just one examiner. Second, the response to errors should take the form of the approach outlined in the Just Culture graphic. Finally, managers should consider the secondary stress produced by a corrective action process, which can often be extended in time, adding to the long-term negative health consequences. Laboratory managers should work toward making corrective actions a last resort, after it is clear that more collaborative approaches have failed.

Fatigue

The effects of shiftwork on fatigue are well-studied in policing, and many of the findings likely transfer to forensic activities. Sleep deprivation has demonstrable effects on cognitive functioning, which has been traced to neuromarkers recorded via electroencephalograms (EEG). These indicate altered sensory processing [31]. Other studies document substantial changes to cognitive performance which can contribute to depression [5]. Simple tests exist to measure sleepiness [32].

In fingerprint examiners, fatigue has been demonstrated to produce more inconclusive results, more cursory visual search, and reduced visual working memory [3].

Chronic fatigue should be distinguished from temporary fatigue from occasional events such as such as late night watching of the Oscars or the Superbowl. Poor sleep may be aggravated by smartphone device use in the evening hours, and operating systems are now suggesting a more red-shifted or even grayscale display in the evening to reduce exposure to blue light.

Management should develop fatigue management systems as part of a quality management system and should discourage working 4 × 10 schedules (i.e. 10 h days, 4 days a week) with the addition of overtime added to each 10-h shift. Although this temporarily reduces backlogs, it allows upper management to avoid addressing the more fundamental problem of not enough personnel to address the caseload without overburdening existing practitioners in the laboratory. The following resource contains definitions and information about fatigue along with information about fatigue management systems.

http://aifema.ca/wp-content/uploads/2016/09/Dead-Tired-Mike-Harnett.pdf.

Pressure from outside

The concerns raised about the potential influences of contextual information include pressure from outside elements of law enforcement or even members of the public. This suggests the need for a range of buffers to protect practitioners from undue influence. These could include a case manager who interacts with consumers and removes irrelevant contextual information, as well as an ombudsperson to negotiate conflict that may arise between the laboratory and outside agencies. Management must also act as a buffer from the pressure of prosecutors and judges demanding laboratory results during times of high backlogs. Threats of being held in contempt of court if a court ordered deadline is not met should be handled by management as they have the responsibility of requesting funding and operational measures to address the unworked cases and the displeasure of the courts.

Various models to remove the real and/or perceived undue pressure from law enforcement have been proposed for forensic laboratories. Operational independence is the issue and is achievable in some law enforcement parent agencies while other laboratories have become fully independent [33]. In 2005, the Virginia legislature created the Department of Forensic Science as a separate department under the Secretary of Public Safety and increased the laboratory's autonomy. The Houston Forensic Science Center represents a model in which true independence from law enforcement is a foundational principle. This isolates examiners from external pressures while still providing forensic services to law enforcement clients.

Ambiguous Decision Thresholds

Most forensic fields have an element of subjectivity as an inherent part of the decision-making process. However, decision thresholds for some fields are more poorly defined than others. For example, consider three fields: toxicology, single-source DNA, and fingerprint comparisons. Toxicology has statute-mandated thresholds for amount and percentage quantities of banned substances. Single-source DNA has a foundation in science (genetics) for what constitutes the presence of a marker in the DNA and these have been supported by the equipment manufacturers as well as ground-truth testing. Fingerprint examinations are much less defined and this adds an additional stressor: there are no fixed standards such as number of corresponding minutiae and policy-makers have had relatively little input into the placement of the decision thresholds that lead to an identification or exclusion. These thresholds seem to have been developed through a general consensus mechanism among the community of practitioners, along with the contributions of proficiency-testing organizations and black box testing [34]. In general, these tests indicate that examiners have a fairly conservative decision criterion that produces many more erroneous exclusions than erroneous identifications. However, it is unclear whether they align with the expectations of the general public [35].

The reasons for this conservative criterion are unclear, but we can offer some speculation. First, current practice in laboratories can often result in a fairly punitive response to erroneous identifications, and these generate a great deal of negative press for the agency. Second, there is a natural desire to keep innocent persons out of jail, although this may result from an availability heuristic, because unsolved crime must also be considered a negative outcome. Examiners operate with criteria that guard their accuracy, keeping them out of trouble.

Although the subjectivity of fingerprint comparisons may persist even in the face of technological advancements, there are steps that management can take to alleviate the concomitant stress brought on by poorly-defined job criteria. These include an opportunity for the establishment of consensus thresholds through shared examples from casework. Conflict resolution can also contribute to this consensus process, although care must be taken to avoid dominance by strong personalities.

Finally, agencies can attempt to make their criteria as explicit as possible, then train all personnel on how to apply the criteria. These criteria can then be tested, and the agency can provide annual refresher training (recalibration). Finally, managers can use their quality management system to monitor casework decisions to look for procedural drift.

Laboratory responses to stressors

Transparency approaches

A number of different laboratories have worked to become more transparent on not only their standard operating procedures (SOPs), but also overall statistics on outcomes, including corrective actions and nonconformities. An example is the Houston Forensic Science Center (HFSC), which lists incidents and corrective actions, calibration and performance checks, and a summary of blind QC and verification procedures on an eDiscovery website. HFSC also provides turnaround statistics for each month and year, weekly backlog numbers and monthly board meetings on its website www.houstonforensicscience.org.

This transparency is in part the result of problems found in the Houston PD laboratory in the early 2000s, which led to the creation of HFSC. HFSC's transparency has helped rebuild public trust in forensics in Texas. However, a department or agency does not need to wait for a crisis to adopt a transparent approach. Although in many cases it does allow the defense an opportunity to examine corrective actions for an examiner, these are discoverable anyway. By providing it on the front end, the jury will not be surprised by the reported actions. It also allows the examiner an opportunity to anticipate and prepare for questions about prior cases.

Computer-based procedures that enforce policies

One approach to the problem of dry-labbing, as well as data collection and reporting, is to adopt computer-based procedures to enforce policies. Similar approaches exist to ensure compliance with FDA/EPA regulations [36].

These approaches have a number of benefits. First, they provide an audit trail for an instrument. This helps protect the integrity of evidence, as well as access control for who can see this evidence. Although it can be difficult to split some cases, a workflow management software solution can demonstrate that evidence has not been altered, and can help if an employee is challenged on the integrity of their work. In some cases these elements can be built into case management systems.

Centralized error reporting

Part of the High Reliability Organization philosophy is an emphasis on tracking errors and close calls. This is also true in aviation and medicine, where close calls can be an important indicator of more serious problems that have potentially worse consequences if left unaddressed. NASA has been tasked to create anonymous databases for both aviation and medicine, in the form of the Aviation Safety Reporting System and the Patient Safety Reporting System (PSRS). These focus more on what kinds of errors occur and why, with less emphasis on the actual agency or person (due to the anonymity of the site). Confidentiality is a major emphasis, which encourages reporting and allows for overall statistics to be reported and mined for trends. From the PSRS web site:

“The PSRS is available to act as a reporting system that is complementary to a medical facility's internal reporting or other reporting system. Additionally, this can serve as the primary system responsible for capturing close calls, events, patient safety concerns, and suggestions.” [37].

Such a model might also work with forensics, either with a similar model run by NASA, or through state-wide initiatives that aggregated data over individual laboratories. The advantage of the NASA model is that the reports are processed to ensure an element of veracity, and then anonymized through the physical destruction of the submitting information from the paper form. Thus there is nothing that is discoverable. Action is not possible on a case-by-case basis, and this discourages submissions of a malicious nature because there is nothing to gain from a single report.

This approach would take state- or nationwide initiatives, but individual laboratories can approximate this by creating databases that are anonymized and therefore not directly discoverable. A culture of openness and transparency requires a certain level of courage among management, especially given the potential for misuse by defense counsel and prosecution alike. However, the job of a forensic scientist is to serve justice, and whether bad actors have the potential to exploit this should not deter a laboratory from adopting transparent policies.

Risk management vs. reducing errors

The “errors-are-unacceptable” approach that typifies current forensic science should be contrasted with the FAA's focus on risk management. This highlights the fact that aviation is inherently risky and the only way to avoid errors is to not fly. However, forensic examinations are also high stake and not doing them could have severe consequences. The FAA has taken the approach that risks can be categorized by likelihood and severity and characterized as identified vs. unidentified and acceptable vs. unacceptable. Extending this approach to the forensic sciences community could produce actionable processes for mitigating risks and improving laboratory culture. The 2017 update to the ISO/IEC 17025 General Requirements for the competence of testing and calibration laboratories now has a chapter on risk-based thinking [38]. To make this change managers must determine the risks that must be managed in forensics. Examples might include: Whether a sample is of value; will this evidence help answer a question posed (by either side) in this case – or is it already ambiguous; is the sample contaminated; what are the limits of fatigue; how does contextual information affect decision-making; and what are the utility of different possible outcomes such as correct vs erroneous identifications.

Laboratories should try to categorize these risks, and then develop data and performance monitoring plans that include multiple fallbacks. This might allow for individual examiner decision thresholds to be relaxed (for example, processing weaker samples) with the security of separate, independent processing by a different examiner.

In the two case studies presented in call-out boxes, we've illustrated two responses to errors in narrative form. The Las Vegas example addresses what to do after errors; the first part of the HFSC example shows how to organize laboratory culture to prevent some types of errors. Both are reprinted by permission.

Case Study: Houston Forensic Science Center.

The Houston Forensic Science Center has what might be viewed as a model for the implementation of many of the recommendations from this article. Although the main stakeholder is the Houston Police Department, there are layers of buffer between the analysts and the customers/stakeholders. This includes a client services/case management team that receives phone calls from stakeholders, freeing analysts up from distracting phone calls and preventing some bias or pressure from stakeholders. A case manager reviews requests and prompts for more information if necessary. Analysts get only the item and the testing method requested; they have limited background information on the circumstances of the case.

As an example, here is how DNA is organized:

Screeners get the most biasing information. They see the physical evidence (rape kits, clothing, etc.). They are looking for the presence of biological material and taking swabs off of the items of evidence. This is the most emotionally charged of the steps. Rape kits do have some medical detail on the trauma received, age, and genders, etc.

Once the swabs are taken, they are portioned and passed on to the technicians who do the quantification, amplification, etc. They have far less information. They are given the case file with the items to be tested and the methods to use to do the testing. They have very little knowledge of the case, and their work is done in batches, so multiple cases can be processed at the same time. It is all test tubes by the time they get it. Not much emotion involved here.

Finally, the results data go to the report writers who do the stats and interpret the results. Not much emotion involved here either. The emotional pressure definitely decreases as the process moves forward.

Errors are part of a larger management style that is different at HFSC, and the tone is set by the CEO, Dr. Peter Stout. Here is an excerpt of an email that was sent to the entire laboratory after a string of mass shooting in the Houston area (reprinted with permission):

Why we do what we do.

I have heard recurrent comments that to me are variations of the tension and struggle between the priority and value placed on an individual and that which is put on the collective. A very human conversation and something we make choices about daily.

This is part of what I hear in comments like “I worry about when am I going to make the mistake that is too big a mistake.” Put differently: “When and where do my needs as an individual become less important than those of the community.” (I will resist inserting the obvious Spock quote here.)

What we do matters. It matters like few other things that people choose to do. The criminal justice system, of which we are a part, directly impacts millions of people and fundamentally influences all 350 million people in this country. It is part of the very foundation of this nation. Every day we put out roughly 100 reports that permanently alter the lives of hundreds of people. Every year, the work we do permanently changes the lives of at least 100,000 people. What we do irrevocably changes the lives of victims, survivors, defendants and their families

This is an incredible privilege that few people ever have in a lifetime.

But that privilege comes with a difficulty and a burden that we carry with us daily. We have responsibility to our families and our children, but we also have this incredible responsibility to the community and the public. This work is hard. Things that matter are hard. Everyone in the criminal justice system takes a real personal risk with every report they issue and every action and decision they make.

This is because what we do matters.

A prosecutor wrestles with whether they have offered a plea bargain to someone who will be back in a few years after having committed a more heinous crime. Judges fret their rulings. Defense attorneys worry whether they do enough. That worry is because this work matters. It is bigger than we are as individuals and, yet, the risk we take we feel both as individuals and as members of the community to which we are responsible.

There are plenty of jobs in this world. Most jobs do not matter like ours do. For those jobs, there is less personal risk. There is less worry about is this the mistake. Those jobs are not always bigger than the individual.

We each make a choice every day to come do a job that is more important than we are as individuals.

I am in awe of all of you. Thank you for taking the risk, and for coming back every day for the unrelenting “hard” that is this work. Thank you for being willing to take on “hard” when everything in our current society seems to scream “hard is wrong. You deserve everything to be easy.” Thank you for caring beyond the easy.

This email establishes the importance of the job, while not minimizing the risk of an error. This email is part of a larger management style that establishes trust and consistent messaging. Trust involves not tolerating retaliation while allowing staff to question authority, not punishing honest mistakes, allowing staff input and providing an environment that allows for a little stress relief and fun.

Summary

The theme of this article is that errors are a result of human decision making that has variability, and will therefore sometimes result in undesirable outcomes. How an organization responds to those errors, and how it works to prevent them, may distinguish well-functioning from dysfunctional organizations. However, even among exemplar organizations, there exists a tension between adopting a nurturing/counseling response to errors and the need to prevent future bad actions. Encouraging an open discussion of various aspects of the laboratory workload may help address problems through collaborative approaches, and this seems to be the theme of successful organizations.

Funding statement

This work was supported, in part, by the Forensic Technology Center of Excellence (Award 2016-MU-BX-K110), awarded by the , , . The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the author(s) and do not necessarily reflect those of the Department of Justice.