The need for research-based tools for personnel selection and assessment in the forensic sciences.

The popularity of forensic science television programs has created a hiring challenge for forensic science laboratories. Laboratories receive unprecedented numbers of applicants, yet struggle to identify highly-qualified candidates. Forensic examiners must possess a unique set of knowledge, skills, and abilities (KSAs) plus other characteristics. They must be critical thinkers, detail-oriented, decisive, and self-regulated; be able to communicate clearly and effectively within the laboratory, with customers (typically investigators or attorneys), and in the courtroom; and demonstrate the required core competencies. Currently, no validated instruments, standardized job descriptions, or lists of KSAs exist to aid in forensic science personnel recruitment and assessment, often resulting in high turnover leading to costly new recruitment and training cycles. This article describes how industrial/organizational psychology develops methods and tools to improve workforce selection; describes some tools currently in use in forensic science; and advocates for research and development of better tools for use in forensic science.

The popularity of forensic science television shows has escalated the number of people seeking employment in crime scene investigation or forensic science. These individuals will often ask forensic professionals, “How do I get a job in forensics?” Forensic professionals typically struggle to answer this question succinctly due to the incredible diversity in hiring requirements across the country. As a result, the forensic professional will often provide their personal “forensic origin story” and warn the perspective candidate that each agency has its own requirements and processes. The only ubiquitous piece of information that can be provided is: “You will need to pass a background investigation, so stay out of trouble.” In fact, some police agencies lose as many as 50% or more of applicants during the background investigation.

Why such diversity in personnel selection and assessment among forensic service providers? The establishment of forensic job classifications, position descriptions, and assessment and selection processes for a given agency are typically influenced by a unique combination of one or more of the following:

Federal, state, and local laws

Civil service rules

Organized labor agreements

Parent organization policies and procedures (e.g., a forensic laboratory located within a city police department must follow the rules established by the Human Resources section of the parent police department or the Human Resources section of the parent city)

Specific forensic services provided by the Forensic Service Provider (FSP)

Accreditation requirements

FSP policies and procedures

FSP experience (specific hiring experiences of the FSP management team)

Accordingly, no real standards, research, or validated instruments currently exist in forensic science on which hiring managers can rely. This article provides an overview of the status of personnel selection and assessment from the perspective of industrial/organizational (I/O) psychology in the hopes that these principles will begin to be adopted by forensic science laboratories. This academic overview endeavors to wash away the differences between forensic service providers and consider: 1) the knowledge, skills, and abilities an ideal candidate possesses and 2) assessment methods that could be used to select the best candidates from a pool of applicants. The authors recognize that FSPs may be limited in their ability to use certain assessment methods due to their unique conditions and requirements (e.g., a labor contract that specifies only sworn peace officers can process a crime scene).

Toward the end of the article, examples of existing hiring practices of local, state, and federal FSPs are provided. It should be noted that many FSPs hire only occasionally for a given position and often alter the process with each recruitment. As a result of this “trial and error” approach to hiring, FSPs are generally incapable of providing data as to the efficacy of a given assessment and selection process. Because nobody has developed and empirically tested “tried and true” methods, there is no published literature describing or advocating for these techniques. This article thus serves two purposes: 1) introduce forensic service providers to personnel selection and assessment tools from I/O psychology with which they may not be familiar and 2) encourage forensic professionals and researchers to engage in order to develop structured research plans involving the assessment and selection of forensic professionals that will yield validated instruments and best practices that can be adopted by forensic laboratories.

Background

Local police departments often rely on county, state, or federal forensic laboratories for the scientific analysis of evidence. The forensic services offered by the forensic laboratory may include latent print examination, firearms and toolmark examination, footwear and tire track examination, forensic document examination, toxicology, drug chemistry, breath alcohol, trace analysis, DNA examination, and others. If the county/state/federal forensic laboratory has significant backlogs and sufficient funds are available, the local police chief or sheriff may decide to establish a forensic laboratory within the local agency. Even though the requirements of a laboratory are different from the requirements of a police station, police administrators generally understand the need for secure buildings, secure computers, appropriate equipment, and consumable supplies. But what about the people who work in the forensic laboratory? What positions should exist? What are the minimum qualifications for hiring experienced scientific staff, technical support staff, and supervisors? Are the qualifications going to depend on the forensic discipline? Once the laboratory is established, will trainees be hired in the scientific and support positions? If so, what are the requirements for trainees in these positions?

Often, the police department will hire an experienced laboratory director to assume the responsibilities of building and staffing the forensic laboratory. Before staffing the laboratory, the laboratory director must establish position descriptions that detail the knowledge, skills, and abilities for each job classification (scientific staff, technical support staff, and management). To that end, the laboratory director will rely on their own experiences and often evaluate the job descriptions of other forensic service providers. The human resources section of the police department or local government (e.g., the city human resources department) will typically be involved to ensure adherence to administrative processes and legal requirements. Organized labor unions will become involved if any of the job duties to be performed under the new job classifications are currently performed by existing union members within the department.

The process of writing job descriptions is critical to the future success of the FSP because the assessment, selection, and continued employment of each staff member rests on their ability to meet and maintain the minimum qualifications listed in position description. Unfortunately, the development of position descriptions rarely involves professionals who specialize in job analysis, task analysis, organizational development, employee behaviors and attitudes, and candidate assessments. These professionals are called industrial/organizational (I/O) psychologists and their expertise can help FSPs establish robust position descriptions and implement an effective personnel selection and assessment program that can help identify the job candidates who will best perform a job and who are a good fit with the organization.

Historically, selection programs in FSP have focused on identifying skills important for performance early in a career. However, due to the extensive and costly training requirements tied to the forensic science disciplines, identifying analysts who will succeed beyond the first few years on the job is immensely important. To develop effective selection measures and hiring assessments that predict long term success, it is crucial to better understand the context of the various forensic jobs, including the work environment, key job tasks and duties, and social and group structural characteristics. Ultimately, understanding how these work and worker requirements fit together should lead to improved personnel selection models that will lead to better hiring decisions and improved employee performance and workplace culture.

In the following sections, we describe how the forensic science community can apply selection methods and tools from industrial/organizational psychology to improve workforce selection decisions and practices. We begin by discussing the role of job analytic techniques which serve as the foundation upon which all other personnel selection and assessment decisions are made. Then, we discuss the role of predictors in the selection process and common predictor measurement techniques. This is followed by a review of job performance criteria and how performance on an assessment can be aligned with job performance to establish the diagnostic utility of a selection practice. The article ends with a discussion of several extant selection processes used in the forensic science domain and a list of recommendations for forensic science hiring managers.

Establishing position descriptions: job analysis

The first step in establishing position descriptions is performing a job analysis, which is a systematic examination of the job [1]. A job analysis seeks to understand the complete range of tasks performed for a given job or job family, the frequency of performing these tasks, the importance of these tasks, and the context and environment in which they are performed. Ideally an I/O psychologist or human resources practitioner with experience and knowledge conducting job analyses guides this process; however, it is unknown how many FSPs have access to such professionals. In practice, the method for developing position descriptions ranges from an informal process wherein an individual uses their existing job duties as the basis of the description, to a formal process that includes practitioners, management, and human resource personnel. Difficulties arise when job descriptions are not conceived in a forward-thinking manner, particularly for a trainee position. For instance, the trainee's “job” is to acquire knowledge about the agency (institutional knowledge), the foundation of their discipline (academic knowledge), and the methods of their discipline (technical knowledge). Given the intense learning-curve, what are the job requirements of the trainee upon hiring? What are the job requirements at certain milestones of the training process? What job requirements must be met to progress beyond the trainee position? How can these job requirements be assessed? If the trainee is having difficulty meeting the job requirements, to what extent will remediation be performed? Are the assessment and remediation processes legally defensible if the trainee is not meeting job requirements and must be terminated? Poorly conceived position descriptions can have long term effects on personnel selection, assessment, and retention.

For a hiring manager in the forensic sciences, the outputs of a job analysis should help identify which knowledge, skills, and abilities (KSAs) can be evaluated during the selection process, which can be evaluated during training or probation, and which KSAs are required for more experienced analysts. The results can be used to establish job requirements, performance standards, and promotion requirements and to identify training needs and requirements. Using this knowledge, human resource managers and practitioners can begin the task of selecting or developing a set of tests to assess each competency to be administered to job candidates during the selection process. Many of the essential tasks and duties of a position may already be captured in formal job descriptions and performance review information. These documents should provide a starting point for identifying critical duties, worker attributes, and ways to measure these skills. If not, Lab supervisors should work with their human resources department to conduct a formal job analysis to identify the critical job duties, tasks, and elements performed on the job as well as the human-attributes required of analysts to successful perform these tasks.

Competencies for the forensic sciences

Traditionally, forensic science laboratories have sought applicants with academic training in science with the recognition that this type of formal schooling would better prepare students for careers in drug analysis, trace analysis, and forensic biology compared to other disciplines. Although having a strong scientific background is indeed important for many jobs in the forensic sciences domain, the critical job requirements of the pattern evidence function require analysts to also have visual perceptual skills, such as pattern recognition and pattern discrimination. Analysts must diligently inspect and interpret minute differences and similarities in impression patterns, which requires a high level of detail orientation, visual attention, and conscientiousness. Often, the samples that analysts must inspect are imperfect; the images can be partially smudged or contain noisy areas that makes inspecting and comparing an impression to a standard difficult. Thus, analysts must be able to mentally filter out this background noise and identify key features for comparison. These skills sets are important because pattern evidence interpretation is a complex task that still fundamentally depends on the judgements and expertise of highly skilled human examiners.

One recognized challenge in the pattern evidence field is that there is little time to develop or assess skills like pattern recognition in a college or degree curriculum. Rather, such skill development is assumed to evolve during apprenticeship experiences or through on the job training. While research shows that visual and cognitive perceptual skills and working memory develop and improve through training [[2], [3], [4], [5], [6]], identifying applicants with aptitudes in pattern matching and relevant visual search skills could lead to better hiring decisions and reduced training costs in the field. In addition, research points to the importance of analysts having metacognitive skills, an individual's awareness of his or her own cognition. As noted by Ref. [7]; these higher-order cognitive skills may enable analysts to generally recognize when they are likely to make an error on a comparison and recognize when other examiners are likely to make an error as well. Such metacognitive awareness may also help analysts determine when it is best to use different tools or feature extraction strategies or decide when to not compare an image because it is too noisy to draw a reliable conclusion.

Forensic analysts must also exercise other (i.e., non-technical) skills as part of their daily duties. Analysts must be able to think critically, make well informed and timely decisions, exercise self-restraint, and communicate effectively [8]. These skill sets are not surprising given that analysts must be able to make sense of, combine, and organize information as well as analyze and evaluate information in a non-biased manner. There is also an increasing need for examiners to understand issues like human factors and statistics and to be able to communicate their results with the appropriate language and communication skills to juries and judges in the courtroom [9]. Being a persistent learner is another important characteristic required of the position as analysts who keep up with the latest scientific advances and developments in their domain are often the most successful on the job.

In summary, the outcomes of a job analysis can identify the work activities, tools, and equipment used in performing work activities, the context of the work environment, and the requirements of personnel performing the job, such as the KSAs and other characteristics required for successful job performance. Once the critical KSAs and core constructs of a job have been examined, the next step is to identify or develop tools and tests to assess these skills. By determining which predictor variables most strongly correlate with job performance, human resource practitioners and hiring managers can begin to develop a selection and assessment system that will validly and reliably predict training and job performance. In the next section we discuss the issue of predictor measurement, which aims to identify ways to measure the critical worker requirements of a position.

Predictors of job performance

It is well established that individual differences exist and that these differences can range across physical and psychological dimensions, which can ultimately influence an individual's effectiveness in performing a job. For instance, differences in critical thinking may allow some analysts to better comprehend the significance of scientific advances in forensic science than others, differences in working memory capacity may lead some analysts to work more efficiently than others, and differences in interpersonal communication skills may lead some individuals to perform better on the witness stand than others. Because organizations invest a considerable amount of money in their employees over time, it makes sense to link the levels of KSAs job applicants possess with how they perform on the job.

In personnel selection and assessment theory, predictors refer to observable tests or assessments that are designed to measure the underlying KSAs deemed important for a position. Such tests or methods can be administered to job applicants during the selection process to identify the best candidate, or set of candidates, for the job. If tests are properly developed, administered, scored, and interpreted, their ability to predict an applicant's job performance (or other job-related criteria) improves [10].

At present, FSPs attempt to evaluate predictors for success when filling vacancies. There are two ubiquitous screening tools for job candidates beyond the background investigation: 1) biographical review to ensure the candidates meet the minimum qualifications (e.g., educational requirements) and 2) the interview. For many FSPs, these two screening tools are the only screening tools used for all positions and interviews are often limited to a small set of questions. Some FSPs use written tests to assess general knowledge of science (if a science degree is required), to assess discipline knowledge (if not a trainee position), or to assess basic math and language competencies. Some FSPs may include work sample tests that gauge perceptual ability, particularly for the pattern evidence disciplines. Fewer agencies appear to use work sample tests to gauge critical thinking, technical writing abilities, or personality traits.

In the following section, we discuss the state of the science of methods to measure common predictors that could be considered by FSPs. These predictors include cognitive ability, personality, biographical information, and work sample testing. Guidance on how these methods could be applied in the forensic science domain will also be provided.

Cognitive ability

General cognitive ability measures have many advantages in personnel selection: (1) they show the highest validity for predicting training and job performance, (2) they may be used for all jobs from entry level to advanced, and (3) they are relatively inexpensive to administer [11]. Specific examples of abilities measured through cognitive ability tests include memory span, numerical fluency, conceptual classification, semantic relations, conceptual foresight, spatial orientation, visualization, and logical evaluation [10]. An example of a popular cognitive ability test is the Wonderlic Personnel Test. First developed in 1938, and still widely used, the Wonderlic is a 12-min test that contains 50 multiple choice items. Test items are designed to assess vocabulary, reasoning, formal syllogisms, arithmetic reasoning and computation, analogies, perceptual skills, spatial relations, and a number of other mental abilities. One of the appealing features of the test is its extensive set of norms scores that have been developed through its long history [12].

Another example of a mental ability test is the Raven's Progressive Matrices (PM), which is a nonverbal test of general intelligence. The test is designed to measure a person's ability to form perceptual relations and to reason by analogy independent of language and formal schooling. Items are presented in the form of a 2 × 2, 3 × 3, 4 × 4, or 6 × 6 grid of patterns with one pattern missing from the grid. Participants are asked to select the missing pattern from a list of possible alternatives that best completes the grid. Items become increasingly difficult as progress is made through each set. Successful performance on the Raven's PM requires an accurate perception of features and patterns between items in the matrix, the ability to notice patterns and reason, and the ability integrate knowledge to determine which pattern would come next [13]. Because the Raven's PM measures abstract reasoning and requires visual observation and spatial working memory skills, this test may be particularly relevant for selection and assessment in the pattern evidence domain.

For the personnel selection practitioner, cognitive ability tests offer several advantages. The outcomes of cognitive ability tests have been linked to a range of important organizational and job-related outcomes including job performance and success in training [[14], [15], [16]]. The predictive utility of cognitive ability increases as the complexity of the job increases; thus, cognitive ability tests may be particularly useful for finding applicants for cognitively demanding jobs. Cognitive ability tests are also typically easy and cost effective to administer and score. Some can be administered via pencil and paper, and many can be administered online. A further advantage is, unlike other personnel selection methods, cognitive ability tests cannot be influenced by a test taker's attempts to impress management or fake responses. In contrast to these advantages, a concerning disadvantage of cognitive ability tests is that they typically produce subgroup differences - they produce racial and ethnic differences - that are larger than other valid predictors of job performance such as personality tests and structured interviews [15]. Thus, to reduce adverse impact, cognitive ability tests can be used in combination with other selection methods, such as interviews or biographical data, to reduce any potential effect on under-represented groups [17]. If practitioners are going to use cognitive ability as a screening test, then there needs to be a strong linkage between test scores and job performance (or the criterion of interest). In FSPs, it may be best practice to use cognitive ability tests in a pass/fail manner to screen out, for example, the bottom 20% of candidates or those who are minimally qualified for the job rather than combining the score with tests that contribute to the overall ranking score of a candidate.

Personality

Personality tests are designed to systematically elicit information about a person's motivations, preferences, interests, emotional make-up, and style of interacting with people. Interest in personality as a predictor of job performance stems from the desire to predict the motivational aspects of work behavior. Research has shown at least modest validity for some personality traits in predicting job performance (e.g., Barrick and Mount, 1991; [18,19]. First, we will discuss resilience and emotional intelligence. Then we will discuss the Big-5 personality traits.

Resilience. Resilience is repeatedly mentioned as an important component for succeeding in highly stressful jobs. Given the stress, long hours, and case load experienced by analysts, it is also likely to be an important predictor of retention and performance in the forensic sciences domain. The concept of psychological resilience has been studied for several decades with roots traced back to developmental psychology, health psychology, and positive psychology. Researchers generally describe psychological resilience as a process that involves interaction between an individual, his or her past experiences, and current life context [20] rather than a personality trait. This characterization implies that reliance is a malleable state that changes. Psychological resilience is important for the forensic sciences community in that examiners must be kept fit for duty and mentally healthy for completing their daily duties despite facing chronic and acute stressors associated with the job. Being able to cope with these experiences, adapt to challenges, and maintain positive thinking and control in the presence of or after experiencing significant adversity is an important, but often overlooked, component of the job. Recent research shows positive correlations between resilience, well-being, and job performance [21].

Emotional Intelligence. Another skillset that has drawn interest in the forensic sciences community as a potential predictor of job performance is emotional intelligence (EI). EI can be defined as “the ability to monitor one's own and others' emotions, to discriminate among them, and to use the information to guide one's thinking and action” [22]; p 189). This definition identifies EI as an actual ability or facet of intelligence. By contrast, other leading researchers conceptualize EI as an umbrella term that “encompasses a constellation of personality traits, affect, and self-perceived abilities, rather than actual aptitude” [23]. As such EI is different from emotions or emotional style and unique from traditional measures of intelligence that are based on general mental ability. Individuals with high levels of emotional intelligence are thought to have more success, have a greater peer network, and be savvier at navigating and engaging in interpersonal relationships.

Regarding the predictive validity of EI in selection contexts, meta-analytic studies have shown that EI correlates with job performance and can robustly predict performance criteria beyond cognitive ability [24]. The predictive validity offered by EI measures may stem from the fact that many EI measures included items that tap well established psychological constructs such as conscientiousness, extraversion, general self-efficacy, self-rated performance, emotional stability, and cognitive ability that have well established linkages to job performance [23]. Still, additional research is needed to better understand how these measures can be used to support better hiring and selection decisions in the forensic sciences community.

The Big-Five Personality Traits. Today's well-known, hierarchical, five-factor model (FFM) of personality (alternatively, “the Big Five”) has been the subject of much research (Barrick & Mount, 1991; [[25], [26], [27]]. The five traits include: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Openness incorporates such traits as imaginative, curious, and creative; conscientiousness includes traits such as organized, thorough, and reliable; Extraversion is comprised of traits such as talkative, assertive, and active; agreeableness includes the traits kind, trusting, and warm; neuroticism includes traits such as nervous, moody, and temperamental [28].

Meta-analytic studies have repeatedly found linkages between personality traits and important job outcomes with the results pointing towards a major tenet in organizational research: Conscientiousness has a strong relationship with job performance across a variety of domains, and the remaining four personality traits show modest correlations with outcomes and success for some occupations [29]. For example, Hurtz and Donovan's [30] meta-analysis found personality to be predictive of task performance and contextual job performance (which refers to engaging in behaviors such as helping others, favorably representing the organization, and persisting with extra effort). Specifically, they found that conscientiousness exhibited the highest validity coefficient of the Big Five Dimensions with task performance (technical performance, job knowledge, completion of specified job duties, and objective performance data), contextual performance (ratings of work dedication, effort, persistence, self-direction, and commitment to others) and interpersonal facilitation. Emotional stability showed consistent, but low, levels of criterion related validity across these criteria as well. These findings have been supported in a number of additional studies [26[26], [31] [[26], [31]].

In FSP, personality testing could be used in combination with other predictors of job performance to identify candidates who are likely to succeed in training, succeed on the job, and fit and contribute towards a positive organizational culture. Personality tests also offer several practical benefits: they are easy to administer, the cost of purchasing a personality test is typically less expensive than developing customized tests, and test administrative costs are generally inexpensive.

Biographical data

Biographical data (biodata), factual information about an applicant's background and life experiences, offers a powerful noncognitive alternative to cognitive ability tests and have shown significant promise as predictors in selection. The principle of biodata is that past behaviors are relevant to predicting criteria such as performance, absenteeism, and other work-related outcomes. Schmidt and Hunter [11]; in a meta-analytic review of over 85 years of personnel psychology research, reported that biodata is strongly correlated with training and job performance as well as turnover, absenteeism, job proficiency, and performance appraisal ratings (e.g., Refs. [[32], [33], [34]]. Based on these meta-analytic results, researchers have concluded that biographical inventories have almost as high validities as cognitive ability tests [33]. In addition, research indicates that biodata show less adverse impact against protected groups than that of cognitive ability tests [35]. Importantly, the high predictability associated with biodata, the ease of administration of biodata instruments, the low cost, and the lack of adverse impact have led to the widespread use of biodata in both the public and private sectors [36].

A significant advantage of using biodata as a predictor of job performance is that the information can be quickly and easily gathered from applicants. It can be gathered by asking an applicant about his or her accomplishments, experiences, and general interests during the application or interview process. It can be gathered through a behavioral narrative in which applicants are asked to describe verbally or in a written statement how they have successfully demonstrated a competency required for the job. It can also be gathered through self-report questionnaires that ask applicants about a variety of topics including educational experiences, hobbies, interests, attitudes, and values presumed or demonstrated to be related to personality factors, personal adjustment or success in social, educational, or occupational pursuits. The items are typically grouped into clusters and assessed and weighted based on their strength of relationship with some criterion of job success, such as job tenure or turnover [37].

Work sample tests

Work sample tests are another predictor commonly used to assess critical skills in a selection program. These types of assessments measure job skills by requiring an individual to demonstrate competency in a situation parallel to that at work, under realistic and standardized conditions. One of the biggest differences between work sample tests and other assessment methods is that the focus is not on reviewing historical information about a candidate or examining their knowledge or how they would respond to a hypothetical situation, but on examining what he or she can do when placed in a simulated work scenario. By observing behaviors and decision-making in a realistic work scenario, work sample tests have the potential to provide a wealth of information about how someone would perform on the job. Notably, work sample tests often require a degree of learning to complete the task, which provides an indication of a candidate's ability to learn new techniques. They also offer greater stimulus and response fidelity than surveys or tests of general cognitive ability. The tradeoff, however, is they take longer to develop, administer, and score. Work sample tests have been used effectively to evaluate decision making and interpersonal skills, investigative skills, and leadership.

Work sample tests offer several distinct advantages over other assessment methods. First, they provide direct evidence of an applicant's ability to perform the job. Meta-analytic studies show that candidate performance on a work sample test is highly correlated with job performance (Schmidt & Hunter, 1989). Second, they are less likely to have a discriminatory effect on protected groups compared to written tests or tests of general mental ability [1,38]. Third, work sample tests serve as a realistic job preview and can lead applicants to self-select out of the selection process if they do not like the task [10]. Below are two examples of work sample tasks for the forensic science community.

Pattern Recognition Work Sample Test. One example of a work sample test for the forensic science community is the Pattern Recognition Test being developed by the National Institute of Standards and Technology (NIST), which can be found online as of 2021 at https://www.nist.gov/quiz/do-you-have-what-it-takes-be-forensic-fingerprint-examiner. This test contains sample items designed to assess an applicant's pattern matching skills. Items are designed to assess applicants' abilities to: compare and judge the orientations of lines; match and perceive the widths of lines; mentally rotate patterns and conduct visual searches to find matching features in different orientations; judge and compare features of interest; separate superimposed images while examining feature patterns; filter out background noise; detect if a small pattern is present within a larger pattern; and follow lines while viewing sample items. Although the test has not yet been validated, it offers a significant step forward compared to existing selection practices, such as reviewing academic transcripts, conducting interviews, and writing samples, towards examining the pattern matching skills of job applicants. Once validated, this test, or a similar test, could be administered to applicants during the selection process and the scores would be used to identify individuals with aptitudes in visual perception and pattern matching. Because test items also require individuals to form representations of features and mentally rotate and manipulate them, it can also assess the spatial working memory skills required of analysts.

Form Blindness Test. Another example of an ability test that has been used in the pattern evidence disciplines to predict future job performance is the Form Blindness Test. “Individuals who suffer from form blindness are not able to see minute differences in size, shape, or form until they are magnified to a level within an observer's threshold of discrimination … an analogy in the realm of sound would not being able to hear a specific pitch until it reaches a certain volume [39]; p. 461).” While only a small percentage of the population is affected by form blindness, given the time and energy invested in training pattern evidence examination, being able to identify individuals who have difficulties detecting minute differences early in the selection process would be a significant benefit to hiring managers and lab supervisors. These applicants could be guided towards or selected for other activities that do not require a high degree of visual acuity. The Form Blindness test was developed approximately 80 years ago, for document examiners. It contains five sub-tests that require test takers to arrange objects from smallest to largest according to their size, identify shapes of equal size, and arrange items in terms of degree of curvature and width. A higher score on the exam reflects a smaller likelihood that one is form blind.

One limitation with the Form Blindness test is that it is designed to test for deficiencies rather than specific levels of ability. That is, the test only provides information on a minimal threshold requirement (can an individual distinguish between test form features) and does not provide a score that can be used to judge the relative talent or ability levels of candidates. Another limitation is that it does not assess critical pattern recognition skills such as the ability to filter noise, contrast sensitivity, or the ability to mentally rotate and transform images. Still, several studies have shown promising results of using form blindness as a predictor of performance on a latent print examination [8,39]. Here again, it may be best practice to use a form blindness test in a pass/fail manner to screen out, for example, the bottom 20% of candidates, rather than combining the score with those tests that contribute to the overall ranking score of a candidate.

Structured interviews

The final predictor method we will discuss is the structured interview. The employee interview is one of the most widely used methods for assessing job applicants. Due to its popularity, a great deal of research has been conducted to determine the best method for conducting interviews and how to improve the reliability and validity of interview evaluations. Much of this research has demonstrated that structured interviews, which use structured rules for eliciting, observing, and evaluating responses, are superior to unstructured interviews [[40], [41], [42]], that they provide incremental validity over personality tests and cognitive ability tests [43,44] and that they can be used effectively to measure different constructs and predict different criteria [45,46]. Structured interviews are effective because the predetermined rules and formal structure reduce the number of extraneous factors that can affect an interviewer's decisions about what questions to ask and how to evaluate performance. This focusing, in turn, helps to increase interviewers' agreement on their overall evaluations.

Two question formats are generally used during the structured interview: situational and behavioral. Situational questions require applicant to indicate what they would do in a hypothetical job-related situation, whereas behavioral questions require applicants to describe what they did in past job-related situations. For example, a behavioral interview question may ask a job candidate to “describe a time when you had two or more competing deadlines at work. What did you do? How did you handle the situation? What was the outcome?” A situational question may ask “Tell me how you would handle an employee who is always turning in assignments late and not performing well on the job?” Results of recent meta-analyses indicate that both question types have criterion-related validity, but questions regarding past performance questions yield slightly higher validity coefficients [45,47].

From a practical standpoint, structured interviews offer an affordable and effective approach for assessing a variety of applicant skills, including job knowledge, interpersonal skills, and other personal attributes (e.g., work habits, leadership, dependability, stability, and perseverance). Within the forensic sciences discipline this technique could be used for assessing these abilities as well as for assessing basic personality tendencies and organizational fit, which can be critically important when selecting individuals into mid-career or leadership positions.

One important note regarding the use of structured interviews is that to be effective, members of the interview panel should be properly trained how to ask questions, how to rate performance, and how to avoid common errors when evaluating applicants and their responses (e.g., stereotyping applications, being more favorable of applicants based on one or two characteristics, giving out higher ratings to applicants more similar to themselves in a way, allowing the quality of the previous applicant to influence ratings of the present applicant; [10]. It is also important to note that when it comes to the issue of determining how many KSAs to measure with the interview, more is not always better as research shows that interviews are most effective when they are designed to assess only a few job-related KSAs [48].

Summary of predictor methods

In summary, identifying assessment techniques and methods, such as the ability tests and work sample tests discussed above, is a critical part of the assessment process. Methods such as structured interviews, work sample testing, cognitive ability tests, and biodata inventories offer ways to gather information regarding KSAs deemed critical for the job. There has been a considerable amount of research conducted on each of these methods in different selection contexts and the reader is encouraged to explore additional resources to learn more about these methods (e.g., Society for Industrial and Organizational Psychology, www.siop.org).

Another critical step in the selection and assessment process is identifying criteria that accurately represent job performance. Once these criteria have been identified, a selection practitioner can begin to establish linkages between the observed predictor measures and observed job performance. In the next section, we review the current conceptualization of job performance and how job performance can be measured.

Measuring job performance

Assessing job performance or the attainment of training outcomes is challenging if the job description or training program performance metrics are not clearly delineated and do not mirror the actual job tasks to be performed. For instance, a trainee job description might include a requirement that the employee achieves a certain level of accuracy during the comparison of unknown and known samples within the first year. However, the job description may unintentionally omit a requirement for attaining a certain level of accuracy selecting the appropriate samples for comparison (i.e., utility decisions). If the trainee is exhibiting significant instability with utility decisions during training, how does an FSP articulate the trainee's inability to perform the actual job tasks associated with casework since this critical task was not included in the job description or appropriately assessed during the training program? Furthermore, it has long been recognized that job performance is best thought of as a multidimensional construct [49] beyond the performance of the basic job tasks. These constructs include task performance, contextual performance, counterproductive work behavior, and adaptive performance. We briefly summarize each of these dimensions below.

Task performance

Task performance refers to the technical proficiency component of job performance; that is the core technical behaviors and activities involved in the job [50,51]. According to Ref. [52]; there are two types of task performance. One aspect of task performance includes activities that produce a finished product. For instance, the procedures associated with collecting DNA samples from items of evidence, analysis of DNA samples, and on to a final forensic report of analysis. A second type of task performance consists of activities that service and maintain the technical core through support such as replenishing chemicals and forensic kits; distributing reports; or providing important planning, coordination, supervising, or staff functions that enable the organization to function effectively and efficiently.

Contextual performance

Contextual performance refers to such behavioral dimensions as: (1) Personnel Support -- Helping others in the organization (e.g., peers or supervisors); cooperating with others and putting team objectives ahead of personal interests; (2) Organizational Support – Favorably representing the organization by defending and promoting its objectives, as well as supporting the organization's mission and objectives; and (3) Conscientious Initiative – Persisting with extra effort despite difficult conditions; taking the initiative to do all that is necessary to accomplish objectives even if they are not normally a part of one's duties [59].

A useful way to view contextual performance is by comparison to task performance. Contextual activities differ from task activities in at least three important ways. First, task activities contribute either directly or indirectly to the technical core. Contextual activities, on the other hand, do not support the technical core itself as much as they support the organizational, social, and psychological environment in which the technical core must function. Second, task activities usually vary between different jobs. Contextual activities, however, are common to many or all jobs. Their peripheral details vary because they are performed in environments that change from job to job, but their central features are the same. Contextual activities within a forensic laboratory might include volunteering for additional work, following organizational rules and procedures even when personally inconvenient, assisting and cooperating with coworkers. And third, because the source of variation in task performance is proficiency with which task activities are carried out, the important human characteristics are KSAs that covary with task proficiency. The main source of variation in contextual performance, however, is not proficiency, but volition and predisposition, and such behaviors are probably better predicted by motivational and personality characteristics. In fact, studies examining the relationship between contextual performance and personality (using the Big 5 personality traits discussed earlier) have shown a relationship between contextual performance and both conscientiousness and agreeableness (see, for example, Borman et al. [60]).

Counterproductive work behavior

Counterproductive work behavior (CWB) involves intentional acts by employees that potentially violate the legitimate interests of, or do harm to, an organization or its stakeholders [53]. It may include behaviors such as theft, improper substance use, misuse of time and resources, inappropriate verbal or physical actions toward coworkers or supervisors, or destruction of property at work. By definition, counterproductive work behaviors are voluntary acts that are detrimental to an organization. They have important implications for the well-being of an organization. Theft alone is estimated to cause worldwide losses in the billions of dollars each year. These estimated losses do not include losses from other sources, nor do they consider the fact that many losses attributable to CWBs go undetected. The consequences of CWBs and their persistence in the workplace have led to increased attention being given to the study of such behaviors.

Strategies used to manage counterproductive work behaviors (CWB) in the workplace are common at the recruitment stage, where companies use procedures like integrity screening and personality screening to identify individuals who may have a history of propensity to CWBs. Among incumbents, one of the biggest ways to reduce the incidence of CWBs in the workplace is by enhancing organizational justice and balancing perceptions of fairness.

Adaptive performance

There is evidence to support that the adaptive performance construct is also an important component of job performance [54]. Adaptive performance in the work environment refers to the ability of an individual to change or adjust behavior to meet changing task or situational demands. Modern work is frequently characterized by jobs where adaptive performance is crucial for employees to succeed in light of new or altered task demands. This recognition has fueled growing interest in adaptive performance as an important component of job performance. Selecting individuals who have strong adaptability skills can also be important in forensic labs. Being able to adapt to a new lab culture, possessing the flexibility to adapt to and abide by new lab rules, and work well with new team members is important. This may be particularly true for forensic specialists who have been working in their career field for a number of years in a different setting and who are used to following a different set of standard operating procedures. Being able to adapt to a new lab culture, new lab rules, and new team members is equally, if not more, important than task and conceptual performance in these cases.

Identifying job performance measures

Identifying and selecting a measure or measures of job performance to validate a selection system is a critically important task. These may be measures of behavior, performance, attitudes or outcomes or tallies that typically reflect a job-related outcome [55]. Examples include output metrics (e.g., number of units produced), quality measures (e.g., number of errors), trainability scores (time to reach performance standard), and ratings of performance. High scores on these measures are intended to reflect what is meant by “successful” job performance. These measures, however, can be difficult to extract from an operational forensic setting. The challenges for FSPs include infrequent hiring of a few candidates at a time or changes to the hiring process between recruitments. A brief discussion of some ways to measure job performance in an ideal setting follows.

Production data

A production measure refers to some quantifiable job output than can be linked to an individual's performance. Some organizations may seek ways to use such “hard” criteria (e.g., number of cases completed, number of comparisons performed, or number of positive associations) because they tend to avoid or minimize human judgment, and thus are believed to produce more objective performance data. For this reason alone, using objective production measures has some degree of intuitive appeal. In addition, these data are usually relatively accessible. Unfortunately, these approaches can create weaknesses in the quality system if the forensic analyst feels pressure to complete examinations with an eye towards attaining a performance metric rather than accuracy. The appropriateness of production data to measure performance is highly dependent on the type of performance being assessed and the potential consequences of using production data.

Performance ratings

Performance ratings are generally described as falling on the subjective side of the objective/subjective continuum due to their reliance on human judgment for their data. In an ideal world, performance ratings should be able to provide performance scores that are free from contamination (e.g., skewed scores due to rater biases) and deficiency (e.g., rating scale inadequately reflects performance) [56]. Unfortunately, rater errors, biases, and other judgment-based inaccuracies detract from the scores’ validity [57]. Still, performance ratings continue to retain their predominance as a “go to” criterion measure, if for no other reason than ease of development, use, and cost.

Knowledge testing

Job knowledge tests are typically most appropriate as training criteria, in large part because it is often important to gauge the amount of learning that has occurred in a training context, and knowledge testing is an easy way to do that. Within the context of forensic science, repeated knowledge testing during training assists in the retention of the academic knowledge that supports the forensic discipline. Long-term retention of foundational information supports the practitioner's testimony skills and provides a framework to which new information can be added as the forensic discipline evolves. Knowledge can be monitored through periodic testimony review and refreshed through continuing education.

Training proficiency

In many instances, performance in training provides evidence of possession of the necessary components of job performance. Training makes an obvious contribution to job performance because a person must know what to do and be able to do it before performing a job. Much of the information and skill that influence performance in jobs like those held by forensic specialists is acquired through technical training. Therefore, knowledge and skill demonstrated in training may be regarded as evidence of a person's ability to perform on the job. In a forensic setting, competency tests at the end of training and regular proficiency testing throughout the practitioner's career often gauge basic task performance.

Validation studies: putting the pieces together

So far, we have discussed predictors of job performance and measures of job performance. Within a traditional selection framework, once predictor measures and criteria have been identified and linked, the next step would be to use these pieces of information in a validation study to identify which predictors offer the best utility in identifying potential job candidates. The two most common validity study design types are predictive and concurrent.

With a predictive validation design, predictor measures are completed by job applicants as part of the hiring process. Scores on these measures are filed away and selection decisions are made on the basis of other available section data [1,10]. After several months on the job, criterion data are collected (e.g., job performance scores, supervisor ratings, or training scores) and scores are correlated with predictors from the selection test to determine if a significant relationship exists. Thus, the predictive design is characterized by the length of time between when predictor and criterion data are gathered and by the fact that data are gathered from actual job applicants. One of the benefits of the predictive design is that because job applicants complete the selection tests (as opposed to existing employees) results provide a direct assessment of operational validity of the selection system. One of the biggest drawbacks of the predictive design, however, is the time interval between when predictor and criterion data are collected [10]. Typically, researchers must wait several months between these two intervals, which can be a significant burden depending on organizational needs and the timeline of the validation study. Another disadvantage is that if an organization hires only a few new employees per month, it may take a significant amount of time to obtain a sufficient sample size to complete a validity study.

In a concurrent validation design, predictor measures are completed by job incumbents and criterion data are collected for these individuals at roughly the same time. After the two sets of data have been collected, researchers can examine whether scores from the predictor measures are associated with job performance scores. Thus, compared to the predictive validity design, the concurrent design offers a more efficient timeline. However, there are several trade-offs with this increased efficiency. First, it is possible that job incumbents will perform differently than job applicants on the selection tests. This may be especially true for analysts who complete a rigorous training curriculum and practice many of the tasks targeted by the selection tests as part of their daily duties. As a result, the generalizability of the tests may not apply to all applicants [10]. Second, differences in job tenure may affect scores on predictor and criterion measures. Third, because rejected applicants, poor performers, and employees who have left the organization are not available for the validation study, the scores on the criterion measures may be significantly restricted. The scores would only be gathered from existing employees, who will typically score higher on the selection measure than would a full range of job candidates as primary participants. This restriction of range restriction can attenuate the value of validity coefficients [58].

In addition to these two approaches, a third validation approach called content-related validation relies on the judgments of subject matter experts to determine whether a test or method is a valid way to measure job performance. With content related validation, subject matter experts are asked to judge the extent to which KSAs are needed to perform job tasks and duties and judge the extent to which the test or predictor method measures these KSAs. Content-related validation typically does not require the collection of any criterion; the expert judgments are used to establish the utility of the measure. As such, content related validation approaches typically take less time to complete and requires a lower financial investment. A limitation of this approach is that it cannot address bias or fairness concerns and it may not be sufficient for certain measures, such as personality or intelligence.

In practice, many organizations may interconnect the different validation approaches to continuously build evidence of validity. They may use content related study to justify the use of a selection practice, then after it is implemented, they might collect applicant and performance data and see if there are any strong linkages between performance on the selection test and performance on the job.

Examples of existing personnel selection systems used in forensic sciences

While some organizations continue to rely on hiring practices that include an appraisal of educational requirements and oral interviews, other have begun to incorporate additional testing and tools within the selection and assessment process. For instance, Ruggiero [8] reports that the Los Angeles County Sheriff's Department included the addition of a written exam to their hiring process. The exam covers reading comprehension, written expression, data analysis, error analysis, and pattern recognition. In late 2015, her department had approval to fill 12 open positions. Ruggiero reported they received over 450 applications. Of those, 79 applicants met the minimal educational requirements and were invited to take the written exam. Scores from the exam were used to invite 52 candidates to an oral interview. Thirty-eight candidates successfully passed the oral interview and were placed on the eligibility list. This list was then rank ordered and the top 13 people were invited to another selection interview. By including the written test component within the selection process, her department was able to significantly reduce the number of applicants invited in for the oral interview and identify candidates who would likely excel in the position based on a sample of test performance. Although published data are not available, Ruggiero [61] indicated that all trainees hired from this testing process have completed the latent print comparison training, a significant improvement from previous testing processes.

A similar type of selection system has been successfully implemented in the Las Vegas Metropolitan Police Department Forensic Laboratory for hiring forensic scientist trainees in latent prints. The system includes a series of selection hurdles. First, applicants are evaluated based on their educational and other minimal requirements. Applicants who meet these minimum requirements are invited to take a written multiple-choice test that asks questions pertaining to basic science and math, lab safety, and questions about common forensic science principles and practices. Applicants who score above the cut-off proceed to a basic fingerprint comparison test. Those who meet the cutoff for the comparison test are invited to take a written practical examination. The written practical examination is administered in two parts. For Part 1, candidates complete a work sample test in which they must read scientific literature, make technical decisions based on the information, and justify their decision. Part 2 is another work-sample test that resembles a case scenario. Candidates are given a set of laboratory procedures and case records. The candidate must find the major mistake, discuss possible consequences of the mistake, and recommend what should be done with the employees involved.

Applicants who pass these two practical exams are invited to complete an oral board where they must answer questions about their background, life experiences, and general knowledge of forensic science. The purpose of this oral interview is to assess applicants’ public speaking ability, potential as an expert witness in court, and organizational fit. Applicants who pass all of these tests are then referred for background investigations. Importantly, this last step still offers a significant complication for some hiring agencies as approximately half of the applicants may not meet the stringent background requirements.

The process for hiring an experienced forensic scientist in latent prints is somewhat similar for the Las Vegas Metropolitan Police Department. Applicants are first selected based on minimal educational requirements and years of experience in latent prints. Those applicants meeting the minimum requirements are invited to participate in additional testing. The first test is a series of three practical exercises (work sample tests) that evaluate core latent print decisions. Applicants are required to (1) analyze latent prints and make suitability decisions, (2) analyze latent prints and indicate anatomical regions and distal orientation and (3) complete a set of latent print comparisons. If the candidate meets the performance metrics for the three practical exercises, they are invited to the oral board. The candidate must answer questions about their background and current issues in the field. Again, the purpose of this board is to assess candidate's oral communication skills, gauge their potential as an expert witness in court, and evaluate their organizational fit. As with the trainee, candidates that successfully pass the oral board are then referred to background. Not all experienced candidates pass the background requirements.

Discussion and recommendations

The goal of a selection and assessment program is to identify the individuals who will best perform a job but also who are a good fit with the organization. There are several steps for creating a more effective selection and assessment program for your organization. First, leverage the subject matter expertise internal to the organization. Human resources departments are staffed with human resource management professionals who are trained in the legal requirements and guidelines for selection and assessment programs. Human resources departments should include industrial organizational psychologists who have expertise in conducting job task analyses, developing and evaluating assessments, and creating valid selection and assessment programs. These individuals have the tools and experience to develop valid selection practices. Second, start small and then expand the selection program. Adding one or two assessments to a selection program may significantly improve the selection process. Third, consider which competencies are most easily and effectively measured as part of the selection process and which can be measured during the probationary period. For example, some attributes such as interpersonal skills and teamwork may be better measured during the probationary period. Finally, involve trained co-workers to participate in structured interviews and oral examinations. Again, the internal expertise of the human resources department can be leveraged to provide training in conducting structured interviews to potential co-workers.

Conclusions

Personnel selection and assessment is a critically important discipline that can ultimately help organizations and units identify which job candidates will succeed at performing the tasks and duties associated with a job and to hire these individuals. Bad hiring decisions can have significant repercussions on a forensic sciences unit. It can lead to lost revenue, lost productivity, and low morale. Conversely, a valid and reliable selection process can offer forensic laboratories several distinct benefits including improved job performance, reduced turnover, reduced training costs, reduced errors, reduced counter productivity, and enhanced legal defensibility. The purpose of this article was to review several popular personnel selection and assessment methods, the research associated with these methods, and discuss their strengths and weaknesses. Accordingly, we reviewed the importance of conducting a job analysis, which establishes a foundation upon which all other personnel selection and assessment decisions are made. Then, we discussed the role of predictors in the selection process and reviewed common predictor measurement techniques, including measures of cognitive ability, personality, and methods such as structured interviews and work sample tests that can be designed to assess different constructs of interest. We also discussed common job performance criteria and the importance of selecting relevant criteria for evaluating job performance. Finally, we reported two case studies of existing selection processes used in the forensic science domain and offered a set of grounded recommendations that a forensic scientist practitioner could use if they wanted to begin to build a more rigorous selection process for their unit.

Moving forward, and given the growing forensic science workforce, investing in selection and assessment practices seems like a promising way to add significant value to an organization and ensure labs and units are hiring the right people, with the right skill sets, into the right jobs.

Funding statement

This work was supported, in part, by the Forensic Technology Center of Excellence (Award 2016-MU-BX-K110), awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the author(s) and do not necessarily reflect those of the Department of Justice.