Human Factors in Financial Trading: An Analysis of Trading Incidents.

OBJECTIVE: This study tests the reliability of a system (FINANS) to collect and analyze incident reports in the financial trading domain and is guided by a human factors taxonomy used to describe error in the trading domain.
BACKGROUND: Research indicates the utility of applying human factors theory to understand error in finance, yet empirical research is lacking. We report on the development of the first system for capturing and analyzing human factors-related issues in operational trading incidents.
METHOD: In the first study, 20 incidents are analyzed by an expert user group against a referent standard to establish the reliability of FINANS. In the second study, 750 incidents are analyzed using distribution, mean, pathway, and associative analysis to describe the data.
RESULTS: Kappa scores indicate that categories within FINANS can be reliably used to identify and extract data on human factors-related problems underlying trading incidents. Approximately 1% of trades (n = 750) lead to an incident. Slip/lapse (61%), situation awareness (51%), and teamwork (40%) were found to be the most common problems underlying incidents. For the most serious incidents, problems in situation awareness and teamwork were most common.
CONCLUSION: We show that (a) experts in the trading domain can reliably and accurately code human factors in incidents, (b) 1% of trades incur error, and (c) poor teamwork skills and situation awareness underpin the most critical incidents. APPLICATION: This research provides data crucial for ameliorating risk within financial trading organizations, with implications for regulation and policy.

Introduction

Financial trading organizations buy and sell products (e.g., equities, physical commodities, financial options) in order to generate profit and optimize their portfolios. Large-scale failures (e.g., Société Général, UBS, JPMorgan) have resulted in multibillion-dollar fines from regulators and have undermined the global economy. Investigations into their causes have highlighted problems in organizational culture (e.g., risk taking) and “rogue traders” who manipulate rules and systems. Increasingly, however, the role of human factors–related issues in managing risk within financial trading are also considered, with parallels being drawn between the financial trading industry and other “high-risk” industries (Young, 2011). For example, investigations of trader performance have highlighted the importance of nontechnical skills (cognitive and social skills that underpin performance), human error (e.g., attention), and human–computer interfaces for influencing performance in financial trading (Ashby, Palermo, & Power, 2012; Fenton-O’Creevy, Nicholson, Soane, & Willman, 2003; Leaver & Reader, 2015; Willman, Fenton-O’Creevy, Nicholson, & Soane, 2002). This outcome is similar to many other high-risk sectors (e.g., nuclear power, aviation, health care), yet relatively little is known about the link between human factors–related problems and incidents in trading (e.g., how many incidents occur and the causes of them).

In this article, we report on the development and application of the Financial Incident Analysis System (FINANS). This is the first system developed to (a) collect voluntary operational trading incident reports (where trading activity results in an avoidable financial loss, for example, due to poor decision making or a compliance breach: Zhao & Olivera, 2006) from employees working on financial trading floors and (b) analyze incidents in order to identify the human factors issues reported within them. In this study, we test the reliability of FINANS and apply it to examine the nature and prevalence of incidents caused by human factors–related problems in a trading organization.

Human Factors and Financial Trading

Financial trading is an inherently complex and risky domain. Traders make high-stakes decisions within complex, large, noisy, high-pressured, and technologically advanced environments. They aim to generate profit for the organization and its stakeholders, and to do so, they must monitor market information (e.g., through screens), interact virtually and physically with other traders and stakeholders, make rapid investment decisions, and ensure that rules and procedures are followed (e.g., trading limits). Effective traders have good technical and nontechnical skills; however, the complexity and pressure of trading lead to error and risk taking (Leaver & Reader, 2015). This combination can result in “operational incidents,” whereby trading activity results in an avoidable financial loss (e.g., making a trade without assessing market-related risk) or compliance failures (e.g., breach of trading limits), which place the integrity of the financial organization at risk even if no loss has occurred (e.g., overexposure to volatile markets; Zhao & Olivera, 2006). Crucially, such events are typically caused not by rogue traders (employees making unapproved financial transactions) but by systemic problems across an organization (e.g., failure of the system to generate breach reports, inaccurate reporting on risk) that impair human performance (Leaver & Reader, 2015).

Thus, financial trading is increasingly conceptualized as similar to a high-risk industry (Sutcliffe, 2011; Young, 2011), with risk constantly being monitored and, when possible, reduced. However, unlike many high-risk industries, the success of financial trading organizations hinges on overt risk taking by traders (as it leads to a competitive advantage). This feature of the domain is consistent with Amalberti’s (2013) description of an “ultra-resilient” organization, where rather than engineering risk out of a system (e.g., through automation), risk is managed through improving employee skills and system design. Typically, this improvement is achieved through gathering data on mishaps and examining the role of human performance and system design in those incidents. Yet, to date, no system exists for capturing operational incidents in financial trading and analyzing the human factors–related issues that contribute to them (Leaver & Reader, 2015). To address this gap in the literature, we report on the development and application of the first tool for capturing and analyzing human factors–related operational incidents within financial trading: FINANS.

Using incident reports to investigate human factors in financial trading

Investigations into how human factors–related issues influence the management of risk within complex industries often begin with the examination of incidents (e.g., mishaps, near misses) and their causes (Barach & Small, 2000), because such analyses are useful for understanding recurrent and systemic problems in risk management. Incident-reporting systems can lead to insight on the number and types of incidents occurring within an organization, their consequences, and the complex network of issues (e.g., errors, skill gaps, resources) that underpin them. Incidents are often collected through incident-reporting systems, whereby employees submit a narrative text and/or structured report on incidents they observed or participated in. Reports describe the types of events that took place (e.g., mechanical, procedural), the personnel involved (e.g., identifying the teams), the activities leading to the incident (e.g., behaviors), and how the event was detected (e.g., system, observation). Incident reports can be anonymous or identified, can triangulate with existing monitoring systems (e.g., instrument data), or can be the primary source of data on mishaps (e.g., in health care). Crucially, to be effective, incident monitoring systems rely on good procedures for capturing incidents (e.g., independent, with nonpunitive results), high-quality data (e.g., freeform narratives that provide an ecological explanation of the event), strong analysis (through coding frameworks that identify causal factors), and robust feedback and learning mechanisms (e.g., for developing interventions, organizational learning) (Mahajan, 2010).

Incident-reporting systems have been used extensively to identify and understand safety problems in a number of high-risk industries. For example, the Aviation Safety Reporting System (ASRS; developed by the Federal Aviation Administration and National Aeronautics and Space Administration [NASA]) is a voluntary and confidential incident-reporting system used by pilots and engineers (via a Web-based platform) to report near misses and incidents (Billings, 1998; Helmreich, 2000). These data are used to understand the role of employees and systems in detecting and coping with incidents and to identify systemic and growing threats to safety. In other industries, for example, health care, incident-reporting systems have also become relatively commonplace although are generally not as developed as in aviation (Itoh, Omata, & Anderson, 2009; Wu, Provonost, & Morlock, 2002). For example, in health care, staff often experience cultural barriers in reporting incidents, and poor attitudes on incident reporting can limit institutional learning (Anderson, Kodate, Walters, & Dodds, 2013; Waring, 2005). Furthermore, in aviation, incident-reporting methodology has continuously evolved, for example, through the presence of a “callback” function that serves to gather additional information by interview prior to anonymization (NASA, 1999).

To understand and learn from incident reports, people tend to analyze them using reliable and theoretically derived taxonomies that classify the types of problems (e.g., error, skills, and systems) that contributed to an incident (Baker & Kronos, 2007; Barach & Small, 2000; Olsen, 2011; Vincent & Amalberti, 2016, Chapter 5). Such taxonomies should be tailored to the industry and should utilize human factors concepts to codify data on the types of incident experienced by operators (e.g., their technical nature, their outcomes), the workplace problems that lead to them (e.g., human–computer interfaces), and the skills and behaviors important for a work domain (e.g., in team vs. noncollaborative roles). The data collected can be used to collect headline data on incident occurrences within a given industry—for example, that in surgery, 43% of incidents involve team communication problems (Gawande, Zinner, Studdert, & Brennan, 2003) or that in military aviation, errors are more likely in rotary than in fixed-wing aircraft (Hooper & O’Hare, 2013). Furthermore, incident reporting is used to identify in-depth data on the causes of specific forms of mishap that can be used to develop interventions (e.g., new software, training), or for example, aspects of system design that lead to errors in the flight cockpit (Billings, 1999; Moura, Beer, Patelli, Lewis, & Knoll, 2016) or aspects of clinician behavior that either contributed to an adverse event (e.g., loss of situation awareness) or helped to avert it (e.g., teamwork skills; Schulz, Endlsey, Kochs, Gelb, & Wagner, 2013; Undre, Sevdalis, Healey, Darzi, & Vincent, 2007).

In summary, the incident-reporting literature highlights a number of principles for how incidents should be collected, analyzed, and used to influence safety-related practices. We apply these principles to develop a system for investigating operational incidents in financial trading.

FINANS

In the current study, we report on FINANS, which was designed to achieve three principle goals: first, to provide a standardized method for collecting data on operational incidents that occur on the trading floor; second, to develop a reliable method for analyzing and extracting human factors–related contributors to operational incidents; and third, to provide practical insight into how these contributors might be ameliorated. In the scope of this paper, we consider human factors as aspects of human performance and system design that contribute to problems in managing risk in financial trading.

FINANS comprises two parts. The first part is an “incident log” for capturing operational incidents on the trading floor. To recap, an incident in this context is an event that did lead or could have led to losses or unwanted market or credit exposure. Incidents can be wide ranging and can include technical systems failure (e.g., pricing tool failures), erroneous human input errors, misunderstandings of instructions or strategy between departments (e.g., between a trader and his or her risk department), and rule violations (e.g., late trade entry). Drawing on previous research, we use a Web-based design (Macrae, 2007; Mahajan, 2010; Wu et al., 2002). The system is accessed online, with reports being voluntary and anonymous (unless trading staff wish to identify themselves) due to the generally accepted negative culture toward “whistleblowing” and admitting error in the financial trading industry (Atkinson, Jones, & Eduardo, 2012; Keenan & Krueger, 1992).

Trading staff complete a reporting form, which includes a narrative section for eliciting a description of the incident in the staff’s own words and a drop-down menu section to elicit contextual details about the incident, for example, whether it was resolved or ongoing and the departments involved. The risk type drop-down menu provides a focus on key risks defined by the organization and helps to create specific and detailed reporting criteria that can evolve over time to meet the changing risks of the firm. This design utilized observations that the common language provided by taxonomies in addition to free-text narratives can retain the richness of narrative reports and at the same time allow for systematically organizing and analyzing the reported data (Macrae, 2016; Holden & Karsh, 2007). Figure 1 is a graphic representation of the reporting form.

Figure 1.

Graphic of the Financial Incident Analysis System (authors’ own rendering).

The second part of FINANS is a taxonomical system for interpreting incidents and near misses in terms of contributory factors. This system consists of three parts.

Based on incident analysis frameworks in aviation, military, and health care (Mitchell, Williamson, & Molesworth, 2016; O’Connor, O’Dea, & Melton, 2007; Wiener, 1993), a framework for codifying problems in nontechnical skills was developed. Nontechnical skills are the cognitive and social skills that complement a worker’s technical skills and underpin safe activity in high-risk environments (Flin et al., 2003). Research has shown their importance for managing risk on the trading floor. For example, the decision-making strategies of successful traders can be understood utilizing theory on situation awareness (e.g., information-gathering strategies, comprehension of complex market data, and course of action) and teamwork (e.g., communication on trading). The taxonomy was primarily based on a systematic review of nontechnical skills in financial trading (situation awareness, decision making, teamwork, leadership) and their association with good and poor trader performance (Leaver & Reader, 2015).

Drawing on error theory and other incident reporting systems (Reason, 1990; Saward & Stanton, 2015), we collected data on slips and lapses. Slips and lapses occur as a failure of execution of the intended task, whereby the actions deviate from the current intention (Reason, 1995). Slips are observed actions and are typically associated with attentional failures. Within FINANS, an example of this type of error is classified as “fat fingers,” whereby, for example, the trader accidently enters an extra zero to the pricing of a deal. Lapses, on the other hand, are associated with more internal events (e.g., failures in memory, distraction), and they can also influence performance in trading (e.g., during high-volume trading, the trader can forget to follow procedures, such as recording data on a trade).

Utilizing the ergonomics literature (Stanton, Salmon, & Rafferty, 2013), data on problems with human–computer interactions were also coded. Human–computer (or human–machine) interaction refers to the errors associated with the incomplete interpretation of system input and outputs as well as the flaws or inadequacies in system design that limits the user’s performance (Lang, Graesser, & Hemphill, 1991; Newell & Card, 1985; Rasmussen & Vicente, 1989). The successful interaction of human and computer is crucial in high-technology domains, such as trading, whereby the incorrect interpretation of data output (such as risk variation) can lead to traders’ taking the wrong position and potentially large losses or unwanted risk exposure.

It is notable the taxonomy consists of “category” and “element” levels. Categories function at a relatively generic level (e.g., situation awareness), and elements reflect aspects of activity specific to the trading floor environment that illustrate the categories (Flin, O’Connor, & Crichton, 2008). The list of categories and elements within the first-stage FINANS taxonomy is shown in Table 1.

Table 1:

FINANS Taxonomy

Category	Associated Elements
Situation awareness	• Attention (distraction, lack of concentration, divided or overly focused attention)• Gathering information (poorly organized information, not enough gathering of information)• Interpretation of information (miscomprehension, assumptions based on previous experience)• Anticipation (i.e., thinking ahead, judging how a situation will develop)• Other
Teamwork	• Role and responsibilities (e.g., unclear segregation of roles)• Communication and exchanging of information between team members• Shared understanding for goals and tasks• Coordination of shared activities• Solving conflicts (e.g., between team members and teams)• Knowledge sharing between teams• Other
Decision making	• Defining the problem• Cue recognition (e.g., finding and recognizing the cues to the decision)• Seeking advice on a decision• Noise and distraction (e.g., that reduce capacity to take a decision)• Bias and heuristics (e.g., overoptimism, overconfidence)• Other
Leadership	• Authority and assertiveness (e.g., taking command of a situation)• Listening• Prioritization of goals (e.g., team/organizational)• Managing workloads and resources• Monitoring activity and performance of team members• Maintain standards and ensuring procedures are followed• Other
Slip/lapse	• “Fat fingers”• Procedural (not following a protocol or following a protocol incorrectly)• Routinized task (e.g., a loss of concentration)• Forgetfulness (forgetting information or how to perform an activity)• Memory• Distraction• Other
Human–computer interface	• Use of the tools (e.g., spreadsheets)• Training on the tool• System did not detect the error• Design of the software and application• Maintenance and testing of the tool• Other

Note. FINANS = Financial Incident Analysis System.

Subject matter experts (SMEs) were involved in the development of the taxonomy, and a preliminary pilot (prior to Study 1) was used to determine whether SMEs agreed with the overall usefulness and fitness of the taxonomy to the incidents. For example, feedback from the SMEs led to the incorporation of further systems elements. To analyze operational incidents reported through FINANS, the subsequent procedure was followed. On an incident being electronically reported by a trading floor employee, a human factors expert reviewed the details and short description, and a risk type was assigned. Risk types are defined by the risk control team and are used for the categorization of the data in the monthly reporting of incidents and can change over time to address the current concerns of the organization (e.g., systems glitch, data entry error, late confirmation of a trade, physical risk leading to force majeure). The narrative text describing the incident was then analyzed using the FINANS taxonomy in order to identify any human factors–related antecedents to the incident.

To test and apply FINANS, we report on two studies using the system. The purposes of the studies were

to test the reliability (e.g., interrater reliability) of using the FINANS coding taxonomy to classify human factors–related problems described within operational incidents reported in financial trading (Study 1) and

to describe the nature and prevalence of human factors–related problems underlying operational incidents in financial trading (Study 2)

Study 1

In this study we test the reliability and usability of the FINANS coding taxonomy (Table 1) for classifying human factors–related problems described within operational incidents reports. Drawing on incidents collected through FINANS, we compare whether different coders perceive similar issues within an error report or incident when applying FINANS. Because FINANS is designed to be used by trading staff to analyze incidents (i.e., that they need not rely on a psychologist), and to reflect the types of errors and problems they experience, in the current study a group of expert trading staff (N = 19) applied the coding framework to analyze 20 incidents. To assess reliability, we examine the interrater reliability of coding by trading staff for the system as a whole, individual categories, and the elements underpinning each category (Butterfield, Borgen, Amundson, & Maglio, 2005). We also examine whether expert participants analyzed incidents in a similar fashion to human factors experts (through creating a “referent” standard) in order to assess whether domain experts unfamiliar with human factors concepts can use the taxonomy in the manner intended (Gillespie & Reader, in press).

Method

To test the reliability of the taxonomical system for interpreting incidents that occur on the floor, an expert user group was recruited from within the participating organization: a leading energy trading firm active in both physical and financial commodity markets. Hedging products include forward contracts, swaps, vanilla options, over-the-counter and exchange-based transactions, and derivatives and futures contracts. Approximately 37,500 transactions are booked with the exchange or over the counter annually on a spot (prompt), medium (futures/forward), or long-term (contract) basis. The sample consisted of three trading managers, two trading supervisors, and 14 midlevel trading staff. Using the FINANS taxonomy, the user group analyzed 20 incidents selected from the incident log. Incidents were selected on the following criteria:

At least one of the FINANS categories was evident in the scenario.

Each of the teams was represented.

The incidents covered frequent and infrequent error types.

The scenarios were presented sequentially and through the Web-based interface. Participants read each scenario and, using an online coding form, indicated which FINANS categories and subcategories (e.g., elements) were contributory to the scenario. In addition, a referent standard was developed by two human factors experts, who coded the 20 incidents separately and then reviewed the incidents again to resolve any differences in coding (and to outline a final set of codes for each incident).

Prior to coding, participants were given a 1.5-hr background tutorial on human factors research and the concepts underlying the FINANS system. Although this tutorial falls below the recommended training time of 3 hr (O’Connor et al., 2002), time constraints in releasing trading staff from their work during market hours (and also asking them to code 20 incidents) meant training was limited. To compensate for this limitation, the initial training was supplemented with a training document distributed to each participant detailing human factors definitions and examples of incident analysis. Moreover, the principal study investigator, whom questions could be directed to, was present in the workplace.

Analysis

The data analysis consisted of comparisons between respondents within the user group (to test interrater reliability) and between respondents and the referent standard.

We ran the following analyses. First, to examine the interrater reliability of the referent users (e.g., the human factors experts), we applied a Cohen’s kappa. Cohen’s kappa coefficient is a statistic that measures interrater agreement for two raters for qualitative (categorical) items and takes into account the agreement that may occur by chance (McHugh, 2012). Second, to establish interrater reliability among the expert users, we applied a Fleiss kappa (Fleiss kappa is applied to extract the nominal scale agreement across many raters; Fleiss, 1971). We also used this statistic to examine the interrater reliability between the referent ratings and the expert user group. It is suggested that kappa results can be interpreted as values k ≤ 0 indicating no agreement; 0.01 ≤ k ≤ 0.20, none to slight; 0.21 ≤ k ≤ 0.40, fair; 0.41 ≤ k ≤ 0.60, moderate; 0.61 ≤ k ≤ 0.80, substantial; and 0.81 ≤ k ≤ 1.00, almost perfect agreement (Fleiss, Cohen, & Everitt, 1969; McHugh, 2012).

Results

First, we examined the reliability of coding for the two human factors experts, from which the referent standard was generated (k = 0.894).

Second, we examined the reliability of coding within the expert user group. Overall, we found good reliability for applying the FINANS taxonomy at the categorical level (k = 0.840). However, greater variance was found in the reliability of coding at the element level (k = 0.453). This finding is consistent with previous empirical studies in other high-risk domains (Baker & Krokos, 2007; Yule, Flin, Paterson-Brown, Maran, & Rowley, 2006). We summarize the findings next through considering the categories and subcategories of the taxonomy that had low versus high reliability.

Low reliability

Consistently low reliability was noted across the element subcategories: procedural (slip/lapse category), k = 0.400; authority (leadership category), k = 0.400; roles and responsibilities (teamwork category), k = 0.400; and anticipation (situation awareness category), k = 0.348. Elements that were not able to be calculated via the kappa method due to an absence of data (e.g., they were never chosen in the coding exercise), were problem definition, cue recognition, selecting a course of action, noise and distraction (all decision making), use of tools (human–machine interface category), solving conflicts (teamwork category), prioritization, monitoring, listening, and managing workload and resources (within the leadership category).

High reliability

All categories were reliably estimated with a range of kappa scores from k = 1.0 (decision making) to k = 0.8 (slip/lapse). Elements were also found to be statistically significant, with interrater reliability ranging from k = 0.655 (human–machine interface elements) to k = 0.859 (teamwork elements). The within-group elements did not test as reliably across all elements within the cases. The highly reliable elements are gathering information, k = 0.8 (situation awareness); system design, k = 1.0; maintenance of the system, k = 0.696; training of the tool, k = 0.696; detection of the tool, k = 0.696 (human–machine interface); knowledge sharing, k = 1.0; communication, k = 1.0; coordination, k = 0.769; shared understanding, k = 1.0 (teamwork); maintaining standards and procedures (leadership), k = 0.65; fat fingers, k = 0.783; forgetfulness, k = 0.737; and routine task k = 1.0 (slip/lapse category).

Overall, high reliability was observed for the category and elements within the teamwork, slip/lapse, situation awareness, and human–machine interface skill sets. Lower reliability was observed for the leadership and decision-making categories.

Finally, the kappa agreement when analyzing the reliability between the reference ratings (n = 2) and the expert ratings (n = 19) for each FINANS category was good (k = 0.871).

Discussion

This study was designed to test the reliability of the FINANS taxonomy for codifying incident reports in the financial trading domain. Given the limitations in training, the results are encouraging and suggest that the human factors problems underlying error in the financial domain can be reliably identified and extracted by trained experts in financial trading. In establishing statistically significant reliability, we can confirm that experts generally agree on the human factors problems underlying operational incidents in financial trading and that the frame of reference held by these experts can be validated (Leeds & Griffith, 2001). This finding is important for demonstrating the appropriateness of FINANS for analyzing operational incidents within financial trading (i.e., it fits to the needs of the domain and its users) and indicates it can be administered with light-touch support. Most crucially, FINANS provides a reliable tool through which to examine the role and extent of human factors–related problems underlying operational incidents in financial trading. This tool has the potential to provide data crucial for identifying, understanding, and ameliorating risk within financial trading organizations. Yet, as indicated in the results, some of the categories and subcategories within FINANS tend either to not be used reliably (e.g., the procedural element within slip/lapse category) or to be used very minimally. This finding indicates FINANS requires further refinement, and we examine this issue further in study 2.

Study 2

In Study 2 we examine the nature and prevalence of human factors–related problems underlying operational incidents in financial trading. We refer to the incidents as “operational” to remain consistent with terminology in the financial domain used to describe error reporting and investigation. At present, relatively little is known about the types of human factors–related incidents that occur in financial trading or, indeed, the number of incidents that occur relative to total transactions. This finding compares poorly to other domains, for example, aviation, where the number of incidents and fatalities in relation to the number of flights per year is systematically documented (Boeing, 2014). We used FINANS to collect and analyze operational incidents in a large financial trading company over a period of 2 years. The analysis was conducted with four principle aims: (a) to provide data on the number of trades that lead to an incident, (b) to identify the distribution of human factors problems within the cases, (c) to provide evidence on the outcomes of these human factors problems, and (d) to explore the co-occurrence of human factors codes in the data set (i.e., clusters of problems that occur together). In addition to these aims, we utilized the larger data set to further refine the FINANS taxonomy.

FINANS was used to collect incident reports in the participating organization over a period of 2 years (from January 2013 until January 2015). Prior to study commencement, and with the support of the organization, trading floor staff were given presentations of the incident collection log as well as practice entries and demonstrations by a human factors expert (separate to the reliability study, although all participants in the reliability study were present during the briefings). Presentations and demonstrations were approximately 1 hr in duration (given four times due to turnover in teams and “maturing” incident reports). Following each reporting month, a trained human factors expert provided feedback reports (e.g., histogram and patterns of events by risk type, deconstructed complex events, incidents, and solutions for four to five logged incidents from the month of reporting) to the participating staff and management. Over this period, approximately 750 unique incident reports (i.e., each incident reporting on a problematic trade was different) were collected and deemed suitable for analysis (e.g., clear text).

Of the 750 incidents, the lead author coded all the cases; a further 375 (50%) cases were coded by the second author to provide a reliability assessment for coding. These cases were randomly selected from the batch. The coding process was made up of 8 steps: (1) identification of the incident type (e.g., slip, mistake, violation), (2) selection of the relevant human factors category (e.g., situation awareness, decision making, teamwork, leadership, human–computer interface, or slip/lapse), (3) the selection of the relevant subcategory (e.g., element) of nontechnical skills (e.g., if situation awareness is chosen as a main category, the element[s] can be selected from distraction, gathering information, interpreting information, and anticipation of future states), (4) identification of single team or multiple team, (5) identification of an ongoing state or isolated nature of the incident, (6) reporting whether the incident was a near miss or a failure, (7) identification of the trigger of the incident (e.g., a text box entry), and (8) filling in the blanks in the following sentence: “The main cause of the issue is [blank], and is caused by [blank].”

Descriptive analysis

First, we calculated the number of erroneous trades identified by the system in relation to the total number of trades within the organization. Second, we used Cohen’s kappa to calculate the reliability of the second coder against the first coder for 375 cases. Third, we described the distribution of human factors problems using frequency and mean calculations for the categories and elements with FINANS, including category and elements that are not reliably coded or not coded for in the n = 750 cases.

Serious incident analysis

Next, we adopted a pathway analysis within SPSS to determine whether the incidents classified as near miss or failure had a common set of human factors antecedents. Pathway analyses describe all the variations of the coded data and then are used to predict whether some codes or sets of codes significantly predict an outcome (e.g., financial loss).

Associative analysis

Third, through bivariate correlation and backward likelihood ratios, we conducted an associative analysis to examine co-occurrence of FINANS category codes within incident reports (e.g., to establish whether there are certain patterns of codes that occur together). The importance of investigating the co-occurrence of codes was revealed when we observed how the data were repeatedly coded for multiple human factors codes, and thus this part of the investigation is exploratory.

Financial trading staff reported 750 incident reports through FINANS. This number equates to 1.08% of transactions within the company. Across the total data set, 70% of incidents were a near miss (an error did occur but was detected and fixed by system controls), and the majority of incidents (90%) involved activity distributed across more than one team.

Of the incidents coded by both the lead author and second author (n = 375), good overall reliability was found using Cohen’s kappa (k = 0.78). All incidents had at least one code from the FINANS taxonomy applied to explain the incident (e.g., incidents can be coded as multiple categories and elements). At the category level, the reliability was generally good, with the exception of decision making. Substantial reliability was determined for leadership (k = 0.83), teamwork (k = 0.79), slip/lapse (k = 0.72), situation awareness (k = 0.72), and human–computer interface (k = 0.67). Moderate reliability was determined for decision making (k = 0.49). Elements were also coded for each case. At the element level, the reliability was disparate, ranging from good to poor or not applicable. High-reliability elements included maintenance and testing (k = 0.77; human–computer interface category), roles and responsibilities (k = 0.62; teamwork category), and maintaining standards (k = 0.65; leadership category). Acceptable-reliability elements included attention (k = 0.57; situation awareness) and communication (k = 0.48; teamwork). Similar to Study 1, several elements were never or rarely coded, which led to poor reliability (k < 0.4). These elements included bias and heuristics, listening, goal prioritization, managing workload, monitoring activity, memory, and training; and many elements were coded interchangeably, which led to poor reliability. The implications are explored in the discussion.

In terms of applying FINANS taxonomy to the incidents, Table 2 provides a fine-grained analysis of the frequency and percentage for each human factors category and element used to classify human errors. To illustrate the context of data collection (and the potential for intervention), and the types of problems being codified using the FINANS taxonomy, qualitative examples are included within Table 2.

Table 2:

Financial Trading Human Factors Taxonomy Descriptions and Frequency

Category and Element Skill	Description	Example of an Incident	Element Coding Frequency When the Category Is Coded
Situation awareness
Anticipation	Comprehending the situation, understanding what might happen next	Downloading deals with incorrect volume units, leading to incorrect current risk projection	57 (15%)
Attention	Maintaining concentration and avoiding distraction	Inverting the price and volume of the trade in the system	213 (56%)
Gathering info	Perception of the elements in the current situation (e.g., visual information, screens, auditory information)	Volumes in the system not matching the physical deal sheet	84 (22%)
Interpreting info	Processing the current information to make sense of the current situation in order to understand what is going on (involves the interpretation of various cues)	Hedging a flat position due to inaccurate interpretation of information in the system	28 (7%)
Total			382 (51%)
Teamwork
Communication	Exchange of information, feedback or response, ideas and feelings	A change in contractual specs poorly communicated between the teams	53 (18%)
Coordination	Coordination within and between teams, improved by equal distribution of task work, monitoring each other, and effective exchange of information	Two members of the same team duplicating the data entry during work flow	87 (30%)
Roles and responsibilities	Lack of adherence to clearly and appropriately segregated roles	Weak definition of business rules in the system leads to the incorrect assignment of access	75 (26%)
Shared understanding	Knowledge held by members of a team that enable them to form accurate explanations and expectations for the task, to coordinate their actions, and to adapt their behaviors accordingly	Validating an erroneous buy trade when the desk wants to short a product	78 (27%)
Total			293 (40%)
Decision making
Bias and heuristics	Simple rule people use to form judgments and make decisions (e.g., availability, representativeness, anchoring and adjustment, affect)	Undervaluing the information provided in a credit risk report	17 (63%)
Cue recognition	The primary situation assessment (e.g., what is the problem) through the recognition and interpretation of environmental cues	Currency units not equal to geographical trade location	7 (26%)
Problem definition	Decision-making method (e.g., what should I do)	Recognizing the input value is incorrect, using the closest settle price as a placeholder until the true value could be determined	3 (11%)
Total			27 (3.6%)
Leadership
Authority and assertiveness	Ability to create a proper challenge and response atmosphere by balancing assertiveness and team member participation and being prepared to take decisive action	Failing to generate a timely risk assessment and assignment of trading limits of a new trading instrument	2 (2%)
Maintaining standards	Compliance with essential standards (e.g., operating procedures)	Not entering trades on the transaction date	64 (62%)
Manage workload	Understanding the basic contributors to workload and developing the skills of organizing task sharing to avoid workload peaks and dips	Mismanaging staffing schedules, leading to task overload during end-of-month procedures	9 (8%)
Monitor activity	Maintain team focus and monitor the output of the team	Underutilizing the daily reports to cross-check trading limit breach levels (e.g., 80%) with activity forecasts	29 (28%)
Total			104 (14%)
Slip/lapse
Distraction	Avoiding the prevention of concentration	Entering the wrong affair for a number of trades	39 (9%)
“Fat fingers”	The mistyping or mis-entry of data information	Entering an extra digit on the price (e.g., 0.01 vs. 0.1)	185 (40%)
Forgetfulness	A lapse of memory	Updating contractual quantities without amending price details	51 (11%)
Memory	The faculty by which the mind stores and remembers information	Skipping a step in the procedure	27 (6%)
Procedure	An established or official way of doing things (written or oral)	The fitness of the procedures to the task (e.g., adaptation to new changing product definitions)	83 (18%)
Routine task	Task work that is commonplace or must be completed at regular intervals (e.g., data input)	Adherence to daily procedural tasks (e.g., time stamp on all deals)	74 (16%)
Total			459 (61%)
Human–computer interaction
Maintenance and testing	The system is tested regularly and adaptations are timely to reflect the task work	Multiple downloads of electronic platform transactions by the broker	52 (31%)
Software design	The design of the software does not inhibit task work (e.g., low complexity, interface-friendly)	Transactions for Product A entered on the market for valuation of Product B	9 (5%)
System detection	The system controls work properly	System fails to send out timely and accurate breach reports	40 (24%)
Training	The team members involved in the task have sufficient experience and training	Team member lacks the ability to cross-check data output from the system with confidence	32 (19%)
Use of tools	The team members can navigate the system with proficiency	Ability to enter a new product transaction in the system independently and model the risk	37 (22%)
Total			170 (23%)

Table 2 shows that over half of incidents involve a slip/lapse or situation awareness problem. Within these subcategories, the most common elements were fat fingers (40%) and attention (56%). Teamwork problems were identified in 40% of incidents, with coordination being the most commonly coded element (30%). The least coded category was decision making (3.6%). In terms of elements, the most commonly coded was attention (213), followed by fat fingers (185) and coordination (87). Again, some elements were never coded; these included noise, seeking advice on a decision, and the prioritization of goals. Similarly, some elements were rarely coded, such as authority and assertiveness, problem definition, software design, and manage workload. Furthermore, elements within more commonly applied categories (e.g., distraction within the slip/lapse category) were also rarely used.

In terms of refining FINANS for future use, a number of observations might be made. Table 1 indicates a number of rarely occurring elements (e.g., training in human–computer interaction, authority and assertiveness in leadership). This finding is consistent with the data in Study 1, and these elements might be removed or amalgamated with other elements (e.g., use of tools, maintaining standards) in future iterations of FINANS. Furthermore, the larger reliability exercise conducted for Study 2 indicates some subcategories to demonstrate low reliability as they are used interchangeably, in particular, fat fingers and routine task, and forgetfulness and attention (within slip/lapse). In order to strengthen the reliability of the tool, the data indicate that these codes might also be combined. Last, although the literature search that informed the taxonomy used in this study does not include stress management, there is a likely benefit in studying the influence of stress and fatigue upon trading staff performance. For example, research shows that traders are less likely to make use of stress coping strategies despite stress resistance being identified as a characteristic of good traders.

In the next analysis we investigated whether the incidents had a common set of antecedents. In the coding framework, incident outcomes were coded as a near miss or failure, and we focused on the distinction between these incidents. Specifically, we assessed whether there were particular human factors issues leading to near misses (system controls detected and corrected the error) or actual failure (systems controls failed to detect the error). For example, the data collected through FINANS indicate that errors that typically originate in the front office may pass through the “layers of defense” in the middle office and then are either detected at the tertiary cross-check by the back office team (leading to a near miss) or left undetected.

This finding indicates that particular aspects of team coordination lead to actual losses, and to ascertain whether a distinct pattern of contributory factors was underlying near misses or failures, we applied a pathway analysis to the data set in SPSS. This pathway analysis describes all the variations of the coded data and then is used to predict whether some codes or sets of codes significantly predict an outcome. Figure 2 illustrates the relationship between the human factors categories and how they are related to the outcomes (e.g., near miss or failure).

Figure 2.

Sets of human factors that lead to near miss or failure in operational trading incidents. SA = situation awareness; TMWK = teamwork; DM = decision making; LDSHP = leadership; SL = slip/lapse; HCI = human–computer interaction.

Figure 2 reveals two significant relationships as a function of outcome (e.g., near miss or failure). First, the interaction between situation awareness and teamwork most often predicts a failure outcome, and second, coding for slip/lapse alone commonly results in a near-miss outcome (indicating it is noticed and prevented by other trading staff). For the most serious incidents, situation awareness and teamwork factors are most commonly attributed to these outcomes. This observation led us to conduct an exploratory analysis into the particular patterns of categories within FINANS that occur together within incidents.

Spearman correlation coefficient is used to achieve the bivariate correlation between the (noncontinuous) variables (Hauke & Kossowski, 2011), and we used this statistic to examine the associations between FINANS categories applied to the incident data. The results of this analysis are presented in Table 3. This analysis reveals patterns of association or lack of association between certain categories, and we consider the findings next.

Table 3:

Bivariate Correlation of Incidents (n = 750)

	SA	TMWK	DM	LDSHP	SL	HCI
SA
Correlation coefficient	1.000
Significance (two tailed)	.
n	750.000
TMWK
Correlation coefficient	.370	1.000
Significance (two tailed)	.000	.
n	750.000	750.000
DM
Correlation coefficient	.061	.080	1.000
Significance (two tailed)	.096	.029	.
n	750.000	750.000	750.000
LDSHP
Correlation coefficient	.131	.288	.171	1.000
Significance (two tailed)	.000	.000	.000	.
n	.000	750.000	750.000	750.000
SL
Correlation coefficient	−.179	−.445	−.184	−.322	1.000
Significance (two tailed)	.000	.000	.000	.000	.
n	750.000	750.000	750.000	750.000	750.000
HCI
Correlation coefficient	−.072	−.071	−.013	−.102	−.344	1.000
Significance (two tailed)	.049	.053	.725	.005	.000	.
n	750.000	750.000	750.000	750.000	750.000	750.000

Note. SA = situation awareness; TMWK = teamwork; DM = decision making; LDSHP = leadership; SL = slip/lapse; HCI = human–computer interaction.

Codes that occur together

The strongest positive correlation was found between teamwork and situation awareness. This correlation means that when an event is coded for teamwork, it is significantly likely that situation awareness will also be coded for (and vice versa). This finding indicates that when breakdowns in teamwork occur, it is likely that a breakdown in situation awareness has also occurred. This coupling occurs significantly within the data set, indicating its presence to increase the likelihood for error in the trading domain. This finding is consistent with previous research in the trading domain showing understanding and sharing insight into risk is underpinned by the distribution of cognition and understanding across teams—often termed “team situation awareness” (Endsley & Jones, 2013; Leaver & Reader, 2015; Michel, 2007). The second most common association was between teamwork and leadership. This close association is unsurprising, given the current evidence that leadership behaviors in the trading domain are determined by situational factors (e.g., incoming team revenue) and that monitoring fluctuates according to team performance (Willman et al., 2002; Willman, O’Creevy, Nicholson, & Soane, 2001).

Codes that do not occur together

There are two striking non-associations that emerge from the data set. First, slip/lapse is significantly likely to occur alone than with any other category of human factors, and the strongest opposition is with teamwork. This finding exemplifies the nature of slip/lapse incidents, which are typically easily detectable by the many layers of defense built into the system and typically low complexity (e.g., characterized by a fat-fingers incident). The second observation from the data set is that human–computer interaction also occurs alone more often than with other categories. This finding indicates that when faults in the operating system or equipment occur, they are detected and reported before elevating in complexity (e.g., interrupting team processes). Inconsistent with the literature on human–computer interaction, an association between situation awareness and human–computer interaction was not observed (Weyers, Burkolter, Kluge, & Luther, 2010).

Study 2 revealed approximately 1% of financial trades annually to incur some form of error. This figure is likely a conservative estimate due to potential underreporting and is less than in domains such as health care but greater than in aviation (Boeing, 2014; de Vries, Ramrattan, Smorenburg, Gouma, & Boermeester, 2008). Consistent with the notion of financial trading as a high-risk industry, FINANS provides a practical tool for identifying and understanding the causes of error. In regards to generalizability to other financial organizations, the research was conducted on a large commodity-trading floor, with generally analogous features (personnel, systems, and organizational design) to other trading organizations (e.g., banks). Yet this generalizability requires examination, and FINANS should be used, albeit cautiously, to inform the development of incident analysis in similar trading floor environments.

In terms of the human factors problems underlying critical incidents in financial trading, slip/lapse-related errors (e.g., fat fingers) was the most frequently coded category, occurred often in isolation from other human factors problems (e.g., teamwork), and were more likely to be associated with near-miss outcomes (indicating errors were being caught by trading staff). It is perhaps not surprising that slip/lapse errors are more likely to be reported in the operational incident log than others (e.g., decision-making skills), as they are relatively easy to detect retrospectively, and participants may show a bias for reporting less punitive, easily detected events (e.g., fat fingers, following procedures) than complex, punitive issues (e.g., failing to consider options). In general, slip/lapse problems did not lead to serious incidents, as they were often fixed quickly through organizational procedures (e.g., team cross-checks), and this finding has also been observed in industries such as aviation (Vincent & Amalberti, 2016, Chapter 5).

In addition, we observed that a significant proportion of critical errors originated from failings in situation awareness and teamwork processes. This finding may indicate team-based processes, such as communication and coordination (e.g., cross-checking of information, monitoring of information), to influence team situation awareness on the trading floor and resonates with research in health care and aviation (Jentsch, Barnett, Bowers, & Salas, 1999; Reader, Flin, Mearns, & Cuthbertson, 2011). Thus, future research may focus on how teamwork and situation awareness interact to influence performance on the trading floor, for example, how errors migrate and develop on the trading floor (e.g., typical error migration is from the front office, through the middle office, to the back office) and awareness of interdependencies among team members.

Relatively few incidents were reported as having leadership or decision-making problems, and this finding is contrary to experimental work in the finance domain. The analysis presented in this two-phase study reveals that decision making is a less present indicator of team performance in the trading domain, and this finding may reflect limitations in the abilities of trading staff to self-monitor decision-making activities. Also, the absence of decision making may indicate that incident reporting may not be an optimal way to collect data on decision making in financial trading, and other forms of study (e.g., observations) may be more useful. In terms of leadership, this category might be conceptualized as a more “distal” cause of incidents (e.g., setting and maintaining standards) and perhaps more difficult to isolate as a contributory factor to incidents.

Finally, the findings of this study might lend themselves to develop interventions and inform regulators on the causes of problems in risk management in financial trading, for example, in terms of training programs (e.g., on interdependencies between teams), software design, and changes to systems and procedures.

Study Limitations

The results are constrained by the nature of the incident reporting, which is susceptible to underreporting and incomplete information about incidents (O’Connor et al., 2007). Incident reporting in trading is limited by the need for an individual to be aware that the event has occurred, his or her limited perspective on the incident, and his or her motivation to report. Furthermore, for Study 1, experts undertook a relatively short training exercise, potentially affecting their ability to accurately code incidents—in the future, it is suggested a longer training exercise is utilized. For Study 2, a further limitation was that only one coder analyzed the incidents (with a second coding half of the incidents to assess for interrater reliability), and the data analysis was constrained by the clarity of the text and the potential biases of trading staff in recalling the incident. Finally, the FINANS taxonomy may require further development. Issues such as stress, fatigue, and organizational culture were not examined, and the reliability analysis indicated scope for improving the FINANS taxonomy (which will be the focus of future work).

Concluding Remarks

This study reports the first system for capturing operational incidents on the trading floor and analyzing the human factors–related issues that led to them. Through two studies, we found that experts in the trading domain can reliably and accurately code human factors underlying in incidents in financial trading and that approximately 1% of all trades incur error. Although slip/lapse is the most common factor underlying incidents, problems in teamwork and situation awareness underpin the most critical incidents. In order to develop a more fine-grained analysis of the nature of these errors, authors of future research should aim to further improve FINANS and to identify the specific skills and conditions that lead to effective risk management on the trading floor.

Human factors problems underlying error in the financial domain can be reliably identified and extracted by trained experts in financial trading using the Financial Incident Analysis System (FINANS).

FINANS is both appropriate for analyzing operational incidents within financial trading (i.e., it fits to the needs of the domain and its users) and can be administered in financial trading organizations without the assistance of psychologists to monitor and analyze data.

FINANS provides a reliable tool through which to examine the role and extent of human factors–related problems underlying operational incidents in financial trading. This tool has the potential to provide data crucial for identifying, understanding, and ameliorating risk within financial trading organizations.

Approximately 1% of trades incur some form of error per year, which provides a useful benchmark for financial organizations against other high-risk industries.

A significant proportion of the underlying causes of the most critical errors originates from failings in situation awareness and teamwork processes. In particular, we find a significant likelihood of teamwork and situation awareness to occur together and lead to critical outcomes (e.g., loss events).