Engaging Gatekeepers, Optimizing Decision Making, and Mitigating Bias: Design Specifications for Systemic Diversity Interventions.

In this contribution to the Journal of Applied Behavioral Science Special Issue on Understanding Diversity Dynamics in Systems: Social Equality as an Organization Change Issue, I develop and describe design specifications for systemic diversity interventions in upward mobility career systems, aimed at optimizing decision making through mitigating bias by engaging gatekeepers. These interventions address the paradox of meritocracy that underlies the surprising lack of diversity at the top of the career pyramid in these systems. I ground the design specifications in the limited empirical evidence on "what works" in systemic interventions. Specifically, I describe examples from interventions in academic settings, including a bias literacy program, participatory modeling, and participant observation. The design specifications, paired with inspirational examples of successful interventions, should assist diversity officers and consultants in designing and implementing interventions to promote the advancement to and representation of nondominant group members at the top of the organizational hierarchy.

Meritocracy is a principle or ideal that prescribes that only the most deserving individuals are rewarded. As such, meritocracy can operate accurately only in an unbiased system.

A strong belief in meritocracy underlies the way we perceive success in society, in organizations, and in individual careers. We like to believe that those who make it to the top do so because they have more merit (i.e., worth, superior quality) than those who do not. Put differently, “in true meritocratic systems everyone has an equal chance to advance and obtain rewards based on their individual merits and efforts, regardless of their gender, race, class, or other non-merit factors” (Castilla & Benard, 2010, p. 543). In many organizations, and especially in those that are characterized by a pyramid structure and matching notions of careers success in terms of climbing the ladder, organizational members tend to think this is not only how the career system should operate but also how it does operate. As a consequence, the distribution of success (e.g., rewards, promotions) across members from different societal groups, especially when looking at the top of the pyramid, is generally perceived to reflect the true distribution of merit between group members (Block, 2016).

However, reward allocation and performance evaluation practices that appear to be meritocratic (Joshi, Son, & Roh, 2015) can result in an unequal distribution of success in favor of some compared with others, regardless of the actual distribution of merit. This means that the way we typically assess merit is biased in favor of dominant group members in terms of the criteria, the tests, and/or the evaluation process (Son Hing et al., 2002).The biased assessment of merit, through which superior qualities are ascribed to dominant group members, is thus rather a sign of bias for (or favoritism) instead of bias against (or discrimination; DiTomaso, 2015).

The paradox of meritocracy holds that when an organizational culture actively promotes meritocracy, decision makers in that organization may ironically show greater bias in favor of the dominant group (Castilla & Benard, 2010). In other words, if managers believe that the way people in the organization are selected and promoted is meritocratic, their decisions about the careers of others are more biased and less based on merit (Castilla, 2016). Efforts to increase diversity in terms of the representation and advancement of nondominant group members at higher hierarchical levels, such as diversity training, networking, mentoring, and introducing work–family policies, may not address the decision processes behind performance evaluation and reward allocation (Joshi et al., 2015; Morse, 2016). Such diversity efforts thus do not expose the paradox of meritocracy nor the biased assessment of merit. Indeed, diversity interventions that do try to challenge and change reward allocation and performance evaluation practices are often met with considerable resistance, even from nondominant group members. A critical revision of how we assess merit might “lower standards and compromise qualifications” (Krefting, 2003, p. 261). In true form, the paradox of meritocracy is not an easy one to resolve—and promoting diversity by addressing meritocracy demands awareness of the simultaneous existence of tensions, conflicting demands, and opposing perspectives (Eisenhardt, 2000).

As is evident from the call for papers for this Special Issue, diversity interventions in organizations often fail to change the racial and gender composition of senior leadership, and do not diminish disparities in workplace outcomes (Block & Noumair, 2015). In addition to their limited effectiveness and meeting resistance, interventions can produce unintended or ironic effects, and may even backfire (Dobbin & Kalev, 2016; Kaiser et al., 2013). Having diversity policies in place may result in an illusion of fairness, supporting the legitimization of the status quo and the normalization of inequality by the dominant group (Kaiser et al., 2013). Specifically, if success or effectiveness of diversity interventions in upward mobility career systems is measured by diversity at the top of the pyramid, many common diversity interventions introduced in such systems fail to produce the desired effect. I argue that there are three main reasons why this is the case: The design of these interventions does not incorporate a systems perspective, the intervention is not aimed at improving decision making and mitigating bias, and the target group does not include gatekeepers. If diversity interventions would be designed more effectively, the representation of nondominant group members at all steps of the intraorganizational career (i.e., after organizational entry) would be improved, including less homogeneity at the top.

In this contribution to the Special Issue, I develop design specifications for systemic diversity interventions in upward mobility career systems, aimed at optimizing decision making, mitigating bias, and engaging gatekeepers. As argued by Moss-Racusin et al. (2014), only a scientific approach to diversity interventions, aimed at reducing biases, will ultimately increase meritocracy. I will ground these specifications in the limited but existing empirical evidence on systemic diversity interventions. Building on systematic reviews of diversity training, I will illustrate these specifications using examples of such interventions in academic settings, namely bias literacy, participatory modeling, and participant observation.

Taking a Systems Perspective

The paradox of meritocracy is alive and well under conditions where careers are primarily evaluated in terms of upward mobility (Vinkenburg & Weber, 2012). A prime example of such conditions is an “up-or-out” career system, common to academia, the military, and professional service firms (PSFs). Careers in such settings are subjected to a linear promotion process shaped by regular and elaborate performance measurements based on predetermined criteria and fixed time frames. In order to sustain the pyramidal shape of the hierarchy, those not advancing to the next career level according to the given parameters are “counseled to leave” (Ossenkop, Vinkenburg, Jansen, & Ghorashi, 2015b). Generally, an up-or-out system strongly suggests meritocracy through its reliance on formal procedures and transparent criteria. Indeed, in many high-prestige settings such as academia and law, performance evaluations are based on relatively measurable criteria or indicators such as billable hours or research productivity (Joshi et al., 2015). However, as in other types of organizations, the selection and promotion decision-making process remains rather subjective (Perry, Davis-Blake, & Kulik, 1994). Scores on seemingly objective criteria (e.g., billable hours, impact factor), are not decisive in the process, but only serve as a threshold for being considered, are systematically overestimated for dominant group members, or even dropped from the deliberations altogether (Van den Brink, Holgersson, Linghag, & Deé, 2016; Vinkenburg, Jansen, Dries, & Pepermans, 2014). Decision making on promotion to the highest levels of organizational hierarchies, due to structural conditions and situational components, can be characterized as especially imperfect (Vinkenburg et al., 2014). Room for bias appears when managers have (or use) discretion in translating performance scores to pay and promotion decisions in PSFs (Ossenkop et al., 2015b). In trying to explain discrepancies between outcomes of performance evaluations versus reward allocations, Joshi et al. (2015) label decisions on pay and promotion in prestigious settings as “highly subjective, opaque, and adversarial” (p. 1533). Up-or-out promotion norms exacerbate the situation by raising the stakes—as not getting promoted on time means leaving. Under such conditions, and regardless of actual merit, the likelihood of upward mobility is higher for dominant group members than for others. The consequence is the typical image of almost exclusively White men at the top of the hierarchy.

In a career system that thus creates and maintains outcome disparities between members of different social groups, and where the paradox of meritocracy feeds into a limited awareness of biased decision making (Block, 2016), breaking the cycle is complicated. While many organizations that embrace an up-or-out career system recruit and select a heterogeneous or diverse pool of entry-level candidates, after 12 to 15 years typically a disproportionate number of dominant group members will have made it to the highest level. Nondominant group members will be absent at this level, as the career system does not allow “up-or-stay.” Taking a systems perspective, Martell, Emrich, and Robison-Cox (2012) conceptualize the process by which micro-level forces such as bias in performance assessment and macro-level forces such as demographic composition operate together to (re)produce gender segregation at the top of the organizational hierarchy. In upward mobility systems, promotion signals such as early success and velocity, as well as promotion models shaped as tournaments and gatekeeping exacerbate segregation (Martell et al., 2012). Erfurt Sandhu (2013) explains the persistent homogeneity at the highest levels in PSFs as a consequence of self-reinforcing mechanisms. Organizational members increasingly adapt to and adopt behavioral and interactional rules for reasons of efficiency and uncertainty reduction, and over time the unwritten rules of the game (Scott-Morgan, 1994; Sydow, Schreyögg, & Koch, 2009) become so normalized that members are locked-in and no longer see alternative, broader scopes of action. As an example, the rule to spend Friday in the office instead of at the client site (invented for efficiency reasons) becomes a way to increase visibility with gatekeepers, and emerges as a criterion for promotion, effectively excluding those who do not spend Friday in the office for religious or care-related reasons.

O’Brien, Scheffer, Van Nes, and Van der Lee (2015) effectively illustrate that because of system dynamics and self-reinforcing mechanisms, achieving more diversity at all hierarchical levels will take much longer than most decision makers tend to think. As a system tends to reproduce and reinforce itself, targeted interventions are needed to make change possible and enduring. Common diversity interventions in organizations such as training, mentoring, and networking are criticized because they fail to transform organizational structures and cultures, because they aim to fix “other” employees rather than fix the system, and because they do not target dominant group members (Benschop, Holgersson, Van den Brink, & Wahl, 2016; Bird, 2011; Ely & Meyerson, 2000).

Designing systemic diversity interventions with transformative power, such as the examples described later on in this article, requires basic knowledge of system dynamics, an approach that is useful in understanding the behavior of complex systems over time. Experts in system dynamics draw causal loop diagrams that show the structure of the system, including its components or variables and various relationships between components. Characteristic elements of such diagrams are stocks, flows, and feedback loops[1]. An example of such a model or causal loop diagram is found in Figure 1. When drawing a causal loop diagram representing diversity in a career system, stocks represent the relative representation of members of various social groups at different levels in the hierarchy, flows represent the relative advancement rates from one level in the hierarchy to the next, and feedback loops represent self-reinforcing positive or negative feedback processes in the diagram (Bleijenbergh & Van Engen, 2015). O’Brien et al. (2015) use system dynamics, including stocks and flows, to predict the rate of change in workforce composition at different levels of the hierarchy depending on the rate at which different categories of employees enter, are promoted, and leave the organization. Taking a systems perspective is thus imperative to decide where to intervene to promote diversity in careers.

Figure 1.

Model developed by technical university, reprinted with permission © Emerald Group Publishing, from Bleijenbergh and Van Engen (2015).

Mitigating Bias in Selection and Promotion Decisions

The main challenge in designing an effective diversity intervention lies in the identification of the main lever for change. This identification is tricky because the relationships between system components are characterized by dynamic complexity, as in constantly changing, tightly coupled, nonlinear, and feedback-driven (Sterman, 2001). While there are many possible levers, most of these are and should be context specific. This requires mapping the system under consideration in detail, to make sure we are not pushing the wrong buttons.

For the purpose of this article, however, I would like to choose a lever that is universal and has been identified as one of the most tenacious causes of homogeneity and inequality—bias. Bias in the strictest definition of the term is a cognitive distortion and is evidenced in decision making. Bias is especially prominent in the construction, operationalization, and application of criteria for selection and promotion. I will show how bias plays a role in reproducing the overrepresentation of a homogeneous group at the top of organizations—by looking at cognitive distortions in general and gender bias in particular. While racioethnic bias is another common and tenacious kind of bias, I do not discuss it here, because underlying stereotypes of various racioethnic categories are first more particular to the societal and organizational context than gender stereotypes (Roberson & Block, 2001), second are more conflated with stereotypes about immigration and religion than gender stereotypes (Vinkenburg, 2014), third are more ambivalent in terms of content (Fiske, 2012) than gender stereotypes, and fourth are less well described in the literature in terms of career consequences (Ossenkop, Vinkenburg, Jansen, & Ghorashi, 2015a) than gender stereotypes.

Bias reflects limits to our cognitive ability to make objective informed decisions about people’s careers, on account of bounded rationality (Kahneman, 2003), intransitivity (Suppes & Zinnes, 1963), high cognitive loads and time pressure (Chugh, 2004), and the influence of emotion and gut feelings (Rivera, 2015). Career-related decision making (e.g., selection, performance evaluation, promotion) is rarely objective and decidedly suboptimal (Vinkenburg et al., 2014), especially so at higher hierarchical levels. Regardless of group membership, this type of bias limits the effectiveness and predictive validity of our decisions about people.

Gender bias is defined as our prejudice in favor of one gender over the other, generally used as “bias against women.” Gender bias follows from gender stereotypes, which show a greater degree of stereotype fit or larger role congruency between “men and work” and between “women and care” than the other way around (Eagly & Heilman, 2016; Heilman, 2012). This lack of stereotype fit or incongruence results in women being “presumed incompetent” in professional roles (Heilman, Block, & Lucas, 1992; Heilman, Manzi, & Braun, 2015), and in shifting standards in performance evaluation (Biernat & Fuegen, 2001). Women who display stereotypically male behavior such as assertiveness and self-promotion in a work environment may experience penalties and backlash (Rudman & Phelan, 2008). Gender bias affects careers—in limiting chances of being promoted (Vinkenburg, Van Engen, Eagly, & Johannesen-Schmidt, 2011). As gender bias is often implicit and subtle, it is more difficult to recognize and thus harder to counter than blatant and explicit discrimination (Biernat, Tocci, & Williams, 2011). In up-or-out career systems, where White men are relatively overrepresented at the top of the hierarchy, the biased assessment of merit is in fact more a matter of favoritism (bias for White men) than of discrimination (bias against all others).

Evidence of bias is not readily accepted by the dominant elite, who firmly believe in their own ability to identify competence (Moss-Racusin, Molenda, & Cramer, 2015). Raising awareness of bias is clearly not enough to mitigate its effects, may produce unintended ironic effects, and could even backfire by reinforcing stereotypes and prejudice (Apfelbaum, Sommers, & Norton, 2008; Duguid & Thomas-Hunt, 2015; Kulik, Perry, & Bourhis, 2000; Macrae, Bodenhausen, Milne, & Jetten, 1994; Shelton, Richeson, Salvatore, & Trawalter, 2005). Bias is notoriously hard if not impossible to eliminate—and being confronted with one’s biases can make decision makers angry, confused, and defensive (Burrell, 2016; J. L. Howell & Ratliff, 2016; Pendry, Driscoll, & Field, 2007). According to learning theory, raising awareness alone makes individuals conscious but incompetent. In order to build their competence, in this case literally developing the skills to mitigate the effects of bias on decision making, experiential learning is required (W. C. Howell & Fleishman, 1982; Raelin, 1997).

While gender bias generally predicts disadvantage for women relative to men, men may also be trapped by bias that follows from gender stereotypes. People in general but professionals in particular share underlying normative expectations of careers as linear (Sabelis & Schilling, 2013), which implies that we value nonlinear, interrupted, or rerouted careers less than careers that reflect steady, smooth, upward mobility (Vinkenburg & Weber, 2012). The finding that men are more penalized for career breaks or other requests for flexibility than women (Coltrane, Miller, DeHaan, & Stewart, 2013; Vinkenburg, Van Engen, Coffeng, & Dikkers, 2012), is clear evidence of bias. If criteria for promotion reflect the uninterrupted career trajectories traditionally followed by male breadwinners, anyone with care responsibilities loses the contest.

It is important to realize that the degree of bias does not have to be large to have effects, especially over time. In “From bias to exclusion,” Martell et al. (2012) argue that if every career-related decision along the way is only a little bit biased in favor of one group versus another, the top of the hierarchy will remain segregated. If decision making is suboptimal in terms of filtering out the best candidates, favoring dominant group members is a likely result (Vinkenburg et al., 2014).

Research evidence suggests that while the criteria used in selection and promotion decisions appear objective, their application may be less so. First, if the description of the criterion is abstract, allowing discretion in the operationalization of vague terms such as “potential,” “talent,” or “excellence” leaves room for bias (Festing, Kornau, & Schäfer, 2014; Robinson, Fetters, Riester, & Bracco, 2009). Second, previously unspecified and perhaps irrelevant criteria such as social class or cultural capital may emerge during the process (Rivera, 2016; Vinkenburg et al., 2014). Third, decision makers tend to redefine the criteria as requiring the specific credentials that a candidate from the dominant group happens to have (Uhlmann & Cohen, 2005), effectively redefining merit. Fourth, criteria as well as candidates’ strengths and weaknesses are selectively mentioned, downplayed, or stressed in committee deliberations (Ahlqvist, Andersson, Söderqvist, & Tumpane, 2015; Van den Brink et al., 2016). Finally, linguistic bias creeps into evaluative language, including using negations more frequently for nondominant group members (e.g., “she does not have a bad CV”; Beukeboom, Finkenauer, & Wigboldus, 2010; Kaatz, Magua, Zimmerman, & Carnes, 2015). Criteria, their construction and application appear to be an especially promising venue for designing diversity interventions and mitigating bias.

Ellemers (2014) argues that the first step is to recognize implicit bias in hiring and promotions, followed by the development of clear criteria for hiring, compensation, and promotion, “instead of relying on subjective impressions [or] ambiguous prototypes” (p. 52). In “What works: gender equality by design,” Bohnet (2016) recommends using comparative selection methods in promotion decisions, which means waiting until there is a pool of candidates which can then be evaluated per criterion rather than by candidate. In addition, strongly structuring the decision-making process removes the possible influence of irrelevant personal characteristics. The effects of intransitivity on the decision-making process can be reduced by discussing the meaning and relative weight of criteria prior to discussing candidates, and by leaving enough time at the end of the meeting to compare candidates explicitly in terms of the weight and value of their scores on each of the criteria. In practice, this often involves reopening the debate on the already established rank order, because of the recalibration of the importance of criteria (Vinkenburg et al., 2014).

Engaging Gatekeepers

Gatekeepers are decision makers, those in organizational positions of power who shape careers by selecting, promoting, and supporting organizational members (Bosley, Arnold, & Cohen, 2009; Clerc & Kels, 2013; Van den Brink & Benschop, 2014). In elite PSFs, gatekeepers responsible for hiring procedures “receive only minimal training in interviewing, use their personal theories of what constitutes merit and how best to judge it, and receive little feedback on the quality of their decisions” (Rivera, 2015, p. 1382-1383). Meta-analyses of gender differences in performance evaluations clearly show that men are more favorably evaluated than women by their supervisors in field studies, in particular on ratings of promotability (Roth, Purvis, & Bobko, 2012), especially when raters are men (Bowen, Swim, & Jacobs, 2000), and more so when job complexity is high (Joshi et al., 2015). Interestingly, sex differences in rewards were much larger than differences in performance evaluations, and differences in performance evaluations did not explain reward differences between men and women (Joshi et al., 2015).

It appears that supervisors and other seniors who make career decisions hold an important key in disrupting the effect of bias on outcomes in up-or-out career systems. Engaging power holders is essential for making diversity interventions successful (Benschop & Van den Brink, 2014; Pendry et al., 2007), especially (and paradoxically) when the goal is to fundamentally alter power relations in organizations (Meyerson & Kolb, 2000). System interventions requires “engaging organizations members up and down the hierarchy, men and women, to question their own and others’ deeply held assumptions about work, productivity and effectiveness including what constitutes and contributes to individual and organizational success” (Ely & Meyerson, 2000, p. 591). Block (2016) recommends cultivating awareness of diversity dynamics in organizations, combining bias awareness and power and privilege awareness. In order to expose and ultimately work through the paradox of meritocracy, I recommend to specifically engage gatekeepers in diversity interventions as they are power holders and important “carriers” of the belief in meritocracy. Raising awareness of diversity dynamics, questioning implicit values, and working through paradox is a process that takes time and effort of a significant group of organization members in higher positions (Bleijenbergh, Van Engen, & Vinkenburg, 2013). Castilla (2016) recommends implementing accountability and transparency in the pursuit of meritocracy—decision makers are key in assigning or taking on these responsibilities. Without the engagement of those with formal authority, diversity interventions may fizzle out (Dobbin & Kalev, 2016; Kalev, Dobbin, & Kelly, 2006).

Design Specifications for Diversity Interventions

I identify three main design specifications for systemic diversity interventions that can be drawn from the argumentation above; namely engaging gatekeepers, mitigating bias, and optimizing decision making related to selection and promotion. The first refers to the target group, the second to the main lever, and the third to the relevant organizational or human resources processes at which the intervention is aimed. While the three intervention types described below share these three main design specifications, they are different in a few salient ways.

The interventions differ in scope—which is why I have ordered them from broad to narrow. Interventions with a broad scope pertain to the entire institution or organization; those with an intermediate scope pertain to organizational units; and those with a narrow scope pertain to committees, councils, or panels responsible for career decisions. The interventions also differ in the degree to which they provide opportunities for rigorous field experimentation and/or established intervention techniques, including using control groups, piloting, pretesting, establishing a baseline, and posttesting or “proper” impact evaluation. The evidence-based movement would recommend such techniques—but sometimes these get sacrificed in the struggle to gain access to an organization or population, the battle for scarce resources for a proposed intervention, or the one-off chance of jumping aboard an already moving train. While certainly a strong recommendation, these techniques should not stand in the way of designing an intervention using the three main specifications.

The backbone of the description of each of the three interventions is a concrete example of this type of intervention, as published in a peer-reviewed journal article or publicly available report. Each in its own way is built using the general specifications outlined above, which combined with their unique characteristics translate into design specifications for implementing these or similar interventions elsewhere (see Table 1 for an overview).

Table 1.

Design Specifications for Systemic Diversity Interventions (With Examples).

Name	Bias literacy	Participatory modeling (group model building)	Participant observation
Scope	Broad (entire organization)	Intermediate (department, unit)	Narrow (committee, team)
Duration	2.5 Hour sessionPretest and posttest	3 sessions of 3.5hoursIndividual preparation	Ongoing (meetings)Feedback session
Method	Interactive training	Workshops	Feedback on process
Experimentation	Field experiment with control groups	Follow-up, model building software	Metrics, linguistic software
Structure	Content: structuredProcess: structured	Content: unstructuredProcess: structured	Content: unstructuredProcess: unstructured
Facilitators	Expert trainer(s)	System dynamics experts, gender experts	Trained observer
Participants (gatekeepers)	Faculty members (20-25)	Department chairs and deans (10-15)	Committees, panels (depending on size)
Focus	Personal bias, self-efficacy, motivation, behavioral intentions	System dynamic model, customized policy development, and implementation	Evaluative language, criteria construction, and application
Key strength	Mitigating bias	Engaging gatekeepers	Optimizing decisions
References (academic examples)	Carnes et al. (2015); Carnes et al. (2012)	Bleijenbergh and Van Engen (2015)	Ahlqvist et al. (2013); Ahlqvist et al. (2015)

Any diversity intervention including the ones described here has much to gain from incorporating data-analytics or metrics (Williams, 2014). The data to be analyzed might include (longitudinal, individual) career outcomes, representation, and progression rates, but also the impact evaluation of the intervention on several measures. Data-analytics can serve as a starting point in diagnosing where bias occurs, and allow insight into intended and unintended intervention outcomes (Kaiser et al., 2013). Measuring effects of interventions on various relevant indicators, including implicit measures and objective measures such as advancement rates could bring these ironies to light. Additionally, all of the interventions below reflect design characteristics that follow from systematic reviews including meta-analyses on “what works” in diversity training (Benschop et al., 2016; Bezrukova, Spell, Perry, & Jehn, 2016; Kalinoski et al., 2013; Paluck & Green, 2009).

In summary—as also reported in Table 1—diversity interventions designed using these characteristics are conducted over a considerable period of time; combine awareness and competence or skills building, use experiential learning, include various instruction methods, and are complemented by other diversity initiatives (Bezrukova et al., 2016). In addition, Benschop et al. (2016) stress the importance of using an interactive approach and of addressing power relations in diversity interventions to help transform the structure and the culture of the organization.

Interestingly, all examples described below come from the domain of academia—the peer-reviewed evidence on similar interventions in the corporate world is extremely limited (Benschop & Van den Brink, 2014; Bezrukova et al., 2016; Kalinoski et al., 2013). The very few examples of systemic diversity interventions from the corporate world (to the best of my knowledge) are based on action research using a “dual agenda” approach—by focusing on work redesign, these interventions contribute to improved gender equity and workplace performance (Bailyn, Rapoport, Fletcher, & Pruitt, 2002; Charlesworth & Baird, 2007; Perlow & Kelly, 2014). As their main focus is on improving work–life integration but not on mitigating bias, I will not discuss these examples here. In academic settings, faculty members and peers act as gatekeepers by assessing merit and making career decisions about others regularly, including hiring, tenure and promotion, funding, and award nomination (e.g., Bird, 2011)—which makes this a prime location for interventions that engage gatekeepers, mitigate bias, and optimize decision making on careers.

Broad Scope Intervention: “Bias Literacy” Program

The first type of intervention can be described as large scale or broad scope, involving most if not all members of an organization. Distinguishing features of the intervention are its considerable opportunity for field experimentation including long-term follow-up and control groups, and its emphasis on a combination of increasing bias awareness and building competence through experiential learning.

The “bias literacy” program is a large scale systemic diversity intervention conducted at the University of Wisconsin (Carnes et al., 2012; Carnes et al., 2015). Using a pair-matched, single-blind, cluster-randomized, controlled intervention design, faculty members from 46 separate departments participated in a 2.5-hour gender bias habit-reducing training, with faculty members from 46 control departments waitlisted. Bias literacy is about becoming conscious and competent in dealing with the biases that typically affect people decisions.

During the workshop, research on the pervasiveness of stereotype-based gender bias in decision making and judgment is reviewed. The first module addresses the origins of bias as a habit, the second module promoted “bias literacy” by describing six kinds of stereotype-based gender bias, such as redefining credentials and stereotype priming, and the third module enhanced self-efficacy for overcoming gender bias by providing behavioral strategies, such as individuation, and by cautioning against counterproductive strategies such as a strong belief in one’s ability to make objective judgments (Carnes et al., 2015). Participants built positive outcome expectations by envisioning a link between their own actions and desired outcomes, as the facilitator reflected on “benefits articulated by participants that can come from reducing implicit bias, and on the empowerment of aligning actions with personal beliefs” (Carnes et al., 2012, 63-77).

To obtain a baseline premeasure, a short-term and a long-term postmeasure of effects of the intervention, participants were surveyed within 2 days before the intervention and at 3 days and again at 3 months postintervention. The results show significant differences in measures of (explicit) personal bias, external motivation to mitigate bias, and self-efficacy to engage in gender-equity-promoting behaviors (Carnes et al., 2015). In addition, significant increases in self-reported action to promote gender equity occurred at 3 months in departments where more than 25% of faculty had participated. Such self-reported actions involved the degree to which participants said they were able to apply the behavioral strategies for reducing bias in decision making they had deliberately practiced during the workshops. Moss-Racusin et al. (2014) recommend combining self-report with behavioral observations from colleagues, students, and trained raters to strengthen the outcome measures. Ideally, the intervention would be geared toward specific decision-making processes such as tenure or faculty hiring, rather than taking a more generalistic approach. In designing interventions of this type, data-analytics could be used to identify crucial career transitions where bias trumps merit-based assessment, or where certain criteria suddenly become more or less important in the process. Participants could also be asked to bring in their own critical incidents, to be interactively discussed and resolved. In addition, the intervention could be strengthened by explicitly adding a system perspective to the curriculum (adding to the duration of the workshop), for example, by explaining how a little bit of bias adds up to cumulative disadvantage and homogeneity in upward mobility systems (Martell et al., 2012), and how redesigning the decision-making process could help mitigate bias (Bohnet, 2016).

This type of intervention is especially successful in building not only participants’ awareness of bias but also their self-efficacy in being able to interrupt the effect of bias in future decision making. As has been argued elsewhere and shown meta-analytically, diversity self-efficacy plays a key role in facilitating diversity skill acquisition and implementation (Combs & Luthans, 2007; Kalinoski et al., 2013). Active and especially experiential learning activities that enhance self-efficacy increase participants’ receptiveness to evidence of bias relative to more passive formats (Moss-Racusin et al., 2014). Through bias literacy’s focus on promoting self-efficacy and behavioral intentions for future decision making, it moves beyond the bias awareness that is the common goal of many diversity interventions.

Intermediate Scope Intervention: Participatory Modeling

Participatory modeling or “group model building” is the only diversity intervention illustrated in this article that explicitly and purposefully takes a system perspective. Using a method that has been developed for interventions in social systems, the overall goal is to engage problem owners from one organizational unit (e.g., department) in “building and playing with a system dynamics model to help them tackle an organizational issue” (Lane, 2010, as cited by Bleijenbergh & Van Engen, 2015, p. 423). A complex problem is translated into a causal loop diagram of system components and feedback loops that shows the dynamics underlying the organizational issue.

Based on their experiences with this type of intervention in two separate Dutch university settings, Bleijenbergh and Van Engen (2015) describe the details of participatory modeling to support gender equality with gatekeepers such as department chairs, HR director, and dean. The intervention consists of three sessions (a, b, and c) of 3.5 hours each, with a 1-month interval to allow reflection and preparation. Together with two facilitators and two gender experts, the participants strive to (a) identify and understand the underlying system dynamics of gender inequality in their own department, (b) to reach a shared problem definition and analysis, and (c) to identify and implement custom-made policies to promote change. The causal loop diagrams are literally built, adapted, and projected on the screen during the sessions, visualizing how feedback processes are interrelated and can stabilize or reinforce themselves. In Figure 1, a visualization of the causal loop diagram or models developed during one of these interventions by Bleijenbergh and Van Engen (2015) can be found, mapping the percentage of women at various stages of the academic career ladder. Participants identified the relative overrepresentation of men at the highest level as resulting from self-reinforcing feedback processes such as the masculinity of norms. Interestingly, while the goal of the intervention was to promote gender equality, both generation and nationality emerged as additional sources of inequality, intersecting with gender. With an extremely diverse population of PhD students and postdocs in terms of nationality, the continued overrepresentation of White Dutch men with an undergraduate degree from the same university in senior faculty positions was striking.

A major strong point of this type of intervention is related to the active involvement of gatekeepers. As a gender diversity expert participating in these sessions at one of the two universities, I personally experienced the strength of including problem owners in the modeling process to increase their commitment and facilitate implementation of interventions. In a personal account, my coauthor describes how confrontations that took place before and during the participatory modeling sessions between the research team and organizational stakeholders, while certainly challenging, ultimately supported the transformation of stakeholders into change agents (Bleijenbergh, 2016).

The masculinity of norms is a central element of the model built in both institutions. The bias that follows from the perceived lack of fit between women and the shared norm of the “ideal academic” who has no outside responsibilities negatively affects women’s chances for tenure and promotion relative to men (Bleijenbergh et al., 2013). While participants recognized the masculinity of norms, they argued it would be difficult to change norms directly. However, by viewing norms as a crucial element of a larger system, the participants identified measures that could over time bend norms including introducing contract extensions for postdocs to compensate for parental leave.

In a book chapter titled “Dare to care” (Herschberg, Vinkenburg, Bleijenbergh, & Van Engen, 2014), we describe in detail how the public negotiation of norms about career and care during the participatory modeling intervention resulted in a lively debate on what some participants considered “normal” and others did not. The dialogue as it spun out was literally transcribed into the chapter, as it proved to be a turning point in making change possible and in minimizing the impact of bias for all academics with care responsibilities (not only mothers!) in promotion and tenure decisions.

Others have used a systems dynamics approach to uncover and investigate underlying feedback mechanisms that reinforce the overrepresentation of White men at the top of the hierarchy (O’Brien et al., 2015), but participatory modeling is especially effective because gatekeepers identify particular rather than universal barriers and opportunities to increasing diversity in their own context. Getting and keeping gatekeepers on board throughout the course of the intervention is a challenge, but if successful, the impact is palpable. Bleijenbergh (2016) describes the very public actions to promote gender equality taken by initially quite resistant deans and other senior administrators after the invention. The rector (i.e., university president) successfully defended one of the policies implemented in a discrimination case filed against the university by a male employee with the Dutch national equal treatment commission.

This intervention benefits from the use of data collected and analyzed prior to the first session by the gender experts, ranging from detailed longitudinal career metrics (including pay differentials) to interview or focus group data on how organizational members experience career opportunities, work–family issues, and the atmosphere in the department. The evidence thus collected serves to challenge or confirm assumptions voiced by the participants during the sessions. While bias in merit assessment is not explicitly on the agenda, the “perceived incompetence” of women (Heilman, Manzi, & Braun, 2015) and ambiguity in hiring and promotion processes and criteria (Bird, 2011) are invariably a topic of discussion. Implicit bias training to raise awareness and build competence is often high on the list of recommendations. Bleijenbergh and Van Engen (2015) recommend tracking and measuring the implementation success of policies decided on, if needed by means of another round of participatory modeling.

Narrow Scope Intervention: Participant Observation

A more organic systemic intervention that puts less pressure on precious resources is participant observation during panel or committee meetings dedicated to reward allocation (e.g., research grants, prizes), performance evaluation, promotion or tenure decisions, and appointments. This intervention is aimed directly at optimizing decision making. Academic committees rely on peer review, performance evaluation, and merit assessment (in terms of excellence, independence, track record, etc.) as input for their deliberations and decisions. Assuming a normal distribution of merit among candidates from different social groups, the urgency of mitigating bias in such decisions is relatively easy to establish if outcomes (e.g., success rates) are unequal. In the context of PSFs, Joshi et al. (2015) recommend that trained neutral observers attend the so-called “calibration meetings” in which managers jointly discuss performance ratings and make bonus and promotion decisions about their direct reports. Such observers “may be able to direct discussion away from decision making based on stereotypes or biases toward performance-related information” (p. 1535).

Ideally, the intervention would last throughout the complete lifecycle of a committee’s main goal, be it hiring (for one or multiple positions), annual performance evaluation or promotion rounds, semiannual or biannual selection panels or election councils, or similar. It would not demand any additional time from the committee members in terms of preparation, except for reserving some time at the end of each meeting or in an overall final session for feedback and reflection. The impact of the intervention could be measured in terms of the number of nondominant group members long-listed, short-listed, and selected or promoted, but also in terms of the awareness and competence of committee members in mitigating bias. Eventually, if this kind of impact has been established, the task of addressing bias could be taken over by one of the committee members (or even the chair).

Participant observation has been effectively applied at the Swedish Research Council (Ahlqvist et al., 2013; Ahlqvist et al., 2015). It requires the presence of a well-prepared, trained observer with status and voice who is allowed to give feedback on the content, process, and procedures of the committee meetings and decisions made. Observers must be aware of bias and how it may affect the committee members and their decision making. Observers may use several paper-and-pencil or app-supported tools to underscore their observations; including counting the incidence and duration of time spent discussing various candidates or criteria, the use of evaluative language (e.g., masculine vs. feminine adjectives, positive vs. negative adjectives, negations), and the incidence and duration of talk by different committee members (Ahlqvist et al., 2015). If candidates are present during the meeting (e.g., interview, presentation), additional dimensions of the interaction and potential sources of bias may be observed. Collecting and systematically analyzing the underlying documentation of the committee work, such as research proposals or personal statements, CVs, recommendation letters, performance reviews, and evaluation reports, could supplement observations (Kaatz et al., 2015; Madera, Hebl, & Martin, 2009).

Due to the inherent intransitivity of the decision-making process, calibration meetings or final discussions during which several candidates are discussed and compared are highly informative in locating the emergent nature or differential application of criteria (Rivera, 2016; Vinkenburg et al., 2014), and in charting the role of the chair in reducing cognitive load and allowing for joint evaluation or comparative selection (Bohnet, Van Geen, & Bazerman, 2015; Vinkenburg et al., 2014). Observers can with their feedback expose and challenge normative expectations and paradoxes underlying the decision-making process, including the meritocracy myth or linear career bias. Acting as change agents, dealing with resistance, and nudging gatekeepers toward optimization of the decision-making process, the observations and reflections from the participant observer and the responses from the committee may create a ripple effect that is noticeable throughout the system.

Discussion and Conclusion

With these three examples of systemic diversity interventions in academic settings, I have highlighted the design specifications for interventions effective in promoting diversity in upward mobility systems. By optimizing decision making, mitigating bias, and engaging gatekeepers, each of the three interventions will improve promotion and advancement rates of nondominant group members and thus reduce the overrepresentation of White men at the top of the pyramid. In contrast to many other diversity interventions, the interventions described here do challenge and change existing merit assessment, performance evaluation, and reward allocation practices—and address the resistance that both dominant and nondominant organizational members may experience when these practices are exposed.

All three examples of the interventions provided here share a distinctive action research characteristic, namely that researchers as experts work together with organizational members in creating organizational change and generating robust, actionable knowledge (Coghlan, 2011). But even without a distinct research goal or the active involvement of researchers, interventions build using the design specifications outlined here and provided in Table 1, should encourage action learning, through reflection and (for some participants at least) reflexivity (Raelin, 1997). Follow-up is needed to ensure the measurement of long-term results and the sustained transformation of ongoing decision-making processes. Taking a systems perspective, by looking at career decisions in particular, each of these interventions aims to debias the system rather than to debias the mind-set (Bohnet, 2016). In doing so, in all three interventions the underlying paradox of meritocracy is exposed and worked through to promote equal representation at all levels of the hierarchy.

Without a formal method to compare the effectiveness of these interventions, each of course can be said to have its strengths and weaknesses. The bias literacy program finds its main strength in bias mitigation, as it shows how to move beyond the stereotype-induced “presumed incompetence” of nondominant group members being evaluated. In its current form, it is perhaps too generalistic and does not cover enough ground in terms of system dynamics. Participatory modeling or group model building finds its main strength in truly engaging gatekeepers as problem owners in the identification and solution for specific problems within their own setting. Whether bias mitigation becomes part of the solution depends strongly on whether participants identify this as an important factor and possible solution in the model. Participant observation, finally, finds its main strength in optimizing decision making as it takes place, thus directly improving the operationalization and application of criteria for evaluation, selection, and promotion. Without explicitly linking the intervention and its consequences to the wider organization and its decision-making habits, learning and change may be limited to individual committee members.

Which of these interventions fits the diversity issue a policy maker or diversity officer would like to tackle, depends on the situation. But the basic design specifications of engaging gatekeepers and optimizing decision making by mitigating bias can be applied in many organizational settings where people decisions are made and careers develop over time. The most natural habitat for these interventions are upward mobility career systems with their fixed steps or routes and formal promotion criteria—such as PSFs (e.g., Kumra, 2014), but other organizational settings in which performance evaluations serve as input for promotion and remuneration decisions would benefit as well (e.g., Van den Brink et al., 2016).

As many diversity interventions seem to fail, and as efforts to promote diversity at the top of the organizational hierarchy are especially unsuccessful, we need more evidence on what does work and how to do it. With this collage of successful systemic diversity interventions and their matching design specifications, I hope to have inspired both researchers and practitioners in designing and studying the effects of diversity interventions both inside and outside career systems that value upward mobility as well as social equality.