Evaluating the quality of the list of occupations recommended for further exploration.

Access to online career information increases the complexity of career decisions (choosing a major or job). When the number of alternatives is large, the first step is to compile a list of promising career alternatives for further exploration, often by using interest inventories (e.g., the Self-Directed Search). But what makes such a list useful? The judgments of 20 career counselors and 103 graduate students supported the hypothesis that higher list quality is associated with a greater similarity between the occupations on the list, fewer occupational fields represented by the occupations on the list, and a list length approximating seven occupations.

Career decisions are among the most important decisions individuals face over their lifetime (Bimrose & Mulvey, 2015). For many individuals, career decisions, such as choosing a major, an occupation, or a job after college graduation, are complex due to the many alternatives from which to choose. In today's dynamic and fast-paced world of work, the number of alternatives, the amount of information about them, and the variety of information sources increase daily; hence, to avoid information overload, it becomes essential to prescreen alternatives to produce a manageable list. The question remaining is, what are the indicators of the quality of such a list of career alternatives for further exploration?

Holland's (1997) RIASEC typology of vocational personalities and work environments often serves career counselors as a framework for career counseling. Holland’s RIASEC model is elegant in its simplicity, appropriate for describing the individual's interests and potential work environment, and has good research support (Tracey, 2020). Vocational interest inventories are often used to assess deliberating individuals' vocational personalities (Crites, 1999). Individuals' interests often guide subsequent career exploration to compile relevant information, and hence can help individuals make informed career choices (Hansen, 2005). Indeed, Holland (1997) proposed to use interest inventories, such as the Vocational Preference Inventory (Holland, 1985) and later, the Self-Directed Search (Holland et al., 1994), to compile a list of occupations compatible with the individual's vocational personality type for further exploration.

Specifically, interest inventories are used, explicitly or implicitly, to guide individuals in compiling a list of promising alternatives worthy of in-depth exploration when there are numerous available options. Information about the connection between interest areas and occupations can facilitate the individual’s career decision-making process. Many interest inventories, such as the Strong Interest Inventory, the ACT Interest Inventory, and the O*NET Interest Profiler, provide feedback in the form of the six RIASEC scale scores, which facilitate translating the individual's interests through a 3-letter Holland code to a list of recommended occupations.

Evaluating the quality of the list of occupations is relevant for both face-to-face career counseling, when the counselor asks the client to come up with a list of occupations or interprets the list of occupations that are compatible with the client's vocational interests; and for self-help contexts, when interactive career planning systems generate lists of recommended alternatives for the client (e.g., the O*NET; www.onetonline.org). The quality of such a list impacts subsequent stages and outcomes of the career decision-making process. A low-quality list may steer the deliberating individual to a sub-optimal career choice or prolong the decision-making process. In contrast, a high-quality list would enhance the prospect of better career decisions (Gati et al., 2019). In the present research, we describe and test three criteria for evaluating the quality of such occupation lists for further exploration.

Compiling a list of promising occupations

A list of promising career alternatives worthy of further exploration may be derived from numerous sources: life experiences, chance circumstances, childhood dreams (e.g., firefighter, teacher, physician), an acquaintance's occupation, parents' expectations, a protagonist in a TV series, elicited daydreams (Holland & Messer, 2017a, p. 2), or by directly being asked (e.g., "List the occupations you think of pursuing"; Gadassi & Gati, 2009). Deliberating individuals often turn to career counseling for assistance in compiling such a list. A list of promising career alternatives can also be assembled using an Internet-based career planning system (e.g., O*NET), using various search factors (e.g., vocational interests, work values, skills). Whatever way this list is compiled, its quality is likely to affect both the progress and outcomes of the career decision-making process. At the in-depth exploration stage, the individual examines the compatibility between their sought attributes for a future career (e.g., economic security, professional advancement, flexibility of working hours) and the corresponding attributes of particular promising occupations by collecting pertinent information (Gati & Asher, 2001). Thus, a high-quality list of promising occupations can facilitate the individual’s career decision-making, particularly at its subsequent stage––in-depth exploration.

The quality of the list of promising occupations

Assessing the quality of a list of promising occupations requires defining criteria for quality. To compile a list of promising occupations, counselees or their career counselors often rely on the results of vocational interest inventories. The Self-Directed Search (SDS; Holland & Messer, 2017a) is frequently used to elicit individuals’ vocational interests and translate them into Holland's (1997) six RIASEC types. Upon completing the SDS, the individual receives a 3-letter RIASEC code that can help identify a list of promising occupations for further exploration (Holland & Messer, 2017b, p. 2). A list derived from the 3-letter RIASEC code will likely possess two features. First, it is homogeneous, as all occupations match the same 3-letter code. Second, the list can vary considerably in length, ranging from 3 (corresponding to all six permutations of the IAC code) to 194 (for the SEC code; Holland & Messer, 2017b). These two features reflect two implicit assumptions: First, a homogeneous (rather than heterogeneous) list of occupations is the desirable outcome, and second, the number of occupations recommended for further exploration is of no importance. The present research goal was to examine these implicit assumptions using career counselors’ and graduate students’ judgments regarding the quality of such lists.

Indicators for list quality

The present study focused on three potential indicators for assessing the quality of a list of promising occupations. In the following sections, we describe two indicators related to homogeneity and one related to list length.

List homogeneity

Homogeneity refers to the degree of resemblance among the occupations on the list. A heterogeneous list comprises occupations that differ in many respects: certain occupations may be compatible with the individual's preferences in some aspects (e.g., social work is compatible with interests in helping and counseling), whereas other occupations may match the individual’s preferences in other aspects (e.g., computer science is compatible with an inclination to analytic thinking). When the list of promising occupations is homogeneous and comprises closely related occupations, comparing them becomes relatively straightforward because their characteristics align with each other. Conversely, heterogeneity increases the complexity of the comparison and may produce choice overload (Chernev et al., 2015). Sagi and Friedland (2007) found that individuals with a more diverse set of considered alternatives are at risk of regretting their choice.

Degree of similarity

The degree of similarity between the listed occupations refers to the degree to which the examined occupations have more common than distinctive attributes. For example, a social worker and a counseling psychologist share many common attributes (high similarity), whereas a social worker and a truck driver differ in many attributes (low similarity). When the attribute-based similarity between the compared alternatives is high, it becomes easier to focus on the more nuanced features that distinguish the alternatives. Thus, a high similarity would facilitate a more refined comparison of the relative desirability of those features and assess which alternative is more compatible with the individual's preferences. Accordingly, we tested the hypothesis that greater attribute similarity among the listed occupations is positively associated with better list quality.

The number of occupational fields

The second proposed indicator of the list's homogeneity is the number of occupational fields characterizing the list. Our use of occupational fields derives from Holland’s (1966, 1997) six RIASEC types or Roe's (1956) eight fields. The number of occupational fields characterizing the occupations on the promising list partially overlaps with the occupations’ similarity, but these indicators are not identical. For example, certain occupations may seem similar (e.g., accountant and economist) but represent different Holland types (Conventional [CSI] and Investigative [IAS], respectively) and Roe's fields (organization and business, respectively). This distinction appears to justify considering both the occupations’ similarity and the number of occupational fields as discrete indicators of homogeneity. A smaller number of occupational fields would reflect higher homogeneity and vice versa. Thus, we hypothesized that fewer occupational fields would characterize higher list quality.

List length

The third potential indicator of list quality is its length––the number of alternatives on the list. The size of the list directly impacts the amount of information to be gathered and processed during the in-depth exploration stage. Using the Occupations Finder (Holland & Messer, 2017b), the resulting 3-letter RIASEC codes yield lists of occupations of varying lengths. Interestingly, Holland and Messer (2017b) did not address variable list length explicitly, nor to the consequences of possible variations on the exploration of the occupations on the list. Nevertheless, the number of occupations on the list may comprise another indicator of list quality. When asked about the preferred number of occupations for further exploration, young adults deliberating on future careers regarded seven as the optimal list length (Gati et al., 2003). Their preference of 7 may reflect natural limitations on information processing, and interestingly, this preferred number alludes to Miller’s (1956) seminal article, "The magical number seven, plus or minus two". Young adults whose recommended list had fewer than five (i.e., 7–2) alternatives regarded it as “too short” whereas a list with more than nine alternatives (i.e., 7 plus 2) was viewed as “too long” (Gati et al., 2003).

A list too short or too long may have ramifications on the career decision-making process. A too-short list may miss potentially suitable career alternatives or may lead to the feeling of having limited options and diminished choice, thus increasing the "fear of missing a better option" (McGinnis, 2020). A too-long list may cause the individual to be deterred by the time, energy, and cognitive resources required to process it fully. Moreover, being fully engaged in processing a too-long list may lead to information overload (Chernev et al., 2015), thus hindering the career decision-making process. A 7-occupation list (plus or minus two) involves gathering and processing a manageable amount of information (Gati & Ram, 2000), likely facilitating in-depth exploration. Therefore, we hypothesized that as the number of listed occupations diverges from seven (above or below), the list of promising career alternatives will be viewed as less optimal.

The present research

The goal of the present research was to test indicators for evaluating the quality of a list of promising career alternatives, stemming from Holland's theory: list homogeneity and list length. We tested two indicators of list homogeneity: (a) the degree of similarity among the recommended occupations on the list and (b) the number of different occupational fields characterizing the occupations on the list. We also tested a third indicator of list quality––(c) list length. We hypothesized that higher list quality would be associated with: (a) a greater similarity between the occupations on the list, (b) a smaller number of occupational fields characterizing the occupations on the list, and (c) the proximity of the number of listed occupations to seven. To test these hypotheses, we elicited career counselors' and graduate counseling students' judgments regarding the quality of lists of occupations recommended for further exploration of varying homogeneity and length. In the absence of objective criteria for list quality, the expert judgments of career counselors served as a proxy for quality (e.g., Shimoni et al., 2019).

Method

Participants

Two groups served as judges. One group comprised 20 vocational psychologists and career counselors (all women; mean age = 49 [range 28–65], all held at least a master’s degree in psychology or counseling, and their mean professional seniority was 21 years [range 3–30]). The second group comprised 103 graduate students in psychology or counseling (89% women; mean age = 35 [range 24–53]). Both groups were familiar with Holland's (1997) typology and the Self-Directed Search and the Making Better Career Decisions career planning system (Gati, 1996; see below). They consented to participate voluntarily in an anonymous study as experts or future experts (see Procedure).

The occupational database

Lists of occupations derived from any three-letter Holland code (e.g., SEC) are necessarily homogenous as all occupations have the same 3-letter code). Hence, to enable testing the hypothesis about list homogeneity as an indicator of quality, we used lists of occupations for further exploration offered to users of a computer-based career planning system—Making Better Career Decisions (MBCD; Gati, 1996). The goal of MBCD is to identify career alternatives that are compatible with the attributes considered desirable by the deliberating individual and therefore recommended for further exploration. MBCD accomplishes this by using a sequential elimination process that matches the individual’s preferences with the respective attributes in its database1 (Gati, 1986). The database included 305 occupations, each characterized in terms of 31 attributes or career-related aspects (Gati et al., 1998; Prediger & Staples, 1996; Pryor, 1981). Some of the aspects can be regarded as a refinement of vocational interests because they refer to more subtle features of each type (e.g., the aspects of counseling, community service, and teaching correspond to Holland’s [1997] Social type). Other aspects can be viewed as an extension of interests or comprise factors that complement vocational interests (e.g., income, length of training, the prospect of professional advancement; Gati et al., 1998). Each occupation in the database (e.g., accountant) is described by its typical level (out of five levels) for each of the 31 aspects (e.g., accountant: level of teamwork = 3 [moderate], using numerical skills = 5 [all the time], work environment = 1 [only indoor work], independence = 4 [above average]).

A six-year follow-up study (Gati et al., 2006) supported the predictive validity of the MBCD’s list of occupations recommended for further exploration, adopting the criterion of occupational choice satisfaction. Specifically, individuals' satisfaction from their occupational choice was higher after six years for those choosing (vs. those not choosing) an occupation from the list recommended by MBCD (Cohen's d = 0.85). This finding supports the validity of the MBCD's occupational database used in the present research to determine the number of occupational fields that the occupations represented on the individual's list of recommended alternatives and to compute the similarity between the occupations on the list.

Research questionnaire

Each anonymous research questionnaire included a page of general demographic questions (i.e., age, gender, and professional seniority), a page of instructions (see Procedure), and a page with a practice list of occupations, followed by 18 lists presented in a different random order (see examples in Appendix A). Six lists portrayed three actual and three constructed cases, adopted from Shimoni et al., (2019, Appendix B). Twelve additional lists of occupations were selected from lists recommended to actual users of MBCD; the lists varied in the number of occupations (i.e., short [3-4], intermediate [6-7], and long [11-15]) and in their homogeneity (both in terms of the number of occupational fields and the similarity between the occupations in the list). Table 1 presents the two indicators of homogeneity for each list, the number of occupations in each list, and the list length closer to 7 (i.e., the absolute gap between the number of occupations on the list and 7). The Preliminary Analyses describe how the three quality indicators were calculated. Each list presented the occupations in alphabetical order (according to the Hebrew alphabet).

Table 1

Career Counselors' (N = 20) and Graduate Counseling Students' (N = 103) mean ratings of list quality, and the three list quality indicators

List#	List quality Ratings		Number of occupational fields	Dissimilarity1 among occupations	Number of occupations on the list	List length closer to 7
	Career Counselors	Graduate Students
1	3.40	4.99	5	1.16	4	3
2	6.30	6.83	2	1.24	6	1
3	6.65	6.26	2	0.52	15	8
4	4.70	5.76	2	1.01	6	1
5	7.95	7.57	2	0.69	7	0
6	4.35	4.49	4	1.18	15	8
7	3.55	5.02	5	0.79	6	1
8	6.90	6.74	2	0.73	7	0
9	4.85	5.53	3	1.06	11	4
10	7.40	7.29	2	0.75	7	0
11	6.45	6.98	2	0.15	3	4
12	4.00	4.75	5	1.15	15	8
13	6.50	6.09	4	0.91	7	0
14	5.65	6.50	2	0.43	3	4
15	4.60	5.41	3	1.05	4	3
16	5.15	6.47	2	0.89	4	3
17	–2	5.31	3	0.82	11	4
18	–2	6.32	3	0.62	11	4

A smaller number of occupational fields, higher similarity among the occupations, and list length closer to 7 are assumed to represent higher-quality lists

1Represents the mean gaps among the occupations across the 31 aspects

2No data is available due to a technical omission

Procedure

We approached career counselors during professional meetings or by mail. The questionnaires were distributed to graduate students during or after class. We asked the participants to voluntarily take part in an anonymous study of about 12–15 min as experts or as future career counseling experts. They were reminded that the career decision-making process often begins with prescreening the career alternatives and identifying occupations worthy of further, in-depth exploration. The participants were asked to evaluate how helpful each list of occupations would be as a starting point for further exploration. They rated their evaluations on a 9-point Likert-type scale, ranging from 1 (not good at all) to 9 (very good). We deliberately did not provide information on what comprises a high-quality list to avoid skewing the participants’ judgments.

Preliminary analyses

The three indicators of a list of promising career options ––similarity among occupations, number of occupational fields, and list length––were calculated for each list of occupations in the following manner.

Similarity between the occupations on the list of recommended alternatives

The attributes of the occupations comprising the lists (in terms of the 31 aspects) were retrieved from the MBCD’s occupational database. To estimate the degree of similarity between the occupations in terms of their attributes, we calculated two indices: (a) the mean distance between the listed occupations and (b) the standard deviation of these distances. For each pair of occupations on the list, we first calculated the absolute gap between their aspects’ most characteristic levels of each aspect (e.g., “only indoors” [rated as “1”] for one occupation and “mostly indoors” [“2”] for the other occupation, resulting in an absolute difference of 1 [|2-1|= 1]). Next, we computed the mean of these absolute differences across the 31 aspects. Finally, we computed the mean differences across all pairs of occupations on the list. A higher mean reflects greater differences and hence a more heterogeneous list. To compute the standard deviation, we first computed the standard deviation of the absolute differences for each pair of occupations across aspects; then we computed the mean of the standard deviations across all occupation pairs. A larger standard deviation indicates greater heterogeneity of the listed occupations. As these two indices were highly correlated (r = 0.86), we merged them to obtain an overall indicator of attribute-based similarity between the occupations on the list. Thus, a lower value for this overall indicator denotes greater similarity between the occupations.

Number of occupational fields

A list of promising occupations derived from Holland’s RIASEC typology is necessarily homogeneous in terms of occupational fields, as all occupations are reflective of the same 3-letter code and hence represent the same three fields (i.e., types). Therefore, to allow for variance in the number of occupational fields, we used the MBCD's occupational database. Each occupation in the MBCD’s database represents one or more of Roe's (1956) eight occupational fields: service, business, organization, technology, outdoors, science, general culture, and arts and entertainment. A set of occupations appearing on a list of promising alternatives (e.g., petroleum engineer, material engineer, and mechanical engineer) can represent a single occupational field (e.g., technology, in Roe's classification). In contrast, in Roe’s classification system, another set of occupations of the same length (e.g., mechanical engineer, chemical engineer, and industrial engineer) can represent several occupational fields (e.g., technology, science, and business). Some occupations represent only a single field, whereas other occupations represent two or more fields. For example, an actor's classification is limited to the field of arts and entertainment, whereas a lawyer can be classified in multiple domains: general culture, service, and business. Thus, in case we have a list with three occupations—social work, computer science, and pharmacist, the number of occupational fields was calculated as follows: Social work belongs to only one occupational field––service––and thus represents only a single occupational field (n = 1); computer science belongs to two occupational fields, science and technology (n = 2); and pharmacist belongs to three occupational fields, science, service, and organization (n = 3). Thus, these three occupations represent four occupational fields (i.e., service, science, technology, and organization). Fewer occupational fields characterizing the listed career alternatives reflect higher list homogeneity.

List length

We calculated the absolute divergence of the number of occupations on the list from 7 (see the far-right column labeled as List Length closer to 7 in Table 1), representing the optimal number of items for cognitive processing (Miller, 1956) and rated as the preferred list length by deliberating young adults (Gati et al., 2003).

Results

Inter-judge agreement

The mean list-quality ratings of each list by both the expert career counselors and graduate students are presented in Table 1. The correlation between the mean list-quality ratings of the career counselors and those of the graduate students (see Table 1) was very high (r = 0.91, p < 0.001), reflecting the high agreement between the two groups. Agreement among the participants within each group was assessed by Kendall’s W coefficient of concordance, a non-parametric statistical test used to assess agreement among raters. The career counselors’ degree of agreement regarding the list-quality ratings (W = 0.51) was higher than that for the graduate students (W = 0.26), Z = 1.84, p = 0.033. Disagreements between the counselors and the graduate students were mostly related to the shorter lists (see Table 1). Notably, the graduate students rated the shorter lists' quality as higher than the career counselors. Despite the high correlation between the two groups’ ratings of list quality, we chose to report the results for each group of raters separately due to the difference in the inter-judge agreement.

The associations between the three indicators of list quality

As expected, the correlation between the indicators of homogeneity––number of occupational fields and similarity between the occupations on the list––was positive as expected, r = 0.48 and 0.47 (p < 0.05), for career counselors and graduate students, respectively. These correlations are moderate, supporting the distinction between the indicators. The correlation between the number of occupational fields and the proximity of list length to 7 (i.e., lesser deviation from 7) was also positive but low r = 0.26 and 0.26, ns). and list length of greater proximity 7 was nil (r = 04 and 0.02, respectively).

The associations between the rated and the calculated list quality indicators

Table 2 presents the Pearson correlations between the three calculated list-quality indicators and the career counselors’ and the graduate students’ ratings of list quality. As hypothesized, these correlations were positive and statistically significant (with a single exception) for both counselors and graduate students. For both groups, the highest correlation was obtained between the list-quality ratings and the number of occupational fields represented by the occupations on the list (r = 0.73 and 0.80, p < 0.01, for counselors and students, respectively). This result indicates that lists with fewer fields were judged higher in quality. The correlations between list-quality ratings and the similarity among the occupations on the list were also positive (r = 0.52 and 0.57, p < 0.05, for counselors and students, respectively), indicating that homogenous lists were judged of higher quality. Lastly, whereas the correlations between list-quality ratings and list length approximating 7 were also positive for both groups, they were statistically significant only for the graduate students (r = 0.39, p = 0.07 one-tailed, and r = 0.53, p < 0.05, for counselors and students, respectively). The graduate students’ list-quality ratings were more strongly associated with all three list-quality indicators than were career counselors’ ratings; still, the differences were slight and did not reach statistical significance.

Table 2

Pearson Correlations between Career Counselors’ and Graduate Students’ Ratings of List Quality and the Calculated Indicators of List Quality

Calculated indicator1	Correlation with ratingsof list quality by
	Career Counselors(N = 20)	Graduate Students (N = 103)
Number of occupational fields	− .73**	− .80**
Similarity among occupations	.52*	.57*
List length closer to 7	.39+	.53*

A smaller number of occupational fields, higher similarity among the occupations, and list length closer to 7 are assumed to represent higher-quality lists

*p < .05; **p < .01. +p = .07 one-tailed

Predicting career counselors’ and graduate students’ list quality ratings

We conducted two multiple linear regression analyses separately for the career counselors and the graduate students. In each analysis, the group’s list-quality ratings served as the criterion, whereas the number of occupational fields, the similarity between the occupations, and list length (i.e., divergence from 7) served as the predictors. As can be seen in Table 3, for career counselors, the three indicators explained 52% of the variance in their list-quality ratings, F (3, 12) = 6.44, p < 0.01; fewer occupational fields were associated with higher list-quality ratings (β = − 0.55). For career counselors, the other two list-quality indicators (i.e., list length and similarity among the occupations) did not emerge as statistically significant predictors of list-quality ratings beyond the number of occupational fields.

Table 3

Predictors of Career Counselors' and Graduate Students' ratings of list quality

	R2	AdjustedR2	β	t
Career Counselors1	.62	.52
Number of occupational fields2  List length closer to 7  Similarity among occupations			− .55 − .23    .24	− 2.60* − 1.24    1.18
Graduate Students	.81	.77
Number of occupational fields			− .57	− 4.13***
List length closer to 7			− .37	− 3.08**
Similarity among occupations			.29	2.19*

1The number of lists judged by the counselors was 16 rather than the 18 judged by the graduate students due to a technical omission; hence df = 12 for counselors and df = 14 for students

2Smaller number of occupational fields, higher similarity among the occupations, and list length closer to 7 represent higher-quality lists, hence β and t are negative for the number of occupational fields and list length closer to 7

***p < .001, **p < .01, *p < .05

For the graduate students, the three predictors explained 77% of the variance in their list-quality ratings, F(3, 14) = 20.23, p < 0.001. The number of occupational fields emerged as the strongest predictor of list-quality ratings (β = − 0.57), followed by list length (β = − 0.37) and the similarity between the occupations (β = − 0.29). Thus, based on the graduate students’ ratings, all three indicators contributed to list quality: the smaller the number of occupational fields, the higher the similarity among the occupations, and the closer the list length was to 7, the higher the perceived list quality.

Discussion

The prescreening of alternatives is typically the first step in many important decisions, especially those characterized by many alternatives (e.g., purchasing a new car, renting an apartment, choosing a major, or hiring a new faculty member). The prescreening stage aims to identify a set of promising alternatives and thus reduce the number of alternatives worthy of further exploration to a manageable set. Individuals deliberating about their future careers often begin by prescreening career alternatives to identify promising ones for further exploration (Gati & Tikotzki, 1989). Such exploration often focuses on considering additional information including abilities, personality, needs, and values to narrow their options (Hansen, 2005).

Interest inventories are often used in career counseling to prescreen career alternatives and identify occupations recommended for in-depth exploration (Harrington & Long, 2013; Owens et al., 2016). Previous research has not addressed how to evaluate the quality of interest inventory-generated lists of promising occupations, namely, what characterizes lists that would be helpful for the clients and facilitate their career decision-making. Two implicit assumptions underlie the recommendation to use the 3-letter RIASEC code to compile the list of occupations using the Dictionary of Holland codes (Holland & Messer, 2017b). The first is that a homogeneous list of occupations is more useful than a heterogeneous one. The second is that the number of occupations recommended for further exploration is of no consequence. The present study tested these assumptions empirically, using career counselors' and graduate students’ judgments about the characteristics of a high-quality list of promising occupations. Specifically, we hypothesized that a list with greater homogeneity (i.e., greater similarity between the listed career alternatives' attributes and fewer occupational fields on the list) and a list length closer to 7 alternatives would be regarded as a higher-quality list.

We solicited the judgments of expert career counselors and graduate students of psychology and counseling as the criterion for list quality. Rather than asking them directly, "Which kind of list is preferable for a client, a homogeneous or a heterogeneous one?" or "How many occupations should appear on an optimal list of promising alternatives?" we presented them with lists of variable homogeneity and length, and asked them to rate the quality of each list. We then analyzed their ratings to ascertain how they characterized a "high-quality" list using the three proposed indicators. As hypothesized, list-quality ratings were associated with all three indicators for career counselors and graduate students alike: greater similarity between the occupations on the list, fewer occupational fields representing the listed occupations, and a list length of greater proximity to 7 occupations.

When examining what explains the variance in list quality, the number of occupational fields emerged as the strongest predictor for both career counselors and graduate students: fewer occupational fields were associated with higher list quality. The prominence of the number of occupational fields is noteworthy, as it justifies the addition of this indicator, despite its resemblance to the similarity between the occupations on the lists.

For the graduate students, but not for the career counselors, the greater attribute-based similarity between the occupations was another contributing factor to list quality. They considered the number of occupational fields and the degree of similarity between the occupations on the list as two distinct indicators of homogeneity. Although the occupations’ similarity was also correlated with the career counselors' list-quality ratings, it did not contribute to the prediction of list quality beyond the number of occupational fields. This may indicate that for professional career counselors, the number of occupational fields is a sufficient representation of homogeneity. The career counselors' settling for a single familiar representative indicator of homogeneity––the number of occupational fields––may be explained by their professional workload. They may have passed on the need to invest the time and effort to evaluate other less easily discerned homogeneity criteria. Compared with career counselors, graduate students may be more flexible with their time and energy and more inclined to consider several indicators of homogeneity, including the number of occupational fields and the similarity between the occupations. It might also be that the disparities between the two rater samples may reflect the process of novices becoming experts. Experts may be more inclined to apply a macro level heuristics of a holistic "snap" judgment that focuses on the occupational field level to assess homogeneity. In contrast, the graduate students who are less confident in their expertise may have adopted a micro-level approach, applying a more analytic approach to processing the information.

List length explained additional variance in the list-quality ratings for the graduate students (but not for career counselors). This finding is compatible with Miller's (1956) “the magic number seven plus or minus two” and the finding that a list with seven occupations was rated as optimal by young adults deliberating about their future careers (Gati et al., 2003). The disparities between career counselors and graduate students regarding list length may also reflect a gap in expertise. Whereas graduate students regarded a list length of 7 plus or minus 2 as manageable, expert career counselors may feel more confident in managing information overload, making them less averse to considering longer lists.

As could be expected, agreement among the career counselors was greater than for graduate students. A similar pattern was observed in Shimoni et al. (2019) concerning judgments attribute-based career-preference cohesiveness. Although the graduate students' list-quality ratings correlated more strongly with the three list-quality indicators relative to the career counselors, the differences were slight and did not reach statistical significance. Future research should explore the replicability of the rating disparity between the two groups and, if substantiated, discover its possible sources.

Theoretical implications

We tested three indicators reflecting two facets of list quality––homogeneity and list length. These indicators empirically test Holland's (1997) implicit assumptions about the characteristics of a list of promising occupations. First, Holland's approach to compiling such a list using the individual's particular 3-letter RIASEC code always generates a homogeneous list derived from the particular three career types or occupational fields. The current findings supported the assumption that a homogeneous list is of higher quality than a heterogeneous list. However, the results also revealed that the number of occupational fields characterizing the listed occupations is informative: When the number of occupational fields is not constrained to three, it can serve as an indicator of homogeneity. Specifically, homogeneous lists in this respect (e.g., fewer occupational fields) were more likely to be rated higher in quality. This finding is compatible with previous research that demonstrated the informativeness of a variable number of occupational fields: when career experts were asked to interpret individuals' responses to vocational interest inventories, they tended to do so in terms of a variable rather than a fixed number of occupational fields (Gati & Blumberg, 1991) or Roe's fields (Gati, 1987).

Second, Holland's approach did not address the desirable number of occupations on the list for further exploration. Using the Dictionary of Holland's codes (Holland & Messer, 2017b), the number of recommended occupations congruent with a client's vocational personality can vary considerably, ranging from 0 (for the RAI code) to 71 (for the ESR code), and up to 210 (for all six permutations of the ESR 3-letter code; Holland & Messer, 2017b). We hypothesized that the number of occupations comprises an additional indicator of list quality. This hypothesis was based on the association between the number of occupations and the resources (e.g., time, finance, cognitive, and emotional investment) needed to follow up on the recommended list. Indeed, we found that the more proximate the list length is to 7, the higher the list-quality rating.

Prescreening in today's dynamic and uncertain world of work

Today's world of work is characterized by dynamic changes and uncertainty, which result from technological advancements, the emergence of new professions and the decline of others, and sudden, unanticipated events (e.g., the COVID-19 pandemic, fluctuations in the global economy). Despite these increasing challenges, individuals still need to make career decisions: deciding whether to attend college and, if so, which major(s) to select, which professional training to acquire, and after graduation, for which job to apply. In most of these decisions, the pool of potential alternatives is quite large, and we speculate that it will continue to increase. Therefore, the need to prescreen alternatives is inescapable, and questions regarding the optimal length or range of promising options on the promising list remain pertinent. Furthermore, determining the optimal variance among the alternatives on the promising list is even more critical for multidisciplinary professions, a growing phenomenon. Naturally, list quality in terms of length and homogeneity is also dependent on the context of the specific decision to be made and the characteristics of the individual making the decision.

Limitations and future research

Before exploring the practical implications of the present research, its limitations should be acknowledged. First, the number of career counseling experts in this study was consistent with the sample of experts participating in previous research (5 ≤ N ≤ 39; e.g., Amir et al., 2008; Lipshits-Braziler et al., 2017; Shimoni et al., 2019); however, they were only 20. The relatively small sample resulted from the challenge of recruiting experts to participate in the research without compensation. The observed agreement among the career counselors was quite high (W = 0.51), and the correlation between their and the graduate students’ mean quality ratings was high (r = 0.91); this high agreement may be due to all raters having attained their professional training in Israel. Future research should examine the generalizability of these findings using a larger and more diverse sample of experts.

Second, future research can seek additional factors contributing to list quality. First, the differentiation and the consistency of the individual's vocational interests (Holland, 1997) may affect list quality. The cohesiveness of career preferences may also affect list quality, with cohesiveness referring to the degree to which the individual's search criteria during prescreening are differentiated, consistent, and coherent (Shimoni et al., 2019). We speculate that more differentiated and consistent vocational interests and more cohesive career preferences would yield higher list quality.

In addition, how the list was compiled is likely to affect its quality. This process may include the measures used to assess vocational interests or attribute-based career preferences, the time invested, the individuals consulted with, and the extent of guidance and support received from career counselors or an online career planning system. Furthermore, in the case of online career planning systems, the quality of the database is critical and would likely affect list quality: Among the database’s critical features are the number of provided career alternatives, the scope of considered attributes, and the quality of the search algorithm.

Third, we investigated the quality indicators of lists of promising occupations that can be applied to lists of occupations that resulted, for example, from vocational interest inventories that are often used to identify occupations for further consideration. Whereas a list of seven promising alternatives was regarded as being of adequate length for deliberations concerning the choice of a major or occupation by both groups of raters, the optimal number may be different in other decision contexts. We speculate that for choosing a midlife career re-training, the optimal number of alternatives would be smaller than seven, whereas, for one's first job after graduation, it may be higher than seven. Future research can explore the optimal list length for prescreening in various career decisions.

Finally, the present study proposed and tested indicators assessing the quality of the outcome of the prescreening stage in the career decision-making process. Future research may investigate the quality of the outcomes of subsequent stages of the career decision-making process––in-depth exploration and choice (Gati & Asher, 2001). A possible criterion for assessing the quality of an in-depth exploration could be the thoroughness of the data gathering. Possible criteria to evaluate the quality of the choice stage could be the individuals' confidence in the choice they made (Amit & Gati, 2013), their satisfaction with the chosen occupations (e.g., Gati et al., 2006), lack of regret in subsequent years (Kovach Clark et al., 2009), as well as their career certainty (Tracey, 2010). Future research can also examine the optimal number of alternatives for the choice stage that follows in-depth exploration. For instance, does Tversky's (1964) suggestion that three is the optimal number of alternatives from which to choose also apply to career decisions?

Career counseling implications

Some deliberating individuals may approach a career counselor with several career alternatives already formulated. Would such a list promote or hinder career counseling and the client's career decision-making process? We believe it depends on the list’s quality. A high-quality list can facilitate face-to-face counseling when it comprises a suitable starting point for in-depth exploration. In some cases, the client's list may appear quite challenging to the counselor because it is too heterogeneous, too short, or too lengthy. When the promising list was derived by translating the client's 3-letter RIASEC code into an array of occupations using the Occupations Finder, it may be too short (e.g., only three occupations for all the permutations of IAC). In other cases, it may also be too long: We speculate that few career counselors would regard a list of 71 occupations (for an ESR code, for example) as a constructive starting point for further exploration.

To help the client who presents a too-short or too-long list, the counselor can review with them their career decision-making process to date and ask how they arrived at the alternatives on the list. The career counselor can then determine if the list resulted from a systematic prescreening or a haphazard assemblage. Then, if needed, prescreening might be conducted again, this time systematically with the counselor's guidance (Gati & Asher, 2001). However, suppose the client's initial list appears to be judiciously assembled. In that case, the counselor can collaborate with the client to refine the list for greater homogeneity and modify it to be neither too short nor too long until it is suitable for this client’s further in-depth exploration. During the following in-depth exploration stage, the counselor can guide the clients to engage in self- and environmental exploration to reduce the list length by substantiating compatibility with the client's relevant attributes (Hansen, 2005).

Implications for career planning systems

List-quality indicators can also be integrated into self-help Internet-based career planning systems, with direct implications for career counseling. Systems that provide users with a list of occupations based on their responses to an interest inventory can monitor the quality of the output (i.e., the list of recommended career alternatives). Monitoring the quality of the list of alternatives recommended for in-depth exploration facilitates prompting the individual to seek professional face-to-face counseling in cases of low list quality. Career counselors should be prepared to meet clients who come equipped with a list of recommendations made by such systems that alert the user of the low quality of the list, including the reasons for regarding it as "low quality." The assessment of the quality of the list of recommended occupations may thus serve career counseling, both directly and indirectly.

To conclude, the present research addressed two of Holland's (1997) implicit assumptions. First, compatible with Holland's approach, our results supported the assumption that a homogeneous list of promising occupations is better than a heterogeneous one for guiding further exploration of the alternatives. Second, supplementing Holland's approach, we found that the number of occupations recommended for further exploration can comprise an additional quality indicator. Thus, a list of homogeneous promising alternatives that is neither too short nor too long can facilitate progress in career decision-making. Adopting the proposed list-quality indicators can benefit career counselors, career planning systems, and, most importantly, the individuals seeking their advice.