Knowledge or Abilities? How Undergraduates Define Intelligence.

Whether students view intelligence as a fixed or malleable trait (i.e., their "mindset") has significant implications for their responses to failure and academic outcomes. Despite a long history of research on mindset and its growing popularity, recent meta-analyses suggest that mindset does a poor job of predicting academic outcomes for undergraduate populations. Here, we present evidence that these mixed results could be due to ambiguous language on the mindset scale. Specifically, the term "intelligence" is a referent in every item of the mindset scale but is never defined, which could result in differing interpretations and measurement error. Therefore, we conducted an exploratory, qualitative study to characterize how undergraduate students define intelligence and how their definitions may influence how they respond to the mindset scale. We uncovered two distinct ways that undergraduates define intelligence: knowledge and abilities (e.g., ability to learn, solve problems). Additionally, we found that students' definitions of intelligence can vary across contexts. Finally, we present evidence that students who define intelligence differently also interpret and respond to the items on the mindset scale differently. We discuss implications of these results for the use and interpretation of the mindset scale with undergraduate students.

INTRODUCTION

Students who are equal to their peers in knowledge, skills, and abilities can end up leaving science for reasons unrelated to their competence or potential as scientists. Students’ persistence and academic performance are influenced by a variety of thoughts and behaviors, such as what kinds of goals they set, how they attribute their successes and failures, and how they react when they struggle academically (Yeager and Walton, 2011; Yeager and Dweck, 2012; Smiley ; Henry ). These thoughts and behaviors are directly influenced by how students think about intelligence, referred to as their mindset (also called “implicit theories of intelligence”; Dweck, 1999). Mindset refers to the degree to which a student believes that intelligence is a trait that is stable and unchangeable (“fixed” mindset) or malleable and improvable (“growth” mindset; Dweck, 1999).

Students’ mindsets carry important implications for their academic careers. Students who believe that their level of intelligence cannot be changed aspire to prove that their fixed level of intelligence is high (Dweck, 1999). These students may avoid challenging situations that pose a risk of revealing insufficient intelligence (Henry ). Students who hold a fixed mindset are more motivated to prove their intelligence by achieving good grades or reaching other milestones (i.e., achievement goals) than they are to deeply understand the material (i.e., mastery goals; Burnette ; Smiley ). When they struggle or encounter failure, they attribute their struggles to not being intelligent enough and they can experience strong negative emotions, such as shame (Burnette ; Smiley ). They are more likely to respond to struggle and failure by engaging in helpless strategies, such as giving up and withdrawing (Burnette ; Smiley ). Ultimately, students with a fixed mindset tend to avoid challenges and be discouraged by them, which harms their academic performance (Dweck, 1999; Burnette ).

In contrast, students who believe that their intelligence can be improved aspire to learn and grow their intellectual abilities (Dweck, 1999). Thus, these students are more likely to take on challenging tasks that will help them improve their intelligence (Henry ). Students who hold a growth mindset are more motivated to gain a deep understanding of material (mastery goals) rather than prove their current level of intelligence through achievements (achievement goals; Burnette ; Smiley ). When they struggle with a challenging task or encounter failure, they attribute their struggles to the need to dedicate more effort or use different strategies and they experience fewer negative emotions (Burnette ; Smiley ). They are more likely to respond to struggle and failure by engaging in mastery strategies, such as devoting more effort to the task or trying a new approach (Burnette ; Smiley ). Ultimately, students with a growth mindset seek out challenges and are resilient to struggle, which leads to increased academic achievement (Dweck, 1999; Burnette ).

Practical efforts to enhance student success have attempted to leverage the effects of mindset by implementing interventions to encourage students to adopt growth mindsets. Some of these interventions have benefited students—improving academic performance, increasing persistence in science, technology, engineering, and mathematics (STEM), and eliminating gender and racial achievement gaps (e.g., Blackwell ; Good ; Yeager , 2019; Fink ). However, many attempts to replicate these benefits have failed to do so (e.g., Dommett ; Sriram, 2014; Orosz ; Burnette ). In fact, two recent meta-analyses have raised questions about the strength of the link between mindset and academic performance. Costa and Faria (2018) found no relationship between mindset and academic achievement among college students in their meta-analysis of 94 independent data sets. Sisk and colleagues’ analysis of 162 independent data sets also revealed no association between mindset and academic performance at the college level (Sisk ). Furthermore, Sisk and colleagues evaluated the impact of 38 mindset interventions on short-term and long-term academic outcomes, finding no effect on average. However, both studies revealed high heterogeneity of effect sizes, meaning that mindset is predictive of academic outcomes in some cases but not others. In fact, Sisk and colleagues conducted additional moderation analyses, revealing that interventions were beneficial for students from low socioeconomic backgrounds, but not for students from middle- and upper-class backgrounds (Sisk ).

These results raise the important question of why mindset is predictive of college students’ academic outcomes in some cases but not others. One possible source of this inconsistency is measurement error. There is only one scale that is widely used to measure mindset, the Implicit Theories of Intelligence Scale, which we will refer to as the “mindset scale” (Dweck, 1999; Sisk ). The mindset scale is an eight-item Likert-scale instrument in which four items measure growth mindset and four items measure fixed mindset, which are distinct but strongly correlated constructs (Dweck, 1999; Cook ). All items on the mindset scale use the term “intelligence” (e.g., “Your intelligence is something about you that you can’t change very much”). The original scale consisted of three Likert-scale items measuring fixed mindset and was later expanded to the currently used eight-item scale (Dweck ; Dweck, 1999).

The mindset scale was originally developed by Carol Dweck and colleagues for use with primary school children and has been subsequently used with students at all academic levels, including undergraduate student populations, without rigorous empirical validation of its use in these new contexts (Levy ; Dweck, 1999; Hong ). Results from the mindset meta-analyses noted earlier provide some evidence that scores from the mindset scale may be valid when used with children, but not college students (Costa and Faria, 2018; Sisk ). Both analyses show that scores from the mindset scale have stronger average correlations () with academic outcomes among younger students than older students. They found

moderate correlation in primary and middle school students: = 0.19, p < 0.01 (Sisk ); = 0.15, p < 0.01 (Costa and Faria, 2018);

lower correlation in secondary school students: = 0.15, p < 0.01 (Sisk ); = 0.09, p < 0.01 (Costa and Faria, 2018); and

negligible correlation in undergraduate students: = 0.02, p = 0.11 (Sisk ); = 0.06, p = 0.03 (Costa and Faria, 2018).

The relationship between mindset and academic outcomes may be observable in younger students but may disappear in older students because the mindset scale measures the mindsets of young children, but not college students.

Some studies using the mindset scale with college students have reported some validity evidence, such as indicators of the measure’s internal structure (e.g., Cronbach’s alpha, omega-total, and factor analyses) and test–retest reliability (Burkley , 2017; Dai and Cromley, 2014; Scott and Ghinea, 2014; Smiley ; Flanigan ). An important component of validity that has yet to be assessed with the mindset scale is response process validity, which means that respondents engage in a common set of cognitive processes when responding to the items (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014; Bandalos, 2018). For example, all of the items in the mindset scale use the term “intelligence,” so evidence of response process validity would demonstrate that students interpret the term “intelligence” consistently. However, undergraduates may bring to mind different ideas about and perspectives on intelligence as they respond to this measure. Specifically, the mindset scale asks about the malleability of “intelligence” as a unidimensional concept, yet people typically think of intelligence as a multidimensional concept that includes problem-solving ability, verbal ability, and social competence (Berg and Sternberg, 1985; Sternberg, 1985). Thus, questions about the malleability of intelligence may be difficult for students to answer if they hold differing views about various dimensions of intelligence. For example, students may view problem-solving ability as fixed but knowledge acquisition as fluid.

Not only do people tend to think of intelligence as multidimensional, there is also evidence that people differ in what they identify as important elements of intelligence and that these conceptualizations are influenced by background and experience (Fry, 1984; Berg and Sternberg, 1985; Sternberg, 2000; Buckley ). For example, teachers’ conceptualizations of intelligence are related to the age of students they teach; teachers of younger students emphasize social skills as a critical part of intelligence and teachers of older students emphasize cognitive skills (Fry, 1984). Cultural values and experiences also influence how individuals conceptualize intelligence (Okagaki and Sternberg, 1993; Sternberg, 2000, 2007). For example, there is evidence that African and Asian cultures tend to place more value on social aspects of intelligence, while Western cultures tend to place more value on cognitive aspects of intelligence (Okagaki and Sternberg, 1993; Sternberg, 2000). Lim found that Koreans emphasize social skills when defining intelligence more than Americans and other Asian groups. Other research has shown that the way people conceptualize intelligence changes as they mature (Berg and Sternberg, 1985, 1992; Yussen and Kane, 1985; Sternberg, 2000; Kinlaw and Kurtz-Costes, 2003). One study of elementary students found that, as students mature, definitions of intelligence become more varied and increasingly emphasize cognitive over social skills (Yussen and Kane, 1985). These developmental changes could contribute to the weaker association between mindset and academic outcomes in older students (Costa and Faria, 2018; Sisk ).

Considered together, these findings suggest that college students may interpret the term “intelligence” on the mindset scale in a variety of different ways, implying that the mindset scale lacks process validity when used with undergraduates. To more fully understand this problem, we conducted an exploratory qualitative study to investigate two research questions: 1) How do undergraduate students think about intelligence?, and 2) How do students’ definitions of intelligence relate to how they respond to the mindset scale?

METHODS

This study is part of a larger mixed-methods exploratory investigation of students’ mindsets and reactions to failure in a challenging, upper-level STEM course. This investigation included surveys with open- and closed-response items and interviews with a subset of participants. Data for this study are drawn from the interviews and open-response survey items. The study was reviewed and determined to be exempt by the University of Georgia Institutional Review Board (STUDY00005634).

Participants

Our study participants were enrolled in a challenging upper-level STEM course (Organic Chemistry II) at a single large southeastern public university with high research activity. We recruited participants from Organic Chemistry II because it is a required course for many professional schools and most STEM majors, including life sciences majors. Further, because mindset influences how students respond to academic struggle, we wanted to target a particularly challenging course.

Students were surveyed four times throughout the semester as part of a larger project, but only responses from the first survey were relevant to the research questions for this study. Participants were compensated with a small amount of extra credit for completing the surveys. The course enrolled 510 students, of whom 356 students completed the first survey. The research questions for this study were not a priori questions, but rather emerged during data analysis. Thus, participants were not directly asked about their definitions of intelligence; rather, 100 students spontaneously wrote survey responses that were relevant to this research question. The study sample consisted of mostly second-year students majoring in the life sciences and a small proportion of members of racially underrepresented groups (Table 1). We used students’ written survey responses to select 20 participants to interview for this study, five of whom had also written survey responses that were analyzed for this research question. We purposely selected participants who varied in their mindset beliefs and were diverse in their personal characteristics (Table 1).

TABLE 1.

Demographic information of survey and interview participantsa

	Survey respondents (n = 100)	Interview participants (n = 20)
Gender
Female	56	15
Male	43	5
Major
Life sciences	65	14
Other STEM	34	5
Non-STEM	0	1
College year
First year	2	1
Second year	67	15
Third year	21	2
Fourth year	9	2
Race/ethnicity
White	61	12
South Asian	25	4
East Asian	9	0
African American/Black	5	3
Latin(x)/Hispanic	7	1
Middle Eastern/North African	2	1
Native American or Alaskan Native	0	1
Parents’ education
Continuing generation	84	15
First generation	16	5

aCounts may not sum to 100%, because some participants chose not to respond, and participants were able to select more than one racial/ethnic identity. “First generation” indicates that none of the students’ parents/guardians earned a bachelor’s degree or higher. Life sciences indicates that students have at least one major in life sciences, including animal sciences but excluding pharmaceutical sciences. Other STEM majors are students who have at least one major in a non–life sciences STEM field as defined by the National Science Foundation, which includes the social sciences. Participants who identified with more than one race/ethnicity are counted in both groups. South Asian includes individuals identifying as Filipino, Asian Indian, Vietnamese, and other South Asian. East Asian includes individuals identifying as Chinese, Korean, and Japanese.

Data Collection

Surveys.

The first survey consisted of two versions of the mindset scale (Dweck, 1999) and two open-ended questions (see the Supplemental Material). To gain insight into the extent to which students’ views on intelligence are domain specific, the first mindset scale queried students about their views on “chemistry intelligence” and the second scale focused on “general intelligence.” After each scale, there was a constructed-response question that asked students to describe why they hold these beliefs. This question was an attempt to explore factors that influence college students’ mindsets. The variety of ways that students defined intelligence was discovered during exploratory data analysis. Because it was not an a priori question, students were not explicitly asked how they define intelligence during the survey. Rather, about one-quarter of participants (100 of 356 total responses) spontaneously explained their conceptualizations of intelligence in response to these open-ended questions. The constructed responses and responses to the mindset scale items of these 100 participants were used as one of the data sources for this study.

Interviews.

Interviews were the second data source for this study. Interviews were semistructured, meaning that we used an established outline of questions for every interview, but interviewers were free to vary the order of questions to follow the flow of conversation and ask spontaneous follow-up questions based on participant responses (Fontana and Frey, 2000). Interviews took place in the second half of the semester. During the interviews, participants were asked about their beliefs about intelligence, the academic challenges they had experienced, their approaches to coping with academic challenges, and their experiences in Organic Chemistry II (see the Supplemental Material). Questions about how students define “intelligence” and whether it is distinct from “knowledge” were added to the beginning of the interview protocol after researchers identified variation in how students define intelligence from written responses in the first survey. Analysis for this study focused primarily on answers to these questions, but any comments students made that revealed their conceptualization of intelligence were also included in the analysis. Participants were compensated with a $25 Amazon gift card for completing the interview, which lasted 45–60 minutes.

Data Analysis

Qualitative Content Analysis.

Interviews were recorded and transcribed verbatim. Written survey responses and interview transcripts were analyzed qualitatively using the software MAX-QDA 12 (VERBI Software, Berlin, Germany). While we were reviewing the data to answer another research question in the larger study, it became apparent that students were defining intelligence in different ways and that this difference was important for understanding the broader context of the student experience. Therefore, we decided to characterize how undergraduates in our sample defined intelligence. Because we were unable to identify any agreed-upon definition of intelligence in the literature or theoretical framework to explain conceptualizations of intelligence, we adopted a conventional content analysis approach, which involves deriving coding categories directly from the text data, also referred to as “inductive coding” (Hsieh and Shannon, 2005). We created codes by identifying sections of text that communicated a participant’s definition of intelligence and creating a label (i.e., “code”) that captures the idea. All authors conducted initial coding of a set of text (e.g., one interview or 50 survey responses) independently, then met to discuss to consensus. As necessary, codes were combined, split, or refined, and previous sections of text were revisited and recoded with the new definitions. During this process, one major dichotomy between the codes became apparent. We grouped codes along this dichotomy in order to create the two categories described. We also coded counterexamples of definitions of intelligence that did not fit either category. In addition to definitions of intelligence, we identified and coded related ideas, such as flexibility of definitions and how definitions appeared to relate to how students respond to the mindset scale.

During this process, five authors read surveys and interviews separately, then met to discuss coded quotes to consensus. Two authors (L.B.L. and J.C.) read all content and were involved in all coding meetings, while these authors (H.G.H., H.R.M., and A.B.) read, coded, and discussed different subsets of the data. Two authors (L.B.L. and J.C.) read and discussed all of the data to ensure trustworthiness of the interpretations, while three authors coded partially overlapping subsets of data to increase the diversity of perspectives in analysis and ensure all data had been reviewed by at least three authors (H.G.H., H.R.M., A.B.). Interview participants’ quotes are presented using pseudonyms. Survey respondents’ quotes are presented anonymously with light editing for grammar and spelling.

Mindset Scale Scores.

Our qualitative analysis suggested that students’ definitions of intelligence may impact how they respond to the mindset scale. We decided to preliminarily investigate this possibility by examining how our participants responded to the mindset scale. Only 74 participants provided comments that allowed us to categorize how they defined intelligence. For these 74 participants, we calculated a growth mindset score and a fixed mindset score by averaging their responses to the four respective items. Responses were on a seven-point scale of agreement where 1 = strongly disagree; 4 = neither agree nor disagree; and 7 = strongly agree. We plotted participants’ growth and fixed mindset scores by their definitions of intelligence. We did not calculate any statistical tests.

RESULTS

We present and discuss three conclusions drawn from the analysis of participants’ responses. First, students differ in how they define intelligence. Second, students’ definitions of intelligence are not necessarily stable across contexts. Third, students’ definitions of intelligence affect how they respond to the mindset scale.

Students Differ in How They Define Intelligence

Of the 100 respondents who commented on how they thought about intelligence, 74 explicitly defined intelligence. Students described a number of different definitions, from which we identified two categories that represented the majority of students’ conceptualizations of intelligence: intelligence as knowledge and intelligence as ability. Most participants’ definitions of intelligence could be categorized as one or both of these ideas, but we also present a few counterexamples of students whose definitions did not fit these categories.

Some Participants Defined Intelligence as Possessing Knowledge.

For these participants, intelligence refers to how much knowledge someone has about a topic. About one-third of the survey respondents (26/74) and three of the 20 of the interview participants defined intelligence as knowledge. For example, in her interview, Alice stated, “Your intelligence is your knowledge. I think they completely go hand in hand.”

Alice treated intelligence and knowledge as synonyms; she made no distinction between the two terms. Students who defined intelligence as knowledge often described knowledge as an entity that someone possesses. For example, Rina used the terms interchangeably and described both knowledge and intelligence as entities that someone can obtain. She said,

Once you obtain knowledge, whether that’s through studying or reading or observing, once you have that knowledge, you’re considered intelligent. Like, you have that intelligence.

Students who defined intelligence as knowledge were focused on one’s current level of knowledge, regardless of the effort or time it took to gain that knowledge. For example, one participant wrote,

Intelligence can be gained through hard work and commitment. Yes, some people are able to understand concepts faster. However, this does not mean that others cannot understand the same concept—it may just take more time. That is not to say that the second person is any less intelligent than the first, because they both acquired knowledge.

According to this student, how quickly or easily one learns does not determine how intelligent one is, only the amount of information one has learned.

Some Participants Defined Intelligence as Abilities.

In contrast to students who viewed intelligence as knowledge, a second group of students viewed intelligence as what students are able to do with the knowledge that they possess. About half of the survey respondents who defined intelligence (37/74) and about half of the interview participants (11/20) defined intelligence as abilities. For instance, Amy described knowledge as “knowing stuff and then intelligence is your capability to do stuff with that knowledge.”

In their definitions, students described a variety of abilities related to working with knowledge and information, such as ability to learn, solve problems, apply knowledge to new contexts, and think critically. The most commonly cited ability was the ability to learn. Participants described ability to learn as how quickly or easily one learns new material. For example, Christian explained,

I generally think of intelligence as your natural ability to grasp concepts. It can take many forms, such as being able to grasp chemistry easy [sic].… I think intelligence is key to processing knowledge. So, for example, if you have someone who [has] high literature intelligence, they are able to process and understand things taught to them. Maybe grasp concepts easier. But if you are someone who’s not skilled, you’ll be sitting there staring at this poem be like, “Alright, I see what they wrote. But I don’t really understand it.” … The way I’m thinking about [intelligence] is your aptitude.

Christian described how ability to learn can manifest in different ways, such as how easily one learns new concepts and how deeply one is able to understand complex concepts. This conception directly contradicted some students who defined intelligence as knowledge, who felt that the amount of time it takes to learn new information is irrelevant to one’s intelligence.

Problem-solving ability was the second most commonly discussed ability. For example, in his interview, George described intelligence as one’s ability to use knowledge to solve problems:

I think intelligent people have a better way of utilizing knowledge. Knowledge is something that you can have and you cannot necessarily be super intelligent. I think intellect and knowledge are, many times they’re use[d] kind of interchangeably, but I don’t think that they’re necessarily the same thing. I feel like knowledge is something that you can just memorize. You can look up something, that’s knowledge. Google is knowledge. But Google can’t solve a problem all the time for you. It knows everything in the world. But you can’t come to a solution when given a problem with just knowledge.

George’s quote highlights a notable contrast between students who defined intelligence as knowledge and those who defined intelligence as abilities. While students who defined intelligence as knowledge used the terms “intelligence” and “knowledge” interchangeably, students who defined intelligence as abilities often drew a sharp distinction between these two terms. George recognized this difference and argued that these terms are distinct, and that possessing knowledge does not make someone intelligent.

Students also defined intelligence as the ability to apply what they know to make sense of new contexts. Jade explained,

Intelligence is more like how well you can apply previous knowledge you have into [sic] situations.… Knowledge is more like content, like how many facts can you memorize and quizzes and things like that. But I feel like intelligence is what you can do with that content, with that knowledge. Like, can you understand how things relate to each other and use knowledge they [sic] have from one experience and apply it to a whole different experience to explain other things?

These ideas are related to problem-solving abilities, because both involve applying knowledge in a new context. However, some students discussed the ability to solve problems in new contexts (like George), and others focused on achieving understanding in different contexts (like Jade).

Other participants described intelligence as one’s ability to think critically about a topic, which included things like making connections between ideas and approaching new ideas with curiosity and inquisitiveness. For example, one student wrote that a more intelligent person thinks critically by asking questions about new information:

[Intelligence] is also how you react to new information. Two people with similar knowledge may both learn the same new fact. However, a more intelligent person would ask “why or how does this work” or “what evidence supports this”?

Similarly, in her interview, Lily explained that “the ability to make connections within a subject and see things from a bigger picture is intelligence.”

Ability to learn, solve problems, and think critically were all mentioned by multiple students. However, George also mentioned two abilities that no other students mentioned: spatial reasoning and creativity. George described how spatial reasoning ability is important for solving problems in organic chemistry:

Some people’s minds don’t visualize things in three dimensions. Some people can’t see a wedge and dash diagram and see it in three dimensions. Somebody can’t see a chiral center and pick it out in their hand and pick it off the paper, kind of pick it up with your head and like look it around, spin it and see the pieces. I guess that goes with intelligence. It’s just a spatial awareness kind of thing, and visualizing reactions and mechanisms and movements.

George also described intelligence as being creative and coming up with novel ideas while solving problems. He explained,

When you’re solving a problem for instance, someone that’s able to see different angles of the same problem and come up with a solution looking one way or another as opposed to one person who can just see one way. I feel like someone who is intelligent is able to solve problems in more than one way and kind of create their own means of getting to a solution.… Being intuitive and also being ingenious, and coming up with new ways of doing something. I feel like someone who’s intelligent can see things in new ways and be inventive with their thinking.

Creativity is a term like intelligence, in that it is commonly used in daily life, but poorly defined. George described a creative person as someone who is “able to see different angles” and “able to solve problems in more than one way.” Thus, we classified this idea as a subset of “intelligence as abilities.”

Some Participants Defined Intelligence as Both Knowledge and Abilities.

A smaller group of students considered both having knowledge and one’s ability to use that knowledge as important components of intelligence. About 10% of the survey respondents (8/74) and three of the 20 interview participants defined intelligence as both knowledge and abilities. One student explained it simply: “I think intelligence is just the sum of one’s knowledge and their [sic] ability to apply this knowledge.” These participants were explicit that either aspect alone was not sufficient to define intelligence, but rather both knowledge and abilities comprise intelligence.

A Few Participants Defined Intelligence as Neither Knowledge nor Abilities (Counterexamples).

A handful of students did not define intelligence at all or defined intelligence in ways that did not fit these main themes. Some participants were unsure of how to define intelligence at all. For example, when Amy was asked to define intelligence during the interview, she said: “Umm, intelligence? I’m not sure. It’s like—I don’t wanna say smart because it’s not really like how smart you are. Because you can be intelligent in different ways.… I don’t know.” Amy indicated that she did not know how to define intelligence, because there are many different forms of intelligence. She indicated that defining “intelligence” without context was not possible given its nuances and context specificity.

Three students (one interview participant and two survey participants) did not define intelligence as either knowledge or abilities. Two survey participants defined intelligence as their academic performance. One student wrote, “I took ‘chemistry intelligence’ to mean performance in undergraduate chemistry courses.”

One interview participant, Eric, had a particularly complex and nuanced view of intelligence. He rejected the idea that intelligence should be defined as abilities to process information. He was double majoring in biology and psychology, and he drew on his training in psychology to inform how he thinks about intelligence. He explained,

I know a lot of times, [in] psychological testing, you wouldn’t describe something as intelligence. You might describe an ability to do this thing of pattern recognition or ability to learn something new quickly, or the ability to memorize. You typically don’t say that that’s their intelligence.

On the basis of his understanding of psychology, he did not view intelligence as the sum or collection of particular abilities. He further reasoned that intelligence should not be defined by these abilities or performance, because external circumstances influence their abilities and performance. He said,

There could be something inherent about that person’s situation. Upbringing, resources, security, and whatever other resources, food, stability. In those ways, I think we often see that situation and we only see the results of a test. We only see these ultimate outcomes and so we might want to say there is something as inherent intelligence. But I usually stay away from using the term of intelligence to describe the situation.

Eric argued that someone could have low abilities or performance due to extenuating circumstances, such as having insufficient resources, rather than an inherent trait that is typically labeled “intelligence.”

He also rejected the notion that intelligence is how much knowledge someone has. He explained that a very knowledgeable or high-performing person who does not think deeply about concepts is not intelligent:

[For example,] this person knows a lot, but they are all over the place. I don’t see them as an intelligent person, even if they get excellent scores on things. Like, they know material, but am I going to think of them as necessarily an intelligent person if they’re not able to engage with deep ideas or anything? Probably not.… Some people don’t seem to engage with the ideas very concretely at all, they didn’t think about any deep topics or analyze their own life in any meaningful way, but they did really well on a chemistry exam. It wouldn’t make sense for me to call that intelligence.

He explained that neither earning good grades nor memorizing facts makes someone intelligent. Eric was reticent to use the term “intelligence” at all and disputed the idea that intelligence is a trait or an attribute. Instead, he was more comfortable describing intelligence as an “attitude” or “philosophy”:

I don’t know if there can really be said to be a strict definition of intelligence.… I don’t know what to call intelligence at all. But, oddly enough, I’m more comfortable seeing a person as intelligent than ascribing an attribute of intelligence to a person. Which sounds completely contradictory, but if you’re analyzing a person’s knowledge, I wouldn’t categorize that in an area of intelligence. But, if I’m analyzing a person’s being, like how they come off, I might think of them as a more intelligent person.… Usually, if I would think of an intelligent person, I would usually more think about their outlook above [their] academic situation. Or their attitude towards learning more than actual substantial knowledge.… it’s more of a philosophical system of thinking.

In sum, Eric considered but dismissed the common definitions of intelligence used by his peers. In contrast to all other participants, he did not view intelligence as a trait that could be defined or measured.

Students’ Definitions of Intelligence Are Not Stable across Contexts

In many cases, participants were unwilling to commit to a single definition of intelligence. Instead, they acknowledged that intelligence has different meanings in different contexts. Additionally, some participants discussed how their thinking about intelligence had developed over time.

Intelligence Definitions Were Context Dependent.

Many participants’ ideas about intelligence were highly context specific, but the ways in which students think about intelligence as context specific varied. Some students rejected the idea of a general intelligence that affects all contexts. For example, after responding to the first set of the mindset scale about “general intelligence,” David wrote that intelligence is so contextualized that there is no such thing as general intelligence:

These questions rely entirely on one’s definition of general intelligence. If this series of questions is taking general intelligence to mean the basic level of their intelligence, I believe no such form of intelligence exists and that all intelligence is genre specific, for lack of a better phrase.

Other students did believe that a general intelligence exists, but that there are also types of intelligence that are domain specific and that these operate by different rules. For example, another student wrote,

I believe general intelligence can definitely be changed and built. I don’t think it can change as much as chemistry intelligence can just because chemistry intelligence has more to do with finite information instead of abstract thinking. I find abstract and creative thinking to be more of something you are born with.

This student explained that general intelligence is more abstract than chemistry intelligence and that chemistry intelligence is thus more malleable, because concrete intelligence is easier to change.

Some participants who defined intelligence as abilities noted that “intelligence” is context dependent, because different tasks or subjects require different kinds of skills. For example, when asked whether there are different types of intelligence, Lydia said,

I think to answer that question you would have to break down what kind of skills are required to analyze biology or what type of skills are required to analyze math. I think there’s a reason why STEM programs are lumped together, because a lot of the same skills required to analyze math are required to analyze science and engineering, and that we see individuals in that field of study have an affinity and have a knack for problem-solving and mathematics and numbers and calculations. And we see individuals in that field have similar talents for similar things and it takes a certain brain to be an engineer, it takes a certain brain to be a mathematician. Whereas if we look at different fields like fine arts, you know a school teacher, it takes a totally different skill set for someone to be a great school teacher and to be a well-spoken public speaker. It takes a certain level of intelligence for them to be good at their craft as it does for [my professor] to be good at teaching OChem and understanding OChem and synthesizing OChem in his lab. So, I think that you have to break down into what skills are required for this craft to answer that question.

Lydia felt that intelligence is defined by one’s abilities, and as different fields of work require different abilities, then someone’s intelligence will depend on the abilities needed in that context.

Intelligence Definitions May Change over Time.

Some participants described how the meaning of intelligence changed over time as they matured academically and intellectually. Five of the 20 interview participants described how their conceptualizations of intelligence became more complex and nuanced over time. For example, Aliyah described how she used to think of intelligence in a much more simplistic manner as a younger student:

When I was younger, I just thought your intelligence was just how smart you are.… I just thought, like, “Oh, you can always get more information. You know, like you can always just know things or not know things.” … So, I thought it was more knowledge-based back then, and then I came here [to college] and somehow changed … having to really think about different things and learn different things … That changed the whole thing. It’s not really about how much you know, but your ability to learn multiple things and actually get the information.

Aliyah described transitioning from thinking about intelligence as knowledge as a high school student to now thinking about intelligence as abilities, specifically the ability to learn. However, this transition in her thinking was neither smooth nor complete. She described how her thinking about intelligence varied from day to day based on her circumstances. When asked how she defined intelligence, Aliyah responded,

Oh, man. You know, that changes so much. I don’t know. I think it depends on what day it is for me. Sometimes I feel like intelligence is about how much you can learn, maybe?… Some days, usually when we get our tests back, I think, “Yikes, maybe it is knowledge-based.”

Thus, while she tended to think about intelligence as her ability to learn, she sometimes wondered whether intelligence was defined by her knowledge base, especially when she received poor grades. For some of these students, like Aliyah, transitioning to college affected their thoughts on intelligence. Other students described changing ideas over shorter timescales, such as during a semester. For example, Lydia said,

I do think that at the beginning of this class I thought that intelligence was memorizing numbers and figures and packing information into your brain and being able to reproduce that on a test. And now I’m realizing, “Okay that’s not really intelligence.”

At the beginning of the semester, Lydia defined intelligence as knowledge, but she shifted to describing intelligence as abilities, saying, “Intelligence is comprehension of a concept. So, more than just being able to memorize a process from start to finish, but actually be able to understand why something works the way it does and not just memorizing facts.”

Students’ Definitions of Intelligence Relate to How They Respond to the Mindset Scale

Participants comments indicated that 1) uncertainty about the definition of intelligence made it difficult to respond to questions about intelligence, including the items on the mindset scale; and 2) how students defined intelligence influenced how they responded to the items on the mindset scale.

Uncertainty Made Responding to Questions about Intelligence Difficult.

Some participants indicated that they did not know how to respond to questions about intelligence without knowing how it should be defined. For example, when the interviewer asked Amy whether she believed intelligence can be changed, she replied: “Umm—I think—what is your definition? My definition of intelligence is loose. Like I’m not really sure what to apply it to.”

Amy asked the interviewer to define intelligence, because she did not know how to interpret the question. In another interview, Emily explained that the lack of a clear definition made it difficult to respond to the mindset scale. When we asked her what the term “intelligence” means to her, she answered, “That was what was tricky as I was taking the survey, because I was like, so much depends on your personal definition of intelligence.”

Sometimes students indicated that they were aware of multiple different ways intelligence could be defined and which definition they used would impact how they answered the questions. For example, after responding to the mindset scale on the survey, one student wrote,

This is an extremely subjective questionnaire since each person defines intelligence differently. If we go by the definition that it is the ability to reason and problem-solve, that can definitely be modified by study. As far as the ease by which you can learn, that can’t be changed.

This student indicated that if intelligence was defined as problem-solving ability, then he would endorse a growth mindset. However, if intelligence was defined as ability to learn, then he would endorse a fixed mindset. This student also keenly observed that, because people define intelligence differently, responses to these items are “subjective.”

How Students Defined Intelligence Influenced How They Responded to the Items.

Students sometimes explained how their definitions of intelligence related to their ideas about whether intelligence is malleable or fixed and thus how they answer items on the mindset scale. It seemed obvious to students who defined intelligence as knowledge that their intelligence would change over time as they learned new things. For example, one student wrote, “At one point in time, we all knew nothing about chemistry. So then how could people become chemistry majors or professors if we couldn’t change our chemistry knowledge and understanding?”

For these students, learning new things and the existence of experts is proof that intelligence is malleable and supported a growth mindset. On the other hand, comments from students who defined intelligence as abilities reflected both growth and fixed mindsets. For example, one student who defined intelligence as the ability to learn and problem-solve believed that these abilities can be improved through practice and training. The student explained, “I think that you can change your general intelligence. You can do this through doing brain games. These games sharpen your problem-solving ability and learning skills.”

In contrast, another student who defined intelligence as the abilities to learn and understand believed that these abilities could not be developed:

Basic intelligence isn’t something that can be changed that much. Some people learn better than others, have better memorization skills, or understand subjects better than others. Putting in work and learning new things won’t change this, it would just make someone more knowledgeable in that subject.

On the basis of these responses, we hypothesized that students who defined intelligence as knowledge would respond to the mindset scale differently than students who defined intelligence as abilities (Figure 1A). Specifically, we predicted that students who defined intelligence as knowledge would express agreement with growth items on the mindset scale and disagreement with the fixed items (orange dashed oval in Figure 1A). Conversely, students who defined intelligence as abilities would span the range of agreement and disagreement with growth and fixed items on the mindset scale (purple dashed oval in Figure 1A).

FIGURE 1.

Responses to the mindset scale by intelligence definition. Ovals represent hypothesized responses to the mindset scale by intelligence definition (A), and points represent participants’ actual responses to the mindset scale by intelligence definition (B). Overlapping data points are jittered for visibility.

We tested these hypotheses by plotting how students who defined intelligence as knowledge versus abilities responded to the mindset scale (Figure 1B). As predicted, students who defined intelligence as knowledge tended to agree with growth mindset items, and none of them agreed with fixed mindset items (orange triangles in Figure 1B). Further, students who defined intelligence as abilities spanned the range of the scale, with some expressing fixed mindsets and others expressing growth mindsets (purple squares in Figure 1B). These data suggest that the way students think about intelligence may influence how they interpret and respond to the items on the mindset scale.

LIMITATIONS

This study is an exploratory investigation of undergraduates’ conceptualizations of intelligence. There are a number of limitations to consider when interpreting the results and drawing conclusions. First, the study was limited to a single institution and included a limited number of students representing limited backgrounds and experiences. Thus, students in this sample represent a limited portion of the population of undergraduates in the United States. Given the evidence that students’ experiences influence how they think about intelligence, these results may not reflect the full range of perceptions of students with different backgrounds and educational experiences.

Furthermore, the results presented in this paper were emergent findings, rather than derived from an a priori question. The study was not designed to fully explore how students conceptualize intelligence or determine how these conceptualizations influence how students respond to the mindset scale. Rather, our results only show that there is variation in how undergraduates think about intelligence, but likely do not capture the full range of undergraduates’ conceptualizations of intelligence or how these conceptualizations relate to their mindset scale responses.

Despite these limitations, we believe it is important to present these results before conducting additional, more generalizable studies because of the widespread interest in measuring mindsets and testing mindset interventions in undergraduate STEM settings. Future studies should be designed to capture the full range of undergraduate students’ conceptualizations of intelligence from a broader, more nationally representative sample and to characterize the relationship between conceptualizations and mindset responses.

DISCUSSION

Here we begin to characterize how undergraduate students think about intelligence. While prior work has investigated how primary and secondary students conceptualize intelligence, there has been little to no exploration of how undergraduates conceptualize intelligence (Kinlaw and Kurtz-Costes, 2003). We also explore how undergraduate students’ definitions of intelligence relate to how they respond to the mindset scale, a measure that has been widely used to understand and test interventions related to mindset despite the limited evidence of its validity among college student populations. Our data indicate that there is variation in how undergraduate students define intelligence and that individual students’ definitions are not necessarily consistent across contexts or over time. This raises concerns about the use of the mindset scale with undergraduate students.

We found that there was a lack of consistency in how undergraduate students defined intelligence. There was variation in how undergraduate students define intelligence, and individual students’ definitions were not necessarily consistent across contexts. Together, these results raise questions about the utility of the mindset scale for college students. All eight items on the mindset scale use the term “intelligence” (e.g., “Your intelligence is something about you that you just can’t change very much”). Thus, when students who define intelligence differently read and respond to the items on the mindset scale, they are likely interpreting the items differently. Indeed, our data suggest that how students defined intelligence related to how they responded to the mindset scale. Specifically, students who defined intelligence as knowledge tended to agree with growth items and disagree with fixed items, because it is logically obvious that a student’s knowledge increases over time. On the other hand, students who defined intelligence as abilities varied in the extent to which they viewed these abilities as fixed or malleable and showed variation in their responses to the mindset scale. This implies that the mindset scale may not measure mindset in undergraduates reliably.

These results have practical implications for interpreting results of studies using the mindset scale with undergraduates. Both recent meta-analyses investigating mindset indicated that, on average, there is negligible association between undergraduate students’ mindset scale scores and academic outcomes (Costa and Faria, 2018; Sisk ). Our findings suggest that these null results could be the result of variation in how students interpret the items on the scale, leading to measurement error. This variation may introduce noise to mindset scale data, which may mask underlying relationships among variables.

Additionally, both meta-analyses indicated that scores from the mindset scale predict academic outcomes in younger students and are less predictive of academic outcomes among older students (Costa and Faria, 2018; Sisk ). Prior research indicates that students’ conceptualizations of intelligence change as they mature (Sternberg, 2000; Kinlaw and Kurtz-Costes, 2003). Our data corroborate this pattern; a few students in our sample mentioned how their ideas of intelligence have changed over time, tending to become more complex and nuanced. Developmental changes in how students define intelligence may lead to systematic differences in how younger and older students interpret and respond to the mindset scale. These differences could potentially explain the observed weakening of the association between mindset scale scores and academic outcomes (Costa and Faria, 2018; Sisk ).

Our results also carry practical implications for future research on mindset, which is becoming an increasingly popular topic of research in undergraduate education (e.g., Henry ). We recommend that researchers studying mindset in undergraduate student populations take steps to address ambiguous language in the mindset scale. One possible approach is clarifying the definition of intelligence. For qualitative, interview-based studies, this could involve explicitly asking participants how they define “intelligence” in order to establish a common language and ensure that researchers clearly understand their participants’ ideas. For quantitative studies using the mindset scale, this could involve providing a definition of intelligence in the survey instructions. In this case, it will be important to empirically examine whether providing a definition actually induces students to think about intelligence in the directed way. Another possible approach is to avoid using the term “intelligence” altogether. Researchers could modify the language of the items on the mindset measure or develop a new measure altogether. This should involve an extensive development and validation process to ensure that the new wording is not ambiguous and that valid inferences about undergraduate students’ mindsets can be drawn from their responses to the new items. The affordances and constraints of each of these approaches should be examined empirically.

To provide a definition of intelligence or replace the term “intelligence” with less ambiguous language, researchers will need to decide what precisely they intend to conceptualize and measure (National Research Council, 2001; Bandalos, 2018). In other words, when we conceptualize and operationalize mindset, are we really interested in students’ beliefs about the malleability of their knowledge or their abilities? We recommend using a definition or terms that align with the abilities conception rather than the knowledge conception of intelligence for two reasons. First, defining intelligence as abilities aligns more closely with the original conceptualization of mindset. As Dweck wrote in her seminal book on mindset, a growth mindset is the view that “everyone, with effort and guidance, can increase their intellectual abilities” (Dweck, 1999, p. 3). Second, our results suggest that the abilities definition is more psychometrically useful. Students who defined intelligence as knowledge all responded to the mindset scale in a similar way, whereas students who defined intelligence as abilities had greater variation in their responses. Thus, the abilities definition allows researchers to discriminate among students (i.e., provides greater discriminant validity).

Additionally, our results indicate that students’ definitions of intelligence are highly context specific. Other studies have indicated that students’ mindset beliefs are context specific. For example, students may hold different beliefs about the malleability of their own versus others’ intelligence (De Castella and Byrne, 2015; Gunderson ) and in math versus general intelligence (Shively and Ryan, 2013). Context specificity is not unique to mindset; it has been identified as a critical element of other psychological constructs as well (e.g., self-efficacy; Wang and Richarde, 1988). Together, these results suggest that researchers may wish to be specific and clear about the context of questions about intelligence to ensure that responses are relevant to the context of the study.

Finally, future research should further characterize how undergraduates conceptualize intelligence and how their definitions relate to how they respond to the mindset scale using a more diverse, nationally representative sample of students. Given that students who defined intelligence as abilities described many different types of abilities (ability to learn, solve problems, think critically, etc.), future research could tease apart how these different ideas relate to how students respond to the mindset scale.

CONCLUSION

We found that students define intelligence in different ways that are not necessarily consistent across contexts, and that these differences likely influence how they respond to the mindset scale. This raises concerns that the mindset scale may not be consistently and accurately measuring the mindsets of undergraduate students. This concern warrants further, critical evaluation by studies with larger and more diverse samples of students. It may be necessary to develop a new mindset scale to more precisely measure mindset among undergraduates.

Click here for additional data file.