An exploratory study comparing students' science identity perceptions derived from a hands-on research and nonresearch-based summer learning experience.

Although multiple efforts have been initiated to increase students' science proficiency scores, most of the schools in the United States do not reach the expected student academic performance. This study addresses the impact of a one-week summer scientific learning experience on students that worked with experimental procedures and students that did not. We describe and evaluate these two different interventions to explore what components influence high school students' perception of their scientific competence, performance, and recognition, using science identity as an analytical lens. Science identity score was increased at the end of both interventions. Interestingly, science identity change index was higher for the group that did not work with experimental procedures. Although this group did not perform any hands-on experiments, they report, through reflexive diaries and interviews that working with CRISPR-Cas9 models, being in a research laboratory, and seeing the instrumentation made them feel like scientists. Regarding science competence, both groups report exponential learning gains, although the group that performed the experiments reports more difficulties. Both groups report that mentorship was key in their competence and performance development. These findings suggest that our one-week scientific learning programs influence participants' perception of scientific competence and performance and create an opportunity to develop further studies on short scientific learning experiences using models and active learning activities.

INTRODUCTION

Student outcomes of long‐term research experiences have been reported.1, 2, 3, 4 Nevertheless, the effectiveness of short scientific learning interventions is not fully understood nor which aspects influence STEM performance, particularly for Latino/Hispanic high school students. The context and cultural aspects of students need to be considered to understand the best practices that positively influence STEM performance. Our work focuses on a group of Puerto Rican students that attend low‐performing public high schools. These schools have limited resources to provide their students access to laboratory, research facilities or diverse scientific learning experiences. Furthermore, their exposure to STEM professionals to serve as mentors or role models is limited.

We aimed to understand the impact of scientific learning experiences on a group of Puerto Rican high school students by addressing the following questions: First, how scientific learning experiences influence high school students' perceptions of their scientific competence, performance, and recognition? Second, is it critical to include authentic experiments as part of the intervention or is simulated experimentation sufficient to improve the students' perceptions of their scientific competence, performance, and recognition?

To answer these questions, we have developed and evaluated two different scientific learning experiences conceptualized on cancer research. This topic allows for the discussion of a variety of molecular biology concepts and techniques such as DNA, gene expression, gene modification tools, cell culture techniques, transfection, and ethics. Specifically, we based our scientific learning experience on essential thrombocythemia (ET). This type of cancer is characterized by an increase in platelet count, and three different mutations have been associated with this disease: calreticulin (CALR) Type I and Type II, thrombopoietin receptor (MPL), and Janus Kinase (JAKV617F).5, 6 Since CALR mutations were recently found, the molecular mechanisms that lead the CALR mutation to increase platelet count are not well known. Our intervention used the cutting edge CRISPR‐Cas9 gene modification tool to create a CALR cell line that will help us to further understand the mechanisms of the CALR mutant protein.

THEORETICAL FRAMEWORK

The framework that encompasses our study is science identity. Identity can be described as a series of representations that give meaning to the role of individuals or groups and describe “the kind of person one is seeking to be and enact in the here and now.”7, 8 A person's identity is in constant change; it emerges, evolves, and incorporates societal structures, but at the same time, endures over time and context.2

Although many science identity dimensions have been defined,9, 10, 11, 12 we have chosen to focus on the three main categories or dimensions described in Carlone and Johnsons' model: competence, performance, and recognition. Although there are other science identity influencers that are not included in this model,9, 10, 11, 12 we chose this model because this model takes into account both the intrinsic factors (self‐ concepts) and the extrinsic factors, (perception of individuals on how they are recognized by others) that frame science identity.

Competence

The perception of the students' competencies in research is defined as how the student perceives what he learns and understands. Different key points have been shown to influence the academic performance of students such as being part of underrepresented groups, low family and faculty expectations or support, low social and academic college integration, inculcation of enthusiasm, effective mentoring, and research experiences.4 Competence brings in accountable factors like logical, academic, analytical, cognitive, and retention skills in science disciplines.3 Experiences within educational and practical settings, like research, enhance academic outcomes and critical thinking.3

Performance

Performance is defined by how the student believes that they act in different relevant scientific practices such as: public speaking, doing experiments, and using equipment.2 Engaging in scientific research such as hands on experience with laboratory technical skills is tied to this component.13 Learning about science can serve as personal encouragement to go in depth with their acquired skills and pursue their goals as a science person.

Recognition

Recognition of self as a scientist is strongly influenced by the recognition from others. It is also tied to social judgment and cultural norms.2 Recognition can be viewed as an essential component to develop identity, for example, a certain pressure to thrive in studies or career can rely on satisfying a family community, or intrinsic motivation to satisfy their own recognition of self‐scientist to pursue their independent goals.2 In other words, if the student recognizes himself as a scientist and he feels recognized as such by the scientific community, then he will do what it takes to continue receiving recognition depending on his innate beliefs.2

METHODS

Design

A mixed‐methods design was used to understand the influence of short scientific learning experiences in high school students. Individuals' competence, performance and recognition categories were based on the individual perceptions. A sequential explanatory approach was used to analyze the impact of a hands‐on research experience and a hands‐on nonresearch experience. Quantitative data was collected using the Spanish translated science identity survey (SIS) before and after the experience.14 A committee of bilingual experts in translation, education, and/or biology translated and evaluated the Spanish Translated SIS. Content validation was achieved using the think‐aloud process with high school students as previously reported.14 Suggested changes were incorporated in the survey. Subsequently, qualitative data were collected using daily‐guided reflexive diaries and focal interviews. Both quantitative and qualitative findings were integrated to understand the influence of short scientific learning experiences. Survey data was used to determine a science identity score index change before and after the experience; diaries and focal interviews addressed how the scientific learning experiences influence students' perception of their competence, performance, and recognition. Cronbach's Alpha was used to measure the survey internal consistency. To address trustworthiness, relevant questions were repeated during diaries and interviews, and three independent researchers compared the outcomes.

Participants and intervention

The study was approved by the UPR Río Piedras Institutional Review Board (IRB protocol 1718‐036) and the Puerto Rico Department of Education. All participants provided informed consent to participate and did not receive any incentives for their participation. Anonymity of all participants is guaranteed.

Participants were recruited from eight low academic proficiency high schools in the municipality of San Juan, Puerto Rico.15 The participants were selected according to their (a) availability of five constitutive days during their summer break, (b) study grade level (11th grade), (c) availability and willingness to complete a pre and post‐survey, reflexive diary, and interview, and (d) parent's availability and willingness to fill out the informed consent. Since the hands‐on research and hands‐on nonresearch experience where held in two different weeks, the selection of participants for one experience or the other was according to the week in which the participants were available. Group classification (hands‐on research or hands‐on nonresearch) was not disclosed to the participants, and groups did not overlap during the intervention. In order to make groups homogeneous, the participants who were available both weeks were distributed among groups, considering the equal participation of the different schools in both groups. We recruited 40 prospect participants. From those prospects, 12 had the hands‐on research experience and 13 had the hands‐on nonresearch experience. The hands‐on research group was part of an authentic research experience using the CRISPR‐Cas9 technique and the hands‐on nonresearch group was part of a scientific learning experience using models and active learning activities (Table S1). The scientific learning experience was conducted in the facilities of the College of Natural Sciences at the University of Puerto Rico—Rio Piedras Campus. As part of the learning experience, both groups received orientation on college admission, sources of funding for college, internships, myths and realities of research, and laboratory safety practices. The instructor‐to‐participant ratio for both experiences was 1:2. The same instructors (undergraduate and graduate biology students) guided both groups on the same topics, places, amount of time in the laboratory setting, and outside the laboratory area.

Assessments

The participants answered the Spanish Translated SIS survey before and after the intervention. Students wrote daily on a guided reflexive diary to record their perception about the scientific learning experience. The guided questions of the reflexive diary explored project gains, students' perception of a scientist and its workspace, as well as participants' science competence, performance, and recognition. Additionally, the reflexive diary assessed the relevance and pertinence of the scientific project for the participants, the impact of the experience on study plans, and the contribution of the instructors and staff to their scientific learning experience. A focal group interview of 1 hr took place with six participants at the end of both interventions.

Data analysis

Quantitative data analysis

Reliability was addressed using Cronbach's Alpha, which is a statistical measure of the internal consistency of the survey.16 Measurement criterion was as followed: α ≥ .90 (high internal consistency or items may be redundant) α ≥ .80 (good internal consistency) α ≥ .70 (adequate internal consistency).17 Cronbach's Alpha and descriptive statistics were calculated using IBM SPSS Statistics software package, version 24. In order to comply with the safety requirements of the laboratory, the sample size of each of the groups was 12 and 13 students. The data was analyzed using the Mann–Whitney U test and descriptive statistics. Both analysis and figures were performed using GraphPad Prism Software. Since missing values were less than 10% of the total survey answers, the missing values for each item were replaced according to the median of the particular item.

Qualitative data analysis

The interviews were transcribed at verbatim. Content analysis was used to interpret the participants' experiences expressed in the interviews and daily reflexive diaries. A deductive approach was used to study scientific learning interventions. We focused on manifest content to code the visible and surface content of text.18 The justification for the selection of content analysis is based on the fact that our research seeks to understand the influence of short scientific learning interventions and recognize whether the pre‐selected categories permeate our data. Before beginning the analysis, we proceeded to identify the preconceived categories from the literature (i.e., competence, performance, and recognition). Data were coded according to the predetermined categories. The content of each category was compared between groups. Three independent researchers reviewed the data and the data analysis consensus is presented in the following results section. Adjectives for each of the groups were quantified and the proportions of positive and negative adjectives are reported by categories. The reported quotes were taken from diaries and interviews.

RESULTS

Participants

Participants gender distribution was similar for both groups; 3 females: 1 male. Almost all the participants lived in the metropolitan area of San Juan (93%). Age range was 15 to 18 years old and the median was 17. Mother or father highest degree obtained, field of study, or occupation is reported in Table S2.

Survey results

A Cronbach's Alpha index of .836 was obtained, confirming that the instrument is reliable (.70 = low, .80 = moderate, .90 high). Using Mann–Whitney test, a significant difference between the pre and post‐test of the hands‐on nonresearch group (p‐value <.0001) was found. A comparison between the pre and post‐test of the hands‐on research group also was performed and a p‐value of .0560 was calculated (Figure 1).

Figure 1

Science Identity Scores after scientific learning experiences. Hands‐on R stands for Hands‐on research experience and Hands‐on NR stands for Hands‐on nonresearch experience. Hands‐on NR n = 13, Hands‐on R n = 12. The lines in the scatter dot plot diagrams represent mean with SD. Significance was established at p‐value <.5; *** p‐value: <.0001

Interviews and reflexive diaries results

Participants report that they felt like scientists when they gained competence, performed like scientists, recognized themselves as scientists, and perceived that others recognized them as scientists.

In the next sections, we report (a) the students' perception of their science competence, performance, and recognition before the scientific learning experiences; (b) perception changes, if any, after the scientific learning experience, and how the perception changed; and (c) which components of the scientific learning experiences influenced students' perception of their science competence, performance, and recognition. To support each of the main points of this article we included translated (Spanish to English) quotes of the participants. Spanish language original quotes are available in the Supplemental Material section (Supplemental Material 1).

Competence

The 74% of the participants explained that before the scientific learning experience their knowledge about DNA, cancer, and gene modification techniques was either very basic, none, or wrong. Also, they commented that these topics were difficult or confusing to understand at school. When the participants compared our intervention with the school experience, they expressed that the school did not offer demonstrations of the studied material, it only focused on the lecture, and they did not have confidence to clarify concepts with their teacher. On the other hand, through the intervention, they had the opportunity to go to laboratories, ask questions, and understand topics much more easily. Interestingly, participants reported an exponential learning throughout the program.

“the teachers that I've had are like: we sit to read a book and we start to discuss and discuss. There is no dynamic, it's like they don't demonstrate [the material].”

“… [Teachers] should take us to the universities, into the laboratories to teach us, to have a bit more demonstration, and make it more of a dynamic experience, it should be much more different, they don't have to lock you up in a classroom with a book, it's not the same, because here, in this program, for example, we had the opportunity to go to a laboratory and be able to be with all of you [instructors] …”

“I did not have the confidence to ask… and now…, I feel that I can ask about my doubts and I know that they will clear them up.”

“everything that was discussed, was done in a much easier and interactive way for my style of learning…”

“…here, in only a few hours, they taught me what the school couldn't do in a year, and I'm not saying it just to say it, I say it because it's true.”

“this program makes up for all of the years that I have been in school. The school has not properly taught me some things that this program has…”

Mentorship

Participants emphasized the influence of instructors in their gain of competence. They highlighted that the instructors explained concepts in different and creative ways that allowed them to understand very easily. Several participants commented that their interaction with the instructors was very comfortable, which allowed them to have the confidence to ask and clarify their ideas.

“the instructors explained very well and with lots of patience to understand it.”

“I liked this project because the staff was very well prepared and willing to help us as many times as necessary.”

“That confidence that the staff gave us (instructor name)… since the day that I came… I could ask questions… about his life in the university, how he could control or use money from university scholarships for his studies, his personal things like gasoline, emergencies, food, car, rent, and a lot of advice, it was of great help.”

“In school I felt embarrassed to ask and here I felt comfortable, I felt at ease to ask anything to anyone, be it my instructor or others, or any of my peers. I truly think that the comfort that I felt was the reason I was able to learn.”

“…and then here they taught me, and made sure you understood, that you learned, and they took their time to explain it again if you did not understand.”

Performance

Participants acknowledged the fact that being in the laboratory influenced their science performance. In the laboratory, they learned what instruments looked like and what their function was. Participants expressed that in the program they were not just part of traditional, static lectures as in school, but they were part of the research and experimentation.

“from the dynamics, they took you to the laboratory, you could see what was really happening. You could see the cell count, we could see the instruments that were used for the cell culture and that was what I liked.”

“but I think the difference is that there [school] they told you more and that here [project] they explained to us too, but we also did experiments… “

During the course of both programs, participants reported that multiple practices made them feel like scientists such as the: use of the scientific method, experiments, analysis, present their findings, coming up with ideas, explain and discuss what they have learned, and find information. They expressed that seeing and counting cells in the microscope, and the genomic comparison of the human being and the mouse in the database of NCBI were the most significant experiences of the program.

“talking or sharing my knowledge with my classmates was cool.”

“I felt like a scientist while we were talking and discussing about our research”

“at the end of the project to be able to explain what was learned”

“I had many opportunities to do things that a scientist does such as see cells and do research.

Self‐instruction

Participants described that the scientific learning experience prompted them to read biology books, to find more information, ask, and learn more about science. The experience aroused an interest and desire in them to keep looking beyond the information that was provided.

“My performance changed because I am a person that likes to read but not much about science… and when I got here, it was like “boom” and I got home, and I started to see science related things and research on my own different topics. Being able to search for information by myself and elaborate an idea, that was something that changed in me “

Recognition

Recognition of self

Participants commented that they felt like “scientists” when they participated in the activities and demonstrations, but more than anything when they entered the lab, used a pipette and observed cells through a microscope. Also, participants described that they felt like scientists when they searched for universities, understood concepts, answered questions, and discussed what they learned. They shared that they felt like professionals and, due to the acquired skills and knowledge in the program, almost all of them felt like scientists.

The 84% of the participants believed that they were capable to contribute to the scientific community. Those who used the following statement, “scientific work is very complicated” were deemed not able to contribute to the community. On the other hand, those who report that they were able to contribute to the scientific community, focused on what they were able to accomplish during the experience, for example, they were capable of following the scientific method, solve problems, and find an answer. Others focused on their enthusiasm, dedication to their studies, what they like, what they do on their own as scientists, and their careers goals. Interestingly, some of them commented that they were able to contribute to the community because of what they were doing in the learning experience. They felt that they can contribute to science research and they included themselves as part of the research community. Participants understood that what they were doing is relevant to the community because they were able to understand the disease and potentially find a cure so lives could be saved. Other participants commented that they felt like part of the scientific community because their results could be useful to cure the studied disease.

“the information we obtain can benefit by helping experts to be closer to a cure”

“if we know more about this disease, we can create a remedy”

Career goals

Before the scientific experience, participants' interest on STEM careers was very diverse. A total of 36% of the participants were already interested in STEM and the experience helped them to awaken their interest in science and see other fields in STEM. Before the experience, 16% of the participants were interested in non‐STEM fields, and after the scientific learning experience, they wanted to pursue a career in STEM. For other participants (48%) this experience helped them feel empowered and follow their dreams.

“For what I want to study, a lot of science is needed, and this woke up my interest in science”

“After I finished high school I wanted to study only cosmetology and after the experience in Science in Action [the summer program], I want to study or know the world of chemistry, which interests me a lot.”

“After the experience I want to continue and improve my studies in order to graduate from a good University”

Recognition from others

Some participants (48%) reported that they perceive that others see them as scientists and the 52% of the group perceived that they are not seen as scientists. Those that reported that they are not seen as scientists, explained that because they do not see themselves as a scientist, others will not see them as such. In addition, they reported that they do not see themselves as scientists because of their performance, and competence: “it is not my vocation,” “I am not a professional,” “I do not act as such,” “It is hard for me to learn science.” Participants that reported that they perceive that others recognized them as scientists point out that it is because of their competence and performance: “I like investigating on my phone,” “I am always asking questions about everything. “I love inventing things,” “I like helping people,” and “because I have curiosity, passion, and potential.”

Hands‐on research versus hands‐on nonresearch experience

In order to understand if experimentation is the key component of scientific learning experiences that help high school students to perceive themselves as scientists, we divided our cohort of participants into two groups: one group performed experiments (research) and the other group developed models with crafts (nonresearch). When science identity change index scores were compared between groups, we found that the score for the nonresearch group (median 14) was significantly higher (p‐value: .0006) than the research group (median 5.5) (Figure 2). To understand why nonresearch group had a higher science identity score, we compared the diaries, interviews, and the use of adjectives to describe the experience for both groups. Interestingly, the group that performed the experiments described their competence and performance with more negative adjectives (competence:19%; performance 29%) than the comparison group (10%; performance 20%), for example they mentioned that the learning process was complicated or difficult (Table 1).

Figure 2

Science Identity Survey Index Score Change. Hands‐on R stands for Hands‐on Research experience and Hands‐on NR stands for Hands‐on nonresearch experience. Hands‐on NR: n = 13 Hands‐on R: n = 12. The lines in the scatter dot plot diagrams represent mean with SD. Significance was established at p‐value <.5; *** p‐value: .0006

Table 1

Students' reported percentages of positive and negative adjectives for science identity pre‐determined categories

	Hands‐on nonresearch		Hands‐on research
	Positive (%)	Negative (%)	Positive (%)	Negative (%)
Competence	90	10	81	19
Performance	80	20	71	29
Recognition	100	0	100	0

Participants of the nonresearch experience report that they were experimenting when they were observing the cells in the microscope and working with models. The group that was part of the authentic research experience highlights its performance describing the techniques that they learned: the use of the pipette and other instruments, and the cell culture work. Both groups report that they were able to experiment. No differences between groups in terms of the participants' self‐recognition as a scientist were found.

DISCUSSION

Previous studies have found that summer or long‐term research experiences influence students' academic outcomes.1, 3, 19 These experiences allow participants to grow professionally, intellectually, and reinforce their critical thinking skills. Similarly, our participants reported that the one‐week scientific learning experiences had influenced their critical thinking and learning. Moreover, demonstrations, mentorship, laboratory setting, and the ability of students to ask questions and clarify their doubts (as described by participants in their reflexive diaries) influenced participants' perception of their competence and made them feel like scientists. Our results demonstrated that the science identity of both groups increased at the end of the learning experience, indicating that they did feel like scientists.

Studies on long‐term research experiences also have shown that scientific experiences develop students' critical thinking, their ability to be independent, and develop students' own ideas.1, 3, 19 Similarly, our results show that participants' performance was influenced by their ability to instruct themselves, find information, develop their own ideas, and explain what they have learned. Noteworthy, both groups claimed that they performed experiments although the hands‐on nonresearch experience group only practiced demonstrations and active learning activities. These results suggest that participants' interpretation of scientific tasks influences participants' perception of their performance and ultimately their science identity. Because our students' performance was based on self‐perceptions and not directly tested, it would be interesting to add direct assessment of skills gained. Furthermore, it would be exciting to follow the longer‐term effects of this study. We would expect that both performance and retention would be improved in both groups. These are subjects of ongoing studies in our laboratory.

A growing amount of evidence points out that mentorship is key for students' retention and persistence in STEM.20, 21, 22 Interestingly, Daniels et al. showed that mentoring is more significant and efficient to promote students personal, and skill gains than the time spent doing research.23 Our results support these findings; although time spent in research is important, mentorship is key to develop students' confidence, competence and performance. Participants' gained confidence in themselves from interactions with mentors and, due to this interaction, they report that their future career plans and educational possibilities in science have broadened. Students also report that being with younger scientists helped them unravel the paradigm that just older people are scientists; they realized that they could contribute to science too.

These findings suggest that our one‐week scientific learning experiences enhance participants' science identity and create an opportunity to develop further studies on short interventions using models and active learning activities. Although engaging in experimentation, as reflected by participants' comments, is very important for the development of their perception on scientific performance, other components such as mentorship, laboratory setting, and active learning activities influence participants' perception of their competence and performance.

Interestingly, through the reflective diaries and interviews, participants of the research learning experience described the experience with more negative adjectives (such as difficult, complicated, and deficient) than the comparison group. We infer that this may be a result of their naivety doing research experiments and their lack of experience in problem solving strategies that are required during authentic‐research activities. Problem‐solving tasks may be a challenge for those who are not used to laboratory work or have low confidence performing activities that require analysis.24 We can also not dismiss that this perception was influenced by the short duration of this learning experience, as this observation has not been made in long‐term research experiences.1, 2, 3, 4

The effectiveness of hands‐on research experiences is well established in the literature.1, 2, 3, 4 Our finding is novel as our work suggests that it may be possible to develop science identity at an early level (i.e., high school) with short term, low‐budget, hands‐on scientific learning experiences. The significance is highlighted by considering that our population, like many other underrepresented minorities, is normally deprived of significant scientific learning experiences and/or laboratories due to budget or space limitations. Furthermore, their access to researchers or STEM professionals to serve as mentors or role models is very limited. Hence, we suggest that inexpensive, short, hands‐on nonresearch activities can be implemented and may have a positive impact on students' science identity.

Since both research and nonresearch scientific learning experiences had a positive influence on students' science identity, we suggest the use of either method to develop science identity in high school students. These findings can guide efforts for the development of low‐cost strategies (i.e., hands‐on activities and demonstrations) that can be easily implemented in high school classrooms. Our findings support, and we encourage, the collaboration between high schools and Universities to provide high quality mentorship to high school students.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

AUTHOR CONTRIBUTIONS

L.H.M. wrote the manuscript, conducted the interviews, coordinated the program activities, analyzed the data, prepared most of the program lectures and revised all lectures. Also coordinated and offered the instructors training, and created Table 1, Supplemental Table 2. L.P.D. was the co‐coordinator of program activities, prepared some of the program lectures, served as instructor of the program, analyzed statistical data, did the Supplemental Table 1, and formatted part of the manuscript for submission. F.L.T. was the co‐coordinator of the program and activities and prepared some of the program lectures. N.C.J. prepared some of the program lectures. S.M.T. coordinated some program lectures and activities, served as instructor of the program, wrote part of the introduction of the article, and translated the participants' quotes. P.L.R. gave remarks in the analysis. A.V.W. assisted on the logistic plan, lectures to the students and writing of the manuscript. M.B. took part on the logistic plan and program content coordination. Also, M.B. assisted on the manuscript writing. All authors reviewed the final manuscript.

Supplemental Table 1 Description of the intervention, activities and topics covered for each group

Supplemental Table 2. Mother or father highest degree obtained, field of study, or occupation

Click here for additional data file.