Development and Evaluation of the Tigriopus Course-Based Undergraduate Research Experience: Impacts on Students' Content Knowledge, Attitudes, and Motivation in a Majors Introductory Biology Course.

Within the past decade, course-based undergraduate research experiences (CUREs) have emerged as a viable mechanism to enhance novices' development of scientific reasoning and process skills in the science, technology, engineering, and mathematics disciplines. Recent evidence within the bioeducation literature suggests that student engagement in such experiences not only increases their appreciation for and interest in scientific research but also enhances their ability to "think like a scientist." Despite these critical outcomes, few studies have objectively explored CURE versus non-CURE students' development of content knowledge, attitudes, and motivation in the discipline, particularly among nonvolunteer samples. To address these concerns, we adopted a mixed-methods approach to evaluate the aforementioned outcomes following implementation of a novel CURE in an introductory cell/molecular biology course. Results indicate that CURE participants exhibited more expert-like outcomes on these constructs relative to their non-CURE counterparts, including in those areas related to self-efficacy, self-determination, and problem-solving strategies. Furthermore, analysis of end-of-term survey data suggests that select features of the CURE, such as increased student autonomy and collaboration, mediate student learning and enjoyment. Collectively, this research provides novel insights into the benefits achieved as a result of CURE participation and can be used to guide future development and evaluation of authentic research opportunities.

INTRODUCTION

For several decades, evidence has suggested that engagement in authentic research practices is of significant importance for novices’ development of reasoning and literacy skills in the science, technology, engineering, and mathematics (STEM) disciplines (Holt ; Sundberg ). In response, recent efforts within the bioeducation community have focused on the development and evaluation of course-based undergraduate research experiences (CUREs)—opportunities that extend beyond traditional laboratory exercises and instead often challenge students to collaboratively develop or identify appropriate researchable questions, experimental protocols, and analytical approaches to make meaning of collected data (Auchincloss ; Spell ; Corwin ). Though the structure of these authentic research experiences has been observed to vary widely (for instance, building upon faculty research interests [e.g., Brownell and Kloser, 2015] or aligning with national models such as the Howard Hughes Medical Institute’s SEA-PHAGES program [Jordan ]), the documented impacts of student participation in CUREs have largely been positive. Using both quantitative and grounded theory approaches, Brownell , 2015) have shown, for instance, that students enrolled in introductory cell/molecular and organismal biology CUREs report a deeper appreciation for and interest in scientific research as compared with their peers completing traditional laboratory coursework. Furthermore, students enrolled in these research-intensive opportunities exhibit marked postinstructional shifts in their confidence in conducting authentic scientific research and in their ability to “think like a scientist” (Brownell )—essential mind-sets for success in the STEM workforce (National Research Council, 2003; American Association for the Advancement of Science [AAAS], 2011).

Despite the significance of these findings, the majority of empirical studies to date have used student self-reported metrics to assess participants’ attitudinal, motivational, and skills-based outcomes as a result of participation in CUREs (e.g., CURE survey; see Lopatto ; described also in Corwin ), and few, if any, have provided a comparative account of traditional versus CURE student outcomes within nonvolunteer laboratory courses at the introductory level (Spell ; Makarevitch ). Consequently, recent research indicates that these practices, which likewise include the use of unpublished or nonvalidated instruments within reported studies (Beck ), recruitment bias (Brownell ; Corwin ), and overestimation of learning and aptitude within student self-reported data sets (Boud and Falchikov, 1989; Falchikov and Boud, 1989), make it difficult to discern the true extent to which CUREs impact cognitive and noncognitive student attributes. To address these concerns, we adopted a mixed-methods approach to explore the following central questions:

What impact does participation in the Tigriopus CURE have on students’ content knowledge in the biological sciences as compared with a matched comparison group that participated in a parallel, traditional laboratory experience?

What impact does participation in the Tigriopus CURE have on students’ attitudes and motivation in biology as compared with a matched comparison group that participated in a parallel, traditional laboratory experience?

What differences, if any, exist in STEM versus non-STEM students’ shifts in attitude and motivation in biology following participation in the Tigriopus CURE?

To what degree were course and programmatic learning outcomes achieved as a result of implementation of the Tigriopus CURE?

We hypothesized that students participating in the CURE would exhibit greater expert-like shifts in attitudes, motivation, and content knowledge in the discipline than those students within the matched comparison group given the active- and inquiry-based nature of the authentic research opportunity. This hypothesis is supported by existent literature, which indicates a positive correlation between participant engagement in student-centered learning environments and affective and/or performance-based outcomes (Tai ; Hansen and Birol, 2014). Because the academic interests of students enrolled in the Tigriopus CURE were anticipated to be diverse, given the dual function of the course as both a liberal arts core (non-STEM) option and a required survey course for several STEM disciplines on campus (see Supplemental Table S1; Batzli, 2005), we likewise found it imperative to assess for potential differences in affective shifts between STEM and non-STEM cohorts enrolled in the CURE. From a broader perspective, we believed the Tigriopus CURE allowed for greater targeting of programmatic learning objectives, namely, the enhancement of students’ scientific reasoning and professional skills in the domain, in a manner that had the potential to promote learning for all students (AAAS, 2011).

The CURE described herein, and the central research questions detailed above, are novel in several aspects. To the best of our knowledge, this is the first CURE to adopt a structure in which the initial hour of each week’s laboratory meeting is devoted to engaging students in an active learning-based supplemental instruction (SI) session designed to reinforce their understanding of content presented in the lecture portion of the course. This structure was adopted intentionally to ensure that a connection between lecture and laboratory experiences remained despite the fact that the CURE was no longer aligned with core content presented in the lecture. In addition, although the structure of the Tigriopus CURE (see Methods) precludes examination of associative or causal relationships between student engagement in SI and knowledge-based or affective outcomes within the context described, previous research suggests that voluntary student participation in SI results in increases in performance on summative course exams and more expert-like attitudes toward learning in the biological sciences (e.g., Batz ).

Moreover, and with specific regard to the identified research questions, our approach offers a novel perspective on how students’ attitudes and motivation change not only in relation to participation in the course itself (whether traditional or research-intensive) but also macroscopically at the disciplinary level. As a result, our analyses provide insight into many aspects of traditional versus CURE laboratory experiences, including their impact on students’ career interest, self-efficacy, self-determination, problem-solving skills, and enjoyment in the biological sciences. These data are enhanced by students’ self-reported perceptions of learning gains related to the structure of the CURE itself. Collectively, such findings are of importance in understanding not only the immediate impacts of authentic research experiences on students’ growth in a cognitive and noncognitive sense but also how the structure of the CURE could be contributing to those outcomes as well.

CONCEPTUAL MODEL

The research presented here is situated within Corwin conceptual model for evaluation of CUREs. In this model, several critical nodes are identified that exemplify short- and long-term outcomes associated with participation in CUREs and research-driven internship experiences previously documented within the literature. These nodes, which include increased communication skills and increased analytical skills, among other factors, are further linked to centralized hubs that serve as “gateways” to achieving longer-term outcomes (e.g., persistence in science). Within the parameters of our own research, we specifically sought to use the tenets of this model, including student self-efficacy (medium-term outcome), attitudes/motivation in biology (medium-term outcome), and increased content knowledge (short-term outcome), as dependent variables designed to compare student outcomes following participation in either a traditional laboratory course or the Tigriopus CURE. Although our objective was not to explicitly test the model proposed by the authors, we elected to focus on these short- and medium-term outcomes due, broadly, to their established relationships to student success and retention in the STEM disciplines (Seymour, 2000; Tai ) and their posited links to student persistence resultant from engagement in CUREs (Corwin ). Mixed-methods approaches were likewise adopted to provide a nuanced account of the potential intersection of these measured outcomes and students’ perceptions of the Tigriopus CURE. Collectively, these data are designed to provide insight not only into student outcomes obtained from engagement in either traditional or authentic research experiences but also the structural characteristics of the CURE that could be contributing to those outcomes.

METHODS

Participant Recruitment and Matching Procedures

Participants (n = 125; 97% of sampled population) represented a convenience sample consisting of all students enrolled in an introductory cell and molecular biology CURE at a midsized, doctoral degree–granting institution in the Spring 2015 semester. For comparative purposes, a matched comparison group (n = 125) consisting of students who had previously completed the traditional version of the laboratory course during the Fall 2014 term was likewise identified. Participants were matched on several demographic factors (e.g., index score [a measure of college readiness], major, and minority and first-generation statuses) demonstrated to be significant correlates of student success in introductory science coursework (Tai ) so as to reduce confounding due to heterogeneity between groups. These demographic data were collected through the university’s system for institutional reporting (Supplemental Material). In an effort to further account for between-semester differences in student populations not captured explicitly through provided demographic information, only those students who were completing the course for the first time and who participated in all aspects of data collection were included in our analyses.

Specifically, CURE participants were first matched to one or more nonparticipants on those demographic variables referenced earlier. This resulted in exact matching on all factors excluding index score (Table 1). To account for the variation observed in this last variable, only those nonparticipants with an index score within one-half SD (± 5 units) of participants’ index scores were retained in the matching process. In instances in which multiple nonparticipants were identified as near matches to any single CURE participant, a random number generator was used to determine a one-to-one pairing. Once matching had been completed, an independent t test was performed to assess for differences in index score between groups. This analysis revealed no statistically significant, between-cohort difference in index score (t(248) = 0.065, p = 0.948). Although a high degree of homogeneity was achieved between the experimental and matched comparison groups, we acknowledge, as is also customary with similar matching procedures (e.g., propensity score matching; Stuart and Rubin, 2008), that it is only possible to account for covariates that are observable (and that were observed). In that same vein, however, it is important to note that participants were not selected on the basis of any other qualifying factors and did not receive course credit or compensation for their enrollment in this research.

TABLE 1.

Demographic data for CURE and non-CURE participants

Category	CURE participants (%)	Non-CURE participants (%)
Class standing
Freshman	55.4	55.4
Sophomore	34.4	34.4
Junior	6.3	6.3
Senior	3.9	3.9
Index scorea	103.9 (12.6)	104.0 (11.7)
Major
STEM	40.0	40.0
Biological sciences	9.6	9.6
Non–biological sciences	30.4	30.4
Non-STEM	60.0	60.0
Gender
Male	28.0	28.0
Female	72.0	72.0
Minority status
Caucasian	31.2	31.2
Non-Caucasian	68.8	68.8
First-generation status
First generation	31.2	31.2
Continuing generation	68.8	68.8
Supplemental instruction (SI)
Participated in SI	100.0	0.0
Did not participate in SI	0.0	100.0

aIndex score is reported as (M; SD) for each cohort. Index score is used as a measure of college readiness and is determined based on precollegiate metrics related to academic ability (e.g., Scholastic Aptitude Test/ACT scores, high school grade point average).

CURE Learning Goals and Suboutcomes

The primary function of the CURE described herein was to increase student engagement and confidence in implementing authentic scientific practices, including both the processes of experimental design and science communication, and to improve students’ attitudes and motivation in the biological sciences. To accomplish these goals, we developed several suboutcomes in alignment with previous research (Auchincloss ; Brownell ). Specifically, these outcomes were 1) student development of a researchable question and subsequent hypothesis, the answer to which was previously unknown by the course instructor or other students participating in the CURE; 2) student–student and student–instructor collaboration throughout all aspects of the CURE, including defined roles for each student in a given laboratory team (Table 2; adapted from Luckie ); and 3) iterative, constructive feedback on formative and summative assessments (e.g., weekly research updates, research proposals, and final oral presentations) provided by both the course instructor and students enrolled in the CURE. Specific details regarding course structure and implementation are described in the following section.

TABLE 2.

Student roles within CURE groupsa

Titleb	Assigned tasks
Principal investigator	• Organization and scheduling of group members
	• Conducting background research on question
	• Writing the introduction of the research prospectus and presenting the introduction during group oral presentations
Protocol expert	• Writing protocols
	• Modifying protocols as necessary
	• Writing the methods section of the research prospectus and presenting the research methods during group oral presentations
Data expert	• Generating graphs and tables as data were collected
	• Entering data into combined course file
	• Writing the results section of the research prospectus and presenting the results during group oral presentations
Analysis expert	• Analyzing the data and summarizing how it relates to existent research on the topic being investigated
	• Developing new hypotheses based on the data
	• Writing the discussion section of the research prospectus and presenting the discussion during group oral presentations

aModified from Luckie .

bImportantly, while students had defined roles within their teams, each student was responsible for demonstrating an understanding of all aspects of the research project.

CURE Description and Implementation

Principles of Biology, an introductory cell and molecular course, is a mandatory class for students pursuing a degree in the biological sciences or a closely related field of study (e.g., chemistry, dietetics and nutrition, nursing). This course is typically completed by freshmen during their first semester at the university, and students are required to concurrently enroll in both the lecture (three 50-minute sessions weekly) and laboratory (one 3-hour session weekly) portions of the course. While this is the case, students retain the flexibility to complete the course in either semester and do so for numerous reasons (e.g., more equitable distribution of STEM courses across their first academic year, scheduling conflicts). Therefore, matching procedures were used to account for student demographic differences between semesters (see Participant Recruitment and Matching Procedures). In the Spring 2015 semester, the traditional laboratory exercises that had historically been used (Dickey, 2003) were replaced with the Tigriopus CURE (see Table 3 for a comparison of the traditional and CURE course learning objectives and laboratory curricula). This CURE was implemented across all sections of the laboratory portion of the course.

TABLE 3.

Comparison of the traditional and CURE laboratory curricula

Course learning objectives and corresponding assessments stratified by laboratory experiencea
Learning objective	Forms of assessment (traditional)	Forms of assessment (CURE)
1. Students will be able to apply critical thinking skills to biological problems.	• Synthesis and submission of two group laboratory reports (topics include enzymes and team-designed experiment on cellular respiration or photosynthesis)	• Synthesis and submission of a group research proposal, final laboratory report, and both weekly and final oral presentations
2. Students will acquire and be able to demonstrate basic research skills, including data collection, organization, and interpretation.	• Completion of weekly laboratory exercises (Dickey, 2003)	• Completion of weekly structured exercises and independent research projects
	• “Questions for Review,” which are based on the previous week’s laboratory exercise and are submitted each session	• Preparation and submission of an individual laboratory notebook
3. Students will demonstrate a basic understanding of the quantitative methods needed to interpret data.	• Synthesis and submission of two group laboratory reports	• Synthesis and submission of a final group laboratory report and oral presentation
	• Formative assessment of in-class analyses of group data	• Statistics moduleb
4. Students will develop increased conceptual understanding in the field of biological sciences.	• Weekly quizzes, which cover material presented both in the previous laboratory and the current laboratory	• Weekly quizzes, which cover material presented both in the previous laboratory and the current laboratory

aCourse learning objectives were identical for both the traditional and CURE laboratory experiences.

bA brief description of the statistics module is presented in the article.

cFor a schematic overview of project milestones, please see Figure 1.

dTo provide further contextualization between the traditional and CURE laboratory experiences, it is important to note that independent experiments conducted within the traditional laboratory setting only involved students manipulating a single independent or dependent variable. The experimental protocol and analysis methods were preprovided.

FIGURE 1.

Analysis of CLASS-Bio metrics indicates that CURE (n = 125) participants exhibited less novice-like shifts on multiple attitudinal factors (e.g., Real-World Connections, Enjoyment) across the duration of the semester relative to their non-CURE (n = 125) peers. *, p < 0.001; **, p < 0.01.

Building upon the expertise of faculty in the department, the planktonic copepod Tigriopus californicus was used as a model system for experimental investigations. This species is a common inhabitant of tidepools along the West Coast of the United States, where it serves as the foundation of many marine food chains. Although its congener, T. japonicas, has been well studied (e.g., Machida ; Lee ), relatively little is known about the biology of T. californicus. This feature provides an ideal platform for students to pose basic, unanswered questions about the organism, including inquiries about the impact of common abiotic factors, such as salinity, temperature, and diet, on survival. Furthermore, T. californicus is amenable to rearing under laboratory conditions with minimal cost, possesses a brief yet complex life cycle, and can tolerate a wide range of abiotic conditions, making it ideal for use in a large-enrollment introductory biology course.

In addition to being novel in its topical focus, the structure of the Tigriopus CURE is likewise noteworthy. Students spend the first 6 weeks of the semester mastering basic laboratory techniques essential for success in subsequently performing independent investigations, including microscopy, pipetting, serial dilutions, and statistical analysis. These modules are traditional in the sense that they possess prescripted procedures and predetermined outcomes similar to those exercises presented in the pre-existent laboratory manual (Dickey, 2003; Supplemental Material). While students are engaged in these exercises, they are simultaneously drafting a collaborative research proposal outlining the independent studies they intend to conduct. These proposals are critiqued and the experimental designs refined at multiple time points throughout the 6-week duration (Table 3). The remainder of the term is then allocated for students to conduct authentic research, with the final week devoted to oral group presentations of research findings. This structure was intentionally adopted to reduce the cognitive load associated with students’ assimilation and enactment of technical and scientific process skills.

Furthermore, and to the best of our knowledge, this is the first CURE to incorporate a format in which the initial hour of each 3-hour laboratory session is composed of an active learning–enhanced SI session intended to increase student interaction with lecture course content rather than experimental design principles (Brownell ; Makarevitch ; for sample activity, see Olimpo, 2015). This structural aspect was adopted in response to feedback on student course evaluations and surveys (J.T.O. and G.R.F., unpublished data), as well as departmental faculty input, which indicated a strong preference for retaining a bridge between lecture and laboratory portions of the course—a concern echoed within the literature (Temple ; Banta ).

CURE Student Research Projects

As evidenced in the preceding discussion (see also Table 3), a central hallmark of student engagement in the Tigriopus CURE was the development and enactment of an authentic independent research project. In selecting foci for these inquiries, student teams elected to pursue such questions as “What effect does temperature have on egg production in T. californicus?” and “What are the effects of exposure to low-sulfur diesel on T. californicus survival rates?” Content analyses of student research proposals (unpublished data) indicated a high degree of within-section variability in the type of question posed despite repetition of questions between sections. Regardless, the resultant experiments generally required students to grow and maintain copepods within a series of 24-well plates (for the purposes of replication), which were incubated at 20°C under a 14L:10D photoperiod and supplied with Tetraselmis algae (a common food source for T. californicus), unless otherwise dictated by the team’s experimental protocol. Data collected in accordance with students’ proposed research question(s) were analyzed using basic statistical approaches, including independent t test and analysis of variance procedures, and were included both in student laboratory notebooks as well as in final written and oral presentations of students’ experiments.

Measures of Content Knowledge

To determine the impact of the Tigriopus CURE on students’ development of content knowledge in the domain, we used a multivariate analysis of variance (MANOVA) procedure to compare midterm and final exam scores for CURE versus non-CURE participants. Summative assessments were identical between groups and consisted of 50 multiple-choice questions ranked at the recall, knowledge, and application levels of Bloom’s taxonomy (Bloom ; Crowe ; Supplemental Table S2). To maintain the integrity of test items between the Fall 2014 and Spring 2015 semesters, the course instructor retained all exams in a locked filing cabinet, and students were only permitted to review their exams in her office under strict observation. Student responses were recorded on Scantron forms and scored electronically, and a total percentage correct metric was obtained. Exam scores were subsequently entered into SPSS, version 22 (IBM, New York), for the purpose of future analysis.

Measures of Students’ Attitudes and Motivation in Biology

Student attitude and motivation data were collected using the Colorado Learning Attitudes in Science Survey–Biology (CLASS-Bio; Semsar ) and the Science Motivation Questionnaire II–Biology (BMQ; Glynn ), respectively. These instruments were selected based on both their previous validation for use in majors introductory biology contexts and existent literature detailing their use to examine shifts in majors and nonmajors students’ attitudes and motivation in traditional and active learning–based environments (Perkins ; Semsar ; Batz ). The CLASS-Bio consists of 31 Likert-item questions designed to examine the extent to which students agree with expert responses on seven scales ranging from enjoyment to problem-solving strategies and reasoning in the domain. Percent favorable scores (0–100%) were generated for each participant on each scale (Semsar ), indicating their degree of alignment with experts’ attitudinal beliefs on those factors. With regard to interpretation of these findings, previous research has demonstrated that students enrolled in introductory biology courses tend to exhibit negative, pre/postsemester shifts in attitudes, indicating a decline in expert-like thinking (Semsar ). It is important to note, however, that student engagement in active learning–based STEM curricula has been found to result in no or more expert-like, pre/postsemester shifts in scientific attitudes among novices (Perkins ).

In comparison, the BMQ consists of 25 Likert-item questions regarding intrinsic and extrinsic factors related to students’ motivation in the biological sciences (e.g., career and grade motivation, self-efficacy, self-determination). Each item is scored on a scale ranging from zero to four points (corresponding with “strongly disagree” to “strongly agree”), and a total score is tabulated for each factor. Both diagnostics were administered during the first and 14th weeks of the semester in one 45-minute block so as to assess for pre/postsemester shifts in student outcomes. Student responses were recorded on Scantron forms, scored electronically, and entered directly into SPSS, version 22, for future analysis.

Overview of CLASS-Bio and BMQ Quantitative Data Analyses

Psychometric analyses indicated a high degree of construct validity (as established through expert panel and participant review processes) and reliability for both the CLASS-Bio and BMQ (Cronbach’s α = 0.847 and 0.852, respectively). MANOVA procedures were used to assess student outcomes resulting from administration of these instruments, in accordance with previously published protocols (Glynn ; Semsar ).

Student Perceptions of Learning Gains (SPLG)

To better understand how the Tigriopus CURE might have impacted student learning and attitudes in biology, we asked CURE participants to complete a brief survey during the final week of the semester. This survey consisted of two open-ended questions (“What elements or characteristics of this semesters’ laboratory experience did you find most enjoyable, and why?” and “What do you believe you learned from taking part in this experience?”) and three Likert-item questions (Supplemental Material) adapted from Kloser and based on semistructured and focus group interviews conducted with students, faculty, and graduate teaching assistants (GTAs) at the university at which this research occurred (J.T.O., unpublished data). We elected to focus on the construct of enjoyment, specifically, as a mechanism to 1) further expand upon data collected from the Enjoyment scale on the CLASS-Bio and 2) provide formative feedback on successful elements of the course itself. Such analysis was likewise motivated by existent literature, which suggests an integral role of laboratory investigations, particularly those of an authentic nature, in stimulating student enjoyment and success in the STEM disciplines (Thompson and Soyibo, 2002; Lunetta ). Open-ended responses were analyzed using a descriptive interpretive approach (Tesch, 2013), with emergent themes identified following iterative cycles of open and axial coding. Each student response was coded independently by two researchers with expertise in the field of science education. High interrater reliability was observed between coders (κ = 0.97, p < 0.001), with all disputes resolved by a third researcher with similar expertise. Descriptive statistics were tabulated for Likert-item questions, with potential rankings ranging from 1 (strongly agree) to 5 (strongly disagree), and the data entered directly into SPSS, version 22, for future analysis.

RESULTS

To control for potential bias introduced as a result of variation in laboratory GTA instructor, we first analyzed the aforementioned outcome variables using a MANOVA procedure. Results from this analysis indicated no significant difference in student outcomes based on GTA instructor (F(108,1680) = 0.97, p = 0.575, Wilk’s Λ = 0.64, ηp2 = 0.05). Furthermore, it is important to note that, although we acknowledge that natural variation will exist in how material is delivered across various semesters of a given course, the lecture instructor, PowerPoint lecture content, general pedagogical methods used (e.g., use of clicker questions and case studies), and summative assessments administered as part of the lecture courses referenced herein were identical between groups. Remaining assumptions for performing parametric tests were confirmed prior to data analysis.

Participation in the Tigriopus CURE Facilitates Students’ Development of Content Knowledge in the Discipline

To test our hypothesis that engagement in the Tigriopus CURE enhanced students’ content knowledge in the biological sciences relative to the matched comparison group, we compared students’ midterm and final exam scores between treatment groups using a MANOVA procedure. These data indicated a statistically significant difference in overall percentage grade obtained between CURE and non-CURE participants on the first and third midterm assessments (F(4,245) = 3.20, p = 0.014, Wilk’s Λ = 0.95, ηp2 = 0.06; Table 4).

TABLE 4.

Student performance on midterm and final course exams

	Traditional (M; SD)a	CURE (M; SD)	F statistic	p Value	Cohen’s db
Exam 1c	64.73 (16.13)	71.58 (15.95)	10.29	0.002	0.43
Exam 2	65.27 (16.74)	68.53 (15.58)	2.29	0.132	0.20
Exam 3	64.29 (15.94)	68.44 (15.71)	3.88	0.049	0.27
Final exam	65.13 (15.63)	69.19 (15.45)	3.86	0.051	0.26

aMean scores and SDs for each assessment are reported as percentages (ngroup = 125).

bCohen’s d values were tabulated based on individual, planned comparison analyses (unpublished data) of between-group differences in mean performance on each exam.

cThe following topics were covered on each assessment: 1) exam 1: introduction to biology, chemistry fundamentals, and macromolecules; 2) exam 2: cells and energy; 3) exam 3: cellular respiration, photosynthesis, replication, transcription, and translation; and 4) final exam: mitosis/meiosis, genetics, and cumulative review of previous material.

Importantly, though student performance on the second midterm and final course exams was not found to be statistically different between groups, CURE participants scored an average of 3–4% higher on these assessments than their non-CURE peers.

Student Engagement in the Tigriopus CURE Leads to More Expert-Like Attitudes and Motivation in Biology than Student Engagement in a Traditional Laboratory Course

Students’ overall shift on the CLASS-Bio was compared initially using an independent t test procedure to assess for between-group differences. Results were statistically significant (t(248) = −4.89, p < 0.001), suggesting that it was appropriate to proceed in further analyzing students’ responses on the remaining seven subscales of the diagnostic. This analysis, accomplished through use of a MANOVA procedure, revealed statistically significant differences between the CURE and matched comparison groups on all subscales with the exception of the Problem-Solving Difficulty factor (F(7,242) = 4.27, p < 0.001, Wilk’s Λ = 0.89, ηp2 = 0.11; Figure 1 and Supplemental Table S3).

Similar analyses were subsequently used to assess for differences in CURE versus non-CURE students’ shifts in both internal and external motivation as measured via the BMQ. Results from a MANOVA procedure indicated a significant between-group difference on all factors, excluding grade motivation (F(5,244) = 5.23, p < 0.001, Wilk’s Λ = 0.90, ηp2 = 0.10; Figure 2 and Supplemental Table S4).

FIGURE 2.

BMQ data reveal that CURE (n = 125) participants exhibited more positive shifts in intrinsic motivation, career motivation, self-determination, and self-efficacy across the duration of the semester than their non-CURE (n = 125) counterparts. *, p < 0.001.

Participation in the Tigriopus CURE Results in Parallel Shifts in STEM and Non-STEM Students’ Attitudes and Motivation in Biology

To further examine the impact of participation in the Tigriopus CURE on students’ attitudes and motivation in the discipline, we stratified data by type of major (STEM vs. non-STEM) in accordance with published classification guidelines (Louis Stokes Alliance for Minority Participation, 2015; Supplemental Table S1). Results from a MANOVA procedure revealed no statistically significant differences between STEM (n = 50) and non-STEM (n = 75) individuals who had participated in the CURE on either the CLASS-Bio or BMQ diagnostics (FCLASS-Bio(7,117) = 0.57, p = 0.783, Wilk’s Λ = 0.97, ηp2 = 0.03; FBMQ(5,119) = 0.95, p = 0.445, Wilk’s Λ = 0.96, ηp2 = 0.04; Figure 3; p ≥ 0.139 for all planned comparisons). A posteriori multiple linear regression analyses were performed, in which the 12 CLASS-Bio and BMQ factors served as individual output variables, and type of laboratory experience (CURE vs. non-CURE), major (STEM vs. non-STEM), and type of laboratory experience × major were included as input variables. This analysis indicated no significant interaction effect for any of the output variables measured (p ≥ 0.155 for all potential interactions). Collectively, these data suggest that there are no statistically significant within-group differences in attitudinal or motivational outcomes for STEM versus non-STEM participants enrolled in either the traditional or CURE laboratory experience.

FIGURE 3.

Comparison of STEM versus non-STEM students’ shifts in attitudes (CLASS-Bio; top) and motivation (BMQ; bottom) in the biological sciences reveals no statistically significant, between-group differences on these constructs as a result of participation in the CURE laboratory experience.

Increased Autonomy, Opportunities for Collaboration, and Technical Training Identified as Critical Factors Related to Student Enjoyment and Learning Following Participation in the Tigriopus CURE

In addition to explicitly performing a comparative assessment of the impact of traditional versus authentic laboratory experiences on students’ development of content knowledge, attitudes, and motivation in the domain, we sought to understand what specific features of the CURE might be mediating those observed outcomes. Descriptive interpretive analysis (Tesch, 2013; Supplemental Table S5) of student responses to those open-ended questions presented on the end-of-term SPLG revealed four emergent themes within the data set: 1) an appreciation for autonomy in all aspects of the experimental design process; 2) increased understanding of and confidence in performing laboratory techniques, as well as a perceived improvement in acquiring both general (e.g., critical thinking) and laboratory-specific skills; 3) increased interest as a result of collaborative efforts in the laboratory, leading also to improvement in written and oral science communication skills; and 4) a deeper respect and appreciation for what it means to “do” science. Importantly, our analyses indicated that these themes, albeit to differing extents, were apparent in students’ responses to both of the open-ended prompts.

Specifically, among these themes, autonomy was cited most frequently (72.8% of participants; Table 5A) as contributing to student enjoyment in the CURE, with techniques/skills and collaboration cited in relatively equal proportion (11.2% and 14.4% of participants, respectively) thereafter, and appreciation for “doing” science cited least frequently (6.4% of participants). Conversely, in regard to those self-identified factors contributing to student learning, appreciation for “doing” science and techniques/skills were cited most frequently (47.2% and 39.2% of participants, respectively; Table 5B), with collaboration (23.2% of participants) and autonomy (15.5% of participants) cited thereafter. In interpreting these outcomes, it is important to note that, across both open-ended questions, more than 85% of CURE participants’ responses were identified as belonging to two or more of the aforementioned coding categories. One student stated, for instance, that “I enjoyed all of it—learning how to use microscopes, graphing, and, finally, working with a group to conduct our very own experiments with our own test subjects. I felt like I was actually being prepared for real world research.” Responses such as this demonstrate that, in agreement with Corwin ) framework, both CURE-specific activities (e.g., collecting novel data, working collaboratively with peers) and short-term outcomes (e.g., increased project ownership, increased technical skills) are likely exerting a combinatorial influence on students’ perceptions of the CURE learning environment, including students’ attitudes in the discipline (e.g., enjoyment, a medium-term outcome).

TABLE 5A.

Student responses to the question “What elements or characteristics of this semester’s laboratory experience did you find most enjoyable, and why?”

Theme: autonomy	Percentage of student responses within theme: 73a
Sample student responses
• “I enjoyed being able to work at our own pace and on our own experiment.”
• “Conducting our own experiment was the most enjoyable because no one knew what results we would (or should) get, and if things went wrong, we had to use that [data] to modify our experiment.”
• “I enjoyed being able to design our own experiment and having that be the majority of the semester because it’s a lot more fun and interesting than the usual lab manuals that tell you exactly what to do.”

an = 125; student responses were coded into multiple categories, as appropriate.

TABLE 5B.

Student responses to the question “What do you believe you learned from taking part in this experience?”

Theme: autonomy	Percentage of student responses within theme: 15a
Sample student responses
• “I learned to be self-sufficient in the lab.”
• “I do not need a laboratory manual to tell me what to do; I can do it [design an experiment] myself.”

an = 125; student responses were coded into multiple categories, as appropriate.

Data Indicate That Course and Programmatic Goals and Outcomes Were Achieved

Owing to the novel nature of the CURE and the fact that it was developed by faculty at the institution where this research was conducted, we found it imperative to ensure that learning goals and outcomes were being achieved from both a course-based and programmatic standpoint. Student responses to the end-of-term SPLG Likert-item questions were first analyzed using Friedman’s test to assess for within-group differences on the following three categories: 1) confidence in developing scientific questions/hypotheses; 2) confidence in designing an experimental protocol to examine a research question of interest; and 3) confidence in interpreting and presenting one’s findings. Results were significant (χ2(4) = 37.54, p < 0.001), indicating a statistical difference in mean rank between categories. Pairwise comparisons were subsequently performed using the sign test, with Bonferroni correction applied. These analyses indicated no significant difference in students’ self-reported confidence in developing scientific questions or hypotheses (M = 1.95, SD = 0.73) and designing experimental protocols to test those hypotheses (M = 1.95, SD = 0.66, p = 0.742). However, a statistically significant difference was noted between students’ self-reported confidence in interpreting and presenting their results (M = 2.17, SD = 0.66) and their confidence in developing research questions and hypotheses (p = 0.007) as well as in students’ self-reported confidence in interpreting and presenting their results and their confidence in designing experimental protocols to test those hypotheses (p = 0.010). These findings suggest that, although data analysis and interpretation are integral components of the CURE described herein, future iterations of the CURE might benefit from additional attention being placed on these critical aspects of the experimental design process.

Because students from both the STEM and non-STEM disciplines were enrolled in the CURE, we wanted to confirm, furthermore, that the aforementioned course learning outcomes were being met equally between groups. To address this concern, we performed a Kruskal-Wallis test. This analysis indicated no significant difference between groups in any of the aforementioned categories (Table 6), suggesting that course learning objectives (as well as the first course suboutcome) were attained for both STEM and non-STEM participants.

TABLE 6.

STEM versus non-STEM CURE student responses to Likert-item questions on the end-of-term SPLG survey indicating their median level of confidence in developing and conducting a scientific investigation

	Mediana	χ2	p Value
Development of research question/hypothesis
STEMb	2.00	0.34	0.563
Non-STEM	2.00
Development of experimental protocol
STEM	2.00	0.05	0.833
Non-STEM	2.00
Interpretation and presenting results
STEM	2.00	0.23	0.632
Non-STEM	2.00

aOn the Likert scale: 1 = strongly agree; 2 = agree; 3 = neutral; 4 = disagree; and 5 = strongly disagree.

bSTEM participants (n = 50); non-STEM participants (n = 75).

For those interested in developing and evaluating CUREs at their own institutions, it is also important to note, from a broader standpoint, that the CURE described here not only was inclusive of those programmatic learning outcomes required for students majoring in the biological sciences at the institution at which this research was conducted (S. Keenan, personal communication), but also expanded upon those outcomes in novel ways as compared with the traditional laboratory experience previously in existence (Table 7).

TABLE 7.

Programmatic goals/learning outcomes achieved by the traditional laboratory experience versus the CURE

Programmatic goals and learning outcomes	Traditional lab	CURE
Goal 1: A broad knowledge of the core content areas in biology
Recall, explain, and illustrate major concepts and topics within the content area	X	X
Demonstrate competency in the core content areas	X	X
Goal 2: The ability to integrate new knowledge into existing scientific frameworks
Synthesize concepts and knowledge from multiple content areas and sources		X
Illustrate the relationship between existent knowledge and new findings		X
Apply core scientific concepts to new problems, including real-world applications	X	X
Goal 3: Research skills
Demonstrate and employ technical skills to collect data	X	X
Use the hypothetical deductive/scientific method to generate research questions and/or hypotheses	X	X
Design a research project to test one or more hypotheses or to answer a research question		X
Select and apply appropriate statistical or qualitative methods to analyze data		X
Interpret data to form relevant conclusions	X	X
Goal 4: Critical-thinking and quantitative reasoning skills
Integrate concepts		X
Generate a new or modified hypothesis from results	X	X
Understand and apply mathematical formulas to biological systems
Distinguish scientific thinking from subjective opinion	X	X
Formulate and evaluate alternative solutions and identify fallacies of logic	X	X
Evaluate the validity of sources		X
Goal 5: The capability to interpret and evaluate popular and primary biological literature
Distinguish between primary, secondary, and popular literature		X
Access primary, secondary, or popular literature using library resources and digital search engines		X
Comprehend the breadth, including the historical perspective, of scientific literature on a given topic
Integrate information from multiple sources		X
Read, understand, and explain primary literature		X
Goal 6: Oral and written communication skills for multiple audiences
Read and discuss both primary and popular literature		X
Research a topic and create presentations for both scientific and lay audiences
Research a topic and develop written summaries for both a scientific and lay audience
Goal 7: Professional skills
Demonstrate an understanding of scientific conduct and misconduct
Demonstrate professionalism and scientific collegiality	X	X
Collaborate with others to create a product that surpasses an individual effort		X
Goal 8: An understanding of the nature of science
Comprehend the meaning of theories and uncertainties in a scientific context
Understand that science is an iterative and collaborative process		X
Understand the goals and process of scientific peer review	X	X

DISCUSSION

For several decades, evidence has suggested that student engagement in authentic research opportunities is important for their development of scientific reasoning and process skills in the STEM domains (Holt ; Sundberg ). Current reports in the literature corroborate this assertion, indicating that not only do students participating in such experiences develop the skills necessary to “think like a scientist,” they also demonstrate a deeper appreciation for and interest in scientific research (Brownell , 2015; Kloser ). Despite these critical benefits, little remains known about the broader impact of CUREs on students’ attitudes, motivation, and content knowledge within the discipline. This is especially true for CUREs implemented in nonvolunteer course contexts. Likewise, few studies have performed a comparative examination of these outcomes among STEM versus non-STEM students engaged in CUREs (e.g., Caruso ). To address these concerns, we adopted a mixed-methods approach to broadly evaluate the implementation of a novel CURE in an introductory cell and molecular biology course, with specific assessment of the aforementioned cognitive and noncognitive outcomes conducted.

With regard to students’ development of content knowledge in the discipline, our findings indicate a statistically significant, between-group difference, in which CURE participants outperform the matched comparison group on the first and third midterm examinations by approximately half a letter grade. These data support a now well-described phenomenon within the science education community—student engagement in active learning–based exercises promotes intellectual growth (Freeman ). Furthermore, this research is in agreement with current studies of the impact of SI on student learning (Rath ; Batz ; Snyder ). Batz have shown, for instance, that struggling students who regularly attend peer tutoring exhibit increased exam performance, better attitudes in biology, and greater course persistence relative to a matched control group that did not attend the tutoring sessions. Although our current analyses preclude us from establishing a causal link between SI implementation and increases in students’ content knowledge in the domain, our data indicate that the structure of the Tigriopus CURE, as a whole, was effective at promoting this latter outcome.

In addition to differential gains in content knowledge, we observed significant between-group differences in CURE versus non-CURE students’ shifts in attitudes and motivation following participation in the laboratory experience. In support of our original hypothesis, students enrolled in the CURE exhibited markedly expert-like shifts (or less novice-like shifts) on attitudinal constructs relative to the matched comparison group, whose responses were, collectively, more novice-like in nature. Similarly, more positive motivations were observed for those individuals participating in the CURE versus those individuals completing traditional laboratory coursework. It is important to note that such shifts were witnessed across a multitude of factors, including self-determination, career interest, and problem-solving strategies, suggesting that the CURE had a wide-reaching impact as compared with the traditional laboratory experience. These data are in accordance with previous reports, which have demonstrated that student engagement in authentic research opportunities results in increases in self-efficacy and intrinsic motivation in the sciences, including increases in students’ perceived ability to prepare for and conduct studies (Drew and Triplett, 2008), greater clarification of students’ academic and/or career paths (Lopatto ; Shaffer ), and a deeper understanding of how scientists engage in real-world problem solving (Shaffer ).

In addition, the results presented herein are novel for several reasons. First, in comparison with previous studies, which have largely provided a subjective account of students’ self-reported perceptions of the CURE itself or of students’ competency in performing experimental design tasks (Lopatto ; Brownell ), the data presented herein speak more objectively to the broader, discipline-based outcomes associated with participation in authentic, course-based research opportunities. Though such metrics are not examined in this paper, we argue that they are of significant importance in offering a common foundation upon which to evaluate and compare CUREs and longitudinal student outcomes resulting from engagement in such opportunities, as they are designed independently of any one particular CURE. Second, the similarity in attitudinal and motivational outcomes observed between STEM and non-STEM participants enrolled in the CURE is critical given that introductory cell and molecular biology continues to remain a “gatekeeper” course for several other programs of study at institutions nationwide (including our own; Gasiewski ) and is identified as a liberal arts core course at the university at which this research was conducted. Consequently, and in light of persistent concerns regarding student success and retention across academic disciplines (e.g., Seymour, 2000; Whalen and Shelley, 2010; Palmer ), it is essential that the curricular and cocurricular elements incorporated into such courses serve to enhance the educational experience in a manner that maximizes opportunities for intellectual growth and professional development for all students (AAAS, 2011).

CONCLUSIONS: CONSIDERING COURSE AND PROGRAMMATIC OUTCOMES

The development and evaluation of novel CUREs requires considerable time and resources (Spell ). As such, we found it imperative to assess the degree to which student engagement in the Tigriopus CURE allowed for both course and programmatic learning outcomes to be achieved. Mixed-methods analyses of end-of-term SPLG survey data revealed that, while students perceived themselves to be competent in constructing a researchable question and appropriate experimental protocol to address that question, they were less confident in their ability to interpret and present their results. We believe this difference in perception is associated with the statistics-based analytical techniques embedded within our CURE. Though students enrolled in the CURE receive a module dedicated specifically to statistical methods (and though these methods are considered basic in terms of statistical approaches), participants have not yet had an opportunity to complete any coursework in this area. In alignment with previous research (Feser ; Makarevitch ), we argue that a greater focus on quantitative reasoning skills and quantitative literacy is needed in contexts such as the one described herein, particularly as a mechanism to increase student awareness of the relevancy of mathematics to biological sciences research. Current efforts within our laboratory are focused on the impact of curricular and cocurricular interventions designed to address these concerns and on the relationship between students’ ability to analyze research data and their beliefs about mathematics in the context of biology.

In addition to students’ perceived competence in using scientific process skills, qualitative analyses of open-ended SPLG items indicated that student learning and affect, as they were related to the authentic research opportunity, were primarily associated with four characteristics or features of the experience, namely, 1) an appreciation for autonomy in all aspects of the experimental design process; 2) increased understanding of and confidence in performing laboratory techniques, as well as a perceived improvement in acquiring both general (e.g., critical thinking) and laboratory-specific skills; 3) increased interest as a result of collaborative efforts in the laboratory, leading also to improvement in written and oral science communication skills; and 4) a deeper appreciation for what it means to “do” science. These data are supported by previous evidence in the field (e.g., Brownell , 2015) and speak directly to Corwin ) framework for evaluating CUREs. Specifically, we contend that these results provide novel insight into how CUREs impact student outcomes, with core structural elements of the Tigriopus CURE, including increased technical and communication skills, sharing a potential, positive relationship with both medium- and long-term outcomes (e.g., greater career interest in biology, increased self-efficacy and enjoyment) within Corwin ) model. Though beyond the scope of the present study, ongoing research within our group seeks to more rigorously test these assumptions and Corwin and colleagues’ model across multiple CURE experiences and with larger participant samples—an area we strongly encourage other CURE education researchers to engage in as well.

Finally, and perhaps of greatest importance to those seeking to create CUREs at their own institutions, the CURE presented here addresses several programmatic learning outcomes not achieved previously through use of the traditional laboratory experience (see Table 7). While we acknowledge that each institution and program possesses its own complement of goals and measurable outcomes, we argue that authentic research experiences of this nature are essential, in a broader sense, because they provide all students with an opportunity to engage in the research process from question and hypothesis development to presentation of results (AAAS, 2011; Bangera and Brownell, 2014), further offering a potential solution to the limited number of opportunities available for students to engage in research in independent faculty laboratories (Desai ). In addition, Corwin and colleagues (2015) state, importantly, that

Students within CUREs are legitimate participants in scientific research, because their actions contribute to achievement of research goals. However, students generally do not perform more central tasks that determine the overall direction and scope of research. For example, in many CUREs, instructors do the central work of posing overarching research questions, which help steer students in scientifically fruitful directions. Students then do the very real (legitimate) but more peripheral work of collecting and analyzing data to answer those questions. (p. 2)

As we, the educational community, continue to purposefully construct and embed authentic research experiences within our courses, we challenge this mindset by creating contexts in which students possess the autonomy to engage in real-world scientific practices that address questions of interest to them and to the scientific community at large.

FUTURE DIRECTIONS

Continued development and integration of CUREs in the biological sciences and other STEM disciplines predicates a need for objective assessment of outcomes associated with implementation of these experiences. At present, much of the research associated with CUREs, including that described herein, has focused on student outcomes following engagement in either traditional or authentic research-based laboratory coursework (see Table 1 in Corwin , for a summary of such studies). While such findings are imperative and continued studies of this nature are needed (e.g., identifying appropriate measures of students’ scientific process skills; Dasgupta ), we contend, as have others (see Corwin ), that a critical focus on alternate contextual features of CUREs is likewise warranted. For instance, though limitations in sample size within the present study preclude us from doing so, future research employing generalized linear mixed modeling approaches might seek to better understand the role GTAs have in promoting student learning and interaction in CURE versus traditional laboratory environments.

In addition, reported outcomes within the literature have historically been associated with implementation of an institution-specific CURE or, alternatively, multi-institutional implementation of a national CURE model (e.g., HHMI SEA-PHAGES [Jordan ]; Genomics Education Partnership [see Shaffer , as an example]; Brownell ). Because these experiences are each arguably unique in topic and structure, CURE–CURE comparisons using the same series of assessments and defined outcomes could be of great interest to the community (akin to Freeman ) description of “second generation” research), particularly in allowing for better articulation of which features of CUREs are beneficial and for whom. Indeed, Spell suggest that varying conceptions of what it means to engage in authentic research exist. Specifically, the authors indicate that these conceptions often focus either on the “products” of science, wherein the expectation is to generate novel questions leading to publishable results, or the “process” of science, in which students are expected to generate their own questions/hypotheses, conduct experiments, and analyze/present the findings of their study. It is not our intent, as Spell contend, to suggest that all CUREs should encompass both conceptions. Instead, we propose that a collective evaluation of such experiences, both individually as well as comparatively, when coupled with existent, ongoing efforts in the field (e.g., CUREnet: http://curenet.cns.utexas.edu), offer a potential mechanism to better represent both the small- and large-scale outcomes associated with CURE implementation across a multitude of diverse contexts.