A Single, Narrowly Focused CREATE Primary Literature Module Evokes Gains in Genetics Students' Self-Efficacy and Understanding of the Research Process.

Exposure to primary literature using CREATE tools has been shown to have a positive impact on students' self-efficacy and beliefs when incorporated into semester-long courses taught by extensively trained faculty. However, it is unknown whether similar benefits can occur with a brief exposure to CREATE in an otherwise traditionally taught course. We hypothesized that students who experienced a short-term CREATE module taught by faculty with minimal training in this pedagogy would make gains in scientific literacy and self-efficacy while also experiencing epistemological maturation. To test this hypothesis, we compared sections of students who experienced the CREATE module with sections of the same course taught without CREATE. Our hypothesis was partially supported by the data in that students in CREATE sections made significant gains in self-efficacy but did not gain transferable data analysis skills. Students in those sections also self-reported significantly enhanced understanding of the research process. Thus, this study suggests that analysis of primary literature using CREATE, even in short modules, can significantly and positively affect students' self-efficacy and their views of science. ©2020 Author(s). Published by the American Society for Microbiology.

INTRODUCTION

Recent writings on undergraduate STEM education emphasize the importance of students learning how scientific knowledge is generated (1–4). Many believe that hands-on lab work is the best way to expose students to knowledge creation, but standard labs have been criticized as involving more demonstration and replication than discovery (5). Open-ended inquiry labs (6, 7) offer some advantages but are limited in scope and can be expensive and difficult to implement across multiple laboratory sections. Semester-long classroom-based undergraduate research experiences (CUREs) increase students’ attitudes, motivation, and content knowledge (8–11). However, these valuable experiences also require a great deal of coordinated organization, lab space, equipment, consumable reagents, course assistants, and/or easy student access to computing resources. Thus, colleges and universities with limited resources, such as community colleges, and, all too often, institutions serving primarily underprivileged and underrepresented students, may be limited to a single simple CURE experience or may not offer one at all.

Additionally, CUREs focus on topics and principles of the scientific process that can easily be addressed in short laboratory sessions over the time frame of one semester. Other interesting areas of science (e.g., astronomy, explorations of deep ocean vents, virology research) are not directly accessible through CUREs. As such, CUREs are limited to certain areas of study and only provide students insight into a short-term inquiry.

Close study of primary literature, in contrast to inquiry labs or CUREs, offers an inroad into any scientific topic as well as many aspects of the process of science that are difficult to address in a semester-long laboratory experience. These include the reiterative nature of science, creativity in experimental design, and the diversity of potential experimental directions. However, such literature has traditionally been considered too difficult for undergraduates. Undergraduate biology majors do not routinely read foundational documents such as Darwin’s The Origin of Species or Watson and Crick’s 1953 paper on DNA structure. Nor are undergraduates typically challenged with papers on breaking developments, especially in introductory courses or courses not specifically dedicated to scientific literature. Instead, many courses rely solely on textbooks that, due to the explosion of information in biology, cannot provide in-depth coverage of topics and often fail to adequately address the research process or scientific discourse (12, 13). Reliance on textbooks can lead instructors to “cover” a myriad of facts at a superficial level, an outcome that has been criticized for miring students in a morass of facts rather than developing their research skills and understanding (14, 15). A sense that “biology is overwhelming” has for decades been linked to the exit of able students out of the major (16). While efforts have been made to correct this situation—the creation in 2013 of a valuable data-focused textbook aligned with recommendations of the Vision and Change project (17) is one example—the effective use of primary scientific literature offers a compelling way to improve student learning in STEM courses.

CREATE was originally tested as a novel method for using primary and other scientific literature in undergraduate elective courses that emphasize a specific research topic and the process of scientific inquiry (18). Taking advantage of the narrative nature of science, the strategy uses intensive analysis of interrelated papers to demystify and humanize the research process. Students in CREATE courses have made a variety of cognitive and affective gains, in first-year (19, 20) and upper-level electives (18, 21) at a minority-serving institution, as well as in a variety of four-year (22, 23) and two-year (24) college/university settings. All implementations of CREATE tested to date were semester- or quarter-long topical courses taught by faculty who learned the CREATE technique through multi-day faculty development workshops (22–24). Given the benefits of the original CREATE approach, we are interested in devising novel ways to apply the technique, for example by using it to teach key concepts in the research process as part of a required, more-traditional course with minimal faculty training in using CREATE tools.

We designed a short CREATE module to be taught in parts of three to four class sessions by a faculty member with minimal CREATE training. We tested this short “dose” of CREATE in a required content-rich course (Introduction to Genetics), with the expectation that the module would provide insight into the process of science and help develop students’ skills with primary literature. We compared students who experienced the CREATE module with those in different sections of the same course taught without CREATE. In this way, we tested the hypothesis that students who experienced a single CREATE module taught over a few class periods would make gains in scientific literacy and self-efficacy while also experiencing positive shifts in epistemological beliefs about scientific skills and knowledge (18–24). Our hypothesis was partially supported by the data in that students in the CREATE sections made significant gains in self-efficacy but did not gain transferable data analysis skills. Interestingly, both CREATE and comparison students felt they gained significantly in literature analysis ability over the semester, but only the CREATE sections showed significant gains in self-rated reading skills and in their sense that reading papers deepened their understanding of how research is done. Additionally, students in the comparison sections, but not in the CREATE sections, showed significant “regression” of one aspect of epistemological views of science during the term, ending their course with more agreement that “knowledge is certain.” Thus, this study suggests that analysis of primary literature using CREATE, even in short modules, can significantly and positively affect aspects of students’ self-efficacy and their views of science.

METHODS

Study overview

To test condensed CREATE methodologies in a required course, we designed a CREATE genetics module around a single paper and tested it in six iterations of a required introductory genetics course on a campus in the northeastern United States. Outcomes in these cohorts were compared with outcomes in six iterations of the same course that did not use the module. We used this study design to test whether a limited exposure to CREATE teaching would have an impact on students’ science literacy skills, and/or their self-rated attitudes/abilities and epistemological views of science.

Module development

The module (Appendix 1; Table 1) was jointly developed by three faculty experienced in CREATE methodology to specifically address genetics concepts that are often difficult for introductory students. To identify these challenging topics, we surveyed faculty experienced in undergraduate genetics teaching from multiple campuses. From these topics, the topic of mutation and phenotypic change was selected to help reinforce concepts related to central dogma, alleles, and dominance. The PLOS Genetics paper “A Mutation in the Myostatin Gene Increases Muscle Mass and Enhances Racing Performance in Heterozygote Dogs,” by Mosher et al. (25), was selected due to its focus and potential accessibility for undergraduates. We provide a delineated script as well as homework assignments used by implementers to apply the CREATE toolkit (Appendix 1 a, b). The pace of the module was determined by the length of a class session. Students completed associated activities across parts of three to four class periods.

TABLE 1

CREATE tools, their purpose, and how they were used in the Genetics module.

CREATE Tools	What the Tool Encourages Students to Do	How the Tool Is Used in the Genetics Module
Concept mapping	Critically read an introduction Define what they do and don’t know about a topic and look up concepts they do not understand Relate old and new knowledge Review to fill gaps in understanding	Students look up unfamiliar concepts Students use a concept map as a springboard for review of fundamental genetics concepts Students develop an understanding of the experimental questions and approaches of the paper
Paraphrasing	Read closely Look up unfamiliar words Learn to express key concepts in their own words	Students paraphrase the paper’s title after concept-mapping the introduction
Sketching	Visualize the experiments by representing “what went on in the lab” in a drawing Link specific methods to specific data obtained Triangulate information in methods/captions/narrative Construct a context for the data	Students sketch the experimental design used in the paper using figures, legends, text narrative, and methods
Elucidating hypotheses	Define in their own words the question being asked or the hypothesis being tested in experiments related to each figure or table	Students define questions being addressed in each figure
Annotating figures, interpreting data	Actively engage with data Determine the significance of each figure Closely read captions and narrative Prepare for in-class analysis of the data’s significance Define in their own words the question being asked or the hypothesis being tested in experiments related to each figure or table	Students annotate all figures Students look closely at how data were collected and the extent to which measurements were standardized (or not)
Designing a follow-up experiment	Recognize research as a never-ending process Exercise creativity in experimental design Consider that multiple options exist; science is not necessarily linear and predictable	Students list potential “next steps” for the research project Class discusses study follow-up ideas
Grant panel exercise	Consider how research funding decisions are made Use critical analysis to rank student-designed experiments Develop verbal ability by pitching/defending particular experiments Learn to work in small groups and reach consensus	Not part of the Genetics module due to time limitations
Email interviews of paper authors	See scientists as humans much like themselves, not stereotypes of pop culture Make personal connections to research/researchers Get their own questions answered Recognize diversity of personalities—that all can be “scientists”	Students read and discuss transcribed interview with study PI, who addressed student-generated questions

Adapted from Hoskins SG, Stevens LM. Learning our L.I.M.I.T.S.: Less is more in teaching science. Adv Physiol Educ 33:17–20, 2009.

Course description

The genetics module was implemented in a required introductory genetics course with lecture (150 min/week) and laboratory (165 min/week) components. The course is required in the first semester of the Biochemistry major and the second semester of the Biological Sciences and Bioinformatics majors. Thus, most of the students in the study were first-year undergraduates. Enrollment in each section is capped at 26 students and the course is typically taught in an active learning environment, with a mix of lecture, problem solving, and group activities including data interpretation using figures from primary sources. Thus, the “comparison sections” used primary literature, but not in the way outlined in the CREATE module. The module was implemented in the last third of the course. Because students self-selected into particular course sections based on their schedules, we could not randomize students into the various sections.

Instructor recruitment and training

Eight faculty participated in this study (Table 2) during spring semesters over a period that spanned three academic years. For the first semester, a faculty member teaching two sections of the Introduction to Genetics course was recruited personally. Subsequently, faculty assigned to teach this course were invited via e-mail to participate; approximately half of those contacted agreed. A random number generator was used to decide which instructors would implement the module and which instructors would teach using their normal methods. Four faculty members (identified as A, B, C, D) taught both a CREATE section and a comparison section (Table 2). Of the remainder, two (E, F) taught only a CREATE section and two (G, H) taught only a comparison section. Two instructors (A, F) had over a decade of teaching experience, but only one (F) had taught this course more than once prior to the study (Table 2).

TABLE 2

Implementer experience.

Instructor	Rank	Undergraduate Teaching Experience (# of Years)	Experience Teaching “Introduction to Genetics” (# of Times)
CREATE Module Sections
A	Adjunct	11	0
B	Assist. Prof.	3	1
C	Adjunct	3	1
D	Adjunct	4	1
E	Adjunct	2	0
F	Assoc. Prof.	16	11
Comparison Sections
A	Adjunct	12	1
B	Assist. Prof.	3	1
C	Adjunct	3	1
D	Adjunct	3	0
G	Adjunct	2	0
H	Adjunct	0	0

Eight faculty participated in the study over a three-year period. Four of the participants (A–D) taught both a CREATE module and a comparison section, either in the same semester (B, C) or in different semesters (A, D).

Faculty were sent an email with explicit instructions on their assigned roles for each section and were reminded that students should not be told about the specific type of instruction that would be used. None of the faculty had previously taught with the CREATE method; those who implemented it received a brief amount of training (approximately one hour). Faculty teaching the module used the same script to ensure consistency across implementations (Appendix 1 a, b). Instructors who taught comparison sections were told to teach in a manner consistent with their typical methods.

IRB, Assessment, and Student Participation

Institutional Review Board (IRB) approval for the study was obtained (HRPP # 2015-0685 and IRB 2014-021). The primary investigator (PI) scheduled times with each faculty for giving the assessments (pre, post) in each section. The PI invited students to participate voluntarily and anonymously in a study designed to “improve undergraduate science education” and obtained informed consent from students enrolled in each section of the course. Students did not receive incentives (e.g., gift cards, points, class credit) for participation. Students used personal codes on all surveys, allowing pairing of pre- and post-course surveys for analysis. Students who chose not to participate received course-related news articles to read during the survey time. Students who participated and finished the surveys early also read these articles.

The Test of Scientific Literacy Skills (TOSLS; 26) and the Survey of Student Attitudes, Abilities and Beliefs (SAAB; 21) were administered over two class periods, both in the first week (pre) and last week (post) of the semester. The placement of the module was dependent on the course syllabus and occurred in the last third of the semester. Means, standard deviations, and paired t tests for TOSLS were calculated in Excel; Wilcoxon signed-rank tests (for SAAB) were calculated using Vassarstats (http://vassarstats.net/wilcoxon.html) in Excel. Effect sizes (Cohen’s d; 27), were calculated using www.uccs.edu/lbecker.

To test for possible gains in science literacy across the term, we used the TOSLS, a survey with 28 multiple-choice questions that address students’ ability to read and evaluate data presented in a variety of scenarios but do not relate directly to the course content (26). We scored the percentage of questions answered correctly. One comparison section did not complete the TOSLS.

To examine the possibility of shifts in students’ self-efficacy with regard to science process skills and/or epistemological beliefs about science, we administered the Survey of Student Attitudes, Abilities and Beliefs (SAAB; 21). The Likert-style survey includes 31 statements (e.g., “I am comfortable defending my ideas about experiments”) to which students respond on a five-point scale (strongly disagree; disagree; I’m not sure; agree; strongly agree). The statements have been classified previously into six factors that address self-efficacy (e.g., ability to decode primary literature) and seven that address epistemological beliefs (e.g., the certainty of knowledge, whether science is creative). We scored surveys by assigning numbers to Likert choices (strongly disagree = 1; disagree = 2; I’m not sure = 3; agree = 4; strongly agree = 5) and grouping statements into their factor categories. Statements for which the more mature, postcourse response was lower than the precourse response (e.g., “The scientific literature is difficult to understand”) were reverse-scored (21). The SAAB survey also contains three summary statements addressing students’ perceptions of 1) their confidence in reading articles, 2) the extent to which they understand the research process, and 3) the degree to which they feel journal articles have influenced their understanding.

Overall, the majority of students in the 12 sections of this course participated in our research study (three semesters in total). We used the student code numbers to select “matched pairs” from these data, each matched pair (mp) representing the precourse and postcourse scores of a single student. The mp data represent nearly three-quarters of the students who participated in the research study. Thus, we think the mp data accurately represent the students who participated in the research study.

RESULTS

SAAB

We present outcomes on the six self-efficacy and seven epistemological belief factors for matched-pair cohorts (Table 3) and for the three summary statements (Table 4). The pooled CREATE cohort (N = 89 mp) made significant gains on self-efficacy factor 1 (Decoding primary literature; p < 0.013; ES = 0.3) and factor 3 (Active reading; p < 0.004; ES = 0.34); (Table 3; see Appendix 2 for the factors and relevant substatements). The pooled comparison cohort (N = 92 mp) made no significant gains on any factor but decreased significantly on epistemological belief factor 7 (Certainty of knowledge; p < 0.01; ES = −0.31). That is, students believed more strongly postcourse than precourse in the certainty of knowledge.

TABLE 3

Outcomes on survey of Student Attitudes, Abilities and Beliefs (SAAB).

SAAB Factor	Measure	CREATE Sections		Comparison Sections
		Pre	Post	Pre	Post
1. Decoding Primary Literature	Mean (SD)	3.22 (0.53)	3.39 (0.59)	3.26 (0.51)	3.36 (0.58)
	Wxn; ES	0.013; 0.3		0.11; 0.18
2. Interpreting Data	Mean (SD)	3.63 (0.61)	3.73 (0.57)	3.73 (0.50)	3.77 (0.53)
	Wxn; ES	0.13; 0.17		0.11; 0.18
3. Active Reading	Mean (SD)	3.56 (0.48)	3.72 (0.47)	3.63 (0.42)	3.64 (0.45)
	Wxn; ES	0.004; 0.34		0.65; 0.02
4. Visualization	Mean (SD)	3.46 (0.58)	3.57 (0.59)	3.60 (0.53)	3.58 (0.56)
	Wxn; ES	0.13; 0.19		0.85; −0.04
5. Thinking Like a Scientist	Mean (SD)	3.31 (0.64)	3.45 (0.58)	3.66 (0.77)	3.63 (0.80)
	Wxn; ES	0.14; 0.23		0.48; −0.04
6. Research in Context	Mean (SD)	4.07 (0.58)	4.1 (0.55)	4.17 (0.54)	4.20 (0.61)
	Wxn; ES	0.63; 0.05		ns; 0.05
7. Certainty of Knowledge (R)	Mean (SD)	3.69 (0.41)	3.70 (0.45)	3.86 (0.44)	3.72 (0.46)
	Wxn; ES	0.70; 0.02		0.01; −0.31
8. Innateness of Ability (R)	Mean (SD)	3.44 (0.78)	3.32 (0.75)	3.41 (0.82)	3.28 (0.79)
	Wxn; ES	0.25; −0.16		0.17; −0.16
9. Scientific Creativity	Mean (SD)	4.10 (0.67)	4.10 (0.81)	4.24 (0.74)	4.18 (0.66)
	Wxn; ES	0.87; 0		0.6; −0.16
10. Sense of Scientists	Mean (SD)	3.02 (0.95)	3.25 (0.93)	2.9 (0.90)	3.11 (0.95)
	Wxn; ES	0.09; 0.24		0.07; 0.23
11. Sense of Scientists’ Motivations	Mean (SD)	3.75 (0.90)	3.59 (0.94)	3.61 (0.99)	3.51 (0.97)
	Wxn; ES	0.21; −0.17		0.52; −0.10
12. Known Outcomes (R)	Mean (SD)	3.82 (0.8)	3.71 (0.82)	3.78 (0.8)	3.66 (0.35)
	Wxn; ES	0.43; −0.13		0.41; −0.19
13. Collaboration	Mean (SD)	4.28 (0.65)	4.29 (0.56)	4.35 (0.61)	4.28 (0.63)
	Wxn; ES	1; 0.02		0.43; −0.11

SAAB factors 1–6 address students’ self-efficacy; factors 7–13 address epistemological beliefs about science (21). We pooled outcomes from six classes that used the CREATE module (N = 89 matched pairs of students) and six comparison non-CREATE classes (N = 92 matched pairs). Negatively phrased statements were reversed (R) for analysis, thus factors for which scores are higher postcourse than precourse indicate a student-assessed improvement in ability (factors 1–6) or more mature view of science (factors 7–13). Means and standard deviations (SD) calculated in Excel; significance (Wilcoxon signed-rank test, Wxn) determined using Vassarstats (http://vassarstats.net/wilcoxon.html); effect sizes (ES) calculated using https://www.uccs.edu/lbecker. ns = non-significant.

TABLE 4

Outcomes of SAAB summary statements.

Summary Statements		Confidence in Reading Ability		Understanding of Research Process		Influence of Journal Articles on Understanding of Science
		Pre	Post	Pre	Post	Pre	Post
CREATE Sections	Mean (SD)	2.92 (0.15)	3.30 (0.17)	3.43 (0.14)	3.75 (0.28)	2.94 (0.30)	3.19 (0.29)
	Wxn; ES	p < 0.05; 2.48		p < 0.05; 1.44		p < 0.05; 0.85
Comparison Sections	Mean (SD)	3.15 (0.10)	3.48 (0.17)	3.63 (0.14)	3.86 (0.23)	3.05 (0.22)	3.26 (0.10)
	Wxn; ES	p < 0.05; 2.36		ns; 1.21		ns; 1.23

The SAAB survey includes Likert-style summary questions regarding students’ self-rated confidence in ability to read/analyze articles (scale: 1 = zero confidence; 2 = slightly confident; 3 = confident; 4 = quite confident; 5 = extremely confident); their understanding of “the research process” (1 = I don’t understand it at all; 2 = I have a slight understanding; 3 = I have some understanding; 4 = I understand it well; 5 = I understand it very well); and the extent to which journal articles have influenced their understanding of science (1 = no influence; 2 = very little influence; 3 = some influence; 4 = a lot of influence; 5 = a major influence). Scores for the six CREATE and six comparison sections were pooled. Means and standard deviations (SD) calculated in Excel; Wilcoxon signed-rank tests (Wxn) performed using Vassarstats (http://vassarstats.net/wilcoxon.html); effect sizes (ES) calculated per https://www.uccs.edu/lbecker. ns = non-significant.

On the summary statements, pooled CREATE groups showed significant gain on all three statements, addressing confidence in literature analysis ability (p < 0.05; ES = 2.48), understanding of how research is done (p < 0.05; ES = 1.44), and the extent to which journal articles had influenced the respondent’s understanding of science (p < 0.05 ; ES = 0.85). The pooled comparison groups showed significant gain on the literature-analysis-ability statement only (p < 0.05 ; ES = 2.36) (Table 4).

TOSLS

Neither the CREATE nor the comparison groups made significant gains on the TOSLS, a test of transferable data analysis skills (Table 5). Neither the pooled CREATE cohort of matched pairs (N = 79; p = 0.45) nor the pooled non-CREATE cohort (N = 75; p = 0.98) made significant gains on the TOSLS postcourse. We conclude that experiencing the condensed CREATE module did not strongly influence science literacy skills measured by the TOSLS.

TABLE 5

Outcomes on the Test of Scientific Literacy Skills (TOSLS).

Measure	CREATE Sections		Comparison Sections
	Pre	Post	Pre	Post
Mean (SD)	58.9 (4.6)	57.5 (5.3)	67.8 (3.5)	67.9 (2.6)
t test	p = 0.45		p = 0.98

We pooled outcomes from completed TOSLS surveys (all questions answered) for six sections that used the CREATE module (N = 79 matched pairs of students) and six comparison non-CREATE sections (N = 75 matched pairs). Means, standard deviations (SD), and two-tailed t tests (type 1) were calculated in Excel.

DISCUSSION

We tested the effect of a short-duration experience with CREATE by comparing student outcomes in sections of a genetics course (Introduction to Genetics) that either did or did not employ a newly developed CREATE module using the Test of Scientific Literacy Skills (TOSLS) and the Survey of Student Attitudes, Abilities and Beliefs (SAAB). Neither the CREATE nor the comparison sections made significant gains on the TOSLS. Perhaps the duration of the CREATE module and/or the specific data analyzed were not sufficient to promote development of transferable literacy skills as measured by the TOSLS. Positive differences in favor of the CREATE-based instruction were observed when comparing student outcomes on the SAAB survey. The results of this survey suggest that the close and active analysis of primary literature with CREATE tools helps to develop students’ understanding of research processes and their self-efficacy with regard to science process skills. Defined as an individual’s confidence in their ability to successfully undertake a goal-directed task in a domain (28), self-efficacy is essential to student success. The genetics module required the use of diverse CREATE tools, including concept mapping, sketching, and illustrating experimental design (Table 1). These activities align with the types of “mastery experiences” that promote development of self-efficacy (29).

Only the CREATE sections made significant gains on self-efficacy factor 1 (Decoding scientific literature) and factor 3 (Active reading). In contrast to previous studies of full-semester CREATE courses where students made significant gains on most SAAB factors across a term (18, 20), students in CREATE module sections did not significantly shift their epistemological beliefs about science. Surprisingly, the comparison groups held a significantly less mature epistemological view on the certainty of knowledge (factor 7) postcourse. Students enrolled in the CREATE sections did not regress in their view of the certainty of knowledge during the term. This suggests that a semester of standard genetics instruction may drive epistemological understanding in a less mature direction and that the inclusion of the CREATE module helped to prevent this shift to a more naive understanding. The finding is reminiscent of outcomes on the Colorado Learning Attitudes about Science surveys (30–32). On such surveys in introductory biology, chemistry, and physics courses, it is common to find students’ perceptions of science moving significantly in the “more naïve” rather than “more expert-like” direction across a semester. We speculate that the problem-solving aspect of genetics courses, where most homework has a single correct answer, may drive students’ sense of knowledge as “certain.” In contrast, a literature module that emphasizes hypothesis-driven inquiry and challenges students to think about how investigators pose questions and plan experiments (see Appendix 1) may support development of a more mature view of the nature of science (33).

As many courses at the testing institution, including the introductory genetics course studied here, use scientific literature, it was not a surprise that students in both the CREATE module sections and comparison sections perceived an improvement in their self-rated ability to read and analyze scientific literature (Table 4). However, only the CREATE group also made significant gains on the reading-related self-efficacy factors, factor 1 (Decoding scientific literature) and factor 3 (Active reading). This suggests that, while both groups of students believe they have improved in their ability to read primary literature, the students who studied literature using CREATE methodologies also report gains in specific skills necessary for deciphering primary literature. The CREATE sections, but not the comparison sections, also made significant gains on summary statements related to 1) self-rated understanding of how research is done and 2) the extent to which journal articles had influenced this understanding (Table 4). This finding suggests that adding active analysis of even a single paper to this lecture/laboratory course deepened students’ understanding of the research process. These findings argue that CREATE interventions are impactful in providing novel insight into what scientists do and how they do it.

Our finding that the inclusion of a single CREATE module prevents a shift to a more naive perception of the stability of scientific knowledge further supports including targeted analysis of primary literature throughout the curriculum, an approach that aligns with the Vision and Change recommendation to “Introduce the scientific process to students early, and integrate it into all undergraduate biology courses” (5). We suggest that primary literature provides a direct inroad into the nature of scientific investigation. Analyzing papers deeply via the CREATE toolkit, and complementing this process with e-mail surveys of paper authors, offers unique insight into researchers and their approaches. Unlike the majority of textbooks, papers have the considerable advantage of including the specific methods used and the actual data accrued. Directed examination of a given study’s logical design and methodology challenges undergraduates to integrate and apply their understanding of core concepts while simultaneously examining how specific methods were employed to address particular questions or hypotheses. The approach encourages students to imagine themselves in the role of scientists who designed the experiments, giving students a nuanced perspective of overall research design (e.g., sample size, controls and their functions, and techniques of data analysis). This process forestalls passive acceptance of reported conclusions and underscores the reality that while an article’s title typically focuses on a single key finding, the article itself builds the research “story” in a stepwise manner with a series of scaffolded sub-parts. The method aligns well with understandings of how students learn both science content and process (33–35). Because the CREATE module has no wet lab component and CREATE is thus inexpensive to implement, the finding that students who experienced the module felt they understood the research process better is especially notable. The module’s interview of the lead author of the research paper highlights the open-ended nature of biological research, possibly surprising students who assumed that investigators could predict outcomes in advance or that all investigations were planned by the head of the laboratory.

In summary, this pilot test of a brief CREATE module in an introductory genetics class resulted in gains that may enhance students’ ability to read primary literature assigned in upper-division coursework. Further research is needed to determine 1) whether use of multiple modules would result in broader cognitive as well as affective gains, as has been documented previously in semester-long CREATE interventions (18, 20) and 2) whether students who have been exposed to a short genetics module possess literature analysis skills transferable to other coursework. Of particular interest is the apparent ability of the CREATE module to prevent a turn to more naive thinking. We do not know whether the gains observed were due to the use of authentic data, the discussion of data variability, and/or the interview with the paper’s lead author, but we think it likely that the approaches work synergistically.

Previous studies of the CREATE method have focused on full-semester courses taught by PIs or by faculty who learned and practiced CREATE approaches in multi-day CREATE workshops taught by experienced CREATE practitioners (22–24). This study demonstrates that a short CREATE module taught by faculty with little CREATE training can also produce some important gains. Most faculty involved in the study were relatively inexperienced at teaching genetics, arguing that the CREATE pedagogy is accessible. Additionally, the inclusion of adjunct faculty in the study further demonstrates CREATE’s utility to diverse educators and suggests that the broader implementation of CREATE modules effectively encourage adjunct and other contingent faculty to use primary literature in a way that deepens students’ understanding of authentic research processes. Thus, the benefits of primary literature study can easily be provided using CREATE methodologies.

Click here for additional data file.