Define Your Goals Before You Design a CURE: A Call to Use Backward Design in Planning Course-Based Undergraduate Research Experiences.

We recommend using backward design to develop course-based undergraduate research experiences (CUREs). The defining hallmark of CUREs is that students in a formal lab course explore research questions with unknown answers that are broadly relevant outside the course. Because CUREs lead to novel research findings, they represent a unique course design challenge, as the dual nature of these courses requires course designers to consider two distinct, but complementary, sets of goals for the CURE: 1) scientific discovery milestones (i.e., research goals) and 2) student learning in cognitive, psychomotor, and affective domains (i.e., pedagogical goals). As more undergraduate laboratory courses are re-imagined as CUREs, how do we thoughtfully design these courses to effectively meet both sets of goals? In this Perspectives article, we explore this question and outline recommendations for using backward design in CURE development.

INTRODUCTION

For over 50 years, the biology community has recognized the importance of teaching science as it is practiced (1–11). Dubbed the “purest form of teaching” (6), doing scientific research with students has been promoted as an effective strategy for teaching students the true nature of science (4,10,12–14).

There are many benefits associated with participating in undergraduate research, but the limited number of apprenticeship-style undergraduate research spots available in faculty labs has historically restricted opportunities to only select students (11,14). As a solution to this access problem, the emergence of course-based undergraduate research experiences, or CUREs (also called discovery-based courses), has made research opportunities available to large numbers of students enrolled in undergraduate laboratory courses (10,11,13,15,16). Students in a CURE use the process of science while working collaboratively and iteratively to make discoveries that answer research questions with results unknown to the scientific research community (17,18). Because CUREs result in novel research findings, they represent a unique course design challenge, as the dual nature of these courses requires course designers to consider two distinct, but complementary, sets of goals for the CURE: 1) scientific discovery milestones (i.e., research goals) and 2) student learning in cognitive, psychomotor, and affective domains (i.e., pedagogical goals). As more undergraduate laboratory courses are re-imagined as CUREs, how do we thoughtfully design these courses to effectively meet both sets of goals?

We recommend an iterative backward design process based on the principles proposed by Wiggins and McTighe (19). Using this process, instructors delineate their scientific research goals and develop corresponding learning objectives before designing their CUREs. Once research and learning goals are established, instructors determine appropriate evidence and assessment, and then scaffold specific laboratory learning experiences to incrementally foster such outcomes. By using backward design, CURE course designers can ensure that they are designing specific elements of their CURE to meet scientific research milestones without sacrificing student pedagogical outcomes.

USING “BACKWARD DESIGN” TO DEVELOP CURES

Proposed by Wiggins and McTighe (19), the “backward design model” has been encouraged by the biology education community in the development of student-centered biology lecture courses and traditional lab courses where students’ data are not broadly relevant to stakeholders outside of the lab course (10,14,20–22). A course designer using backward design for a lecture course or traditional lab course would start with the pedagogical goals of the course and then work “backward” to identify appropriate assessments to measure the goals. Only then would an instructor craft a curriculum to achieve the goals.

The true hallmark of a CURE, distinguishing it from other types of lecture and lab courses, is that students work on a research project with an unknown answer that is broadly relevant to people outside of the course (18,23). This means that, in addition to the typical pedagogical function of lab courses, CUREs have a unique scientific knowledge-building function that intertwines research and learning. Thus, when designing a CURE, course developers must navigate a balance between scientific discovery milestones and student learning gains, making careful choices about whether to start with research goals or pedagogical goals. While there is substantial evidence that instructors are backward designing biology lecture and traditional lab courses by considering pedagogical goals, there is little evidence that instructors of CUREs are backward designing their courses by considering the combination of pedagogical goals and research goals. To our knowledge, backward design considering both pedagogical goals and research goals has not yet been specifically recommended for CUREs.

By choosing to teach a CURE, instructors have already defined the overarching learning outcome for students: to provide them with the experience of conducting scientific research, allowing them to integrate into the scientific community. Engaging in any part of the research enterprise, even a simple task, is considered legitimate peripheral participation in science and provides opportunities for students to become more experienced members of the scientific community of practice (24). Further, integrating research into the undergraduate biology curriculum has been recommended to improve students’ abilities to understand how biologists conduct research (10). By deciding to teach a CURE, course designers have also determined the overarching research goal: to contribute new scientific knowledge to the scientific community. The next step for CURE developers is to consider more specific learning and research goals. Historically, faculty developing CUREs design specific research goals first based on their research interests (25), then define learning goals within the context of the research project. Although the research goals can constrain the possible learning goals, defining the research goal first ensures that the CURE is built around a research question and will result in broadly relevant novel data. After deciding to develop a CURE, we recommend that the next step be to identify specific scientific discovery milestones (e.g., synthesizing a compound, annotating a gene, or characterizing a phenotype) that can be met, given student skill levels and course resources. Once scientific discovery milestones have been established, course designers can develop learning goals around these research goals. For example, course designers can ask: what is the guiding research question for this CURE, and what do we want students to know or be able to do after engaging in this research question? Once these research and pedagogical goals are defined, instructors should then determine the type of evidence that would signify success in achieving these goals. For example, which scientific results are we aiming for? Which skills and competencies should students be able to demonstrate? After determining what constitutes acceptable evidence, instructors should plan the daily or weekly experiments and instruction that will support the students in meeting these larger goals. Finally, we acknowledge that backward designing a CURE is an iterative process with regard to both research goals and pedagogical goals; after identifying class activities that will lead to desired outcomes, designers will likely reexamine their plan for assessment to ensure that assessment aligns with planned activities. Figure 1 illustrates a backward design model applied to CUREs, which encourages instructors to sync the dual functions of research and pedagogy in a CURE.

FIGURE 1

Backward design (19) applied to scientific discovery and learning outcomes for CUREs. Starting broadly and becoming more specific, we recommend that instructors use the suggested reflection prompts to guide course planning and design for research goals (a) and learning goals (b). CURE = course-based undergraduate research experience.

Below we unpack four critical steps of backward design and highlight how research and learning goals can fit into these steps.

Step 1: Identifying desired results: defining outcomes

The first step in backward design is articulating the intended goals of the course. Although the overarching goals of engaging students in real research and contributing novel data to the scientific community have already been established by choosing to develop a CURE, more specific research and learning goals need to be articulated. While course designers could in theory begin with either research or learning goals, we recommend that course designers who want to develop a CURE start with research goals to ensure that their course meets the definition of a CURE and will result in data that are novel and broadly relevant to the scientific community. Then we recommend that the specific learning goals be superimposed on the research project.

Research goals can encompass a broad range of possibilities and organizational structures—from the SEA-PHAGES program, where students identify novel phage from soil samples (26), to sequencing genomes in the Genomics Education Partnership (27), to faculty-specific research in diverse areas of biology (28,29). We recommend that research goals be selected with consideration for the technical and developmental abilities of the undergraduate learner, in addition to practical considerations such as access to equipment and resources. See Kloser et al. (30) and Hekmat-Scafe et al. (31) for suggestions in engaging novice students in producing research.

Once research goals are established, learning goals for the course may include cognitive outcomes (e.g., content knowledge, ability to design an experiment, conceptions about the process of science), affective outcomes (e.g., sense of belonging, self-efficacy, attitudes about science, group membership), and psychomotor outcomes (e.g., increased technical skills). Some of these student outcomes may extend beyond a single course (e.g., retention in a major) and thus are difficult to measure. However, measureable single-term proxies (e.g., sense of belonging or science identity (32,33)) exist to gauge some of these longer-term outcomes. Given the plethora of possible learning objectives for undergraduate involvement in research, it is impossible for a single CURE to meet all of them, and thus careful choices must be made. Learning goals should be adjusted to the learning needs of a specific student population (e.g., introductory or advanced, nonmajors or majors) within a given curriculum to maximize the potential for short- and long-term success. Further, biology departments can consider how CUREs fit into the overall undergraduate biology curriculum and encourage CURE developers to use backward design so that the course helps students meet broader curricular goals (34). For example, CUREs can be developed to help students develop core biology competencies such as the ability to apply the process of science or the ability to use quantitative reasoning (10). Further, core concepts such as information flow could be integrated into the CURE curriculum (10,34). Additionally, CURE designers could consider developing interdisciplinary CUREs in order to develop students’ ability to tap into the interdisciplinary nature of science and communicate and collaborate with other disciplines (10). Importantly, broader curricular outcomes may be difficult to achieve in one-term laboratory courses, and thus, departments may need to consider how CUREs build upon other laboratory courses that are already part of the curriculum or provide the foundation for future courses.

Step 2: Determining acceptable evidence: choosing suitable assessments

Once the goals for the CURE have been identified and the relative importance is assigned to each goal, course designers are tasked with evaluating progress toward these goals by determining acceptable evidence and assessment strategies. This requires systematically measuring whether the lab course is achieving the intended research and learning goals.

To measure the course’s success in achieving research milestones, we recommend the same approach that is used in apprenticeship-style undergraduate research experiences— evaluate the quality and uncertainty of student-generated research in terms of its ability to contribute new knowledge. In cases where student efforts in the course do not achieve the expected research milestones, it is important to educate students about the role of “failure” as a common component of the process of science. Even if a CURE does not achieve the research milestones, if it has the potential to produce broadly relevant and novel data, the course is still a successful representation of the scientific enterprise and would be considered a CURE.

Learning goals can be measured using concept inventories, performance assessments, surveys, classroom observations, and/or analyzing course artifacts (e.g., lab notebook, final paper, poster). When using surveys or open-ended response questions to assess student gains, we encourage instructors to be wary of student self-reporting. While self-reporting is the most appropriate way to measure affective outcomes such as sense of belonging to the scientific community, it is an indirect measure of students’ scientific skills such as analyzing data. Because students’ assessment of their own abilities can be influenced by social desirability or the general tendency to report positive gains (35), instructors should consider more direct measures for non-affective outcomes. For example, although one could use a survey to obtain students’ self-report of their ability to analyze data, a more direct measure of their ability to analyze data would be to have students actually analyze data as part of a test or assignment. Assessment of learning goals could be done by comparing post-course scores to pre-course scores (30) or taking multiple measures over the duration of the term (36), and may also involve a comparison group (28,37). Shortlidge and Brownell have compiled a suite of previously developed assessments that could be used to assess a CURE, and they also suggest the importance of contextualizing one’s assessments to the specific topic of the CURE (38). An additional assessment that measures a suite of student outcomes, specifically in the affective domain, is the student persistence in the sciences (PITS) survey (39), but we encourage caution when administering broad surveys if the survey components do not align with the specific learning goals of one’s CURE. For a broader discussion of assessment of learning goals related to a CURE, the reviews by Corwin, Graham, and Dolan (23) and Brownell and Kloser (18) are useful resources. It may be helpful for a CURE developer to work with an education research specialist to help plan an appropriate assessment strategy.

Step 3: Planning experiences and instruction

Once desired outcomes and an assessment plan are established, course designers can begin to plan specific scientific experiments, classroom activities, and instructional practices that comprise the lab course. For example, if course designers want students to improve their ability to analyze data, then developing multiple opportunities throughout the course for students to practice this skill is beneficial. Students may analyze practice data as pre-lab assignments or conduct several similar experiments where they interpret and compare results. CUREs can also be designed to emphasize affective aspects of research such as improving students’ sense of belonging to a larger scientific community. Such a course may be built around an opportunity to contribute novel data to a database used by many other scientists or to present research at a professional conference. These activities give students access to a broader network of the scientific research community and can promote student conversations with scientists outside the class, building a sense of community. See Corwin et al., 2015 (23) for a set of proposed relationships between CURE activities and student outcomes.

Step 4: Iteration and revision

We recognize that it is not always easy to predict which course structures, activities, and pedagogies will lead to maximal and equitable opportunities for student learning. Therefore, to the extent that it is possible, we encourage instructors to take an evidence-based approach, consulting the education literature when making design and pedagogical decisions (10). While the literature establishing causal relationships between design components and outcomes of CUREs is limited, the advantage of using the backward design process is that it allows instructors to begin to systematically evaluate what is effective for their population of students and contribute to this growing body of knowledge surrounding the efficacy of CUREs.

Even though there is a growing literature describing the gains associated with CUREs over cookbook lab experiences, few studies have attempted a reductionist approach to determine which specific elements of lab course design lead to specific cognitive and affective outcomes (but see Shaffer et al., (40) and Brownell et al. (36) for exceptions). Recently, the CURE community has emphasized the importance of aligning assessments with outcomes to decipher the relative impact of each CURE component (17,18,23). After hypothesizing about which activities will most likely lead to the desired benefits, course designers will probably need to return to the assessment step of backward design in order to revise the assessment to be able to determine whether the activity leads to the goal. For example, an instructor may decide that improving the sense of belonging of first year students is an important desired outcome for their lab course. Next, they may choose an assessment to measure whether or not this goal is achieved and incorporate an activity (e.g., frequent collaboration with peers) in order to foster an increased sense of belonging. However, the instructor will need to continue to refine the assessment strategies to determine whether this specific component of the course (e.g., collaboration with peers) leads to an improved student sense of belonging.

AN EXAMPLE OF BACKWARD DESIGN APPLIED TO A CURE: SMALL WORLD INITIATIVE

The Small World Initiative (SWI) is an international collaborative delegating the discovery of new antibiotics to undergraduate researchers using a crowdsourcing model (www.smallworldinitiative.org; 41). The initiative uses scientific discovery in the context of a common CURE design to explore soil ecosystems on a worldwide scale for antibiotic-producing microbes. Students engage in a series of experiments during the semester, including culturing microbes from soil, screening these isolates for antibiotic activity against tester strains, performing 16s rRNA gene sequencing, and chemically extracting antibiotic-containing metabolites for toxicity testing of eukaryotic cells. Scientific data are contributed to a central database, and students present original research at an annual national symposium. Table 1 illustrates the backward design process applied to representative scientific and short-term learning outcomes of the Small World Initiative curriculum.

TABLE 1

The backward design process applied to representative scientific and short-term learning outcomes of the Small World Initiative (SWI) curriculum.

Backward Design Step	Scientific Discovery (a)	Student Learning (b)
1. Identify desired outcomes	Students identify an antibiotic-producing soil microbe.	Students 1) develop ownership of their research project and 2) gain a sense of belonging to the SWI scientific community.
2. Determine acceptable evidence	Students obtain a zone of inhibition from patches of soil isolates, signifying antibiotic secretion and killing of tester strains.	1) Adaptation of Project Ownership Survey (46) and 2) Custom survey for sense of belonging.
3. Plan learning experiences and instruction	Students 1) design and execute a method of culturing bacteria from soil, 2) measure CFU/g from serial dilutions of soil, 3) design and execute an assay for zones of inhibition against tester strains.	Students keep a laboratory notebook and complete pre-lab assignments (which include experimental design, hypothesis generation, and data analysis) as well as post-lab oral presentations.
4. Revise and iterate	Troubleshoot experiments to measure whether the experiment leads to an accurate scientific result.	Adapt assessments to measure whether activities lead to learning goals.

THE LIMITATION OF BACKWARD DESIGNING WITH RESEARCH GOALS FIRST

While we are recommending that CURE course designers first design their CURE to align with their research goals, we acknowledge that this constrains possible student learning outcomes. For example, a CURE derived from a faculty member’s research project may lead to only a select number of student benefits due to the model organism, methodology, and appropriate analyses—which may or may not be the intended learning outcomes of the lab course. For instance, a faculty member who develops a CURE based on her research on deep sea vents will be able to give students the experience of analyzing data, but not collecting their own data. Not collecting data may impact student project ownership (23), which has been suggested to contribute persistence in science (42). If the faculty member is interested in persistence in science as a long-term student outcome of the CURE, then it will be important to identify whether analyzing data is enough to garner sufficient project ownership to persist in science. Alternatively, a faculty member who develops a CURE based on her research in urban ecology may be able to give students the experience of collecting data, but the statistical analyses may be too complicated for introductory-level students; if the analyses are too difficult for introductory students, then the course may not meet its goal of improving student ability to analyze data. Thus, adjusting the learning outcomes of the course to fit within the research project may be necessary. Likewise, adjusting the research project to meet learning outcomes is key to the course revision process, especially in relation to larger departmental curricular outcomes.

SHOULD BACKWARD DESIGN BE APPLIED TO INDEPENDENT RESEARCH EXPERIENCES?

A corollary to this entire argument is that we should be using backward design to enhance student gains in independent research experiences. Similar to CUREs, independent research experiences merge learning goals with research goals when students are receiving course credit for participating in these experiences. To what extent have faculty considered learning outcomes when students are engaged in undergraduate research? How do these learning outcomes map onto the research goals of the independent research project? While the goals of apprenticed undergraduate research experiences are inherently research outcomes, we encourage faculty to systematically consider learning outcomes in addition to their research goals rather than assume that students are benefiting from these experiences by just engaging in research. This is especially important when students are enrolled in research for course credit or when research is required for a student’s degree program because student learning is then an expectation of this experience (43). While there are many self-reported student benefits of undergraduate research, there is limited evidence supporting student gains in conceptual understanding (44). Due to the variability among research experiences, we cannot assume that all students benefit from doing research in the same ways. As such, faculty using student-centered backward design for undergraduate researchers have an opportunity to explore which aspects of independent research experiences lead to reported outcomes, which is currently lacking from the literature (43–45).

CONCLUSION

CUREs are a powerful way of exposing students to the process of producing potentially publishable data in the context of a lab course, but we as a community would benefit by more thoughtfully designing these courses using the principles of backward design to maximize scientific research milestones and student learning.