Developing Course-Based Research Experiences in Discipline-Based Education Research: Lessons Learned and Recommendations.

In this perspective, we highlight the opportunity for the biology education research community to develop course-based research experiences (CREs) or course-based undergraduate research experiences (CUREs) in discipline-based education research. Building on our prior experience developing and teaching four biology education research CREs, we present opportunities, potential pitfalls, and recommendations for discipline-based education researchers interested in integrating their research and teaching in the context of a CRE.

INTRODUCTION

Course-based research experiences (CREs), also referred to as course-based undergraduate research experiences (CUREs), are research experiences embedded into formal lab courses. Course-based research experiences have been shown to have many positive benefits for students, including enhancing students’ critical thinking skills, enculturing students into the community of academics, and improving student persistence in college (1–3). The hallmark of a CRE is the opportunity for students who are enrolled in a course to work on a research project that produces novel data that is of interest to people outside the course (4, 5). Course-based research experiences can be developed for a diversity of research topics across disciplines. However, there has been concerted interest in developing CREs in the natural sciences (6–10, https://serc.carleton.edu/curenet/index.html) and, to our knowledge, there are only a few published examples of CREs outside of the natural sciences and no published examples in discipline-based education research (DBER) (11).

Discipline-based education research refers to research that is focused on improving the educational experience of undergraduates in a particular discipline; for the purpose of this perspective, when we refer to DBER, we mean education research done in the context of biology, chemistry, physics, and geosciences (12). We highlight three reasons why developing DBER CREs could be advantageous: 1) By integrating education research into a formal course, DBER CREs merge teaching and research, which can directly benefit DBER faculty if their teaching responsibilities include an education course. Course-based research experiences have been shown to contribute positively to promotion and tenure, and students in a CRE can collect and analyze data that can benefit a faculty’s research program, particularly for faculty who develop CREs based on their own research (13, 14). 2) The nascent state of the field of DBER means that there are many accessible research questions that are ripe for exploration. Many questions have not been studied at all in the context of undergraduate classrooms, whereas other research questions have been investigated in more well-developed disciplines, such as communication or psychology, but not yet in science courses (15). Further, DBER CREs could serve to enhance the generalizability of work published in DBER fields by implementing DBER CREs at different institutions and classrooms, with different instructors and student populations, so that they could serve as replication studies. 3) Because of their focus on education, DBER CREs can have a unique impact on science students’ thinking. In addition to reaping many of the same benefits as basic science CREs, we hypothesize that science students enrolled in DBER CREs may become more metacognitive about their own learning and gain a deeper understanding of instructor decision making if the DBER questions that they explore are related to instructional practices. Further, students may develop a personal interest in evidence-based teaching or even DBER. In sum, given the potential benefits for DBER faculty, DBER disciplines, and science undergraduate and graduate students, we recommend that discipline-based education researchers consider developing DBER CREs.

In this perspective, we draw on our own experience teaching both unsuccessful and successful DBER CREs, specifically in biology education research. We highlight possible opportunities associated with DBER CREs and present a suite of recommendations for discipline-based education researchers interested in developing a DBER CRE.

UNSUCCESSFUL AND SUCCESSFUL ATTEMPTS AT BIOLOGY EDUCATION RESEARCH CRES

By definition, CREs merge teaching and research and should have both pedagogical and research goals (13, 14, 16, 17). Pedagogical goals consist of student cognitive, psychomotor, and affective learning and can be measured by assessing student outcomes such as their ability to collect data, their ability to write a manuscript, or how much they enjoy participating in the CRE (16). Research goals consist of scientific discovery milestones and can be measured by tangible outcomes such as a set of research findings presented as a poster or publication (16). Thus, a CRE is “successful” when the course’s pedagogical and research goals are met. In this perspective, we focus exclusively on how we designed the CREs in order to reach our research goal of potentially publishable research findings in DBER.

We have taught four biology education research CREs. All four CREs were implemented in a single semester course that met in person once or twice a week for a total of three hours per week. Below we describe the course setting, the projects, and why we deemed them successful or unsuccessful at achieving the research goals. In each case, the instructor identified the research topic to be explored in the class, but some of the research projects were less well-defined at the beginning of the semester. We present these examples chronologically and hope that they can provide DBER CRE developers with insights into how to maximize their own CREs to meet research goals.

Bioed CRE #1: what does “#whatshouldwecallgradschool” tell us about student perceptions of graduate school?

The first time we taught a CRE was in a seven-person, graduate-level biology education research course. All seven students were graduate students in science disciplines. The first seven weeks of the course were dedicated to readings and discussions about biology education broadly and the final seven weeks focused on the research project. For the research project, students analyzed GIFs on the Tumblr “#whatshouldwecallgradschool,” (18) for themes that illustrated graduate student perceptions of graduate school. Specifically, we were interested in graduate student perceptions of research and teaching. Graduate students in the class seemed engaged in the project because of its relevance to them, and through this project, students learned how to code written response data. However, due to the nature of this Tumblr website, we were unable to determine who created the GIFs and who decided which GIFs to post, which could have biased our findings. Further, these GIFs included both images and words, which sometimes were presented in a sarcastic way, so students in the class often struggled with how to analyze the GIFs. While we came up with many interesting research ideas, ultimately we did not focus quickly enough on a specific research question, and we ran into many issues creating a coding rubric because the source material was difficult to interpret. We ran out of time to complete the project, and students were frustrated by the lack of progress on the research project at the end of the semester. We did not meet our research goals.

Positive features of the CRE:

The research project was interesting and engaging to students in the class.

We used existing data to code and did not have to collect data, which saved time.

Pitfalls of the CRE:

The scope of the project was too large for the time designated to train novices to make substantial progress, and there was a lack of focus on a specific research question.

The research design was flawed because the source material was not easy to code and we did not have enough information about the dataset to draw unbiased conclusions.

Bioed CRE #2: exploring the differences between inquiry-based courses and CUREs

This CRE was taught to four students enrolled in a graduate-level biology education research course. All four students were graduate students in science disciplines. We shortened the amount of time dedicated to broad discussions about biology education to only the first three weeks of class and spent the remainder of the semester reading and thinking about two types of biology lab courses—course-based undergraduate research experiences (CUREs) and inquiry-based lab courses. We intended to have students use published research articles on inquiry and CURE courses to write a perspective/essay on the benefits of CUREs and inquiry-based courses. The small class size was a challenge because it took more time to do an in-depth literature review than we anticipated. Further, we realized that perspective articles are difficult to write since they are less formulaic than typical research papers, so it was challenging for science graduate students who were new to discipline-based education research to write a perspective in a cohesive way. Although we developed interesting novel ideas through this course, we found that students needed a deeper theoretical foundation and understanding of CUREs and inquiry-based courses to be able to write the intended perspective piece in a single semester. Some of the ideas generated from this CRE ultimately resulted in a manuscript (16), but it was not a direct product of the class. We did not meet our research goals.

Positive features of the CRE:

As a result of conducting an in-depth literature review, students by the end of the semester seemed to have a strong foundation in the literature on CUREs and inquiry courses and were able to generate interesting research ideas.

Pitfalls of the CRE:

The literature on CUREs and inquiry-based courses is extensive, and it was difficult for science graduate students who were new to DBER to conceptualize, which made it challenging to write a perspective during a single semester.

Writing a perspective-style article was difficult for novices.

Bioed CRE #3: exploring student perceptions of instructor use of humor in science classrooms

Learning from our previous failures to produce a research product, we decided that we needed to explore a specific research question that would be highly accessible for novice biology education researchers, that we needed to spend the majority of the semester on the project to increase the probability of achieving research success, and that we needed a larger class size to handle the scope of collecting and analyzing enough data in a single semester to report a research finding. We therefore offered the class to both undergraduate and graduate students in science disciplines and enrolled a class size of 16. Before the semester started, we spent more time backward-designing the course, outlined research and pedagogical goals (16), and identified an accessible research topic with questions that could be answered by coding open-ended questions. Coding open-ended questions can be time intensive for a small research team but can be done more quickly with a large number of students. Specifically, we decided to explore instructor use of humor in science classrooms because all students have experienced instructors trying to use humor, yet a prior review of the literature indicated that no studies had been done in the context of undergraduate science courses on this topic.

Although we chose the broad project idea for the students, we had the students review the published literature to identify what specific research questions they wanted to explore. As a class, we decided to explore how students’ social identities influenced student perceptions of instructor humor in college science classrooms. Importantly, this decision about the specific research questions was made within the first three weeks of the semester, so the research project was well defined for the majority of the semester. During the course, students also completed the Human Subjects Research course from the Collaborative Institutional Training Initiative (CITI) Program, created a survey, iteratively revised the survey through think-aloud interviews and piloting, and deployed the survey to students in 25 college science courses. Students were trained on how to use open coding methods (18, 19) to code data from open-ended responses: we used large data sets of de-identified student responses that we had collected previously for other projects and re-analyzed them with the students. Students learned how to develop coding rubrics, code responses, and calculate inter-rater reliability. Therefore, once we had data from the humor survey, students were able to analyze open-ended responses from over 1,600 survey participants. We broke students out into different teams of two to four students, and each team analyzed different open-ended questions, working in parallel to develop their own rubrics for their specific questions. By the end of the course, students had analyzed most of the data and had created final tables and figures that they were able to present to the class. By doing the tasks in parallel, we were able to maximize the progress that students could make on a group project. After the course, the instructors drafted the manuscript and gave students the opportunity to edit it; the manuscript is now published, with all 16 students as co-authors (20). We achieved our research goals in this CRE.

Positive features of the CRE:

Because we selected an accessible research topic and backward designed the research methods considering student ability, students were able to participate in formulating the research questions, reviewing the current literature, designing a survey, collecting data, and analyzing data.

This project would have taken far longer with fewer students, so we maximized the usefulness of having a large research team to code responses from ~1,600 survey participants.

We had multiple students code each response, so we had a built-in check for accuracy, which we felt was important given the subjective nature of coding.

Pitfalls of the CRE:

Different groups of students were coding different questions at the same time, but we would have benefited from coding them in sequential order because we realized that some questions were less interesting based on the coding of other questions.

It took longer than expected to iteratively develop the survey, making the data analysis at the end of the semester more time intensive than we anticipated, and students complained about the workload during those last few weeks.

Bioed CRE #4: why do undergraduates consider leaving undergraduate research

Tasked with teaching a course to prepare ten undergraduate transfer students to be involved in basic science undergraduate research in subsequent semesters, we decided to teach the course as a CRE focused on undergraduate research experiences. Students read the literature on undergraduate research experiences and brainstormed interesting projects. They came up with the idea of exploring factors that influence whether students consider leaving and actually leave undergraduate research experiences.

We mirrored the structure of our previous successful CRE. Students designed a survey, iteratively revised it, and sent it out to ~2,000 students, obtaining ~200 student responses. They began data analysis of the open-ended responses but unfortunately did not finish by the end of the semester due to the extended time taken to decide on a specific research question at the beginning of the semester. We did not meet our research goals within a single semester, but the CRE has potential for success in subsequent semesters. This course is part of a program that will continue for three additional semesters. We anticipate completing the CRE by the end of the program.

Positive features of the CRE:

The previously-defined and accessible research topic was highly relevant for these students and allowed them to become more knowledgeable about undergraduate research, meeting our primary pedagogical goal.

We made sufficient progress on the research project in a single semester so that students in subsequent semesters can build on our initial findings.

Pitfalls of the CRE:

Because the purpose of this course was to help transfer students better understand research, we focused our attention on the pedagogical goals of the course, which limited our research progress.

We did not identify a specific research question early enough in the semester.

OPPORTUNITIES ASSOCIATED WITH DEVELOPING DBER CRES

Based on our experiences teaching CREs, we have expanded our reasoning on why discipline-based education researchers should consider teaching CREs and highlight three CRE outcomes that each positively impact a different stakeholder: the student, the instructor, and the DBER community.

Students in a DBER CRE can learn how to critically think through a problem just as in a basic science CRE. Undergraduate students and first-year graduate students may especially benefit from the opportunity to analyze and interpret data, work in collaborative teams, present their work, and co-author a publication early in their academic career. Further, few basic science projects are accessible enough for students to design the project in a single semester, but the accessibility of some education research projects allows students to design and execute the research in a semester (16).

Depending on the research question, students can also learn more about their own learning process, undergraduate research, or evidence-based teaching. Thus, even if students who participate in a DBER CRE do not choose to go on in DBER, they may develop an enhanced understanding of teaching and learning that could be useful for their future learning and in any future position involving teaching.

Prior research has shown that instructors personally benefit from teaching CREs (13, 14), and in our opinion, we benefitted from teaching these CREs. Instructing a CRE can be an enjoyable way to teach, often feeling more like leading a research team than teaching a class. Further, designing CREs with the goal of publishing a paper allows instructors to better balance teaching and research demands by publishing manuscripts on work resulting from the CRE.

Discipline-based education research is a relatively young field, and students in science who are interested in teaching and learning may not be aware of DBER literature or the opportunity to study education in the context of their discipline. A DBER CRE could be an entry point into education research for science students, which it was for some of the students in our courses. Several of the students who took these courses decided to pursue additional research opportunities in DBER.

Additionally, DBER CREs could be used to replicate studies in different contexts to allow researchers to be able to make more generalizations about their findings. One advantage of replication projects is that students can use research questions and methods that have already been published, simply applying them to a new institutional context, thereby increasing the probability of research success and meeting research goals.

TIPS FOR TEACHING A DISCIPLINE-BASED EDUCATION RESEARCH CRE TO UNDERGRADUATE AND GRADUATE STUDENTS

We experienced a number of pitfalls associated with teaching biology education research CREs and base the following recommendations on our successful and unsuccessful CREs.

Before beginning a semester-long CRE, identify an interesting research topic that students can easily grasp, and quickly develop a conceptual understanding of the topic.

Design a project to take advantage of a larger number of people to meet research goals and keep students engaged. This can take the form of data collection (e.g., students using observation protocols to observe classes), data analysis (e.g., students coding large amounts of open-ended data), or a literature review on an accessible topic. Projects that can be done by one person (e.g., statistical analyses of closed-ended datasets) would likely not benefit from having a large number of students involved in the project.

Novice students are unlikely to have the skillset to do complicated statistical analyses or deeply theoretical qualitative analyses in a short period of time, so we recommend projects that either require simpler analyses or that are designed so that students conduct aspects of the research that do not need those analyses.

Students have varying levels of motivation to be accurate, so we encourage instructors to have multiple students do each task to reveal discrepancies.

Some steps of the research process will likely take longer than expected. We encourage instructors to have a plan but not expect to stick to a week-by-week schedules, because it is likely that the schedule will change. Students should also be aware that, due to the nature of research, some weeks will be lighter in workload and others will be heavier.

Organization is key because there are multiple students doing different tasks. It is imperative that the CRE instructor(s) stay organized and develop standardized ways for students to organize and analyze data.

CONCLUSION

We hope that this perspective can serve as a helpful resource and guide for instructors who are interested in developing course-based research or course-based undergraduate research experiences in discipline-based education research.