Using Critical Analysis of Scientific Literature to Maintain an Interactive Learning Environment for In-Person and Online Course Modalities.

Every instructor has concerns about effectively balancing the amount of course content with experiences to enhance a student's skills for professional success. The COVID-19 pandemic made this process even more challenging by requiring many instructors to shift rapidly from in-person to online instruction while maintaining academic integrity. The objective of this course on tissue engineering, a multidisciplinary field that aims to repair and/or replace body damage, was to increase undergraduate students' ability to read primary scientific literature and use critical analysis to creatively solve problems. Every week, a lecture covered the necessary background information to identify the current research questions and prepare students for reading the assigned research article. Students completed an analysis worksheet prior to the subsequent class, and a summary presentation followed by a student-led critical analysis discussion occurred in class. Small student groups completed an in-class thought exercise that designed several experiments that built on the article's data. The modular course design enabled a quick and successful transition to an online asynchronous modality in less than two weeks due to the COVID-19 pandemic. A recorded weekly lecture was posted online by the instructor, and students completed the analysis worksheet, watched a student-recorded summary presentation, and posted to a discussion board. The experimental design worksheet became an individual assignment to provide more flexibility. Pretransition and posttransition assessment showed no significant differences and provided positive proof of concept evidence. This process can be adapted to a number of topic-themed scientific courses that use in-person, online, or hybrid modalities. ©2021 Author(s). Published by the American Society for Microbiology.

INTRODUCTION

Nearly every science instructor struggles with how to balance the amount of content with augmenting a student’s skill set for academic and professional success when designing a new course or refining an existing one. Some of the skills desired by postgraduate advisers and industry are reading primary scientific literature, critical analysis, critical thinking, communication, creativity, and problem-solving (1–11). However, not all class assignments and activities translate into developing all of these abilities. This process was even more difficult during the spring 2020 semester, when the COVID-19 pandemic caused in-person courses to rapidly transition to an online format.

In 2017, Sola et al. reviewed the literature and found that several studies indicated a downward trend in the ability of students to think creatively. For their particular study, the authors hypothesized that senior students would be better at critical thinking but less creative compared to freshman students. The authors found strong evidence that the freshmen were more creative, but no evidence that the freshman had a lower critical thinking ability compared to the seniors. They concluded that more focus on developing critical thinking and creativity was needed to address the problem (4). In another example, Ralston and Bays conducted a longitudinal study that followed three cohorts of undergraduate students from their freshman year through their senior year to study their critical thinking development. The hypothesis was that a significant increase in the ability of students to think critically would occur based on the incorporation of critical thinking assignments from their freshman to senior year. For all three cohorts, a significant increase in critical thinking scores between the sophomore and junior years occurred, but no other consistent comparisons between the three cohorts were measured. The authors accepted their hypothesis as the critical thinking scores did significantly increase over the four-year period (3).

The majority of the existing literature concentrates on the development of one to three of the aforementioned skills, but Hoskins et al. developed a CREATE (Consider, Read, Elucidate hypotheses, Analyze and interpret data, Think of the next Experiment) method for science undergraduates that focuses on the development of all the skills (1–20). This method was used in several science undergraduate single-semester courses to help students when reading and analyzing journal articles (2, 11, 13). The student assessment survey showed a significant increase from pre-test to post-test scores for decoding primary literature, interpreting data, active reading, thinking like a scientist, and research in context. The authors concluded that the CREATE method enhanced students’ ability to confidently read, understand, and explain research, which led to an increased ability in designing experiments, visualizing methods, critically analyzing the data, and relating the information to the bigger picture (2).

A similar approach to the CREATE method was used in an undergraduate, elective Tissue Engineering (TE) course. TE is an interdisciplinary field that combines life sciences, engineering, and medicine to repair and/or replace damaged tissues and organs. Cells, biomaterials, and/or bioreactors are used to create a functional and viable tissue or organ (21). This TE course focused on reading primary scientific literature, critical analysis, critical thinking, creativity, and problem-solving; however, it used a modular design that employed assignment scaffolding within each week’s chosen TE topic. A scaffolding approach was chosen to help guide students on how to solve more difficult problems by having several smaller-stakes assignments spread out over the week (7, 22, 23). This course was initially adapted to online instruction due to COVID-19 in the spring 2020 semester; however, a similar methodology could be applied to many scientific courses that range in topic and contain majors and/or nonmajors, while maintaining an interactive learning environment for in-person, online, or hybrid courses.

Intended audience

This elective TE course was designed for undergraduate students who had at least one semester of Anatomy & Physiology. Students previously enrolled in the course were biology majors, biology minors, biochemistry and molecular biology majors, or interested in the medical health field. It was designed for smaller classroom sizes (25 students or fewer) because of the student-led discussion following the student presentation, which was intended to engage the entire class. For larger classes, small groups facilitated by a teaching assistant or small recitation courses could be used and allow for enrollment to be increased by a factor of two to four. This TE course served as a proof of concept, and its modular design could be easily integrated into other topic-themed science courses. For example, an article that focuses on the p53 gene could be assigned when discussing the cell cycle and cancer in a cell biology or a genetics course. In an ecology course, an invasive species, such as zebra mussels, and its effect on the native populations could be assigned when discussing population growth and regulation.

Learning time

The first three weeks of the course focused on providing students with relevant background information that included introducing TE, cells, materials, and bioreactors. Figure 1 shows a schematic representation of the weekly process that transitions from course content to skill building starting with week 4. Each week began with a lecture that covered basic anatomy, basic physiology, past medical treatments, current medical treatments, and the research status on a specific tissue and/or organ. This helped students identify the current research questions and prepared them for reading the assigned research article chosen by the professor. Each student uploaded a completed Journal Article Analysis (JAA) worksheet to Canvas, a learning management system, before the following class (Table 1). Then, a student gave a summary presentation on the assigned article and led the critical analysis in-class discussion. Afterwards, student groups (two to four students per group) completed the Design the Next Series of Experiments (DNSE) worksheet as an in-class thought exercise (Table 1). Initially, this structure was designed to work in an in-person class that met twice each week; however, it could easily be adapted for a course that meets three times per week. Due to the COVID-19 pandemic, this course was rapidly transitioned to an asynchronous online modality.

FIGURE 1

Schematic representation of the weekly modular design that displays the assignment scaffolding process as it transitioned from course content to skill building. LO, learning objective.

TABLE 1

Queries for the individual Journal Article Analysis and group Design the Next Series of Experiments worksheets.

Journal Article Analysis Worksheet
Query 1	Write the title and authors of the paper.
Query 2	Summarize the introduction in 3 to 4 sentences, including the importance of studying this.
Query 3	Give the goal and/or hypothesis.
Query 4	Summarize the results in 8 to 10 sentences (Hint: Try to write 1 sentence or less for each experiment).
Query 5	What were the authors’ interpretation of the results and final conclusion? How does this study increase our understanding of the research topic?
Design the Next Series of Experiments Worksheet
Query 1	Brainstorm ideas below about how to use the data from the paper to create a series of experiments for the next paper to be published.
Query 2	What is your hypothesis?
Query 3	Describe 3 or 4 experiments below and explain why you chose them/why they would be good to conduct to move the research forward.

The DNSE worksheet was changed to an individual assignment following the transition to an online modality.

Prerequisite student knowledge

Students should have knowledge of all the human body’s systems so each student understands what is needed to design a fully functioning organ or tissue replacement. A student must have passed one to two semesters of Anatomy & Physiology depending on how the curriculum is structured.

Learning objectives

At the end of this course, the student learning objectives (LOs) were:

Apply physiological, anatomical, and engineering vocabulary to read and summarize primary scientific literature,

Critically analyze data from primary scientific literature, and

Design experiments that build on the data analysis.

The challenge presented by the COVID-19 pandemic was to transition the in-place modular design with assignment scaffolding to an asynchronous online format in less than 2 weeks without losing any progress made with the LOs.

PROCEDURE

Materials

Each student needed access to the JAA worksheet (Table 1) and each week’s assigned research article (LO 1). The weekly student presenter accessed their presentation the day of the presentation, and all other students were given a rubric to fill out that assessed the presentation (Table 2). The DNSE worksheet was then given out to each small group (two to four students) to be completed prior to leaving class (Table 1). After the transition to online learning, all materials for students were available online on Canvas.

TABLE 2

Rubric used to assess each student’s presentation grade.

Delivery	Comments
Clear voice and professional tone (no umms, likes, etc.) Uses correct pronunciation and usage of terms Does not read talk off slides. It is a talk Talk fits within specified time range (~15–20 min)
Slides
Contain standardized headings and fonts; easy to read Do not contain too much information per slide Use professional-style images, not silly or distracting ones. Images not from the article are properly cited Figures/tables from paper are easy to see
Introduction
Title slide and authors listed Relevance/importance of current study Project goal and/or hypothesis stated Concise
Materials & Methods/Results
Focus more on results than mats & methods Experimental n values, statistics, etc. explained Figures/tables adequately presented and explained Take-home message from each experiment stated
Conclusion
Overall conclusion Assesses data and interpretation of data by authors. If disagree w/interpretation, why? Was the project goal and/or hypothesis completed? Was the hypothesis accepted or rejected?
Questions/Discussion Period
Able to answer questions about the paper Able to answer questions tangential to the paper Starts discussion off with a question Keeps discussion moving

A scale of 1 to 5 was used, where 1 = unacceptable, 2 = poor, 3 = fair, 4 = good, and 5 = excellent.

Student instruction

At the beginning of the semester, students were instructed to read the assigned research article and upload the completed JAA worksheet prior to the following class. They were also instructed to make note of any questions they had regarding the article. Nonpresenting students filled out the rubric that assessed the student’s presentation and ability to lead the discussion. The average student score counted for 40% of the presentation grade and the instructor’s assessment for 60%. All students were expected and encouraged to participate in the student-led class discussion that was directly tied to their grade. Since the professor completed the first presentation and in-class discussion, students were instructed to follow a similar format when completing their own presentations.

During the COVID-19 pandemic, there was concern about Internet access and reliability, combined with other stressors such as mental health, physical health, financial concerns, etc. affecting each student’s ability to learn and complete assignments (24). It was for these reasons that an asynchronous instructional modality was adopted. A recorded lecture for students was posted at the beginning of each week on Canvas. Students read the assigned article and uploaded the JAA worksheet a minimum of 22 hours after the recorded lecture was available (LO 1). Next, a discussion board was set up on Canvas for each topic that contained the recorded article summary presentation. Students watched the presentation, commented at least four times over a 48-hour period (LO 2), and uploaded a completed rubric that assessed the presenter. The DNSE worksheet was changed to an individual assignment to provide the students with more autonomy, and the number of experiments to design was changed to one or two (Table 1). The completed worksheet was due ~36 hours after the discussion board closed (LO 3).

Faculty instructions

The first three weeks were lecture-based, with the instructor providing background information on TE, cells, materials, and bioreactors. As TE is a relatively new field, no textbook was used; instead, background information was found in scholarly publications. However, a textbook could be used in conjunction with the instructor chosen peer-reviewed articles found using databases. Starting in the fourth week, a specific tissue, organ, or organ system was chosen as the focus and the lecture described its anatomy and physiology, potential injuries and diseases, and past plus current medical treatments including any current limitations to completely restoring normal anatomy and physiology. At the beginning of the semester, journal articles chosen by the instructor have several obvious discussion points that students can easily find; however, this was required less and less over the course of the semester. It was found that better student presentations and in-class discussions occurred if the instructor modeled both of these for the students in week 4. The instructor facilitated the discussion by asking open-ended questions or asking students to analyze data, but this necessity decreased over time. Some example questions were: Do the figures and tables support the written results? Why or why not? What did the authors do well? Were the best materials and methods chosen to accomplish the goal? This type of facilitation was also needed when the small groups were completing the DNSE worksheet. For example: How long will the experiment run for? How often will data be collected? Do the experiments you have chosen fall in line with your hypothesis? What is the sample size for each group? Both the completed rubrics and DNSE worksheets were submitted prior to leaving class.

This modular format helped successfully and rapidly transition the course to an online asynchronous instructional modality. Student expectations, listed above, were clearly communicated through either email or using the announcements feature on Canvas. As this was a new format for presentations and discussions, the instructor modeled both of these again. The instructor also continued to facilitate the student-led discussion on the discussion board. Communication was important during this stressful time, and a weekly announcement was made on Canvas to remind students of assignments and due dates. This helped the students feel more connected and kept them on track. If a student was struggling, a personal email to the student was sent to check on them and remind them of the assignments that were due.

Suggestions for determining student learning

The challenge was to rapidly transition the course to an asynchronous online format in less than two weeks without losing progress on the student LOs during the COVID-19 pandemic. Tables 3 and 4 display the LO assessment statements and the scoring system for each statement, with 1 being excellent and 5 being unacceptable, meaning that higher scores were more unsatisfactory. Possible scores ranged from 7 (best) to 35 (worst) for LO 1 and 5 (best) to 25 (worst) for LOs 2 and 3. Microsoft Excel was used to perform a t-test where p < 0.05 was considered statistically significant in LO scores before and after the online transition. In addition, course evaluations were used to assess whether the online transition was successful.

TABLE 3

Assessment statements for the Journal Article Analysis worksheet.

Relevant background information is given including the importance of the current study.

The goal and/or hypothesis are clearly and correctly stated.

All results are correctly summarized.

Final conclusion and interpretation of the results are clearly expressed.

Explanation given for how this study advances our understanding of the research topic.

Past tense used throughout. Writing quality is up to standard.

Writing is clear and concise, with the minimum number of words necessary to convey relevant information.

The assessment was based on LO 1 and graded on the following scale: 1 = excellent, 2 = good, 3 = fair, 4 = poor, 5 = unacceptable. A score of 5 was given for each part that a student failed to complete.

TABLE 4

Critical analysis, critical thinking, creativity, and problem-solving ability assessment statements.

Identify 3 design considerations for the specified tissue or organ.

Analyze the article description given and write 2 pros and 2 cons.

Describe 1 experiment that builds upon the data given.

Formulate a project goal.

Construct a hypothesis.

Assessments were based on LOs 2 and 3 and graded on the following scale: 1 = excellent, 2 = good, 3 = fair, 4 = poor, 5 = unacceptable. A score of 5 was given for each part that a student failed to complete.

Safety issues

There are no safety issues associated with this course.

Sample data

Since no permission was sought from the Human Research Council prior to teaching the course due to the small sample size, and the COVID-19 pandemic could not be foreseen, sample data cannot be provided.

DISCUSSION

Field testing

While this was the second time that this course was taught to undergraduates at a small liberal arts college, it was the only time that a pandemic prompted a rapid change to online learning. Student enrollment was lower this time (n = 5) compared with the previous enrollment in this course (n = 11) due to other course conflicts. As the class consisted of all upper-level students and synchronous learning had occurred for 8 weeks, the shift in learning style early in the remote learning process worked well. It also allowed for greater flexibility for the instructor and the students, which helped to alleviate some stress.

Evidence of student learning

Although rapidly transitioning a course from in-person to completely online was challenging, the modular course design allowed for a successful and rapid transition to asynchronous online instruction during the COVID-19 pandemic. For LO 1, the average pre-score was 15.4 ± 3.7 (n = 5) compared with 12.7 ± 1.8 for the post-score (n = 5). A t test [degrees of freedom (d.f.) = 4, t statistic =1.62, one-tailed t-test p value = 0.09, two-tailed p value = 0.18) determined there was no significant difference between these two averages. The average class pre-score was 8.4 ± 1.0 (n = 5) compared with the average class post-score of 8.4 ± 3.9 (n = 5) for LOs 2 and 3. Again, there was no significant difference between the pre-score and post-score averages (d.f. = 4, t statistic = −0.14, one-tailed t-test p value = 0.45, two-tailed p value = 0.90). Academic integrity and an interactive learning environment were maintained since the LO assessment showed no decrease after the asynchronous online instruction transition. These results also indicate that there was a good balance between the amounts of course content and student skill development.

Course evaluations revealed that students strongly agreed that instructional methods were altered effectively for the sudden transition to online learning, and class changes were effectively communicated as the students understood and supported the decisions made. Weekly announcements reminded students when presentations were available, assignment due dates, and reiterated instructor availability. Evaluations also indicated that using Canvas for announcements, assignments, and grades was very helpful. There was also more flexibility in accepting late assignments as this fell in line with the goal of continuing student learning. For the JAA worksheet, the number of late submissions increased from 5% to 24%. The DNSE worksheet went from zero late/missing assignments to 12% for missing and 4% for late submissions. These results support that this topic-themed modular course design works for an in-person or an online course and could be adapted to fit a hybrid modality.

Possible modifications

Although the LO progress was sustained, it would be better if the assessment had shown an increase. For future online or hybrid classes, synchronous lectures and article presentations will aid in social and teaching presence; however, these would be recorded and uploaded for students who could not attend to view later. A synchronous discussion followed by an asynchronous discussion board would be used in the hopes of capturing the best of each method (25, 26). Breakout groups could be used for smaller group discussions that then report to the larger group, and multiple discussion boards could be used. The student presenter would be in charge of the discussion board, with instructor facilitation as students prefer an instructor in this role, and an individual reflection assignment would ensure each student did not just post and leave (26). These small groups or multiple discussion boards could be managed by a teaching assistant to accommodate larger class sizes. Small student groups would then complete the DNSE worksheet virtually. Though there was a small sample size, the data demonstrate a positive proof of concept. This methodology can be easily adapted to other scientific topic-themed courses, such as cell biology, genetics, ecology, wildlife biology, and biochemistry, by choosing journal articles that relate to the current classroom topic and following the described format.