The Impact of an Interactive Statistics Module on Novices' Development of Scientific Process Skills and Attitudes in a First-Semester Research Foundations Course.

Evidence suggests that incorporating quantitative reasoning exercises into existent curricular frameworks within the science, technology, engineering, and mathematics (STEM) disciplines is essential for novices' development of conceptual understanding and process skills in these domains. Despite this being the case, such studies acknowledge that students often experience difficulty in applying mathematics in the context of scientific problems. To address this concern, the present study sought to explore the impact of active demonstrations and critical reading exercises on novices' comprehension of basic statistical concepts, including hypothesis testing, experimental design, and interpretation of research findings. Students first engaged in a highly interactive height activity that served to intuitively illustrate normal distribution, mean, standard deviation, and sample selection criteria. To enforce practical applications of standard deviation and p-value, student teams were subsequently assigned a figure from a peer-reviewed primary research article and instructed to evaluate the trustworthiness of the data. At the conclusion of this exercise, students presented their evaluations to the class for open discussion and commentary. Quantitative assessment of pre- and post-module survey data indicated a statistically significant increase both in students' scientific reasoning and process skills and in their self-reported confidence in understanding the statistical concepts presented in the module. Furthermore, data indicated that the majority of students (>85%) found the module both interesting and helpful in nature. Future studies will seek to develop additional, novel exercises within this area and to evaluate the impact of such modules across a variety of STEM and non-STEM contexts.

INTRODUCTION

Recent efforts within the science education community have emphasized the importance of incorporating quantitative exercises into existent curricular frameworks within the science, technology, engineering, and mathematics (STEM) disciplines, in part to promote students’ development of scientific process and reasoning skills in these domains (1, 10, 11). Empirical evidence demonstrates that these efforts are necessary not only for students’ professional development as scientists and researchers within their fields (6–9), but also for personal decision-making in everyday contexts (8, 10, 14). In their analysis of student outcomes associated with engagement in an authentic research course, Makarevitch et al. (8) illustrated, for instance, that quantitative and experimental design skills are integral for generating novel hypotheses related to the processing and visual representation of complex datasets within the biological sciences. These findings corroborate the earlier work of Speth et al. (12), who, in addition to demonstrating statistically significant increases in novices’ ability to construct graphical representations of biological data following integration of quantitative literacy into an introductory biology course, furthermore recognized the need to prepare scientists and non-scientists alike to utilize quantitative evidence to make reason-based decisions regarding scientific and pseudoscientific claims in society.

Despite such findings, these and other studies acknowledge that students often fail to appreciate the relevance of mathematical thinking to problem-solving within the STEM disciplines, and, as a result, often perceive themselves to be poor quantitative reasoners and scientific researchers (4, 7). Likewise, it remains largely unclear to what extent various curricular activities can support students’ development of scientific process skills in the STEM domains. To address these concerns, we developed a two-session, interactive module that focuses on core statistical and research-based concepts within the scientific disciplines, including: a) hypothesis testing; b) experimental design; and c) interpretation of research findings. Given the active nature of the exercises comprising this module (2, 5), we hypothesized that student engagement in the lessons would result in a statistically significant increase in student learning gains in each of these areas, as well as an increase in students’ confidence in assimilating and applying those skills. The module was implemented in a first-semester research foundations course for science and nonscience majors and was evaluated using a quantitative study design.

Intended audience and Prerequisite student knowledge

The statistics module described in this manuscript was implemented in a research foundations course (SCI 1301: Inquiry in Mathematics and Science) whose primary purpose was to introduce novices to fundamental analytical reasoning skills necessary to effectively engage in authentic scientific research. Specifically, the activities detailed here were designed to introduce students in the STEM disciplines to basic statistical concepts and to facilitate development of their experimental design and graphical interpretation abilities. Students were predominantly first-semester freshmen majoring in the biological sciences (>85% of participants) who were enrolled in a First Year Research Intensive Sequence (FYRIS) (http://buildingscholars.utep.edu/web/index.php/student-research-training/fyris). However, the overarching structure of the module allows it to be implemented in both lower- and upper-division coursework regardless of students’ academic standing or institutional/instructional context. While no specific scientific or statistical content knowledge is prerequisite, a rudimentary understanding of the scientific process (i.e., formulating a hypothesis, testing a hypothesis, gathering data, interpreting data, drawing conclusions) and prior exposure to peer-reviewed primary literature are recommended.

Learning time

The lectures and activities comprising this module are designed to encompass two, 80-minute sessions, as detailed in Table 1. However, these times may be adjusted, as required, to align with the needs of the instructor. The entirety of each session is needed for the planned activities, as each exercise is followed by small- and/or large-group discussion (see Appendices 1 and 2).

TABLE 1

Outline of module learning activities indicating approximate time spent on each exercise.

Session 1
Activity	Approximate Time Devoted to the Activity
Instructor-facilitated lecture	20 min.
Interactive height exercise + small-group discussion	20 min.
In-class assignment (Assignment #1)	15 min.
Large-group discussion	25 min.
Session 2
Activity	Approximate Time Devoted to the Activity
Session 1 “recap”	10 min.
Instructor-facilitated lecture	20 min.
Evaluation of figure in peer-reviewed article	15 min.
Student team presentations	20 min.
Large-group discussion	15 min.

Instructor preparation

Height pre-activity survey for session 1 activities

In preparation for the interactive height activity (described below), the instructor is strongly encouraged to survey the students prior to the first session. In addition to enabling the instructor to prepare appropriate height bins for the exercise, this approach could likewise be utilized to: a) introduce students to concepts and practices related to the ethical conduct of research, namely, procedures for obtaining informed consent; b) demonstrate best practices for data collection (e.g., survey methodology); and c) predict potential outcomes associated with implementation of the activity and use that knowledge to scaffold classroom discussions (or to generate discussion prompts in advance).

With respect to students in the SCI 1301 course specifically, participants’ self-identified and self-reported height and gender information was collected on 3″ × 5″ index cards that were submitted during the session immediately preceding the statistics module. Students could elect whether or not they wished to participate and were instructed not to include any form of personal identification (name or university ID number) in order to protect their anonymity.

Demarcation of bins for the interactive height activity implemented in session 1

Immediately prior to the first session, the instructor should prepare indicators for the class height bins. This will serve to expedite the process of sorting students as they engage in the interactive height activity. In our session, six-inch increments ranging from 4′7″ to 6′6″ were marked off on dry-erase boards along the back of the classroom. Alternate materials (e.g., labeled cardstock or poster board) could likewise serve as bin indicators, as necessary.

Article selection for session 2

In preparation for the second session, the instructor should select an article that contains empirical evidence (e.g., figures) supporting the relevant statistical topics addressed in the module. In addition, the Materials and Methods section of the article should clearly state the sample size and criteria for establishing treatment groups and should be suitable for the target audience (first-year students). As part of our lesson, we selected an article used in a prior session to introduce the various sections of a peer-reviewed publication (e.g., abstract, introduction, materials/methods, results, and conclusions). Figures presented in the Results section of the article had not been reviewed during this prior session. The specific article chosen for the statistics exercise was:

Shirai, O., et al. 2002. Alcohol ingestion stimulates mosquito attraction. Journal of American Mosquito Control Association 18(2):91–96. (www.biodiversitylibrary.org/content/part/JAMCA/JAMCA_V18_N2_P091-096.pdf)

This article was selected because it provided a clear research question and a sufficient number of figures such that each student group could analyze a unique figure for the purposes of classroom discussion. The straightforward nature of the article also enabled students to easily deconstruct and discuss aspects of the authors’ experimental design.

Learning outcomes

Student learning objectives and methods of assessment for the module are detailed in Table 2. From a holistic standpoint, these objectives are designed to promote the development of scientific process skills relevant to the design and evaluation of experimental studies.

TABLE 2

Module learning objectives and methods of assessment.

Student Learning Objective	Method of Assessment
1. Utilize knowledge of statistical concepts and experimental design principles to evaluate a figure published in a peer-reviewed, scientific article	Student Team Presentations (Formative Assessment – Session 2)a
2. Apply statistical concepts and experimental design principles to evaluate scientific claims and interpret graphical data broadly within the STEM disciplines (e.g., in the biological sciences)	E-EDAT, GI (Pre-/Post-/Delayed Post-Activity Test)b
3. Demonstrate increased confidence in understanding and applying statistical concepts	SPLG (Pre-/Post-Module Survey)b

STEM = science, technology, engineering, and mathematics; E-EDAT = expanded experimental design ability tool; GI = graphical interpretation; SPLG = student perceptions of learning gains.

See Appendix 3 for a sample evaluation rubric for this portion of the module.

See the “Evidence of student learning” section for complete descriptions of each of these assessments.

PROCEDURE

In the first session, the course instructor guides students to create a physical representation of a normally distributed curve based on participants’ heights. In low-enrollment courses, this curve will potentially possess a non-normal/skewed distribution, which can be addressed through large-group dialogue. Students utilize the visual and physical information represented by the student-generated curve to describe core statistical concepts, including mean and standard deviation. This exercise promotes discussion of sample selection criteria, in particular, as the class will likely observe differences between the height distributions for men and women. Furthermore, student-driven conversations resulting from the activity offer a platform for organic exploration of height demographics in different regions of the world, as well as the mediating effect of race/ethnicity, gender, genetic variation, and environmental conditions on this outcome.

In the second session, student teams (n = 9/team) work collaboratively to identify critical elements of a figure published in a scientific, peer-reviewed article provided by the course instructor (e.g., appropriate display of error bars based on collected data), to critically analyze the experimental sample size, and to contextualize the validity of reported results in relation to the authors’ hypothesis. Students’ conclusions from this exercise are discussed in class through informal presentations (“chalk talk”) facilitated by instructor-selected team leaders from each group, allowing immediate access to formative feedback from both the instructor and peers (Appendix 3).

Materials

The following resources and materials are required for this module:

Access to writing surfaces, such as chalkboards, whiteboards, poster boards, or paper affixed to the wall, to be used by both the instructor and students

A scientific, peer-reviewed article (to be made available by the course instructor) that aligns with the learning objectives of the module and the target audience (first-semester freshmen)

A personal computer or portable device equipped with internet access that can be used in class to retrieve data or information pertinent to the interactive height activity or the article discussion

Student instructions

Student procedures are provided as verbal prompts by the course instructor, as outlined in the lecture materials (Appendices 1 and 2).

Faculty instructions

Lecture slides for each session are provided in Appendices 1 and 2, with specific remarks to the instructor presented in the ‘Notes’ pane. As mentioned previously, the first session (Appendix 1) is intended to introduce students to descriptive statistics and their application within the scientific disciplines. Within this session, the instructor should encourage students to provide familiar examples of descriptive statistics, including grade point averages, distribution of class performance on a summative assessment, and conversion of percent values to a letter scale (transformation of scores), the express intent of which is to extend discussion resulting from the interactive height activity. Instructors are likewise encouraged to allocate time for valuable, spontaneous discussion that might arise. In our experience, relatable concepts such as height can often prompt subsequent student questioning, offering opportunities for students to independently pursue answers to those questions through use of electronic databases and/or peer-peer dialogue.

In contrast, the focus of the second session (Appendix 2) is to articulate the overarching purpose and interpretation of inferential statistics. Because of the nature of our student demographic, no specific methods or testing procedures (e.g., analysis of variance [ANOVA], Chi-square analysis) were introduced. Instead, the session is designed to target and address common student misconceptions, such as “proving the alternative hypothesis” (3). As with the first session, detailed notes are provided to guide instructors through the various group activities and primary concepts that should be covered as part of the session itself.

A comprehensive overview of the statistics module, including all pre-, post-, and intra-lesson procedures and activities, is presented in Appendix 4.

Suggestions for determining student learning

We recommend pre- and post-module assessment of students’ conceptual understanding and affect. This includes administration of experimental design and graphical interpretation diagnostics (e.g., expanded experimental design ability tool [E-EDAT] and graphical interpretation [GI] assessment; Appendix 5) as well as measurement of students’ perceived learning gains (SPLG; Appendix 6). Delayed posttest methods could likewise be implemented to assess development of scientific process skills over the course of the semester. Collectively, these instruments can be administered in one, 30-minute block. To ensure the fidelity of assessment items, pretest responses should not be discussed or returned to students, and all diagnostics should be administered under identical conditions (e.g., same presentation format and instructions/completion requirements).

Sample data

With the guidance of the course instructor, students voluntarily engaged in an interactive height activity, leading to the physical representation of a normal curve designed to intuitively illustrate basic descriptive statistics (e.g., mean, range, standard deviation). Variations on this theme (e.g., restricting participation to men only) prompted meaningful discussion regarding how to locate and interpret height measurements within public databases, including interpretation of measurements following stratification by gender, age, and race/ethnicity. Within their own teams, students were likewise prompted to evaluate a figure from a peer-reviewed, scientific article and present their evaluation to the class.

Safety issues

There are no safety issues associated with this module.

DISCUSSION

The primary objective of this statistics module was to use interactive demonstrations and critical reading exercises to facilitate novices’ comprehension of fundamental concepts in the field of statistics and to apply those concepts to “real world” scientific practices. Importantly, the concepts we elected to focus on are commonly used within the STEM disciplines and, as such, are likely to contribute to increases in students’ self-efficacy in applying quantitative reasoning skills in future research endeavors (10, 11). Analysis of pre-, post-, and delayed post-module data revealed statistically significant increases in students’ development of scientific process skills, as well as increases in students’ perceived confidence in acquiring and applying basic statistical concepts. Furthermore, formative feedback suggested that the majority of participants (>85%) found the module to be both interesting and helpful in nature.

Evidence of student learning

Participant recruitment procedures

Participants (n = 32) included all students enrolled in a first-semester research foundations course at a large, doctoral degree-granting institution in the Southwest. Participants were not selected on the basis of any qualifying factors and did not receive compensation for their engagement in this research. This research was approved by the university’s institutional review board under protocol ID 789648-2.

Measures of experimental design and graphical interpretation skills

Students’ development of experimental design skills was assessed using the expanded experimental design ability tool (E-EDAT), which has been validated for use within research-oriented contexts such as the one described here (Brownell et al., (3); Cronbach’s α = 0.723, in our context). This diagnostic presents students with a scenario in which ginseng consumption is reported to increase human endurance and requires them to design an experiment to test the accuracy of this claim. Student responses were scored in accordance with those criteria established by Brownell and colleagues (3) and entered directly into IBM SPSS STATISTICS v. 22 (IBM), hereafter referred to as “SPSS,” for future analysis.

Students’ development of quantitative reasoning skills was assessed using the graphical interpretation (GI) assessment, a course-specific diagnostic designed to measure participants’ understanding of basic statistical concepts as represented within pre-published figures (Appendix 4). Psychometric analyses indicated a high degree of construct validity (as established through expert panel and participant review processes) and reliability for the GI (Cronbach’s α = 0.765). Student responses were scored using a pre-determined rubric (Appendix 7) and entered directly into SPSS for future analysis.

Participants completed both the E-EDAT and GI immediately prior to and following the statistics module so that shifts in students’ experimental design and graphical interpretation abilities could be determined. These same assessments were administered as a delayed posttest at the end of the last course session to measure students’ long-term development of scientific process skills.

Because both the E-EDAT and the GI were created to assess students’ development of scientific process skills, a counterbalanced design was implemented to reduce the likelihood of confounding due to instrument presentation bias (i.e., viewing GI prompts first versus E-EDAT prompts first). Results of a multivariate analysis of variance (MANOVA) procedure indicated no statistically significant between-group difference in pre-, post-, and/or delayed post-module test scores on either the E-EDAT or the GI (F(6,25) = 0.308, p = 0.927; Wilk’s Λ = 0.93; ηp2 = 0.07), suggesting that the order in which these instruments were presented to students did not influence their responses.

Student perception of learning gains (SPLG) survey

In addition to determining the impact of the statistics module on students’ development of experimental design and graphical interpretation skills, we examined the degree to which students’ perceived confidence in understanding and applying basic statistical concepts changed as a result of participation in the module. Participants were asked to complete a brief survey consisting of 11 Likert-item questions immediately preceding and following the module. The post-module survey contained an additional six Likert-item questions designed to assess the extent to which students found module exercises to be helpful and interesting (Appendix 6). Descriptive statistics were tabulated for Likert-item questions, with potential rankings ranging from “1” (not confident or not helpful/interesting) to “5” (extremely confident or extremely helpful/interesting), and the data were entered directly into SPSS for future analysis.

Engagement in the statistics module results in increases in students’ development of experimental design and graphical interpretation skills

Student performance on the E-EDAT and GI was assessed using a series of paired t-tests with Bonferroni correction. These analyses revealed statistically significant learning gains on both diagnostics (tE-EDAT (31) = −2.98, p = 0.006; tGI (31) = −5.74, p < 0.001; Fig. 1), indicating increases in students’ development of experimental design and graphical interpretation skills following participation in the module. Subsequent analyses of posttest and delayed posttest data demonstrated a statistically significant gain in student learning for the E-EDAT (t(31) = −2.37, p = 0.024; Fig. 1) but not for the GI (t(31) = 0.77, p = 0.443).

FIGURE 1

Pre-, post-, and delayed posttest analyses of student performance on the E-EDAT and GI following participation in the module reveal a positive impact of the statistics module on students’ development of scientific process and reasoning skills. *p < 0.025; **p < 0.001. E-EDAT = expanded experimental design ability tool; GI = graphical interpretation.

We believe these latter results can be attributed to the structure of the course as a whole. Specifically, while graphical interpretation skills are a microcomponent of SCI 1301, the course’s overarching purpose is to help students develop the critical thinking skills necessary to effectively design experiments and generate research proposals. To that end, course activities and assignments peripheral to the statistics module focused on reinforcing and developing students’ experimental design abilities.

Participation in the module increases students’ perceived confidence in understanding and applying basic statistical concepts

Student responses on the pre-and post-module SPLG survey were analyzed using a series of paired t-tests with Bonferroni correction. Results indicated a statistically significant increase in self-reported confidence on all assessment items (p < 0.004 for all comparisons; Fig. 2) following participation in the interactive exercises. In particular, self-reported increases in interpreting p-values and error bars, explaining α-values, and recognizing the difference between Type I and Type II errors were of interest. Calculated effect sizes (data not shown) indicated that these elements exhibited the lowest initial self-reported confidence and the largest subsequent increase in confidence, attesting to the success of the approaches implemented in the statistics module.

FIGURE 2

Post-module shifts in students’ confidence in understanding and applying basic statistical concepts are indicative of self-reported development of essential statistical knowledge. p < 0.004 for all comparisons.

The interactive exercises comprising the module were helpful and interesting

Subsequent descriptive analyses of post-module, SPLG Likert-item questions assessed the extent to which students perceived module activities to be both helpful and interesting in nature. Data indicate that students believed that the instructor-facilitated lecture session was most helpful (with 100% of participants rating this activity as either helpful or extremely helpful), followed by evaluation of figures within a scientific, peer-reviewed article (96.9% of participants), and, lastly, engagement in the interactive height activity (93.8% of participants). Conversely, the interactive height activity was perceived as most interesting (with 93.8% of participants rating this exercise as either interesting or extremely interesting), with the figure evaluation activity and the lecture session rated equally thereafter (90.6% of participants in each category) (Fig. 3).

FIGURE 3

Student perceptions regarding the degree to which module activities were helpful and interesting in nature are positive, indicating a high level of utility among all components of the statistics module. “Not Helpful/Interesting” represents the percentage of students indicating a Likert-item score of one (“1”) or two (“2”). “Helpful/Interesting” represents the percentage of students indicating a Likert-item score of three (“3”). “Extremely Helpful/Interesting” represents the percentage of students indicating a Likert-item score of four (“4”) or five (“5”).

Possible modifications

Select exercises, including end-of-class review questions/discussion and analysis of figures published within a peer-reviewed scientific article, were allotted approximately 15 minutes of class time and were conducted in groups of nine students. Our analyses of collected audio recordings of student conversations (data not shown) indicated that the time required to complete these tasks ranged from five to seven minutes. Thus, future iterations of the module might benefit from extending these activities to further promote student learning and engagement or from incorporating additional exercises to illustrate, in alternate modalities, statistical concepts addressed within each session. Additionally, in-class assignments administered at the end of each session could easily be issued as homework or presented as in-class activities during the following session as a means of review. One final observation from the student recordings was that only four or five students (out of nine) in each group were usually actively engaged in the assigned task. We thus recommend that groups for in-class activities be restricted to four or five students to encourage greater individual participation in the group conversations (13).

CONCLUSION

The goal of this study was to determine the impact of interactive demonstrations and critical reading exercises on students’ comprehension of basic statistical concepts essential to a successful research experience. To that end, we integrated a series of activities focused on hypothesis testing, experimental design, and interpretation of research findings, particularly in relation to “real world” scientific events, into the existing framework of a research foundations course for science and nonscience majors. Using a pre-/post-module quantitative research design, we discovered a statistically significant increase in both students’ scientific reasoning and process skills as well as students’ self-reported confidence in understanding the statistical concepts presented in the lessons. Furthermore, the majority of students found the module to be both interesting and helpful in nature. Based upon these findings, we predict that students’ increased confidence in their ability to apply quantitative reasoning skills will foster their successful participation in future research environments. We likewise encourage educators to incorporate similar exercises into their own curricula in a manner that best serves to increase students’ scientific and quantitative literacy.

Appendix 1: PowerPoint lecture notes accompanying session 1

Appendix 2: PowerPoint lecture notes accompanying session 2

Appendix 3: Rubric for evaluation of student team presentations (session 2)

Appendix 4: Comprehensive instructional overview of the SCI 1301 statistics module

Appendix 5: GI and E-EDAT assessments

Appendix 6: SPLG survey

Appendix 7: GI scoring rubric