Student Use of Self-Data for Out-of-Class Graphing Activities Increases Student Engagement and Learning Outcomes.

Two out-of-class graphing activities related to hormonal regulation of the reproductive cycle and stress responses are used to determine whether student use of self-data vs. provided data increases engagement, learning outcomes, and attitude changes. Comparisons of quizzes and surveys for students using self- vs. provided data suggest that while both activities increase learning outcomes, use of self-data compared with provided data has a greater impact on increasing learning outcomes, promotes recognition that hormones are relevant, and enhances confidence in graphing skills and graphing efficacy.

INTRODUCTION

The purpose of this study is to identify whether using self-data compared with provided data for graphing activities emphasizing hormone regulation increases student engagement and learning. Identifying strategies to increase student engagement is important because it correlates with increased academic performance and retention (1). Inquiry-based-learning not only increases motivation for learning and reasoning skills (2) but also correlates with increased GPA (3). Course-based research experiences increase student engagement, scientific and critical thinking, data interpretation, and student retention (4–8). Small-group debates also correlate with greater participation, changed attitudes, and higher grades (9), while current events discussions can increase learning gains and change student perception of biology (10).

Dee Fink explains that significant learning prompting lasting change can be achieved by incorporating six overlapping categories into pedagogy: caring (values), human dimension (learning about oneself), integration (connecting ideas), application, foundational knowledge, and learning how to learn (11). Furthermore, each category synergizes with the others to promote integrative learning (11). Inquiry-based learning, course-based research, and other strategies that increase engagement incorporate most, if not all, of Fink’s categories.

This study, therefore, investigates whether student use of self-data compared with provided data increases engagement and enhances learning outcomes. The hypothesis is that, if students use self-data, they will be more engaged and demonstrate increased learning outcomes. This is expected because students will connect concepts to themselves, participate in inquiry-based learning for which they must discover an unknown answer, and incorporate all six of Fink’s categories. To test this, I developed two out-of-class graphing activities related to hormonal regulation of the reproductive cycle and the stress response. For each activity, students either used self- or provided data to create graphs illustrating their predictions for hormonal and physiological changes. Differences in engagement, learning gains, and attitude changes were compared using pre vs. post quizzes and surveys.

Identifying strategies that engage students outside of class is valuable to maximize class time and motivate students to complete assignments. Data from this study suggest that student learning outcomes increase after each activity and that using self-data compared with provided data promotes increased learning outcomes, engagement, and attitudinal changes related to the importance of hormones, confidence in graphing skills, and graphing efficacy.

METHODS

Basal body temperature out-of-class graphing activity

Students recorded basal body temperature (BBT) corresponding to either their own reproductive cycle or 30 consecutive days (if not expecting a reproductive cycle) or were given a data set. Students graphed BBT and predicted corresponding hormonal changes regulating the reproductive cycle using a textbook and journal article as models (12, 13). Students included a discussion of the changes they would expect in the ovaries and uterine lining. Appendix 1 includes instructions and rubric.

Stress reduction out-of-class graphing activity

The objectives of this activity were of equal complexity to those of the BBT activity and similarly related to hormone regulation. All students learned how to collect pulse and stress level perception during an in-class meditation and/or deep breathing activity. Students using self-data performed three additional stress reducing activities, three times each, collecting pulse and stress level perception before and after each technique. A data set was given to students using provided data. All students entered data into a spreadsheet template to graph the average changes for each experiment and to determine the p value. Based on stress level changes, students predicted changes for hormone levels and explained effects on biological systems. This activity was expanded from a laboratory experiment (14). Appendix 2 includes instructions and rubric.

Student and course characteristics

This study was conducted in a 200-level general education capstone course, Biology of Women, for biology majors and nonmajors at the women’s College of Arts and Sciences at Trinity Washington University, which has a student population comprised of approximately 96% students with minority backgrounds. Therefore, participants in the study were women with ethnicity designated as mostly either African American or Hispanic, but specific demographic information was not collected.

Developing and comparing groups using self- vs. provided data

This study was conducted over five years during which each class had approximately 15 students. Classes were designed such that either: 1) all students collected self-data for the BBT activity; 2) students were randomly assigned to collect self-data for one activity and provided data for the other (two classes); or 3) students were given a choice about collecting self-data (all but two students used provided data for both activities).

Comparisons of learning outcomes and engagement measures were then made between: 1) students from a class in which all students used self-data for the BBT graphing activity and students using only provided data from a class in which self-data collection was optional; 2) students using self-data for one activity and provided data for the other within classes; 3) students using self-data for one graphing activity and provided data for the other using combined classes; and 4) students from a class using only provided data and students from combined classes using self-data for one graphing activity.

The number of students per analysis is indicated in the results section and varies according to whether comparison groups are made between entire classes, a class divided in half, or students from multiple classes. Additionally, while each class started with 15 students, only students who completed related pre/post quizzes and surveys were included in the analyses.

Learning outcome measurements

Students completed quizzes at three points: after a lecture about hormones but before completing each graphing activity, after each graphing activity, and at the end of the semester (final). The average percentage of students correctly answering multiple questions was compared using Student’s t-test between the quiz given before the graphing activity and the final for groups that used self- vs. provided data. For comparisons made within classes, or combined from classes in which students were split into groups using either self- or provided data, only students completing all pre/post quizzes and graphing activities were included in the study. Percentages of students correctly answering complex specific questions requiring integration of concepts related to hormone regulation for each activity were also compared between groups using self-data and provided data. Student’s t-test was used to determine significance (p < 0.05).

Grade comparison

Grades from three separate classes were compared. In two classes, students used self-data for one graphing activity, while students from the third class used only provided data. Student’s t-test, was used to compare the percentage earned on the first quiz of the semester in the class using only provided data with the other classes, and no statistical significance was observed. The grade distribution for each class was computed.

Surveys

After completing the graphing activities, students answered open-ended questions to determine whether those who used provided data would have liked to use self-data, and whether those who used self-data liked using self-data and thought it helped their learning and why. Responses were grouped according to categories emerging from the survey. Survey responses were collected from two classes in which the students used self-data for one activity and provided data for the other. Students were also asked what part of the graphing activity was most helpful to learning.

A Likert survey was given at the beginning and end of each semester for students to fill out anonymously. Only students who completed both the initial and final surveys were included in this analysis. Students indicated whether they collected self- or provided data. Cronbach’s α was calculated to determine reliability for similar questions that were combined for the analysis (see Appendix 3). Student’s t-test was used to determine whether there were statistically significant differences (p < 0.05) between initial and final responses.

IRB approval

The Institutional Review Board at Trinity Washington University granted permission for these studies to be conducted with exempt status.

RESULTS

The basal body temperature (BBT) graphing activity was first assessed to determine whether it effectively promotes student learning when students collect self-data. The average percentage of students correctly answering questions on the quizzes after completing the graphing activity with self-data was significantly higher (p < 0.05) than on the quiz given before the activity (Fig. 1A).

FIGURE 1

A) Average percentage from 14, 14, and 15 students answering 4, 9, or 3 questions correctly on the quiz after the lecture, after using self-data for the BBT activity, and on the final quiz, respectively (*p < 0.05 compared with quizzes after the BBT activity and on the final. Error bars represent standard deviation.) B) Percentage of students correctly answering questions asked on both a quiz before the BBT activity and on the final were compared between a class in which everyone used self-data for the activity and students from a class using all provided data (12 students in the provided group, and 14 for the initial quiz and 15 for the final quiz in the self-group). BBT = basal body temperature.

To determine whether using self-data enhanced learning gains compared with using provided data, the percentages of students correctly answering two questions on a quiz both before and after the BBT activity were compared between a class in which everyone used self-data for the activity and a class in which students chose to use provided data. (The questions can be found in Appendix 4.) While similar percentages of students correctly answered questions on the quiz before the activity, there is a greater increase in the percentage of students correctly answering questions after the graphing activity in the group using self-data (65.2% and 37.6% increase, respectively, for the two questions) compared with the group using provided data (29.5% increase for both questions) (Fig. 1B). (Data were not based on individual student responses, and, therefore, statistical analysis was not calculated.)

These data suggest that learning gains are made after using the BBT graphing activity and that using self-data enhances learning gains. However, the data compared were from different students in different classes. To minimize the effect of these differences, comparisons were next made within classes in which half of the students used self-data for one graphing activity and provided data for the other. Both activities relate to hormone regulation and have similar objectives, difficulty, and instructions. Differences in learning gains were therefore compared between groups comprised of the same students completing one activity using provided data and one activity using self-data. This was repeated for two classes.

For the first class, more students in the group using self-data for the BBT graphing activity correctly answered multiple questions on the final exam than on the quiz given before the graphing activity; the difference was statistically significant, and there was no significant difference on quiz outcomes in the group using provided data (Fig. 2A). For the stress activity, both groups had a statistically significant increase in the average percentage of students correctly answering multiple questions on the final exam compared with the quiz taken before the graphing activity, but the group using self-data also had a significantly higher average on the final exam (p < 0.05) than the group using provided data. Appendix 5 lists quiz questions.

FIGURE 2

Average percentage of correct responses on pre-activity quiz and final exam. A) From a class with 6 students in each group. There were 11 and 12 questions related to the BBT (reproductive) activity on the pre quiz (before the activity) and final, respectively, and 4 and 3 questions related to stress on the pre quiz and final, respectively (*p < 0.05). B) From a class with 7 students in the group using self-data for the BBT activity (provided for the stress activity) and 5 students in the other group, with BBT quizzes consisting of 5 questions on the pre quiz and 8 questions on the final, and the stress quizzes consisting of 3 questions for each. Error bars represent standard error of the mean. BBT = basal body temperature.

This intervention was repeated for a second class, and a statistically significant increase was observed in the average percentage of students correctly answering multiple questions on the final exam compared with the quizzes taken before both graphing activities (stress and BBT) in groups using self- and provided data (p < 0.02). While this suggests that both activities increase learning gains when students use either self- or provided data, the group using self-data for the BBT graphing had a higher average percentage of students correctly answering multiple questions on the final exam that was statistically significant compared with the group using provided data (p < 0.02) (Fig. 2B). Students using provided data for one activity are the same students using self-data for the other. These data suggest that while both graphing activities increase learning outcomes, using self-data provides an advantage to increase learning outcomes compared with using provided data.

To determine whether using self-data for the graphing activities increased student understanding of hormone interaction, correct student responses to one open-response question for each activity (Appendix 6) were compared for students using self- vs. provided data on quizzes preceding and following the graphing activities (not the final exam, as included in Fig. 2). Student responses were combined from classes in which students used self-data for one activity and provided data for another. (Only students completing all four quizzes were included.) While there was no statistical difference in the number of students correctly answering the questions before the activity, a significantly greater number of students (represented as percentage in Fig. 3) using self-data for the BBT activity answered the question correctly compared with the group using provided data (p = 0.04). Similarly, a greater percentage of students in the self-data group had correct responses related to stress hormones than students using provided data, although the difference only approached statistical significance (p = 0.17). Importantly, a higher percentage of students correctly answered the question when they used self-data for one activity, compared with the group using provided data, while the percentage of these same students correctly answering questions related to the other activity when using provided data was lower compared with the group using self-data.

FIGURE 3

Percentage of students correctly answering a hormone regulation question before and after completing BBT or stress graphing activities when using self- or provided data. BBT = basal body temperature.

To determine whether using-self data increased engagement, final grades from three classes were compared in which students had similar results on the first quiz (no statistical difference was detected using Student’s t-test). In two of these classes, students used self-data for one graphing activity. For the third class, students chose to use self- or provided data. All but two students (who were excluded from this analysis) chose to use provided data. Notably, the group using exclusively provided data had final grades with more Fs (29%) than the other two classes (0%), and fewer final As and Bs than the other classes (Fig. 4).

FIGURE 4

Grade comparison between classes using self-data (n = 15 in each) and a class using provided data (n = 13). p > 0.05 for percentages on initial quizzes.

To ascertain whether student use of self-data increased engagement and interest, students for each graphing activity were given the following survey questions:

If you did not collect self-data would you have wanted to and why or why not?

If you collected self-data, did you like it, and was it helpful to your learning and why or why not?

The percentages of students providing answers related to common themes are represented in Figure 5. The majority of students using provided data for the BBT or stress activity indicated that they would have liked to collect self-data (60% and 86%, respectively) and an even greater percentage of students using self-data for the stress activity indicated that they liked using it and that it was helpful to learning (93%). The majority of students using self-data for the BBT activity indicated that they liked it (75%; data not shown), while 92% found using self-data helpful to learning.

FIGURE 5

Percentage of survey responses from students using A) provided data from the BBT activity, B) self-data from the BBT activity, C) provided data from the stress activity, and D) self-data from the stress activity. BBT = basal body temperature.

The predominant reason indicated from all groups except those using self-data for the BBT activity for why they liked or would have liked to use self-data was the connection to themselves, by either learning directly about themselves or their body, or relating information to their own body. Other reasons include increasing interest, engagement, or understanding. The predominant reason indicated for why using self-data for the BBT activity helped their learning was increasing understanding (33.3%), while only 16.7% wrote about learning about their body, lower than survey responses in all other groups. Other responses indicated that using self-data for the BBT activity made it more engaging/interesting (16.7%), helped them relate information from themselves to hormones (8.3%), or helped them remember (8.3%). These data suggest that connecting information to themselves increases students’ understanding, and helps them become more fully engaged and interested.

A greater percentage of students using provided data for the BBT activity indicated that they did not want to use self-data when compared with students using provided data for the stress activity (40% vs. 14%). Additionally, when students used self-data for either activity, a smaller percentage indicated that they did not like doing so, compared with the group using provided data. This suggests that students have more concerns about completing the activity with self-data when they are given provided data than when they actually use their own data. Responses further suggest that students recognize that using self-data data enhances their learning and engagement.

Students indicated not wanting to collect their own data because it would take more time and/or they would forget. For the BBT activity, however, there were some activity-specific concerns. Twenty percent of students were concerned about collecting self-data because they didn’t have an average cycle while 7% were concerned about what they might find. These concerns were only raised during the first year the survey was given, likely because concerns were addressed by the professor when the activity was described in subsequent years. Furthermore, student responses to a question about what they learned by completing the BBT activity indicate that it is not necessary for students to have an average reproductive cycle for the activity to be valuable to learning. One student using self-data indicated that, “everybody’s body is different, and using contraceptives has an impact on that as well.” Another student wrote, “I have a better understanding of how my birth control affects my hormones and how I can tell if I ovulate.” These answers suggest that students without an average reproductive cycle benefitted from using self-data.

To further differentiate how using self-data impacted engagement and learning, students were asked which part of the graphing activity was most helpful to their learning (Table 1). While several responses from both groups illuminated that graphing helped their learning, there were some differences noted. Responses from students using self-data suggested that students integrated information from the class into their own lives and it helped with understanding. In contrast, some responses from the group using provided data indicated a greater disconnect with the graph, a more superficial understanding, factual errors, and less motivation towards working through the challenges.

TABLE 1

Students’ responses to open-ended questions after completing the BBT activity using self- or provided data.

Question: What part of the BBT graphing activity was most helpful to your learning?
Provided data users	Self-data users
“I think trying to analyze the different temperatures and reading over the PowerPoints.”	“Taking my temperature every morning let me get an idea of my hormone levels and I could incorporate what I was learning in class to real life.”
“None of it. All I did on my graph was plot the data. That is all I knew how to do.”	“The graph. When I actually graphed the information I obtained and delivered the conclusion of progesterone and estrogen levels, I was able to understand better.”
“The part of the menstrual cycle that was helpful was the Follicle Stimulating hormone and Luteinizing hormone and how it affects the temperature levels.”	Predicting what should happen to the hormones because it helped me learn the functions of each involved in the reproductive cycle.”
“Having the graph already there, so I just had to follow the trend.”	“Even though I had trouble with the graphing activity, … what I found helpful to learn while doing this activity was how to draw lines and how the estrogen and progesterone cycle vary. Now when I see a graph, I can identify them and ovulation as well.”
“Having a data set although I would have preferred my own data.”	“Describing based on the temperature where the hormones would be located.”
“The most helpful was learning what happens to your egg during follicle development, ovulation, corpus luteum and luteal regression.”	“The part of the menstrual graphing activity that was most helpful was learning the body and if ovulation will occur and when.”
“Being able to have a visual of the follicle and endometrium lining throughout the cycle.”	“The most helpful was learning how to read the graph: knowing when and where the phases began and ended.”
“Designing the graph was the most helpful for my learning. It helped me visualize what was happening.”	“Creating the graphs was more helpful in learning.”
“I think the most helpful part of the graphing activity was having to show all the changes that happen while keeping one’s BBT in mind.”	“The hormonal part because each hormone has a direct effect on every part of the uterine/ovarian cycle overall.”
“Learning about the endometrial shedding and ovarian changes. I am able to have a better understanding about how the body changes/experiences during menstruation.”	“Recording my personal data helped me see how different my body functions.”
“I consider graphing ovulation was most helpful for my learning because the levels of hormones rise and the graph illustrates the level increase.”	“Seeing when I’m most fertile. In case I ever decide to have a child.”

The italicized statement demonstrates inaccurate understanding of hormonal interactions.

BBT = basal body temperature.

To determine whether self-data impacted attitudes, changes in Likert survey responses were compared from the beginning to the end of the course for students who used self-data for at least one activity and for students who used all provided data. Increases with significant differences (p ≤ 0.02) in responses were obtained from students who used self-data but not from students using only provided data for the following assertions: hormones are relevant, students are good at making and interpreting graphs, and students will improve their graphing skills with practice (Fig. 6). (Appendix 3 lists specific survey questions combined for each topic.) These data suggest that using self-data increases student engagement and impacts students’ values, learning, and efficacy for learning.

FIGURE 6

Likert survey results comparing average responses from students in a class using only provided data (9 students) with responses from students in classes using self-data (28 students) for one graphing activity. (Strongly agree = 5, Agree = 4, Neutral = 3, Disagree = 2, Strongly disagree = 1)

DISCUSSION

These data suggest that the described graphing activities increase learning outcomes. Furthermore, a comparison of learning outcomes from the same groups of students using self-data for one activity and provided data for another indicates that learning outcomes increase when students use self-data. For two classes in which half the students used self-data for the BBT activity, higher learning outcomes were observed, with statistical significance (Fig. 2) for students using self-data compared with those using provided-data. A statistically significant difference between groups using self- vs. provided data for the stress activity, however, is only observed in one of the two classes. Likewise, the number of students correctly answering a complex question assessing understanding of hormone interactions is significantly higher for students using self-data for the BBT activity (p < 0.05) but only approaches significance for the stress activity (Fig. 3). The benefits for using self-data for the BBT activity are therefore consistent. It is possible that the inconsistent statistical significance for using self-data for the stress activity is due to the small numbers of students and questions used for assessment. Alternatively, class instruction involving all students in a stress reduction activity, or the relatability of stress, may enable students to more readily connect hormone interaction to themselves without collecting their own data and thus curb the impact of using self-data.

Free responses from survey questions indicated that most students liked using or would have liked to use self-data because they would have learned more about their own body, while others indicated it increased their interest, engagement, and understanding and allowed them to make connections between information and themselves. This emphasizes the importance of learning about oneself to increase engagement. Interestingly, more students using self-data for the BBT activity indicated that it was helpful to learning because it helped them understand or remember. This corresponds to the increase observed for learning outcomes for students using self-data for the BBT activity. Taken together, data suggest that students value learning about themselves, learning outcomes are impacted by student engagement, and relating topics to themselves increases engagement.

The most common reason students indicated that they did not want to collect self-data was the additional time involved. However, the benefits of increased engagement and potential for increased learning outcomes likely outweigh the drawback of additional time required for collecting self-data, as increased time on task is related to student performance (1). Interestingly, when given the choice to collect self-data, most students (while initially planning to collect their own data) chose to use provided data. This class ended with the most Fs and fewest As and Bs when compared with two classes for which all students were required to use self-data, even though they began with approximately the same initial quiz scores. While it is possible that this can be explained by differences in the students or instructional differences, it is likely, as data from this report suggest, that not using self-data resulted in students either being less engaged or lacking the advantage of using self-data to enhance understanding.

In further support of self-data impacting engagement, changes in attitudes as determined by a Likert survey were identified in students from classes using self-data but not for students using only provided data. Attitude changes include believing that hormones are relevant, perceiving their own graphing competencies, and recognizing that they can improve with practice.

Taken together, these data suggest that using self-data enables students to incorporate all six dimensions Dee Fink explains as essential to promote significant learning (11). Data from Likert survey responses indicate that self-data impacted student values (hormones are important) and learning how to learn, indicated by an increased sense of graphing competency and efficacy. Students who can interpret graphs more skillfully will likely more effectively read and evaluate information and are thus more empowered to build knowledge. Student survey responses also indicate that, when using self-data, students learned about themselves (human dimension) and connected ideas from themselves to the information or vice-versa (integration). Finally, assessment data indicate that students built and applied knowledge. Therefore, using self-data incorporated all six components of Fink’s integrated model, increased student engagement and promoted significant learning. This study specifically identified benefits of using self-data compared with providing a data set, with the difference lying with two of Fink’s categories: caring and learning about oneself, which impacts a third category, learning how to learn.

A limitation of this study is that it was conducted in small class sizes and therefore collected over multiple years. Additionally, the students were all women from minority populations (predominantly African American and Hispanic). Therefore, while these data suggest that using self-data is an effective strategy to engage and promote learning for this student population and would likely be effective for any student population, the impact was not tested for a broader student population. However, the impact of identifying a strategy to engage and promote learning outcomes for minority women in STEM is a topic of great national importance. For instance, according to Integrated Postsecondary Education Data (IPED) numbers, the four-year completion rate for females in nonprofit institutions entering college in 2008 was 60.4% for white students but only 34.8% and 50.5% for Blacks and Hispanics, respectively (15). Furthermore, increased engagement for both first-year and senior college students corresponded to increased GPA, with an even greater impact identified for Hispanic students and students entering college with lower ACT scores (1). Engagement also positively affected persistence from first to second year, with the greatest impact found for African American students (1). Therefore, identifying strategies to engage and promote learning for women of minority backgrounds is essential to chip away at racial disparity in completion rates.

The benefits of using self-data, including increased learning gains, motivation, engagement, and understanding, are consistent with evidence showing that problem-based learning and course-based research are beneficial to students’ retention and engagement (1, 3, 7). This report therefore provides a specific strategy to promote small scale open-ended inquiry-based learning for projects in or out of class to increase the impact for significant learning. Additionally, changes in attitude related to confidence in graphing skills and increased efficacy raise the possibility that self-data may also enhance quantitative literacy and self-efficacy, particularly related to quantitative skills.

Click here for additional data file.