The enhanced effect of a twitter intervention on Pell grant recipients 6-year STEM graduation rates.

This study evaluates the long-term impact of a randomized controlled study which utilized Twitter as an educational intervention to extend the course beyond the physical and time constraints of face-to-face instruction. Participants included 125 undergraduates from a mid-sized midwestern doctoral granting university in the United States. Outcomes previously published include increased engagement and grades for the intervention group (Twitter) compared to the control group. In the longitudinal study presented here, interaction effects were analyzed between GPA, ACT, gender, and socio-economic status with first-to second year retention rates, 6-year graduation rates and 6-year graduation rates in STEM fields. A statistically significant interaction effect was found between socio-economic status and the intervention (Twitter), resulting in an increased likelihood of graduating in STEM compared to students with low-socioeconomic status (SES) in the control group. This finding indicates that low cost or free technology has the potential to expand the face-to-face classroom in meaningful ways, can increase access to faculty, can increase access to peer-support and ultimately enhance 6-year STEM graduation among low SES students.

Introduction

Low-income students and Pell eligible students

Low-income students face overwhelming challenges in entering and succeeding in higher education (Med and Ma, 2016). Social constraints, economic barriers, and cultural differences have prevented many students from pursuing or remaining in higher education (Braxton et al., 2000; Paulsen & St. John, 2002; Walpole, 2003). While support structures such as financial aid are made available to students to support college enrollment, persistence, and achievement, mounting college costs are compounded with immense shifts from grant aid to loans and from need-based aid to merit-based scholarships. These factors have amplified the gaps in college affordability and postsecondary educational achievement among college students (Des Jardins and Chen, 2008, 2010; Chen and St John, 2011; Zheng et al., 2010).

Post-secondary enrollment declined after 2010 after the end of the Great Recession, following a growth period between 2000 - 2009. In both 1970 and 2012, public two- and four-year institutions enrolled 76% of all undergraduate students. In 2012, the high school student graduates who entered a college was 89% for the upper family income quartile (>$108,650), up from 79% in 1970. Among the bottom quartile (<$34,160), the rate was 46% in 1970 and 62% in 2012. The difference in high school graduates’ college enrollment rates between the highest and lowest quartile at 27 percentage points in 2012 was down slightly from 33% in 1970 (Cahalan et al., 2019).

Due to rising educational costs, low income students and families need financial resources to attend and persist in higher education. The U.S. Federal Pell Grant program financially supports students from low-income families and independent students with low incomes (College Board Trends in College Pricing, 2015). Eligibility for Pell Grants is based on family income, family size, number of family members attending college, and other factors (Kerry et al., 2010). In the year 2012–13, 61.2% of the more than 3.78 million Pell Grants were awarded to dependent students with family incomes below $30,000; 76.8% of grants were to those with family incomes below $40,000; and 88.6% to those from families below $50,000 (Snyder, 2015). The lack of financial support is driving more low-income students to attend two-year institutions. The share of Pell Grant recipients enrolling in a four-year institutions declined from 57% to 55%, while the share of non-Pell recipients enrolling in a four-year institutions increased from 71% to 75% (Snyder, 2015).

Importantly, although there has been success in the early stages of the educational pipeline (high school graduation rates and college enrollment rates), growth in college graduation rates has remained stagnant. Those in the lowest income quartile actually declined in college graduation rates between 1970 to 2013 — from 22% to 20% — whereas their peers in the highest income quartile increased from 55% to nearly 100% (Cahalan et al., 2019). This data showcases the apparent link between income and educational attainment that belies the hope for equal opportunity through financial aid and college access in this country.

Although many studies explore the effects of financial aid and identify its positive role in enhancing college access, choice, and future purpose (Des Jardins and Chen, 2008, 2010; Hossler et al., 2009), they differ widely on whether and how precise sorts of aid meet the needs of low-income students. Studies have documented that Pell grants are related to narrowing the dropout gap between students from low- and middle-income groups, although overall the interaction between Pell grant and income was not significant (Knapp et al., 2012). The institutional variables that have larger outcome impacts on graduation rates were associated to Pell recipients, those enrolled in private colleges, and SAT scores (Cahalan et al., 2019). In 2012, students who received Pell Grants were approximately 3.5 times as likely to attend a private institution compared to non-Pell Grant recipients, which is up from two times as likely in 2001 (Cahalan et al., 2019). It was reported that Pell Grant recipients were less likely than their counterparts to attend a four-year rather than a two-year college (Knapp et al., 2012). Moreover, studies have also indicated the increased rate of college enrollment when those in low socio-economic status (SES) were supported by Pell grants (Protopsaltis and Parrott, 2017; Turner, 2016). Socioeconomic status is defined by the APA as “the social standing or class of an individual or group. It is often measured as a combination of education, income and occupation” (American Psychological Association, 2021). Additionally, Hoxby & Bettinger (2013) reported that support from Pell grants promotes retention and persistence to graduation.

These outcomes have significant implications for policy and practice when considering the need more supportive financial aid for college students. As stated, numerous factors contribute to successful outcomes for undergraduate students. The current study aims to specifically understand the impact of student-faculty interaction through a free and widely accessible social media platform on retention and graduation rates among Science, Technology, Engineering and Mathematics (STEM) students who are Pell recipients. An approach grounded in student engagement research guided the social media intervention, and the outcomes related to retention, graduation, and Pell award are discussed.

STEM student success

Nearly 50% of undergraduates and 69% of those pursuing associate degrees in STEM do not persist (Chen and Soldner, 2013). In order to increase degree attainment, the STEM discipline based education research (DBER) faculty, teachers, researchers and staff around the world invests significant time, energy, and funding in creating highly impactful undergraduate teaching practices and, due to this work, science instruction has been experiencing a deep reformation in the past twenty five years (American Association for the Advancement of Science, 2011; Aulck et al., 2017; Brewer and Smith, 2011; Mervis, 2010; Handelsman et al., 2004; Wood and Gentile, 2003).

Student support services and student success

Additionally, Student Success Programs (SSP) are designed to facilitate students' success in college (Chaney, 2010; Jehangir, 2009). Student Success Programs on college campuses facilitate this success through increased student involvement and enhanced academic supports (Cummings, 2014; McCracken, 2004; Wiley, 2005). SSP programs encourage students involvement and integration into university culture. According to the U.S. national performance reports (2004; 2010), disadvantaged college students who participate in SSS programs achieve improved educational outcomes. The impact of the implementation of SSP's has been supported by research (Bettinger et al., 2013; Thayer, 2000). Fike and Fike (2008) included SSP's as impactful in college student retention; other positive predictors were taking internet courses, receiving financial aid, father having some college education, semester credit load, and student age. Further, a sound body of literature has recognized strong associations between student involvement in an array of educationally supportive student activities and encouraging outcomes of student success (e.g., growth, fulfillment, diligence, academic success and social engagement, etc.) (Pascarella and Terenzini, 2005; Trowler, 2010). Additionally, practicums, community service, and projects that integrate knowledge build social capital and yield higher educational success rates (Kuh, 2009a). Thus, universities can and should help students to positively participate in the behavioral, social, cultural, and intellectual facets of their education. Mallette and Cabrera (1991) found a direct relationship between students graduation rates and academic involvement in the university. Chaney, 2010 established that when SSP's are connected to precise student services at the same institution 7% points higher retention and degree completion are reported. The dominant types of involvement are academic involvement, participation by faculty, and by student peer groups (Astin, 1996). For decades, the federal government has directed funding to federal SSPs like TRiO in an effort to increase graduation of first-generation and low SES students (Congressional research service, 2018). Furthermore, researchers found a strong correlation between the academic support services and student learning (Chambers, 2004). Many SSPs seek to provide academic opportunities for disadvantaged students. They provide programming tailored to preparing students for global, interdisciplinary and diverse work and also provide them supports aimed at graduation and workforce transition.

According to Andrepont-Warren (2005), academically disadvantaged students can be just as successful in college as any other student population. Their study confirmed that if students are offered and asked about advising services and other SSP's they can be successful in college. There are at present 947 SSPs across the nation, engaging over 200,000 students (Council for Opportunity in Education, 2013). Students who participate in SSPs are more than three times as likely to earn a bachelor's degree when compared to their counterparts with no participation but the same qualifications (Allen et al., 2013; Jean, 2011). Numerous studies have found a necessity for SSPs that aid first-generation, economically disadvantaged students during their pursuit of a bachelor's degree (Collier and Morgan, 2008; Woosley and Shepler, 2011; Padgett et al., 2012).

Improved student retention rates have also been reported by the use of peer-mentoring programs at colleges and universities (Hall and Jaugietis, 2011; Holt and Fifer, 2018; Ward et al., 2012). The personnel at Curtin University in Australia gathered data from students who were assigned a mentor and concluded that 26 students who considered dropping out of the class referenced their mentors as a vital element in their decision to stay (Wheeler, 2018). Further, a two-year mentoring program study conducted at a University in Canada found an increase in students’ GPAs and a decrease in course failure in their first semester (Salinitri, 2005).

Social media and student success

Research using mobile technology has been shown to enable partnerships between students and instructors (Martin and Chamberlain, 2015). The practice of infusing technology throughout the educational organization has become a vital piece of higher education, with most universities now assimilating this technology into their broader educational and teaching methodologies. Thus, learner engagement has altered dramatically in recent years, with an increase in the number of faculty using technology that offers engaging and varied student practices and communication platforms (Graham and Perin, 2007; Trowler, 2010).

An extensive literature review evidenced that 25% of science faculty utilize social media in instruction (Moran et al., 2012). These instructors typically used blogs, wikis, podcasts, and other social networks like Twitter or LinkedIn in teaching (Seaman and Tinti-Kane, 2013). These findings were affirmed by a study that identified the most commonly-used social media technologies in higher education as blogs, podcasts, social networking, and virtual environments (Liu et al., 2012). Communication technology can comprise earlier tools, such as email, notes in pdf files, or an online discussion forum. It can also include communication on social media like student blogs, wiki projects, or Twitter discussions (Williams and Jacobs, 2004). The objectives of instruction employing social media generally involve improving how to connect and work in groups, enhancing student engagement, and providing a platform for alternative course content delivery (Bawden and Robinson, 2009).

Chickering and Gamson (1987) proposed seven principles of effective undergraduate education as instructional approaches that (a) inspire connections among students and teachers, (b) inspire mutuality and collaboration among students, (c) endorse active learning, (d) offer swift feedback, (e) motivate time on task, (f) communicate high expectations and (g) respect diverse talents and ways of learning. Multiple studies have shown the need to contextualize and customize educational interventions using technology to the student population, institution, and faculty, as when these interventions follow well-researched pedagogical approaches like Chickering and Gamson's, (1987) they can produce strong educational outcomes for students (Heiberger and Harper, 2017).

Instructors in traditional classrooms, however, can find new technologies challenging to implement. Loss of student engagement and retention can occur due to failure to apply well-developed educational principles (Mervis, 2010; Wood and Gentile, 2003). Brewer and Smith (2011) claim that this issue can be addressed by retaining traditional methods of instruction in addition to new media and ensuring that learners are active contributors in all platforms. Further, introducing social media into courses may help make them more engaging and even fun, as well as enhancing students’ communication, collaboration, community, and creativity skills (Hovorka and Rees, 2009). Thus, there is a convincing need to explore educational methods that use new media to address the need for improved student engagement and student success.

Despite the fact that some students favor formal pedagogical course instruction, one learner-centered approach that has been rarely explored by STEM educators is the use of casual shared networking sites (Seaman and Tinti-Kane, 2013). Studies have reported that these platforms have been more utilized in instruction in non-STEM disciplines (Moran et al., 2012; Seaman and Tinti-Kane, 2013). In recent years, new social networking options based educational approaches have been revealed as placing the learner at the center of webs of communication and information. The skills that students develop utilizing these media lead to new forms of learning, regardless of the fact that these media themselves make no scholastic promises (Mao, 2014). Mobile computing strategies can deliver instructive prospects for students to access course content, as well as network with teachers and classmates anywhere they are situated (Cavus and Ibrahim, 2008, 2009; Francisco, 2013; Sarab et al., 2015).

Instructors in K-12 have supported assimilating Facebook, Ning, and additional sites into academia (Davis et al., 2010; Munoz and Towner, 2009). Teachers have endorsed the use of social media as part of a connecting learning theory. Downes (2004) proposed that blogs can engage learners in thoughtful ways that are critical, intrusive, and responsive. Blogging, when used as an educational tool by educators and learners, is shown to be positive in the classroom. Nevertheless, despite its likely benefits, according to a national study by Salaway et al. (2007), 26.1% of undergraduate student learners used blogs for academic purposes. However, this proportion was said to rise to 37.3% in 2009 (Smith et al., 2009).

There is a lack of rigorous research, and specifically a lack of research on social networking site's ability to improve student learning, most research focuses on common uses by faculty. But studies show that microblogging has the potential to encourage participation, engagement and reflective thinking (Gao et al., 2012; Greenhow and Askari, 2017). Manca and Ranieri (2017) indicate there are many opportunities for teacher training & professional development. This includes but is not limited to student-teacher communication; professionalism training and social media pedagogy training. Some researchers call for more understanding and use of social media by academics because it shows clear benefits for career development, research and teaching, but many academics do not use social media in their work (Chugh et al., 2021). Social Media use is limited in higher education due to cultural resistance, institutional constraints and pedagogical issues. Faculty are ambivalent (Manca and Ranieri, 2016). Metanalysis by Tang and Hew (2017) suggest to make social media use mandatory, provide regular academic and personal support, broader access to learning communities, clear expectations, training sessions.

Social media tools can be used to teach students how to better collaborate with others. A study among high school students in the United States inspected how different types of social investment associated with parents, friends, and Facebook were linked to students' self-reliance about their views about being successful at college. A study by Donghee et al. (2013) shows that for students whose parents did not graduate from college, social media use plays a noteworthy role in increasing student performance in college. Further access through social media to a larger network of people who could aggressively distribute informational support was positively associated to these students’ ability to succeed in college (Wohn et al., 2013). STEM faculty have often encouraged using technology for improved student engagement (Tsaushu et al., 2012; Walsh et al., 2011). These studies documented significant enhanced student performance for STEM students who are underrepresented in the field.

The study author's use of Twitter in the Biology Majors First Year Seminar may differ on the face from the trends of faculty use of social media in teaching, but there was a convincing rationale to do so. The validation for the methods to use this strategy included meeting all seven principles as outlined by Chickering and Gamson (1987) and Pai et al. (2017). Literature-supported principles suggest that Facebook and other social media platforms are encouraging avenues for science education innovation (Czerkawski, 2011; Downes, 2005, 2012; Gao et al., 2012; Junco, 2012; Junco et al., 2011). The faculty leading this project implemented this approach with the short-term intent of creating strategies that extend engagement with faculty and with academic content beyond the classroom walls. Additionally, the research team analyzed the long-term effects of this intervention on first-to second-year retention and on STEM graduation rates.

Previous analysis of the specific intervention in this paper indicated the social media communications used for class engagement provided a highly engaged learning process between instructors and students (Junco et al., 2011). Additionally, this study affirmed faculty participation on the platform, integration of Twitter into the course based on a theoretically driven pedagogical model, and requiring students to use Twitter as essential components of improved learning outcomes (Junco et al. 2013). This evidence provides additional rationale for our use of Twitter as an educational tool to help engage students and to mobilize faculty into a more active and participatory role.

The current state of higher education outcomes and interventions will likely not enable the creation of additional expensive SSP's, the broadening of or significantly increased funding of the Pell program, nor will dramatic shifts in faculty-student engagement occur due to the resource scarcity felt across higher education. However, the social media intervention described here has potential to be replicated across courses, institutions, and the full higher education landscape due to the simplicity of its use for faculty and students and the ability for the intervention to impact student engagement beyond of the classroom with limited additional investments of time, training and funding by faculty, administrators, and students. As fiscal resources are unlikely to grow in meaningful ways in the future for underrepresented and economically disadvantaged students the intervention below is important because it impacts educational outcomes for all students but importantly does so at higher rates for Pell-eligible students.

Theoretical background, study purpose and research questions

The theoretical background for the interventions in Junco et al. (2011) focused on comparing educational activities delivered under Chickering and Gamson (1987) seven principles of undergraduate education through two social media platforms and the differences in their ability to produce favorable educational outcomes like grades and student engagement.

The analysis in Junco et al. (2011) yielded increased engagement scores and first semester GPA for students in the intervention. No study has been conducted on the longitudinal effects of this intervention, or the differential effects on students. Therefore, the current study serves to extend the previous research by analyzing the long-term effects of the intervention. Specifically, due to the increased engagement scores and GPA of the twitter group, this study focuses on long-term student success outcomes. Student engagement scores and GPA are predictive or highly correlated with graduation rates (Kuh, 2009). Those relationships have also been found to be confounded by other variables including ACT sub-scores and gender, so interaction effects between these variables were analyzed. Additionally, low-income students (i.e., Pell grant recipients) graduate college at lower rates than their peers but have been shown to be more strongly impacted by interventions to support student success, so the effect of this intervention on Pell grant recipients will also be conducted.

The research questions investigated were:

2.1 Do the first-to second-year retention and six-year graduation rates differ between participants in the intervention and control groups?

2.2 Do interaction effects exist between the intervention and 1.) undergraduate GPA, 2.) ACT science sub-scores, and 3.) gender?

2.3 Do interaction effects exist between the intervention and PELL recipient status?

Materials & methods

Sample (with permission from Junco et al., 2011)

Seven sections of a one-credit first-year seminar course for pre-health professional majors (students planning to apply to dental, chiropractic, medical, physical therapy, etc. schools) were used for the study. Random section assignment allocated four of the sections into the experimental group and three sections into the control group. Students were asked to participate in the study by completing a pre- and post-test engagement instrument derived from the National Survey for Student Engagement (NSSE). Although participation was voluntary, participants could enter to win drawings of cash deposits to their University flex accounts throughout the semester. The drawings were announced via Twitter for the experimental group and via Ning for the control group. Of the 132 students in the seven sections, 125 took the pre-test survey for an overall 95% participation rate. In the experimental group, 70 out of 74 (95%) students participated, while 55 out of 58 (95%) participated in the control group. There was no significant difference between groups in participation rate. Sixty percent of those who took the pre-test were female and 40% were male. The mean age of our sample was 18.2, with a standard deviation of 0.445. The age of our participants ranged from 17 to 20, although over 98% were between 18 and 19 years old. Twenty-eight percent of the sample reported that neither parent had a college degree. Our sample was 91% Caucasian, 6% Latino, 3% Native American, and 1% Asian American. We had no African Americans in our sample. The race and ethnic breakdown of our sample was similar to that of the overall University population, with the exception of a slight overrepresentation of Latinos and a slight underrepresentation of Asian Americans in our sample. The study ran for 14 weeks.

Rationale for utilizing ning and twitter in this study

Ning (https://www.ning.com) and Twitter (https://twitter.com) were chosen as platforms for out of class communication between students, and between faculty and students. Ning was chosen due to the similar features with commonly used social media platforms during the study period (i.e., Facebook, Myspace, etc.). Additionally, Ning offered features to familiar to the students including public discussion boards which were in high use directly prior to the study period. Finally, the functionality of Ning were similar to the capabilities of collegiate learning management systems, specifically: direct messaging, group creation, discussion board posting and search capabilities. The Internet Archive (2021) provides a good example of the technological features of this site during the study period.

Twitter was chosen due to its unique ability to be accessible across devices for the students and faculty (e.g., smartphones, desktops, laptops and importantly via text message for those with limited access to hardware and devices). This newly emerging, accessible, and less robust platform was untested in the educational space at the time. Twitter provided a unique opportunity to test the efficacy of more simplistic but much more accessible social media platform for educational purposes outside the classroom walls (Kan, 2020).

Twitter procedure (with permission from Junco et al., 2011)

The experimental group used Twitter as part of the class while the control group did not. None of the students used Twitter before participating in this study. Both groups used Ning (http://www.ning.com; a service that allows users to create their own social networking site) instead of a learning management system as a regular part of the course. During the second week of the semester, the sections in the experimental group received an hour-long training on how to use Twitter. This training was supplemented by question-and-answer periods over the next few class meetings. Students were taught the basics of Twitter [e.g., how to sign up for an account, how to send tweets (Twitter messages), how to use hashtags (clickable keywords within tweets) and @ replies (replies to other users)] and were shown how to enable privacy settings. All students were asked to send an introductory tweet during the training session. Students from experimental group sections were asked to follow a single Twitter account created for this study as well as follow each other so that they could interact across sections. Immediately after the Twitter training sessions, both the experimental and control groups were sent links to the online engagement instrument. The posttest instrument was sent during the last week of the study, which ran for 14 weeks. The Twitter class account was administered by two of the authors. Based on previous research on engagement ((Chickering and Ehrmann, 1996); Pascarella and Terenzini 2005; Kuh 2009), engagement in social media (HERI, 2007; Heiberger and Harper 2008), and case studies of Twitter use, we used Twitter for the following educationally relevant activities:

Continuity for class discussions: Because the first-year seminar met only once a week for an hour, Twitter was used to continue conversations begun in class. For instance, students were asked to discuss the role of altruism in the helping professions.

Giving students a low-stress way to ask questions: Oftentimes, first-year and/or introverted students are less comfortable asking questions in class. The dynamics of Twitter allow students to feel more comfortable asking questions given the psychological barriers inherent in online communication (Kruger et al., 2005)

Book discussion: All first-year students read the same book as part of their first-year reading program. The book, Mountains Beyond Mountains (Kidder, 2004), focuses on Dr. Paul Farmer's medical relief work in Haiti and was used to stimulate discussion about altruism and the helping professions.

Class reminders: As all students took a similar sequence of courses, we were able to remind them of due dates for assignments and dates for exams in multiple classes via one Twitter feed.

Campus event reminders: At the beginning of the semester, we used SocialOomph (formerly Tweetlater) to schedule tweet reminders for the entire semester. These reminders included campus events, speakers, concerts, and volunteer opportunities.

Providing academic and personal support: We regularly posted information about academic enrichment opportunities on campus (e.g., the location and hours for the tutoring center), both periodically and in response to student requests for help. Additionally, we provided encouragement and support when students reported things such as feeling “stressed out” or worried about exams.

Helping students connect with each other and with instructors: The “cohort effect” or the intentional creation of learning communities is an important concept in ensuring student persistence (Keup, 2005–2006). Additionally, student/faculty interaction is an National Survey for Student Engagement (NSSE) factor shown to be related to student success (Kuh, 2002).

Organizing service-learning projects: As part of this course, students needed to participate in a service-learning volunteer opportunity. Students used the Twitter feed to coordinate volunteer times with each other.

Organizing study groups: With only a little encouragement from the authors via the Twitter feed, students organized study groups for two of their more difficult courses, Chemistry and Biology.

Instruments and measures

Data were collected through the University's Institutional Research Office under IRB-0908007-EXM. There were 123 students in the study; their average high school GPA was 3.47; the average ACT was 24.14, with an average ACT Science Reasoning at 24.26; 28% received the Pell grant during the semester of the study, and the percentage of Pell recipients at the University was 24% (US News and World Report, 2018). The first-to second-year University retention rate of the students in the study was 80%. By comparison, the overall retention rate for the University was 76.5% in 2009–2010. The six-year graduation rate of those in the study was 62%. The University's graduation rate for the 2009 First Time Full Time was 54.5%. And the six-year graduation rate in a STEM major was 41% for participants in the study.

STEM majors for this study include NSF and IPEDS defined CIP codes for STEM. In addition, health professions and related programs (as defined by IPEDS) are included in the STEM definition for this study. The rationale for this is that the workforce needs in the state and region include Pharmacy, Nursing, and Exercise Science majors offered at this institution.

Results

Research Question 1: Do the first-to second-year retention and six-year graduation rates differ between participants in the intervention and control groups?

The mean rates of college retention and graduation among the experimental (Twitter) group are significantly higher than those in the control group. Applying Fisher's exact test with ∝<.05, the retention and graduation rates yield statistically significant results between control and experimental groups in first-to second-year retention (p = .0407), six-year graduation rates (p = .0086), and six-year graduation rates in STEM (p = .0414). See Tables 1, 2, and 3.

Table 1

First-to-second year retention.

	Retained	Not retained	% Retained
Experimental (Twitter) group	60	9	87.0
Control Group	38	16	70.4
Total (n = 123)	98	25	79.7

Table 2

6-year graduation.

	Graduate	Not graduate	% graduate
Experimental (Twitter) group	50	19	72.5
Control Group	26	28	48.1
Total (n = 123)	76	47	61.8

Table 3

6-year graduation in STEM.

	Graduate	Not graduate	% graduate
Experimental (Twitter) group	34	35	49.3
Control Group	16	38	29.6
Total (n = 123)	50	73	40.7

Research Question 2: Do interaction effects exist between the intervention and 1.) undergraduate GPA, 2.) ACT science sub-scores, and 3.) gender?

Logistic regression

Tables 4, 5, and 6 show logistic regression models with no statistically significant interactions between GPA and intervention (Twitter), ACT science sub-scores and intervention (Twitter), gender and intervention (Twitter).

Table 4

Logistic model with interaction between GPA and treatment.

	Graduate	Graduate STEM	Retention
ACT Composite score	1.19	0.87	1.11
ACT Science Reasoning sub-score	1.04	1.22	0.99
Pell Grant Recipients	0.95	1.88	1.62
Female	1.07	1.69	0.60
GPA	18.20∗∗	4.26	3.75
Group = Intervention/Control = 1	841.65	1.50	21.61
Group = Intervention/Control = 1#GPA	0.17	1.12	0.49
Observations	121	121	121
Pseudo R-squared	0.27	0.16	0.12

Exponentiated coefficients.

∗p < 0.05, ∗∗p < 0.01,∗∗∗p < 0.001.

Table 5

Logistic model with interaction between ACT science sub-score and treatment.

	Graduate	Graduate STEM	Retention
Pell Grant Recipients	1.05	1.87	1.71
Female	1.03	1.69	0.55
GPA	5.29∗	4.55	2.55
ACT composite score	1.23	0.87	1.15
ACT science reasoning sub-score	0.93	1.21	0.91
Group = Intervention/Control = 1	0.02	1.81	0.09
Group = Intervention/Control = 1#ACT science	1.21	1.01	1.15
Observations	121	121	121
Pseudo R-squared	0.26	0.16	0.13

Exponentiated coefficients

∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001

Table 6

Logistic model with interaction between gender and treatment.

	Graduate	Graduate STEM	Retention
GPA	5.65∗∗	1.62	2.61
ACT Composite score	1.21	0.87	1.13
ACT Science Reasoning sub-score	1.03	1.22	0.98
Pell Grant Recipients	1.04	1.86	1.73
Female	1.86	1.42	0.66
Group = Intervention/Control = 1	3.58	1.90	2.33
Group = Intervention/Control = 1#gender	0.37	1.32	0.82
Observations	121	121	121
Pseudo R-squared	0.26	0.16	0.12

Exponentiated coefficients.

∗p < 0.05, ∗∗p < 0.01,∗∗∗p < 0.001.

Research Question 3: Do interaction effects exist between the intervention and PELL recipient status?

Table 7 indicates a statistically significant interaction effect between Pell grant recipients and intervention (Twitter). An odds ratio of 11.64, p < .05. Pell grant recipients in the Twitter intervention group recipients had a 11.64 times likelihood of graduating in STEM compared to Pell grant recipients in the control group.

Table 7

Logistic models with interactions between PELL and treatment.

	Graduate	Graduate STEM	Retention
GPA	5.93∗∗	5.71∗	2.85
ACT Composite score	1.20	0.85	1.12
ACT Science Reasoning sub-score	1.02	1.24	0.98
Pell Grant Recipients	1.09	1.71	0.59
Female	0.79	0.59	1.30
Group = Intervention/Control = 1	1.70	1.16	1.75
Group = Intervention/Control = 1#Pell Grant	1.93	11.64∗	0.35
Observations	121	121	121
Pseudo R-squared	0.26	0.19	0.12

Exponentiated coefficients.

∗p < 0.05, ∗∗p < 0.01,∗∗∗p < 0.001.

Specifically, Table 8 shows 31% of students who did not receive Pell grants in the control group graduated in STEM in six years compared to 42% of those students in the Twitter group. By contrast, 28% of Pell recipients in the control group graduated in STEM in 6-years, and 75% of Pell recipients in the Twitter group graduated in STEM in 6-years. The Twitter intervention increased 6-year STEM graduation by 11% for the Non-Pell grant recipients, but by 47% for Pell Grant recipients.

Table 8

6-year graduation in STEM.

	Graduation Rate	Graduation Rate (Non-Pell)	Graduation Rate (Pell)
Experimental (Twitter) group	49.3%	42%	75%
Control Group	29.6%	31%	28%
Total (n = 123)	40.7%

Discussion

This study provides a longitudinal analysis of previous work (Junco et al., 2011) on the impacts of applying Chickering and Gamson's (1987) seven principles for good practice in education to Twitter. The initial analysis of this experiment showed that the experimental group had a significantly greater increase in engagement than the control group, as well as higher semester grade point averages. Analyses of Twitter communications showed that students and faculty were both highly engaged in the learning process in ways that transcended traditional classroom activities (Junco et al. (2011).

Research Question 1: Do the first-to second-year retention and six-year graduation rates differ between participants in the intervention and control groups?

First semester engagement and GPA have been shown to be highly correlated with first-to second-year retention and 6-year graduation rates. The increase in retention rates and graduation rates within the experimental group is likely a long-term outcome of increased engagement and GPA, an outcome of the support, engagement and learning facilitated through Twitter with faculty and peers. Connection to fellow students, STEM faculty, and increased engagement with these groups can be attributed to the twitter intervention, and these can be extrapolated to increased retention and thus persistence to graduation within STEM fields.

Research Question 2: Do interaction effects exist between the intervention and 1.) undergraduate GPA, 2.) ACT science sub-scores, and 3.) gender?

Although GPA, ACT and gender do not have interaction effects with the experimental design, student GPA alone does show significant correlation with graduation which is a logical conclusion. Students who do not obtain minimum grades to progress in STEM courses in addition to institutional minimum GPA for progression and continuing enrollment. These students are forcibly screened out of the STEM pathway either through course or institutional level policy.

Research Question 3: Do interaction effects exist between the intervention and PELL recipient status?

This study shows that in addition to achieving GPA increases and higher levels of engagement in the first year (Junco et al., 2011) using Twitter to apply Chickering and Gamson's (1987) practices may prove effective at supporting low-income students toward the goal of STEM degree attainment. The pathway towards successful completion of STEM degrees includes numerous barriers, including but not limited to motivation, successful completion of gateway coursework, peer support and modeling, all of which low-SES students are less likely to have within their background, experiences and support networks.

Students from the lowest income quartile in the United States are experiencing an increasing high school graduation rate and increasing college going rate, but decreasing college graduation rate (Cahalan et al., 2019). Additionally, the likelihood that federal, state, and local governments will infuse additional resources into proven programs like SSS and Pell grants seems unlikely. Using Twitter in educationally relevant ways through this intervention had strong effects on STEM graduation rates and that impact was greater with Pell grant recipients. Low or no-cost interventions, such as this specific model using Twitter, can be effective strategies for increasing success in STEM fields, especially of low-income students.

Limitations

Previous research, educational theory, and the data analysis from this study support the model that the Twitter intervention led to higher grades and increased student engagement (Junco et al., 2011), and this impact may have led to major retention. The increased grades and engagement, and major retention may have led to the increased first-to second-year retention rate and those two outcomes combined (major retention and year over year retention) could also then positively impact STEM graduation rates six years later for students in the intervention group in this study. Additional randomized, controlled, and longitudinal studies should be conducted prior to extrapolation of these findings to a larger audience. There are a number of limitations to the current study. It was conducted with a small sample from a University that is not representative of the population of higher education students in the United States or internationally (with respect to student demographic characteristics). Although the intervention was randomly assigned, the sample size is small. There are also limits to the study design, sample size, and intervention in confirming the long-term outcomes discussed here from an intense and unique but relatively short 14-week experience.

Conclusion

This study provides the first evidence that applying Chickering and Gamson's (1987) principles to Twitter may have long-term effects on student success, and importantly that the impact is greatest with students who are low income (as indicated by Pell grant eligibility). We hope that this study motivates further longitudinal student outcome analysis of previous controlled studies with technology in higher education and the impact on graduation rates in STEM for students with low socio-economic status.

Declarations

Author contribution statement

Greg Heiberger: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Reynol Junco: Conceived and designed the experiments; Performed the experiments.

Sumadhuri Pamarthi: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

The data that has been used is confidential.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.