Undergraduate R Programming Anxiety in Ecology: Persistent Gender Gaps and Coping Strategies.

The ability to program in R, an open-source statistical program, is increasingly valued across job markets, including ecology. The benefits of teaching R to undergraduates are abundant, but learning to code in R may induce anxiety for students, potentially leading to negative learning outcomes and disengagement. Anecdotes suggest a gender differential in programming anxiety, with women experiencing greater anxiety. Currently, we do not know the extent to which programming anxiety exists in our undergraduate biology classrooms, whether it differs by gender, and what instructors can do to alleviate it. Instructor immediacy has been shown to mediate related anxieties such as quantitative and computer anxiety. Likewise, students' use of adaptive coping strategies may mitigate anxieties. We investigated students' R anxiety within a lower-division ecology course and explored its relationships with gender, instructor immediacy, classroom engagement, and reported coping strategies. Women reported significantly higher R anxiety than men, a gap that narrowed, yet persisted over the semester. In addition, several specific coping skills were associated with decreases in R anxiety and increases in self-concept and sense of control; these differed by gender identity. Our findings can guide future work to identify interventions that lessen programming anxiety in biology classes, especially for women.

INTRODUCTION

Rapid increases in technology, data accumulation, and data science have led to a world that is steeped in information ready to be accessed and employed (Friedman, 2017). Indeed, no other time in history has been marked by such a sheer amount of available information. Yet, because of this acceleration in technology and data collection, organizing, accessing, and analyzing information has increasingly become a challenge. Computer programming skills are essential to tackle this challenge and are increasingly recognized as a valuable skill set for undergraduate students (Barone ; Wilson-Sayres ; Wright ). Programming skills can lead to more job opportunities (seven million openings called for programming in 2015) and higher-paying jobs ($22,000/year more, on average, just within career-track jobs) and are useful across a variety of job categories (Burning Glass Technologies, 2016). By introducing a programming language in undergraduate courses, identifying the barriers that inhibit the development of programming skills, and designing instructional elements that improve student learning and engagement in programming, universities can help their graduates access these more abundant and higher-paying jobs and can contribute to national and global efforts to better use the information currently available.

One program used frequently in the field of ecology, and thus likely being introduced more regularly into ecology classrooms, is the statistical and graphical program R (Auker and Barthelmess, 2020; also see Touchon and McCoy, 2016). R is open source, robust, and adaptable; it can be used for a range of purposes, including linguistic analysis of text files, complex statistical analyses, and imputation of missing data for data sets with thousands of entries (e.g., with packages in R such as Quanteda and MICE). R programmers across the world contribute to and iteratively improve R on a daily basis. Thus, it is both broadly used and useful across fields, including biology and industry. For example, in the journal Ecology, peer-reviewed papers that used R for statistical analysis increased from 14.3% in 2008 to 82% in 2018, demonstrating its increasingly widespread use in the field (Auker and Barthelmess, 2020; also see Touchon and McCoy, 2016). Because it is open source, there are fewer financial barriers to access (computer and Internet access are still required), contributing to its potential for aiding students long after college and reducing barriers for low socioeconomic status individuals.

While the benefits of learning to program, and specifically learning to program in R, are clear, anxiety experienced while learning R may present an obstacle for many students (Connolly ; Nolan and Bergin, 2016). In academic contexts, anxiety is often associated with the emphasis on right answers and fear of failing, not understanding, and not being capable of executing an expected task, all of which can threaten students’ sense of self-worth and competence (McInerney, 1997; Harper and Daane, 1998; Cooper ). Programming-specific anxiety is “a psychological state engendered when a student experiences or expects to lose self-esteem in confronting a computer programming situation” (Connolly , p. 3). Students can experience and develop anxiety in programming contexts because they often arrive at college unprepared in programming skills, a situation that is then exacerbated by frequent negative feedback from programming software (e.g., frequent error messages when learning how to code). Programming anxiety in undergraduate biology courses can also relate to quantitative or math anxiety and computer anxiety. Quantitative or math anxiety, described as “feelings of tension and anxiety that interfere with the manipulation of numbers and solving mathematical problems” (Kelly , p. 173), can arise and overlap with programming anxiety when students use a programming language to conduct statistical analyses, such as in R. Mathematic abilities have also been shown to predict programming skills (Owolabi , p. 715). Additionally, computer anxiety, or the “generalized emotion of uneasiness, apprehension, anxiousness of coping, or distress in anticipation of negative outcomes from computer-related operations” (Chang, 2005), is similarly related to programming anxiety due to computing-specific challenges that students can encounter. Thus, in working across these constructs, we define R programming anxiety as “feelings of uneasiness, apprehension, or distress in anticipation of negative outcomes from programming within R; such feelings may be associated with anticipation of programming, statistical, or computer-related difficulties.” Conversely, a positive sense of control and self-concept, two constructs related to and influential for anxiety, can help combat anxiety and are critical indicators when considering ways to decrease anxiety (McInery, 1997). Sense of control relates to students’ sense that they have control over their own actions when engaging with R, and self-concept measures students’ overall confidence and beliefs about themselves in the context of a given R task. Students’ task-specific sense of control and self-concept are inversely correlated with the anxiety they experience when engaging with that task; positive increases in these two constructs can help reduce learners’ fear of task failure and thus their anxiety (McInerney, 1997).

While low levels of anxiety can sometimes motivate students who are interested in the topic at hand (Pekrun ), moderate to high anxiety typically demotivates students and has a negative effect on student learning (Yerkes and Dodson, 1908; Seipp, 1991; Akgun and Ciarrochi, 2010; England ), especially when the task at hand, such as learning to use R, is cognitively difficult (Teigen, 1994). Indeed, anxiety in science and biology classrooms is well documented and may be one factor that contributes to low rates of student retention in science, technology, engineering, and mathematics (STEM) fields, exacerbated by low student preparedness for quantitative course work upon entry to college (Kelly ; England et al., 2017, England ; Cooper ; Cooper and Brownell, 2020). Further, programming anxiety may be a particularly salient issue in biology classrooms, as degree programs often vary widely in their statistics pedagogy, and many students view programming as a means to study biology rather than an intrinsically motivating activity in and of itself (Metz, 2008). Finally, the way in which R coding is taught may have an effect on student programming anxiety. There are increasingly calls for biology classrooms to move to active styles of instruction (e.g., American Association for the Advancement of Science [AAAS], 2011; Theobald ), and while these offer clear benefits in terms of cognitive gains (Freeman ; Theobald ), they also can result in increased student anxiety while learning via fear of negative evaluation by one’s instructors or classmates (Cooper ; Downing ). Despite its likely importance and demonstrably increasing role in students’ biology education experience, most research relating to programming anxiety 1) is focused on computer anxiety and not specific to programming; 2) was published more than 10 years ago, at a time when programming was less ubiquitous; or 3) if published within the last 10 years, was carried out within the context of computer science classrooms (Chua ; Connolly ). This research leaves a large gap in our understanding of how undergraduates learn and build programming efficacy in modern contexts and in biology classrooms.

Although there is a paucity of formal studies on biology undergraduates’ programming anxiety, with related research showing mixed results and maintaining the same limitations of era and scope described (Chua ; Stoilescu and McDougall, 2011), instructors often cite anecdotal evidence that suggests trends in students’ experience. Notably, it is generally assumed that women experience higher degrees of programming anxiety in the biology classroom. Although biology majors are typically 60% female (Eddy ; Wright ), there is evidence that women are underrepresented in computational biology (Bonham and Stefan, 2017) and that, within the field of biology, women underperform on exams (while controlling for grade point average) and participate less in class (Eddy ; Wright ). The fact that gender gaps remain a factor in biology student success and representation in computational biology necessitates further investigations into mechanisms underlying these gaps and requires that we periodically test our assumptions about how gender affects different facets of the student experience (Eddy ; Flanagan and Einarson, 2017). Indeed, the relationship between gender and programming anxiety is likely to change over time as societal contexts shift, computers become ubiquitous, computer skills gain importance, and more diverse identities are represented in science and technology (Powell, 2013). Furthermore, it is also important to recognize that there is a paucity of research on whether such gaps exist for the populations of biology students who identify as nonbinary, gender fluid, transgender, or elsewhere on the gender spectrum—identities that are not well-represented or well-characterized within the field. As biology programs and training incorporate more programming into their curricula, we posit that gender-mediated differences in programming anxiety are an important consideration when attempting to support the achievement and participation of students of all genders in our classrooms.

Understanding how our teaching techniques influence student anxiety when learning quantitative skills is critical to students’ skill development and their self-efficacy. It has been posited that instructors may be able to mitigate student anxiety in several ways, such as by introducing group work (e.g., Cooper ) or validating students’ thinking (e.g., Downing ). One factor that has been shown to decrease quantitative anxiety specifically is instructor immediacy, or the perceived social and/or physical distance between the student and instructor (Witt ; Williams, 2010; Kelly ). Instructor immediacy is measured as verbal and nonverbal behaviors exhibited by the instructor that have been correlated with a decrease in perceived social distance, having strong positive impacts on student learning (Gorham, 1988; McCroskey ; Witt ; Chesebro and McCroskey, 2001). In addition to decreases in student anxiety, instructor immediacy leads to increased student engagement in the classroom, which may serve to combat student disengagement in the face of tasks that induce anxiety (Roberts and Friedman, 2013). This may be especially important for programming anxiety, as computer anxiety is correlated with computer avoidance (Chua ). While instructor immediacy has been shown to decrease student math and quantitative reasoning anxiety (Witt ; Williams, 2010; Kelly ), and increase student engagement, we have encountered few, if any, studies examining how instructor immediacy impacts student anxiety or engagement while learning to program.

In addition to instructor behaviors, specific coping strategies students use when confronting challenging tasks, such as programming in R, likely influence the degree of anxiety they experience. Coping, or an individual’s behavioral response to a stressor, is described by many researchers as context specific, with the individual context influencing the coping behavior (Lazarus, 1993). Coping responses are also malleable, meaning that they can be influenced, learned, and changed (Lazarus, 1993), but they tend to become more stable over time with repeated use (Spencer ). Coping strategies can either be maladaptive (leading to negative well-being and negative outcomes) or adaptive (leading to positive well-being and productive outcomes; Skinner ; Henry ). Thus, due to their malleability and positive outcomes associated with adaptive coping strategies, students’ coping responses to R-induced stressors may be fruitful targets for instructional interventions. In this study, we examine both instructor immediacy and student coping behaviors as they relate to students’ reported R anxiety, R self-concept, and R sense of control over the course of the semester.

Research Questions, Aims, and Predictions

Here, we investigated whether Q1) gender (being a man or woman) is related to R programming anxiety, Q2) instructor immediacy is related to changes in R programming anxiety, Q3) R programming anxiety or instructor immediacy are related to student engagement when learning to code in R, and Q4) coping strategies employed when experiencing R coding challenges relate to changes in reported anxiety metrics. Together, these questions build a framework for understanding whether students who identify with specific genders are more at risk of experiencing programming anxiety, and whether instructors can alleviate student anxiety and increase student engagement in their classrooms by modifying their interpersonal relationships with students. Further, by identifying the coping skills that students do and do not use, we can work to improve instructional materials in ways that facilitate student persistence in challenging or anxiety-producing tasks. Notably, Q4 arose as a later addition to the study as a result of early findings that led us to seek additional information to explain observations in anxiety metrics.

We explored this framework in an undergraduate ecology lab class at a large research university. This class affords most of these students their first opportunity to write code for an independent project. We specifically addressed student anxiety while learning to code in the statistical program R, due to its increasing prevalence across STEM and its widespread use in our classrooms. We predicted that: P1) women would report higher programming anxiety and lower sense of control and self-concept in R (hereafter referred to as “R anxiety”) than men, but that anxiety for all students would decrease and their sense of control and self-concept would increase over the semester due to more R exposure; P2) students with more immediate instructors would report greater decreases in R anxiety over the semester and greater increases in R sense of control and self-concept; P3) students with higher R anxiety would engage less in the classroom, but students with more immediate instructors would engage more; and P4) students who use adaptive coping strategies would show decreases in R anxiety over the semester and increases in R sense of control and self-concept.

METHODS

University and Course Context

We conducted this study in the Principles of Ecology course, at the University of Colorado Boulder (CU-Boulder), a large research university (>34,000 students). This course enrolls between 90 and 150 students per section and is a required course (typically taken in the second year) for undergraduates seeking a degree in ecology and evolutionary biology. Two lecture sections of the course are taught in the Fall by two different faculty instructors, with one lecture section in the Spring. Students also enroll in laboratory sections of the course, which are capped at 14 students per section and taught by graduate teaching assistants (TAs). Beginning at ∼5 weeks into the semester and for the remainder of the course, every student works as part of a small lab group within the laboratory section to conduct an independent research project as part of the lab requirements. This includes every step of the scientific process, including statistical analysis, which students must do in the program R. Most students have not had any experience with R before entry to this course, and even if they have, it is many students’ first opportunity to write their own code. Students are taught basic programming in R as part of the lab curriculum during two 2–3 hour lessons. TAs in the Ecology and Evolutionary Biology department at CU-Boulder (around seven per semester, each teaching two lab sections) are the sole instructors of the labs and have almost complete control of how content is taught (i.e., they choose among active-learning techniques and lecture techniques to deliver the same content across labs). All TAs are experienced in using R for statistical analysis and have ample access to the R code they need to know to teach the class before facilitating the R lessons. Overall lesson length varies slightly due to TA instructional style and students’ pace. In this study, all TAs used workshop-style active-learning strategies while teaching R, leading students through a script and checking in with their students frequently, and based on our observations, only small variations in teaching strategy and lesson length existed. Though teaching strategies were similar, active learning presented opportunities for potential differences in instructor immediacy, because TAs could control the pace and style of content presentation. Students are also exposed to R code during the lecture sections but are not asked to write their own code for lecture activities. Thus, the majority of their R learning occurs during the laboratory sections. This research was reviewed and approved by the University of Colorado Boulder’s Institutional Review Board (IRB no. 18-0471).

Study Design

We employed a pre–post study design to examine changes in programming anxiety over the semester as related to our factors of interest (gender, engagement, etc.). At the beginning and end of each semester for three semesters, we deployed a survey to students we recruited from the Principles of Ecology course, with a total of 376 students taking part in our presemester survey and 362 taking part in our postsemester survey across semesters (credit for completion was given). Of the students invited to participate in the study, 63% participated in both pre- and post-semester surveys, and 215 students completed both pre- and post-semester surveys (we only analyzed data from full responses [= completed surveys?]). In all three semesters, students were surveyed on: demographics (pre- and postsemester survey, see Table 1 for demographic summary), R anxiety (pre- and postsemester survey), and instructor immediacy (postsemester survey only). In the third semester of this study, we added a coping skills scale to the postsemester survey and observations of student engagement in the lab sections of this course.

TABLE 1.

Self-reported demographic breakdown of students (by semester) who participated in both pre- and post-semester surveys and whose data were analyzeda

Demographic	Semester 1	Semester 2	Semester 3	Total	Department total (Spring 2019)b
Genderc
Woman	47 (61%)	18 (69%)	63 (68%)	128 (59%)	486 (60%)
Man	28 (36%)	7 (27%)	29 (32%)	84 (39%)	324 (40%)
Nonbinary	2 (3%)	1 (4%)	0 (0%)	3 (1%)	—
Race
Asian	3 (4%)	2 (8%)	5 (5%)	10 (5%)	59 (7%)
Black	4 (5%)	0	0	4 (2%)	21 (3%)
Latino	2 (3%)	1 (4%)	8 (9%)	11 (5%)	95 (12%)
Mixed	2 (3%)	0	2 (2%)	4 (2%)	—
White	64 (83%)	22 (85%)	77 (84%)	163 (76%)	585 (72%)
Additional informationd
First-generation	—	2 (8%)	16 (17%)	18 (8%)	136 (19%)
Nontraditional	—	4 (15%)	12 (13%)	16 (7%)	—
Learning disability, not registered	—	0	4 (4%)	4 (2%)	—
Learning disability, registered	—	1 (4%)	4 (4%)	5 (2%)	—

aData not available or collected are denoted by dashes. We did not include metrics under “additional information” in analyses due to small sample sizes.

bThe last column reflects 2015 demographic data (where available) for the CU-Boulder Department of Ecology and Evolutionary Biology, wherein this study took place. Importantly, students were not required to answer demographic questions.

cResponses to an open-ended question asking students to report on their gender included woman, man, female, male, and nonbinary. The responses “female” and “male” were interpreted respectively as “woman” and “man” to align with the construct of gender.

dDemographic data that were only collected in semesters 2 and 3 of this study, which is why there are no data present for semester 1 (denoted by a dash).

R Anxiety Measure

While there is no developed R-specific anxiety measure to our knowledge, much work has been done to understand the factors that contribute to related anxieties, such as math, programming, and computer anxiety (Heinssen ; Connolly ). As explained in the Introduction, these anxieties are likely to intersect when students are engaged in using R. Thus, for this study, we drew upon prior work in these areas and lightly edited the programming anxiety measure developed by Connolly and colleagues (2008) to focus on programming in R through simple language changes (e.g., “learning computer terminology” was changed to “learning R terminology”). We employed the lightly edited survey in the pre and post surveys at the start and end of the semester. Connolly and colleagues originally adapted the Computer Anxiety and Learning Measure (McInerney, 1997) for their research (they changed “computer” to “computer programming” for each item). This measure was originally written and validated by McInerney (1997) in a large lower-division undergraduate population, similar to the students in this study. There are four constructs that this measures in regard to computing situations: 1) gaining initial skills, 2) sense of control, 3) self-concept, and 4) state of anxiety. In testing the dimensionality of this measure, McInerney (1997) found that these four constructs were stable over multiple samples and showed the following structures. The “gaining initial skills” construct was explained by a model with a single higher-order factor (gaining initial computing skills) and four factors beneath it. The “sense of control” construct was explained by a two-factor model with one substantive factor and one methodological “artefactor,” which stands for artificial factor; in other words, the two factors both measure “sense of control” and arise due to the fact that some items were positively worded and some were negatively worded. The “self-concept” construct was similarly explained by a substantive factor and a methodological artefactor arising from positively and negatively worded questions (artefactors can commonly arise from method effects associated with negatively worded items; e.g., Tomas ). Finally, the “state of anxiety” construct was explained by a model with a single higher-order factor (state of anxiety) and four factors beneath it. These structures gave us confidence in using these scales to measure the four constructs. Because we were interested in reporting on and analyzing the four overarching constructs, we averaged across items and reported overarching results for the four main constructs. Negatively valenced items were reflected so that all items reflect increases in the given construct (see Supplemental Material). Thus, our use of this instrument results in measurement of four constructs of R anxiety with Likert scale responses, for which we calculated internal reliabilities for our population; “gaining initial skills in R” (α = 0.95), “sense of control in R” (α = 0.87), “R self-concept” (α = 0.96), and “state of anxiety in R situations” (α = 0.95). We refer to the groups of questions that measure each of these constructs as “scales” of the broader R anxiety measure throughout the paper.

The gaining initial skills in R scale includes 20 items that ask students about how anxious they would feel while performing specific learning tasks in R, including learning about basic functions, using R, and receiving feedback on R skills. An example item from this scale is “Rate the extent to which taking a course in R would make you anxious.” This scale has the highest possible score of 5, with 1 being least anxious about gaining initial skills in R and 5 being most anxious. McInerney (1997) included the gaining initial skills in R construct due to the assumption that anxiety about computers is context specific, necessitating exploration of the beginner-specific experience. We did not deploy this scale in the third semester of this study, in order to prevent survey fatigue when a coping skills scale was added. We chose to remove this scale as we felt that the other three anxiety scales best encapsulated potential overall, long-term changes in student self-efficacy and R anxiety, while anxiety about gaining initial skills in R is more specific to the initial introductory stages of a student’s R learning trajectory. The sense of control in R scale, with 14 items, is designed to measure students’ sense of self-control over situations that include R, which are examined through asking students how often they engage in positive and negative self-talk. For example, students are asked to rate how often they think “I feel in control of what I do” or “What if I hit the wrong key?” while using R. This scale has the highest possible score of 5, with 1 being the lowest sense of control in R and 5 being the highest sense of control in R. The R self-concept scale, with 22 items, is designed to measure students’ self-image and self-efficacy in regard to situations with R. For example, students were asked to rate their agreement with items such as “I am sure I could solve any problems I had while I was using R.” This scale has the highest possible score of 5, with 1 being the least positive R self-concept and 5 being the most positive R self-concept (McInerney, 1997; Connolly ). McInerney (1997) describes sense of control and self-concept as two critical constructs when considering ways to decrease anxiety. That is, a student’s sense of control and self-concept are correlated with anxiety; increases in these two constructs can help reduce the learner’s fear of task failure and anxiety (McInerney, 1997). Finally, the state of anxiety in R situations scale, with 22 items, is designed to measure student worry, distractibility, comfort, and physiological symptoms while using R. For example, students were asked to rate how often “I feel helpless when I use R” and how often they experience “sweaty palms.” This scale has the highest possible score of 4, with 1 being the lowest state of anxiety and 4 being the highest state of anxiety. The state of anxiety in R situations construct was included by McInerney (1997) because of its importance in measuring situation-specific anxiety and its relevance to anxiety students experience when being evaluated. Together, these four constructs (78 total items) can help us to assess and understand students’ overall anxiety when coding in R.

Immediacy Measures

We used two scales to measure instructor immediacy: The Revised Non-verbal Immediacy Measure (RNIM; developed and validated by McCroskey ) and the full Gorham’s Verbal Immediacy Measure (developed and validated by Gorham, 1988). The RNIM is a revised version of the Non-verbal Immediacy scale developed by Richmond, Gorham, and McCroskey (1987), asking students to score the frequency of instructor behaviors such as “gestures while talking to the class,” “smiles at the class while talking,” and “has a very relaxed body position while talking to the class,” for 10 items on a five-point Likert scale, with 1 representing “never” and 5 representing “very often” (McCroskey ). McCroskey collected the original evidence of scale validity for undergraduate students across 2,300+ undergraduate students from five different universities in five countries, demonstrating reliability across all, including the U.S. population (n = 365, α = 0.85, α = 0.82 in our study). Gorham’s Verbal Immediacy measure uses 20 items on a five-point Likert scale for the frequency of the instructor’s verbal behaviors, such as “uses personal examples or talks about experiences she/he has had outside of class,” “addresses students by name,” or “asks questions that solicit viewpoints or opinions” (Gorham 1988). Gorham (1988) collected the original evidence of scale validity in a population of 387 undergraduate students, demonstrating stable dimensionality and reliability (α = 0.94 for a subset of the original items, with all items loading onto a single factor). We used the full version of Gorham’s scale (α = 0.8 in our study population). These two measures (30 items total) were included in the post survey at the end of the semester (see Supplemental Material).

Coping Skills Measure

We used coping skills scales to quantify self-reported 1) Avoidance/Behavioral Disengagement (e.g., “I reduce the amount of effort I put into solving the problem,” five items), 2) Active Coping (strategies students employ in the moment; e.g., “I concentrate my efforts on solving the problem,” two items), 3) Planning (future-oriented, using organization and planning to approach a problem; e.g., “I try to come up with a strategy about what to do,” two items), 4) Instrumental Support Seeking (content-specific; e.g., “I get help and advice from my peers,” two items), and 5) Self-Blame (e.g., “I criticize myself,” two items; Carver, 1997). All scales were framed with a statement asking students to consider how they cope when encountering challenges in R. These questions were included in the post survey sent to students in the third semester of this study (n = 91 with full data; pre and post survey responses).

These scales were adopted largely from the Brief COPE (Carver, 1997), which is based on the COPE instrument (Carver ). Notably, the Self-Blame scale is unique to the Brief COPE and was not part of the original COPE. Evidence of validity for the original COPE instrument was collected from 978 undergraduates at the University of Miami. Carver and colleagues used principal-factors factor analyses to investigate the dimensionality of their items. They formulated 11 scales, which included items represented in the Behavioral Disengagement, Active Coping, Planning, and Instrumental Support Seeking scales used in this work (with Active Coping and Planning loading onto a single scale). Item loadings were all above 0.3 for the items used in our study, and Cronbach’s alpha was 0.62 or greater in these initial analyses to assess internal reliabilities (Carver ). The Brief COPE drew on the COPE by selecting two items with high factor loadings and good performance in the field from each scale. Items for the Brief COPE were edited slightly to sharpen their focus, and the Self-Blame scale was added. Evidence of dimensionality and internal reliability of the Brief COPE (from which we drew the vast majority of items used in this study) was gathered from a sample of 168 community residents recovering from Hurricane Andrew (David ). Exploratory factor analyses yielded nine factors for the Brief COPE, which included the scales used in this study. Again, the Active Coping and Planning items loaded onto a single factor. Cronbach’s alpha values investigating internal reliability for these scales were all above 0.64 (Carver, 1997).

For this study, we did not use the entire Brief COPE, but rather chose four scales that were relevant to our context, because we anticipated that 1) they would be used by undergraduates in the context of R challenges, and 2) they had potential to make reasonable targets for future instructional interventions. We chose the 1) Avoidance/Behavioral Disengagement, 2) Active Coping and Planning scale, 3) Instrumental Support Seeking scale, and 4) Self-blame scale from the Brief COPE. In addition, we added back in a few relevant items from the full COPE. Because we chose only a subset of the scales, added back a few items from the full COPE instrument, and were curious about whether our students perceived the scales in the same way as prior populations, we chose to run a confirmatory factor analysis (CFA) to confirm the factor structure of the scales we chose (see Supplemental Material). Our CFA analyses confirmed that a five-factor structure fit our data. The scales included 1) Avoidance/Behavioral Disengagement behaviors, 2) Active Coping, 3) Planning, 4) Instrumental Support Seeking, and 5) Self-Blame. Notably, Active Coping and Planning were split in our best-fit structure, whereas they were previously combined in the Brief COPE.

Classroom Engagement Observations

In the third semester of this study, we collaborated with the Technology in Education Group (authors S.S. and J.F.) on our campus to conduct the Behavioral Engagement Related to Instruction protocol (BERI) with all consenting students in each section. The BERI protocol was chosen over other protocols due to its specific focus on measuring students’ engagement levels rather than students’ performance of specific tasks. For example, while other protocols like the LOPUS define “listening” as “Listening to TA, video, or student presentations as a class,” the BERI defines it as “Student is listening to lecture. Eye contact is focused on the instructor or activity and the student makes appropriate facial expressions, gestures, and posture shifts (i.e., smiling, nodding in agreement, leaning forward),” which specifically addresses the levels of student engagement while listening. The BERI was first designed for large classroom settings, but use in small classrooms allowed us to quantify student engagement for a majority of students in the third-semester sample while they learned to use R (Lane and Harris, 2015). The BERI was validated in undergraduate classrooms to ensure reliability across courses and class sizes, making it valid for use in our study population and class size. Specifically, this protocol quantifies time spent performing engagement behaviors, including listening, writing, reading, engaged computer use, engaged student interaction, and engaged interaction with the instructor (see BERI observation protocol in Lane and Harris, 2015). Conversely, it also tracks the time that students spend exhibiting disengaged behaviors: settling in/packing up, unresponsive, off-task, disengaged computer use, disengaged student interaction, and distracted by another student (Lane and Harris, 2015). Every 2 minutes, observers record what behavior each consenting student is exhibiting using the Generalized Observation Reflection Platform (developed and copyrighted by UC Davis). Each observer (four observers: C.F., S.S., J.F., L.C.) went through the same formal training on how to use the scale and online program. The training consisted of a preliminary discussion of the observation protocol and codes, watching and discussing a video of a classroom in which the students displayed the various engagement codes (video was paused and codes were discussed throughout), and finally watching videos together with other trainees while using the protocol. Video examples were watched and coded until all coders in the training felt confident applying each code and until consensus coding was consistent (i.e., the coders applied the same codes consistently to student behaviors and reached > 90% agreement). After the training, class observations were conducted. One observer attended each class, observed, applied engagement codes, and took notes related to student engagement, especially when something notable happened or the observer was unsure of a code. Observers communicated frequently before and during the observation process to discuss codes, ensure that codes were being applied similarly across contexts, and address any questions. In total, we conducted three classroom observation sessions in each lab section (as recommended by commonly used protocols like the COPUS; Smith ), two of which took place during students’ R data analysis classes at the beginning of the semester, and one of which occurred in a lab section that did not include the use of R, which we used as a baseline (control) measurement of typical engagement (total number of observations = 43). Through these observations, we captured almost the entire time that TAs instructed students in R over the course of the semester, as well as an additional observation in a non-R lecture.

Statistical Analyses

Due to the continuous nature of our data and our desire to include random effects in our statistical tests, we used linear mixed-effects models for each of our analyses. All assumptions were met for these regressions: linear relationships between variables, multivariate normality, and little multicollinearity or autocorrelation (VIF all < 5). Fixed effects (e.g., verbal immediacy) included in each model were determined through a priori hypothesis generation. Variable values for constructs measured via surveys represent the arithmetic average of items within the survey, while values for demographics (e.g., gender) represent responses based on a single question. In an initial analysis for research Q1 (“Is gender related to programming anxiety?”), we initially included student gender (man or woman), ethnicity, first-generation status, and learning disability status in an initial trial model, because we hypothesized that these may all have an effect on anxiety (Onwuegbuzie, 1999; England ). However, because of small sample sizes and lack of statistical significance, we removed the other demographic factors to focus only on gender. Similarly, we were interested in whether immediacy interacted with gender to influence anxiety and engagement. We initially ran all models predicting anxiety and engagement with gender and a gender × immediacy interaction. These models were overparameterized and did not meet the assumptions of linear mixed-effects models described earlier, so we report models without these interaction terms. In addition, we were interested in how anxiety might interact with gender to predict engagement and how coping might interact with gender to predict changes in anxiety. We did not add gender and all of the gender × anxiety or gender × coping interactions to these models (Q3A and Q4), because this would have resulted in models with too many predictor variables, given our sample size (overparameterization). Instead, we ran separate models for men and women. (Gender was self-identified through an open-response question in the survey. Responses of “female” and “male” were interpreted as “woman” and “man,” respectively, to align with the construct of gender.) Notably, we were unable to include nonbinary students in our statistical analyses out of caution to not overextend information gained from a small sample size (n = 3); however, nonbinary students are represented in our graphs, because gender is a spectrum. Thus, when we refer to gender in our results and models, we specifically refer to the difference between students identifying as women and men. Results did not differ by gender for engagement, so we report only the model with gender identities combined. However, results differed by gender for the coping model, so we report those results by gender.

In all analyses, we included semester as a random effect (denoted “1|Semester”) to account for random variation in student responses between semesters, except for in our Q4 model, because we only deployed the coping skills scale in the third semester. For Q3, which tested how instructor immediacy and anxiety relate to student engagement, we included student nested within semester as a random effect to account for random variation within individual students, because we conducted multiple observations of each student. In this model, we use presemester measures of student anxiety, as we thought that anxiety toward the start of the semester would more strongly influence how students behaved and that experience in the classroom would likely influence the end-of-semester anxiety more than vice versa. In some cases, metrics predicting our dependent variables were highly correlated (e.g., Nonverbal Immediacy and Verbal Immediacy were correlated as confirmed through linear regression). To avoid potential type 2 errors that might arise from distribution of variance among correlated predictors, we analyzed each correlated predictor in its separate model as opposed to a larger multivariate model. Equations that we used to test each research question are presented, with “Anxiety Metric” referring to each of our four R anxiety scales (gaining initial skills in R, sense of control with R, R self-concept, and state of anxiety in R situations). Thus, when the term “Anxiety Metric” is used, it indicates that four similar models were run, one with “gaining initial skills in R” as the metric, another with “sense of control with R” as the metric, and so on (note, however, that in model 3A, these metrics are used as predictors and referred to by name). Timing in Q1 refers to a dummy variable with “pre-semester” and “post-semester” as the values. The response variable for Q1 includes both pre and post scores to allow for examination of the Gender * Timing interaction in this model. “Change in anxiety metric” was calculated as the difference between pre and post survey scores for each metric for each student. Thus, we ran four models corresponding to each R anxiety scale for each question below.

Q1. Is gender related to R programming anxiety, and how did this change over time?

Anxiety Metric = Gender + Timing + Gender * Timing + (1|Semester)

Anxiety Metric = Gender + (1|Semester)

Anxiety Metric = Timing + (1|Semester)

Q2. Is instructor immediacy related to changes in R programming anxiety?

Change in Anxiety Metric = Nonverbal Immediacy + (1|Semester)

Change in Anxiety Metric = Verbal Immediacy + (1|Semester)

Q3. Are R programming anxiety and/or instructor immediacy related to student engagement in the classroom? (Analysis run separately for both R and control classroom observations, unique ID removed as random effect for control observations due to the presence of only one control observation, analyses conducted for third semester only.)

A. Programming Anxiety:

Percent Engagement = Presemester Initial Skills Anxiety + Presemester Sense of Control + Presemester R Self-Concept + Presemester State of Anxiety + (1|Semester) + (1|Unique ID)

B. Instructor Immediacy:

Percent Engagement = Nonverbal Immediacy + (1|Semester) + (1|Unique ID)

Percent Engagement = Verbal Immediacy + (1|Semester) + (1|Unique ID)

Q4. Are specific coping skills correlated with greater changes in programming anxiety? (Analysis run for women and men separately, analyses conducted for third semester only.)

Change in Anxiety Metric = Behavioral + Active + Planning + Instrumental Support Seeking + Self-Blame

RESULTS

How Does Gender Affect R Programming Anxiety, and How Does This Change over Time (Q1)?

For each of the results, we ran both the full model including the interaction term for gender and time and simple effects models within gender and within time period (pre or post). This approach allowed us to fully characterize what was happening within each gender at each time point. When a significant (p < 0.05) or nearly significant (statistically insignificant but with a p value below 0.15) result was found from the full model, it is noted, and results from the simplified models are used to describe the specific effects of the interaction. Additionally, for all scales, nonbinary students are represented in our graphs to acknowledge that gender is a spectrum beyond man and woman, despite the fact that we could not include them in our statistical analyses (n = 3).

Gaining Initial Skills in R.

This scale has the highest possible score of 5, with 1 being least anxious about gaining initial skills in R and 5 being most anxious. Women reported significantly higher anxiety presemester but not postsemester when gaining initial skills in R, with 24% higher gaining initial skills anxiety presemester and 11% higher gaining initial skills anxiety postsemester as compared with men (pre p < 0.001, pre df = 97; post p = 0.13, post df = 97; see Figures 1 and 2). Importantly, women reported significantly lower anxiety about gaining initial skills in R after the semester as compared with before the semester (p = 0.02, df = 125, beta = 0.37). Men did not report a significant difference in gaining initial skills anxiety between pre and post surveys (p = 0.8, df = 65). There was a nearly significant interaction between gender and time (pre/post) for this metric (p = 0.1, df = 191, beta = −0.41), which results from women reporting greater decreases in their anxiety about gaining initial skills than men despite their consistently higher anxiety overall (Table 2).

FIGURE 1.

Reported anxiety scores (rows) from pre- and postsemester surveys (columns) for men (green), women (orange), and nonbinary (purple) students. Nonbinary students are shown here for representation purposes, but conclusions cannot be drawn due to a small sample size (n = 3). Statistically significant differences in each metric between students identifying as women and men students are denoted with asterisks. Women consistently reported higher anxiety (state of anxiety, gaining initial skills anxiety) and lower confidence (sense of control, self-concept) both pre- and postsemester compared with men.

FIGURE 2.

Changes in anxiety scores (rows) from pre- to postsemester surveys for men (green), women (orange), and nonbinary (purple) students. Nonbinary students are shown here for representation purposes, but conclusions cannot be drawn due to a small sample size (n = 3). An increase in a given metric would be shown above 0 on the y-axis, while a decrease would be shown below 0 on the y-axis. Statistically significant differences between pre- and postsemester students for men and women separately are denoted with asterisks. All students’ reported anxiety (state of anxiety, gaining initial skills anxiety) and confidence (sense of control, self-concept) generally improved over the course of the semester, although changes did not remove the gender gap.

TABLE 2.

Results of all statistical analyses, organized by research question and analysisa

Modelb	beta	SE	df	t value	p value	VIF	R 2 m	R 2 c
Q1
Anxiety metric = Gender + Timing + Gender*Timing + (1|Semester)
Gaining initial skills in R anxiety	−0.41	0.25	191	−1.62	0.10	2.53	0.09	0.09
Sense of control in R	0.31	0.17	378	1.84	0.06	2.49	0.13	0.13
R self-concept	0.07	0.17	378	0.43	0.66	2.51	0.06	0.07
State of anxiety in R situations	−0.16	0.47	378	−1.09	0.27	2.49	0.08	0.08
Anxiety metric = Gender + (1|Semester)
Pre: Gaining initial skills in R anxiety	−0.68	0.18	97	−3.74	0.0003	1.01	0.13	0.13
Pre: Sense of control in R	0.57	0.12	190	4.87	0	1.01	0.11	0.11
Pre: R self-concept	0.38	0.11	190	3.37	0.0008	1.01	0.06	0.06
Pre: State of anxiety in R situations	−0.37	0.10	190	−3.60	0.0004	1.01	0.06	0.07
Post: Gaining initial skills in R anxiety	−0.26	0.17	97	−1.53	0.13	1.01	0.06	0.06
Post: Sense of control in R	0.27	0.12	190	2.25	0.02	1.01	0.03	0.03
Post: R self-concept	0.31	0.13	190	2.31	0.02	1.01	0.04	0.04
Post: State of anxiety in R situations	0.22	0.10	190	−2.05	0.04	1.01	0.03	0.03
Anxiety metric = Timing + (1|Semester)
Men: Gaining initial skills in R anxiety	−0.03	0.15	65	−0.23	0.81	1	0.00	0.10
Men: Sense of control in R	−0.22	0.11	124	−2.07	0.04	1	0.03	0.03
Men: R self-concept	−0.22	0.11	126	−1.95	0.05	1	0.03	0.03
Men: State of anxiety in R situations	0.16	0.09	124	1.71	0.09	1	0.02	0.09
Women: Gaining initial skills in R anxiety	0.37	0.16	125	2.29	0.02	1	0.04	0.04
Women: Sense of control in R	−0.53	0.11	252	−5.05	0	1	0.09	0.09
Women: R self-concept	−0.29	0.11	250	−2.7	0.007	1	0.03	0.05
Women: State of anxiety in R situations	0.32	0.09	252	3.53	0.0004	1	0.05	0.07
Q2
Change in anxiety metric = Nonverbal Immediacy + (1|Semester)
Gaining initial skills in R anxiety	−0.06	0.15	98	−0.41	0.68	1	0.00	0.00
Sense of control in R	0.22	0.09	191	2.37	0.02	1	0.03	0.03
R self-concept	0.21	0.10	191	2.02	0.04	1	0.02	0.02
State of anxiety in R situations	−0.10	0.08	191	−1.22	0.22	1	0.01	0.01
Change in anxiety metric = Verbal Immediacy + (1|Semester)
Gaining initial skills in R anxiety	−0.21	0.20	98	−1.04	0.29	1	0.01	0.01
Sense of control in R	0.17	0.14	191	1.20	0.23	1	0.01	0.01
R self-concept	0.25	0.15	191	1.62	0.11	1	0.01	0.01
State of anxiety in R situations	−0.17	0.12	191	−1.38	0.17	1	0.01	0.01
Q3
R Anxiety: Percent Engagement = Presemester Gaining Initial Skills Anxiety + Presemester Sense of Control + Presemester R Self-Concept + Presemester State of Anxiety + (1|Semester) + (1|Unique ID)
R instruction observation: State of anxiety in R situations	−0.01	0.01	46	−1.08	0.28	1	0.02	0.02
R instruction observation: Sense of control	0.01	0.01	46	0.99	0.32	1	0.01	0.01
R instruction observation: R self-concept	0.01	0.01	46	1.23	0.22	1	0.02	0.02
Control instruction observation: State of anxiety in R situations	−0.03	0.09	11	−0.38	0.71	1	0.01	0.99
Control instruction observation: Sense of control	−0.03	0.07	11	−0.51	0.62	1	0.03	0.99
Control instruction observation: R self-concept	−0.03	0.08	11	−0.44	0.66	1	0.02	0.99
Percent Engagement = Nonverbal Immediacy + (1|Semester) + (1|Unique ID)
R instruction observation	0.01	0.01	46	0.81	0.42	1	0.01	0.54
Control instruction observation	−0.04	0.07	50	−0.71	0.48	1	0.01	0.01
Percent Engagement = Verbal Immediacy + (1|Semester) + (1|Unique ID)
R instruction observation	−0.01	0.02	46	−0.57	0.56	1	0.00	0.54
Control instruction observation	−0.22	0.12	50	−1.72	0.09	1	0.06	0.06
Q4
Change in anxiety metric = Avoidance/Behavioral Disengagement + Active Coping + Planning + Instrumental Support Seeking + Self-Blame
Men: State of anxiety in R situations						2.75	0.55	0.55
Avoidance/behavioral disengagement	0.08	0.11	23	0.75	0.45
Active coping	0.27	0.16	23	1.64	0.11
Planning	−0.54	0.12	23	−4.38	0.00
Instrumental support seeking	0.14	0.11	23	1.29	0.21
Self-blame	0.14	0.07	23	2.04	0.05
Men: Sense of control in R						2.75	0.43	0.43
Avoidance/behavioral disengagement	−0.31	0.17	23	−1.82	0.08
Active coping	−0.23	0.24	23	−0.96	0.34
Planning	0.41	0.18	23	2.27	0.03
Instrumental support seeking	−0.27	0.16	23	−1.71	0.10
Self-blame	−0.15	0.11	23	−1.41	0.17
Men: R self-concept						2.75	0.45	0.45
Avoidance/behavioral disengagement	−0.06	0.15	23	−0.44	0.66
Active coping	−0.02	0.22	23	−0.11	0.90
Planning	0.47	0.16	23	2.88	0.008
Instrumental support seeking	−0.09	0.14	23	−0.61	0.54
Self-blame	−0.11	0.09	23	−1.17	0.25
Women: State of anxiety in R situations						4.29	0.27	0.27
Avoidance/behavioral disengagement	0.42	0.16	57	2.52	0.01
Active coping	0.27	0.24	57	1.11	0.27
Planning	−0.22	0.25	57	−0.87	0.38
Instrumental support seeking	−0.33	0.14	57	−2.36	0.02
Self-blame	−0.02	0.09	57	−0.19	0.85
Women: Sense of control in R						4.29	0.13	0.13
Avoidance/behavioral disengagement	−0.58	0.22	57	−2.63	0.01
Active coping	−0.40	0.32	57	−1.25	0.22
Planning	−1.54	0.34	57	−0.46	0.64
Instrumental support seeking	0.20	0.18	57	1.11	0.27
Self-blame	0.02	0.11	57	0.18	0.85
Women: R self-concept						4.29	0.35	0.35
Avoidance/behavioral disengagement	−0.54	0.17	57	−3.08	0.003
Active coping	0.11	0.25	57	0.46	0.64
Planning	0.01	0.26	57	0.06	0.95
Instrumental support seeking	0.05	0.14	57	0.34	0.72
Self-blame	0.18	0.09	57	1.98	0.05

aR2m, marginal R2 value; R2c, conditional R2 value.

b“Timing” is a dummy variable and refers to pre- vs. postsemester survey.

Sense of Control in R.

This scale has the highest possible score of 5, with 1 being the lowest sense of control in R and 5 being the highest sense of control in R. Women reported a significantly lower sense of control in R, with women reporting 17% lower sense of control presemester and 7% lower sense of control postsemester as compared with men (pre p < 0.0001, pre df = 190; post p = 0.025, post df = 190; see Figures 1 and 2). Both women and men had significant increases in their sense of control between pre and postsemester survey responses (women’s values: p < 0.0001, df = 252, beta = −0.53; men’s values: p = 0.04, df = 124, beta = −0.22). We observed a nearly significant interaction between gender and time (pre/post) for this metric (p = 0.07, df = 378, beta = 0.31), which results from women making greater gains in sense of control than men despite their consistently lower sense of control in R overall (Table 2).

R Self-Concept.

This scale has the highest possible score of 5, with 1 being the least positive R self-concept and 5 being the most positive R self-concept. Women reported a significantly lower R self-concept, reporting a 17% lower self-concept presemester and 12% lower self-concept postsemester as compared with men (pre p < 0.001, pre df = 190; post p = 0.02, post df = 190; see Figures 1 and 2). Women reported a significant increase in R self-concept after the semester as compared with before the semester (p = 0.007, df = 250, beta = −0.29). Men also reported a significant increase in their R self-concept (p = 0.05, df = 126, beta = −0.22).

State of Anxiety in R Situations.

This scale has the highest possible score of 4, with 1 being the lowest state of anxiety and 4 being the highest state of anxiety. Women reported a significantly higher state of anxiety, reporting a 16% higher state of anxiety presemester and an 11% higher state of anxiety postsemester as compared with men (pre p < 0.001, pre df 190; post p = 0.04, post df = 190; see Figures 1 and 2). Both men and women reported lower levels of anxiety after the semester as compared with before the semester, although the effect was not significant for men according to results from the simple effects model (women’s values: p < 0.001, df = 252, beta = 0.32; men’s values: p = 0.09, df = 124, beta = 0.16).

Is Instructor Immediacy Related to Changes in R Programming Anxiety (Q2)?

Because we hypothesized that nonverbal and verbal immediacy would be highly correlated (which we confirmed through a linear regression), we ran separate models for each of these two metrics.

Nonverbal Instructor Immediacy.

The nonverbal immediacy scale has the highest possible score of 5, with 1 being the least nonverbally immediate and 5 being the most nonverbally immediate. The mean nonverbal immediacy score for women was 4.16, with men reporting an average verbal immediacy score of 3.83. Higher nonverbal immediacy scores were correlated with greater increases in students’ sense of control with R (p = 0.02, df = 191, beta = 0.22) and R self-concept (p = 0.04, df = 191, beta = 0.21; see Figure 3). Nonverbal immediacy was not correlated with changes in students’ state of anxiety while using R (p = 0.22, df = 191) or anxiety about gaining initial skills in R (p = 0.68, df = 98).

FIGURE 3.

Higher nonverbal immediacy scores were correlated with greater increases in students’ sense of control with R (left; df = 191, p = 0.01, beta = 0.23) and R self-concept (right; df = 191, p = 0.04, beta = 0.21). Data are shown with linear regression lines (dark gray line) and 95% confidence interval (gray shading).

Verbal Instructor Immediacy.

The verbal immediacy scale has the highest possible score of 4, with 1 being the least verbally immediate and 4 being the most verbally immediate. The mean verbal immediacy score for women was 2.94, with men reporting an average verbal immediacy score of 2.76. Student-reported verbal immediacy was not correlated with changes in students’ sense of control with R (p = 0.23, df = 191), R self-concept (p = 0.11, df = 191), students’ state of anxiety (p = 0.17, df = 191), or gaining initial skills in R anxiety (p = 0.29, df = 98).

Do Programming Anxiety or Instructor Immediacy Affect Student Engagement in the Classroom (Q3)?

Overall, students were highly engaged during the lab classes in both sections where they were learning R, with 94% mean engagement in each of the two R-focused sections. Students were 8% less engaged, but still highly engaged in the “control” observation during a lab that focused on poster creation and no R content, with a mean percent engagement of 86%.

R Anxiety.

Percent engagement in the classroom during R sessions was not correlated with presemester R self-concept (p = 0.22, df = 46), sense of control with R (p = 0.32, df = 46), or students’ state of anxiety while using R (p = 0.28, df = 46). No significant relationships between any metric of anxiety and percent engagement were found in control sessions either (p values all > 0.7). Here, we do not report on gaining initial skills in R, because we did not deploy that scale in the semester that we did classroom observations (so we could add the coping skills scale without increasing survey fatigue).

Instructor Immediacy.

Percent engagement in the classroom was not correlated with nonverbal immediacy (p = 0.42, df = 46) or verbal immediacy (p = 0.56, df = 46) for the R sessions. No significant relationships were found when examining engagement in control sessions either (p values all > 0.09).

Are Coping Skills Correlated with Changes in Programming Anxiety (Q4)?

Behavioral Disengagement.

For women, lower reported rates of avoidance/behavioral disengagement were associated with greater increases in their sense of control in R (beta = −0.58, p = 0.01, df = 57), greater increases in R self-concept (beta = −0.54, p = 0.003, df = 57), and greater decreases in their state of anxiety while using R (beta = 0.42, p = 0.01, df = 57; see Figure 4). For men, avoidance/behavioral disengagement was not associated with changes in any metric of anxiety (all p > 0.05).

FIGURE 4.

Lower reported rates of avoidance/behavioral disengagement are associated with greater decreases in women’s state of anxiety while using R (left; beta = 0.42, df = 57, p = 0.01), greater increases in their sense of control in R (middle; beta = −0.58, p = 0.01, df = 57), and greater increases in R self-concept (right; beta = −0.54, df = 57, p = 0.003). Data are shown with linear regression lines (dark gray line) and 95% confidence interval (gray shading).

Active Coping.

Active coping was not associated with changes in any metric of anxiety for men or women (all p > 0.05).

Planning.

For men, higher rates of planning were associated with greater increases in sense of control in R (beta = 0.41, p = 0.03, df = 23), greater increases in R self-concept (beta = 0.47, p = 0.008, df = 23), and greater decreases in their state of anxiety while using R (beta = −0.54, p = 0.0002, df = 23; see Figure 5). For women, planning was not associated with changes in any metric of anxiety (all p > 0.05).

FIGURE 5.

Figure showing the relationship between men’s planning scores and changes in their state of anxiety with R, sense of control in R, and R self-concept (in order left to right). For men, higher rates of planning when encountering challenges with R were associated with greater decreases in their state of anxiety while using R (left; beta = −0.54, df = 23, p = 0.0002), greater increases in sense of control in R (middle; beta = 0.40, df = 23, p = 0.03), and greater increases in R self-concept (right; beta = 0.48, df = 23,  p = 0.008). Data are shown with linear regression lines (dark gray line) and 95% confidence interval (gray shading).

Instrumental Support Seeking.

For women, higher rates of instrumental support seeking were associated with greater decreases in their state of anxiety while using R (beta = −0.33, p = 0.02, df = 57), but not with changes in R self-concept or sense of control in R (p > 0.05; see Figure 6). For men, instrumental support seeking was not associated with changes in any metric of anxiety (all p > 0.05).

FIGURE 6.

For women, higher rates of instrumental support seeking are associated with greater decreases in their state of anxiety while using R (beta = −0.33, df = 57, p = 0.02). Data are shown with a linear regression line (dark gray line) and 95% confidence interval (gray shading).

Self-Blame.

For men, higher rates of self-blame were associated with greater decreases in their state of anxiety while using R (beta = 0.14, p = 0.05, df = 23), but were not associated with R self-concept or sense of control (p > 0.05). For women, higher rates of self-blame were related to greater increases in R self-concept (p = 0.05, df = 57, beta = 0.18), but were not related to their state of anxiety or sense of control (p > 0.05). In both cases, the effect of self-blame on the anxiety metrics was very small (beta < 0.2).

Methodological Limitations

This study was an observational study confined to a single course context that used mixed-model regression analysis to identify relationships among variables. Because randomization and a control/comparison group were not used in this study, we cannot infer causation. For example, we cannot infer that the R instruction in the course under investigation resulted in the observed improvements to R anxiety, skills, self-concept, or sense of control. Nor can we infer that specific coping strategies caused changes in self-concept, sense of control, or R anxiety. Furthermore, we recognize that administering the pretest, which asked students about anxiety, might have induced anxiety via suggestion (i.e., via the self-fulfilling prophecy of anxiety or fear suggested by some of the items) and inflated levels of anxiety measured. While this condition was true for all students in this study, it nonetheless may have influenced our results. We also cannot extend the results of this study to other courses; our results should only be considered within the context of the course in question. Additionally, our results should only be considered for the student identities present in our data set. Students at our institution are predominantly white and ages 18–24. Our results may not apply to persons from underserved ethnic or racial groups or nontraditional students. Likewise, our results apply primarily to cisgendered individuals; gender non-conforming and nonbinary individuals made up a very small portion of our sample (n = 3). Furthermore, due to a highly unbalanced design (i.e., all except one of the instructors identified as women), we could not investigate the effect of instructors’ gender identities on the outcomes of interest.

Despite these limitations, the significant relationships observed as a result of our mixed models allow us to hypothesize mechanisms that may have caused these relationships and propose interventions and future studies for further investigation. Also, while our results are not broadly applicable, they can lend insight into courses similar to ours in which R skills make up a small but significant portion of instruction. In addition, our results allow us to add to the body of studies that characterize patterns in R anxiety across demographic variables, such as gender, which we report on here.

Another limitation of this study is that we ran ∼50 regressions to avoid overly parametrized models. This means that 2.5 (5%) of our inferences may be incorrect. However, following Gotelli and Ellison’s (2013) suggestion, we did not globally reduce our alpha from 0.05. Specifically, a global reduction of alpha is excessively conservative, as it assumes that all tests are independent of one another and that all of the null hypotheses are true. Further, alpha is an important standard for comparison across scientific literature, and each test provides an important piece of information in distinguishing between scientific hypotheses. Another limitation in our statistical analyses relates to our analysis of Q3—whether presemester anxiety scores were correlated with percent engagement. We hypothesized that presemester anxiety scores would affect engagement more than post survey anxiety scores, because the labs these observations were conducted in represented the first R learning opportunity for many of our students and were closer in time to the pre survey. However, this does not take into account that anxiety scores are likely to change over time and student anxiety scores could have been different at the exact time of observation. A last limitation of our analyses is that data for Q4, which investigates coping mechanisms, were only collected for one semester (semester 3), and the small sample size could lead to overparameterization. However, this model met assumptions of linear analyses as described in the statistical methods.

An additional limitation of this study is that our participants were drawn from a volunteer sample. Students were not required to participate and were instead offered the opportunity to participate and self-selected into the study. This has the potential to bias study results if the students who choose not to enroll represent a distinct subpopulation with different experiences in comparison with those who enroll. We have no reason to believe that this is the case. Of students who were invited to participate in this study, 63% participated, and their demographics were not different from those typical of the course. Thus, we are reasonably confident that our participant sample is not biased due to students’ self-selection into the study.

A final limitation of our study arises from the observational nature of this work. Our research questions and predictions are aimed at understanding how variation in one observed variable (the predictor) relates to another (the response). If variation in the observed measures of either the predictor or response variable is limited, it becomes more difficult to ascertain if there is or is not a relationship between the two variables, because we are limited to looking only at the range of values represented by the data. Within our data, variation is somewhat limited in our measures of instructor immediacy (all instructors had relatively high immediacy values) and quite limited in our measures of engagement (almost all students displayed very high engagement). Thus, our investigations are limited to investigating relationships among instructors with relatively high immediacy and among relatively engaged students. If we had more variation in our observed measures of these predictor variables or conducted an experiment, we would have more potential to observe significant relationships where currently we see none. More investigations examining a broader spectrum of immediacy and engagement could be done to elucidate whether these factors might affect R anxiety and other metrics under different conditions.

DISCUSSION

All Students Report Improvements in Anxiety over the Course of the Semester, but Women Consistently Report Significantly Higher R Anxiety, Lower Self-Concept, and Lower Sense of Control in R

We found that, relative to their men classmates, women consistently reported 1) higher anxiety about gaining initial skills in R (24% pre and 11% post, although post was not a significant difference), 2) a lower R self-concept (17% pre and 7% post), 3) a lower sense of control in R (17% pre and 12% post), and 4) a higher state of anxiety in R situations (16% pre and 11% post) both before and after the semester. Our findings of a narrowed, yet persistent gender gap between women and men in programming anxiety and self-efficacy are similar to related prior research showing gender gaps in statistics anxiety (Ralston ), math self-efficacy (Pajares, 2005), and computer anxiety (Chua ; He and Freeman, 2010; Powell, 2013). Here, we complement the existing literature by investigating anxiety associated with learning to use a common data analysis tool and coding language (R). We also contribute evidence that can help us to understand how students’ gender identity can impact the experience of learning to program in a biology course. We show that, despite greater numeric representation of women in biology (Eddy ; Spini ) and even as technological literacy in the general population has advanced along with the importance of teaching undergraduate students coding skills (Auker and Barthelmess, 2020), women in biology experience their R course work differently from men. Notably, however, due to sample size, we cannot comment on the experiences of individuals with gender identities other than “man” and “woman” nor whether gender gaps in R anxiety existed, persisted, or dissipated over the term of the study for such students.

The continuation of the historic trend of women reporting lower levels of self-efficacy than men and experiencing negative psychological states when engaging in quantitative tasks is concerning. Notably, experiencing a negative physiological and psychological state during task engagement (i.e., experiencing anxiety) combined with lack of self-efficacy and development of self-concept at a task can decrease motivation and engagement (Wigfield and Eccles, 2000; Usher and Pajares, 2008). This, in turn, could preclude women from accessing mastery experiences while learning R, leading to a positive feedback loop that further threatens self-efficacy and leads to greater disengagement (Wigfield and Eccles, 2000; Usher and Pajares, 2008). While lower self-efficacy and higher anxiety in R are not direct measures of student performance or grades in courses that use programming skills, it is important to recognize that self-efficacy, self-concept, and identity development are important predictors of persistence in STEM (Graham ) Thus, this trend has the potential to limit women’s persistence in statistics-heavy aspects of ecology and evolutionary biology and other subdisciplines such as computational biology, which are increasingly valued in the workforce (Burning Glass Technology, 2016).

Unfortunately, there is evidence that this negative cycle begins early, with gender gaps between women and men in math self-efficacy starting as early as middle school, contributed to by factors including media representation and parents’ perceptions of their children’s abilities (Meece and Courtney, 1992; Wigfield ; Pajares, 2002, 2005). However, we can potentially break this cycle by providing interventions that target programming anxiety. While we did not directly investigate methodology for, or results of, direct interventions on anxiety, several of our results discussed later suggest potential targets for such interventions.

Despite the presence of persistent gender gaps between women and men, anxiety for all students decreased and self-concept and sense of control increased over the course of the semester. We also found evidence that the magnitude of gender gaps between women and men decreased postsemester for anxiety associated with gaining initial skills in R and sense of control in R (as indicated by significant or nearly significant interaction terms). Thus, aspects of the course curriculum may have served to reduce these gender gaps. However, given that we did not have a comparison group in this study, we cannot say with certainty whether this was the case. Given our results, however, we do hypothesize that low-stakes practice in R, frequent feedback, more mastery experiences, learning in an active-learning environment (workshop-style sessions in their lab course), and applying R skills to independent work for which students felt a sense of ownership (using R to analyze data from their independent projects) all may have contributed to these positive outcomes (Pajares, 2002; Corwin ). However, again, due to the observational nature of this study, we cannot say whether these outcomes were because of successful pedagogical techniques or simply increased R exposure over time. Additional experimental or quasi-experimental experiments involving interventions may better serve to elucidate beneficial teaching practices (see the Implications for Teaching and Research section).

Instructor Immediacy Has Minimal Effects on Student R Anxiety

Contrary to a wide array of literature showing that instructor immediacy can decrease students’ quantitative anxiety and positively impact student learning (Witt ; Williams, 2010), we found minimal evidence in the context of our study for instructor immediacy impacting student R anxiety or confidence. Two exceptions were that nonverbal instructor immediacy was positively associated with students’ R self-concept and sense of control in R; however, those effect sizes were extremely small and unlikely to be of high impact. Thus, we do not discuss these in depth.

We hypothesize that we did not find evidence of a relationship between instructor immediacy and student R anxiety due to two main factors. First, there was a lack of variation in instructor immediacy—in general, immediacy levels were high. Statistically, it is harder to view an effect when there is limited variation in the data. This could also suggest there is a threshold after which immediacy does not make a significant impact on student anxiety or self-efficacy. Second, class context likely affected our findings. We conducted this research in small lab courses, with a maximum of 14 students in each section. These small class sizes, taught by graduate TAs likely meant greater opportunities for immediate behaviors, and because students received almost personalized instruction in R through workshop-style lessons, the effects of immediacy could have been eliminated (Furlich, 2016). To this effect, small class sizes may have precluded the effect of immediacy acting on anxiety in part through causing students to feel that they could easily access the instructor (Furlich, 2016). In this course, it is arguable that all students had access to their TAs, despite some small variation in how immediate they may have felt their instructors were.

R Anxiety and Instructor Immediacy Are Not Correlated with Student Engagement

We found no evidence that R anxiety and instructor immediacy impact student engagement in the classroom. Instructor immediacy has been shown to increase student willingness to communicate with the instructor (Allen ), so we predicted that it might increase student engagement during class. However, we did not see an effect, likely because of extremely high overall levels of engagement in our observation sessions. Class sizes were very small, and each lesson in R was active and highly structured—students went through an R script at the same time as the TA, and TAs frequently checked in. One example of this is that some TAs asked students to put different-colored sticky notes on their monitors to denote their progress with a given section of code (e.g., red for stuck and need help from the TA, green for done and ready for the next section). Thus, it was difficult for students to not be fully engaged for the duration of the R workshop. This aligns with prior work suggesting that active-learning approaches increase student engagement (Ambruster ).

Adaptive Coping Skills Are Associated with Improvements in Anxiety but Differ by Gender

We found several relationships between coping responses and anxiety. In addition, we found that men and women reported using different coping strategies, which has also been shown in other research (Lawrence ; Eschenbeck ; Madhyastha ; Martínez ). Notably, avoidance/behavioral disengagement, predicted to be a maladaptive coping response in academic contexts (Skinner ; Henry ), was associated with changes in anxiety metrics for women, but not for men. For women, less frequent avoidance/behavioral disengagement was associated with greater improvements in sense of control, greater increases in R self-concept, and greater decreases in state of anxiety. This has important implications for student persistence, as students who experience a lower sense of control and higher sense of anxiety are more likely to disengage in the long term (Wigfield and Eccles, 2000; Bonneville-Roussy ). This also supports predictions that avoidance/behavioral disengagement is maladaptive in academic contexts, because women who engaged in more frequent disengagement did not experience decreases in anxiety or improvements in R self-concept and sense of control (Henry ). Further, this aligns with empirical findings examining experiences of medical students and college students that indicated that emotion-focused coping, which includes denial, avoidance, and disengagement, resulted in lower motivation and satisfaction and was associated with higher rates of failure in comparison with other coping strategies (Struthers ; Alimoglu )

Patterns in self-blame did not align with our predictions of self-blame as maladaptive, standing in contrast to our results regarding disengagement. We found evidence that higher rates of self-blame were associated with greater decreases in men’s state of anxiety and greater increases in women’s R self-concept. While the effects we found of self-blame are very small, this counterintuitive result is notable and may be due to the complex relationship between self-blame and attribution of control. On one hand, self-blame is known to lead to rumination and inaction (Legerstee ) and is associated with multiple types of anxiety in children (Rodriguez-Menchon et al., 2021) and increased stress in college students (Straud and McNaughton-Cassill, 2019). However, self-blame may also empower individuals to act if they blame their own behavior and perceive their responsibility in causing the stressor (as opposed to blaming their character or disposition; Shaver and Drown, 1986). If individuals perceive an issue to be their own fault as a result of some changeable controllable behavior, as opposed to the result of some uncontrollable external factor, they may feel a greater sense of control over a situation, and thus can feel empowered to improve the situation (Rotter, 1966; Weiner, 1985). Thus, the dual nature of self-blame may help to explain our finding of weak associations between self-blame and improvements in anxiety and self-concept.

Responses such as active coping, planning, and instrumental support seeking are predicted to be adaptive in STEM academic contexts (Henry ) and are typically associated with positive outcomes such as avoiding burnout (Sevinç and Gizir, 2014; Shin ), increased motivation (Struthers ), satisfaction (Alimoglu ), and increased academic achievement (Brdar ). We found some support for this in our data. For men, higher rates of planning were related to greater increases in sense of control and R self-concept and greater decreases in their state of anxiety. This corroborates findings that planning is one component that negatively predicts burnout (anxiety is one correlate of burnout; Shin ; Lyndon ) and positively predicts academic achievement (Struthers ), but adds granularity to who may benefit the most from planning. Similarly, we also found that instrumental support seeking was associated with decreases in women’s state of anxiety when using R, again corroborating evidence that seeking support often occurs in response to anxiety in an effort to alleviate it (Rijavec and Brdar 1997) and helps to avoid burnout (Shin ). These findings suggest that women in particular may have found that the support provided in class helped to alleviate their anxiety. Further, this finding suggests that targeting interventions that build in explicit, readily available opportunities for instructor support may assist in reducing gender gaps in anxiety.

Surprisingly, active coping did not predict any of the anxiety metrics measured in this study, despite our prediction that it might alleviate anxiety and increase R sense of control. We initially predicted that active coping would help students to solve their problems in R, which we assumed would lead to a greater sense of control and lower anxiety. However, we failed to consider an important characteristic of the classrooms we studied. In the active-learning format, students were expected to be active in troubleshooting and solving problems in R. This explicit expectation could have increased the frequency of reporting this coping mechanism, dampening the ability to observe an effect. It could also be that the timescale of our study was also too short to adequately observe relationships between these variables.

IMPLICATIONS FOR TEACHING AND RESEARCH

Our findings show a persistent gender gap in anxiety that undergraduate ecology students experience while learning to code in the statistical program R—a gap that narrows over the course of the semester but is still maintained. Specifically, we found that women reported significantly higher anxiety and lower confidence than men, and that this gender gap remained over the course of a semester. However, our findings suggest that coping strategies share a moderately strong relationship with anxiety and the observed gender gap. As a result, teachers would benefit from focusing on interventions that improve coping strategies. We found that women who reported instrumental support seeking more frequently showed greater decreases in R anxiety, and those who reported disengaging less frequently showed greater increases in R sense of control and R self-concept and greater decreases in R anxiety. This result is particularly notable for R self-concept, given that students displayed lower levels of R self-concept in comparison with other anxiety metrics. Thus, women may especially benefit from interventions that discourage avoidance/behavioral disengagement and encourage help-seeking. Men seemed to benefit specifically when planning was used, especially with regard to improvements in their very low levels of R self-concept. All in all, interventions that provide training for how to develop a plan when facing a coding issue, build in opportunity for seeking instrumental support, and provide alternatives to disengaging in difficult R tasks may help students to alleviate or avoid anxiety while also working toward closing the observed gender gap.

In previous work, growth mindset interventions have been shown to be particularly effective in encouraging instrumental support seeking and decreasing achievement gaps (Yeager ; Casad ; Fink ; Covarrubias ; Henry ). For example, sending an email inviting students to join a peer-led tutoring program in a gateway STEM course that used growth language (e.g., “this program helps you and your peers to build a learning community,” “mastering course material is a process that takes hard work and effort,” “our job is to grow your understanding of the material step-by-step and to support you in this process”) led to significantly higher rates of instrumental support seeking in women compared with when students received an email excluding growth language (Covarrubias ). In another study with first-year college students in an introductory chemistry course, assignments that included growth mindset information (e.g., reading an article titled “You Can Grow Your Brain,” which explains that the brain is malleable and knowledge is not fixed) and reflection practices (e.g., asked to reflect on how the growth mindset article would inform their study strategies) eliminated an achievement gap between underrepresented minority and white students that was present when these interventions were not conducted. Toward increasing engagement and decreasing achievement gaps, more structured courses and greater employment of active-learning techniques may be particularly effective (Tanner, 2013; Eddy and Hogan, 2014; Gavassa ). Future research on 1) programming anxiety across biology programs, not just introductory ecology courses; 2) different contexts including, for example, community colleges and larger classes; 3) interventions that target adaptive coping strategies to alleviate programming anxiety; and 4) interventions to decrease disengagement in the face of challenging materials would help to elucidate how widespread gender gaps in programming anxiety are across our discipline and aid in the development of solutions for mitigating them.

Click here for additional data file.