Computer Science Students' Perceptions of Emergency Remote Teaching: An Experience Report.

In the first 6 months of 2020, the COVID-19 pandemic forced numerous universities across the globe to quickly transfer all their courses online, a response known as Emergency Remote Teaching. Courses initially designed for face to face delivery had to be quickly adapted to a new online format. In this paper, we study the perception of the transition to remote teaching in a group of computer science students. Despite the advantage given by an average higher computer literacy, the results of this study suggest that students found the transition challenging, especially regarding tasks such as asking questions during video lectures and interacting with instructors. The transition seems to have had a greater impact on students of lower level courses. Differences were found also across race and residence status (but not gender). The initial format of the course was also relevant: students fared better if their course relied on online tools before the transition.

Introduction

The beginning of the year 2020 saw the outbreak and worldwide spread of a new coronavirus (COVID-19) so contagious and dangerous to force the World Health Organization (WHO) to declare a global pandemic. Countries across the globe put in place unprecedented measures to contain the spread of the virus, asking people to limit social interactions, work from home, and wear masks in public places [8, 24]. Like many other organizations, universities had to adapt to the new mandates and take drastic actions to protect the students and the population at large, closing campuses and moving all classes online [15]. To this day, many institutions maintain limited access to their campuses and still offer a good portion of their courses remotely [29]. The academic community was quick to emphasize the difference between this emergency shift of classes to a new format from proper online learning [21], and generally refers to what happened in response to COVID as emergency remote teaching (ERT). While courses intended to be delivered fully online are planned over several months to be carried out successfully, giving time to the instructors to carefully tailor content and delivery mode to this format, ERT is an emergency response adopted in time of crisis to guarantee the continuation of courses initially designed to be delivered completely or partially face to face. It is meant to be ad-hoc, temporary, and allows much less time to prepare. There have been several instance of humanitarian and natural disasters that caused an emergency shift to remote teaching we can remember Hurricane Katrina [27], the earthquakes in Christchurch, New Zealand [39] and Amatrice, Italy [35], and the tsunami and consequent nuclear accident in Fukushima, Japan [19]. The COVID crisis is, however, is the first one in recent history to cause ERT to take place on a global scale.

As many others, the University of Houston (a large public university in Texas, USA) suspended all on campus classes in response to the pandemic and switched to remote teaching. The students left on March 6th for Spring Break and were scheduled to return on March 16th. Over the break, however, the University announced that the campus will shut down and all classes will resume online on March 23rd. 2700 instructors had little more than a week to move all classes online for the more than 45,000 enrolled students.

In this paper, we will focus on the impact of the lockdown on the students of the department of Computer Science (CS) at the University of Houston through a survey study. Theoretically, CS departments should be particularly well positioned to move to ERT quickly and effectively, since their faculty and students have, on average, higher levels of computer literacy than other students; moreover, many faculty members already adopt online tools for various tasks, from assigning and collecting homework to sharing course material. In this study, through a survey, we investigated the challenges that students enrolled in CS courses encountered while performing normal course-related tasks. Our results suggest that, despite the theoretical advantages, CS students still struggled completing coursework after moving online, and generally consider the ERT response unfavorably when compared to the initial course format. Our study also shows differences in perception across the student body and helps identify the most vulnerable segments of the population.

General Sentiment on Online Learning

It is not trivial to measure exactly how popular online courses were among the general students’ population before COVID-19 pandemic, even though investments in remote educational technologies were rising, suggesting increasing interest and willingness to adopt this model at least in the universities [26]. However, challenges in designing effective online education remain. The literature shows that, even with plenty of time for the instructors to design the course and for students to prepare for it, students encounter some struggles. In particular, students are not confident in their ability to focus on schoolwork at home, where other distractions are present, and to complete group work [41]. Another challenge is posed by maintaining motivation and pace in absence of a structured schedule [16, 31]. Despite the challenges, some students can thrive in the online classroom. Studies identified self-efficacy, motivation and computer literacy as good predictors of success in online courses [3, 9, 23, 37]. These characteristics are typically found in more mature students (junior and senior level) [23, 30]. Second-generation college students have also been found to join the classroom with higher levels of self-efficacy and better self-regulating skills than first-generation students [38], and are consequently better positioned to succeed in online courses. In addition, female students also seem to do better online than in traditional face-to-face courses, reporting equal or better outcomes than their male counterparts [20, 23, 30].

How about computer science students? A study by Maltby and Whittle [28] shows mixed reviews: 58% of the students interviewed indicated a preference for face-to-face lectures over online ones, believing them to have a higher educational value. Students viewed favorably the fact that online lectures can be self-paced and easy to access, but were frustrated by connectivity issues and the difficulty in asking questions to the instructors. Previous computer and programming experience did not have an impact on the students’ examination performance in courses delivered online. Another study [40] showed no gender-based differences in motivational beliefs and achievement in a self-regulated online programming course, confirming what was found for students in other fields. This is particularly encouraging given the difficulty female students still experience in claiming their place in computing sciences (only 18% of the bachelor degrees in computer science awarded in 2015 went to females, according to the 2018 NSF Science and Engineering Indicators [5].

Learning During the COVID-19 Pandemic

In the short period of time since the beginning of the COVID-19 pandemic, numerous research groups have looked at the effect of the pandemic on the academic population (students and staff alike). Many looked at the impact on the mental and physical health of students and faculties [7, 22, 25, 34], while others focused on its impact on the learning experience. Remote teaching has already proven to be a valuable tool in time of crisis, and the pandemic spurred the growth of Ed-Tech start-ups [13]. However, the sudden lock-down of universities and mass-move of courses online brought several problems to the surface, including some hidden assumptions, such as the idea that all students would have equally easy access to the material. In Portugal, while the country was pushing for a complete lock-down and a move to online education, students were left to face a lack of infrastructure, where 28.5% of them did not even have access to a computer with Internet connection in their own home [36]. Similar situations were found in Canada [17], Ghana [32], Pakistan [1], and in other areas of the world [34]. Not surprisingly, students who were already more vulnerable because of external social and financial constraints were disproportionately affected [17].

Even when access to online courses was possible, the transition was met with some resistance and general dissatisfaction. A recent survey study of a group of Chemistry students at the University of British Columbia by [33] reveals that their experience with transitioning to ERT was mostly negative, with students complaining about lack of motivation and interaction with classmates and instructors . Another study of students’ sentiment on Twitter by [14] found that college students were “likely to express negative feelings towards how social distancing and school closure are affecting their work and study environments”, and a majority of them (81.3%) disliked remote learning. Common issues were difficulty learning subjects that would require physical interaction with the content (i.e., chemistry and biology laboratories) and keeping up with the course pace when attendance to weekly lectures was no longer required, as was already found in pre-COVID studies [16, 31]. The negative sentiment extended beyond the semester, and caused some students to delay their graduation and be less optimistic about their future employment opportunities [2]. Female students in particular were found to have a worse perception of their future outcome, which is interesting considering that, on average, they fare well in online learning environments. Another study by [18], surveying a group of Engineering students, revealed that female students were more likely to opt for a pass/fail grading option when given the opportunity, and described finding themselves with less time available to study because of increased domestic workload.

To our knowledge, the experience of Computer Science students with the COVID lockdown and subsequent transition to remote teaching has not been investigated, although a paper by [12] reports higher confidence in CS instructors in their ability to move to ERT effectively. In this paper, we set out to learn more about how a group of CS students at a large public university dealt with the sudden transition to remote teaching and the possible struggles that followed.

Methodology

To better understand how students experienced the transition from the traditional format to online classes, we asked all students enrolled in at least one course offered by the Computer Science Department at the University of Houston to fill an anonymous survey composed of 21 closed and open-ended questions and scale items. The invitation to participate in the survey was sent at the end of the Spring 2020 semester and responses were collected between May 4th and May 21st 2020. Participation to the survey was completely voluntary. The survey questions were collected online.

The students were asked to identify their degree type, gender, ethnicity, and residence status (U.S. citizens or permanent resident, or international student). The students were also asked about the initial format of their courses and how much their instructors relied on online tools to perform regular course tasks (communicating with students or assigning and delivering homework) to begin with. Another question asked about how lessons were delivered after the transition (synchronous, with or without recording, or asynchronous). In addition, finally, the students were asked to share their impressions regarding the difficulty of performing class-related tasks in the new online format compared with the original format, using a 5-point Likert scale ranging from “Much More Difficult” to “Much Easier”, and to rate how much they believed their ability to learn was affected using a scale from 0 (“I learned much less”) to 10 (“I learned much more”), with 5 being “I learned about the same” as they would have had if the course did not move online. The complete list of scale items is visible in Table 2. With the exception of the demographic (which included a “Prefer not to answer” option) and degree type questions, the students were free to skip questions if they wanted to. We chose to give the option to skip questions, because we believed that it would increase the likelihood that the students would submit at least most of their answers, rather than decide not to submit at all, because they did not feel comfortable answering a question. Note that because each student was asked to fill the survey only once, their responses represent a summary of their experience with all the CS courses they were taking during the Spring semester.

Table 2

Perceived difficulty of performing class-related tasks in the online format

Task	1–MMD	2–MD	3–S	4–E	5–ME	N/A	Avg.
Understanding the lectures/video recordings	13.5%	30.8%	31.8%	12.5%	10.6%	1.0%	2.76
Asking questions in class	19.2%	31.7%	30.8%	8.7%	7.7%	1.9%	2.53
Participating in lab activities	17.3%	22.1%	20.2%	3.8%	1.9%	34.6%	2.25
Attending office hours	17.3%	26.0%	26.0%	11.5%	8.7%	10.6%	2.64
Completing homework assignments	13.5%	18.3%	6.2%	12.5%	8.7%	1.0%	2.84
Keeping up with the course pace	13.5%	33.0%	30.1%	14.6%	7.8%	1.0%	2.70
Interacting with the instructor	18.4%	28.2%	29.1%	17.5%	3.9%	2.9%	2.59
Interacting with the TAs	21.2%	26.0%	29.8%	9.6%	6.7%	6.7%	2.52
Interacting with classmates	22.1%	26.0%	33.7%	6.7%	6.7%	4.8%	2.47

The last column (Avg) shows the average score excluding N/A responses

MMD much more difficult, MD more difficult, S about the same, E easier, ME much easier, NA not applicable

Results

The survey received 105 responses. The demographic composition of the respondents can be summarized as follows:We will start the description of the results with a general summary of the responses. The number of responses for any given question may not match the number of respondents, because answering all questions was not required.

Degree type: 72 undergraduate students, 9 masters, 20 doctoral, 4 post-baccalaureates.

Gender: 68 males, 34 females, 3 preferred not to answer.

Ethnicity: 53 Asians, 20 Hispanics, 15 White, 7 African–American, 2 multiracial, 3 other ethnicity, 5 preferred not to answer.

Residence status: 63 U.S. Citizens or Permanent Residents, 36 International students, 6 preferred not to answer.

The students were asked to identify the level of the course(s) they were currently enrolled in. This is the distribution of students across course levels:Note that the total is greater than the number of respondents, because some students were enrolled in and selected courses of different levels.

1000-level (freshmen level): 13 students.

2000-level (sophomore level): 18 students.

3000-level (junior level): 34 students.

4000-level (senior level): 35 students.

6000/7000 (graduate courses): 29 students.

When asked to identify the initial format of their course(s), 98 students said they were enrolled in at least one face-to-face course, 24 said one of their courses was hybrid, and 15 said it was online. Both undergraduate and graduate students were equally likely to be enrolled in at least one face-to-face course, while the distribution changed for hybrid and online courses: undergraduate students were more likely to describe their course as hybrid (26.3%) than online (10.5%). The opposite was true for graduate students: 24.1% were taking online courses, only 13.8% hybrid courses. To gain more insight into the initial courses’ format, we also asked the students to use a Likert-scale to describe how much their instructors were relying on online tools to perform course-related tasks, such as answering questions or delivering assignments, before the transition. Their responses are summarized in Fig. 1. Then, we asked the students to select the new format of the course among three possible options: synchronous delivery, synchronous delivery with recording, asynchronous delivery. 54 students indicated that at least one of their courses was delivered synchronously, with live video lectures during the usual class time. 66 selected synchronous delivery with recording available after class. In addition, 35 students said that they were taking completely asynchronous courses, with prerecorded video-lectures uploaded by the instructor. As before, the total is greater than the number of respondents, because some students were taking courses in different formats. The distribution of delivery format was very similar for graduate and undergraduate students.

Fig. 1

Students’ assessment of pre-transition use of online tools to perform course-related tasks

The survey continued with 9 scale items to assess the perceived difficulty in performing normal class-related tasks after the courses moved online. The list of tasks is visible in Table 2. The scale went from 1 (Much more difficult) to 5 (Much easier), with a not applicable option if the student never had to perform the task described. The distribution of responses and average scores are also visible in the table.

The last scaled question of the survey asked the students to assess how they felt the current events affected their ability to learn from 0 to 10, with 5 meaning “I learned about the same as I would have regularly”. The final score average was 4.5, with a 2.38 standard deviation. The scores for different students groups are visible in Table 1.

Table 1

Ability to learn score (mean and standard deviation) for different students groups

Group	Mean	Standard dev.
Degree level
Undergraduate	4.30	2.46
Graduate	5.00	2.14
Student level (undergrad.)
Freshman/sophomore	3.11(*)	1.66
Junior/senior	5.11(*)	2.59
Gender
Female	4.59	2.44
Male	4.52	2.35
Ethnicity
African–American	4.17	2.48
Asian	4.75	2.29
Hispanic/Latino	4.05	2.61
White	4.66	2.22
Other/multiracial	5.00	2.24
Citizenship status
Citizen/permanent resident	4.22	2.39
International	5.03	2.43

The students were asked to score their ability to learn remotely on a scale from 0 (“I learned much less”) to 10 (“I learned much more”), with 5 being “I learned about the same” as they would have had if the course did not move online. Statistically significant differences within a group (p value < 0.05) are marked with (*)

Analysis of Confounding Factors

We took a closer look at these results to see if the students’ demographic or other factors affected their responses. In particular, we looked at how degree type, course level, gender, ethnicity, residence status, initial course format and final course format affected the students’ perceived ability to learn and their ability to perform course related tasks.

Degree type We explored differences in responses between graduate (master and PhD) and undergraduate students (including post-baccalaureate). We used Wilcoxon test to evaluate differences in the assessment of their ability to learn and found no significant difference. We used the same test to find differences in the difficulty they encountered with normal course tasks and found that undergraduate students had more difficulty understanding lectures in the new online format, with an average score of 2.61 against 3.14 (p value < 0.05).

Course level We looked more in depth at our undergraduate student population by focusing our analysis on people enrolled in 1000–4000 level courses. We found that students in early level courses (1000–2000) were at a significant disadvantage in several tasks when compared to their seniors (3000–4000 level): they encountered more difficulty in: understanding online lectures (2.14 vs. 2.93), asking questions in class (2.11 vs. 2.74), attending office hours (2.07 vs. 2.94), completing homework (2.53 vs. 3.04), keeping up with the course pace (2.93 vs. 2.18), and interacting with classmates (2.00 vs. 2.95), instructors (2.11 vs. 2.88) and teaching assistants (TAs) (2.04 vs. 2.87), with p value < 0.05 for all comparisons. A significant difference was also found in the impact on their ability to learn (Table 1). No significant difference was found in their perceived difficulty to complete the laboratories.

Gender When grouping students by gender, we found no significant difference in their ability to learn or complete course tasks. Students who answered “Prefer not to say” () were excluded from this analysis.

Ethnicity We used Wilcoxon test and grouped students by ethnicity in a 1-vs-all manner to see if a particular group exhibited a different behavior than the rest, and found two significant relationships: students who identified themselves as Asian had on average an easier time understanding lectures and interacting with their classmates than students from other groups, with a score of 2.94 against the 2.54 of the other students for the first task, and 2.69 against 2.27 for the second task (0.05 < p value < 0.1). Students who identified themselves as White had less difficulty attending office hours (3.75 average score against 2.49 for other students, p value < 0.05). Students who answered “Prefer not to say” (n = 5) were excluded from this analysis.

Residence status When looking at students with a different residence status (U.S citizens and permanent residents versus international students), we found that international students had on average less difficulty keeping up with the course pace (2.91 vs. 2.58, 0.05 < p value < 0.1). Students who answered “Prefer not to say” () were excluded from this analysis.

Initial course format Somehow not surprisingly, we found that the initial course format had a significant difference in the students’ experience following the transition online. Students who declared to be enrolled in at least one hybrid or online course () had a average score of 5.66 (± 2.19) for ability to learn, against 3.97 (± 2.29) for the 73 students who did not (p value < 0.05). Students who had started with an online or hybrid course also found it easier to understand lectures in the new format (3.19 vs. 2.57, p value < 0.05), ask questions (2.87 vs. 2.39, 0.05 < p value < 0.1), complete assignments (3.23 vs. 2.68, p value < 0.05), keep up with the course (3.37 vs. 2.42, p value < 0.05), and interact with instructors (2.93 vs. 2.44, p value < 0.05) and classmates (2.81 vs. 2.32, 0.05 < p value < 0.1) (but not TAs).

Course format after transition We used Wilcoxon test to see if one delivery format among the 3 identified (asynchronous, synchronous and synchronous with recording) was significantly better or worse when compared to the others. We found that the 35 students who were taking at least one course asynchronously had less difficulty understanding lectures (3.06 vs. 2.60, 0.05 < p value < 0.1), but not performing other tasks. People who were taking at least one course synchronously with recording available after class found it easier to ask questions in class (2.69 vs. 2.26, p value < 0.05) and interact with the instructor (2.75 vs. 2.31, 0.05 < p value < 0.1).

Analysis of Open Questions Responses

The survey contained three optional open-ended questions about the students’ learning experience compared to before the transition: (1) what did improve after the transition; (2) what worsened; and (3) students’ general comment about the experience. We used open-coding [11]) to identify and summarize the topics students mentioned in their responses. For open-coding, both authors coded each comment collaboratively. They assigned one or more categories to each comment. With the identification of each new category, we reviewed all prior comments for possible adjustments. Table 3 shows the main categories and their frequencies in the responses in the open-ended questions. Note that the open-ended questions were optional and some participants chose to not respond to them.

Table 3

Frequency of categories in the open-ended questions

Category	Frequency
Offline access	26
Saving commute time	10
Improved focus	6
Flexibility	3
Technology issues	4
Lack of communication with teaching staff	8
Lack of face-to-face	8
Issues with time management	3

In the rest of this section we summarize the favorable and unfavorable students’ experiences in the transition to online learning.

Favorable Experiences

The most frequently mentioned favorable experience that students mention in their responses was offline access to the course materials, which seems to have enabled students to follow the courses at their own pace and refer to them as they need.The second most frequent favorable aspect of moving to online learning format according to their comments was not commuting. Given that the University of Houston is located in a large, populous urban area, and a large portion of students at the university commutes every day, students may spend hours commuting and it is not surprising that they would enjoy saving time by not commuting as stated in the following comment.Few students also mentioned positive changes in the structure and clarity in some class communications as a positive aspect in online learning, suggesting that online teaching and lack of face-to-face communication forced some instructors to avoid ambiguous communications and problem statements.Few students also mentioned increased concentration, increased/improved engagements in the class and with the TAs as positive aspects of transitioning online.

“I liked being able to go back and watch lectures when needed”

“Since we didn’t have to commute to school, I had more time for myself to study and focus on getting the homework done”

“The professor sent out a schedule of what topics we were going to cover until the end of the year. Prior to going online, there was no clear path of what this class was meant to cover.”

Unfavorable Experiences

A large number of participants complained about difficulty in accessing instructors and TAs. They experienced difficulty in communicating with the instructors or TAs, as described by the following student:Technology-related issues were the second most mentioned topic, wherein students found difficulty in using or adopting the technologies in the online classes, e.g.,While online classes offered more flexibility to students to manage their education, participants expressed difficulty in managing time and attending classes. It seems that some students can benefit more from rigid structure of traditional physical classes, e.g.,Several participants expressed dissatisfaction in inadequacy of synchronous classes in simulating physical classes, as they do not allow them to participate and engage in the class as they would in physical classes.Some participants also found lack of face-to-face feel unsatisfactory that negatively impacted their concentration and motivation.

“TAs never responded to emails with questions. I really felt adrift...after we went online.”

“Going to lecture was pretty bad with Blackboard Collaborate. Paying attention was also a bit harder, because online lectures were not as engaging due to not being in person. This does NOT reflect badly on the professor. The professor I had taught fine regardless of medium, but the medium made it harder to learn a bit.”

“It became somewhat more difficult to compartmentalize study and the rest of life.”

“I am generally favored for my class participation and attention. That was nullified due to the typical lag in group responses to questions during live lectures. People that tend to be alert and answer fast are just copied and left to blend into the mix of lagged answers.”

“Maintaining motivation and an immersive college experience were the two aspects that suffered the most during the transition to online classes.”

Discussion

The results collected in this study suggest that the student population overall went through some moderate struggles during the transition. The assessment of how the new courses format affected their ability to learn was 4.5, not significantly below the score of 5 that represents comparable learning outcome with the format before the campus closure. Similarly, as shown in Table 2, in 5 out of 9 scale items asking about the difficulties encountered in performing normal coursework, the most popular answer was “About the same”, suggesting that overall many students felt capable of completing these tasks. However, for all the 9 scale items, the average score is below 3, indicating a distribution skewed toward negative responses. “More difficult” was in fact the first or second most popular answer for all tasks listed, showing that a significant portion of students had some difficulty performing regular course-related tasks. Asking questions in class and interacting with instructors, TAs and classmates registered the worse responses overall, although the lowest score belongs to “Participating in lab activities” (we are singling out this task because of the high number of N/A—not all students were enrolled in a course with a laboratory). Understanding lectures, completing assignments and keeping up with the course pace were the least affected.

These results are not surprising: students struggled more with tasks that benefit the most from face-to-face interaction. In addition, the highly hands-on nature of lab sessions made them particularly difficult to move online while maintaining the same level of effectiveness. Students were more comfortable (or, at least, not affected as much) with tasks they could complete on their own, such as homework. We believe that this is also due to the fact that the majority of the respondents were mature students with at least a few years of college behind them (junior, senior and graduates). The fact that more experienced students were able to self-manage and complete the coursework independently is in line with literature findings [23, 30]. These findings were also confirmed by the results of the Wilcoxon test, which showed early level students significantly worse off than their more senior counterparts in performing the majority of course-related tasks.

Another result in line with existing literature is the absence of significant differences in responses based on gender. As we know, female students are still the minority in many Computer Science departments, are less likely than males to take advantage of learning opportunities, such as internships or lab assistantships, and are more affected by retention problems [4, 6, 10]. We were glad to see that, in this scenario, they did not find themselves in a worse position than their male classmates.

Some more unexpected results came from the analysis of the impact of ethnicity and citizenship status. As we have described in the “Results” section, Asian respondents found it less challenging to understand the material and interacting with their classmates. The first finding may speak to a higher level of comfort for this group to study independently. The second result is more difficult to interpret: it is possible that Asian students were more capable of maintaining their network while studying remotely, but it could also mean that they did not see a difference, because they have less interaction with their classmates to begin with. We suspect it may be possible that students overall were not particularly affected by a lack of interaction with their peers, because this complaint emerged only once in the open-ended questions. We have also seen that white students struggled less than the others attending office hours, and even found it easier on average. Online office hours and not having to commute were favorites in the students’ comments, so it is possible that this segment of the population benefit particularly from the new office hours format. It would be interesting to see if white students of our department are less likely to live on campus and more likely to commute. This could explain why the new format was so popular amongst them.

International students were found to be better capable of keeping up with the course pace. Interestingly, the majority of international students are also graduate students (22 out of 36 respondents), so they may have the advantage when it comes to studying independently.

The initial course format had a much larger impact than the format after the transition. Students enrolled in hybrid or online courses had a better outcome than those only enrolled in face-to-face classes. The new course format chosen by the instructor had a more limited effect. Students found it easier to interact with the instructors and ask questions in class if the course was delivered synchronously, but a recording of the lecture was also made available. This suggests that students able to review the lecture multiple times could come up with questions and approach their instructors with more confidence. We have also found that students enrolled in asynchronous courses found less difficult to understand the material, although this may have a different explanation: out of the 35 students in this group, only 4 were freshmen. It is possible that instructors of more mature students felt more comfortable with the idea of their students studying the material independently and opted for asynchronous delivery more often.

Conclusions

In this study, we have surveyed Computer Science students at a large public university to better understand their perceptions on the transition to online teaching caused by the COVID-19 pandemic and subsequent campus lock-down. We compared the impact of the transitions on different groups of students, and used open coding to gain more insight into what students found particularly challenging and what they liked.

The results of the study show that the students found various course-related tasks, on average, moderately more difficult to complete. Tasks such as asking questions during video-lectures and interacting with instructors were amongst the most affected. The transition seems to have had a greater impact on students of lower level courses, which confirms previous studies that identify students at the beginning of their academic career to be at a disadvantage in online courses. This is probably caused by the more independent nature of remote learning, which requires higher levels of motivation and self-efficacy. Differences were found also across race and residence status, but not gender. Some differences were caused by delivery format (synchronous or asynchronous), but the greatest impact was caused by the initial format of the course: hybrid and online courses, and courses, where the instructor was more reliant on online tools before the transition to perform course-related tasks (delivering homework or communicating), resulted in a better experience for the students.

In the students’ comments, we observed that some aspects of online learning were quite popular: not having to commute, being able to access recording of lectures and having more time to organize their coursework independently were all seen favorably. On the other hand, damaging side effects included difficult communication with instructors and TAs, issues with the technology adopted to deliver lectures, and structure their daily schedule.

Overall, our results paint a similar scenario to the one found when looking at students of other disciplines, indicating that Computer Science students, despite the theoretical advantage when it comes to learning online, experienced the same kind of struggle with the sudden transition to ERT. Our results also show that beginner students were particularly affected; instructors should keep this in mind when approaching students of lower level courses during ERT, possibly favoring compassion and flexibility over extreme rigor, as suggested in [18].

The University of Houston, as well as many others, is offering the majority of its courses online for Spring 2021. The COVID-19 pandemic will not be the last time that we will be required to quickly transition to ERT (for instance, our city is at high risk of hurricanes and flooding, which have caused campus closures). Thanks to the results of our study, we believe that a better remote teaching experience could be achieved by preemptively focusing on the following practices:Different demographics have shown to have different strengths and weaknesses in remote teaching scenarios. Further studies may help to better understand the characteristics shared by the various groups of students, to provide a more tailored experience when moving to Emergency Remote Teaching is required.

Training instructors to familiarize with some remote teaching tools and encouraging them to adopt some of them in their courses even when they are being delivered face-to-face (e.g., a system to assign and collect homework online, or an online forum to communicate with students).

During ERT, maintain clear and frequent communication with students to ensure that they are keeping up with the course material and their motivation is not weaning (e.g., frequent, low stakes quizzes; surveys to spot problems with the course delivery). This is especially important for freshman and sophomore students.

Scheduling online office hours with TAs and instructors even during regular course delivery. The students seem to appreciate this option (especially those who commute), and they will be already familiar with it if a move to ERT is required.