Preservice teachers' perceptions of learning environments before and after pandemic-related course disruption.

When the 2020 semester began in the USA in January, it was unimaginable that the near-total closure of educational system across the globe would become the new normal. To mitigate the spread of the COVID-19 virus, teaching faculty hastily converted to an online learning environment in order for instruction to continue. This mixed-methods study used the What Is Happening In this Class? (WIHIC) questionnaire and analysis of student course evaluations to explore changes in student perceptions of learning environments from before to after the switch to remote learning because of the pandemic. Students perceived a statistically-significant decline in student cohesiveness, teacher support, involvement, task orientation and equity, with the largest decline of 0.56 standard deviations occurring for student cohesiveness. Qualitative comments illuminated reasons for these declines and suggested ways to mitigate declines in the future.

Introduction

Because of government closure of educational institutions to contain the spread of COVID-19, education institutions internationally quickly converted to online learning environments during 2020 (UNESCO, 2020). In a recent book entitled An educational calamity: Learning and teaching during the COVID-19 pandemic, Reimers et al. (2021a) provide 10 case studies of the educational impact of COVID-19 in numerous countries to identify options for mitigating and redressing this impact. The picture painted by these case studies is that 2020 involved a “remarkable collapse of educational opportunities to learn, robbing the current generation students of the opportunities to gain the same skills that their counterparts were able to develop the previous year. Unless effective efforts to correct such loss are put in place soon, the future for those students... will be less hopeful than it would have otherwise been (Reimers et al., 2021b, p. 20).

The USA university involved in our study of changes in the learning environment during pandemic-related course disruption utilizes Canvas, which is an online learning management system that allows instructors to create video lectures and to share video and text content, tests and quizzes, discussion boards, etc. Although teaching staff at this university had prior access to Canvas to support instruction, many instructors were unfamiliar with some of the interactive features used in both synchronous and asynchronous learning and/or had never taught online courses. In addition, there were little time and opportunity for thoughtful, deliberate consideration about course design and instruction.

Several researchers have investigated the effects of the sudden shut-down of face-to-face instruction in schools around the world as a result of the pandemic. For example, Ma et al.’s (2021) study of recent pandemic-related changes in the education system and their effects on students’ mental health concluded that many 7–15-year-old students in China are suffering post-traumatic stress disorder (PTSD) and depression symptoms as a result of the pandemic. Although relatively few researchers have focused solely on instruction and learning, Middleton (2020) addressed how disruptions and issues in learning during the pandemic affect student assessment. Whereas Wang’s (2020) article reflects on best practices for online instructional design, other articles describe how curriculum and instruction have changed to online format and consider some resulting issues (Brown, 2020; Fackler and Sexton, 2020; Graham et al., 2020; Kaden, 2020).

Because of the unfortunate timing of the disruption in the USA in the middle of the semester, data collection and corresponding research stopped. However, Rahayu et al. (2021) accomplished the development and validation of the Online Classroom Learning Environment Inventory (OCLEI) in Indonesia and measured learning environments in newly-created online learning situations. Additionally, Howard et al. (2020) investigated teacher self-efficacy beliefs and perceived institutional support associated with online instruction, which also are salient to teaching during the pandemic. When Davis (2021) investigated the mental health of parents who were forced to become ‘proxy educators’ during the pandemic as schools transitioned to distance learning, they identified elevated mental distress among parents.

Given the uncertainty of future course format guidelines, research about student perceptions during conversion to online learning during 2020 potentially could provide important data for consideration during the future design of online instruction. The purpose of this study was to identify changes in student perceptions of the instructional learning environment during the sudden and unplanned transition to an online environment as a result of the COVID-19 pandemic. Specifically, the study attempted to answer the following research questions.

Did students’ perceptions of their learning environments change during conversion from a face-to-face to an online instructional environment in response to the COVID-19 pandemic?

What factors did students identify as contributing to changes in their perceived learning environments during the conversion to remote learning?

Theoretical perspective: field of learning environments

The field of learning environments research has its historical roots in the first half of the twentieth Century when Lewin’s (1936) introduced a ‘person–environment paradigm’ in which behavior depends on a combination of person and environment. This paradigm was expanded by Murray (1938) who further distinguished between the environment discerned by an external observer (‘alpha’ press) and the environment perceived by internal participants (‘beta’ press). The concept of beta press, internal participants’ view of environment, was further separated into ‘personal’ and ‘consensual’ beta press by Stern et al. (1956). Specifically in the field of education, the foundations of the field of learning environments were laid by Walberg and Anderson (1968) and Moos and Trickett (1974). Since the time of these pioneering contributions, the considerable progress, expansion and internationalization of the field of learning environments has been traced by Fraser (1986, 1994, 2012, 2014, 2018) and Zandvliet and Fraser (2018).

Our study utilized the field of classroom learning environments as a framework and source of criteria of effectiveness for evaluating the impact of the sudden shift to remote learning after the pandemic-related quarantine of the campus. Within learning environment research, there is a long and rich tradition of using classroom environment criteria as measures of the effectiveness of myriad educational innovations or programs. For example, learning environment characteristics have been used to evaluate outcomes-focused education (Aldridge & Fraser, 2008), mixed-mode delivery (Koh & Fraser, 2014), using games to teach mathematics at the university level (Afari et al., 2013), the professional development of inservice teachers (Nix et al., 2005), anthropometry activities in science (Lightburn & Fraser, 2007), computer-assisted learning (Teh & Fraser, 1995), innovative methods for teaching mathematics (Spinner & Fraser, 2005), reality pedagogy (Sirrakos & Fraser, 2017), outdoor learning environments (Zaragoza & Fraser, 2017), middle-school science laboratory activities (Wolf & Fraser, 2008) and alternative science topic arrangements (Long et al., 2020). The study reported in this article adds to the field of learning environments because it is probably the first to use learning environment criteria in investigating the impact of the sudden disruption to instruction caused by the COVID-19 pandemic.

What Is Happening In this Class? (WIHIC)

The field of learning environments research enjoys “a variety of economical, valid and widely applicable questionnaires that have been developed and used for assessing students’ perceptions of classroom environment. Few fields in education can boast the existence of such a rich array of validated and robust instruments which have been used in so many research applications” (Fraser, 1998, pp. 7–8). Reviews show that these surveys have been extensively used around the globe (Fraser, 2012, 2014). Since the historical use of the Learning Environment Inventory to evaluate Harvard Project Physics (Walberg & Anderson, 1968), numerous instruments have been developed, validated, cross-validated and used worldwide: Science Laboratory Environment Inventory (SLEI, Fraser & Lee, 2009; Fraser et al., 1995; Lightburn & Fraser, 2007); Constructivist Learning Environment Survey (CLES, Koh & Fraser, 2014; Nix et al., 2005; Taylor et al., 1997); Questionnaire on Teacher Interaction (QTI, Fraser & Walberg, 2005; Wubbels & Brekelmans, 2012) (see reviews of Fraser, 2012, 2018).

Currently, the most widely-used learning environments questionnaire is the What Is Happening In this Class? (WIHIC, Aldridge et al., 1999; Fraser et al., 1996), which has been so frequently used that Dorman (2008) commented that it has achieved ‘bandwagon status’. While there are several learning environments surveys, the WIHIC was chosen as the most appropriate for this study. The WIHIC was initially developed by Fraser et al. (1996) in order to create an economical learning environments instrument by combining salient scales from well-established surveys with additional scales updated for current relevance (Aldridge et al., 1999). We chose the WIHIC as being particularly well suited for our study because of its successful prior use with university students (Afari et al., 2013) and preservice teachers (McMinn & Aldridge, 2020).

Methods

Sample

The participants in this study were enrolled in teacher education programs at a large research-intensive public university in Texas. These students were participating in face-to-face instruction at the beginning of the spring 2020 semester (January) and all courses were converted to 100% online in mid-March (mid-term) as a result of the COVID-19 pandemic. A total of 230 participants self-identified their ethnicity as 53.9% White, 19.6% Hispanic, 15.2% Black, 6.1% Asian, 1.3% Native American, and 3.9% as Other. The sample self-reported gender as 73% female and 27% male.

Quantitative data: What Is Happening In this Class? (WIHIC)

Quantitative data were obtained by administering a modified five-scale version of the WIHIC questionnaire (Aldridge et al., 1999) to assess Student Cohesiveness, Teacher Support, Involvement, Task Orientation and Equity with 35 items with a 5-point frequency response format (Almost Never, Seldom, Sometimes, Often, very Often). The WIHIC previously has been validated and used in a wide variety of settings, as illustrated by Fraser’s (2018) tabulation of 21 studies in 13 countries and 12 languages. Additional free-response questions were asked to gather specific student feedback and insights about the changes to the spring course’s format.

Because of the unplanned nature of this semester and research, all data collection occurred at one time. In May, all students in the university’s College of Education were invited to complete two surveys—one in relation to their learning environment perceptions for their face-to-face course (January–mid-March) and one for their perceptions after the online conversion (mid-March–May). From the 256 students who responded, 230 completed 100% of the questionnaire.

Table 1 provides details of the WIHIC scales as used in this study. Although the most-recent versions of WIHIC incorporate the seven scales of Student Cohesiveness, Teacher Support, Investigation, Involvement, Task Orientation, Cooperation, and Equity (Skordi & Fraser, 2019), we omitted Investigation and Cooperation in our study because of their lesser relevance to the specific course and to achieve economy of administration time.

Table 1

Scale description and sample items for each WIHIC scale used

Scale	Description	Sample item
Student Cohesiveness	The extent to which students are friendly and supportive of one another	I work well with other class members
Teacher Support	The extent to which the teacher helps, encourages and is interested in the students	The teacher considers my feelings
Involvement	The extent to which problem solving and inquiry skills are emphasized	I discuss different answers to questions
Task Orientation	The extent to which it is important to complete activities and stay on the subject matter	Class assignments are clear so I know what to do
Equity	Extent to which the teacher treats students equitably	I am treated the same as other students in this class

Qualitative data: Student Perceptions of Teaching (SPOT)

The qualitative component of this study consisted of analysis of student responses to three questions added by the university’s College of Education to the student course evaluation form or SPOT:

This semester was unusual because of the health crisis. Was there anything about the instructor and/or the structure of the course that was helpful to you for having a successful semester?

What do you need the [department] (where your course is housed) to know that might improve your educational experiences in the future?

What were some of the challenges you faced in your courses this semester that were unique to the current health crisis?

These qualitative data included over 4500 individual statements which were examined and categorized by four education experts to identify themes for discussion. A consensus model was employed to thematically analyze participant responses. Using a triangulation approach, data were used to identify and categorize participants’ perceptions before and after conversion to a fully-online learning environment.

For this study, we used qualitative thematic analysis of student comments based on Braun and Clarke (2006). First, any identifying information was removed from all students’ comments before uploading them into the Atlas.ti software package for data management. We used a deductive approach based on the WIHIC framework (Aldridge et al., 1999) to develop the initial and holistic codes (Saldana, 2016). This deductive approach allowed a more-detailed analysis of the data (Braun & Clarke, 2006) in terms of the WIHIC framework. These codes were then refined throughout the process using constant comparative analysis and a hybrid inductive approach to the coding that either developed new codes or expanded the a priori codes (Fereday & Muir-Cochrane, 2006). For example, initial codes for Task Orientation were eventually split into more-specific codes related to Certification, Reminders, or Staying on Task. New codes that connected across the WIHIC framework include more-descriptive/conditional codes such as Kindness, Flexibility, or Engagement. These specific codes were then used to compare responses across the three prompts and across the five scales of the WIHIC framework.

Quantitative analysis and results for WIHIC

Table 2 reports a comparison of students’ learning environment perceptions before disruption (face-to-face) and after disruption (online format) associated with COVID-19 in terms of the average item mean and average item standard deviation for each of the five WIHIC scales. Table 1 also reports the statistical significance of differences between before-disruption scores and after-disruption scores on each WIHIC scale for the entire sample of 230 students. MANOVA with repeated measures was used to check the statistical significance of before–after differences for the set of WIHIC scales as a whole and, to reduce the Type I error rate, the univariate ANOVA results for each individual WIHIC scale were only interpreted after the multivariate MANOVA established significant multivariate differences using Wilks’ lambda criterion. Additionally, Cohen’s d effect sizes were used to describe the magnitude of these differences in standard deviation units, as distinct from their statistical significance, as recommended by Thompson (1998).

Table 2

Average item mean, average item standard deviation and difference between learning environments before and after pandemic-related disruption (ANOVA results and Cohen’s d effect size) for each WIHIC scale

Scale	Average item mean		Average item SD		Difference
	Before	After	Before	After	F	Effect size d
Student Cohesiveness	3.87	3.36	0.79	1.01	93.70*	0.56
Teacher Support	4.33	4.16	0.75	0.95	22.37*	0.20
Involvement	3.72	3.50	0.85	1.04	18.32*	0.23
Task Orientation	4.48	4.30	0.67	0.86	27.24*	0.23
Equity	4.49	4.28	0.62	0.85	29.49*	0.28

N = 230

*p < 0.001

Table 2 shows that all five WIHIC scales showed a statistically-significant decline in learning environment scores after the conversion to online learning whose magnitudes ranged from d = 0.20 to d = 0.56 standard deviations. The largest decline occurred for Student Cohesiveness (d = 0.56), suggesting that students felt that they had lost the feeling of being part of the class. Cohen (1988) suggests that effect sizes greater than 0.8 are large, greater than 0.5 are medium, and greater than 0.2 are small.

Qualitative analysis and results for SPOT

To explain the nature of the declines in the WIHIC scores, responses to the three items related to COVID-19 were analyzed for relationships to the WIHIC scales. Although SPOT data originally were not intended as a part of this study, we still analysed them to provide qualitative insights to complement quantitative insights based on changes in WIHIC learning environment scores linked to the sudden shift to remote learning. Table 3 reports the number of student comments that we coded into each of the five WIHIC scales used in this study. Each comment was coded as either positive, negative, or neutral regarding the scale to which it was assigned. In total, 3853 comments were associated with various WIHIC scales.

Table 3

Frequency of positive, negative and neutral Student Perceptions of Teaching (SPOT) comments for each WIHIC scale

WIHIC scale	Frequency of SPOT comments
	Positive	Negative	Neutral
Teacher Support	825	116	0
Task Orientation	594	150	45
Involvement	28	20	59
Equity	24	93	0
Student Cohesiveness	46	11	25

Student statements associated with Teacher Support

Teacher Support had the smallest decline of 0.20 standard deviations among the five WIHIC scales. Analysis of the student SPOT responses offers an explanation of why the decline in scores for this scale was mitigated to some degree. The most-common theme among statements related to Teacher Support was communication. Students identified positive teacher support in connection with frequent announcements, emails, and Zoom meetings, as well as quick responses to student emails. Some examples include.

She was very consistent and kind when it came to emails with questions and feedback on our assignments. I am genuinely so happy that I had her as a professor during this time; she does her job very well and so obviously cares for her students.

The instructor understands the human condition and opened a discussion about our educational experiences as teachers, parents of students, etc... during this time. It was nice to be able to share what was and wasn't working and view the perspective of other classmates.

There were several moments when I have felt as if I did not have my full mental capacity to keep up with the coursework. However, thanks to the support of my professors, I have been able to keep moving forward and to give my best.

Overall, SPOT comments related to Teacher Support were mostly positive with 88% of responses praising the instructors. Despite the positive comments towards many of the instructors, there was a substantial number of negative statements which were shared by the students and which help to explain the change in Teacher Support. While the positive comments towards instructors outnumbered negative comments, it is readily apparent that a handful of instructors can significantly impact the learning environment. Selected negative comments include:

Communication about expectations during the health crisis was not great. New modules were sent to us to do and every single week they were late being sent out. The modules felt like busy work. I felt in the dark about things during the health crisis and wish there had been leadership that checked in regularly (weekly at least) even without updates but just to speak to the weekly module expectations, reassure us, etc.

Every other class I took this semester discussed with the students how the course can be structured moving forward to help us learn and complete assignments. S did not do this. She wrote "I told you" in the email instead. Like she was doing us a favor.

Student statements associated with Task Orientation

Along with Involvement, Task Orientation had the second smallest decline in scales scores of 0.23 standard deviations. Similar to Teacher Support, comments about Task Orientation were 75% positive. Most Task Orientation comments focused on the challenge of having added work in the online/remote format or perceived instructor flexibility regarding how and when assignments are completed. Students also expressed concerns about connection between their coursework and their certification requirements or future needs in the classroom. Much like Teacher Support, several Task Orientation statements expressed positive support:

The professor was extremely forgiving with due dates and opportunities: allowing extra time for students to complete assignments and being understanding when giving extensions.

She gave us very flexible deadlines which helped me to complete all the work with very high quality.

She provided additional support and accommodations such as accepting late work when students are under stress with work. Very compassionate educator who allowed a comfortable amount of time to complete work.

It is likely that the instructors who were perceived as being supportive helped to mitigate the decline in Task Orientation. However, negative comments illustrate why there was a decline in Task Orientation:

The added discussion boards three times a week were difficult to keep up with.

Professor H was very absent. Students who expressed concerns over field experience projects were not met with understanding or real solutions but were redirected to figure it out on their own. There was a lot of added and unnecessary stress from this class. As a class, we had to support each other and try to understand the questions that we had on our own.

More homework was added from other classes onto Canvas, making it more difficult to separate my work and home life balance.

Student statements associated with Involvement

There was a significant but small decline in Involvement of 0.23 standard deviations. Student comments related to Involvement focused on the difficulties associated with learning in a remote environment, such as the reduced ability to ask questions about the material or assignments or to engage actively in practical lessons, such as working with school district administrators or presenting lesson plans. Student comments included:

My schedules completely changed. I was now sitting on my own watching lectures without being able to ask questions in person.

It took a bit of adjusting switching for online classes, especially since I was no longer hearing someone introduce and discuss a topic.

Not having the opportunity to present in person our lesson plans.

I think some assignments were harder to do because of not being in class and being unable to ask questions in person.

Student statements associated with Equity

The significant decline in Equity scores was 0.28 standard deviations, which is small and suggests that some effort was made by instructors to mitigate these concerns. Most student comments addressed Equity by identifying pandemic-related challenges outside the classroom. Comments included loss of employment, increased family responsibilities, working from home, and not having the appropriate technology to be successful. There were several comments detailing the added stress felt by some students:

Connecting and learning new things all at one time. I suffer from ADD and anxiety, and so I really had a difficult time.

In addition to completing this course, I was also responsible for facilitating daily online instruction for my students, overseeing two of my own children's online schoolwork and continuing overseeing/managing department and district directives for the teachers on my campus. It was very overwhelming and stressful.

Juggling financial hardships, my three boys each having online learning of their own, trying to be everything to everyone at home, and doing my own work as a student and teacher were stressful.

My home life exploded (both of my parents ended 7-year relationships, everyone lost their jobs, and entertaining and getting primary-school brothers to do their work was hard. I moved four times and had to get wi-fi at home (new and expensive), making it difficult to focus and find time for school.

I had a lot of stress with the crisis because I had to juggle work, my family, and my classes while still trying to be safe. I was so thankful that some of my professors made it easy for all of us and lowered the workload and made the exams easier to guarantee students' success.

I got stuck in another state without a car, clothes, instruments, enough books, etc... I lost work, summer work, applied for unemployment, dealt with other things, etc.

The only challenge I faced was time management because, in our household, we had one computer. So, I needed my children to use my computer while I did my reading of the book. Then I had to work at night to complete all assignments on time. There was also a week in the beginning phases that the internet would go down and I would get frustrated.

The students whose comments are given above were undoubtedly impacted, while other students did not face similar situations, thus creating a less-equitable learning environment. However, some instructors appeared to recognize these situations and took steps to accommodate the difficulties experienced by some students, as illustrated by comments such as those below:

He was very flexible about the Zoom meetings and understood that people suddenly had different time obligations due to work at home.

Dr R is such a great motivator. Growing up as an ESL student and now a teacher, I always am scared of doing something wrong. Dr R always left me positive comments that made me feel very, very good. She truly gave me confidence in myself and made me feel that I am a good student.

Having the instructor still using equity cards helped to ensure we participated. Using the breakout rooms was also helpful.

The instructor was super helpful in giving grace for late assignments due to the need for my work life to be on the computer and via video meetings daily. I have hearing loss and I suffer from brain fatigue, and so I had trouble with comprehension of material after a long work day.

Student statements associated with Student Cohesiveness

The largest effect size (0.56 standard deviations) was for Student Cohesiveness. Indications of dissatisfaction with low levels of post-quarantine Student Cohesiveness are reflected in comments about missing classmates and the difficulty of completing group work virtually. Many students commented on the difficulty and overall negative experiences in trying to work with classmates after moving to remote teaching during the pandemic. Some examples include:

I think large group discussions don't work in a video class.

Group assignments when learning online are more difficult because they require extra work on part of the students to communicate with others outside the classroom.

Some activities were not useful without peer-to-peer interaction.

The partner work was a lot harder because everything had to be done through technology.

I really had to rely on my own interpretation of the reading instead of getting to know what my classmates thought.

It was hard to message all the people in class separately to get one assignment done. Also having a large group project online is complicated.

Communication fell off between the new group-mates because we did not have classes after spring break to form relationships.

The class translated well to online, but I did miss having lectures and conversations as a class.

Having our class time taken away was the most-challenging aspect of our situation because, as students, we were unable to ask questions right away, enjoy social interactions, and participate in group-building activities.

Some of the activities were not useful without peer-to-peer interaction.

The partner work was a lot harder because everything had to be done through technology.

As with the other scales, not all the Student Cohesiveness commentary was negative. More than half (56%) of the comments were positive and indicated an explicit effort on the part of many instructors to maintain a cohesive classroom environment. There was ample mention of the use technology such as Zoom, online discussion boards, and various Google applications:

We did a lot of group work in class and so the transition from that was difficult, but the instructor helped to make it easier.

I liked how, on one day, we would work in groups and, in the next class, we would come together and have a class discussion like we used to when we were face-to-face.

She even offered incentives for doing well and that helped us to bond as a class. Her encouragement of all of us working together to succeed as a group was nice.

Also, working in groups made things easier to understand.

One comment captured the challenge faced by students and instructors alike in maintaining Student Cohesiveness in a remote setting:

I want the instructors to know that online and virtual can still include authentic connections and relationships which are vital to learning. I would love to have opportunities to connect with professors and classmates in a way that is more authentic than discussion boards.

Discussion, limitations, and further research

With the unprecedented disruption to education worldwide in 2020 because of the COVID-19 pandemic (Reimers et al., 2021a), many face-to-face learning environments were changed hastily to online teaching/learning. In the study reported in this article, we used quantitative and qualitative methods to identify and explain changes in the learning environment of a university course that were attributable to pandemic-related disruption.

Regarding our first research question about whether students’ perceptions of their learning environments changed during the transition from face-to-face to online environment learning environments in response to the COVID-19 pandemic, quantitative WIHIC data revealed statistically-significant deteriorations for all five learning environment scales. The magnitude of deterioration was appreciable for Student Cohesiveness (0.56 standard deviations), but small for the other four scales of Teacher Support, Involvement, Task Orientation and Equity (d = 0.20–0.28 standard deviations). Because the COVID-19 pandemic continued in the USA, the university continued to deliver most courses, including all courses involved in the study, remotely through the Fall 2021 semester, therefore providing an opportunity for course instructors to make modifications to improve their online learning environments. For example, because the largest change in learning environment occurred for Student Cohesiveness, instructors could pay particular attention to encouraging their students to be friendly to and supportive of each other (e.g., designing assignments that require groups of students to work cooperatively via online meeting platforms such as Zoom).

There is potential for university instructors to strive to improve their online learning environments by utilizing feedback from questionnaires such as the WIHIC to assess students’ perceptions of their learning environments. This simple approach involves assessing students’ perceptions of both their actual and preferred learning environments, identifying actual–preferred discrepancies for specific scales, and designing interventions to reduce the discrepancies (Fraser, 1999). Over time, this simple approach evolved into more-sophisticated methods for scoring questionnaires and graphically depicting feedback about students’ perceptions of actual and preferred learning environments. For example, case studies of successful environment improvement studies have been reported for teacher action research (Aldridge et al., 2012) and a principal-led whole-school improvement effort (Aldridge et al., 2021).

Our second research question concerning factors that students report as contributing to changes in perceived learning environments during the conversion to remote learning was answered with our qualitative SPOT (Student Perceptions of Teaching) data. SPOT comments, classified as positive, negative or neutral, provided valuable insight into what did and did not work in promoting a positive remotely-delivered learning environment.

This study was limited in scope to a single college within a single university. Future research with larger and more-diverse samples would make it feasible to cross-validate WIHIC scales when used in remote learning environments, as well as to improve the generalizability of our findings concerning students’ perceptions of online instructional settings to other colleges and other institutions.