Suddenly fully online: A case study of a blended university course moving online during the Covid-19 pandemic.

This study investigates the way students experience synchronous online and offline meetings in university education, using a qualitative approach. The context is a course in teacher training that was forced to go fully online during the Covid-19 pandemic while the course was almost halfway to completion. After completion of the course, a semi-structured interview on the experiences during the online and offline synchronous meetings was conducted with six randomly selected students out of a total of 24. The interview was transcribed, coded, and second coding was performed. The results reveal that students generally prefer offline meetings because of higher levels of all types of student interactions (student-student, student-content, and student-teacher), and the higher level of relatedness. However, students do appreciate the possibility to perform small group work outside of class in an online environment. The results support the application of blended learning. Implications are discussed.

Introduction

Education is based on students, teachers, content, and interactions between the three (Bernard et al., 2009; Garrison, 1989). It is generally assumed that if one of the student-interactions (student-student, student-teacher or student-content) is of a high quality, deep learning will occur, even if the other two interactions are significantly reduced or even minimized (Anderson and Garrison, 1998; Anderson and Kanuka, 1999; Garrison and Vaughan, 2008; Stein and Graham, 2020). Accordingly, one could envisage an effective university course based solely on a high quality student-content interaction. Traditionally, this would mean students studying textbooks. In the digital age, many forms of online content would supplement or replace the books. Students might follow a Massive Open Online Course or MOOC, even though actual MOOCs usually do include at least some level of online student-teacher or student-student interaction (Littlejohn et al., 2016).

The reported high dropout rates (up to 90%) of MOOCs, however, support the stance that focusing solely on student-content interaction may not be the most effective educational strategy, at least not for all students (Li and Moore, 2018; Rivard, 2013). Indeed, most university educators strive for a balance between the three student interactions, since student-content, student-student, and student-teacher interactions all have their specific advantages and disadvantages, for example in terms of availability. “Getting the mix right” has been a matter of considerable debate and research over the past decades (Anderson, 2003; Daniel and Marquis, 1979; Stein and Graham, 2020; Wagner, 1994).

A relatively recent approach to finding this right mix is blended learning (Garrison and Vaughan, 2008). Even though definitions vary, blended learning involves having a significant portion, up to 90%, of all interactions taking place online. Blended learning also incorporates some form of offline and thus necessarily synchronous interaction in face-to-face or physical meetings. The idea is to get the best of both worlds by offering almost unlimited student autonomy in terms of place and time for student-content interaction whilst supplementing this with the best offline education has to offer in terms of high-quality student-student and student-teacher interactions in the physical realm (Stein and Graham, 2020). Indeed, meta-studies show that the blended learning approach results in significantly higher learning outcomes when compared to more traditional forms of classroom education, even though the reported effect sizes are modest (Bernard et al., 2014). Over the past decade or so, most universities have started to offer their courses in blended form.

But now consider the situation where a global disruption, such as the 2020 Covid-19 pandemic, renders the high-quality offline events essential to blended learning not only impractical, but even unlawful. Most universities in the Netherlands responded to the change in situation by replacing the offline sessions by alternative synchronous, but online, events. This move was facilitated by the many available videoconferencing tools (Bakker and Wagner, 2020).

With all synchronous education thus suddenly becoming online, the issue of effectiveness of student interactions and other questions on the educational experience during synchronous education are raised. To what extent can a synchronous fully online meeting replace an offline one? An interesting opportunity for a case study arose when a blended university master's course in the field of teacher education was forced to go fully online almost halfway through, after 4 out of a total of 9 synchronous meetings. This opened up the possibility to investigate the student experiences during online and offline synchronous meetings, keeping other variables equal. This study does not aim to evaluate either the transition as such or the experiences of online versus offline education in general. It explicitly focuses on the way the students experienced the offline and online synchronous meetings of the same course, with course content, structure, schedule, student and teacher population, and assessment remaining the same.

The research question is:

How do students experience the learning environment during synchronous online and offline meetings within the same master course?

Theoretical background

Theoretically, deep learning depends on at least one of the basic student interactions, student-student (S–S), student-teacher (S-T) and student-content (S–C) being of a high level (Anderson and Garrison, 1998; Bernard et al., 2009). Examining more closely the concepts “interaction” and “high level” in this statement, the definition of interaction as put forward by Wagner (1994) is adopted: “Reciprocal events that require at least two objects and two actions. Interactions occur when these objects and events mutually influence one another” (p. 8). This definition is somewhat broader than earlier definitions which limited interactions to interactions between persons (Daniel and Marquis, 1979).

But when is a specific interaction considered to be of a high level? As a basic condition in education, S–S and S-T interactions should preferably be voluntary, reciprocal (i.e., two-way), and collaborative in nature (Anderson and Garrison, 1998). A high level is then considered to be attained when the interaction is also meaningful, i.e., deeply and intricately related to the educational task at hand. This may be fostered by employing authentic tasks, i.e., tasks that are open, complex, with real-world relevance, not having one right solution, etc. (Reeves et al., 2002; Woo and Reeves, 2007). Note that this leaves the possibility of including S–C interaction, since authentic tasks can also be designed in this realm.

Each of the aforementioned student interaction types can be incorporated into a learning activity and contributes to the learning experience in a specific way according to its characteristics. A combination of these interactions will thus generally be more engaging for the students than designs that focus predominantly on only one form of interaction (Stein and Graham, 2020).

Discussing briefly the three student interaction types:

Student-Student (S–S) interaction is essential to any collaborative learning task, the core activity in social constructivism (Vygotsky, 1978). It is also central to inquiry-based learning (Capps and Crawford, 2013). It can be achieved in class through collaborative tasks and group work, but many online tools include forums, chats, document sharing applications, and video conferencing subgroups that can be used to facilitate these collaborations.

Student-Teacher (S-T) interaction appears to have the highest perceived value among students (Anderson, 2003). Traditionally it takes place synchronously in class and this makes the interaction difficult to scale, but the use of online resources can transfer S-T interaction to both online synchronous S-T interaction by means of videoconferencing and to asynchronous S–C interaction by using educational videos, classroom recordings, and personalized and adaptive practice environments. (Anderson, 2004; Stein and Graham, 2020).

Student-Content (S–C) interaction can easily be made accessible online and is easily manipulated, as content has no volition of its own. It can take the form of texts, videos, graphics, animations, adaptive practice environments, etc. (Garrison and Vaughan, 2008; Graham, 2006; Stein and Graham, 2020). Indeed, through the ubiquitous smartphone, S–C interaction can now basically take place anywhere, anytime, blurring traditional boundaries between study and other areas of life (Akkerman and Bakker, 2011; Garrison and Vaughan, 2008).

Most courses will attempt to strike a balance between these three types of student interaction to merit from the advantages of each, combining flexibility with structure and interaction. The physical classroom is considered to be the ultimate “high fidelity” learning environment, where all three types of student interaction can take place at a high level (Broadbent, 2017; Van Doorn and Van Doorn, 2014). A body of research, primarily in the field of distance education, focuses on the subject of finding the right balance between the three types of student interaction (Anderson, 2003; Daniel and Marquis, 1979; Wagner, 1994).

Blended learning in the definition adopted here is an educational approach that aims for a mix of the three student interactions by combining “instruction from two historically separate models of teaching and learning: traditional face-to-face, classroom learning systems and distributed, online learning systems” (Bonk and Graham, 2012) (p.5). The mix offered in blended learning may vary along the axes of online versus offline and formal versus informal learning. In this two-axis model, traditional synchronous classroom education is classified as offline & formal, whereas students chatting about content in an online forum on a Saturday night is classified as online & informal (Garrison and Vaughan, 2008). The percentage of online work within a blended course may vary greatly, reaching as high as 90%, even though some definitions limit the amount of online time to a maximum of 50% (Bernard et al., 2014). Blended learning is showing promise: a 2014 meta-analysis demonstrates that the improvement in achievement attributed to blended learning as compared to traditional classroom instruction is low but significantly greater that zero (Bernard et al., 2014).

The field of distance education sprouted many studies on the effectiveness of online versus offline courses, a subject at the core of the present study. Studies have included short-term and long-term retention rates, but results have not been conclusive so far (Sublett, 2019). Online distance courses have demonstrated fall-to-fall retention rates that were up to three times higher than those reported in classroom courses (Fike and Fike, 2008), but in contrast other studies report a short-term negative effect of online education (Hart et al., 2018). Furthermore, when offered a choice, students reported opting for online education for subjects they themselves judged as more easy, whereas they preferred classroom education for the more difficult or important subjects (Jaggars, 2014). This preference may be grounded in the observation that students in both collaborative and teacher-directed offline conditions, with potentially high level S–S and S-T interactions, outperformed students working online on their own, with a potentially high level of S–C interaction (Means et al., 2009).

A recent study has specifically investigated the quality of S–S, S-T, and S–C interactions in online and offline environments. In this study, students’ face-to-face and online interactions within the same university course were ranked according to their perceived quality, and a sociogram of interactions was constructed. It was found that the level of interaction increased during the first half of the course and then levelled off. More importantly for the present study, however, the level of interaction appeared to be significantly higher in the offline part (Shu and Gu, 2018).

These literature and theoretical considerations can be used to construct a hypothesis on the preference of students for online or offline synchronous or asynchronous meetings within a blended university course. It assumes that the offline classroom synchronous interaction will be preferred:

When offered a choice, students will prefer synchronous offline meetings to synchronous online meetings, based on the higher level of all three student interactions in the former.

Method

A qualitative case study approach was adopted in order to answer the research question and test the hypothesis (Almeida et al., 2017; Morgan, 2007). A qualitative approach was preferred since the research question requires in-depth information on the various dimensions of the issue, rather than quantitative information. The circumstances forcing the course to go fully online were extraordinary. A relatively open, qualitative method was judged to be most likely to accommodate responses in areas that could not easily be anticipated. A focus group method was chosen because the intention was to “generate discussion or debate about a research topic that requires collective views and the meanings that lie behind those views” (O.Nyumba et al., 2018, p. 28). It was decided to use an random sample of six students out of the total of 24, because larger numbers were assumed to result in more students “dodging” the discussion, especially in an online situation, whereas a smaller number might not lead to the desired amount of interaction. Since 10 percent of “oversampling” is recommended in literature ((Morgan, 1996; Nyumba et al., 2018), it was decided to randomly ask seven student students to participate.

The context of this study is a university master course for student teachers across all school subjects within the teacher training program at a large university in the center of the Netherlands: “Excellence and differentiation in secondary education.” It is a blended course with 9 synchronous meetings, originally designed to be offline, given by two teachers. The course overview is given in Figure 1. The total student workload is 5 European Credits, or 140 h, of which 18 h are offline synchronous (nine meetings of two hours, or 13% of the total workload). There were 24 students on the course.

Figure 1

Design of the blended course “Excellence and differentiation in secondary education,” a 5 EC elective course within a teacher training program at a university in the Netherlands.

This study concerns the third edition of the course after its full redesign as a blended course. In this design, almost all the course content was presented in online formats such as articles, videos, forums, and readers. The offline meetings were designed to accommodate a high level of S–S, S-T, and S–C interaction, with rapidly changing educational activities such as group work, presentations, peer feedback sessions, Q&A, discussions, and debates.

The course happened to be about halfway (between meeting 4 and 5) when it was forced to go fully online because of the Covid-19 pandemic. Meetings five through nine were redesigned as two-hour online synchronous meetings facilitated by a videoconferencing platform, several online voting tools, online chats, and an online forum. Group work was facilitated by employing breakout rooms within the videoconferencing platform and students could use these rooms both within and outside of the synchronous meetings. They could thus also work in small groups on the “personal learning objectives” mentioned in Figure 1. The course assessments were in the form of individual essays on two different aspects of the course, to be graded on the basis of pre-shared rubrics. Therefore, the assessments and the grading system were kept identical after the course went fully online.

After the course was finished and every student's work had been graded, seven students (out of 24) were randomly selected to take part in a semi-structured interview, using the same videoconferencing platform that was used during the course. Six students agreed to take part and gave their consent for the interview to be recorded and the recording to be transcribed, anonymized, and coded. One student did not respond to repeated e-mails and may not have received them. The interview was conducted by the researcher, also one of the teachers of the course. The interview questions are included in the appendix, as is the text of the consent statement. In almost every instance the questions were followed up by other questions by the researcher, e.g.: “Why?”, “Could you explain?”, or “Could you give an example?”. The duration of the interview was 40 min. Since the course was English-spoken, the interview was conducted in English as well. During the interview the offline synchronous meetings were referred to as belonging to the “blended part,” whereas the online synchronous meetings were referred to as belonging to the “online part.”

The interview was transcribed and the quotes were coded along three categories:

Bottom-up emerging themes;

The primary type of student interaction involved (S–S, S-T, or S–C);

Favoring either the offline synchronous meeting, the online synchronous meeting, or neutral.

A total of 40 codes was found, of which 16 were subjected to second coding by an independent researcher. Intercoder reliability was established on all three coding schemes, with Cohen's fixed marginal kappa κ = 0.92 for the bottom-up themes, κ = 0.70 for the primary type of interaction, and κ = 1.0 for the preference for online, offline, or neutral. These values indicate a good to perfect intercoder agreement.

This study was approved by the Ethical Board of the Faculty of Sciences and the Faculty of Geosciences, Utrecht University under reference S-20428.

Results

The results of the bottom-up coding process are presented in Figure 2.

Figure 2

The results of the coding of bottom-up emerging themes combined with preference for either type of synchronous meeting.

Figure 2 gives the results of the bottom-up themes that emerged from the transcript, and the students’ preference for the online or offline synchronous meetings reflected in the quotes. The figure describes the way the students experienced the two versions of synchronous course meetings. It clearly demonstrates that in the majority of quotes (78%) the students express their preference for offline synchronous meetings.

Moving to the different categories, some of these quotes refer to their being able to physically move around (physical setting) and to the feelings of relatedness that they associate with physically being together in a single room:

“We're still, you know, using up the interaction that we had before because if we would have started online you could never get the same feeling of, I guess, “welcomeness” that [another student] was trying to explain.”

“And...eh...about teacher support I really noticed that you as teachers really took time to get to know us in the blended course setting. Eh... I was very surprised when I came in the room the first time that I was immediately greeted by you and asked for my name and stuff (...) I thought that was really nice and I really felt supported.”

Not all quotes on the physical setting favored the offline meetings, though. One student referred to the amount of time lost in forming groups and walking around during offline meetings:

“In the blended meetings we did a lot of walking around and giving turns and, I don't know, sometimes it just took up quite some time and now we, because we are online, we didn't really have that anymore. So, it was quicker to jump from person to person and from activity to activity.”

Furthermore, students mentioned a perceived barrier or threshold for asking questions online and reported having trouble keeping focused:

“I couldn't always ask some questions because it was a bit hard to jump in with so many people sometimes because everyone has their own questions and it's just hard to ask them online, I think.”

“In the online part [concentrating] was probably even harder, because then you feel less watched, less interaction, it's easier to, you know, zone out and let things go.”

By far the largest group of quotes (40% of total), however, is related to the quality of the student interactions in a category dubbed “direct interaction”. Students were outspoken about the perceived advantages of offline meetings and the perceived negative influence on the level of interaction when the course switched to fully online. For example, several quotes commented on the sluggishness of the interaction in an online environment:

“In a real class, you just, you know, you hear something and then you continue on that. That easy transitioning from hearing something somewhere in the class and then continuing on that, that is missing [online].”

“I think (…) what was the best part of those (...) [offline] classes, was that you could have direct interaction. I don't know if it is really that you learn so much more – maybe it is, yeah, but…I think direct interaction, yeah, maybe it does teach you more!”

The perceived barrier for opening the microphone and speaking was also touched upon:

“People don't know when to speak online and (…) that is a barrier to speaking up in classes or in phone calls or something like that.”

Furthermore, the eye contact that is so notoriously missing in online meetings was mentioned in relation to directly gauging the understanding of the students:

“…with the direct instruction for some reason you want to have, maybe, eye contact. Maybe that's it. You want to see the students, you want to see how they respond to what you are saying and then you can explain a bit more or a bit less.”

A smaller number of quotes (10%) concerned group work. Interestingly, these all involved neutral quotes and quotes actually favoring the online synchronous meetings, e.g.:

“…when a teacher says okay, well, those four people go into channel 1, those four people go into channel 2 and the rest go in channel 3, or sometimes even let us decide: this is the subject of channel 1, this is the subject of channel 2 and choose where you want to go, this works very well. So...eh…some level of interaction might work and I guess in hearing [another student]'s story it depends on the...eh...on the participation of the...eh...[students] and how active everyone is.”

“…so when we transitioned to online ... actually [the group work outside meetings] was a little bit easier because we knew we had to meet but we could just do it online so – I don't know why we didn't do that before, actually.”

Figure 3 shows the same body of quotes coded according to the primary type of student interaction.

Figure 3

The results on the coding according to the primary type of student interaction referred to.

Figure 3 shows that all three student interactions are clearly present, with quotes favoring the offline synchronous meetings dominating each type of student interaction. The S-T and S–C interaction categories consist almost entirely of quotes favoring the offline meetings. Interestingly, almost all the neutral quotes and quotes favoring the online interaction (7 out of 9 such quotes) are in the S–S interaction category:

“I believe [the university ICT support desk] has a couple of peer feedback tools and (…) I am sure there are many instruments that will support having a more interactive discussion online (...). I think that's more in the course design than in the difference between blended or online.”

Notwithstanding this, in the S–S category as well, the majority of quotes did in fact favor the offline meetings:

“…you can't see the other people. When I am in a class I can anticipate, like, oh, she's, like, sitting like this. She is going to ask something and then I will hold back but ... if I can't see anyone then I don't know if I should, like, speak up or what other people are thinking or when...in class some people start to speak, you're immediately, like: oh yeah, that's my question!”

“I think I only realized how nice it is to actually see people and discuss because we cannot do that anymore.”

Figures 2 and 3 demonstrate that the perceived high quality of direct interaction is the main advantage of offline synchronous meetings of online ones, a notable exception being small group work.

Conclusion

The research question of this case study is:

How do students experience the learning environment during synchronous online and offline meetings within the same master course?

Students generally favor the synchronous offline meetings to the synchronous online ones, 78% of the quotes favoring the former. They refer to higher levels of perceived relatedness both among themselves and with the teachers of the course, they enjoy the low threshold for asking questions, report more focus, and appreciate the physical environment during the offline meetings. According to them, the most important difference between on- and offline synchronous meetings is the perceived higher level of all student interactions in the latter. This higher level of interaction is facilitated by:

rapid reactions on each other's remarks;

direct eye contact between students and teachers;

body language.

One notable exception is S–S group work. In this category, the quotes paint a different picture consisting of quotes favoring both approaches as well as neutral quotes. When discussing small group work inside and outside of class, students generally favor online meetings.

The hypothesis:

When offered a choice, students will prefer synchronous offline meetings to synchronous online meetings, based on the higher level of all three student interactions in the former, is supported by the data of this study.

Discussion

This section is divided into three subsections: implications, limitations of the study, and directions for future research.

Implications

As may be expected from these master students in the field of education, the students in the focus group interview were very vocal and eloquent in their arguments supporting their preferences. For example, students literally stated that they learn more during offline synchronous meetings as compared to online meetings, for the reasons listed above. This is in line with recent research on the level of interaction in online and offline environments in general (Shu and Gu, 2018). Relatedness and the perceived benefits of being together in the same room are also mentioned as advantages of offline meetings, relatedness being one of the three basic psychological needs in self-determination theory (e.g., Niemiec and Ryan, 2009).

The results of this case study support the application of the concept of blended learning but give some rather specific directions on blended course design (Stein and Graham, 2020) as well. Students state that offline meetings are a necessary part of education because of the perceived high quality of all types of interaction and in terms of relatedness and connectedness to each other and to the teacher(s). Apparently, the physical classroom is indeed considered a “high fidelity” environment (Broadbent, 2017; Van Doorn and Van Doorn, 2014) by them.

However, if small group work, especially outside of class, is a part of the course design, students recommend that this could very well, and even preferably, be done in the online realm. This could have implications for course design. Since the use of videoconferencing tools has become so widespread during the 2020 pandemic, the learning environment of any future course could easily be (re-)designed in order to include a tool to facilitate online small group work, while keeping the synchronous meetings mostly offline.

Student small group work outside of the classroom is thus an aspect of S–S and S–C interaction that may actually benefit from online environments. Students do not need to travel to university buildings to interact in small groups while working on assignments. In this context, it is noteworthy that traveling to and from university campus was not once mentioned as a drawback of offline synchronous meetings. Apparently, the advantages of getting together as a whole class in a physical room outweigh the trouble of traveling, but when working in small groups outside of class the balance is tipped the other way.

A remark about the results in terms of student grades is warranted here. The average results of the 2019 edition of the course (average 7.7 with a standard deviation of 1.1 on a scale from 1-10) and the 2020 edition that was the subject of this study (7.9 ± 0.9) are very nearly the same. Whether this reflects the inconclusive results on the differences in learning outcomes for online versus offline courses (Sublett, 2019) cannot be concluded from this small case study.

Limitations of the study

As this is a case study, the questions of generalizability and possible bias should be addressed.

Considering possible sources of bias first: students were selected randomly after they had received their course grades. All of them passed the course. Six out of seven volunteered to take part, the seventh never reacting to repeated emails and messages, maybe not even having received them. Students were explicitly asked not to evaluate either the course material or the teachers, but rather the differences they experienced during on- and offline synchronous meetings within the course. All students of the course were offered the regular course evaluation tool to evaluate the course and the teachers. Based on these considerations, a neutral stance may be expected, even though the interview was conducted by one of the teachers: there was nothing for the students to be either gained or lost from the nature of their answers.

The semi-structured interview took place within the same environment as the online meetings and was conducted in the course language (English). Participants were informed that the recording and all analysis would be treated confidentially and no quotes would ever be able to be traced back to them. All these steps were taken to minimize possible sources of bias and facilitate students to express freely their opinion.

It is perhaps remarkable that no reference at all was made to the Covid-19 pandemic itself during the interview, only to the effects it had on the educational practice. Apparently, after several weeks of partial lockdown the students had gotten quite used to the limiting regulations. On the basis of this, bias towards the offline meetings – the situation before the partial lockdown – is neglected.

As for generalizability: all participants were student teachers in various school subjects studying ways to recognize and develop talented students in their own classes. Many of them were doing secondary school internships which confronted them with the same issues as encountered during the course, as most secondary schools in the Netherlands eventually started to offer fully online courses during the Covid-19 pandemic, albeit a week after the universities did. Most students thus not only had an intimate knowledge of teaching in general and teaching their subject in particular, but some experience with online teaching as well. In that sense, they may be considered to be more familiar with the subject of this study than their fellow students in other university master programs.

A remark on the interrater agreement may be warranted here. The bottom-up coding and the coding concerning the preference both showed a very good to perfect interrater agreement (κ = 0.92 and 1.0, respectively). The interrater agreement for the primary type of interaction was good, κ = 0,70, but not as high as the other two, indicating some possible ambiguities during this coding process.

Directions for further research

The Covid-19 pandemic has given rise to many changes in society, not in the least in education. The fact that education can take place fully online (if so forced), however, does not necessarily mean that this is the best way to conduct it. The results of this case study could be used to construct a questionnaire that can be used to quantitatively gauge the experience with the pros and cons of online education, including the facilitation of small group work outside of class.

Furthermore, the conclusions of this study could be used for design-based research (Bakker, 2018) on blended university courses. Given the fact that interaction and relatedness are key elements of offline meetings, these meetings should be redesigned with a basic question in the back of the educator's mind: “Does this offline course design optimally support relatedness and interaction?” Or alternatively: “Does this offline course design actually warrant coming to university for?”

We may yet learn a great deal from the 2020 pandemic on what exactly can and cannot be done in online education.

Declarations

Author contribution statement

R.F.G. Meulenbroeks: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work was supported by the Faculty of Science, Utrecht University.

Data availability statement

Data will be made available on request.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.