An observational study of learning effects of peer-assisted learning tests in COVID-19 pandemic times.

PURPOSE: This study aimed to determine whether peer-assisted learning (PAL) is a more effective learning and test method in terms of oral- and maxillofacial surgery.
MATERIAL AND METHODS: In July 2020, a total of 267 students took a PAL-based exam on fictional patients with surgical issues, in which they had to evaluate two fellow students and were themselves evaluated by two fellow students. The students evaluated their experience with the PAL-based exam through a questionnaire which consisted of five given statements (answer possibilities: agree, disagree, neutral) and two questions (answer possibilities: better, equal, worse) to rate.
RESULTS: In the survey, 77.9% of the students rated PAL as a better learning method and 21% rated it as at least equally effective to the known multiple-choice (MC) test. A total of 74.9% of the students indicated that they learned more content with PAL and 20.2% said they learned the same amount; 83.7% said that their "clinical thinking has improved" through PAL. In the comments, 73% of the students noted that they think PAL is a good learning method, and at least 22% rated it as useful but in need of improvements. Only 5% did not see PAL as an acceptable learning method. In contrast to this, 1.3% saw PAL as a "bad alternative to MC tests."
CONCLUSION: PAL, especially peer assessment, might represent a better learning method as it might encourage students to deal more intensively with the learning content and to improve clinical thinking.

INTRODUCTION

Training in dental clinics in times of digitalization and higher requirements of the students (i.e., fair and objective assessment) is becoming increasingly challenging. A rapid shift in thinking toward treading new paths will affect the universities due to the pandemic outbreak of the corona virus at the latest.

According to the national German dental licensing laws, students should obtain theoretical knowledge through lectures and perform dental treatments for patients by themselves in a supervised course from the 3rd to the 5th year of study. Currently, the restructuring of practical and theoretical courses is difficult due to common quarantine regulations meant to guarantee the health of students, patients, and university staff. The students’ theoretical education needs to be continued without any conventional lectures in the auditorium, what makes it difficult to keep the students motivated to continue learning. In general, it is challenging and hard to assess students’ skills, knowledge, aptitude, and behavior with only online classes, especially during the current governmental restrictions. However, technology‐mediated distance learning and e‐learning seem to be good alternatives depending on the attitudes of both the faculty and students. Redesigning the theoretical education curriculum requires the examination modalities of conventional multiple‐choice (MC) tests with compulsory presence to be changed as well. ,

In a survey with 69 European dental institutions, the authors determined that the lockdown led dental schools to postpone formative (46%) and summative assessments (42%) or to organize examinations entirely online (50%), and 72% of the schools considered postponing the evaluation of required clinical competences for their students. There was an urgent need to develop new types of assessments for graduate students, especially during the pandemic, rather than postpone them.

Peer teaching or peer‐assisted learning (PAL) could be one such alternative method. To paraphrase PAL, the large number of authors refers to Topping, who defined this term in 1998 as follows: “Peer learning can be defined as the acquisition of knowledge and skill through active helping and supporting among status equals or matched companions. It involves people from similar social groupings who are not professional teachers helping each other to learn and learning themselves by so doing.” PAL is widely used in medical school education, for example, in preclinical anatomy and problem‐based learning courses. , In addition, PAL is also the part of the practical clinical training of physical examinations, learning to communicate with patients or the acquisition or further development of theoretical and/or practical skills in skills labs. , , This broad application is based on the fact that PAL has many advantages, as the literature shows. PAL can increase student confidence in clinical practice and is able to be equivalent, if not more effective, in imparting knowledge of practical work compared to trained teaching staff. , , Furthermore, PAL can offer a positive learning environment (cooperative learning through joint discussions, searching for solutions, etc.) and is characterized by an “openness,” as students explain misunderstandings, seem to be less stressed and a certain “fear of failure” is missing. Moreover, it should be mentioned that PAL can also be carried out successfully online, which is a considerable advantage in times of coronavirus but also in new ways of digital teaching and examination.

Scicluna et al. evaluated a structured peer learning program. They distinguished between students who were taught by other students who had 1 year more experience (vertical integration) and students taught by other students who had between 1 and 6 years more experience (near peer teaching). The researchers developed a comprehensive questionnaire to evaluate the peer learning among the participants, sent it to 1606 participating medical students, and received valid responses from approximately 20% of the medical students (n = 328). In general, the respondents liked the experience of vertical integration, and the more experienced students felt obligated to teach the less experienced students, feeling personally responsible for the students’ education.

One important component of PAL is peer‐assisted tests. Since, to our knowledge, there is a lack of data concerning this component, this study evaluated a peer test to investigate how PAL tests are generally evaluated and how or to what extent they can improve the students’ learning success.

METHODS

This observational study reports findings regarding students from years 3–5 within the Department of Oral and Maxillofacial Surgery at Friedrich‐Alexander University Erlangen‐Nürnberg (FAU) in the summer term of 2020. The study was notified at the ethics committee of FAU, and a submission for approval was not mandatory by them. The level of difficulty of the PAL test was adjusted to the expected knowledge level of the students according to their academic year. Using different patient cases (an anticoagulated patient and tooth extraction, anaphylactic shock, oral squamous cell carcinoma, reconstructive tumor procedures, diabetic patient), the students had to answer theoretical questions (eight to 11) about the case. According to the academic year, students in the third year had a further final question related to suture materials, students in the fourth year had a question related to immune responses, and students in the fifth year had to propose a treatment for the plastic reconstruction of a facial defect. Each student had 24 h to answer the questions and upload the completed file to the internal online platform for students. Afterward, on the second day, peer tutors were selected randomly via the online platform, and each student received two independent peer tutors from the same academic year and thus acted as one of the two peer tutors for another student. Peer tutors rated their peer's work in a rating system (1‐low to 5‐high) and had a text field for providing constructive feedback after carefully reading the answers of their colleagues and comparing them with their own answers. All answers (the test itself and the answers given as a peer tutor) were supervised by one coordinator within the department with professional experience (oral and maxillofacial surgeon), who responded to comments if necessary to determine whether the test had been passed via the online platform during or after the test session within one day.

To assess and clarify the learning effect of PAL as our main question, we used a questionnaire, which was inspired by previous questionnaires about PAL. , The survey was evaluated anonymously by the students and disseminated via an online link. It was mandatory to fill and included five statements and two questions.

The five statements were as follows (rated as agree, disagree, neutral):

I think my clinical reasoning has improved due to the peer test.

Due to the peer test, I have simultaneously improved my critical ability and my ability to give feedback by giving my fellow students constructive feedback.

I believe my personal data are endangered due to the peer test.

Due to the peer test, I fear that my personal data will be exposed by my fellow students.

Due to the peer test, the community of students this semester was strengthened.

The two questions were as follows (rated with better, equal, worse):

How would you rate the learning effect of this peer test compared to that of MC tests?

How do you think you dealt with the learning content in PAL compared to that of MC tests?

In addition, the students were able to write comments such as suggestions for improvement, their points of view, and other notes about PAL. The general message of each comment was classified as “good,” “improvable,” and “not acceptable,” and comments were grouped by their core statements and the frequency with which each type of comment appeared. To analyze the data, we used a univariate descriptive statistic.

RESULTS

A total of 267 students were included and filled the evaluation questionnaire after they took PAL‐based exam. The detailed results of the answers to the five statements and two questions of the survey can be seen in Table 1. In total, 153 comments were able to be analyzed, and the general messages concerning the peer‐assisted test were extracted and classified as good (73%), improvable (22%), and unacceptable (5%). Moreover, Table 2 shows that 63.4% of the students explicitly mentioned that “PAL is generally good.” In contrast, 1.3% of the students rated PAL as a “bad alternative to MC tests.” A total of 26.1% of the students who wrote comments indicated that they had “learned more with PAL than they would with an MC test.” As for fictitious patient cases, 22.9% of the students rated the task as “more practical” than an MC test. Further, in 13.1% of the comments, students indicated that “because of the peer assessment, (they learned) new therapy approaches.” PAL improved the “critical assessment” ability of 3.9% of the participants and the “self‐reflection” ability of 3.3% of the participants.

TABLE 1

Showing the survey results for PAL

Participants’ survey results, n = 267	I agree	Neutral	I disagree
I think my clinical reasoning has improved due to the peer test.	83.7% (224)	15.9% (42)	0.4% (1)
Due to the peer test, I have simultaneously improved my ability to give feedback, by giving my fellow students a constructive feedback, and critical ability.	64.8% (173)	30.3% (81)	4.9% (13)
I believe my personal data are endangered due to the peer test.	66.3% (177)	26.2% (70)	7.5% (20)
Due to the Peer Test, I fear an exposure by my fellow students.	3.8% (10)	14.2% (38)	82.0% (219)
Due to the Peer Test, the semester community got strengthened.	43.8% (117)	47.9% (128)	8.2% (22)
	Better	Equal	Worse
How would you rate the learning effect of this peer test compared to that of MC‐ tests?	78% (208)	21% (56)	1% (3)
How doyou think you dealt with the learning content in PAL compared to MC‐tests?	74.9% (200)	20.2% (54)	4.9% (13)

Note: The numbers in parenthesis are the actual numbers of respondents.

Abbreviations: MC, multiple choice; PAL, peer‐assisted learning.

TABLE 2

Showing the core statements of the students’ comments

Core statements of comments (n = 153)	Frequency of statements
PAL is generally good.	63.4% (97)
I learned more than in an MC‐Test.	26.1% (40)
A fictitious patient case is more practically.	22.9% (35)
Because of peer assessment, I got new therapy approaches.	13.1% (20)
PAL enhances critical assessing.	3.9% (6)
PAL improves my self‐reflexion.	3.3% (5)
PAL is a bad alternative to MC‐test.	1.3% (2)

Note: The numbers in parenthesis are the actual numbers of respondents.

Abbreviations: MC, multiple choice; PAL, peer‐assisted learning.

DISCUSSION

Within this study, we referred to a theoretical part of PAL, examining not only the learning itself but also the assessment of other students and the resulting learning effect through a peer‐assisted test. The vast majority of the students (78%) stated they gained more knowledge through PAL than they would have with an MC test; 21% of them indicated that they learned at least as much as they would have. This makes sense, as 74.9% of the students stated that they learned better or equally well (20%) with the peer‐assisted test compared to an MC test. Previously, all participants had experience with MC tests within the department. It can be assumed that students prepare thoroughly in a short period just before the MC exam, take the exam, and then check to see if they passed, although they may not know which answers were correct. Thus, they may deal with the topic intensively, but this is only during their test preparation and the test itself.

In this PAL exam, in addition to studying, the assessment also required following up on the learning material, as students recapitulated their own answers, compared them with the answers of their fellow students, and identified or improved their incorrect answers. This corresponds to another study that observed a shift from an assessment of learning (founded on the idea of Martinez & Lipson) to assessment for learning. In an assessment of learning, the assessment is seen as separate from the educational process and only serves to determine whether the students have acquired sufficient knowledge and skills. With an assessment for learning, however, the assessment itself is embedded in the educational process in order to teach students individually in the most effective way. Peer tutors have the opportunity to individually comment on the tests of fellow students, thus a high qualitative feedback results, which can be a powerful influence on student learning, since students are significantly more motivated for further learning. , In a study by Cameron et al., a similar result emerged, since (bachelor degree students) BDS‐1 students taught by BDS‐5 students agreed that they had received useful feedback (87% of students in the peer teaching group vs. 61% of students in the traditional teaching group). They concluded that students themselves might have a better ability to teach and correct colleagues and still remain objective than teachers do. Clinical reasoning resulting from PAL is not precisely defined in the literature but is associated with clinical decision‐making or judgment. It is crucial to solve clinical problems by collecting and analyzing patient information strategizing and finally making decisions about treatment. , Clinical reasoning was promoted in two ways in this study. First, through the teaching method itself, since the non‐anonymized evaluation simulates a kind of consultation. The comments of their peers provided new points of view and additional knowledge for students, who thus obtained a comprehensive view of the examination. In this way, they may consider other methods, perspectives, and concepts, whereby extensive knowledge and established treatment concepts have to form a common base. The second aspect that promoted clinical reasoning was the impact of the design of these exams, which contained a fictitious patient with free text answers, unlike MC tests. Such exams based on patient scenarios can be a possible scheme for PAL tests to approximate clinical situations for intensive examination of relevant topics.

In addition to the positive learning effects mentioned above, 64.8% of the participants agreed that two types of crucial soft skills involved should not be underestimated. In contrast to an MC test, in PAL, the students are actively included in the assessment, as they write qualitative feedback, which is a soft skill. Upon receiving this feedback, the students have to rethink their answers and admit possible mistakes, which may lead to a new motivation. Thus, with PAL, students are given confidence, even if they fail, so they can remain motivated and deal with the specific learning content in a different way.

Possible negative effects of PAL tests should be discussed. Putting responsibility for grading an examination on the students could also lead to negative consequences, such as frustration, withdrawal, and counterproductive work behavior. It is important that faculty are available to clarify unresolved questions and other issues. However, the study of Cameron et al. showed that BDS‐1 students in the tutor group totally disagreed with the statement “I did not feel comfortable asking questions” (61% in the peer tutoring group vs. 26% in the traditional teaching group) but agreed that they had received useful feedback (87% in the peer tutoring group vs. 61% in the traditional teaching group). Our evaluation also shows that most of the students did not fear the exposure of their personal data. Moreover, many students thought that the student community was even strengthened during the semester (43.8%). This corresponds to another study, where students from the third academic year (D3) were each assigned to a student from the fourth academic year (D4) for one year and discussed topics such as maxillofacial surgery, endodontics, CAD/CAM technology, and implantology with them. Overall, 86.2% of our students recommended introducing this type of PAL general practice model into dental schools. Another negative effect, and probably the most critical point in PAL tests, is the lack of anonymization. This is reflected in the survey, as 66.3% of the students believed that their personal data were at risk of exposure. This is not solely about the students' data, but rather about the feared exposure in front of their peers and should definitely be taken seriously in further studies. But, we should consider that education also prepares for the profession and in this a presentation of the therapeutic approaches and decisions to patients and colleagues may be required at any time. Moreover, anonymity in a peer‐assisted test might even lead to irresponsible or exceptionally harsh feedback and prevent oral exchange and clarification between assessors and those being assessed. However, this effect can also be seen bidirectionally. In the absence of anonymization and with an assigned peer with stronger social relationship, better judgments could follow than of those with no social relationship at all. The idea behind PAL is open communication among equals, as the learners are all in the same situation and because of their experience and professional status, the challenges they face, etc., and they may have a better connection with one another. In further studies pseudonyms should be used instead of personal data to prevent social exposure and embarrassment. Nevertheless, this study shows that PAL test is widely accepted by students because 63.4% of them commented that their learning experience with PAL was generally good. Specifically, 26.1% of them commented that they experienced a greater learning effect with PAL than with an MC test, and 13.1% said they discovered new learning approaches. This corresponds to a study in which 80% of the students stated experiencing a positive learning effect. In addition, between 86.7% and 94.2% of the participants were able to derive learning benefits from being a near‐peer teacher. This correlates with our results: the students benefitted from providing feedback, deepening their knowledge through the process of teaching, and being motivated by their fellow students. A frequent comment (among 22.9% of the participants) was that a fictitious patient is much more practical and promotes clinical reasoning in a peer‐assisted test.

Ultimately, it must be emphasized how well PAL test was also adapted to the restrictions imposed by the COVID‐19 pandemic. In addition to its numerous negative effects, the pandemic may also catalyze a change in education. PAL is not restricted by social distance, and so soft skills and knowledge can both be promoted. According to the literature, e‐learning alone does not result in greater learning success, but it also does not result in inferior learning ; however, e‐learning does not have to completely replace conventional teaching, as it can be integrated into such teaching and lead to an advantageous blended learning strategy. ,

There are a few shortcomings in this study that need to be mentioned and critically discussed. One limitation of the study is the fact that only the influence of PAL on theoretical knowledge transfer and not the influence practical skill improvement is considered. It should also be noted that PAL is a general term that summarizes various PAL programs (e.g., peer tutoring, peer‐led teaching, peer‐supported teaching). In this study, the focus was only on peer assessment, thus reflecting only one type of PAL and making it almost impossible to gain a comprehensive understanding of PAL, its modes of operation, and the spectrum of PAL activities. Moreover, knowledge gained from staff feedback and the ability to give feedback should be further investigated in future studies. Ultimately, these results come from the subjective impressions of the students. Furthermore, this study only represents a high level of acceptance for PAL among students, but not to what extent this teaching method is more effective compared to MC tests, since no control group is integrated. A more objective approach, that is, an additional control group that takes an MC test, is needed.

CONCLUSION

The authors concluded that peer‐assisted learning, especially peer assessment, results in a greater learning effect and improves clinical thinking of the students, although a small part of the students has voted PAL as a worse exam alternative.

AUTHOR CONTRIBUTIONS

Conceptualization: Jonathan Holfert and Mayte Buchbender. Methodology: Jonathan Holfert and Mayte Buchbender. Software: Jonathan Holfert. Validation: Jonathan Holfert, Marco R. Kesting, and Mayte Buchbender. Formal analysis: Jonathan Holfert. Investigation: Jonathan Holfert. Resources: Marco R. Kesting. Data curation: Marco R. Kesting. Writing‐ original draft preparation: Jonathan Holfert. Writing – review and editing: Mayte Buchbender. Visualization: Jonathan Holfert. Supervision: Mayte Buchbender. Project Administration: Marco R. Kesting. Funding acquisition: Marco R. Kesting.