The use of innovative cost-saving audience response system in orthodontic case-based learning: A potential approach in distance learning.

OBJECTIVE: To provide a cost-saving innovative audience response system (ARS) that permits typing texts and compare its efficiency to the paper-based method in case-based learning (CBL).
MATERIALS AND METHODS: Orthodontic clinical cases were presented to 149 undergraduate dental students for discussion among teammates. Responses were collected using ARS-based and paper-based. ARS was constructed from an online survey platform (Google forms) then QR code was created for easy and fast access. Students used their cellphones to scan code, view questions, discuss, type, and submit answers within 10 minutes. Feedbacks were collected using a feedback survey. Outcome measures included the number of words, spelling mistakes, time required by the instructor to read submissions, and activity time compliance.
RESULTS: The average number of words submitted by females in ARS-based 47 ± 8 was significantly higher than 35 ± 16 paper-based, and male ARS-based 36 ± 18 (P < 0.05). ARS-based submissions required significantly less time to read compared to paper-based among whole group, females, males (P < 0.001, P < 0.05, P < 0.001), respectively. Spelling mistakes were lower 1 ± 1 in ARS-based compared to paper-based 2 ± 2. The ARS-based first submission was (-4.28) minutes before the deadline, while paper-based last submission was (+2.19) minutes after (P < 0.05). ARS-based submissions were 12.5 seconds faster to read than paper-based (P < 0.001). Out of 56.4% of respondents, 63.1% preferred using ARS-based and 80% agreed that it provided immediate feedback, with high overall satisfaction.
CONCLUSIONS: This innovated ARS was found to facilitate CBL. It is superior in time and cost-saving to paper-based and other ARSs. It could be useful in distance learning especially during the COVID-19 pandemic. Copyright:

Introduction

Case-based learning (CBL) has been well adopted in health professional education for a decade.[1234] The main aim of CBL is to develop clinical reasoning by the application of theoretical knowledge into clinical practice using an inquiry-based learning framework.[5678] This method fosters a deeper level of learning through the meaningful application of theory.[5910] In CBL, teachers act as facilitators and learning is student-centered in which students recall previous knowledge to make decisions for a given clinical scenario.[3411]

The limited time available for orthodontics within the dental curriculum and limited clinical exposure to orthodontic cases could jeopardize the overall learning experience.[312] This can be improved by adopting case-based seminars or using standardized patients as teaching strategies.[34131415] Adopting CBL in orthodontics was reported to enhance students’ knowledge and treatment planning skills.[16] Furthermore, students can actively participate in their own learning experience by breaking into smaller groups and work in teams to analyze and solve clinical cases based on previous knowledge.[17] This team-based approach can be utilized in large classrooms to enhance soft skills including teamwork, communication, critical thinking, and problem-solving skills.[1819]

Audience response system (ARS) has been adopted in medical education to allow students interactions with instructors.[202122] It is an electronic system that allows students to respond to multiple-choice questions (MCQs) or dichotomous questions (YES/NO) presented on the screen by using electronic remote devices or clickers.[23] Results are instantly collected, clustered, and displayed anonymously on the screen. This interactive method allows student participation and promotes active learning.[2425] It has been reported to enhance student engagement and learning, stimulate in-class discussion, and improve knowledge retention.[24262728] It provides real-time feedback to students in terms of accuracy of their responses so they compare their performance to the rest of the class. Additionally, it allows instructors to assess students’ learning and gives them an opportunity to reinforce key concepts based on students’ responses.[29] Students perceived ARS positively and preferred to incorporate it in preclinical dental lectures.[303132]

However, a major disadvantage of the currently used ARS is its limited use for dichotomous true/false, YES/NO, or MCQ questions. This alone could only measure the basic level of knowledge without allowing further elaboration and reflection on the answers selected by students.[33] Moreover, there is an existing risk that students could select the right answer by chance, without reflection of actual competence or knowledge gain. Other disadvantages are the added cost of purchasing clickers, limited number of available devices while teaching a large number of students, and the logistics of establishing ARS setting within the institution.[34] Some institutions dedicate a specific location and set-up for ARS that limits availability and requires transportation that adds to the total burden. Therefore, the aim of this study was to provide a user-friendly cost-saving innovative ARS that permits typing texts and compares its efficiency to the paper-based method in CBL.

Materials and Methods

Case-based and team-based model

This study was reviewed and approved by the Research Ethical Committee at the Faculty of Dentistry, King Abdulaziz University (KAUFD) in 2020. It was performed in 2020, during the interdisciplinary orthodontic module as part of the orthodontic curriculum for the sixth year (last year) dental students at KAUFD. This module was designed in a case-based format that is delivered on a weekly basis, one session per week with two to three cases per session. Typically, it is delivered to students during the last semester of their final year in dental school. By this time, higher cognitive and analytical skills are targeted, as students have had already most of their clinical basic knowledge in orthodontics and other dental disciplines.

During these sessions, students were divided into small groups of 5–6 students per group based on their preference to work in teams. The sessions were delivered separately to males and females. Every team distributed roles and duties among team members with one person (the recorder) responsible for collecting the team's answers and submitting them to the instructor within an allocated time frame. In every session, real-life orthodontic case scenarios supported by clinical pictures and radiographic images were presented on a screen infront of the whole class. Every scenario had four open-ended questions related to the case, and students were given a fixed 10-minutes period to work among their teams, discuss the case, and answer the questions. During this time frame, students were encouraged to browse the internet or search for the information in any source of preference. Time was calculated using an electronic timer, and the instructor notified students about the remaining time for each case, 5-minutes and 1-minute before time was over. After submissions were received from all groups, the case was then discussed among the whole class with all teams participating in the discussion, and possible debate, tackling each question. Lastly, a wrap-up summary of the key concepts learned from this case scenario was presented by the instructor at the end of the session.

Innovative audience response system

This part of the article explains the method of utilizing an online survey platform (Google forms) to create an inhouse ARS used in this study. A separate Google form was created for each clinical case, in which the first part was dedicated to collect students’ names within each team to ensure their attendance and participation in the activity. Each form has four open-ended questions that are unique and related to that specific case.

When the form-construction was completed, an electronic link was generated from this form. This link was then used in a quick response (QR) code generator website to produce a QR code image. This generated QR code image is then saved in a portable network graphics (PNG) format and inserted into a separate slide within the PowerPoint presentation following the case scenario slides. This QR code image was enlarged to fit the whole slide so it could be screened from a distance, especially in a large classroom. A white background color was used to eliminate any noise or distractions and to keep the slide simple and clean. This whole process was done in advance while preparing the cases and slides for the session.

Study intervention

In this study, the instructor used two different methods for collecting and reporting answers related to each presented case. The study was performed by the same instructor eliminating the need for calibration. Both methods were applied to the same students, so they act as their own controls.

In one session, students were introduced to the inhouse generated ARS and instructed to use it during the session. At the beginning of the class, the instructor requested from all teams’ recorders to try scanning a test-QR code on the screen using QR code reader built in their own cellphones’ or tablets’ cameras, or any application that has a scanning feature. Then, the case scenario with clinical and radiographic images was presented on a large screen to the whole class. Next, the slide containing the enlarged QR code was displayed. Team recorders were instructed to scan the QR code, view questions on their cellphones/tablets, discuss the case among their teams, type answers, and submit the form on behalf of the team within 10-minutes. While teams were working on the case, the instructor changed the display screen from PowerPoint and opened the original Google form to view students’ submissions on the screen. Each team's submission was captured and displayed in real-time on the responses tab of Google form. After the 10-minutes was over, the instructor switched Google form's responses button off, so any further submissions were not allowed. All submitted answers were instantly clustered per question and displayed anonymously in front of the whole class.

This was followed by a rich discussion and debate among the different teams while all answers were available on the screen. Teams were able to see their submissions and compare it to other teams’ submission who had similar or different views from their own. This provided immediate feedback to team members in terms of their understanding of the clinical case. Additionally, all teams participated with their shared knowledge in building a global overview of the case tackling it from different perspectives. This emphasized the role of students in active learning and minimized the load on the instructor. Furthermore, the main role of the instructor was facilitating the discussion among teams [Figure 1]. At the end, the display screen was switched back to PowerPoint to present a summary slide with key concepts related to the case. Students’ responses were automatically saved to a Google sheet and were retrieved later for further analysis.

Figure 1

Model of innovated ARS

In the following session, new different cases with four questions each, were presented to the same students. Teams were instructed again to discuss the case among team members and submit their answers to the instructor within 10-minutes. However, ARS was not used this time, and teams’ recorders were instructed to write their answers on a piece of paper and hand them to the instructor within the allocated time. After all submissions were received, the case was then discussed among the class similar to the previous week.

The outcome measures used in this study were number of words received per submission, students’ compliance with the activity's allocated time (presented by the time of first and last submission received) and number of spelling mistakes. The time required to read each submission in both methods was calculated and used as an indicator measuring difficulty in reading students’ handwriting.

Audience response system evaluation (Feedback) survey

At the end of the second session, an electronic survey was distributed among students to record their feedback on both methods and to report their preferred learning method. Participation was anonymous and optional. The survey consisted of nine questions. Of those, two questions were in a 5-Likert scale format measuring how they liked ARS versus paper-method and overall satisfaction rate. Another five Yes/No/Indifferent questions were measuring their perception about the effectiveness of using ARS in case-based learning in future sessions. The last two questions were open-ended text format to allow students to comment on advantages and disadvantages and difficulties faced while using the innovative ARS [Appendix]. Based on their feedback, their preferred method was then adopted in teaching subsequent sessions.

Statistical analysis

Data were retrieved from the Google sheet and downloaded in Excel software (Microsoft Office Excel, Redmond, WA, USA). Descriptive statistics were reported in mean and standard deviation. Student t-test was used to compare the number of words, compliance with activity time, spelling mistakes, and difficulty in reading handwriting in both methods. The linear regression model was used to study the association between the time required to read each submission and the method used (ARS-based vs. paper-based) with adjustment of other variables including gender and number of words submitted, best-fitted model was reported. Testing of normality distribution was done using the Shapiro–Wilk test. The significance level was set at P < 0.05. All analyses were performed using STATA Version 16.0 (StataCorp, College Station, TX, USA).

Results

A total of 149 sixth-year students (72 males and 77 females) were involved in this study. Figure 1 shows a diagram of the innovated ARS model used in the sessions. Efficiency of ARS-based compared to the traditional paper-based method is presented in Table 1. The mean number of words submitted by overall students in the paper-based method was 37 ± 16 compared to 42 ± 15 in ARS-based. Females submitted significantly (P < 0.05) higher number of words 47 ± 8 in ARS-based compared to 35 ± 16 words in paper-based. The number of words submitted by ARS-based were significantly (P < 0.05) higher in females 47 ± 8 compared to males 36 ± 18.

Table 1

Effectiveness of ARS-Based Compared to Paper-Based Method

Submission Method		Number of Words Mean±SD	Difficulty to Read Handwriting (in sec) Mean±SD	Number of Spelling Mistakes Mean±SD	Compliance with Activity Time
					First Submission (in min)	Last Submission (in Min)
Paper-Based	Male	41±14	25±8 *	2±2	0	+1.37
	Female	35±16 *	22±11 *	1±2	0	+3.01
	Overall	37±16	23±10 *	2±2	0 *	+2.19 *
ARS-Based	Male	36±18♦	10±5 *♦	1±1	-3.48	0
	Female	47±8 *♦	14±2 *♦	1±1	-5.07	0
	Overall	42±15	12±4 *	1±1	-4.28 *	0 *

*♦ Student t-test was performed to compare both methods. Females type in ARS-based significantly (P<0.05) higher number of words compared to writing on paper-based and to ARS-based submissions in males. A significantly (P<0.05) less time was needed to read females ARS-based submissions compared to reading their handwriting in paper-based. A significantly (P<0.001) less time was needed to read males ARS-based submissions in comparison to reading their handwriting in paper-based. A significantly less time was needed to read ARS-based submissions compared to paper-based among the whole group (P<0.001).Males ARS-based submissions required significantly (P<0.01) less time to read compared to females ARS-based submissions.A statistically significant difference (P<0.05) was found in time of first submission between both methods.Time of last submission was significantly different (P<0.01) between both methods among the whole group

The mean time required to read paper-based submissions in the overall group was 23 ± 10 seconds which was significantly longer than 12 ± 4 seconds in ARS-based submissions (P < 0.001). The time needed to read female submissions in paper-based was 22 ± 11 which was significantly longer compared to ARS-based 14 ± 2 (P < 0.05). The time needed to read female ARS-based submissions 14 ± 2 was significantly (P < 0.01) higher than their male classmates 10 ± 5. A significantly longer time (P < 0.001) was needed to read males’ handwriting in paper-based 25 ± 8 than reading their electronic ARS-based submissions 10 ± 5.

The number of spelling mistakes among the overall class was higher 2 ± 2 compared to 1 ± 1 in paper-based and ARS-based, respectively. Males reported a higher number of spelling mistakes in paper-based 2 ± 2 compared to 1 ± 1 in ARS-based, and higher than their female classmate submissions in both methods.

The first submission received electronically was (-4.28) minutes before the deadline compared to zero submissions received by the paper-based method, and the difference was statistically significant (P < 0.05). On the other hand, the last submission received by the paper-based method was (+2.19) minutes after the deadline, which was statistically significant (P < 0.01) compared to zero submissions electronically [Figure 2]. Regression analysis was performed to compare the association between the number of words and time required to read submissions (indicator for the difficulty in reading handwriting). In the regression model, a statistically significant correlation was observed between the type of method used (ARS-based vs paper-based) and the required time to read it. ARS-based submissions were on average 12.5 seconds faster to read than paper-based submissions when adjusting for the number of words per submission (P < 0.001) with adjusted R square 0.75 [Figure 3].

Figure 2

Compliance with activity deadline measured by time of first and last submissions

Figure 3

Relation between number of words and time required to read it

Figure 4 presents students’ feedback on using electronic ARS. Out of 149, 56.4% of students responded to the feedback survey. Of those, 63.1% preferred using the ARS method throughout the semester and 58 students enjoyed using it. Additionally, 69% preferred using ARS over a paper-based method compared to 21.4% who preferred paper-based. From the respondent group, 79.8% agreed that it was easier to view other groups’ responses and provide immediate feedback, and sixty-two students thought that ARS was effective in teaching clinical cases. A high overall satisfaction rate was reported by 66.7% of respondents compared to 13.1% who did not like using ARS [Figure 5].

Figure 4

Students Feedback on Innovated Electronic ARS

Figure 5

Students’ Satisfaction with the Innovated Electronic ARS Using 5-Likert Scale

All the 84 students responded to the open-ended questions. They perceived the innovated ARS as an easy, fun, and interactive new experience. They reported that this ARS stimulates discussion and deep critical thinking and promotes teamwork. Others enjoyed using their cellphones and described the innovated ARS as fast and efficient technology that is environment friendly. The main disadvantage reported was some difficulty with an internet connection, which could waste some of the activity time.

Discussion

This study attempted to provide an innovative cloud-based ARS that is efficient, time- and cost-saving in comparison to the traditional paper-based method. It was carried out by a single instructor with the same students experiencing both methods and serving as their own controls.

The higher number of submitted words with less time needed to read them together with activity compliance and time gain (4.28 minutes) observed with ARS in comparison to the delay in paper-based (2.19 minutes) indicates high efficiency of using the innovated ARS compared to paper-based. This would help efficiently utilize in-class time and productivity while minimizing waste. Additionally, it preserves the instructors valuable time after class while reading students’ submissions.

Males had a higher tendency to write on papers than typing electronically, yet their handwritings were more difficult to read with more reported spelling mistakes compared to females. Females showed a higher tendency to type electronically with a comparable number of spelling mistakes to writing on papers. These findings could have been influenced by the recorders in each team rather than representing a gender difference however, a pattern was noticed among all teams in both genders. More training on using these technologies could be required especially with male groups prior to initiating those kinds of activities. In ARS, spelling mistakes were underlined with a red dotted line or autocorrected which could explain the lower number of spelling mistakes compared to the paper-based method. Yet, not all spelling mistakes could be detected by spell check options, beside some scientific terminologies could not be recognized by those spelling detectors.

Compliance with activity deadline was evaluated by two outcome measures including the time of first and last submissions received by the instructor in minutes while considering the 10-minutes activity time as a baseline. None of the groups submitted their answers before the 10-minutes timeline in the paper-based method. However, there was observed compliance using ARS-based because students knew once the response button is switched off, no further submissions will be received by the system. Females were less likely to comply with activity deadlines with faster submission electronically and delayed on papers. This reported delay was statistically significant among overall groups, which could strain the session timing and jeopardize the overall learning experience especially with a larger number of students.

In the open-ended questions, students reported ARS as easy, engaging, and stimulated critical thinking which agrees with previous studies.[2433353637] The majority of them liked ARS and preferred to use it in future training which is in line with Barbour et al. and Miller et al.[383940] They thought that ARS was effective in teaching clinical cases and provided an opportunity for them to evaluate their understanding and performance compared to their classmates similar to Uhari et al. and Schackow et al. reports.[214142]

One of the main strengths of the current ARS was allowing students to type texts which were the main limitation of other ARSs.[33] It relied on students’ cellphones/tablets and a freely available cloud-based service that could be incorporated into a standard PowerPoint presentation without a special setting or additional software installment. This eliminates the high operating and maintenance cost of the currently available ARS devices and financial and environmental burden associated with the paper-based method. Even more, this ARS can be embedded into other educational programs and utilized in distance learning and online education to make it more engaging which is of special importance during school closure at the time of the COVID-19 crisis.

Limitations of this ARS included its reliability on internet connection. Poor connection limits students’ ability to submit their responses, they could lose their recorded information and lose some activity time. Although, in-class time was preserved using the innovated ARS, yet preparation time was required in advance. The level of instructor's competency in using technology and being able to prepare those exercises, upload them online, and create a QR code or link correctly is another limitation. Additionally, the severity of the different cases and types of questions presented with both methods might have influenced study findings which could be considered in the future. Further studies enabling students to experience a couple of ARS and paper-based methods with some sort of training prior to distributing feedback survey, and evaluating the quality of provided answers, raised debates, evidence supporting students’ answers could be performed. Lastly, the same method could be applied to individual activities rather than teams to highlight gender differences in future.

Conclusions

In summary, the use of innovated ARS was found to enhance CBL. It is more efficient than the traditional paper-based method in terms of saving in-class time and providing real-time feedback to both instructors and learners. Integrating this model in other teaching strategies including problem-based and team-based learning could enhance student engagement within an educational setting. This is important in distance learning especially with the current global trend of expansion in online education, with the significance heightened during the challenge of school closure at the time of COVID-19 pandemic. It provides a great benefit and opportunity to help educators innovate in their teaching strategies and utilize cost-saving technology to improve students’ learning experiences and achieve better educational outcomes. Finally, the impact of using this innovative ARS on total dollar savings and its applicability to distance learning and online education requires further research.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.