Artificial intelligence-tutoring problem-based learning in ophthalmology clerkship.

BACKGROUND: Artificial intelligence (AI) is an increasingly popular tool in medical investigations. However, AI's potential of aiding medical teaching has not been explored. This study aimed to evaluate the effectiveness of AI-tutoring problem-based-learning (PBL) in ophthalmology clerkship and to assess the student evaluations of this module.
METHODS: Thirty-eight Grade-two students in ophthalmology clerkship at Sun Yat-Sen University were randomly assigned to two groups. In Group A, students learned congenital cataracts through an AI-tutoring PBL module by exploring and operating an AI diagnosis platform. In Group B, students learned congenital cataracts through traditional lecture given with the same faculty. The improvement in student performance was evaluated by comparing the pre- and post-lecture scores of a specific designed test using paired-T tests. Student evaluations of AI-tutoring PBL were measured by a 17-item questionnaire.
RESULTS: The post-lecture scores were significantly higher than the pre-lecture scores in both groups (Group A: P<0.0001, Group B: P<0.0001). The improvement of group A in the part of sign and diagnosis test (Part I) was more significant than that of group B (P=0.016). However, there was no difference in the improvement in the part of treatment plan test (Part II) between two groups (P=0.556). Overall, all respondents were satisfied and agreed that AI-tutoring PBL was helpful, effective, motive and beneficial to help develop critical and creative thinking.
CONCLUSIONS: The application of AI-tutoring PBL into ophthalmology clerkship improved students' performance and satisfaction. AI-tutoring PBL teaching showed advantage in promoting students' understanding of signs of diseases. The instructors play an indispensable role in AI-tutoring PBL curriculum. 2020 Annals of Translational Medicine. All rights reserved.

Introduction

The development of artificial intelligence (AI) provides new tools to solve problems in various medical fields (1). AI algorithms have obvious advantages in identifying and extracting image features (2). Therefore, AI models have been shown to performed equally with or better than human doctors in disease diagnosis based on signs or morphologies, such as skin cancer (3), diabetic retinopathy (4-6), congenital cataract (7), and pathological section identification (8-10). However, AI’s potential of aiding medical teaching has not been explored.

Problem-based learning (PBL) is a learner-centered pedagogy, in which students learn knowledge by solving several open-ended problems found in a trigger material, and take a center stage in case-based, self-directed learning (11,12). PBL enables students to learn more profoundly and improves students’ self-confidence and self-learning ability (13-15).

In recent years, the rapid development of AI algorithm has contributed to the innovation of pedagogy (16). Currently application of AI in teaching is mainly in the form of a robot teacher and/or a robotic examination system, and the AI system replaces teachers’ role in teaching and/or performance-evaluating. Taking the advantages of AI algorithm in image recognition and its widely use in medical field, we proposed an alternative approach of applying AI in medical teaching. We have developed an AI-tutoring PBL module, in which students learn ophthalmic knowledge through solving questions related to the CC-Cruiser, the AI pediatric cataract consultation platform developed by Zhongshan Ophthalmic Center. We validated the effectiveness of the AI-tutoring PBL module by comparing it to the traditional lecture-based teaching in ophthalmology clerkship.

Methods

Participant enrollment and assignment

Thirty-eight Grade-two students participated in ophthalmology clerkship at Sun Yat-Sen University were enrolled in August 2018. The participants were randomly assigned to two groups. In Group A, students learned congenital cataracts through an AI-tutoring problem-based-learning module by exploring and operating an AI diagnosis platform. In Group B, students learned congenital cataracts through traditional lecture given with the same faculty. All the procedures in this study were arranged strictly with the approval of the institutional review board of Zhongshan Ophthalmic Centre of Sun Yat-sen University (IRB-ZOC-SYSU). Written informed consents have been obtained from all students. Flowchart of the study was specific (). Both groups had a pre-lecture test, a 45-minute lecture and an after-lecture test. The same examination paper was used for pre- and after-lecture test. The test consisted of two parts. Part I was about the signs and diagnosis of pediatric cataract, and Part II was about the treatment of pediatric cataract. Both groups were taught by an experienced instructor who is a physician in Home of Cataract Children in Zhongshan Ophthalmic Centre.

Figure 1

Flowchart of the study.

CC-cruiser, the AI consultation platform for pediatric cataract

CC-Cruiser is a pediatric cataract consultation platform developed by our team. This AI platform uses slit lamp photos of pediatric cataracts to identify and classify the location, density and extent of cataract lesions, and provides treatment recommendations for either conservative treatment or surgical treatment. Trained with a large number of labeled ocular images, this AI platform performed as good as ophthalmic experts in diagnosis and evaluation of pediatric cataract (7).

AI-tutoring PBL module

One day before class, students in group A got the introduction materials on CC-cruiser as well as key points for diagnosis and treatment of pediatric cataract. Besides, the tutor provided five questions on diagnosis and treatment of pediatric cataract to students and encouraged them to discuss the questions before class. In the class, 10 minutes were given to students to finish the pre-lecture test. Then the students used 25 minutes to diagnose and evaluate patient with the AI consultation platform, CC-cruiser. Finally, a 10-minute wrapped-up discussion about the diagnosis and treatment of congenital cataract was carried out.

Design of the traditional lecture

One day before class, students in group B got key points for diagnosis and treatment of pediatric cataract. The tutor provided five questions, the same as for Group A, on diagnosis and treatment of pediatric cataract to students and encouraged them to discuss the questions before class. During the class, 10 minutes were given to students to finish the pre-lecture test. Then the tutor gave a 25-minute traditional lecture on congenital cataract. Finally, a 10-minute wrapped-up discussion about the diagnosis and treatment of congenital cataract was carried out.

Student evaluations

Student evaluations of AI-tutoring PBL were measured by a 17-item questionnaire, including 16 one-choice question and one open-ended question ().

Table 1

Seventeen-item questionnaire

No.	Question
One-choice questions (A, strongly agree; B, agree; C, disagree; D, strongly disagree)
1	AI-tutoring PBL helped me to acquire a higher level of knowledge
2	AI-tutoring PBL is more effective and motivate compared with traditional didactic lecture
3	AI-tutoring PBL challenged me to do my best
4	AI-tutoring PBL promoted the learning of essential concepts or skills
5	AI-tutoring PBL promoted effective cooperative learning
6	AI-tutoring PBL promoted increased reading of the textbook by the students
7	Overall, I am very satisfied with this AI-tutoring PBL
8	AI-tutoring PBL should be offered more frequently in the curriculum
9	I will recommend AI-tutoring PBL to other students
10	This activity was preferable to the traditional lecture format
11	The AI-tutoring PBL module is well organized
12	I study with colleagues frequently
13	The instructor highly facilitated the learning process during the process of AI-tutoring PBL.
14	The instructor responded well to questions asked by students.
15	The instructor encouraged and provided opportunities for discussion.
16	AI-tutoring PBL is beneficial to help develop critical and creative thinking.
Open-ended question
17	Compared with the traditional teaching method, what do you think are the advantages and disadvantages of AI-tutoring PBL?

AI, artificial intelligence; PBL, problem-based learning.

The questionnaire in our study were designed according to the previous studies in other subjects of medical education (17). The questionnaire was conducted in Group A after class to collect and assess student satisfaction on AI-tutoring PBL module. Information of the questionnaire have several topics, which included: knowledge acquisition (2 items), motivational dimension (3 items), group cooperation (1 item), creative and critical thinking (1 item), instructor performance (3 items), organization (1 item), overall rating (2 item), recommendations (2 items). Statements were rated on a 4-point scale ranging from “strongly agree” (with the highest score) to “strongly disagree” (with the lowest score).

Data analysis

All data were collected non-anonymously. The effects of the AI-tutoring PBL module on students’ performance were measured by comparing pre- and after-lecture scores using a paired t-test. Independent t-test was used to compare the improvement of student performance between Group A and B. A P value less than 0.050 was considered statistically significant. All analyses were conducted by SPSS software version 22.0 (SPSS Inc., Chicago, IL, USA).

Results

Baseline characteristics

All participants took part in the class and completed examinations as required. There was no difference in baseline characteristics between two groups including gender, pre-lecture scores of Part I (P=0.469), Part II (P=0.097) and total scores (P=0.158) ().

Table 2

Baseline characteristics of subjects

Characteristics	Group A	Group B	P value
Male (n, %)	7, 36.8%	7, 36.8%
Pre-lecture scores (mean ± SD)
Part I: diagnosis	1.12±1.09	1.36±0.94	0.469
Part II: treatment	4.21±1.99	3.26±1.37	0.097
Total scores	5.33±1.90	4.62±0.98	0.158

SD, standard deviation.

Improvement of student performance in each group

The post-lecture scores were significantly higher than the pre-lecture scores in both groups (Group A: P<0.001, Group B: P<0.001). The improvement of both groups in the part of sign and diagnosis test (Part I) was significantly. (Group A: P<0.001, Group B: P<0.001). However, the improvement of group A in treatment plan test (Part II) was not obvious (P=0.083) ().

Table 3

Improvement of student performance in each group

Test content	Difference in post- to pre-lecture scores (mean ± SD)	P value
Group A
Part I: diagnosis	7.11±1.94	0.000*
Part II: treatment	0.95±2.25	0.083
Total scores	8.06±2.86	0.000*
Group B
Part I: diagnosis	5.39±2.26	0.000*
Part II: treatment	1.37±2.11	0.011*
Total scores	6.76±3.33	0.000*

*, statistically significant. SD, standard deviation.

Comparison of student performance between Group A and B

The total and the Part I score of group A were higher than those of group B (P=0.034 and 0.033 respectively). Compared with the pre-lecture test, the improvement of group A in Part I was more obvious than that of group B (P=0.016), which may be the main reason why the total post-lecture score of group A was significantly higher than that of group B ().

Table 4

Comparison of student performance between Group A and B

Test content	Group A (mean ± SD)	Group B (mean ± SD)	P value
Post-lecture scores
Part I: diagnosis	8.24±1.84	6.75±2.26	0.033*
Part II: treatment	5.16±2.14	4.63±1.89	0.427
Total scores	13.39±2.40	11.38±3.17	0.034*
Increase in post- to pre-lecture scores
Part I: diagnosis	7.11±1.94	5.39±2.26	0.016*
Part II: treatment	0.95±2.25	1.37±2.11	0.556
Total scores	8.06±2.86	6.76±3.33	0.203

Student satisfaction

All students in group A responded to the questionnaire. Overall, all respondents were satisfied and agreed that AI-tutoring PBL was helpful, effective, motive and beneficial to help develop critical and creative thinking. Meanwhile, they agreed that the instructor played an important role in AI-tutoring PBL curriculum ().

Figure 2

Students’ responses to Artificial-tutoring problem-based learning courses. Each survey question used a four-point scale.

We also collected students’ answer to the open-ended question: what are the advantages and disadvantages of AI aided PBL teaching methods? Many students confirmed that AI-tutoring PBL benefits them, in terms of preview, communications, collaborations, self-learning ability and critical thinking. However, some students complained on higher requirement of hardware and that they need to spend more time previewing.

Discussion

Clerkship is a transition from general education to learning in a specific field (18). Students in this phase have the advantages of active thinking, strong learning ability, and are eager to accept new concepts and technologies (19,20). Therefore, we combined an AI diagnosis platform with a model of active learning, PBL to set up an AI-tutoring PBL module to improve student performance in ophthalmic clerkship.

We found some interesting results. Group A improved more greatly in the sign and diagnosis test (Part I) compared to Group B, which indicates that the AI model based on image features is better than traditional teaching in deepening medical students’ understanding of signs and morphological features. The AI platform does not directly replace part of the teacher’s work. What CC cruiser presents to students is the disease-evaluating ability that AI has gained through learning from human experts. The AI platform extracts the characteristics of the patient’s ocular image using convolutional neural networks for diagnosis and treatment decision-making (7). Based on the input ocular images and the corresponding evaluation made by the AI platform, the students were encouraged to discuss and collaborate to learn knowledge on diagnosis and treatment of pediatric cataract. The post-lecture scores of Part II were similar in both groups, which demonstrated that the AI module was equally effective to the traditional lecture in conveying knowledge on treatment strategy.

Our anonymous questionnaire gave a subjective assessment of AI-tutoring PBL from students in terms of knowledge acquisition, motivational dimension, cooperation, creative and critical thinking, instructor performance, and preferences. Consistent with the results of the test scores, students agreed that AI-tutoring PBL helps improve student initiative, collaboration, creativity and critical thinking. What is more, the students all agreed that the role of instructor in AI-tutoring PBL is indispensable. The instructor highly facilitated the learning process, also encouraged and provided opportunities for discussion. Consistently, studies showed that the sophisticated role of PBL tutor strikes the right balance between process facilitation and information delivery (21).

The results of this study should be assessed within the context of its limitations. Our data showed that AI-tutoring PBL module could be more effective in conveying knowledge on diseases with specific signs, such as pediatric cataract. However, the effects of this module in teaching other types of diseases require further study. Secondly, the number of participants is relatively small and the results require to be validated with larger sample size.

In summary, application of AI-tutoring PBL into ophthalmology clerkship improved students’ performance and satisfaction. AI-tutoring PBL teaching showed advantage in promoting students' understanding of signs of diseases. The instructors play an indispensable role in AI-tutoring PBL curriculum.

The article’s supplementary files as