Comparison of Multiple Quantitative Evaluation Indices of Theoretical Knowledge and Clinical Practice Skills and Training of Medical Interns in Cardiovascular Imaging Using Blended Teaching and the Case Resource Network Platform (CRNP).

BACKGROUND This study aimed to compare multiple quantitative evaluation indices of levels of theoretical knowledge and clinical practice skills in training medical interns in cardiovascular imaging based on the use of the blended teaching (BT) online artificial intelligence (AI) case resource network platform (CRNP), including time and frequency indices and effectiveness of the CRNP. MATERIAL AND METHODS The study included 110 medical interns who were divided into the routine teaching (RT) group (n=55) and the blended teaching (BT) group (n=55). The two were assessed using the mini-clinical evaluation exercise (mini-CEX) that assessed clinical skills, attitudes, and behaviors and using an objective written questionnaire. The following four indices were compared between the RT and BT groups: the X-ray score (XS), the computed tomography angiography (CTA) score (CS), the cardiac magnetic resonance imaging (CMRI) score (MS), and the average score (AS). Seven assessment indicators included: the imaging description (ID), the qualitative diagnosis (QD), the differential diagnosis (DD), examination preparation (EP), interview skill (IS), position display (PD), and human care (HC). Indicators of CRNP use included: number of times (TN), average duration (AD), single maximum duration (SMD), and total duration (TD). RESULTS AS significantly correlated with AD (rAD=0.761) and TD (rTD=0.754), and showed moderate correlation with TN (rTN=0.595), but weak correlation with SMD (rSMD=0.404). CONCLUSIONS Levels of theoretical knowledge and clinical practice skills during medical intern training in cardiovascular imaging based on BT using the CRNP teaching technology improved theoretical knowledge and practical skills.

Background

Cardiovascular imaging is a practical clinical subject for medical interns that includes X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, nuclear medicine, and other modern cardiovascular imaging diagnosis techniques [1]. Cardiovascular imaging has become one of the most rapidly developing diagnostic disciplines in clinical medicine and is one of the most highly technical medical imaging methods. However, the teaching of cardiovascular imaging to medical interns and their assessment can be difficult and complex. Routine teaching methods may not make the most of the leading role of teachers in the early stages of training of medical interns, and it is difficult to encourage autonomous learning [2].

Blended teaching (BT) is a modern teaching method that not only requires teachers to use the traditional classroom methods to disseminate knowledge, but also includes online artificial intelligence (AI) technology in intern teaching [3]. The case resource network platform (CRNP) is based on AI technology that can integrate traditional face-to-face intern teaching methods with modern online learning. Because the CRNP is a hybrid clinical teaching method of online and real-time communication and coordination of students, teachers, topics, learning materials, and clinical situations, a good teaching environment is created to achieve the required learning objectives through self-learning and student-teacher interactions [4,5].

Therefore, this study aimed to compare multiple quantitative evaluation indices of levels of theoretical knowledge and clinical practice skills in training medical interns in cardiovascular imaging based on the use of the blended online and offline AI case resource network platform (CRNP), including time and frequency indices and effectiveness of the CRNP.

Material and Methods

Study participants

The study included 110 medical interns in the years 2015 and 2016 who were divided into the routine teaching (RT) group (n=55) and the blended teaching (BT) group (n=55). The two groups of students were assessed using the mini-clinical evaluation exercise (mini-CEX) that assessed clinical skills, attitudes, and behaviors and using an objective written questionnaire. The following four indices were compared between the RT group and the BT group: the X-ray score (XS), the computed tomography angiography (CTA) score (CS), the cardiac magnetic resonance imaging (CMRI) score (MS), and the average score (AS). Seven assessment indicators included: the imaging description (ID), the qualitative diagnosis (QD), the differential diagnosis (DD), examination preparation (EP), interview skill (IS), position display (PD), and human care (HC). Indicators of the CRNP included: number of times (TN), average duration (AD), single maximum duration (SMD), and total duration (TD). The background of teaching theory in the two study groups was similar, and they had the same teachers.

Study design

During theoretical teaching and teaching of clinical practice skills in cardiovascular imaging to medical interns, routine methods with the teacher were used in the RT group. The theoretical explanations of the teacher used multimedia, PowerPoint presentations of images from the archives, the use of teaching models of the heart and cardiovascular circulation system wall charts, and on-site practical classroom teaching. During the teaching process, students were encouraged to ask questions at any time [6,7]. In addition to conventional teaching methods, teachers in the BT group combined online and offline teaching methods. Teachers collected relevant scores for theoretical knowledge and used the CRNP for remote teaching, questions and answers, and independent learning. The main functions and components of the CRNP were the availability of images to download (Figure 1A–1D), images based on AI technology [8], and a series of questions related to the content of the clinical curriculum, designed to the level of student knowledge. Through the collection of the responses to the questions, and online questions and answers, the CRNP provided timely and effective learning support.

Figure 1

(A–D) T2-weighted magnetic resonance imaging (MRI) of the heart. (A) T2-weighted MRI of the heart shows myocardial edema (arrow). (B) Artificial intelligence (AI) post-processing T2-weighted pseudo-color MRI shows myocardial edema (arrow) more intuitively, using a visual impact method (red color shows a high signal, and the blue shows a low signal) to aid the student’s memory. (C) Static left ventricular functional image. (D) Dynamic four-dimensional (4D) left ventricular functional image, using spatial AI to reconstruct the endocardial border (red) and the epicardial border (green). The dynamic film of the left ventricular systolic and diastolic viewed 360° rotationally, and the cross-sectional image from the base (yellow strip) to the apex of the heart from blended teaching (BT) using the online case resource network platform (CRNP).

At the end of the internship, the two groups of students were evaluated comprehensively using a written test of theoretical knowledge and objective questions using the modified mini-clinical evaluation exercise (mini-CEX) that assessed clinical skills, attitudes, and behaviors using an objective written questionnaire. The two groups of students were compared. According to the requirements of the teaching syllabus, the objective questions of theoretical knowledge and the mini-CEX were scored using a 9-point system. The objective questions of theoretical knowledge were scored by the XS, CS, and MS, and the AS was calculated. The differences in the XS, CS, MS, and AS between the RT group and the BT group were compared. The mini-CEX included seven assessment indicators: imaging description (ID), qualitative diagnosis (QD), differential diagnosis (DD), examination preparation (EP), interview skill (IS), position display (PD), and human care (HC). The differences between the ID, QD, DD, EP, IS, PD, and HC between the RT group and the BT group were compared. There were four indicators of the use of the students in the BT group of the CRNP: the number of times (TN), the average duration (AD), the single maximum duration (SMD), and the total duration (TD). The correlation between the AS of the objective questions with the TN, AD, SMD, and TD was analyzed.

Statistical analysis

Data were analyzed using SPSS version 17.0 software (IBM Corp., Armonk, NY, USA). Data were expressed as the mean±standard deviation (SD). One-way analysis of variance (ANOVA) was used with the F-value to compare the two study groups. Pearson’s linear correlation coefficient analysis was used to determine the correlation between the measurement data. Spearman’s rank correlation coefficient analysis was used for correlation analysis between the measurement data and the rank data. A p-value <0.05 was considered to be statistically significant.

Results

Study groups and study indices

Medical interns (n=110) were divided into the routine teaching (RT) group (n=55) and the blended teaching (BT) group (n=55). The two groups were assessed using the mini-clinical evaluation exercise (mini-CEX). The indices measured included the X-ray score (XS), the computed tomography angiography (CTA) score (CS), the cardiac magnetic resonance imaging (CMRI) score (MS), the average score (AS), the imaging description (ID), the qualitative diagnosis (QD), the differential diagnosis (DD), examination preparation (EP), interview skill (IS), position display (PD), and human care (HC). Indicators CRNP use included the number of times (TN), average duration (AD), single maximum duration (SMD), and total duration (TD).

Theoretical knowledge scores

The scores of the theoretical knowledge objective questions of the RT group and BT group were: XS 6.56±1.36 vs. 6.73±1.28; CS 5.76±1.95 vs. 6.53±1.46; MS 4.73±2.23 vs. 5.69±1.88; AS 5.69±1.18 vs. 6.32±0.93. Comparison between the RT group and the BT group, showed significant differences for the CS (P=0.022 <0.05), MS (P=0.016 <0.05) and AS (P=0.002 <0.05). The difference for the XS was not statistically significant (P=0.517 >0.05). These results are shown in Table 1.

Table 1

Comparison of the scores of the theoretical knowledge objective questions of the routine teaching (RT) group and the blended teaching (BT) group.

	RT group	BT group	F-value	p-value
X-ray score	6.56±1.36	6.73±1.28	0.422	0.517
CTA score	5.76±1.95	6.53±1.46	5.385	0.022*
CMRI score	4.73±2.23	5.69±1.88	5.987	0.016*
Mean score	5.69±1.18	6.32±0.93	9.670	0.002*

Statistically significant difference.

CTA – computed tomography angiography; CMRI – cardiac magnetic resonance imaging; RT – routine teaching; BT – blended teaching. F-value from analysis of variance (ANOVA).

The mini-clinical evaluation exercise (mini-CEX) scores of the RT group and the BT group were: ID 5.93±1.49 vs. 6.53±1.35; QD 5.29±2.16 vs. 5.87±1.75; DD 4.47±2.31 vs. 4.74±2.21; EP 5.85±1.89 vs. 6.00±1.69; IS 6.76±1.33 vs. 6.58±1.42; PD 5.93±1.56 vs. 6.56±1.46; HC 4.87±2.14 vs. 4.71±2.25. Compared with the RT group and the BT group, the differences in ID (p=0.029 <0.05) and PD (p=0.030 <0.05) were significant. The differences in the QD (p=0.124 >0.05), DD (p=0.528 >0.05), EP (p=0.671 >0.05), IS (p=0.491 >0.05) and HC (p=0.697 >0.05) were not statistically significant. These results are shown in Table 2.

Table 2

Comparison of the mini-clinical evaluation exercise (mini-CEX) scores of the routine teaching (RT) group and the blended teaching (BT) group.

	RT group	BT group	F-value	p-value
Imaging description	5.93±1.49	6.53±1.35	4.918	0.029*
Qualitative diagnosis	5.29±2.16	5.87±1.75	2.408	0.124
Differential diagnosis	4.47±2.31	4.74±2.21	0.400	0.528
Examination preparation	5.85±1.89	6.00±1.69	0.181	0.671
Interview skill	6.76±1.33	6.58±1.42	0.478	0.491
Position display	5.93±1.56	6.56±1.46	4.865	0.030*
Humanistic care	4.87±2.14	4.71±2.25	0.153	0.697

Statistically significant difference.

RT – routine teaching; BT – blended teaching. F-value from analysis of variance (ANOVA).

The AS of the objective questions had a strong correlation with AD (rad=0.761, p=0.000 <0.001) and TD (rtd=0.754, p=0.000 <0.001), a moderate correlation with TN (rtn=0.595, p=0.000 <0.001), but a weak correlation with SMD (rsmd=0.404, p=0.002 >0.001), respectively. These results are shown in Figure 2A–2D.

Figure 2

(A–D) Representation of the average score of objective questions and the number of times (NT), the average duration (AD), the single maximum duration (SMD), and the total duration (TD). Spearman’s rank correlation between the average score of objective questions and (A) the number of times (NT); (B) the average duration (AD); (C) the single maximum duration (SMD); and (D) the total duration (TD), of blended teaching (BT) from the online case resource network platform (CRNP).

Discussion

This study aimed to compare theoretical knowledge and clinical practice skills during the training medical interns in cardiovascular imaging based on the use of blended teaching (BT) using the online artificial intelligence (AI) case resource network platform (CRNP). Indices were compared between the routine teaching (RT) group and the BT group and included the X-ray score (XS), the computed tomography angiography (CTA) score (CS), the cardiac magnetic resonance imaging (CMRI) score (MS), and the average score (AS). Seven assessment indicators included the imaging description (ID), the qualitative diagnosis (QD), the differential diagnosis (DD), examination preparation (EP), interview skill (IS), position display (PD), and human care (HC). Indicators of CRNP use included the number of times (TN), average duration (AD), single maximum duration (SMD), and total duration (TD).

Cardiovascular imaging training is an important component of the internship and requires students to have solid basic knowledge and to be able to flexibly use theoretical knowledge in practice during intern training [1]. Currently, medical interns should not rely only on traditional textbooks but are required to use online training combined with hands-on practical training [9]. The main task of BT is to improve traditional teaching with diverse teaching methods that use online information, combined with traditional case-based learning and problem-based learning in the classroom, supplemented with small courses combining theory, reading, examinations, and the mini-clinical evaluation exercise (mini-CEX) or active learning [5].

The cardiovascular imaging component of interns training requires that the student master not only theoretical knowledge but also develop analytical skills. For the teacher in cardiovascular imaging, the organization of the teaching methods and content are important. In addition to the effective management of teaching activities, it is also necessary to update teaching concepts, apply advanced teaching technology such as the CRNP, and combine traditional teaching methods to provide high-quality teaching to large numbers of students [10]. In the era of information networks, there are also various extracurricular learning methods for students who intern in cardiovascular imaging, such as medical applications (apps), online forum case discussions, and cardiovascular imaging libraries [11,12]. The CRNP online teaching has the advantages of flexible teaching that is not limited by time and place, facilitating the adoption of participatory and interactive teaching, and improving the subjective initiative of learning. The CRNP is not only used for medical imaging interns, due to its content design.

The findings from the present study showed that the BT mode based on the CRNP might be helpful to improve the CS, MS, and AS in the theoretical knowledge of cardiovascular imaging, but may not significantly improve the XS. The reason might be that X-ray was easier to learn than computed tomography angiography (CTA) and cardiac magnetic resonance imaging (CMRI). Also, cardiovascular X-ray was not as widely used as chest X-ray, so that the content and quantity of clinical case images was similar between the two study groups. Although there was no significant difference in XS between the two groups, AS was not significantly affected, and AS between the two groups was more significant than TS and MS. The greater difference between the two groups for the MS than in TS might have been because CMRI was more difficult for students to master than CTA in conventional teaching, while the CRNP might help students to improve their understanding and practical ability for knowledge points with greater difficulty. However, the simpler knowledge points involved in X-ray were of little significance.

This study also showed that the BT mode based on the CRNP might be helpful to improve ID and PD in the practical ability of the mini-CEX in cardiovascular imaging. The difference between the ID of the two groups was statistically significant, while the difference between the QD and DD was not statistically significant. The reason might be due to the overall ability level of the students. Currently, the students’ abilities for description and interpretation of normal or abnormal CTA density and CMRI signal intensity, and for normal or abnormal morphology, such as cardiac cavity enlargement or muscle wall thickening, might have reached the standard levels. However, it is still difficult to master qualitative and differential diagnosis. For example, students might describe the enlargement of heart cavity but might not identify it as being due to dilated cardiomyopathy, which is more difficult to distinguish than an old myocardial infarction with ventricular remodeling. Also, the students might describe the thickening of the cardiac wall, but might not identify it as hypertrophic cardiomyopathy, which is more difficult to distinguish from compensatory hypertensive heart disease with centripetal hypertrophy.

In the present study, the main reason why the differences in the EP, IS, and HC between the two study groups were not significantly different might have been due to the programming and content of the CRNP, as problems in the development of many types of internet teaching have previously been described [13]. For the EP, the CRNP only added the text and did not update the corresponding imaging and flow charts in time to help students understand and learn the content for the IS and the CRNP did not have the corresponding instruction video for students to learn. These factors were the limitations of the CRNP that are areas that remain to be optimized. Secondly, the IS might have been affected by subjective factors of interview raters, while the mini-CEX requires further improvement and optimization [14]. The HC may be the most difficult item to improve in the program content of the current CRNP. Future studies may include data analysis of large-scale and long-term studies from medical student to interns to standardize the training of medical residents after graduation, as previous studies have shown that the communication skills in the doctor and patient communication are not improved in a short time [15].

This study showed that there was a significant difference in the PD between the RT and BT groups. For PD, the BT group had a higher level of accuracy than the RT group, which may have been due to mirror image interpretation problems. For example, the left side of a structure in the naked eye image was the right side of the structure. Because the CRNP focused on the auxiliary memory of clinical case images, the BT group’s understanding of left and right was reinforced, and low-level errors may have been avoided. The AS of the objective questions had a strong correlation with the AD and TD, a moderate correlation with TN, and a weak correlation with SMD. The TN showed reproducibility, which could effectively avoid forgetting and improve learning efficiency [16]. However, the reproducibility of testing depends on the number of repetitions and the duration of repetitions and reinforcement. The TD reflected the duration of long-term reinforcement and might represent the diligence and efforts of the students. The AD reflected the regularity of long-term strengthening and might represent the degree of self-discipline. The SMD might represent the effects of cramming, which indirectly showed that short-term learning methods could not effectively improve the use of the CRNP.

Conclusions

This study aimed to compare multiple quantitative evaluation indices of levels of theoretical knowledge and clinical practice skills in training medical interns in cardiovascular imaging based on the use of the blended teaching (BT) online artificial intelligence (AI) case resource network platform (CRNP), including time and frequency indices and effectiveness of the CRNP. The levels of theoretical knowledge and clinical practice skills in the training of medical interns in cardiovascular imaging based on the BT mode of the CRNP teaching technology improved theoretical knowledge and practical skills.