Effects of problem-based learning in Chinese radiology education: A systematic review and meta-analysis.

BACKGROUND: In recent years, the problem-based learning (PBL) teaching method has been extensively applied as an experimental educational method in Chinese radiology education. However, the results of individual studies were inconsistent and inconclusive. A meta-analysis was performed to evaluate the effects of PBL on radiology education in China.
METHODS: Databases of Chinese and English languages were searched from inception up to November 2017. The standard mean difference (SMD) with its 95% confidence interval (95% CI) was used to determine the over effects of PBL compared with the traditional teaching method.
RESULTS: Seventeen studies involving 1487 participants were included in this meta-analysis. Of them, 16 studies provided sufficient data for the pooled analysis and showed that PBL teaching method had a positive effect on achieving higher theoretical scores compared with the traditional teaching method (SMD = 1.20, 95% CI [0.68, 1.71]). Thirteen studies provided sufficient data on skill scores, and a significant difference in favor of PBL was also observed (SMD = 2.10, 95% CI [1.38, 2.83]). Questionnaire surveys were applied in most of the included studies and indicated positive effects of PBL on students' learning interest, scope of knowledge, team spirit, and oral expression.
CONCLUSION: The result shows that PBL appears to be more effective on radiology education than traditional teaching method in China. However, the heterogeneity of the included studies cannot be neglected. Further well-designed studies about this topic are needed to confirm the above findings.

Introduction

Problem-based learning (PBL) was introduced at the end of the 1960s by Barrows and Tamblyn at McMaster University. There are 4 key principles of PBL, including contextual learning, information processing, cooperative learning, and self-determination.[ For modern educational theory, an ideal teaching method is thought to be beneficial for students learning, in terms of critical thinking, knowledge acquisition, practice skill, and lifelong professional learning.[ It is claimed that PBL meets these requirements, and inspires more comprehensive and profound learning. Therefore, the traditional approach of sequential study has been extensively replaced by an approach based on integrated PBL since the latter part of the twentieth century.[

Radiology is an important branch of medicine that involves in the diagnosis of almost all diseases. PBL shows to be an appropriate tool for the sub specializations of radiology since most of the PBL scenarios are based on organ systems.[ Different from traditional teaching method, PBL is a student-centered teaching approach. There are many advantages to integrate PBL of radiology with clinical scenarios. It places radiology in a clinical context and allows students to obtain information from a wide range of viewpoints. Students learn and comprehend about a clinical subject through the experience of practice in a tutor-guided small group. For example, important and urgent clinical cases such as a cerebral hemorrhage or a cerebral infarction can be effectively identified and taught around a simple brain computed tomography image. The role of radiology in patient diagnosis and management can be more smoothly interpreted by students through this study model. In addition to theoretical knowledge and practice skill, it is also helpful for cultivating student clinical thinking and lifelong professional learning.[

In China, PBL application in radiology education lags behind that developed countries for many reasons, including different curricula arrangement, numerous medical students in 1 class, deficiency of enough tutors, and high tuition fees for PBL education.[ In recent years, PBL teaching model has been extensively introduced in radiology curricula as an experimental educational method. Many studies have demonstrated that the students in a PBL group have better knowledge scores and problem solving skills than those in a traditional group. However, the results of those studies are not easy to be accessed by international researchers as most of them have been published in Chinese. Since most studies are with relatively small sample size, it is necessary to perform a meta-analysis study to quantitatively combine of the existed evidence. The aim of current study was to assess the overall effects of PBL compared with traditional teaching model in Chinese radiology education.

Methods

Search strategy

We searched Chinese National Knowledge Infrastructure, Wanfang Data (Chinese database), VIP Information (Chinese database), Chinese Biomedical Literature, and English language databases, including PubMed and Embase. The databases were systematically searched from inception up to November 2017. The following medical subject heading terms and keywords were used: “problem-based learning OR PBL” AND “China OR Chinese” AND “radiology OR radiology education OR radiolog∗.” There were no language restrictions. All abstracts mentioned PBL in Chinese radiology education were selected for further consideration. Moreover, the reference list of each study was screened for relevant studies.

Inclusion criteria

The inclusion criteria were as follows: participants: medical students in Chinese medical universities or colleges; intervention: PBL teaching in the experimental group; comparators: traditional teaching or lecture-based learning in the control group; outcomes: theoretical scores and/or skill scores of radiology; study design: controlled trials in radiology education. If there were duplicated studies identified, then the study with the largest sample size was retained.

Data extraction and quality assessment

Two independent reviewers identified and reviewed the relevant trials. In cases of disagreements, a third reviewer assessed the study to obtain a consensus. The key parameters extracted from each study were summarized in a table and included the following items: first author's name, publication year, the involved radiology courses, sample size (PBL group and control group), participants’ characteristics, intervention, comparator, and duration of intervention time. The quality assessment of including studies was evaluated with the risk of bias table according to the Cochrane Collaboration.

Statistical analysis

The extracted data were analyzed using the Review Manager 5.3 software (Cochrane Library Software, Oxford, UK) and STATA 12.0 software (STATA Corporation, College Station, TX). Since continuous outcomes from different scales were extracted, the standardized mean difference (SMD) with 95% confidence interval (CI) was applied as effect size to calculate for each study. Forest plots were drawn to show the point estimates of each study related to the pooled results. Homogeneity tests were based on the Q statistic and I2 statistic. In this study, P < .05 or I2 value > 50% were considered to be statistically significant. A random-effects model or, in the absence of heterogeneity, a fixed-effects model was applied to combine the SMD with 95% CI. If heterogeneity was noted, a sensitivity analysis was conducted to investigate the influence of each study on the overall estimate by omitting each study in turn. Publication bias was detected by Begg's funnel plots and Egger's test with STATA 12.0 software.

Results

Search results

The flow chart of the article selection process is shown in Figure 1. The related databases were searched based on the predescribed search strategy to identify the potentially eligible studies. We initially obtained 645 relevant articles. After screening the titles and abstracts, 624 studies were excluded either because of duplicate publications, if they were letters or reviews, or with different topics. Twenty studies were potentially appropriate and assessed for eligibility according to the pre-established inclusion criteria. Two studies were excluded because they did not provide sufficient data.[ Another 1 study was further excluded as it focused on the effects of PBL on Chinese refresher doctors in radiology education.[ Thus, a total of 17 studies were included in this meta-analysis.[

Figure 1

Flowchart of selection process of the included studies.

Study characteristics and study quality

The characteristics of the included studies are exhibited in Table 1. All of them were published in Chinese language between 2009 and 2016. The effects of PBL on radiology compared with traditional teaching were evaluated by the theoretical scores and/or skill scores. The sample sizes ranged from 15 to 90 students in PBL group and 15 to 90 in the control group. A total of 1487 participants were included in this study. Despite the school systems were different; most of the studies were performed with the 5-year system medical students. Five studies[ were carried out with interns and 12 studies[ were carried out with probationers. The included studies were also different in the duration time for the curriculum, of which 7 studies[ were 1 semester, 1 study[ was 4 weeks, and 1 study[ was 3 months, while 7 studies[ did not mention it. Most of the included studies (except for 2 studies)[ have used the questionnaire results to show that PBL are superior to traditional teaching methods in many scopes, such as improving students’ learning interest, scope of knowledge, self-learning ability, participation in class discussion, team cooperation, clinical thinking, and oral expression. Figure 2 shows the risk of bias assessment of the 17 included studies. The panel shows each quality item as percentages across all studies, including the aspects of random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcomes assessment, incomplete outcome data, selective reporting, and other bias.

Table 1

Main characteristics extracted from the included studies.

Figure 2

Summary of each methodological quality item presented as percentages across all included studies.

Effects of PBL on knowledge scores

The scores on the knowledge exam of radiology were used to evaluate how well the students mastered the theoretical knowledge. The effects of PBL compared to the traditional teaching on knowledge scores were reported in 16 of the included studies (except 1 study, which only provided data of skill scores),[ involving 1455 medical students (PBL group = 727, control group = 728). Since a high degree of heterogeneity was noted across the included studies (I2 = 95%, P < .00001), the random-effects model was used for the meta-analysis. The analytical results showed a significant difference in knowledge scores (SMD = 1.20, 95% CI [0.68, 1.71]) in favor of PBL, compared with the traditional teaching (Fig. 3). To identify the effect of PBL on different participants, we divided the students into 2 subgroups, including probationer subgroup[ and intern subgroup.[ The subgroup analysis also showed that a statistically significant difference in the probationer subgroup (SMD = 1.32, 95% CI [0.67, 1.97]) and the intern subgroup (SMD = 0.82, 95% CI [0.28, 1.36]).

Figure 3

Forest plot for the effects of PBL on knowledge scores compared with the traditional teaching. PBL = problem-based learning

Effects of PBL on skill scores

The scores in the skill test of radiology were used to assess the students’ clinical practice for film reading. Thirteen studies[ involving 1090 medical students (PBL group = 545, control group = 545) provided sufficient data for this analysis. The random-effects model was used to combine the SMD values because of significant heterogeneity (I2 = 95%, P < .00001) in the 13 studies. The result showed a significant difference in skill scores (SMD = 2.10, 95% CI [1.38, 2.83]) in favor of PBL, compared with the traditional teaching (Fig. 4). Meanwhile, we conducted a subgroup analysis to identify the effect of participants’ distribution on the skill scores. The results also showed a statistically significant difference in the probationer subgroup (SMD = 2.45, 95% CI [1.36, 3.54])[ and intern subgroup (SMD = 1.56, 95% CI [0.86, 2.27]).[

Figure 4

Forest plot for the effects of PBL on skill scores compared with the traditional teaching. PBL = problem-based learning

Sensitivity analysis and publication bias

To evaluate the stability of the results, sensitivity analyses were performed. For the overall SMD values of knowledge and skill, similarly statistical results were obtained (results not shown) by excluding individual studies sequentially, suggesting the stability of the meta-analysis. Begg's funnel plots of the SMD against the standard error of SMD showed no substantial asymmetries in knowledge scores (Fig. 5A) and skill scores analysis (Fig. 5B). Egger's regression test also showed no evidence of publication bias in knowledge scores (t = 1.70, P = .112) and skill scores analysis (t = 1.54, P = .151).

Figure 5

Funnel plot of the included studies for publication bias. (A) Evaluation of knowledge scores, divided into probationer and intern subgroups. (B) Evaluation of skill scores, divided into probationer and intern subgroups.

Discussion

The PBL teaching model is an educational innovation developed in response to dissatisfaction with traditional medical education. In recent years, it has been extensively applied in Chinese radiology education as an experimental educational method.[ Up to now, many studies have investigated the effects of PBL compared with traditional teaching method, while with inconclusive or inconsistent results. Therefore, a meta-analysis including 1487 participants from 17 individual studies was performed to evaluate the effects synthetically. Despite a high heterogeneity existed in the included studies, the radiology students in the PBL group had better knowledge scores and skill scores than those in the traditional teaching method group. To our knowledge, this is the first systematic review and meta-analysis of the effects of PBL in Chinese radiology education.

The synthesized estimates showed that PBL was superior to traditional teaching method in both theoretical and practical scores. Our results are in line with the findings of recent PBL studies in Chinese pharmacy,[ pediatrics,[ physical diagnostics,[ and dental education.[ However, the performance of PBL has been questioned in Western countries. Some studies even reported that PBL has a negative effect on knowledge exams due to the acquisition of factual knowledge.[ There are some possible reasons to explain the inconsistent results. First, the differences in higher medical education between China and the Western countries are needed to be taken into consideration.[ The PBL teaching model is a novelty for most of Chinese students, because they have not been exposure to it since primary education. Their learning interests can be more easily and strongly inspired by the PBL teaching method, which is helpful in facilitating the knowledge learning processes.[ Second, unlike other medical subjects, PBL suits the subspecialization of radiology subject due to most of the PBL problems and the discipline design of radiology are based on organ systems.[ It allows different imaging modalities to be taught in an integrated clinical scenario. Thus, in radiology subject, the knowledge score in PBL group was higher than that of the traditional teaching method group. Third, PBL encourages students to apply their knowledge to solve practical problems. The superiority of PBL compared with traditional teaching method showed to be more evident when considering the clinical skills.[

In addition to improve knowledge and skill scores, the results have also demonstrated that PBL could make the students carry on the study with better physical and mental state.[ Most of the included studies have used the questionnaires to illustrate the results. The estimation scales covered many different scopes, including improving students’ learning interest, scope of knowledge, self-learning ability, participation in class discussion, team cooperation, clinical thinking, and oral expression. It is difficult to pool these different scales together and quantitatively analyze them. However, the reasons behind the positive effects need to be further explained. Unlike the traditional teaching method most based on lecture-based learning (LBL), PBL is famous in stimulating the learning interests and promoting the self-study ability. The traditional teaching method makes the students rely on passive acceptance of knowledge. By contrast, the PBL pedagogy makes the students study constructively and energetically. Thus, the students can improve the abilities of data collection, information retrieval, and human communication through the problem solving processes. Moreover, it is crucial to cultivate the team working and interpersonal communication abilities for Chinese students, as they are taught to be humility and courtesy since their childhood period.[

Despite many advantages of PBL described above, it is difficult to extensively apply PBL in Chinese radiology education.[ First, the PBL teaching method takes too much time. Fan et al[ surveyed the PBL usage and reported that most Chinese schools used PBL for less than 50% of total clinical curricular hours. Their results suggest that medical schools need to make a balance between the positive effects of PBL and the costing of teaching time and resources.[ Second, the education system in China is characterized by traditional education method, which has been used for many decades. The traditional education method is a teacher-oriented model and paid much attention on how to acquire high theoretical scores by passive learning. It is hard to change the current trend in a short time. Third, some individual students negatively participate in PBL curricula or, they just notice the superficial phenomenon without deep consideration. The dependent mentality limited their knowledge acquisition and practice ability. Therefore, similar to other teaching methods, PBL also has its own defects. The teaching method should be appropriately applied according to the current situation of Chinese education, teaching contents of radiology subject, and students’ specific circumstance.

There are some limitations of the present study. A shortage of large-scale studies on the effects of PBL in Chinese radiology education is an obvious deficiency. The included studies were relatively low in study quality as they were not randomized controlled trials. The experimental and control groups were always divided based on the classes rather than the individual students. It is difficult for the investigators and participants to implement the double blinding method through the whole study period. Therefore, the selection bias and performance bias were unavoidable. Besides, both the standards and levels of radiology education vary significantly among different medical schools in China.[ The diversity is a potential cause of the great heterogeneity in the theoretical and practical scores between the individual studies. Furthermore, a standard questionnaire needs to be developed to quantitatively evaluate the positive effects of PBL, such as learning interests, clinical thinking, and communication abilities.

Conclusion

The PBL teaching model shows more advantages than traditional teaching method in improving knowledge and skill scores in Chinese radiology education. Most of the included studies used questionnaire surveys to demonstrate that PBL has positive effects on radiological students, including learning interest, scope of knowledge, team spirit, and oral expression. However, the heterogeneity among the included studies needs to be considered. Based on the limitations described above, more standardized experimental studies with a well-designed program on this topic are needed.

Acknowledgments

The authors appreciate the work of editors and anonymous reviewers.