Assessing the effectiveness of problem-based learning in physical diagnostics education in China: a meta-analysis.

Problem-based learning (PBL) has been extensively applied as an experimental educational method in Chinese medical schools over the past decade. A meta-analysis was performed to assess the effectiveness of PBL on students' learning outcomes in physical diagnostics education. Related databases were searched for eligible studies evaluating the effects of PBL compared to traditional teaching on students' knowledge and/or skill scores of physical diagnostics. Standardized mean difference (SMD) with 95% confidence interval (CI) was estimated. Thirteen studies with a total of 2086 medical students were included in this meta-analysis. All of these studies provided usable data on knowledge scores, and the pooled analysis showed a significant difference in favor of PBL compared to the traditional teaching (SMD = 0.76, 95%CI = 0.33-1.19). Ten studies provided usable data on skill scores, and a significant difference in favor of PBL was also observed (SMD = 1.46, 95%CI = 0.89-2.02). Statistically similar results were obtained in the sensitivity analysis, and there was no significant evidence of publication bias. These results suggested that PBL in physical diagnostics education in China appeared to be more effective than traditional teaching method in improving knowledge and skills.

Problem-based learning (PBL) is an educational approach usually characterized by the use of real problems as a context for students to acquire knowledge and develop skills1. This pedagogy was developed from educational innovation to resolve dissatisfaction with the conventional medical education practices in the late 1960s2. It has since been implemented in many medical school curricula and other professional training programs around the world.

Unlike traditional teaching model, PBL approach is student-centered. Students learn about a subject through the experience of problem solving in a tutor-led small group. Many studies have suggested that students in a PBL curriculum become better at problem solving and self-directed learning than those in a traditional curriculum345. However, there is also evidence indicating that PBL teaching model is not superior to the traditional approach with respect to the acquisition of factual knowledge167, and the effects of PBL may be different in different subjects.

As a bridge course in medical education, physical diagnostics is very important for medical students to acquire basic clinical knowledge and develop clinical skills. The latter are usually defined as clinical reasoning and clinical problem solving that are mostly related to taking an adequate medical history, conducting an appropriate physical examination and establishing a proper diagnosis8. Although the physical diagnostics course structure has been changed to the PBL approach in many medical schools9, it is still not clear whether PBL in physical diagnostics education is superior to traditional pedagogies such as lecture-based learning and system-based approach.

In China, PBL application in medical education is not a routine pedagogy for many reasons10. The development of PBL is behind that in other countries. In recent years, due to current shifts in approaches to medical education prevalent in China, this teaching model has been extensively applied as an experimental educational method in many medical curricula including physical diagnostics1112. However, the results of PBL studies about physical diagnostics education in China have been inconclusive or inconsistent. Considering the relatively small sample size of most studies, it is possible to perform a quantitative synthesis of the evidence with rigorous methods. The aim of current meta-analysis was to evaluate the overall effectiveness of PBL compared to traditional teaching in Chinese physical diagnostics education.

Results

Search results

The flowchart for identification of studies is shown in Fig. 1. At the stage of identification, we searched the related databases using the search strategy described in the Methods section. In total, 318 potentially relevant articles were identified. At the screening stage, we excluded 250 articles with different topics, and also eliminated 53 reviews and commentaries. Fifteen studies were potentially appropriate and assessed for eligibility according to the criteria described in the Methods section, of which two studies were excluded because they did not provide sufficient data1213. Thus, a total of 13 controlled studies, involving 2086 medical students, met the inclusion criteria and were selected for this meta-analysis14151617181920212223242526.

Figure 1

Flowchart for identification of studies.

A: The number of studies that provided usable data on knowledge scores (A1) and skill scores (A2).

Study characteristics

The characteristics of these 13 included studies are shown in Table 1. All of them were published in Chinese between 2007 and 2014, and evaluated the effects of PBL compared to traditional teaching in physical diagnostics courses (also named clinical diagnostics in China). The sample sizes ranged from 20 to 230 students in PBL group and 25 to 229 in control group. The major of students in almost all studies was clinical medicine. The school system was different, however nearly half of the studies were carried out in the five-year system medical students. Nine studies described that the students in each group were matched for the basic medical knowledge test scores or college entrance exam scores. The duration of study was one semester, which was mentioned in 9 studies. Both knowledge scores and skill scores of physical diagnostics were used as outcome measures in all studies. The scores in the knowledge exam of physical diagnostics were used to assess how well the students mastered the related theoretical knowledge, and the scores in the practical skill test of physical diagnostics were used to evaluate the students’ clinical skills including history taking, physical examination and diagnostic reasoning.

Table 1

Characteristics of published studies included in this meta-analysis.

Study	Course name	Number of students (E/C)	Major of students	School system	Students matched for	Educational approach (E/C)	Duration of study	Outcome measures
An 2012	Clinical diagnostics	30/30	Clinical medicine	Five-year system	Age, Sex, ES	PBL/Traditional teaching	One semester	KS, SS
Chen 2014	Physical diagnostics	60/60	Medicine	Not described	Age, Sex, BS	PBL/Traditional teaching	Unclear	KS, SS
Guan 2007	Clinical diagnostics	48/46	Clinical medicine	Five-year system	Not described	PBL/Traditional teaching	One semester	KS, SS
Hou 2012	Physical diagnostics	60/60	Clinical medicine	Five-year system	BS	PBL/Traditional teaching	Unclear	KS, SS
Lai 2012	Physical diagnostics	100/100	Clinical medicine	Five-year system	Not described	PBL/Traditional teaching	One semester	KS, SS
Li 2007	Physical diagnostics	130/122	Clinical medicine	Five-year system	BS	PBL/Traditional teaching	One semester	KS, SS
Liu 2007	Physical diagnostics	93/95	Clinical medicine	Three-year system	Age, Sex, ES	PBL/Traditional teaching	One semester	KS, SS
Liu 2014	Physical diagnostics	20/25	Clinical medicine	Mixed	Not described	PBL/Traditional teaching	Unclear	KS, SS
Mo 2011	Physical diagnostics	54/52	Clinical medicine	Five-year system	Not described	PBL/Traditional teaching	One semester	KS, SS
Nie 2012	Physical diagnostics	230/229	Clinical medicine	Three-year system	Age, Sex, ES	PBL/Traditional teaching	One semester	KS, SS
Wu 2012	Clinical diagnostics	60/60	Clinical medicine	Mixed	BS	PBL/Traditional teaching	One semester	KS, SS
Xu 2013	Physical diagnostics	40/40	Clinical medicine	Not described	Age, Sex, ES	PBL/Traditional teaching	Unclear	KS, SS
Zuo 2011	Clinical diagnostics	32/210	Clinical medicine	Eight-year system	Sex, BS	PBL/Traditional teaching	One semester	KS, SS

E/C, experimental group/control group; ES, college entrance exam scores; PBL, problem-based learning; KS, knowledge scores; SS, skill scores; BS, basic medical knowledge test scores.

Study quality

The summary of the methodological quality of each study is shown in the left panel of Fig. 2, and the graph of each quality item presented as percentages across all studies is shown in the right panel. Nine studies were designed as randomized controlled trials, and the other 4 studies were unclear. All studies reported complete outcome data and were free of selective reporting and other bias. The allocation concealment and blinding were not stated in these studies.

Figure 2

Summary of each methodological quality item for each included study and graph of each quality item presented as percentages across all included studies.

A: Randomized? B: Allocation concealment? C: Blinding? D: Incomplete outcome data addressed? E: Free of selective reporting? F: Free of other bias?

Effects of PBL on knowledge scores

The effects of PBL compared to the traditional teaching on knowledge scores were reported in all 13 included studies involving 2086 medical students (PBL group = 957, control group = 1129). Seven studies suggested that there was a statistically significant difference between the PBL and the control group in students’ knowledge scores. Because a high degree of heterogeneity was observed across all of the 13 studies (I2 = 95%, P < 0.001), the random-effects model was applied for the meta-analysis. The analytical results showed a significant difference in knowledge scores (SMD = 0.76, 95%CI [0.33, 1.19], P = 0.0006) in favor of PBL, compared with the traditional teaching (Fig. 3). The summary effects also favored the PBL group when using the fixed-effects model (SMD = 0.81, 95%CI [0.72, 0.91], P < 0.001). In addition, we performed a sensitivity analysis by excluding individual studies sequentially. Statistically similar results were obtained (SMD = 0.58~0.84, P < 0.01), suggesting the stability of this meta-analysis. In the stratified analysis by school system, we pooled the data from 6 studies in which target population were all five-year system students using the random-effects model (I2 = 97%, P < 0.001). A statistically significant difference was also observed in knowledge scores (SMD = 0.90, 95%CI [0.04, 1.76], P = 0.04) in favor of PBL, compared with the traditional teaching.

Figure 3

Forest plot for the effects of PBL on knowledge scores compared with the traditional teaching.

Effects of PBL on skill scores

The effects of PBL compared to the traditional teaching on skill scores were reported in all included studies. Ten studies involving 1599 medical students (PBL group = 805, control group = 794) provided usable data for this analysis. Because a high degree of heterogeneity was observed across all of the 10 studies (I2 = 96%, P < 0.001), the random-effects model was applied for the meta-analysis. The analytical results showed a significant difference in skill scores (SMD = 1.46, 95%CI [0.89, 2.02], P < 0.001) in favor of PBL, compared with the traditional teaching (Fig. 4). The summary effects also favored the PBL group when using the fixed-effects model (SMD = 1.40, 95%CI [1.29, 1.51], P < 0.001). In addition, we performed a sensitivity analysis by excluding individual studies sequentially. Statistically similar results were obtained (SMD = 1.28~1.60, P < 0.01), suggesting the stability of this meta-analysis. In the stratified analysis by school system, we pooled the data from 5 studies in which target population were all five-year system students using the random-effects model (I2 = 92%, P < 0.001). A statistically significant difference was also observed in skill scores (SMD = 1.08, 95%CI [0.47, 1.69], P = 0.0006) in favor of PBL, compared with the traditional teaching.

Figure 4

Forest plot for the effects of PBL on skill scores compared with the traditional teaching.

Publication bias

Funnel plots for knowledge scores (Fig. 5) showed no clear evidence of publication bias, and the test using Egger’s method did not suggest publication bias, either (t = −0.47, P = 0.649). Funnel plots for skill scores (Fig. 6) also showed no clear evidence of publication bias, and the test using Egger’s method did not suggest publication bias, either (t = 0.14, P = 0.896).

Figure 5

Funnel plots for the assessment of potential publication bias in knowledge scores.

Figure 6

Funnel plots for the assessment of potential publication bias in skill scores.

Discussion

Although PBL has been widely adopted in medical education all over the world, the application of this teaching method in China is still at the elementary stage. Given that different education systems or cultural backgrounds could influence the effectiveness of PBL method27, the target population should be limited to Chinese medical students in the Chinese education system in order to evaluate the potential effectiveness of the PBL approach in China. This meta-analysis of studies in the field of Chinese physical diagnostics education assessed whether there were differences in quantitative outcomes between PBL pedagogy and the traditional teaching method.

The results of current meta-analysis show that the students in the PBL groups have better knowledge scores than those in the traditional teaching method groups, which are consistent with the findings of recent PBL studies in Chinese pediatric, pharmaceutical and dental education102829. However, in western countries, some education researchers have questioned the performance of PBL students in the knowledge exams and have argued that PBL has a negative effect on the acquisition of factual knowledge167. Given the differences in higher medical education between China and the western countries, there are several possible reasons for the inconsistent results. Firstly, PBL pedagogy is totally different from the traditional teaching method in which students achieve knowledge passively from their teachers. Chinese students have received the traditional teaching model since primary education. PBL teaching model is a novelty for them and has greatly stimulated their interest in learning30. Secondly, most Chinese medical schools use unified textbooks and syllabuses for all students, which is very different from the western countries. In the present study, both PBL and traditional students used the same textbook and syllabus of physical diagnostics. PBL students appeared to be more proactive in learning, which led them to get higher knowledge scores. Thirdly, teachers and other faculty personnel in a PBL program usually engage more with students31. Chinese teachers always emphasize the importance of course examinations and the keen pursuit of high scores promotes the students to excel in the exams30.

Despite the explosion of technological advances, the basic clinical skills of history taking, physical examination and counseling remain the most important and effective diagnostic tools8. Therefore, the development of above skills in the physical diagnostics education is considered crucial for medical students. According to the results of our study, there is a robust positive effect from PBL on the skill scores of students, and no single study reports negative effects. These results are consistent with the previous findings that PBL can effectively enhance clinical skills132333435. The use of problems as a vehicle for developing problem solving skills is one of the key characteristics of PBL, while the important aspects of physical diagnostics education involve the development of students’ clinical diagnostic reasoning skills and ability to solve clinical problems, which represent the advantages of PBL. Hence, the superiority of PBL relative to the traditional teaching method could be more evident when considering the clinical skills.

Obviously, there is a high heterogeneity among the included studies. In the stratified analysis by school system, significant heterogeneity also exists, which might have resulted from the following factors. The definition and implementation of PBL could vary widely among medical schools and educators36. The educational quality of different medical schools in China could be very uneven, and the medical students from different schools could have different learning background. There are still no unified criteria for the evaluation of the effectiveness of PBL on knowledge and skills. In addition, many other factors that are difficult to measure and control could also affect the success of PBL37. Therefore, the random-effects model was applied in this meta-analysis. The high levels of heterogeneity in the effect sizes indicate that more than one population is generating the effect sizes and that the present mean effect sizes are not broadly generalizable. Further research that focuses on moderator variables is required to reduce and resolve the heterogeneity.

It is generally accepted that randomized controlled trials provide the highest level of evidence for the effectiveness of an intervention. However, it is not always possible to use a very rigorous randomized controlled trial approach for educational research. Although most of the studies included in this meta-analysis were designed as randomized controlled trials, none of them described the allocation concealment or blinding method. Overall, the methodological quality of the included studies is not high. In the analysis of knowledge scores (Fig. 3), two of the effect sizes are very high and the mean effect size is large. Similarly, five of the effect sizes are high in the analysis of skill scores (Fig. 4), and the mean effect size is also quite large. Although statistically significant results were also obtained after removing those corresponding studies (data not shown), moderating variable analyses were not performed due to limited studies. Thus, the results of current meta-analysis need to be interpreted with caution. In addition, some of the included studies also used questionnaire surveys as an additional measurement to assess the effectiveness of PBL in physical diagnostics education. However surveys are limited in reliability and validity, subjective and prone to rater biases, and we failed to perform the meta-analysis of those outcomes due to inadequate data.

In summary, the present meta-analysis shows that PBL in physical diagnostics education in China appears to be more effective than traditional teaching method in improving knowledge and skills. Based on the limitations of this meta-analysis, we believe that further well-designed studies on this topic are needed.

Methods

Data sources

We searched China National Knowledge Infrastructure (CNKI), China Science Periodical Database (CSPD), Chinese BioMedical Literature Database (CBM) and English computerized databases including PubMed, EMBASE and Cochrane Database using the following terms: (PBL OR (problem-based learning)) AND (diagnostics OR diagnosis). There were no language restrictions. Searches were current as of April 2016. References of all primary studies and review articles were reviewed for additional references.

Study selection

Included studies met the following criteria: (1) target population: medical students in Chinese medical schools; (2) study design: controlled trials in physical diagnostics education; (3) interventions: PBL teaching in the experimental group and traditional teaching in the control group; (4) outcome measurements: students’ knowledge scores and/or skill scores of physical diagnostics. In addition, we excluded studies with insufficient data for calculating effect sizes.

Data extraction and quality assessment

Two reviewers independently identified trials and reviewed the titles, abstracts and citations (Jianmiao Wang and Jianping Zhao). Two reviewers independently assessed studies for inclusion based on the criteria for population, intervention, study design and outcome measurements (Jungang Xie and Xiansheng Liu). Data were also independently extracted by two reviewers (Yongjian Xu and Weining Xiong). From each study, the following information was extracted: first author’s surname, publication year, course name, sample size, student characteristics, intervention method, duration of study and outcomes. We also contacted study authors for missing data. The methodological quality of each study was evaluated with the risk of bias table according to the Cochrane Collaboration.

Statistical analysis

Standardized mean difference (SMD) for continuous outcomes, with 95% confidence interval (CI), was calculated for each study. Studies were then pooled together using SMD as appropriate. The Z-test was used for overall effect and two-sided P < 0.05 was considered as statistically significant. The Q-statistic was calculated to examine result heterogeneity among studies, and P < 0.10 was considered significant. The fixed-effects model was used when the effects were assumed to be homogenous; otherwise, the random-effects model was applied. The I2 statistic was also calculated to efficiently test for the heterogeneity, with I2 < 25%, 25–75% and >75% to represent low, moderate and high degree of inconsistency, respectively38. Publication bias was examined in funnel plots and tested with Egger’s weighted regression method39. The meta-analysis was performed using Review Manager 5.0.23 (Cochrane Library Software, Oxford, UK) and STATA 12.0 (STATA Corporation, College Station, Texas, USA).

Additional Information

How to cite this article: Wang, J. et al. Assessing the effectiveness of problem-based learning in physical diagnostics education in China: a meta-analysis. Sci. Rep. 6, 36279; doi: 10.1038/srep36279 (2016).

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.