Performance of Chinese medical postgraduate students in literature searching: a two-center survey and analysis.

BACKGROUND: To explore the performance status of Chinese postgraduate medical students in literature searching.
METHODS: A self-designed online questionnaire was used to assess the literature search performance of postgraduate students (PGSs) from the classes of 2016, 2017, 2018, and 2019 from two medical colleges. The items of the questionnaire mainly included: the demographic characteristics of the PGSs, methods of literature review, literature reading habits, and use of literature. We also designed a self-assessed score that ranged from the lowest 1 point to the highest 5 points.
RESULTS: A total of 902 PGSs (482 male, average age: 29.4±5.8 years old, working time range: 0-10 years, average 3.7±2.4 years) completed the questionnaire. Most PGSs investigated literature only at the work tasks (632, 70.1%) and writing papers (571, 63.3%) stages. Of the PGSs, 542 (60.1%) PGSs searched literature frequency (≥1 paper/week), and 114 (12.6%) did not perform advanced searches, and some had no knowledge of advanced search techniques at all. Most PGSs had not read more than 100 Chinese articles or English articles before. Most PGSs were used to read articles from the most authoritative journals (665, 73.7%) or high impact factor (IF) (540, 59.9%). PGSs (845, 93.7%) only read the full text of articles they deemed important. Of the PGSs, 441 (48.9%) did not use literature management tools. For self-assessed score of literature searching and reading skills, the mean was 2.1 (standard deviation, 0.8). Reading literature efficiently (710, 78.7%) and tracking recent literatures (615, 68.2%) were the two needed literature skills reported.
CONCLUSIONS: Chinese medical PGSs still have room for improvement in relation to literature investigation. Intensive training in literature searching should be given to improve their performance. 2020 Annals of Translational Medicine. All rights reserved.

Introduction

Literature searching skills are crucial for researchers and enable related articles or material, including the latest scientific findings, to be found (1,2). In the knowledge era, practitioners should keep up to date with scientific and technological progress, especially in the area of medicine (3,4). However, with the widespread use of the Internet, it is easy to get lost in an ocean of specific information (5); therefore, efficiently obtaining the information we actually need can prove extremely difficult. PubMed and Google scholar may be the most popular professional databases used by investigators worldwide to search for literature. In China, Chinese-language medical literature databases, including China National Knowledge Infrastructure (CNKI), Wanfang database, and Baidu scholar, are also important resources. Some studies reveal gaps in the writer’s literature search performance. In China, considerable emphasis is put on literature reading and searching during the course of study (6). However, in practice, we have found that there are some practitioners who cannot or do not use literature searches professionally, which hampered their career development and had negative influences on the quality of medical research. Therefore, the current study aimed to better understand the actual situation and underlying reasons for this phenomenon by designing a questionnaire for postgraduate students (PGSs) from the classes of 2016, 2017, 2018, and 2019 from two medical colleges.

We present the following article in accordance with the SURGE reporting checklist (available at http://dx.doi.org/10.21037/atm-20-6932).

Methods

Study population

PGSs from the classes of 2016, 2017, 2018, and 2019 were enrolled. These PGSs were educated at two postgraduate medical colleges: 732 (81.2%) came from Medical Postgraduate College of People’s Liberation Army General Hospital (PGH, also known as Medical University of People’s Liberation Army, Beijing, China) and 170 (18.8%) came from Postgraduate College of Medical College of Nankai University (MC of NKU, Tianjin, China). Signed informed consent was obtained from each participant. This survey was about student habits not related to human health and ethical approval was not required. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Questionnaire and score

To gain a deeper insight into the performance of Chinese medical PGSs in literature searching, we designed online version questionnaire (www.wjx.cn). This questionnaire mainly focused on: (I) literature search methods, including search tools, the use of advanced search, and the frequency of searching activities; (II) habits of literature reading, including full-text, intensive, or extensive reading, and the number of articles read per month; and (III) use of literature, including purpose of literature searching and reading, reference management tools, and citation of published articles. We also designed a self-assessed score for researching and reading skills. One point is the lowest, and 5 point is the highest. No incentives were provided.

Statistical analysis

The results data of this cross-sectional survey were recorded in a data sheet in SPSS (version 17.0 for Windows, SPSS, Inc., Chicago, IL, USA). We calculated the response rate. Nonresponse was calculated as missing data. Normally distributed and equally dispersed variables were compared using an unpaired two-sided Student’s t-test. Categorical variables were expressed as numbers and percentages (%). Wilcoxon rank sum test was used to analyze factors grouped by year of class. Self-assessed scores were expressed as the mean ± standard deviation (), and one way ANOVA were used. A P value of <0.05 was considered statistically significant. All statistical analyses were performed with SPSS.

Results

PGSs

Among the 1,166 PGSs enrolled, 902 respondents reported questionnaires with response rate as 77.4%. Two hundred fifty five (28.3%) were studying for a doctor’s degree and 647 (71.7%) were master’s students. Males accounted for a majority of the participants (482, 53.4%). The students had an average age of 29.4±5.8 years, and most (86.6%, 781) were under the age of 35 years. Time of career ranged from 0–10 year, with an average of 3.7±2.4 years. Only 119 (13.2%) PGSs had published English articles before admission. lists the baseline characteristics of the PGSs.

Table 1

Baseline characteristics and literature search performance scores of medical postgraduate students from two colleges in China

Variable	Grouped by year of class					P value
	Overall	2016	2017	2018	2019
N	902	360	123	251	168
College, n (%)
NKU	155 (17.2)	–	54(49.6)	56(22.3)	45 (26.8)	<0.001
PGH	747 (82.8)	360 (100.0)	69 (50.4)	195 (77.7)	123 (73.2)
Degree, n (%)
Master degree	647 (71.7)	221 (61.4)	61 (49.6)	197 (78.5)	168 (100.0)	<0.001
PhD or MD	255 (28.3)	139 (38.6)	62 (50.4)	54 (21.5)	–
Gender, n (%)
Female	420 (46.6)	162 (45.0)	74 (60.2)	97 (38.6)	87 (51.8)	0.001
Male	482 (53.4)	198 (55.0)	49 (39.8)	154 (61.4)	81 (48.2)
Work experience, n (%)
No	357 (39.6)	81 (22.5)	76 (61.8)	108 (43.0)	92 (54.8)	<0.001
Yes	545 (60.4)	279 (77.5)	47 (38.2)	143 (57.0)	76 (45.2)
Age, n (%)
<25	261 (28.9)	55 (15.3)	32 (26.0)	86 (34.3)	88 (52.4)	<0.001
25–29	338 (37.5)	141 (39.2)	48 (39.0)	95 (37.8)	54 (32.1)
30–34	182 (20.2)	106 (29.4)	18 (14.6)	40 (15.9)	18 (10.7)
35–39	97 (10.8)	47 (13.1)	19 (15.4)	23 (9.2)	8 (4.8)
≥40	24 (2.7)	11 (3.1)	6 (4.9)	7 (2.8)	–
No. of papers published, n (%)
None	469 (52.0)	154 (42.8)	49 (39.8)	141 (56.2)	125 (74.4)	<0.001
As-coauthor	2 (0.2)	–	–	2 (0.8)	–
Chinese papers	312 (34.6)	163 (45.3)	38 (30.9)	72 (28.7)	39 (23.2)
English Papers	119 (13.2)	43 (11.9)	36 (29.3)	36 (14.3)	4 (2.4)

PGH, People’s Liberation Army General Hospital; MC of NKU, Medical College of Nankai University ; MD, medical doctor degree; PhD, Doctor of Philosophy Degree.

Literature search methods

PubMed was the main tool used by the PGSs to search for English articles, and CNKI (China National Knowledge Infrastructure) was the main tool used for Chinese-language searches. 542 (60.1%) PGSs searched literature frequency (≥1 paper/week). Of the PGSs, 114 (12.6%) did not carry out an advanced search and some of them even had no knowledge of how to perform one. A majority of the PGSs had not read more than 100 Chinese or English articles before (). Further analysis revealed that some of the participants read the full texts of articles, but some did not because they only read the abstract. Work tasks (632, 70.1%), writing papers (571, 63.3%), writing protocols (41.3, 45.8%), clinical case presentations (350, 38.8%) and doing experiments (312, 34.6%) were main literature prepared scenarios (). In terms of the sources of articles, the PGSs were used to read articles from the most authoritative journals (665, 73.7%) or high impact factor (IF) (540, 59.9%) ().

Table 2

The literature searching habits of Chinese medical postgraduate students

Variable	Grouped by year of class					P value
	Overall	2016	2017	2018	2019
N	902	360	123	251	168
Database or search engine, n (%)
PubMed	505 (56.0)	194 (53.9)	83 (67.5)	149 (59.4)	79 (47.0)	0.001
Medline	7 (0.8)	7 (1.9)	0	0	0
Web of science	4 (0.4)	0	1 (0.8)	2 (0.8)	1 (0.6)
Embase	5 (0.6)	1 (0.3)	3 (2.4)	0	1 (0.6)
Google Scholar	14 (1.6)	2 (0.6)	3 (2.4)	4 (1.6)	5 (3.0)
OVID	1 (0.1)	1 (0.3)	0	0	0
UpToDate	5 (0.6)	3 (0.8)	0	0	2 (1.2)
Yizhi	1 (0.1)	0	0	1 (0.4)	0
CBM	7 (0.8)	5 (1.4)	0	1 (0.4)	1 (0.6)
Wanfang	62 (6.9)	33 (9.2)	6 (4.9)	14 (5.6)	9 (5.4)
CNKI	239 (26.5)	85 (23.6)	24 (19.5)	68 (27.1)	62 (36.9)
Baidu Scholar	41 (4.5)	25 (6.9)	1 (0.8)	10 (4.0)	5 (3.0)
VIP	9 (1.0)	3 (0.8)	2 (1.6)	1 (0.4)	3 (1.8)
Others	2 (0.2)	1 (0.3)	0	1 (0.4)	0
Use of advanced search, n (%)
Never	114 (12.6)	40 (11.1)	9 (7.3)	32 (12.7)	33 (19.6)	0.020
Seldom	247 (27.4)	108 (30.0)	33 (26.8)	58 (23.1)	48 (28.6)
Sometimes	522 (57.9)	205 (56.9)	79 (64.2)	152 (60.6)	86 (51.2)
Frequency	19 (2.1)	7 (1.9)	2 (1.6)	9 (3.6)	1 (0.6)
Chinese papers totally read, n (%)
0	6 (0.7)	1 (0.3)	1 (0.8)	0	4 (2.4)	<0.001
1–9	126 (14.0)	67 (18.6)	3 (2.4)	29 (11.6)	27 (16.1)
10–99	518 (57.4)	182 (50.6)	61 (49.6)	160 (63.7)	115 (68.5)
100–500	206 (22.8)	83 (23.1)	50 (40.7)	54 (21.5)	19 (11.3)
>500	46 (5.1)	27 (7.5)	8 (6.5)	8 (3.2)	3 (1.8)
English papers totally read, n (%)
0	48 (5.3)	24 (6.7)	0	8 (3.2)	16 (9.5)	<0.001
1–9	301 (33.4)	139 (38.6)	13 (10.6)	74 (29.5)	75 (44.6)
10–99	430 (47.7)	150 (41.7)	77 (62.6)	131 (52.2)	72 (42.9)
100–500	110 (12.2)	39 (10.8)	31 (25.2)	36 (14.3)	4 (2.4)
>500	13 (1.4)	8 (2.2)	2 (1.6)	2 (0.8)	1 (0.6)
Searching frequency, n (%)
Never	27 (3.0)	19 (5.3)	0	4 (1.6)	4 (2.4)	<0.001
Seldom (<1 paper/week)	333 (36.9)	156 (43.3)	23 (18.7)	83 (33.1)	71 (42.3)
Frequency (≥1 paper/week)	542 (60.1)	185 (51.4)	100 (81.3)	164 (65.3)	93 (55.4)

CBM, Chinese Biomedical Literature Database; CNKI, China National Knowledge Infrastructure; OVID, Ovid technologies; VIP, Weipu.

Figure 1

When will the literature search be conducted? (A) Work tasks; (B) writing protocols; (C) during experiments; (D) writing papers; (E) clinical case presentations.

Figure 2

According to what to choose a paper to read? (A) Influence of journals; (B) impact factors; (C) affiliation of authors; (D) famous expert authors.

Literature reading habits

In this survey, most PGSs (845, 93.7%) only read the full text of articles they deemed important. Only 42 (4.7%) PGSs were used to read full-text. For the purpose of reading papers, most reason was to improve the research ability (804, 89.1%) followed by performing tasks (123, 13.6%) and preparing papers or fundings (235, 26.1%). With regards to factors hindering searching and reading, limited searching skills and poor English were two top factors (both 641, 71.1%), and these two factors were significantly difference from 2016 to 2019 (P=0.011, P<0.001, respectively) (). The top reading sequence for sections of literature was reading in natural order (Table S1).

Table 3

Literature reading habits of Chinese medical postgraduate students

Variable	Grouped by year of class					P value
	Overall	2016	2017	2018	2019
N	902	360	123	251	168
Full-text reading, n (%)
None	15 (1.7)	6 (1.7)	2 (1.6)	5 (2.0)	2 (1.2)	0.462
Part	845 (93.7)	333 (92.5)	119 (96.7)	232 (92.4)	161 (95.8)
All	42 (4.7)	21 (5.8)	2 (1.6)	14 (5.6)	5 (3.0)
The purpose of reading papers
Improve the research ability, n (%)
No	98 (10.9)	46 (12.8)	10 (8.1)	24 (9.6)	18 (10.7)	0.431
Yes	804 (89.1)	314 (87.2)	113 (91.9)	227 (90.4)	150 (89.3)
To perform tasks, n (%)
No	779 (86.4)	308 (85.6)	107 (87.0)	216 (86.1)	148 (88.1)	0.876
Yes	123 (13.6)	52 (14.4)	16 (13.0)	35 (13.9)	20 (11.9)
Papers or fundings, n (%)
No	667 (73.9)	256 (71.1)	83 (67.5)	186 (74.1)	142 (84.5)	0.003
Yes	235 (26.1)	104 (28.9)	40 (32.5)	65 (25.9)	26 (15.5)
Factors hindering searching and reading
Limited searching skills, n (%)
No	261 (28.9)	105 (29.2)	45 (36.6)	78 (31.1)	33 (19.6)	0.011
Yes	641 (71.1)	255 (70.8)	78 (63.4)	173 (68.9)	135 (80.4)
Poor English, n (%)
No	261 (28.9)	107 (29.7)	45 (36.6)	83 (33.1)	26 (15.5)	<0.001
Yes	641 (71.1)	253 (70.3)	78 (63.4)	168 (66.9)	142 (84.5)
Limited professional knowledge, n (%)
No	609 (67.5)	250 (69.4)	91 (74.0)	177 (70.5)	91 (54.2)	0.001
Yes	293 (32.5)	110 (30.6)	32 (26.0)	74 (29.5)	77 (45.8)
Limited medical research knowledge, n (%)
No	435 (48.2)	188 (52.2)	66 (53.7)	112 (44.6)	69 (41.1)	0.037
Yes	467 (51.8)	172 (47.8)	57 (46.3)	139 (55.4)	99 (58.9)
Other reasons, n (%)
No	898 (99.6)	359 (99.7)	122 (99.2)	249 (99.2)	168 (100.0)	0.560
Yes	4 (0.4)	1 (0.3)	1 (0.8)	2 (0.8)

Use of reference management software and the needed literature skills

A total of 441 (48.9%) PGSs chose to not use literature management tools; some of these PGSs were unaware such tools existed, while others were unsure of how to use them. Of the PGSs who chose to use reference management tools, 379 (42.0%) used Endnote, 49 (5.4%) used Medical Reference King, and 26 (2.9%) used NoteExpress. For self-assessed score for literature searching and reading skills, the mean was 2.1 (SD, 0.8). From 2016 to 2019, the difference of the self-assessed score was significantly (P<0.001) (). The needed literature skills were reading literature efficiently (710, 78.7%) and tracking recent literatures (615, 68.2%) ().

Figure 3

Self-assessed score for literature searching and reading skills (1 is the lowest, and 5 is the highest).

Table 4

Use of reference management software and the needed literature skills in Chinese medical postgraduate students

Variable	Grouped by year of class					P value
	Overall	2016	2017	2018	2019
N	902	360	123	251	168
Which reference management software did medical students often use? n (%)
Endnote	379 (42.0)	118 (32.8)	82 (66.7)	127 (50.6)	52 (31.0)	<0.001
NoteExpress	26 (2.9)	6 (1.7)	2 (1.6)	9 (3.6)	9 (5.4)
Reference manager	4 (0.4)	4 (1.1)	0	0	0
Papers	2 (0.2)	2 (0.6)	0	0	0
Medical reference king	49 (5.4)	29 (8.1)	2 (1.6)	12 (4.8)	6 (3.6)
Zhiyun	1 (0.1)	0	0	0	1 (0.6)
None	441 (48.9)	201 (55.8)	37 (30.1)	103 (41.0)	100 (59.5)
What did medical students want to learn?
Tacking recent literatures, n (%)
No	287 (31.8)	117 (32.5)	46 (37.4)	79 (31.5)	45 (26.8)	0.282
Yes	615 (68.2)	243 (67.5)	77 (62.6)	172 (68.5)	123 (73.2)
Getting the full texts, n (%)
No	497 (55.1)	180 (50.0)	85 (69.1)	141 (56.2)	91 (54.2)	0.003
Yes	405 (44.9)	180 (50.0)	38 (30.9)	110 (43.8)	77 (45.8)
Reading literature efficiently, n (%)
No	192 (21.3)	85 (23.6)	22 (17.9)	59 (23.5)	26 (15.5)	0.105
Yes	710 (78.7)	275 (76.4)	101 (82.1)	192 (76.5)	142 (84.5)
Reference management software, n (%)
No	389 (43.1)	149 (41.4)	53 (43.1)	120 (47.8)	67 (39.9)	0.333
Yes	513 (56.9)	211 (58.6)	70 (56.9)	131 (52.2)	101 (60.1)
Advanced search of PubMed, n (%)
No	363 (40.2)	132 (36.7)	63 (51.2)	111 (44.2)	57 (33.9)	0.006
Yes	539 (59.8)	228 (63.3)	60 (48.8)	140 (55.8)	111 (66.1)

Discussion

To our best knowledge, this study has reported the literature search performance of medical PGSs from two distinguished medical colleges for the first time. We found that not all PGSs had sufficient literature searching skills, which negatively influenced their research. Chinese medical practitioners have the choice of English literature or Chinese literature searches. PubMed is often the resource used to search English-language articles, and count PGH and NKU among its subscribers. PGH and NKU also have subscribed other databases, including Medline, Embase, Web of Science, Lippincott, Williams & Wilkins (LWW) database, and the Cochrane Library (7). Full text articles can be downloaded through links provided by PubMed or directly from the other databases. All of these databases provide a fuzzy search in the homepage search box as well as an advanced search tool. For Chinese-language medical literature, CNKI and Wanfang database are popular resources. These databases also provide fuzzy search and advanced search options and links for full-text downloads. Advanced searching skills are extremely useful (8). In general, PGSs from the two colleges in this study can carry out medical literature searches whenever and for whatever they want. However, the performance of the PGSs in the current study failed to meet our initial expectations. Some doctorate students had more experience in literature searching from their previous research work and had even published English articles previously. Younger PGSs were more familiar with the network and computers technology and had superior English language skills. The PGSs of classes 2018 and 2019 received better literature searching training and had more convenient network facilities, but they self-assessed scores were not better.

As for searching tools, PubMed and other English literature databases could meet PGSs’ basic needs. For Chinese literature, CNKI does not include articles from some high-level journals, such as Chinese Medical Journal (Chinese version), Chinese Journal of Internal Medicine, and Chinese Journal of Cardiology. However, these journals can be found on Wanfang database. On the other hand, unlike CNKI, Wanfang database does not include articles from Chinese journals of general practice. Likewise, some English articles, especially full texts, cannot be found through PubMed alone. Furthermore, some research areas are reported on more or less frequently in English journals compared to Chinese journals. Besides Chinese traditional medicine, recent progress in areas such as integrated elderly care (a Chinese-specific model for taking care of the elderly), reform or administration of public hospitals, or orthopedics, especially surgery of the upper cervical spine, has been reported frequently in Chinese medical journals. Therefore, students should cross-search multiple databases to avoid omitting articles of relevance (9). Unfortunately, the present study revealed that some PGSs had no knowledge of or chose not to use cross-search techniques. Cross-searching is an important skill that enables investigators to dig deeply through literature (10) and should be a focus of future training.

In China, PGSs spend at least 3 years studying for their master’s or doctor’s degrees, and during this time, they should constantly review the latest literature associated with their field (11). However, not all of the PGSs in our study kept up to date with specific scientific progress. Some PGSs only searched literature during the preparation stage of their research work, with some performing a further literature search at the paper-writing stage. Only a few participants carried out literature searches throughout the research process. However, researchers should keep up to date with the progress in their field of study. The latest information can help to inform the direction and design of their research, or support their argument (12); without it, researchers risk missing out on important pieces of data and knowledge. In practice, discovering that similar results have already been published is the worst case scenario for any scholar upon submitting a paper (13).

Further, we analyzed the literature reading habits and utilization of Chinese PGSs. We found that most PGSs only read full texts of selected articles, which is reasonable. In fact, we found that many PGSs in our study read the full text of fewer than 1 articles per week. Considering these findings, we believe that our PGSs did not read enough medical articles in terms of both depth and amount. We also found that many PGSs did not use tools to manage their references. When a large number of references are needed to write a paper, failure to use a reference management tool can reduce efficiency and often leads to sequential errors or the repetition of references. For the PSGs who used tools, Endnote was the main choice. In terms of citations from references, 274 PGSs would copy the original sentence or statement into their own articles while the majority [511] of PGSs would paraphrase.

This survey also has some limitations. First, there were 22.6% non-respondents, these PGSs may be poor compliance. Second, graduate students’ majors may be an important influence factor that was not discussed in this survey.

In summary, this survey showed that the literature search performance of PGSs was not adequate. Generally, the PGSs in our study did not regard literature searching with enough importance, while some did not perform actively in literature searching. Some PGSs were not good at handling references. All of these factors make it difficult for PGSs to excel in their research work (14). In the future, to ensure PGSs have the basic skills to carry out medical research, we should continue to improve our literature investigation training program, make it easier for PGSs to perform literature searching, and set up strict standards for testing their skills and knowledge of literature searching.

The article’s supplementary files as