What factors affect the quality of medical students' doctoral theses? A comparative study in affiliated hospitals of a Chinese university.

Background: The quality of doctoral theses decides whether medical PhD and MD candidates could get their doctoral degree successfully. Good quality theses could be rewarded by Peking University which is a great honor for both doctoral candidates and for mentors. The present study aims to determine factors affecting the quality of medical doctoral theses.
Methods: Honored theses and nonhonored theses were matched 1:3 randomly by specialty and submission year. Conditional logistic regressions were utilized.
Results: Five domains comprising 17 indicators were put forward to evaluate the quality of doctoral theses. 41 honored theses and 119 matched nonhonored theses from years 2012-2016 were analyzed by univariate and multivariate conditional logistic regression. Degree type (OR: 107.56, 95%CI: 1.20-9632.70, P=0.041), first author impact factor (OR:1.24, 95%CI: 1.01-1.53, P=0.040) and correctly reported statistic results (OR: 43.18, 95%CI: 1.88-991.61, P=0.019) are independent factors influencing the quality of a doctoral thesis. Conclusions: The present study indicates that there is a significant gap between PhD and MD students on quality of thesis. The rewarded theses have a feature of high first author impact factor. However, most medical students need more training on statistics to improve the quality of their doctoral theses.

Background

The quality of medical doctoral thesis is highly valued among medical educational institutions around the world. It determines whether medical students are cultivated to be qualified high-level medicine talents. A great many efforts have been put into improving quality of medical theses.1–3 Since the reform and open-up in 1978, rapid development of society in China has gained growing demands for high-level talent in medicine.4 Medical doctoral degrees in China have been divided into two types since 1997: Doctor of Medicine (MD) and Doctor of Philosophy (PhD). For PhD students, the fundamental training goal is to gain the ability to conduct scientific research independently. In practice, quality of thesis and impact factor (IF) of published articles on impacted scientific journals are used for evaluating the accomplishment of the training goal of PhD candidates. While for MD students, the primary training goal is to achieve excellent professional clinical skills. Further, in order to be able to promote the development of the specialty in their future career, MD candidates should also gain the basic ability to perform clinical research, pose scientific questions and utilize appropriate methods. Quality of thesis has also been widely used to evaluate the scientific training of MD candidates. Moreover, writing a doctoral thesis in good quality and passing a blind review by nationalwide experts has become one of the requisites for obtaining doctoral degree not only for PhD candidates, but also for MD candidates. Many medical schools award excellent doctoral theses to encourage the improvement of thesis quality. Questions regarding factors influencing the quality of medical students’ theses have been frequently asked by students, mentors and administrators. However, no answer based on data analysis has been provided. In this study, we aim to explore factors affecting the quality of medical doctoral thesis by a matched comparative study design.

Methods

Materials

Medical doctoral theses from 2012–2016 were available online at Peking University Health Science Library. A total of 50 doctoral theses from affiliated hospitals of Peking University were honored by the university as “excellent theses” (honored theses). Forty-one honored theses with open full texts were downloaded from the online system of the university library. Specialty- and year of submission-matched nonhonored theses with open full texts were randomly chosen in the online library system in a 3:1 nonhonored:honored ratio. For two honored theses, only one matching thesis was downloaded from online library system. Information including name, department, specialty and title of thesis were hidden from two statisticians who performed data collection and analyses. Potential factors affecting thesis quality were determined through an intensive brainstorm.

Analysis

To determine whether the honored theses group has distinct characteristics from nonhonored theses group, indicators for evaluating theses quality were put forward through brainstorming and went for first evaluation by univariate conditional logistic regression. Indicators with P<0.05 and OR >2 were then subjected to a multivariate conditional logistic regression model to select indicators at a significance level of 0.05. STATA 13.1 (STATA Corporation, College Station, TX, USA) was utilized for all statistical analysis. All tests were two-sided and had a significance level of 0.05.

Results

A total of 160 doctoral theses including 41 honored theses and 119 matched nonhonored theses were subjected as study materials. Five domains comprising 17 indicators were put forward by brainstorming.

Results of univariate conditional logistic analyses were summarized in Table 1. There was no statistical difference between the two groups in indicators including number of instructors, word count of review, percentage of last three years references, introduction of sample source, provide sample size estimation and provide ethic approval by the univariate logistic analysis (P>0.05). Among these indicators, both honored and nonhonored groups had similar word count of review (10,600 vs 8,700 words). Nearly half of the candidates put efforts in reading the latest references and providing ethic approval in their theses. Most candidates (honored vs nonhonored group: 97.56% vs 94.96%) have introduced sources of samples, making it clear whether study subjects come from hospitals or communities, what genetic backgrounds of laboratory animals are and what the origins of cell lines are. Less than a third of the candidates utilized appropriate statistical methods for their study, and fewer than 5% of candidates provided sample size estimation in their doctoral theses. The honored group demonstrated statistical significance in writing more words on abstract (P=0.001) and main text (P<0.001), and in citing more references (P<0.001) than the nonhonored group. From the univariate analysis, seven indicators with P<0.05 and OR: >2 were selected for the multivariate logistic regression analysis: degree type, research classification, first author impact factor, scholarship, standard format, provide clear diagnostic criteria and correctly report statistic results (Table 2). Three indicators showed statistical significance in the final multivariate conditional logistic regression model: degree type (OR: 107.56, 95%CI: 1.20–9632.70, P=0.041), first author IF (OR: 1.24, 95%CI: 1.01–1.53, P=0.040) and correct statistic report (OR: 43.18, 95%CI: 1.88–991.61, P=0.019).

Table 1

Comparison of thesis quality indicators between honored and nonhonored group (univariate analysis)

Domain	Indicators	Honored	Nonhonored	OR (95%CI)	P-value
Training characteristics	Number of instructors N (%)
	=1	23 (56.10%)	76 (63.87%)
	>1	18 (43.90%)	43 (36.13%)	1.55 (0.70–3.44)	0.276
	Degree type
	MD	4 (9.76%)	92 (77.31%)
	PhD	37 (90.24%)	27 (22.69%)	62.89 (8.53–463.5)	<0.001
	Research classification (n(%))
	Clinical epidemiology	4 (9.76%)	67 (56.30%)	Ref	Ref
Clinical and basic research	14 (34.15%)	15 (12.61%)	13.18 (3.67–47.3)	<0.001
	Basic research	23 (56.10%)	37 (31.09%)	7.88 (2.60–23.88)	<0.001
Workload	Abstract word count (per 100)
	Median (quartile)	14.61 (9.86, 20.18)	9.70 (7.21, 14.37)	1.10 (1.04–1.16)	0.001
	Review word count (per 1000)
	Median (quartile)	10.60 (7.04, 13.91)	8.70 (6.67, 12.98)	1.03 (0.99–1.08)	0.182
	Main text word count (per 1000)
	Median (quartile)	23.74 (18.51, 31.54)	16.08 (11.70, 22.37)	1.09 (1.04–1.13)	<0.001
	Total number of references
	Median (quartile)	144 (108, 196)	104 (76, 139)	1.01 (1.01–1.02)	<0.001
	Last three years references N (%)
<21%	24 (58.54%)	55 (46.22%)
	≥21%	17 (41.46%)	64 (53.78%)	0.60 (0.29–1.25)	0.175
Student accomplishment	First author IF
	Median (quartile)	8.31 (5.41, 10.97)	0 (0, 1.54)	2.36 (1.45–3.84)	0.001
	Scholarship N (%)
No	9 (21.95%)	55 (46.22%)
	Yes	32 (78.05%)	64 (53.78%)	3.18 (1.32–7.67)	0.010
Thesis standardization	Standard format N (%)
	No	3 (7.32%)	28 (23.53%)
	Yes	38 (92.68%)	91 (76.47%)	3.84 (1.09–13.55)	0.036
	Provide clear diagnostic criteria N (%)
	No	10 (24.39%)	50 (42.02%)
	Yes	31 (75.61%)	69 (57.98%)	2.53 (1.07, 6.00)	0.035
	Introduce sample source
	No	1 (2.44%)	6 (5.04%)
	Yes	40 (97.56%)	113 (94.96%)	2.47 (0.29, 21.20)	0.408
	Provide ethic approval
No	19 (46.34%)	68 (57.14%)
	Yes	22 (53.66%)	51 (42.86%)	1.60 (0.73, 3.48)	0.237
Statistic method	Provide sample size estimation
	No	40 (97.56%)	115 (96.64%)
	Yes	1 (2.44%)	4 (3.36%)	0.72 (0.07, 7.35)	0.782
	Appropriate statistic method
	No	29 (70.73%)	94 (78.99%)
	Yes	12 (29.27%)	25 (21.01%)	1.78 (0.70–4.56)	0.230
	Correct statistic report
	No	13 (31.71%)	95 (79.83%)
Yes	28 (68.29%)	24 (20.17%)	9.14 (3.69–22.61)	<0.001

Table 2

Multivariate conditional logistic analysis of indicators affecting quality of medical doctoral theses

Indicators	Coefficient	Standard error	P-value	OR (95%CI)
Degree type	4.68	2.29	0.041	107.56 (1.20–9632.70)
Research classification
Clinical epidemiology	Ref	Ref	Ref	Ref
Clinical and basic research	−0.18	1.56	0.906	0.83 (0.04–17.68)
Basic research	−0.27	2.02	0.892	0.76 (0.01–39.70)
First author impact factor	0.22	0.11	0.040	1.24 (1.01–1.53)
Scholarship	−0.35	1.34	0.795	0.71 (0.05–9.83)
Standard format	2.77	1.86	0.136	15.94 (0.42–605.50)
Provide clear diagnostic criteria	0.51	1.26	0.685	1.67 (0.14–19.88)
Correct statistic report	3.77	1.60	0.019	43.18 (1.88–991.61)

Discussion

The present descriptive cross-sectional study was conducted to determine factors influencing thesis quality of medical doctoral candidates in Peking University. Independent factors included degree type, first author IF, and correct statistic report.

Like most universities in China, an open and stepwise recommendation system is adopted by Peking University in the process of selecting excellent medical doctoral theses. Both PhD and MD candidates could be recommended by their mentors, reviewers and schools. However, there are also opinions that it is unfair to compare the two degree types. PhD candidates have spent most of their time in scientific training and have more time to polish their scientific writing, whereas MD candidates have spent most of their time in clinical training and have limited time for research and writing theses. Results of this study suggested that PhD candidates have significant advantage in winning the honor of an excellent doctoral thesis than MD candidates. A previous survey5 has demonstrated widespread incompetency in conducting research among Chinese doctors. This situation has aroused demands for clinical doctors to conduct scientific research. However, various factors including shortage of time, knowledge and academic resources contribute to enormous pressure for doctors to do research. Currently, such academic pressure has shifted to MD candidates, making MD candidates facing similar research obstructions with clinical doctors. Comments have been pointed out that it is necessary to avoid the confusion between PhD and MD degrees on training objective and pattern.6 Using same criterion to evaluate scientific trainings as well as thesis quality of MD and PhD students may not be appropriate. Theses evaluations for the two degree types should be considered differently. More investigations on training differences between the two degree types should be conducted in the future.

The median first author IF in the nonhonored group was zero which indicates that most candidates have not made their scientific work visible to academic community upon graduation. A study from Finland found that fewer than 25% of medical students published their theses in impacted scientific journals.7 Data from a specific field of research shows that 86% of MD theses failed to result in publication.8 Most opinions are that the intrinsic value of a doctoral thesis is closely related to the impact factor of published academic articles. When multiple recommended theses are presented, it could be simple and easy to select one or two as excellent theses according to the author’s accumulated impact factors by graduation. However, mentors and students have raised concerns that using first author IF upon graduation may underestimate a good quality thesis, which will result in publication after graduation. Furthermore, using single selecting indicators may cause deviation from the original intention for excellent thesis evaluation. Quality-oriented and comprehensive evaluation system should be studied and put into application.

It seems that one common deficiency in methodological training for medical students is statistics. Earlier studies on German medical doctor candidates showed that more than half of the students need more assistance with statistics.9 A measure of attitudes found that medical students were not very active in their study of statistics.10 The present study shows that the majority of medical doctoral candidates have difficulties in choosing suitable statistical methods and providing sample size estimation. Using correct statistical methods is crucial to give credible answers to study hypotheses. Furthermore, correctly reporting statistical results could significantly improve thesis quality. Medical students need more help from medical statisticians to improve their understanding and application of medical statistics.

The present study demonstrated important factors influencing quality of medical doctoral theses and provided evidence-based suggestions for medical education administrators to consider. There are some limitations of the present study. First, Peking University has been one of the leading universities in China and may represent the top drawer of medical education. However, since China has seen multiple medical education reform,11 cultivation for medical doctoral talent can vary from place to place, which might make the result of this study not applicable to other higher medical education institutions. Second, since the current analyses were based on objective indicators, assessment of subjective attitudes and difficulties in writing and supervising doctoral theses were not reflected in the present study. Last but not least, in the absence of standard evaluation criteria, honored theses are often chosen based on expert judgment. To some extent, factors affecting thesis quality might reflect expert consensus rather than objective standards. Objective evaluation system for medical doctoral theses still needs further research and construction in the future.

Conclusions

Within the limitation of this matched comparative study, it might be concluded that:

(1) PhD candidates might have a distinct advantage in winning the honor of an excellent thesis than MD candidates.

(2) The rewarded theses have a feature of high first author impact factor.

(3) Assistance in statistics could be helpful for medical doctoral students to improve the quality of their doctoral theses.