Scientific research competitiveness in hospitals: A narrative review of major hospital ranking systems in China.

Background and Aims: Several hospital ranking systems have been created in China recently, but there is still a lack of comprehensive analysis of the weight and significance of scientific research in hospital ranking. The present study aimed to identify and analyze the role of scientific research competitiveness in various hospital ranking systems in China.
Methods: Over 200 materials published between 2010 and 2020 and related to three mainstream hospital ranking systems in China were reviewed. The methodologies applied in the three ranking systems were analyzed and compared. In addition, the comparative learning and analysis of Top 10 and Top 46-55 hospitals according to the ranking system of China's Best Hospital Rankings was performed for a longitudinal study.
Results: The three major hospital rankings had different scientific research capability ranking methodologies and emphases of scientific research evaluation systems. The most commonly used indicators were science citation index (SCI) publications, National Scientific Foundation of China funding, a number of national key laboratories, and a number of academicians. The relative standing of several top hospitals showed slightly different in the three major Chinese hospital ranking systems. For the longitudinal study, we found that the fluctuation of the ranking of the Top 46-55 hospitals was significantly higher than that of the Top 10 hospitals, in which scientific research played a vital role.
Conclusion: The proportion of scientific research plays an important role in the hospital ranking systems. The quality and quantity of SCI publications, the key indicators of national projects, and top academic talents are the most important factors used to evaluate the level of hospital scientific research, and thus affect the ranking of hospitals.

INTRODUCTION

In China, most tertiary hospitals are located in big cities, especially in Beijing, Shanghai, and Guangzhou. Due to the lack of well‐accepted criteria patients or research institutions used to have no method to compare and judge the quality of these hospitals. Therefore, there was a need to develop a system to evaluate the hospitals. Over the past decades, hospital ranking evaluation has played an important or even irreplaceable role in promoting the service standardization and continuous improvement of the medical quality of hospitals at different levels. Hospital ranking, which was originated in the United States in 1989, has received more and more attention in China, as it provides a very important reference for patients to choose hospitals for medical treatment. However, there are still many areas that need to be improved to accurately and objectively reflect the level of hospitals.

Currently, several hospital ranking systems have been developed in China. Unlike American ranking systems, which are completely disclosed in detail in publications, , , China's ranking systems were only published online. Thus, not all data in China' hospital ranking systems are publicly available; however, most data can still be accessible online. For example, the Hospital Management Institute of Fudan University was the first to publish “China's Best Hospital Rankings” in 2010, with the aim of identifying the best hospitals for various specialties. This rank has become the longest‐running annual report with regards to hospital quality in China. Several other publicly recognized hospital ranking systems have also been developed, such as “Chinese Hospitals' Competitiveness Rankings” (Hong Kong Institute of Ailibi Hospital Management) and “Chinese Hospitals' Scientific Influence Rankings” (Yeecin Institute of Medical Information, Chinese Academy of Medical Sciences). These systems enable consumers to compare among hospitals, and identify institutions with high quality in each specialty.

It is generally accepted that scientific research activities and achievements at hospitals, along with the clinical practice, are important indicators to represent the advancements of hospitals. Although the above hospital ranking systems cover quite a few aspects, different ranking systems have various methods in accessing research competitiveness. Moreover, the specific role of scientific research competitiveness in determining the rank of a particular hospital has not been examined. Therefore, the present study aimed to identify and analyze the role of scientific research competitiveness in various hospital ranking systems in China.

METHODS

Sample and data

Materials that were related to the Hospital Rankings in China, China's Best Hospital Ranking, Chinese Hospitals' Competitiveness Rankings, and Chinese Hospitals' Scientific Influence Rankings and published between 2010 and 2020 were obtained. More than 200 relevant articles from journals and literature were subsequently retrospectively analyzed. , , , ,

Keywords including “Hospital Rankings in China,” “China's Best Hospital Rankings,” “Chinese Hospitals' Competitiveness Rankings,” and “Chinese Hospitals' Scientific Influence Rankings” were searched using the China National Knowledge Infrastructure (CNKI), PubMed, the Wan Fang resource databases, and the wiper database journal database. Additionally, our materials included some comments and content found through Sohu and Google searches.

Through a synthesis of all these searched materials, we compared and analyzed the evaluation models of three hospital ranking systems (i.e., China's Best Hospital Ranking, Chinese Hospitals' Competitiveness Rankings, and Chinese Hospitals' Scientific Influence Rankings), and the proportion of scientific research in the three hospital ranking systems. Furthermore, we chose the Top 8 ranked hospitals in the three hospital ranking systems in 2016 for horizontal and longitudinal comparisons. First, the scientific research contribution to these well‐recognized hospitals and their respective scientific research and technology levels were compared. Then, the annual ranking changes of Top 10 and Top 46–55 hospitals from 2010 to 2020 under China's Best Hospital Ranking were obtained, and their scientific research and technology levels were compared to analyze the importance of scientific research and technology in hospital ranking.

Search strategy and selection criteria

Database: PubMed; freetext search terms: “Lancet” and “China” or “Chinese” and “health” or “hospital” or “public”;

Database: PubMed; freetext search terms: “China” and “disparity” or “hospital” or “reform”;

Search engine: Google; freetext search terms: “News & World Report” and “Best Hospitals”;

Search engine: Google; freetext search terms: “Truven Health” and “top hospitals”;

Search engine: Google; freetext search terms: “Healthgrades”;

Database: CNKI; freetext search terms: “hospital ranking” or “competitiveness”

China's Best Hospital Rankings: http://www.fudanmed.com/institute/news222.aspx

Chinese Hospitals' Competitiveness Rankings: http://www.ailibi.com/web/rank

Chinese Hospitals' Scientific Influence Rankings: http://top100.imicams.ac.cn/comprehensive?year=2013%26subject=320

Data analysis

The present study was a descriptive study and all data collected were compared and analyzed in a descriptive manner without statistical analysis. Data analyses were performed by using Microsoft Office 2016 and GraphPad Prism 9.0 (https://www.graphpad.com/scientific-software/prism) software was used for figure production.

RESULTS

A summary of the three mainstream Chinese hospital ranking systems

The “China's Best Hospital Rankings” system was created by the Hospital Management Institute at Fudan University in 2010,  and is published annually. It is composed of “China's Best Hospital Rankings (comprehensive)” and “China's Best Hospital Rankings by Specialty.” To create and update these rankings, the Hospital Management Institute of Fudan University maintains a pool of 3969 board‐certified physicians from 27 major specialties at a nationwide level. These hospital rankings are determined through the survey of board‐certified physicians combined with the research performance of each hospital. The Hospital Management Institute selects various physicians from each specialty at random each year and instructs them to complete a survey regarding hospital performance. This survey requires specialists to rank the Top 5 hospitals and Top 5 specialties at each hospital based on hospital performance, discipline, construction, clinical technology, medical quality, scientific research capability, and so forth. A scientific technology score was also added to the final results. The Top 100 hospitals are presented based on these survey results. The evaluation process for this ranking system is shown in Figure 1.

Figure 1

Assessment Process of China's Best Hospital Rankings. CS, overall score of each hospital; MAX(ROS), maximum value of hospital scientific research academic score; MAX(SR), the maximum score of specialty reputation; MIN(ROS), minimum value of hospital scientific research academic score; MIN(SR), minimum score of specialty reputation; N, the number of experts participating in the survey; ROS, academic score for scientific research; SDA, score of sustainability; SDA, score of sustainable development ability; SR, social reputation score of the specialty; SSR, standardized specialty reputation score

The annually updated “Chinese Hospitals' Competitiveness Rankings” was first published in 2011 by the Hong Kong Institute of Ailibi Hospital Management. This ranking system categorizes hospitals as public hospitals, the provincial capital, municipal public hospitals, local city public hospitals, county public hospitals, private hospitals, and traditional Chinese medicine hospitals. The data for these rankings were obtained from health administrative departments, clinics, healthcare associations, and the institute's own database. The underlying methodology of this ranking system is a combined weighting system. Following the specific inclusion and exclusion criteria, these institutes fill the database with hospital data. The specialists initially chose 500 hospitals and all of which are subsequently scored by a composite score using the TOPSIS weighting method. Eventually, the Top 100 hospitals are selected and reported.

The “Chinese Hospitals' Scientific Influence Rankings” were initiated in 2014 by the “Yeecin” Institute of Medical Informatics of the Chinese Academy of Medical Sciences. A total of 846 tertiary hospitals were evaluated in terms of hospital performance in 25 specialties. The ranking is determined by the extensive data‐driven analysis that combines measures of performance in three primary dimensions, which are aimed at measuring each hospitals' scientific influence: science and technology (S&T) input, S&T output, and academic influence. Factors that influence performance include published medical research, citation frequency, human resources, number of scientific research projects, scientific research hubs, authorized patents, education, training, excellence of individual researchers, academic participation, and both domestic and overseas cooperation. Data are obtained from public resources, including the Chinese Citation Database and GoPubMed Database. These rankings primarily include data from the past 5 years.

Data sources of these three major ranking systems

Table 1 presents the data sources for the three major Chinese hospital ranking systems. Data regarding scientific research under all three systems is obtained from publicly available or objective sources, including the thesis retrieval system, National Health Administration, and internal quantitative data from individual hospitals.

Table 1

Comparison of data sources

Ranking system	Data sources
China's Best Hospital Rankings	Survey of specialists at a nationwide level
	Research article published in Science Citation Indexed (SCI) journals, National science and technology awards, other awards, and so forth
Chinese Hospitals' Competitiveness Rankings	Public data
	Data from administrative departments, individual institutions, and healthcare associations
	Hong Kong Institute of Ailibi Hospital Management database
Chinese Hospitals' Scientific Influence Rankings	Chinese Citation Database
	GoPubMed Database
	Quantitative data including S&T input, S&T output, and academic influence

Abbreviations: S&T, science and technology.

Ranking methodologies of these three major ranking systems

Tables 2 and 3 present the specific methodologies that each ranking system adopted. The “China's Best Hospital Rankings” system uses reputational scoring, the impact factor (IF) of science citation index (SCI)‐indexed journals, and sustainable development capacity scores. The “Chinese Hospitals' Competitiveness Rankings” uses exploratory factor analysis, with quantitative scoring determined by TOPSIS weighting. The “Chinese Hospitals' Scientific Influence Rankings” uses the IF of SCI‐indexed journals, and converts it to a score.

Table 2

 Comparison of ranking methodologies and operation approach

A. Comparison of ranking methodologies
Hospital rankings	Methodology
China's Best Hospital Rankings	Hospital rankings reflect “peer review” reputational score combined with scientific research performance
Chinese Hospitals' Competitiveness Rankings	Hospital rankings were quantitatively scored and comprehensively analyzed using TOPSIS weighting
Chinese Hospitals' Scientific Influence Rankings	Hospital rankings data was obtained from Chinese and international databases. Hospital rankings were determined by S&T input, S&T output, and academic influence.

Abbreviation: SCI, science citation index; S&T, science and technology.

Table 3

Comparison of scientific research competitiveness evaluation systems

Hospital ranking system	Dimensions
	First grade	Second grade
China's Best Hospital Rankings	Academic level	IF of SCI publications
		Sustainable development capacity (first or second prize from the national science and technology progress awards or invention awards)
Chinese Hospitals' Competitiveness Rankings	Academic influence	Number of academicians
		Proportion of academicians in leadership
		Number of Ministry of Education approved disciplines or laboratories
		National Science Foundation of China (NSFC) grants
Chinese Hospitals' Scientific Influence Rankings	S&T input	Number of researchers
		Number of national scientific research projects
		National laboratories, national clinical laboratories, and clinical pharmaceutical trials
	S&T output	Number and citation frequency of SCI published studies in Chinese core journals
		Number of authorized patents
		Number of national medical guides formulated by the hospital
		Number of national continuing medical education projects
	Academic influence	Number of academicians
		Number of Changjiang scholars
		Number of National Science Fund for Distinguished Young Scholars appointees
,		Number of high‐level academic participants
		Number of core journal editors

Abbreviations: IF, impact factor; NSFC, National Science Foundation of China; SCI, science citation index; S&T, science and technology.

Scientific research capability ranking methodologies

Table 3 shows the methodologies of evaluating the scientific research capability among three major Chinese hospital ranking systems. “China's Best Hospital Rankings” only relies on S&T output, while “Chinese Hospitals' Competitiveness Rankings” pay more attention to academic influence. The “Chinese Hospitals' Scientific Influence Rankings” takes multiple quantitative indicators into account, such as S&T input, S&T output, and academic influence, and is considered relatively comprehensive.

Although both “China's Best Hospital Rankings” and “Chinese Hospitals' Scientific Influence Rankings” focus on S&T output, the detailed evaluation methods between these two are different. The “China's Best Hospital Rankings” includes articles published in SCI‐indexed journals, IF, and national awards in determining scientific ranking. The “Chinese Hospitals' Scientific Influence Rankings” rely on the citations of papers published in SCI‐indexed journals and Chinese academic journals, numbers of authorized patents, and status of participating in making national clinical management guides and hosting national continuing medical education projects.

The “Chinese Hospitals' Competitiveness Rankings” include in the assessment of academic dimensions, such as Ministry of Education approved key laboratory data and National Science Foundation of China (NSFC) grants. However, the “Chinese Hospitals' Scientific Influence Rankings” simply classifies these data points as S&T input. The number of researchers is also considered a component of S&T input.

In terms of academic influence, “Chinese Hospitals' Competitiveness Rankings” focuses on top academic talents (e.g., the number of academicians and the proportion of academicians in leadership). Meanwhile, “Chinese Hospitals' Scientific Influence Rankings” considers the number of Changjiang scholars, national science fund, or distinguished young scholars, high level of academic participants, and editors of core journals in addition to the number of academicians.

Emphases of scientific research evaluation systems

Table 4 shows the 21 evaluation indicators used by these three evaluation systems: “China's Best Hospital Rankings” uses two indicators, “Chinese Hospitals' Competitiveness Rankings” uses six indicators, and “Chinese Hospitals' Scientific Influence Rankings” uses 17 indicators.

Table 4

Emphases of scientific research evaluation systems

Evaluation index		Hospital ranking system
		China's Best Hospital Rankings	Chinese Hospitals' Competitiveness Rankings	Chinese Hospitals' Scientific Influence Rankings
SCI publications	Quantity	/	/	＋
	IF	＋	/	＋
	Citation frequency	/	/	＋
Publications in Chinese core journals	Quantity	/	/	＋
	H index	/	/	＋
	Citation frequency	/	/	＋
National wards		＋	/	/
NSFC	Quantity	/	＋	＋
	Total money	/	＋	/
National key disciplines		/	＋	/
National key laboratories		/	＋	＋
National continuing medical education projects		/	/	＋
National standard guide formulations		/	/	＋
Authorized patents		/	/	＋
Number of academicians		/	＋	＋
Proportion of academicians in leadership		/	＋	/
Number of Changjiang scholars		/	/	＋
Number of National Science Fund for Distinguished Young Scholars Appointees		/	/	＋
Number of people with high‐level academic participation		/	/	＋
Number of core journal editors		/	/	＋
Number of researchers		/	/	＋

Abbreviations: IF, impact factor; NSFC, National Science Foundation of China; SCI, science citation index.

The most commonly used indicators are SCI publications, NSFC funding, number of national key laboratories, and number of academicians, all of which are used by two hospital ranking systems. The additional indicators are only used by one of these three ranking systems.

The relative standing of several top hospitals according to the three major Chinese hospital ranking systems

Supporting Information: Tables S1–S3 show the rankings of eight top Chinese hospitals under the three major Chinese hospital ranking systems in 2016. The reason for choosing 2016 is that the dynamic changes of hospital rankings were analyzed for a total of 10 years (2010–2020). We considered that the hospitals in the middle year (i.e., 2016) would better represent the trend of changes. These eight top hospitals include Peking Union Medical College Hospital, West China Hospital of Sichuan University, Chinese People's Liberation Army (PLA) General Hospital, Ruijin Hospital affiliated with Shanghai Jiaotong University, Zhongshan Hospital affiliated with Fudan University, Xijing Hospital of the Fourth Military Medical University, The First Affiliated Hospital of Zhejiang University, and Peking University First Hospital. These hospitals are located in Beijing, Shanghai, Chengdu, Xi'an, and Hangzhou, covering North, East, and West China.

As shown in Supporting Information: Tables S1–S3, these rankings contain no specific details regarding how the scores were calculated. Merely the final ranking and total scores are available. Therefore, the investigators were unable to compare the differences in how these rankings were calculated across systems. Supporting Information: Table S4 shows a brief summary of the eight hospitals noted above and the rankings among these three hospital ranking systems. It was found that the top three hospitals in different rankings systems are consistent: Peking Union Medical College Hospital, West China Hospital of Sichuan University, and Chinese PLA General Hospital. These hospitals are ranked in the top three across all three systems, although the specific rankings are slightly different. The average ranking of Peking Union Medical College Hospital, West China Hospital of Sichuan University, and Chinese PLA General Hospital are 1.67, 2.00, and 2.33, respectively. Furthermore, the range in ranking difference across systems for each hospital is 1, 2, and 2, respectively. These findings illustrate that these top hospitals have strong reputations in the medical community, and the data used by each of these three ranking systems are consistent. Similar findings were also observed among other top hospitals, including Ruijin Hospital affiliated with Shanghai Jiaotong University, Zhongshan Hospital affiliated with Fudan University, and Xijing Hospital of the Fourth Military Medical University. All these institutions have good reputations and strong scientific research performance. Therefore, the average ranking lies between 4 and 7 across all three ranking systems, and the difference between the highest and lowest ranking for each hospital is less than five across all hospitals. Financial resources obtained both internally and from the government allow these hospitals to maintain their clinical and scientific strength, which likely accounts for the consistency seen at the top of these rankings across systems. Individual popularity and societal recognition may play a role in the minor differences seen across ranking systems.

It is interesting and worthwhile to discuss the last two hospitals on the list. Peking University First Hospital ranks as the fifth‐best hospital under the Chinese Hospitals' Competitiveness Rankings. In terms of clinical management, it has a very strong institution. However, based on the Chinese Hospitals' Scientific Influence Rankings, Peking University First Hospital ranks as the twentieth best hospital in China. This finding is not surprising, since this hospital is known to be somewhat weak in terms of scientific research competitiveness, especially when compared to that in other top hospitals. However, when its clinical reputation and scientific research performance were combined, Peking University First Hospital still can achieve as the tenth best hospital in China based on China's Best Hospital Rankings. Due to the extensive S&T input and resultant S&T output in recent years, the First Affiliated Hospital of Zhejiang University has gradually accumulated strength in scientific research competitiveness. It now ranks in the top five under the Chinese Hospitals' Scientific Influence rankings. However, it does not even rank among the top ten in other hospital rankings systems. These results suggest that although scientific research can strengthen a hospital, additional time and resources are required to gain public recognition for clinical excellence, and achieve a greater reputation in the medical field.

The comparative learning and analysis of Top 10 and Top 46–55 hospitals according to the ranking system of China's Best Hospital Rankings

At present, there is no objective comparison model to verify the reliability and validity of the hospital ranking systems currently developed in China. Therefore, we used the ranking system of China's Best Hospital Rankings and evaluated it with the analytical idea in mathematics (i.e., analysis of the extreme values including the maximum, median, and minimum values). In the study design, we originally planned to compare the top 10% (ranking 1–10), middle 10% (ranking 46–55), and bottom 10% (i.e., ranking 91–100) among the Top 100 hospitals. However, during the analysis, we realized that the ranking was relatively consistent for the top 10% and middle 10% but fluctuate for the bottom 10% over the study years. In other words, among the top 100 hospitals, while hospitals ranking Top 1–10 and Top 46–55 were relatively stable, hospitals ranking Top 91–100 were changing every year. Therefore, we felt that the inclusion of the hospitals ranking Top 91–100 was not reliable and decided to remove these hospitals from the analysis. Thus, we collected Top 10 and Top 46–55 Hospitals data from China's Best Hospital Rankings in 2016 for a longitudinal study to analyze the dynamic changes of hospital rankings for a total of 10 years (2010–2020), as we considered that the hospitals in the middle year (i.e., 2016) would better represent the trend of changes. Then, we collected the ranking of these hospitals in the latest data released by the ranking system China's Best Hospital Rankings from 2010 to 2020, with their specific scores to interpret the relationship between the changing trend of ranking and social reputation and scientific research weight. After comparison, the hospital change of ranking Top 10 was relatively small, while the ranking change of ranking Top 46–55 hospital was relatively fluctuating (Figure 2A). Three hospitals in the Top 10 ranking trended downward, while the other three hospitals trended upward. Similarly, several hospitals' ranking trended upward and downward among the Top 46–55 hospitals (Figure 2B).

Figure 2

Comparation and learning of Top 10 and Top 46–55 hospitals. (A) Rankings trends of Top 10 Hospitals of China's Best Hospital Rankings. (B) Rankings trends of Top 46–55 Hospitals of China's Best Hospital Rankings. (C) Six hospitals with social reputation and research scores that rose in the rankings from 2010 to 2020; (D) Five hospitals with social reputation and research scores that dropped in the rankings from 2010 to 2020. For (A) and (B), the thin solid line represents the social reputation score and the dotted line represents the research score (sustainability). CHFU, Children's hospital affiliated to Fudan University; PLAGH, The General Hospital of the People's Liberation Army

Then, we combined the rising and falling hospitals in terms of their social reputation and scientific research performance. For the hospitals at the top of the rankings, the social reputation is significantly higher than the proportion of scientific research; however, for the hospitals with Top 46–55 rankings, it is obvious that the proportion of scientific research is higher than that of social reputation (Figure 2C). Most of the reasons for the rise in the ranking are due to the stability of the social reputation, while the level of scientific research is constantly improving. Of course, there was a special one, the rise in Zhongda Hospital was mainly attributed to the improvement of scientific research. For several hospitals that have declined in recent years, further analysis and comparison found that the main reason for the decline in the ranking in the Top 10 was the decline in the social reputation, and the social reputation of several hospitals in the 46th to 55th ranking also contributed a large part (Figure 2D).

DISCUSSION

The present study analyzed the evaluation model and the proportion of scientific research competitiveness in three hospital ranking systems (i.e., China's Best Hospital Ranking, Chinese Hospitals' Competitiveness, and Chinese Hospitals' Scientific Influence Rankings). The importance of scientific research and technology in hospital development was analyzed through horizontal and longitudinal comparisons. As for the horizontal comparison, the recognized authoritative good hospitals ranked top in 2016 were selected to evaluate their scientific research and technology levels and the contribution of scientific research in their rankings. As for the longitudinal comparison, annual ranking changes of Top 10 and Top 46–55 hospitals from 2010 to 2020 were observed and the impact of scientific research and technology levels on the changes were analyzed.

As a typical knowledge‐intensive industry, hospitals must rely on the development of S&T, including novel knowledge and innovative techniques, to improve clinical practice, so as to keep their competitiveness. Globally, Massachusetts General Hospital affiliated to Harvard University, one of the top hospitals in the world, is a leader in promoting the clinical and serviceability of the hospital through scientific and technological innovation. Indeed, the establishment of research hospitals featuring translational medicine has been shown to be a successful model of sustainable development, and thus an important way to promote and maintain the core competitiveness of hospitals, and to build up world‐class hospitals. Hospital Rankings can not only guide patients to seek medical treatment objectively but also promote the competition among hospitals and accelerate the scientific development of the whole medical industry. In the present study, through the comparative analysis of the role of scientific research in the hospital ranking systems, we clearly demonstrated that the quality and quantity of SCI publications and the key indicators of national projects are important factors to improve the ranking of the hospitals in China.

In China's Best Hospital Rankings, scientific research accounts for only 20% of the evaluation system, and the social reputation, which is mainly based on expert voting, accounts for 80%. However, expert voting takes into account the hospital's scientific research ability, such as the SCI publication, the number of national‐level research projects, and other advanced technologies. Thus, when experts' evaluation is concretized and objectified, the proportion of scientific research in the ranking is far more than the superficial 20%, which further indicates the importance of scientific research and technology in the development of hospitals, and even in the development of medical care in the world. Meanwhile, in the Chinese Hospitals' Competitiveness Rankings and Chinese Hospitals' Scientific Influence Rankings, the actual proportion of science research and technology is also obviously much higher than what appears in the systems. Scientific research is required at hospitals in China, which is facilitated by a large number of clinical cases. As such, most hospitals use research outputs, such as SCI publications to determine the position promotion of medical staff in China.

The present study further analyzed the eight top‐ranked hospitals in 2016 under the three hospital ranking systems in China. These top hospitals have a strong reputation in the medical community, and all three ranking systems use consistent data. These hospitals also have great achievements in scientific research. Hospitals with a strong clinical ability or a good social reputation are often inseparable from their superb scientific research levels. Advanced scientific and technological equipment and top‐notch scientific research levels result in accurate diagnosis and efficacious treatment. Thus, the number of SCI publications, the number of national high‐level projects, and the number of high‐level academic talents such as academicians, which directly reflect the levels of scientific and technological research, may indirectly determine the probability of disease treatment.

In our study, we selected the Top 10 and Top 46–55 hospitals according to the ranking system of China's Best Hospital Rankings in longitudinal analysis and found that the fluctuation of the ranking of the Top 46–55 hospitals was significantly higher than that of the Top 10 hospitals. Further analysis found that for the hospitals ranked relatively in the middle, the annual fluctuation of social reputation is not large, indicating that social reputation has not a particularly great impact on the change of ranking. On the contrary, the proportion of scientific papers published by the hospitals tends to be an important factor influencing their rankings. Therefore, for middle and lower‐ranked hospitals, they need to constantly introduce advanced technology, new concepts of diagnosis and treatment, and strengthen scientific research levels.

Due to imperfect methodologies and insufficient data, Chinese hospital ranking systems still exhibit shortcomings. However, these show an improved capacity to evaluate scientific research talent and achievement. Therefore, ensuring the independence of these ranking systems and improving evaluation methodologies are very important. Meanwhile, the establishment of an authoritative database with the help of the government would give clinicians, researchers, and patients a platform for efficient data acquisition. In addition, relevant laws with regard to data protection are necessary, to maintain such a database. It is merely once these medical quality data sources are publicly available would this be able to help in the implementation of patient‐centered healthcare. In recent years, China has made great efforts to expand insurance coverage, strengthen healthcare infrastructures, especially primary healthcare facilities, and develop an Internet‐based healthcare system under the 14th Five‐Year Plan for National Economic and Social Development. The investigators consider that the new Five‐Year Plan will prioritize improvements in healthcare delivery and the establishment of such data platforms in the near future.

There are some limitations to this study. First, as part of the data is unpublic, we cannot do a more in‐depth analysis of the specific process of the ranking systems. Second, in terms of the social reputation of the ranking systems, the prestigious scholars are invited to make the score without standard reference content, which means that we can hardly control the impact of the social reputation on and independently evaluate the contribution of scientific research to the ranking systems. Therefore, more information and data are needed to verify our opinion. Third, the ranking systems used in this study were issued by an authoritative organization in China, but the evaluation tool of the ranking system in this field is not mature, and thus a large number of future studies are needed to verify our opinion.

Chinese hospital ranking systems could be improved in a number of ways. First, a third party should be introduced to increase the credibility of these results.  As these present ranking systems are not independent (the Hospital Management Institute of Fudan University and the “Yeecin” Institute of Medical Information are affiliated to universities), bias or at least a perception of bias may exist. Although the Hong Kong Institute of Ailibi Hospital Management is a commercial hospital management company, it is unable to obtain data from insurance companies, hospitals, or the Ministry of Health. Second, sophisticated databases should be established and improved. , At the present time, China still lacks relevant professional databases. As such, it is difficult to objectively evaluate medical quality. Third, certain medical services should be incorporated into the ranking systems, including clinical management, disease prevention, and health education. Hospital rankings need not only to strengthen publicity, but also to emphasize the experience, feelings, and satisfaction of patients. Both China's Best Hospital Rankings and Chinese Hospitals' Scientific Influence Rankings lack indicators in their methodologies to account for these clinical factors. In omitting such factors, these omit true patient experience. The Hong Kong Institute of Ailibi Hospital Management includes three indicators of medical technology in their assessment: clinical beds, hospital scale, and innovative medical technology. However, they omit quality data, such as mortality rate, readmission rate, the average length of stay, in‐patient expense at discharge, and so forth. ,

CONCLUSION

The proportion of scientific research plays an important role in the hospital ranking systems. The quality and quantity of SCI publications, the key indicators of national projects, and top academic talents are the most important factors used to evaluate the level of hospital scientific research, and thus affect the ranking of hospitals.

AUTHOR CONTRIBUTIONS

Dan Li and Jing Yu contributed equally to this study and should be considered cofirst authors. All authors participated in study design, data collection, and analysis. Jiyu Li contributed to the recommendation sections and has reviewed and approved the final version.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

Supplementary information.

Click here for additional data file.