A Knowledge Map for Hospital Performance Concept: Extraction and Analysis: A Narrative Review Article.

BACKGROUND: Performance is a multi-dimensional and dynamic concept. During the past 2 decades, considerable studies were performed in developing the hospital performance concept. To know literature key concepts on hospital performance, the knowledge visualization based on co-word analysis and social network analysis has been used.
METHODS: Documents were identified through "PubMed" searching from 1945 to 2014 and 2350 papers entered the study after omitting unrelated articles, the duplicates, and articles without abstract. After pre-processing and preparing articles, the key words were extracted and terms were weighted by TF-IDF weighting schema. Support as an interestingness measure, which considers the co-occurrence of the extracted keywords and "hospital performance" phrase was calculated. Keywords having high support with "hospital performance" are selected. Term-term matrix of these selected keywords is calculated and the graph is extracted.
RESULTS: The most high frequency words after "Hospital Performance" were "mortality" and "efficiency". The major knowledge structure of hospital performance literature during these years shows that the keyword "mortality" had the highest support with hospital performance followed by "quality of care", "quality improvement", "discharge", "length of stay" and "clinical outcome". The strongest relationship is seen between "electronic medical record" and "readmission rate".
CONCLUSION: Some dimensions of hospital performance are more important such as "efficiency", "effectiveness", "quality" and "safety" and some indicators are more highlighted such as "mortality", "length of stay", "readmission rate" and "patient satisfaction". In the last decade, some concepts became more significant in hospital performance literature such as "mortality", "quality of care" and "quality improvement".

Introduction

Given the vital role of hospitals in the health care systems, hospitals usually consume more than half of the health care resources in most countries(1). Hospital performance is defined in terms of the achievement of specific goals, either medical or managerial (2). The term performance indicates not only quality but also other factors such as cost of care, access to care, and the relations between patient satisfaction and expectations (3). High performance is identified as the provision of cost effective, high quality and properly available health services resulting in patient satisfaction (4). For achieving healthcare reform’s aims, more efficient and effective hospital care is requested. To improve hospital performance, scientific research and best practice models should be developed (5). WHO has defined “Performance” in relation to specific targets reflecting the values of different stakeholders such as patients, professions, insurers and regulators (6).There are deep gaps in knowledge in hospital performance area and the research should be developed to improve knowledge about factors influencing hospital performance (7).

However, a systematic and widespread review of this field is required to answer certain related questions. What research on Performance has been carried out? What are the main themes of accessible research in this field? And what additional research is needed? All these questions are significant for us to extend useful measures in order to improve and manage performance (8).Knowledge visualization and literature overview are two methods in order to answer these questions by illustrating a comprehensive picture of the field (9, 10).

Knowledge visualization shows the visual forms of data entities of scientific literatures (such as authors, keywords and so on) and their relationships between them(9,10). It may successfully increase knowledge in order to understand large amounts of data and to draw the structure and development of a scientific field (11). The co-word analysis is one of the methods to envision the structure of scientific fields and analyze academic literatures (12–14). It offers a rational picture of the real content of available documents (15, 16). The Co-word analysis has been employed in several academic and experimental studies for investigating the research themes and their interactions and modifications of special scientific fields; but it is hardly ever used in medical investigations (12).

In this study, the knowledge mapping based on co-word analysis and social network analysis has been used to know literature key concepts on hospital performance and to improve knowledge about factors influencing this important issue.

Methods

The co-word analysis is based on the hypothesis that a scientific field could summarize a group of indicator-words to overview literature and reveal its essential contents. The frequency of words occurrence in the whole documents of a special field can reveal the key concepts, and co-occurrence of several vocabularies in the same text shows the relevance of the themes that terms related to (17).

When the co-occurrence of two words in the literature increases, the similarity between the themes they show will raise. The keywords of scientific articles can be used as indicator-words (18, 19).

The Social Network Analysis (SNA) is used to map the interactions between elements in a structure (20). A network in SNA includes a collection of nodes and connections. While the nodes indicate the elements, the connections represent interactions between the nodes. In this study, the keywords network of research on hospital performance has been prepared. In this network, the nodes are the keywords and the connections stand for the co-occurrence of these keywords (21). The relationship between two nodes in a system can be affected and controlled by a node placed between them. The central position for a node happens when a node locates on the shortest direction between other nodes (22).

Data collection and Keywords Extraction

We searched Pub Med for studies on hospital performance from January 1945 (the time of first article publication) to September 2014. To develop an accurate search strategy; the primary search was conducted based on the terms related to the hospital performance. Initially, to identify and collect related documents and also to find appropriate keywords, about eighty full text articles in PDF format were found by using primary keywords which were selected with the help of 3 colleagues of the research team (Hospital Performance, evaluation, Quality indicators, and performance assessment) and through comprehensive databases such as Proquest, Emerald and Ovid. Then, articles were changed into CSV format to perform text refinement process and keywords extraction by the use of R-studio software (R3.1.0 version). To explore more essential keywords, these eighty articles were tokenized and normalized. Before the extraction, stop words, context free phrases, blank spaces, punctuations and numbers were omitted. All words were written in small letters and keywords stemming was performed. On the basis of Term Frequency Method, keywords were extracted. Then key words extraction were finalized for searching in Pub Med database with the help of TFIDF method and based on experts’ approval. As some terms may not be of the same importance as others in a document, the weight of each term was calculated by multiplying term frequencies (TF), using the inverse document frequency (IDF) for that term. Eq. [1] indicates how TF_IDF is calculated. Where tf is the number of times that term K occurs in all documents, N is the total number of documents in the corpus and n is the number of documents where the term K appears.

The search strategy was formulated according to finalized extracted keywords for searching Pub Med database (Fig. 1).

Fig. 1:

The search strategy for searching Pub Med database. TI=Title, AB= Abstract

We used “OR” operator to find all abstracts included each of keywords or combination of them. Also we used “AND” operator, quotations and parenthesis to specific search for exact keywords and combination of them.

Finally, 2775 articles were found. After omitting unrelated articles, the duplicates, and articles without abstract and keywords, 2070 articles entered into the study which included title, abstract and keywords. For the second time, words were tokenized and normalized and then terms were extracted. We extracted unigrams, bigrams and trigrams as keywords (with sparsity less than 0.99, 0.99 and 0.995, respectively).

Overall, 186 keywords are obtained by this method. Then the semantic relations between the keywords are considered via the expert opinions and the synonym keywords are replaced. Therefore, 337 keywords are reduced to 78 keywords. Then the support measure between each extracted keyword (including unigrams, bigrams and trigrams) and the “Hospital Performance” is calculated as [2]: Where S is a set of some keywords.

In this study, we consider S as {a keyword phrase, “Hospital Performance”}.

Finally, among the remained keywords, ones occurring more than or equal to 10 times, called high frequency words in this study, were selected to form the term-document matrix and document-term matrix( Fig.2).

Fig. 2:

The knowledge-mapping process

Term-document matrix indicates whether or not each term appears in a document. Therefore, it is a binary matrix.

Then, Term-Term matrix is calculated from term-document matrix as [3]:

After calculating Term-Term Matrix, the graph G is extracted from it. The nodes of G indicate the selected keywords and the edge E (A, B) between A and B indicate both of A and B exist in some documents. The weight of E (A, B) is high when both of A and B appear in many documents (23, 24). Then, the community including “hospital performance” is detected from G using fast greedy community detection algorithm. This algorithm is described in (25) with more details.

For the first and second periods of time, the identified communities include 25 and 21selected key words, respectively. The number of the selected key words in the detected community for all articles was 24.

Data Analysis and Mapping

The R-studio software (R3.1.0) was used for analyzing and mapping the data. Text Mining package (TM) in R is used for tokenization, stemming words and phrases, stop word removal and stem completion.

The i-graph package was used for social network analysis and drawing the graphs. Mining association rules in a large database is considered as an essential task in the area of data mining in order to determine hidden, interesting relations occurring between various data items (26). One important task for data mining algorithms is to search for models that are specific, logical and “surprising” (27). The criterion on the basis of which the search is performed is called “Interestingness measure”. One of the interestingness measures for mining association rules, called Support measure, was applied in this study. In addition, we used one of the common network centrality measure, called betweenness. In the graph, the size of the nodes indicates the support of keywords with hospital performance, and the thickness of the lines shows the strength of relationships between keywords pairs measured by betweenness value. The edge betweenness is defined as the number of shortest paths between the pairs of nodes that run through that edge (28) (We explained about how to calculate the support value and the edge betweenness value through an example in the appendix 1). To clarify the network structure, we put threshold for the edge betweenness which is equal to two times of the betweenness values average. (Two times of the average of betweenness values)

The findings were reported in two time periods: 1945–2004 and 2005–2014. The World Health Organization Performance Assessment Tool for Quality Improvement in Hospitals (PATH model) was implemented in the year 2004 and introduced a new framework for hospital performance assessment.

Results

Of 2775 articles, 2350 papers entered the study after omitting unrelated articles, the duplicates, and articles without abstract. In the next step, after omitting papers without keywords, 2070 ones were used as the input for further analysis.

The first article was published in 1975, so we changed the beginning year of the first time period to 1975. A summary of the basic statistics of the two networks is given in Table 1.

Table 1:

Descriptive Statistics for Hospital Performance Research 1975–2014

Papers	Total papers	Paper with keywords	Total extracted keywords	Total frequency of keywords	Keywords with high frequency (>10)	Total frequency of keywords with high frequency	Keywords with very high frequency (>20)	Total frequency of keywords with very high frequency
1975–2004	883	741	73	1880	34	1681	22	1492
2005–2014	1467	1329	78	3827	58	3655	34	3312
Total	2350	2070	78	5707	76	5691	45	5226

Knowledge mapping for 1975–2014

Table 1 indicates that there are 34 high frequency keywords extracted from the literatures up to 2004. The most high frequency words after “Hospital Performance” with 200 times frequency were “efficiency” and “financial” with 152 and 138 times frequency respectively.

There are 58 high frequency keywords extracted from the literatures during 2005–2014. The most high frequency words after “Hospital Performance” with 469 times frequency were “mortality” and “efficiency” with 286 and 229 times frequency respectively.

There are 76 high frequency keywords extracted from the literatures from 1975 up to 2014. The most high frequency words after “Hospital Performance” with 669 times frequency were “mortality” and “efficiency” with 413 and 381 times frequency respectively.

Figure 3 illustrates the keywords network showing the knowledge structure of hospital performance in published studies in the given period of time on the basis of support measure.

Fig. 3:

Map of keywords in hospital performance research, 1975–2014. The size of nodes indicates the support of keywords with hospital performance, and the thickness of the lines shows the strength of relationships between keywords pairs measured by between ness values

The map describes the major knowledge structure of hospital performance literature during these years and shows that the keyword “mortality” had the highest support with hospital performance followed by “quality of care”, “quality improvement”, “discharge”, “length of stay” and “clinical outcome” which also had high support, respectively. In addition, there are strong relationships between some keywords pairs in the map which are shown in Table 2 on the basis of betweenness value.

Table 2:

The strength of relationships between keywords pairs measured by betweenness value (edge betweenness>7)

Edge Vertex1	Edge Vertex 2	Edge betweenness
Electronic Medical Record	Readmission Rate	18
Waiting Time	Adverse Events	17
Patient Education	length of Stay	17
Complication Rate	Discharge	16
Medical Record	Admission – Discharge	15
Admission – Discharge	Mortality	14
Waiting Time	Medical Record	12
International Classification of Diseases	Readmission Rate	11
Quality of Life	Triage	10
Readmission Rate	Patient Education	10
Readmission Rate	Patient Follow-Up	10
International Classification of Diseases	Quality Improvement	10
Waiting Time	Readmission Rate	10
Medical Record	Morbidity	9
Patient Education	Post discharge	9
Quality Improvement	Post discharge	9
Electronic Medical Record	Triage	9
International Classification of Diseases	Admission – Discharge	9
Patient Identification	Post discharge	8
Quality of Care	Triage	8
Medical Record	Complication Rate	8
Readmission Rate	Complication Rate	8
Patient Education	Hospital Performance	8
Patient Education	Patient Identification	8
Quality indicators	Patient Identification	8
Patient Education	Medical Record	8
Quality Improvement	Patient Follow-Up	8

As Table 2 indicates, the strongest relationship is seen between “electronic medical record” and “readmission rate”

Furthermore there are strong relationships between keywords pairs such as “patient education” and “length of stay”, “quality of life” and “triage”, “readmission rate” and “patient education”, “readmission rate” and “patient follow up”, “patient education” and “post discharge”, “quality improvement” and “post discharge”.

As the map shows, the strong relationship is seen between “discharge” and “patient education” via “complication rate” and “readmission rate”, in addition to the straight weak relationship between them.

Table 3 indicates the support values of some keywords with hospital performance in 1975–2004, 2005–2014 and 1975–2014.

Table 3:

The support values of some keywords with hospital performance

1975–2004		2005–2014		1975–2014
Keywords	Support	Keywords	Support	Keywords	Support
Hospital Performance	0.269	Hospital Performance	0.353	Hospital Performance	0.323
Financial	0.058	Mortality	0.112	Mortality	0.091
Mortality	0.052	Quality of Care	0.058	Quality of Care	0.046
Efficiency	0.048	Quality Improvement	0.058	Efficiency	0.045
Discharge	0.028	Discharge	0.047	Financial	0.043
Staff	0.026	Efficiency	0.043	Quality Improvement	0.042
length of Stay	0.026	Staff	0.038	Discharge	0.041
Quality of Care	0.026	Financial	0.035	Staff	0.034
Hospital Cost	0.021	Safety	0.035	length of Stay	0.028
Diagnosis Related Groups	0.017	length of Stay	0.028	Safety	0.023
Quality Improvement	0.016	Clinical Outcome	0.024	Clinical Outcome	0.020
Budget	0.013	Quality indicators	0.024	Quality indicators	0.019
Clinical Outcome	0.013	Patient Safety	0.023	Drug Use Evaluation	0.016
Drug Use Evaluation	0.012	Readmission Rate	0.020	Patient Safety	0.014
Outpatient Services	0.010	Drug Use Evaluation	0.019	Readmission Rate	0.013
Quality indicators	0.010	Patient Care	0.016	Case mix	0.012
Accreditation	0.009	Case mix	0.015	Effectiveness	0.012
Revenue	0.009	Morbidity	0.014	Patient Satisfaction	0.011
Cost control	0.009	Adverse Events	0.013	Adverse Events	0.011
Quality Assurance	0.009	Accreditation	0.012	Diagnosis Related Groups	0.011
Adverse Events	0.008	Patient Satisfaction	0.011	Accreditation	0.011

As Table 3 shows, the support values of “mortality”, “quality of care” and “quality improvement” in 2005–2014 period of time are more higher than their support values in 1975–2004 period of time. Table 4 illustrates the keywords with highest support value with hospital performance in the trend mining.

Table 4:

The list of keywords with highest support value with hospital performance in the trend mining (up to five keywords for each year respectively)

Year	Keywords
1977	Mortality, Quality of Care
1978	Mortality, Quality of Care
1979	Efficiency, Mortality, Quality of Care
1980	Budget, Efficiency, Financial, Cost control, Hospital Cost
1981	Efficiency, Hospital Cost, length of Stay, Budget, Case mix
1982	length of Stay, Efficiency, Hospital Cost, Quality of Care, Budget
1983	Efficiency, length of Stay, Quality of Care, Budget, Case mix
1984	Efficiency, Financial, length of Stay, Budget, Discharge
1985	Financial, Efficiency, Budget, Diagnosis Related Groups, Equity
1986	Financial, Efficiency, Quality of Care, Diagnosis Related Groups, Equity
1987	Diagnosis Related Groups, Hospital Cost, Discharge, Mortality, Cost control
1988	Diagnosis Related Groups, Hospital Cost, Accreditation, Discharge, Efficiency
1989	Discharge, Hospital Cost, Diagnosis Related Groups, Mortality, Accreditation
1990	Financial, Mortality, Accreditation, Discharge, Antibiotics
1991	Financial, Mortality, Clinical Outcome, Discharge, Diagnosis Related Groups
1992	Financial, Mortality, Efficiency, Clinical Outcome, Diagnosis Related Groups
1993	Efficiency, Financial, Mortality, Staff, Clinical Outcome
1994	Financial, Mortality, Efficiency, Staff, length of Stay
1995	Mortality, Financial, Staff, Efficiency, Quality of Care
1996	Mortality, Discharge, Financial, Quality of Care, Staff
1997	Discharge, Mortality, Efficiency, Financial, Hospital Cost
1998	Discharge, Mortality, Quality of Care, Efficiency, Financial
1999	Efficiency, Mortality, Staff, length of Stay, Discharge
2000	Financial, Efficiency, Mortality, Staff, length of Stay
2001	Financial, Efficiency, Mortality, length of Stay, Discharge
2002	Financial, Efficiency, Mortality, Quality Improvement, Drug Use Evaluation
2003	Efficiency, Mortality, Financial, Quality Improvement, Quality of Care
2004	Efficiency, Mortality, Financial, Quality Improvement, Quality of Care
2005	Mortality, Quality Improvement, Quality of Care, Efficiency, Financial
2006	Mortality, Efficiency, Quality of Care, Quality Improvement, Financial
2007	Mortality, Quality Improvement, Discharge, Staff, Efficiency
2008	Mortality, Discharge, Staff, Financial, Quality Improvement
2009	Mortality, Discharge, Quality Improvement, Quality of Care, Financial
2010	Mortality, Quality of Care, Quality Improvement, Discharge, Efficiency
2011	Mortality, Quality of Care, Quality Improvement, Efficiency, Safety
2012	Mortality, Quality of Care, Quality Improvement, Discharge, Efficiency
2013	Mortality, Quality of Care, Quality Improvement, Discharge, Efficiency
2014	Mortality, Quality Improvement, Quality of Care, Discharge, Quality indicators

As Table 4 shows, “mortality” has the most co-occurrence with “hospital performance” in all years except 1980–1988 and also this term has the greatest support value with hospital performance since 2006. In the years before 2003, terms such as “financial”, “hospital cost”, “diagnosis related groups” and “efficiency” had more co-occurrence with “hospital performance”.

Discussion

The number of studies on hospital performance has been significantly increased in the last two decades. Hospital performance has become an emerging research field evolving and requiring a systematic analysis of its knowledge structure.

This study integrates the co-word analysis and the SNA to investigate the knowledge structure created by academic journal articles on hospital performance, in order to systematically examine the fundamental knowledge structure.

Hospital performance is a complex issue affected by the interaction of many factors. This concept includes some dimensions such as effectiveness, efficiency, safety, patient centeredness and human resources. Sub-dimensions and indicators for each dimension are determined.

For example, effectiveness dimension includes sub-dimensions/indicators such as outcome of care/mortality rate, readmission rate. Efficiency dimension includes sub-dimensions/indicators such as productivity/length of stay (29–34). The knowledge mapping in our study shows that some of these dimensions/sub-dimensions/indicators are more important than others in research field. Clinical outcomes consist of in-hospital mortality, readmission rate, emergency room visits after discharge, length of stay, Quality of life, satisfaction with care, costs and so on(35). Performance indicators can reflect the quality of care and use clinical outcomes. Clinical outcome data illustrates the quality of care, deliver to patients, through clearly defined set of measures. It relates to the direct administration of treatments in hospitals (36).Clinical outcome measures have some considerations such as differences in type of patient (confounding by patient characteristics), differences in measurement (determination and definition of cases, outcomes and risk factors), chance (random variation, influenced by number of cases and frequency of outcome occurrence) and differences in quality of care (use of proven interventions) (37).

While mortality data are readily available, instantly recognizable and usually recorded consistently, such data are rare regarding outcomes of other aspects of health status after treatment. It is, therefore, inevitable that mortality data will be used as a main indicator for assessing hospital performance (38). Studies show the clear associations between clinical processes and mortality. Mortality is a representative for clinical effectiveness. This indicator can be used to monitor the effect of quality improvement activities (39).

As shown in the map for keywords in hospital performance research during all years included in this study, “length of stay” is one of the keywords which have the high co-occurrence with hospital performance. In addition, length of stay is one of the essential measures used as a formative indicator (casual) of efficiency and at the same time as a reflective indicator (effect) of clinical effectiveness. The length of time patients spend in hospital for specific conditions has a great impact on overall costs of health system. Although longer hospital stays can be due to factors out of control by hospitals, a shorter hospital stay is more efficient from a hospital’s viewpoint so that it makes beds available faster to provide care for more number of patients. It can also reduce the cost per patient. However, too short stays may decrease the quality of care which can result in poorer patient outcomes (40).

In many countries the development of new policy strategies such as focusing on quality improvement and patient satisfaction highlight the importance of efficient and high quality hospital organizations. These policies make the hospital performance issue more considerable (5).

The Electronic Medical Record (EMR) is one of the most advantageous components of health information technology. Some studies showed the relationship between EMR implementation and 30-day re-hospitalization, 30-day mortality, inpatient mortality and length of stay (41).Data from the electronic medical record are used to create a clinical, strong, multivariable model to predict re-admission risk in hospitalized patients (42).

There are some interventions with the aim of reducing readmission within 30 days of discharge. For instance, pre-discharge interventions included patient education, discharge scheduling, and setting up a follow-up appointment before discharge. Post-discharge interventions included follow-up telephone calls, timely contact with ambulatory providers, timely follow-up and post-discharge home visits (43).

With continuous post-discharge home care, the patients’ readmission and mortality rates were decreased about 10% (44). Adib-Hajbaghery et al. have recommended strategies such as tele monitoring, telephone support, or internet follow-up system to affect the post-discharge hospital readmissions (45). As key members of the health care system, nurses have an important role in developing the strategies for follow-up and continuous care programs to prevent readmissions and decreasing the costs.

Initial triage is not only important to determine the treatment priorities, but also help in estimating the medical risk of patients which influences site-of-care decisions, and post-acute care needs (39). This could help physicians and nurses to make better decisions about need for hospital stays and organizing the post discharge process which affect the resource use, length of hospital stay, overall costs and patient’s outcomes in terms of mortality, re-hospitalization, quality of life and satisfaction with care (39).

There are limitations in data set used in this paper including the extent of the database and “indexer effect”. Pub Med does not provide a comprehensive coverage for the scientific studies on hospital performance, but it is well-received by the academic community. Furthermore, its computer based indexing technology significantly decreases the “indexer effect” (8). It satisfied the aims of this study to identify literature key concepts on hospital performance and research structure of this field.

Conclusion

Some dimensions of hospital performance were more important such as “efficiency”, “effectiveness”, “quality” and “safety” and some indicators were more highlighted such as “mortality”, “length of stay”, “readmission rate” and “patient satisfaction”. In the last decade, some concepts became more significant in hospital performance literature such as “mortality”, “quality of care” and “quality improvement” .Applying the knowledge mapping method for text mining especially in healthcare studies may provide comprehensive reference information for the researchers and policy-makers in healthcare systems. Results of this study can help hospital decision makers and policy makers to improve their conception about dimensions and indicators of hospital performance and to design a comprehensive, concise and relevant model for hospital performance assessment.

Ethical considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.

Document (article)	Extracted keywords
1	Accreditation, hospital performance, mortality
2	Hospital performance, case mix
3	Mortality, accreditation, case mix
4	Case mix, hospital performance, mortality
5	Hospital performance
6	Mortality

Path	0	1	2	3	4
0		(0,2),(2,1)	(0,2)	(0,2),(2,1),(1,3)	(0,2),(2,1),(1,4)
1	(0,2),(2,1)		(1,2)	(1,3)	(1,4)
2	(0,2)	(1,2)		(2,1),(1,3)	(2,1),(1,4)
3	(0,2),(2,1),(1,3)	(1,3)	(2,1),(1,3)		(3,1),(1,4)
4	(0,2),(2,1),(1,4)	(1,4)	(2,1),(1,4)	(3,1),(1,4)

Edge	Edge – Vertex names	Edge-betweenness
(0,1)	Hospital performance – Case mix	0
(0,2)	Hospital Performance – Mortality	8
(1,2)	Case mix – Mortality	12
(1,3)	Case mix – Accreditation	8
(1,4)	Case mix – Length of Stay	8
(2,3)	Mortality – Accreditation	0
(3,4)	Accreditation – Length of Stay	0