A bibliometric analysis of the application of artificial intelligence to advance individualized diagnosis and treatment of critical illness.

Background: Artificial intelligence (AI) has been extensively applied in the individualized diagnosis and treatment of critical illness, and numerous studies have been published on this topic. Therefore, a bibliometric analysis of these publications should be performed to provide a direction of hot topics and future research trends.
Methods: A bibliometric analysis was performed on the research articles to identify the hot topics and any unsolved issues regarding the use of AI in individualized diagnosis and treatment of critical illness. Articles published from January 2011 to December 2021 were retrieved from the Web of Science (WOS) core collection database for bibliometric analysis, and a cross-sectional analysis of the relevant studies that had been registered at ClinicalTrials.gov was also conducted.
Results: The number of articles published showed an annually increasing trend, with a worldwide geographic distribution over the past decade. Ultimately, 427 research articles were included in the bibliometric analysis. The relevant articles were divided into four separate clusters that focused on AI application aspects, prediction model establishment, coronavirus disease 2019 (COVID-19) treatment and outcome assessments, respectively. "Machine learning" was the most frequent keyword (147 occurrences, 165 links, and 395 total link strengths) followed by "risk", "models", and "mortality". With 205 articles, the United States of America (USA) had interacted the most with other countries (20 links, and 94 total link strength), while the domestic research institutes in China had infrequently collaborated with others. Approximately 130 trials focusing on the application of AI in the intensive care unit (ICU) and emergency department (ED) had been registered at ClinicalTrial.gov, and most of them (n=71, 54.6%) were interventional. The main research objectives of these trials were to provide decision making assistance and establish prediction models. However, only 3.8% (5 trials) of them had reached exact conclusions which favored the application of AI. Conclusions: The application of AI has raised great interest in critical illness and has mainly been focused on decision making assistance and prediction model establishment. Cooperation between agencies engaged in AI research needs to be strengthened. An increasing number of trials have been registered at ClinicalTrial.gov, and the results of them are promising. Keywords: Bibliometric analysis; artificial intelligence (AI); individualized diagnosis; critical care medicine; emergency department (ED). 2022 Annals of Translational Medicine. All rights reserved.

Introduction

Due to the deepened understanding of critical illness and advancements in treatment, the mortality rate of critically ill patients in the emergency department (ED) and intensive care unit (ICU) has gradually been decreasing (1). However, the pathophysiology of these patients is extremely complex, and it is sometimes difficult to make a clear differential diagnosis. Further, critically ill patients, such as those with sepsis, acute respiratory distress syndrome (ARDS), and infectious diseases, comprise a heterogenous population (2). Therefore, rapid diagnosis and personalized treatment based on dynamic and complex clinical situations are needed to improve the strategy making and prognosis of these patients (3).

Artificial intelligence (AI) refers to the ability of a computer or machine to perform tasks generally conducted by a human being (4). It was originally applied to image recognition, and later applied to critical care medicine to assist with difficult decision making (5). As the latest technologic advance in medicine, AI based on big-data has been conducted in most medical fields, especially in ICU, where is brimming with high volume of complex information (6).

It has been reported that AI can make an accurate diagnosis by using a pathology graphic and text analysis system and identify the subtypes of sepsis and ARDS from the Medical Information Mart for Intensive Care (MIMIC) database (7). What’s more, a lot of hard work has been made to develop AI extensions both to the consolidated standards of reporting trials (CONSORT) and standards for reporting of diagnostic accuracy studies (STARD) guidelines (8,9). Thus, the CONSORT-AI and STARD-AI can provide more evidence-based materials to the practice guidelines. It should also be noted that the most utilized methods are unsupervised learning techniques which work like a “black box”. Thus, the underlying process of getting the outcomes or results is not clear to the user (10-12). There were, however, studies about AI application to advance individualized medical decisions in ICU are fragmented over past decade. Herein, we aimed to conduct a bibliometric analysis and describes the details of published documents qualitatively and quantitatively in this field.

Registered trials have played an important role in changing daily clinical practice, especially during the coronavirus disease 2019 (COVID-19) pandemic (13). With the development of new drugs and therapies, individualized diagnosis and personalized treatment are urgently needed. However, information about the application of AI to critically ill patients is limited, and the application and development directions of AI for the next decade have not yet been clarified.

Thus, we conducted a bibliometric analysis to provide a comprehensive review of the AI field in critical care medicine and identify the currently solved and unsolved issues. Through analysis of the registered clinical studies, we also aimed to reveal the hotspots and future directions in this area.

Methods

Data sources and search strategy

Articles published from January 2011 to December 2021 were screened by two authors of the research team in the Web of Science (WoS) core collection database for bibliometric analysis. Additionally, a cross-sectional analysis of relevant studies that had been registered at ClinicalTrials.gov in the same period was carried out.

Search strategy was based on “PICO” framework. P—population/problem, which referred to the critical illness. I—intervention, in our study it comprised of various AI methods and machine-learning technologies. C—comparison, which indicated the difference between AI assistance and manual handling. O—outcome, which outlined the results of individualized treatment and diagnosis (14,15). Search strategy was adopted from previous work and experts’ opinions (15,16). We refined the query to include keywords related to critical illness (“critical care/treatment”, “intensive care”, “intensive care unit”, “ICU”, “emergency medicine/treatment/service/care/department”, and “EICU”), AI technologies (“artificial intelligence”, “AI”, “algorithmic prognostication”, “computational intelligence”, “machine learning”), and individualized treatment and diagnosis (“individual”, “personalized”) in both Medical Subject Headings (MESH) and titles. The data for bibliometric analysis were extracted and downloaded in text format from the database, including details of the publication, authors, and titles.

There are two standard weight attributes for an item which are defined as the links attribute and the total link strength attribute. They represent the number of links of an item with other items and the total strength of the links of an item with other items, respectively. For example, in the case of co-authorship links between researchers, the links attribute indicates the number of co-authorship links of a given researcher with other researchers. The total link strength attribute indicates the total strength of the co-authorship links of a given researcher with other researchers. Both weight attributes were calculated by VOSviewer with full counting. Further, the full data of the trials, including study type, participant age, gender, status, and study outcomes, were recorded from ClinicalTrials.gov.

Inclusion and exclusion criteria

The articles for bibliometric analysis were restricted to original articles with English written. The exclusion criteria were as follows: (I) documents were written in non-English language and (II) documents were classified into non-original article type according to the classification service.

Statistical analysis

The intrinsic functions of the WoS core collection database were applied to describe the basic characteristics of the retrieved publications. The online analysis platform of literature metrology (http://bibliometric.com/) successfully validated for bibliometric analysis and was used to determine the international cooperative relations in this study (17,18).

VOSviewer (version 1.6.17; Leiden University, The Netherlands) was mainly used to construct and visualize co-occurrence networks of co-authorship, co-occurrence, citation, keywords, and themes extracted from literature. CiteSpace (Version 5.8 R3; Chen Meichao, Drexel University, Philadelphia, PA, USA) was used to reveal the evolution and turning point of the research field. Other analysis tools such as Bibliographic Item Co-Occurrence Matrix Builder version 2.0 (BICOMB; Lei Cui, China Medical University, Shenyang, China) and Graphical Clustering Toolkit version 1.0 (gCLUTO; University of Minnesota, Minneapolis, MN, USA), were also applied to provide matrix of keywords clustering results. The categorical variables of the studies registered at ClinicalTrial.gov were described by frequencies and percentages.

Keywords occurred more than five times were defined as high-frequency keywords in VOSviewer and keywords burst defined as the occurrence of the keywords varied greatly over one year by CiteSpace (19). All clusters were calculated by VOSviewer, CiteSpace and gCLUTO with its own clustering algorithm.

Results

Publications output

A total of 593 eligible publications were selected, of which 432 (72.8%) were original articles, 93 (15.7%) were reviews, 64 (10.8%) were conference papers, and 5 were other types of articles. It should be noted that almost half of the articles (34.6%) were published in 2021. Ultimately, 427 English written research articles were included in the bibliometric analysis ().

Figure 1

Flow diagram of article selection process. WoS, Web of Science.

Growth trend of publications

A rapid increase in the number of articles published in this field from 2011 to 2021 were detected, for which there was a wide global geographic distribution (). The number of articles published in 2021 (161 articles) was about 35 times higher than that in 2011. Further, 54 countries contributed to publications on the application of AI in the ED and ICU. The United States of America (USA) published the most articles (n=201), followed closely by China (n=57).

Figure 2

Overview of publication output between 2011 and 2021. Top 10 countries based on the annual numbers of publication. India was joint 10th along with Brazil.

Bibliometric analysis of the keywords

The keywords were identified by VOSviewer and CiteSpace. Both tools extracted all keywords from the included articles based on the citation index service of WoS core collection database. Finally, a total of 2,504 keywords were identified from the included articles. Among them, 93 keywords were high frequency and included in the analysis. The co-occurrence of high-frequency keywords were divided into six clusters with a total link strength of 1,835 (). “Machine learning” was the most frequent keyword (147 occurrences, 165 links, and 395 total link strengths), and was linked to 88 other high-frequency keywords (). “Artificial intelligence” (51 occurrences, 52 links, and 136 total link strengths), “risk” (43 occurrences, 61 links, and 146 total link strengths), “models” (42 occurrences, 63 links, and 144 total link strengths), and “mortality” (37 occurrences, 59 links, and 149 total link strengths) were the other four top high frequency keywords. Other high frequency keywords that ranked in the top ten included “classification”, “prediction”, “care”, “deep learning”, and “COVID-19”.

Figure 3

Co-occurrence network of keywords from 2011 to 2021. (A) The co-occurrence of all high-frequency keywords. (B) The co-occurrence of “Machine learning” and “Artificial intelligence”. (C) The top 15 keywords with the strongest burst value. The size of the nodes indicates the frequency of occurrence. The curves between the nodes represent their co-occurrence in the same publication. The shorter the distance between 2 nodes, the larger the number of co-occurrences of the 2 keywords. The red stripe demonstrated the time period of keyword burst appearance and the blue stripe was the period of keyword occurrence.

The top 15 keywords with the strongest burst values are summarized in . From 2011 to 2021, “big-data” had the strongest burst strength (strength: 6.74), followed by “machine learning algorithm” (strength: 5.51), “patient care” (strength: 4.51), “real time” (strength: 3.95), and “deep learning” (strength: 3.79). In 2017, the most frequently observed keyword bursts included “precision medicine” (strength: 3.66), “heart rate” (strength: 3.66), and “treatment decision” (strength: 3.05), to the exclusion of “big-data” and “machine learning algorithm”. The two keywords that maintained high burst values from 2019 to 2021 were “deep learning” (strength: 3.79) and “machine learning” (strength: 2.89).

Co-citation clustering and time evolution analysis

The whole silhouette [defined as the homogeneity of a cluster, values ranges (0–1), ≥0.7 represent the clustering is efficient and convincing] of the co-cited references clustering was 0.93, and the Q score was 0.88 [defined as modularity, values ranges (0–1), ≥0.3 represent the clustering is significant] by analysis tool CiteSpace. Articles on AI application research in the ED and ICU were divided into 81 clusters, and the clusters of the largest connected components (as labeled by the log likelihood ratios) are shown in . The top five largest clusters were “COVID-19” (#0, size: 54, silhouette: 0.94), “artificial intelligence” (#1, size: 44, silhouette: 0.90), “optimization” (#2, size: 38, silhouette: 0.94), “triage” (#3, size: 16, silhouette: 0.96), and “risk prediction” (#35, size: 3, silhouette: 0.99). Additionally, the time evolution analysis indicated that the top five largest clusters were highly cited after 2013 ().

Figure 4

Co-citation clustering and time evolution analysis. (A) The top 5 clusters of the co-cited references analysis. (B) The time evolution analysis of the top 5 clusters of the co-cited references.

Research themes and topic trends

In the application of AI, four theme clusters were found among the ED and ICU studies (). The red cluster mainly represents the fundamental and new research areas in the application of AI. The green cluster shows the prediction model establishment. The blue cluster represents the performance and future prospect of AI in COVID-19 treatment. The yellow cluster indicates the outcomes of patients with critical illness. shows the trends of these topics, and the dot color, which ranges from purple to yellow, illustrates different research times. Recently, studies have mainly focused on the relation of machine learning models and clinical outcomes.

Figure 5

Network map of the trend topics. (A) Research themes listed by cluster. (B) Topic trends.

According to the matrix generated by BICOMB and bi-clustering by gCLUTO, the keywords that appeared more than three times were divided into six clusters (). The different clusters of the keywords represent the application of AI in different disease study fields. For example, Cluster 0 represents the application of AI to dementia, Cluster 1 represents the application of AI technology to human immunodeficiency virus research, Cluster 3 and Cluster 4 summarize the algorithmic design in the pediatric and mental health fields, respectively. In addition, Cluster 5 focuses on the clinical application of AI to sepsis, COVID-19, and severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).

Figure 6

Visualized matrix of bi-clustering of keywords.

Analysis of co-authorship

According to the domestic and international literature search, 56 countries published relevant papers. displays the cooperative relationship among the countries or regions. Notably, with 205 articles, the USA had the highest frequency of interactions with other countries (20 links, and 94 total link strength). The main active collaborators with the USA were China, England, and Canada. Of the 1,018 institutions, 34 institutions that had published five or more papers were included in the analysis. Harvard Medical School had the most institutional partnerships with 17 organizations (25 total link strength), and their main partner was Massachusetts General Hospital (4 link strength), followed by Massachusetts Institute of Technology (18 total link strength), and Stanford University (16 total link strength). Additionally, with six articles and 28 citations, Zhejiang University had no links to other organizations.

Figure 7

Network of countries/regions cooperation.

Analysis of the registered trials

From 1st January, 2011 to 31st December, 2021, 130 trials were identified that examined the application of AI in the ED and ICU. Among them, 59 (45.4%) were observational study and 57 (43.8%) are applied to prospective data. Additionally, 25 trials (19.2%) had recruited children as participants. In terms of the status of the trials, only 36 (27.7%) had been completed. The results of most (125 trials, 96.2%) were not able to be retrieved during our study period. Further details are provided in .

Table 1

Characteristics of 130 clinical trials registered in ClinicalTrial.gov

Characteristics	n (%)
Study type
Interventional	71 (54.6)
Observational	59 (45.4)
Time perspective
Prospective	57 (43.8)
Retrospective	17 (13.1)
Others	56 (43.1)
Participant age (years)
<18	25 (19.2)
18–64	114 (87.7)
≥65	116 (89.2)
Trial status
Not recruiting	20 (15.4)
Recruiting	47 (36.2)
Completed	36 (27.7)
Suspended	1 (0.8)
Terminated/withdrawn	7 (5.4)
Unknown	6 (4.6)
Results
With results	5 (3.8)
Without result	125 (96.2)
Funding source
Industry	15 (11.5)
Other	115 (88.5)
Locations
America	85 (65.4)
Asia	21 (16.2)
Europe	52 (40.0)
Middle East	5 (3.8)

The registered trials had three predominant basic objectives. First, a great number of trials sought to evaluate or create deep learning and machine learning models for predicting the clinical or cognitive status of patients admitted to the ED or ICU. Notably, the algorithms used in these studies sought to assist practitioners in patient care decision making. Second, other trials sought to establish a medical database that included standardized, structured clinical diagnosis, and treatment information, and provide a multi-modal data technology system for future AI technology. Third, other trials sought to assess AI-assisted operations or robotic surgery in the improvement of public health.

Discussion

In this study, we conducted a bibliometric analysis of the application of AI in critical illness and undertook a comprehensive review of the relevant trials registered at ClinicalTrials.gov. Though 427 articles from WoS core collection database and 130 clinical trials from ClinicalTrials.gov were limited quantities according to our study, the results based on authoritative databases illustrated the achievement of AI application in individualized diagnosis and treatment were gradually increased. Therefore, it has great importance and applicability in the clinical practice. One attractive result from our analysis was that the number of published articles has continued to increase annually, and in the last decade, the global geographic distribution of the articles has been wide. The relevant articles focused on the following four aspects: (I) AI application; (II) prediction model establishment; (III) COVID-19 treatment; and (IV) outcome assessment. Among the trials that had been registered, only 3.8% (5 trials) of them had reached exact conclusions which favored the application of AI.

With the development of modern science, more external life support equipment has been applied in the treatment of critically ill patients, such as mechanical ventilation (MV), extracorporeal membrane oxygenation (ECMO), and continuous renal replacement therapy (CRRT). Thus, the success rate of resuscitation in the ED or ICU is gradually improving and a great volume of data can be collected during the hospital stay (20,21). “Big Data”, “Machine Learning” and “AI” have already formed an “iron triangle” (22,23), and our bibliometric analysis revealed that most of these terms were keywords in the included studies.

Critical illness management always requires rapid diagnosis and timely treatment. However, sometimes decision making is very difficult due to the similarity of the clinical symptoms. Thus, AI assistance has been applied based on “Big Data” analyses to establish prediction models for disease diagnosis and outcome evolution (24,25). For example, multimodal models based on dynamically associated biological markers and basic characteristics of the hosts have already been established for risk and mortality prediction in ICU patients with COVID-19 (24,26). Meanwhile, natural language processing (NLP) plays another important role in extracting valuable information, which can provide high-quality data for AI and benefit for treatment strategy making, dosage of medications adjusting and parameters changing in MV (27). Furthermore, medical imaging, offers potential prospects for diagnosing, monitoring and surveying complications (28). It is also reported that the combination of AI and specimen morphological recognition has a promising future in differential diagnosis of interstitial exudative inflammation of lung in ICU (29).

Due to the heterogeneity of the diseases and population, individualized treatment is recommended in the ICU. It has been demonstrated that AI assistance can clearly distinguish phenotype and retype in sepsis, ARDS and other symptom complexes (30,31). Thus, the combination of a reliable prediction model and personalized medicine may improve the outcome. However, both internal and external validation is required to establish a credible prediction model. Digital twin, a virtual/digital replica of physical entities aimed to access cost-effective simulations, was always combined with AI analysis tools, which can quickly evaluate many possible ICU treatment alternatives. It is still in its infancy and data collection and team establishment are quite difficult. Unify data and model standards, share data and models, innovate on service and establish forums would make research and development of digital twins more coherent in critical care medicine (32,33). Meanwhile, it is also important to strengthen the cooperative relationship between different countries and institutions to make up for the small sample size of a single institution.

The ethical issue of AI is another unavoidable matter and it has already become one of the mostly controversial challenge of emergency and critical care medicine. AI and algorithms are established from the electronic systems and database and algorithms written may naturally contain errors which may lead to unforeseen consequences. Thus, current regulatory oversight is quite necessary and “double check” is recommended (34,35).

Clinical trials play an important role in the diagnosis and treatment of critically ill patients. The safety and effectiveness of new drugs and therapies have been tested in these trials, and some of them might be suggested in practice guidelines and recommendations as changes to the treatment strategy (36,37). ClinicalTrials.gov is a public registry platform that increases the transparency of research projects in progress or those that have been completed (38). As many as 130 trials have been registered on this platform, and 43.8% of them are applied to prospective data which will coincide with the development in critical care medicine. It can be expected that they may have more impact and can provide more practical instruction. Notably, only 27.7% of the registered trials have been completed and only 3.8% of the clinical trials have presented favorable results. Thus, the feasibility of these trials may still be problematic, and a high-quality control process and good coordination among multidisciplinary teams are needed.

The present study provided a comprehensive review of the application of AI in the ED and ICU, but still has some limitations. First, the WoS core collection database and ClinicalTrials.gov are updated dynamically, and thus, the most recent data might be missing from this study. Second, we only included English written articles and trials in the final analysis. However, the findings of articles and trials written in other languages may not concur with our results. Thus, bibliometric analysis and review of the registered trials should also be dynamic and updated in the future.

Conclusions

The application of AI to critical illness for individualized diagnosis and treatment is a research field with great potential. Cooperation between agencies engaged in AI research needs to be strengthened. A growing number of trials focusing on decision-making assistance and prediction model establishment have been registered at ClinicalTrial.gov, and a quality-control process should be implemented to ensure research completion.

The article’s supplementary files as