The most influential articles in critical care medicine.

PURPOSE: The study aimed to examine query strategies that would provide an exhaustive search method to retrieve the most referenced articles within specific categories of critical care.
MATERIAL AND METHODS: A comprehensive list of the most cited critical care medicine articles was generated by searching the Science Citation Index Expanded data set using general critical care terms keywords such as "critical care," critical care journal titles, and keywords for subsubjects of critical care.
RESULTS: The final database included 1187 articles published between 1905 and 2006. The most cited article was referenced 4909 times. The most productive search term was intensive care. However, this term only retrieved 25% of the top 100 articles. Furthermore, 662 of the top 1000 articles could not be found using any of the basic critical care search terms. Sepsis, acute lung injury, and mechanical ventilation were the most common areas of focus for the articles retrieved.
CONCLUSION: Retrieving frequently cited, influential articles in critical care requires using multiple search terms and manuscript sources. Periodic compilations of most cited articles may be useful for critical care practitioners and researches to keep abreast of important information. Copyright 2010. Published by Elsevier Inc.

Introduction

Modern methods to search for important medial literature have recently flourished with the dissemination and near ubiquity of Internet-based search engines (eg, PubMed, MEDLINE, OVID). Articles that have been cited most often, referred to as “citation classics,” are frequently important sources of both detailed information that has changed clinical/research practices as well as of useful connectors to other relevant literature that cites these articles. Finding landmarks or classic articles still requires an understanding of the literature and how articles are referenced. Different databases may categorize disciplines and journals from the same clinical areas under different terms and yield different results when using the same search term.

These referencing differences as well as other conceptual arguments have led to some debate regarding attributing the number of times an article is cited to its quality [1], [2], [3], [4] However, there is general agreement that the number of times an article is cited does reflect the “impact of that article on the scientific market” [5].

Garfield [6] first published citation classics of the Journal of the American Medical Association in 1987. Similar studies (citations classic or impact factors) have been published with respect to various clinical specialties [7], [8], [9], [10], [11], [12], [13], [14], [15]. In 2004, Baltussen and Kindler [16] published citation classics of the critical care literature and enumerated the 71 top-cited articles in critical care journals and the 45 top-cited critical care articles in non–critical care journals. This article was somewhat limited by the number of citations referenced as well as the absence of specific critical care topic citations groups. The first and primary aim of this study was to expand on Baltussen's earlier work and provide an updated and more comprehensive catalogue of the most influential articles pertaining to critical care medicine and within specific categories of critical care, something not done in previous studies of this type. We also examined the methods used to retrieve articles to provide an exhaustive search method that future researchers may use in modern electronic databases to find frequently cited critical care articles.

Creation of master comprehensive list

To develop the most comprehensive master list of most often cited critical care medicine–related articles, we first focused on querying the ISI Web of Knowledge (2008 Thompson Corporation) using the Science Citation Index Expanded (SCI Expanded) data set. The SCI Expanded is a database that provides bibliographic information, abstracts, references, and citations counts for more than 6650 scholarly science and technology journals. The SCI Expanded search was limited to articles published from 1900 to December 31, 2007. To find any article among the broadest selection of scientific journals related to critical care, we searched for any article that contained the keywords “critical-care,” “intensive-care,” “ICU,” or “critically-ill OR critical-illness” and that had been cited more than 100 times. A total of 472 articles were retrieved with this technique after duplicates were removed (see Fig. 1A).

Fig. 1

Methods.

Because many critical care articles found in the SCI Expanded were not tagged by the general keywords noted above, we also searched within critical care journals. To do this, we used the Journal Citation Report (JCR) 2006 to identify journals specializing in critical care medicine from more than 7500 scholarly journals. We first reduced the target list to 16 English-language journals of 18 critical care journals identified by the JCR. We reviewed any article that had been cited more than 100 times and saved articles pertaining to critical care medicine (Fig. 1B). This search resulted in 470 hits; however, we removed 159 duplicate articles that were already found by keyword searches to bring the total database to 783 articles (Fig. 1).

Creation of disease/condition-specific lists

A representative list of most cited articles for each of the specific critical care topics was created from the existing database and by searching each topic individually using the terms in Table 1 . Using these search terms, articles pertinent to critical care and referenced more than 100 times were added to the database. These searches by specific disease or conditions added 404 articles to the master database as 194 articles were already in the database from either the initial search or from the journal search. From this master database of 1187 articles, we then generated a list of all articles cited more than 150 times (Table 2 ). We organized this table by topics particularly relevant to critical care. Within each category we listed both the most cited articles within that topic as well as that article's ranking within the top 200 most cited articles. Finally, we also listed the absolute number of citations for each article.

Table 1

Specific critical care subjects

Subject area	Search terms
Sepsis and infectious disease	“sepsis” “septic*” “nosocomial” “blood stream”
Mechanical ventilation and lung injury	“ventilation” “respiratory failure” “pulmonary failure” “weaning” “extubation” “ventilator” “PEEP” “positive end expiratory pressure” “noninvasive” “ARDS” “respiratory-distress” “lung injury”
Monitors	“hemodynamic-monitor” “CVC” “physiologic-monitor” “noninvasive-monitor” “catheter*” “central venous” “pulmonary-artery-catheter” “Swan-Ganz” “ultrasound” “Doppler” “pressure variation”
Mental status	“coma” “brain-damage” “brain-injury” “ICU AND delirium” “ICU AND sedation”
Physiologic predictors	“physiology score” “severity of illness” “severity of disease” “mortality-predict” “severity-score”
Organ dysfunction	“organ-failure” “organ-dysfunction” “MODS” “MSOF”
Resource utilization	“intensivist” “leap frog” “ICU-administration” “critical care management” “ICU management”
Kidney injury	“kidney-injury” “renal-failure” “dialysis” “renal-replacement-therapy” “CVVH*” “CRRT”
Resuscitation	“life-support” “cardiac-arrest” “cpr” “resuscitation” “cardiac massage” “cardiac-life-support” “fluid” “saline” “lactated” “albumin” “colloid” “crystalloid”
Shock (excluding infectious causes)	“shock” “hypoperfusion” “CHF” “heart-failure”
Transfusions	“transfusion” “fluid-therapy”

⁎Denotes any form of the word preceding the symbol, i.e., septic shock, septic.

Table 2

Most cited articles in critical care medicine by subject

Overall Citation Rank
A. Sepsis/systemic inflammatory response syndrome (no. of times cited)
1. (#6) Tracey, KJ et al. Anti-cachectin TNF monoclonal-antibodies prevent septic shock during lethal bacteremia. Nature 1987. (1983)
2. (#7) Bone, RC et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest 1992. (1838)
3. (#8) Bernard, GR et al. Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med 2001. (1827)
4. (#10) Bone, RC et al. American-College of Chest Physicians Society of Critical Care Medicine Consensus Conference—definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Crit Care Med 1992. (1724)
5. (#12) Van den Berghe, G et al. Intensive insulin therapy in critically ill patients. N Engl J Med 2001. (1608)
6. (#19) Ziegler, EJ et al. Treatment of gram-negative bacteremia and septic shock with HA-1a human monoclonal-antibody against endotoxin—a randomized, double-blind, placebo-controlled trial. N Engl J Med 1991. (1072)
7. (#22) Angus, DC et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001. (1016)
8. (#25) Bone, RC. The pathogenesis of sepsis. Ann Intern Med 1991. (982)
9. (#28) Waage, A et al. The complex pattern of cytokines in serum from patients with meningococcal septic shock—association between interleukin-6, interleukin-1, and fatal outcome. J Exp Med 1989. (892)
10. (#31) Wichterman, KA et al. Sepsis and septic shock—a review of laboratory models and a proposal. J Surg Res 1980. (846)
11. (#34) Bone, RC et al. A controlled clinical-trial of high-dose methylprednisolone in the treatment of severe sepsis and septic shock. N Engl J Med 1987. (791)
12. (#37) Parrillo, JE. Mechanisms of disease—pathogenetic mechanisms of septic shock. N Engl J Med 1993. (756)
13. (#41) Annane, D et al. Effect of treatment with low doses of hydrocortisone and fludrocortisone on mortality in patients with septic shock. JAMA 2002. (712)
14. (#43) Petros, A et al. Effect of nitric-oxide synthase inhibitors on hypotension in patients with septic shock. Lancet 1991. (705)
15. (#52) Cannon, JG et al. Circulating interleukin-1 and tumor necrosis factor in septic shock and experimental endotoxin fever. J Infect Dis 1990. (642)
16. (#55) Casey, LC et al. Plasma cytokine and endotoxin levels correlate with survival in patients with the sepsis syndrome. Ann Intern Med 1993. (624)
17. (#56) Vanzee, KJ et al. Tumor-necrosis-factor soluble receptors circulate during experimental and clinical inflammation and can protect against excessive tumor-necrosis-factor–alpha in vitro and in vivo. Proc Nat Acad Sci USA 1992. (600)
18. (#59) Hack, CE et al. Increased plasma-levels of interleukin-6 in sepsis. Blood 1989. (589)
19. (#62) Moore, FA et al. Early enteral feeding, compared with parenteral, reduces postoperative septic complications—the results of a metaanalysis. Ann Surg 1992. (578)
20. (#64) Lowenstein, CJ et al. Nitric-oxide—a physiological messenger. Ann Intern Med 1994. (574)
21. (#69) Fry, DE et al. Multiple system organ failure—role of uncontrolled infection. Arch Surg 1980. (549)
22. (#70) Parker, MM et al. Profound but reversible myocardial depression in patients with septic shock. Ann Intern Med 1984. (548)
23. (#72) Kudsk, KA et al. Enteral versus parenteral-feeding—effects on septic morbidity after blunt and penetrating abdominal-trauma. Ann Surg 1992. (543)
24. (#75) Fisher, CJ et al. Treatment of septic shock with the tumor necrosis factor receptor:Fc fusion protein. N Engl J Med 1996. (540)
25. (#77) Hotchkiss, RS et al. Medical progress: the pathophysiology and treatment of sepsis. N Engl J Med 2003. (538)
26. (#78) Greenman, RL et al. A controlled clinical-trial of E5 murine monoclonal IgM antibody to endotoxin in the treatment of gram-negative sepsis. JAMA 1991. (536)
27. (#81) Martin, GS et al. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003. (529)
28. (#83) Glauser, MP et al. Septic shock—pathogenesis. Lancet 1991. (522)
29. (#89) Dellinger, RP et al. Surviving sepsis campaign guidelines for management of severe sepsis and septic shock. Crit Care Med 2004. (513)
30. (#94) Bone, RC et al. Sepsis syndrome—a valid clinical entity. Crit Care Med 1989. (506)
31. (#96) Fisher, CJ et al. Recombinant human interleukin-1 receptor antagonist in the treatment of patients with sepsis syndrome—results from a randomized, double-blind, placebo-controlled trial. JAMA 1994. (499)
32. (#97) Parrillo, JE et al. Septic shock in humans—advances in the understanding of pathogenesis, cardiovascular dysfunction, and therapy. Ann Intern Med 1990. (499)
33. (#105) Assicot, M et al. High serum procalcitonin concentrations in patients with sepsis and infection. Lancet 1993. (481)
34. (#109) Ochoa, JB et al. Nitrogen-oxide levels in patients after trauma and during sepsis. Ann Surg 1991. (464)
35. (#114) Moore, FA et al. TEN versus TPN following major abdominal-trauma–reduced septic morbidity. J Trauma 1997. (435)
36. (#120) Abraham, E et al. Efficacy and safety of monoclonal-antibody to human tumor necrosis-factor–alpha in patients with sepsis syndrome—a randomized, controlled, double-blind, multicenter clinical-trial. JAMA 1995. (425)
37. (#121) Meakins, JL et al. Delayed-hypersensitivity—indicator of acquired failure of host defenses in sepsis and trauma. Ann Surg 1977. (425)
38. (#122) Pinsky, MR et al. Serum cytokine levels in human septic shock—relation to multiple system organ failure and mortality. Chest 1993. (422)
39. (#131) Danner, RL et al. Endotoxemia in human septic shock. Chest 1991. (409)
40. (#132) Eickhoff, TC et al. Neonatal sepsis + other infections due to group b beta-hemolytic streptococci. N Engl J Med 1964. (408)
41. (#134) Wheeler, AP et al. Treating patients with severe sepsis. N Engl J Med 1999. (405)
42. (#151) Sprung, CL et al. The effects of high-dose corticosteroids in patients with septic shock—a prospective, controlled-study. N Engl J Med 1984. (386)
43. (#153) Damas, P et al. Cytokine serum level during severe sepsis in human il-6 as a marker of severity. Ann Surg 1992. (382)
44. (#157) Askanazi, J et al. Influence of total parenteral-nutrition on fuel utilization in injury and sepsis. Ann Surg 1980. (377)
45. (#162) Docke, WD et al. Monocyte deactivation in septic patients: restoration by IFN-gamma treatment. Nature Med 1997. (372)
46. (#168) Petros, A et al. Effects of a nitric-oxide synthase inhibitor in humans with septic shock. Cardiovasc Res 1994. (366)
47. (#173) BrunBuisson, C et al. Incidence, risk-factors, and outcome of severe sepsis and septic shock in adults—a multicenter prospective-study in intensive-care units. JAMA 1995. (361)
48. (#174) Nava, E et al. Inhibition of nitric-oxide synthesis in septic shock—how much is beneficial. Lancet 1991. (361)
49. (#178) Szabo, C. The pathophysiological role of peroxynitrite in shock, inflammation, and ischemia-reperfusion injury. Shock 1996. (350)
50. (#180) Warren, BL et al. High-dose a randomized antithrombin III in severe sepsis—a randomized controlled trial. JAMA 2001. (348)
51. (#181) Bone, RC. Sir Isaac Newton, Sepsis, SIRS, and cars. Crit Care Med 1996. (348)
52. (#182) Schumer, W. Steroids in treatment of clinical septic shock. Ann Surg 1976. (348)
53. (#183) Damas, P et al. Tumor necrosis factor and interleukin-1 serum levels during severe sepsis in humans. Crit Care Med 1989. (346)
54. (#185) Cohen, J et al. The immunopathogenesis of sepsis. Nature 2002. (345)
55. (#187) Rush, BF et al. Endotoxemia and bacteremia during hemorrhagic-shock—the link between trauma and sepsis. Ann Surg 1988. (341)
56. (#190) Fourrier, F et al. Septic shock, multiple organ failure, and disseminated intravascular coagulation—compared patterns of antithrombin-III, protein-C, and protein-S deficiencies. Chest 1992. (339)
57. (#192) Munoz, C et al. Dysregulation of in vitro cytokine production by monocytes during sepsis. J Clin Invest 1991. (336)

Article's rankings listed by subgroup then by its position within the overall top 200 citations.

Results

The final database included 1187 articles (Fig. 1). The articles collated in this study came from almost 100 different journals with publication years ranging from 1905 (#149) to 2006 (average publication year = 1991). Surprisingly, of the top 1000 most cited critical care articles, 662 were not found using any of the basic critical care search terms—critical care, intensive care, ICU, or critically ill/critical illness. These articles were discovered only after searching within a specific critical care journal (n = 227) or by specific critical care topics (Table 1) (n = 373). In fact, only 4 of the top 1000 articles could be found using any 1 of the 4 basic critical care terms listed above. Of these search terms, the most productive was “intensive care” (25 of top 100 articles), then “critically ill” or “critical illness” (19/100), “ICU” (11/100), and finally “critical care” (5/100). Sepsis/systemic inflammatory response syndrome was the most common topic (193 articles), followed by acute lung injury and mechanical ventilation (177 articles). Overall, 41% of the most cited articles came from critical care journals as designated by the JCR. The most articles were from Critical Care Medicine (23%) followed by American Journal of Respiratory and Critical Care (7%), Journal of Trauma (5%), Intensive Care Medicine (4%), and Shock (1%). Among the non–critical care journals, the New England Journal of Medicine (11%) followed by JAMA (9%) provided the most articles.

Discussion

One method to quickly retrieve scholarly articles related to critical care medicine is by using readily available, Internet-based, bibliographic database search tools supported by most medical libraries. We have found, however, that commonly used basic search terms retrieve under only 50% of the most cited and therefore arguably most influential articles in critical care. Therefore, although searches may have become much faster and easier, they may not be sufficient to retrieve a comprehensive list of articles.

We found that there is no single term that exhaustively retrieves all critical care medicine articles. The best generic search term, “intensive care,” only retrieved 25% of what one would consider the 100 of the most influential articles in critical care. Furthermore, the search term “critical care” was linked to only 5 of the top 100 articles, although it is a common keyword used by many search engines. Not to mention the fact that it is in the title of 8 of the 18 “critical care” journals.

One goal of this study was to determine the most efficient search terms for future literature searches. The capriciousness of how efficient a search term is perhaps no better illustrated than with our experience retrieving articles on ICU mortality prediction models. That not one of the basic critical care search terms in the Science Citation Index was tagged to these most influential articles highlights some of the difficulties in finding the most cited studies in critical care. It is clear that even for other types of articles, the best yield of only 25% indicates the need for a comprehensive search strategy such as the one used in this study.

Although this is a problem for those looking for research articles, it can also be an issue for the way published research becomes cited by others. For example, articles without abstracts or very brief abstracts were often found only after using exact topic words found in the article's title or by starting a search from within a specific critical care journal itself. Our experience has been previously demonstrated where the choice of keywords and abstract construction significantly impacts the likelihood that an article will be found using modern electronic searches [17].

This study does highlight the importance of sepsis and infectious disease as well as respiratory physiology in critical care with more than half the articles falling into 1 of these 2 broad categories. On the other hand, the most cited article was related to physiologic predictors of morbidity and mortality and the next 3 articles were related to mental status.

Limitations

This study, like previous ones focusing on citations, has several limitations. Perhaps the most relevant is the debate as to what constitutes the most influential articles within a professional discipline [1], [4], [18]. Citation data found in all bibliographic databases rely not only on correctly acquiring cited references but also on the assumption that a primary study cites other references that are most relevant to the published article. It is likely that this study captured many of the classic articles in critical care as each article in our list of the top 200 articles was cited at least 300 times. Considering that 46% of articles published in medicine are never even cited, this is a remarkable observation [19]. Certainly, these articles deserve attention and must have made an indelible impact. However, this is not always the case. Authors may preferentially cite their or their colleagues' previous works both because of familiarity or to increase the citation of that article. Recent scholarship on social networks, especially in the age of accelerated knowledge transmission, suggests that there may be an enhanced level of “connectedness” related to these networks [20], [21]. It is not known whether, or how, this connectedness may be influencing the dissemination and therefore the pattern of references within the critical care community or among specific critical care topics. However, it is possible that the pattern of some citations may reflect the influence of an individual, as some have referred to as the “ceremonial citation,” rather than the specific findings within an article [16], [22], [23]. In addition, authors are more likely to cite articles of their own language [8], [24] or articles that come from a highly cited journal. Not surprisingly, Baltussen found that among the most cited 45 articles, most of them came from non–critical care-focused journals [16]. Nevertheless, using citations as a proxy for influence and importance has both face validity and is supported by empiric data demonstrating that citation analyses correlate with articles with the highest quality hierarchies of evidence and research design [25].

Most importantly, the use of citations is also confounded by the effect of time from the year of publication [26] with peak of citations differing for different journals and areas of research. Once citations do peak, they eventually become part of common knowledge and are no longer cited. Previous studies suggest that articles peak 7 to 10 years after publication [15], [26]. It has been suggested that “classic” articles are relevant to only a few decades and many important articles are lost to the passage of time [15]. These findings may be supported by the fact that among the top 200 articles in this study, 158 were published during or after 2000. It is probably too early to tell how electronic databases will change the epidemiology of citation classics because it is easier to retrieve and perhaps cite influential articles than before the age of the Internet.

In conclusion, although Web-based search engines can produce lists of references within seconds to minutes, the utility of these searches can be quite limited as a complete end effective literature search remains an art. Periodic reviews of the literature may prove helpful to trainees mastering the most influential literature of our field as well as more established professionals searching for starting points for new investigations.