Tackling the growth of the obesity literature: obesity evidence spreads across many journals.

OBJECTIVE: This study identified the journals with the highest yield of clinical obesity research articles and surveyed the scatter of such studies across journals. The study exemplifies an approach to establish a journal collection that is likely to contain most new knowledge about a field. DESIGN AND METHODS: All original studies that were cited in 40 systematic reviews about obesity topics ('included studies') were compiled and journal titles in which they were published were extracted. The journals were ranked by the number of included studies. The highest-yielding journals for clinical obesity and the scatter across journal titles were determined. A subset of these journals was created in MEDLINE (PubMed) to test search recall and precision for high-quality studies of obesity treatment (that is, articles that pass predetermined methodology criteria, including random allocation of participants to comparison groups, assessment of clinical outcomes, and at least 80% follow-up).
RESULTS: Articles in 252 journals were cited in the systematic reviews. The three highest-yielding journals specialized in obesity, but they published only 19.2% of the research, leaving 80.8% scattered across 249 non-obesity journals. The MEDLINE journal subset comprised 241 journals (11 journals were not indexed in MEDLINE) and included 82% of the clinical obesity research articles retrieved by a search for high-quality treatment studies ('recall' of 82%). Of the articles retrieved, 11% were about clinical obesity care ('precision' of 11%), compared with precision of 6% for obesity treatment studies in the full MEDLINE database.
CONCLUSION: Obesity journals captured only a small proportion of the literature on clinical obesity care. Those wishing to keep up in this field will need to develop more inclusive strategies than reading these specialty journals. A journal subset based on these findings may be useful when searching large electronic databases to increase search precision.

Introduction

The prevalence of overweight and obesity has increased at an alarming rate (1). The World Health Organization projects that by 2015, approximately 2.3 billion adults will be overweight and more than 700 million will be obese (2). As the prevalence of obesity increases, the demand for research in this area has also increased, creating more articles in this topic area for clinicians and others to consider. Clinicians are expected to keep up to date and apply the current best evidence from health care research to their clinical care (3) but are challenged by more than two million new research articles published each year in medical and scientific journals (4). Retrieving high-quality and methodologically sound research articles in a given topic area is difficult (5), making it more important than ever for clinicians to carefully select which journals to read (6). A study by Burke et al (7) found that at least half of the surveyed residents in physical medicine and rehabilitation looked for important articles within one of six journals; four of the six were specific to their discipline. A study by Weiner et al (8) revealed that journal selection should not be based on intuition alone.

Overweight and obesity can lead to serious health consequences and chronic diseases such as diabetes, cardiovascular disease, musculoskeletal disorders and some cancers (2). Thus, clinical research articles on obesity care could be scattered across various discipline-specific journals not routinely scanned by the clinician working with a patient who is overweight. One approach to this dilemma is to identify the top yielding journals on overweight and obesity care that produce methodologically sound research evidence (5) and determine the scattering of the relevant literature across journals, as this can provide the number and range of journals needed to capture the key literature in the field (9). If these journals can be identified, searching for the best current evidence in the large electronic databases, such as MEDLINE, could be enhanced by restricting the search to just these journals. To identify journal subsets in a particular disease content area a logical place to start would be systematic literature reviews because such reviews are based on a comprehensive search for all available evidence on a given clinical question and the conclusions of systematic reviews hold the highest position in the hierarchy of evidence for clinical decisions (10).

We sought to build on, and test, the work of Garg et al (11), who found that half of all renal practice evidence, was published in non-renal journals. To identify the highest yielding journals for the care of overweight and obese patients, we looked at studies cited in obesity-relevant systematic reviews. Our primary aim was to identify the top yielding journals in clinical obesity care and to determine the scatter of clinical studies across journals of other disciplines. A secondary aim was to test the retrieval performance (“recall and precision”) of a standard search filter for studies of treatment in the obesity journal subset, compared with the same search conducted in the full MEDLINE database.

Methods

To conduct this analysis, a representative sample of obesity relevant systematic reviews from four databases (MEDLINE, EMBASE, CINAHL, and PsycINFO) was compiled. The representative sample sought to simulate a simple user search in each of the four databases. The systematic reviews were identified by searching on the term “obesity” as a major heading and “exploding” obesity to include all its subtopics using the OVID interface. The strategy was further narrowed in OVID by using the “limits” function and choosing Reviews (specificity) from the Clinical Queries.

A research assistant (RA), trained and calibrated to identify obesity content, systematic reviews, and the clinical purpose of the article (e.g., treatment), reviewed the full-text of each retrieved review. The first step was to determine if the content was relevant to human obesity. The inclusion criteria for obesity relevancy were: review must be about human subjects, and the focus/aim/objective must be to study subjects who have metabolic syndrome or are described as adipose, obese or overweight, or who are being studied to see if they are at risk of becoming overweight. Overweight was defined as an adult body mass index (BMI) of ≥25 (12) and a child BMI percentile reported as ≥85th percentile for age and sex (13, 14).

During the second step, the RA independently determined if each review met specified methodological criteria for quality: a statement of the clinical topic, explicit statements of the inclusion and exclusion criteria applied for selecting primary studies for detailed review, and methods used to identify primary studies for inclusion in the review (i.e., which bibliographic databases were searched).

If the review was found to be relevant and systematic, during the third step the RA ascertained the purpose category of the article (i.e., the type of question posed in the systematic review): treatment, prognosis, diagnosis, etiology, or economics. Once this was determined, in order for the review to be included, at least one of the included primary studies had to meet the methodology criteria for the specified purpose category (e.g., for studies of treatment, at least one of the included primary studies had to have random allocation of participants to comparison groups; for additional criteria, see http://hiru.mcmaster.ca/hiru/InclusionCriteria.html). For each included study, the date of publication, language of publication, and journal title were extracted. Each study was also classified according to the type of question posed in the systematic review: treatment, diagnosis, prognosis, etiology, or economics. Studies published in the form of a conference proceeding, book, or thesis, were not considered further.

All identified journal titles were ranked according to the number of relevant articles published, from highest to lowest. Each journal was also classified according to its primary discipline using the U.S. National Library of Medicine’s MEDLINE journal subject terms as a guide (http://wwwcf.nlm.nih.gov/serials/journals/index.cfm).

In a post hoc extension of our study, we created a journal subset for MEDLINE (15), based on our findings and using the “My NCBI” function in PubMed. We tested recall (defined as the proportion of relevant citations retrieved) and precision (defined as the proportion of all citations retrieved that are relevant (i.e., articles retrieved that were about obesity and passed on methods) compared with searching the entire MEDLINE database for high quality studies of obesity treatments (passed on methods), using the sensitive (broad) Clinical Query for therapy (16). Of the 252 journals in our obesity subset, 11 were not indexed in PubMed; leaving 241 journals to enter in “My NCBI” (see supplemental data Table 1). The obesity journal subset was “ANDed” with the PubMed Clinical Query for therapy (broad (sensitive) search) and the search term of obesity. The results were further limited to being published in the last 6 months. Each retrieval was assessed for relevance to obesity and methodologic quality using the criteria for treatment studies that was applied when deriving the Clinical Query for treatment studies (i.e. random allocation of participants to comparison groups, assessment of clinical outcomes, and at least 80% follow-up) (16).

Results

A total of 40 obesity relevant systematic reviews in MEDLINE, EMBASE, CINAHL, and PsycINFO were compiled (See supplemental data Table 2). The systematic reviews examined a broad range of questions relevant to the study of obesity such as: intragastric balloon for the treatment of obesity, the risk of dementia with increased body weight, and routine monitoring of growth to screen for obesity in children. Reviews most often addressed questions of treatment (62.5%), followed by etiology (30%), diagnosis (5%), and diagnosis and treatment (2.5%). No reviews were retrieved on prognosis or economics.

1091 unique included studies were extracted from the 40 reviews. 16 included studies were removed from the database as they came from conference proceedings, dissertations, or unpublished work, leaving 1075 included studies. The 1075 studies were published across 252 journals (list of journals provided on request). Of the 1075 studies, 534 (50.5%) were published in the top 20 journals ranked by publication frequency, while the remaining 541 (49.5%) studies were published across another 232 journals.

A list of the top 20 journals that published studies relevant to obesity is presented in Table 1. The top 3 yielding journals were obesity specialty journals: International Journal of Obesity (2005) (9.6% of the studies), Obesity Surgery (5%), and Obesity Research (Silver Spring, Md.) (4.6%). When the disciplines of all 252 journals were considered, 3 journals or 1.2% were classified as obesity journals which contributed 19.2% of all the articles (Table 2). Other common specialties included: pediatrics (12.4% of articles), general and internal medicine (11.9%), psychiatry (11.2%), nutrition and dietetics (6.5%), endocrinology (diabetes) and metabolism (6.4%), and surgery (6.3%) (Table 2). 15 of the journals were published in languages other than English (5.9%) and 3 journals (1.2%) were published in both English and another language.

Table 1

Top 20 journals that published the highest number of studies relevant to clinical obesity care

Rank	Journal	PercentofRelevantStudies(n=1075)	CumulativePercent
1	International Journal of Obesity (2005)*	9.6	9.6
2	Obesity Surgery	5	14.6
3	Obesity (Silver Spring, Md.)**	4.6	19.2
4	The American Journal of Clinical Nutrition	2.6	21.8
5	American Journal of Epidemiology	2.5	24.3
6	Pediatrics	2.4	26.7
7	Journal of Consulting and Clinical Psychology	2.4	29.1
8	Annals of Surgery	2.3	31.4
9	JAMA : The Journal of the American Medical Association	2.2	33.6
10	Archives of Pediatrics & Adolescent Medicine	2.2	35.8
11	Behavior Therapy	2	37.8
12	Diabetes Care	1.9	39.7
13	The Journal of Pediatrics	1.9	41.6
14	Archives of Internal Medicine	1.5	43.1
15	The New England Journal of Medicine	1.4	44.5
16	Preventive Medicine	1.4	45.9
17	Journal of the American Dietetic Association	1.2	47.1
18	Archives of Disease in Childhood	1.2	48.3
19	Obstetrics and Gynecology	1.1	49.4
20	Cancer Causes & Control : CCC	1.1	50.5

International Journal of Obesity (2005) encompasses International Journal of Obesity (Lond), International Journal of Obesity, and International Journal of Obesity and Related Metabolic Disorders (reflecting journal name changes).

Obesity (Silver Spring, Md.) encompasses Obesity (Silver Spring) and Obesity Research (reflecting journal name changes).

Table 2

Obesity evidence was published across journals from many disciplines

Primary Discipline	No. ofJournals(n=252)	% ofJournals	No .ofArticles(n=1075)	% ofArticles
Obesity	3	1.2	207	19.2
Pediatrics	31	12.3	133	12.4
General and Internal Medicine	29	11.5	128	11.9
Psychiatry	32	12.7	121	11.2
Nutrition and Dietetics	13	5.2	70	6.5
Endocrinology (Diabetes) and Metabolism	17	6.7	69	6.4
Surgery	11	4.4	68	6.3
Oncology	8	3.2	41	3.8
Epidemiology	6	2.4	37	3.4
Obstetrics and Gynecology	9	3.6	35	3.2
Cardiovascular Diseases	11	4.4	25	2.3
Sports Medicine	5	2	15	1.4
Public Health	7	2.8	14	1.3
Gastroenterology	7	2.8	12	1.1

An additional 63 journals that contributed 100 articles were scattered across other disciplines

The search for high-quality obesity articles in the 241 journal subset in PubMed retrieved 415 results for a six month period. Of the 415 results, 46 passed criteria (11%) (i.e., were about obesity and passed on methods) and 369 failed (89%). The subset search had a sensitivity of 45% and a specificity of 70%. The same search was run in PubMed without the obesity journal subset. This search retrieved 915 articles of which 415 (45%) were duplicates from the subset search including the 46 pass articles. Overall, the full PubMed search retrieved 56 (6%) articles that passed criteria and 859 that failed (94%). The full PubMed search gleaned 10 additional pass articles that were missed in the obesity subset search because they were in journals not included in our journal subset. The full PubMed search had a sensitivity of 54% and a specificity of 30%. Thus the subset search had an approximate recall of 82% and a sensitivity that was 9% lower than the full PubMed search. However, the subset search had a specificity that was 40% higher than the full PubMed search and a precision that was 83% higher than the same search in PubMed (11% vs 6%).

Discussion

To our knowledge, this study represents the first formal assessment of the publication scatter of clinical obesity research articles in the medical literature. We used systematic reviews to identify studies for this analysis. This sampling approach was similar to that taken by Birken (5) and Garg (11) and reduces the chance that selection biases influenced the results, as systematic reviews intentionally include all relevant studies, albeit usually for a narrow clinical question, such as the effect of a given treatment. Accordingly, the numbers provided in this report should not be viewed as comprehensive for all obesity topics.

Our results suggest that reading International Journal of Obesity (2005), Obesity Surgery, and Obesity (Silver Spring, Md.) will capture less than 20% of the overweight and clinical obesity research articles. These findings differ from those for nephrology, reported by Garg and colleagues (11): half of all renal practice evidence was published in renal journals. For obesity, the scatter of publication outside obesity specialty journals is much wider. Some of the scatter may be influenced by the focus of the systematic review. Some reviews had patients with a comorbid condition (e.g., psychiatry comprised 12.7% of the journals and endocrinology [diabetes] and metabolism journals comprised 6.7% of the journals); some focused on a specific population (e.g., pediatric journals totalled 12.3% of the journals) and others focused on a specific intervention (e.g., nutrition and dietetics comprised 5.2% of the journals and surgery journals comprised 4.4% of the journals).

Our preliminary assessment of searching an obesity journal subset for six months showed good recall (82%) with an improvement in precision (from 6% to 11%), but these findings clearly need more extensive validation with longer time periods and for other types of studies than therapy.

One small limitation of creating journals subsets is that journals may discontinue and change names, leaving the researcher or the database provider to be diligent about keeping on top of these changes. A case in point is that Obesity Research became Obesity (Silver Spring, Md.) in 2005, so our journal subset had to be amended accordingly.