General internal medicine and family medicine journals: Comparative study of published articles using bibliometric data.

Like research in general internal medicine, family medicine research can play an important role in improving medical knowledge. We aimed to compare articles published in family medicine journals with articles published in general internal medicine journals. In this bibliometric study, we retrieved 658 randomly selected quantitative articles published in 2016 in 18 high impact factor journals of family medicine and general internal medicine. We extracted the following data: author (gender, number of publications, and place of residence of the first author), paper (number of participants, study design) and journal characteristics (journal discipline, 2015 impact factor). We compared the two groups of articles, using multivariate logistic regressions adjusted for impact factor and intra-cluster correlations. The first author of the articles published in family medicine journals, compared to general internal medicine journals, was more often a woman (OR 2.8 [95%CI 1.8-4.4], P-value < .001), living in the Western world (OR 14.4 [95%CI 6.0-34.4], P-value < .001), and a less experienced researcher (<5 vs >15 publications: OR 2.4 [95%CI 1.5-4.0], P-value .01). In addition, these studies generally included more participants (>1000 vs <100: OR 3.5 [95%CI 1.4-8.6], P-value .02). There was no statistically significant difference in the study design between the two groups of articles (P-value .25). Despite some differences between the two groups of articles, studies published in family medicine journals do not appear to be any less ambitious in terms of study design and sample size than those published in general internal medicine journals.

Introduction

Publishing scientific articles is not an easy task but it is crucial because it allows the dissemination of new knowledge[ and plays an important role in the recognition and career advancement of researchers.[

Family medicine is a relatively new medical discipline.[ With the creation of a large number of family medicine academies and increased support for research by national medical societies, the number of publications by family medicine researchers has increased worldwide.[ It is encouraging to observe this evolution, as scholarly activity can be considered a key factor in the successful development of this medical discipline.[ Building research capacity is essential to the ability of family medicine to provide better patient care; indeed, although family physicians benefit from research in other disciplines, they often need answers to questions from studies involving family medicine patient populations.[

However, the number of publications per se does not necessarily reflect the quality and/or scientific importance of research. In theory, the quality of research could be roughly measured by the number of citations and the journal impact factor, and its scientific importance by its relevance to implementation by clinicians and policy makers.[ Unfortunately, these indicators are imperfect (number of citations and journal impact factors) or difficult to measure (relevance to implementation).

Another way to assess the quality and scientific importance of research is to use the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach.[ Many clinical practice recommendations are based on summaries of evidence, as determined by this tool. Several factors determine the quality of evidence, including study design (the quality of evidence from systematic reviews and experiments is usually higher than that from observational studies), study quality (there is a risk of bias if the design or conduct of a study is flawed) and precision of data (imprecision may occur, especially in the case of small studies).[ Moreover, it has been shown that the quality of research is generally influenced by researchers’ experience,[ and citations per article are consistently higher for more productive (vs less productive) researchers.[

General internal medicine is a medical discipline recognized for the quality of its research. Therefore, using the above indicators (study design, sample size, and number of publications) to compare family medicine with general internal medicine can help to assess the current state of family medicine research.

In this study, we aimed to compare articles published in family medicine journals with articles published in general internal medicine journals in terms of author and paper characteristics. We planned to restrict the analysis to quantitative studies in order to better compare articles published in the two disciplines, for example in terms of sample size. Since most family medicine specialists have little time for research,[ we hypothesize that articles published in family medicine journals are generally written by less experienced researchers and come from less ambitious studies (i.e., studies with smaller sample size and/or weaker study design).

Methods

Identification of studies and data collection (extraction of the data)

As planned, this study was part of a larger project designed to assess the publication time of articles submitted to general internal medicine and family medicine journals. We described in detail the selection process of the articles in another paper.[ Only the main points are summarized below.

We randomly selected 781 articles published in 2016 in the 9 highest impact factor journals of general internal medicine and the 9 highest impact factor journals of family medicine. For the purpose of our initial study, we limited the selection of journals to those that reported submission and publication dates. After having excluded qualitative studies, we retrieved 658 articles for this study. We divided journals by specialty (general internal medicine vs family medicine) according to the Journal Citation Reports (JCR) classification. This is a well-known classification used by researchers worldwide, for example in bibliometric studies.[

We extracted the following data from the articles:

author characteristics: gender, number of publications (using Web of Science [v 5.25.1]) and place of residence of the first author;

paper characteristics: submission and publication dates (these two variables were not used in this study), number of participants in the study and study design;

journal characteristics: journal discipline (general internal medicine or family medicine), 2015 impact factor and number of articles published in 2016.

As planned, the number of publications was classified as under 5, 5 to 15, and over 15, the number of participants as <100, 100 to 1000, and over 1000, and the study design as systematic review, experiment, cohort study, cross-sectional study, case–control study, and other design.

By definition, a researcher was considered more experienced than another if she/he published more articles, and a study was considered more ambitious than another if it included more participants and/or if its design was stronger (systematic review or experiment).

Statistical analyses

We summarized numerical data using medians and interquartile ranges (IQRs) and categorical data using frequency tables. We compared articles published in general internal medicine journals with articles published in family medicine journals using logistic regressions adjusted for intra-cluster correlations.[ As there were meaningful differences in the impact factor of general internal medicine and family medicine journals, we also adjusted the findings for the journal impact factor. Since there were two Asian journals among the general internal medicine journals but none among the family medicine journals, we excluded articles published in these two Asian journals for the comparative analysis of the variable “place of residence.”

Statistical significance was set at a two-sided P-value of ≤.05. All analyses were carried out with STATA version 12.0.

Results

A total of 658 quantitative articles published in 2016 were included in the study (393 in general internal medicine and 265 in family medicine), representing 42% of the articles published in 2016 by the 18 journals included in the study (Table 1). The number of articles selected in the two disciplines was not the same because qualitative studies were excluded from this study. Impact factors ranged from 19.7 (British Medical Journal) to 1.6 (American Journal of the Medical Sciences) for general internal medicine journals, and from 5.1 (Annals of Family Medicine) to 1.1 (Primary Health Care Research and Development) for family medicine journals.

Table 1

List of the 18 journals included in the study, stratified by discipline (general internal medicine or family medicine) and sorted by 2015 impact factor (N = 658 quantitative articles).

Table 2 shows, for each journal included in our study, the median number of publications of the first authors, the median number of study participants, and the proportion of systematic reviews and experiments. For each of these three variables, the figures were generally higher for journals with higher impact factors and lower for journals with lower impact factors. In summary, the median number of publications ranged from 22 (British Medical Journal) to 5 (American Journal of the Medical Sciences) for general internal medicine journals, and from 9 (Annals of Family Medicine) to 3 (Atencion Primaria) for family medicine journals, the median number of study participants from 20,312 (British Medical Journal) to 153 (Journal of the Formosan Medical Association) and from 6017 (British Journal of General Practice) to 156 (Australian Journal of Primary Health), and the proportion of systematic reviews and experiments from 19% (British Medical Journal) to 2% (American Journal of the Medical Sciences) and from 10% (Annals of Family Medicine) to 2% (Primary Health Care Research and Development).

Table 2

Median number of publications of the first authors, median number of study participants and proportion of systematic reviews and experiments for each journal included in our study (N = 18 journals and 658 quantitative articles).

Tables 3 and 4 compare the two groups of articles in terms of first author and paper characteristics (Table 3 using frequencies and medians, Table 4 using odds ratios). In multivariate analysis, the first author of the articles published in family medicine journals, compared to general internal medicine journals, was more often a woman (54.9% vs 45.1%, OR 2.8 [95%CI 1.8–4.4], P-value <.001), living in the Western world (55.3% vs 44.7%, OR 14.4 [95%CI 6.0–34.4], P-value < .001), and a less experienced researcher (median number of publications: 5 (IQR 13) vs 10 (IQR 25); <5 vs >15 publications: OR 2.4 [95%CI 1.5–4.0], P-value .01). In addition, these articles generally included more participants (median 490 (IQR 3149) vs 359 (IQR 2943); >1000 vs <100: OR 3.5 [95%CI 1.4–8.6], P-value .02). There was no statistically significant difference in the study design (P-value .25). Only a few researchers in Asia have published in family medicine journals (12 articles in family medicine journals compared to 88 in general internal medicine journals).

Table 3

First author and paper characteristics of 658 quantitative articles, stratified by journal discipline (family medicine or general internal medicine).

Table 4

Unadjusted and adjusted associations between publication in family medicine journals, and first author and paper characteristics of 658 quantitative articles.

Discussion

Summary

We found that the first author of the articles published in family medicine journals was more often a woman, living in the Western world and a less experienced researcher; in addition, these articles generally included more participants than those published in general internal medicine journals. Finally, there was no statistically significant difference between the two groups of articles in terms of study design.

Strengths and limitations

First, several high impact factor journals could not be included in our study because they did not provide submission and/or publication dates, which limits the generalizability of our results. Although the non-inclusion of “important” journals can be considered a weakness of our study, there is no reason to imagine that providing or not providing the submission and publication dates of published articles could be associated with our outcome. Therefore, the inclusion of journals that do not provide these data should not affect our results. Second, data extraction was limited to a single year (2016). However, some journals may have changed their editorial strategy since 2016, which could lead to changes in the profile of published articles. Third, in this study, we did not measure the methodological quality, scientific importance or novelty of the published studies. These indicators could be used in future comparative studies. Finally, this study compared articles published in general internal medicine journals and articles published in family medicine journals. However, family medicine researchers also submit their research to general internal medicine journals, for example because they may feel that family medicine journals have less prestige and/or that their institution gives less weight to articles published in these journals.[ These articles would have been considered in our study as general internal medicine research. We cannot assess the extent of this bias, but the number of family medicine articles published in general internal medicine journals appears to be relatively small (<5%).[

Comparison with existing literature

Although researchers who published in family medicine journals were generally less productive than those who published in general internal medicine journals (their median number of publications was half as high: 5 vs 10), their published articles did not appear to be less ambitious in terms of study design and sample size than those published in general internal medicine journals. Indeed, there was no significant difference in study design between the two groups of articles (in particular, the number of systematic reviews and trials published in family medicine journals did not differ significantly from those published in general internal medicine journals) and the median number of participants was even slightly higher.

However, we included only quantitative studies in our analyses in order to accurately compare the two groups of articles. If we had also included qualitative studies, we would have observed different results for study design and sample size for at least two reasons. Qualitative studies are generally more common in family medicine than in general internal medicine (in our study, the number of qualitative studies we excluded was 118 in family medicine journals but only 5 in general internal medicine journals). Qualitative studies generally include far fewer participants than quantitative studies. In our study, if we had included all qualitative studies, the median number of participants would have been only 269 for family medicine journals (instead of 490) while the figure would not have changed much for general internal medicine (358 instead of 359).

In summary, as hypothesized, articles published in family medicine journals were generally written by less experienced authors. However, these studies did not appear to be any less ambitious than those published in general internal medicine. These findings are encouraging and confirm that family medicine research has the potential to play an important role in contributing to the improvement of medical knowledge.

We found that female researchers were more likely to publish in family medicine journals than their male counterparts. A number of studies suggest that there are meaningful differences in the way female and male doctors practice medicine.[ For example, it has been shown that, compared to their male counterparts, female family medicine physicians tend to spend more time listening to their patients.[ In a similar way, gender differences could also occur in research and explain at least in part where researchers decide to submit their research. For example, some studies showed that female and male researchers differ in their patterns and strategies for research collaboration.[

More surprisingly, we found that Asian researchers were much less likely to publish in family medicine journals. These differences could be explained by the fact that there are generally fewer academic departments of family medicine in Asia than in the Western world or that they do not necessarily include a research component in their training program.[ Alternatively, family medicine research may not be a top priority for a number of Asian countries compared to research in other medical disciplines; as a result, researchers may face serious difficulties in terms of financial resources as well as training, support and supervision.[ Finally, Asian family medicine researchers may be more inclined to submit their research to journals that are not family medicine journals,[ which may lead to reduced academic visibility. Our results comparing researchers’ place of residence for the two groups of journals are only valid for non-Asian journals (we excluded the two Asian journals from the analysis, as they were both general internal medicine journals). Our findings could have been different if we had included for example only Asian journals in our study.

We also showed that researchers in Oceania tended to publish more often in family medicine journals than in general internal medicine journals, whereas the opposite was true for researchers in South America and Africa, but these findings are less obvious to analyse due to relatively small numbers of observations.

Implications for research

First, academic research is an important factor in the successful development of a medical discipline. In this sense, the continued growth in the number of publications by family medicine researchers is encouraging. However, many of the questions in the discipline remain unanswered and it is not known whether family medicine research meets the information needs of family physicians.[

Second, it is of course important to maintain both the quantity and quality of research. Therefore, further studies are needed to assess the quality of family medicine research by developing quantitative methods to measure this indicator.

Finally, our finding of a small number of studies from Asian researchers in family medicine journals deserves more attention: does it reflect lower productivity in Asia or the fact that Asian family medicine researchers are more likely to submit their research in journals that are not considered family medicine journals? In any case, this finding must be addressed, as the continent's contribution to research has increased significantly in recent decades and family medicine must contribute to this development.

Conclusion

Despite some differences between the two groups of articles, studies published in family medicine journals do not appear to be any less ambitious in terms of study design and sample size than those published in general internal medicine, a medical discipline recognized for the quality of its research. However, further research is needed to confirm our results and explore the reasons of these differences. In particular, the finding that only a few researchers in Asia have published in family medicine journals should be addressed.

Acknowledgments

I would like to warmly thank Hubert Maisonneuve (HM) and Jean-Pascal Fournier (JPF), co-investigators of the project, Claire Ragot (CR) and Pierre-Henri Gorioux (PHG), doctoral students, and Amir Moussa, administrative assistant, for their precious contribution to the study. I would also like to thank François Herrmann for his statistical support. These persons give permission to be named. This project was supported by institutional funding from the Faculty of Medicine, University of Geneva.

Author contributions

PS, HM, and JPF were involved in the conception of the study. PS was involved in data interpretation, data analysis, and data interpretation. PS drafted the manuscript. PS can be contacted for access to the dataset underlying the current analysis.