An Altmetrics Analysis of the Articles Published in the Medical Journal of the Islamic Republic of Iran (1987-2020).

Background: As a newly-emerged metric for evaluating scientific research, altmetrics captures the online activity regarding individual scientific items and is increasingly used in disseminating scientific information in a real-time span. This study aimed to conduct an altmetrics analysis of articles published in the Medical Journal of the Islamic Republic of Iran (MJIRI) during 1987-2020.
Methods: Using the archives of MJIRI's articles (during 1987-2020) and the four databases of Google Scholar, Scopus, Dimensions, and Altmetrics needed data on received citations as well as altmetric indicators and altmetric attention scores of these articles were extracted manually in December 2020. Data analysis was done in Excell and SPSS-25.
Results: Only 1274 MJIRI articles (about 51%) were present in the Altmetric Institute and had an altmetric attention score. Only 109 papers (8.5%) were shared at least once on online social media. Twitter was the most frequent social medium used for sharing the articles (n=91, 7.14%). These articles were twitted 171 times in total and the mean rate of twitting them was 1.88 per paper. Users from 21 countries in the world tweeted the articles. The top three twitting courtiers/regions were the United States (n=47), the United Kingdom (n=14) and India (n=3), respectively. Regarding twitters' membership status, the top three ranks were dedicated to the members of the public with 137 twits, practitioners (doctors and other healthcare professionals) with 18 twits and scientists with 16 twits. In Mendeley, the top three ranks were dedicated to master students (n=284), bachelor students (n=240) and Ph.D. students (n=155), respectively. The top three disciplines in this regard were medicine and dentistry (n=335), nursing and health profession (n=190), and biochemistry, genetics and molecular biology (n=68). Most of the highly-mentioned articles were review papers. The relationship between the altmetric attention score and citation performance of MJIR articles was not significant (p>0.05).
Conclusion: This study is one of the first studies to investigate the altmetrics indicators of articles published in an Iranian high-prestigious internationally-wide medical journal. Using social media tools can certainly promote medical scholars' scientific interactions and make added value for research published in medical journals. Editorial boards, including that of MJIRI can use altmetrics for detecting research trends and publishing approaches and consequently increased citation counts and research impact.

What is “already known” in this topic

Many altmetrics studies on different medical fields and subfields revealed the importance of using social media and tools in promoting scientific research.

What this article adds

This study highlights the necessity of conducting an altmetrics study of a medical journal in increasing its impact and widening its scope and can be a sample altmetrics analysis for similar studies.

Introduction

New types of scientific products beyond traditional information resources are thriving, and some data sources other than citation data are increasingly accessible for research evaluation. As a result, stakeholders are seeking evidence of scientific impact beyond traditional bibliometric measures, such as those of altmetrics (1). Coined in 2010 (2), "altmetrics" refers to all measurements of research output not included in the traditional bibliometric indicators. Scholars' being visible on the social web lies beyond citation and manifests their social impact. The concept of “altmetrics” is a new metric that seeks to capture the online activity regarding individual articles as a measure of research impact (3). It provides a summary of how research is shared and discussed online by various readers, including the general public, in real time (4).

Interest in altmetrics has increased rapidly since the concept was first introduced for its potential to address the main limitations of conventional citation-based counts (5). It measures attention a paper receives among widely diverse audiences, thereby potentially better indicating its overall societal reach as well as public awareness (6). Altmetrics chooses online tools for measuring research impact, including among others, social media, online reference managers and news sources. It quantifies the dissemination of a paper using its views on a number of different websites and blogs as well as existed mentions and posts on social media networks, such as Facebook, Twitter, etc. (7).

Various stakeholder groups in the field of medicine use a range of social media platforms to share research output and engage in dialog on their individual research interests (7). In addition, the social media presence of medicine and its related fields has received much attention from researchers (8). In this regard, altmetrics is a measure of importance and is increasingly recognized as a tool to measure the real-time reach and influence of an academic article, especially in medical fields that need immediate reactions and reaching in real-time (9). Therefore, it is needed that authors and editors of medical journals pay more attention to altmetrics as a new and fast tool to measure scholarly social impact in a wider range.

To date, a limited number of studies in the medical areas have formally evaluated research trends regarding their altmetrics (e.g., in emergency medicine (10), pharmacoepidemiologic research (11), and endodontology (12)). Top altmetric articles in some medical fields have been investigated (e.g in neurosurgery (13), in neuro-intervention (14), in neuroimaging (15), and in medical imaging (16)). Some correlated altmetric indicators and bibliometric ones in medical fields (e.g., in cardiovascular research (17), in plastic surgery (18), in general surgery (19), in anesthesiology (20).

Since the concept of altmetrics was introduced, it has captured the attention of higher impact scientific journals, and they have started to report altmetric indicators alongside their usual citation-based ones. As noted by Wong et al. (2018), social media is often underutilized and remains an untapped resource by many journals to increase their readership and impact and disseminate research output. Journals with social media profiles had significantly higher academic metrics, and among journals with such profiles, increased online activity was a predictor for their academic influence (21). Besides, Hughey, Hughey and Morphy (2017) showed that trauma and orthopedic surgery journals with dedicated Twitter profiles have a higher impact (22). In an altmetric analysis of Archivos of Bronconeumologia (AB), as a highly-impact worldwide respiratory journal, López-Padilla et al. (2020) found such an analysis helps in determining journals' research trends (23). In some studies, the altmetric indicators of medical journals in group aggregated levels have been investigated (e.g., Salajegheh et al. 2019 for top Scopus-indexed medical journals and Serati et al., 2020 for Iranian medical Journals) (24, 25).

The Medical Journal of the Islamic Republic of Iran (MJIRI), as an editorially independent peer-reviewed online open-access journal, is owned and published by the Iran University of Medical Sciences. As a highly-prestigious medical journal, it is currently indexed and abstracted by PubMed, PubMed Central, Scopus, Google Scholar and some other databases (26). Based on Scopus, the journal's CiteScore, SNIP and SJR are .9, .699 and .268, respectively (27). Despite a bibliometric study on analyzing the 32-year activity of the journal from 1987 to 2018 (28), there is no altmetric study on it.

It is needed that the journal is evaluated from an altmetric perspective to know the status of its real-time readership, impact, dissemination, and attention within social media and among the research communities present in online tools. This can help the journal in directing a planned online activity for reaching and developing its interested fields as well as increasing its research influence worldwide. Therefore, we aimed to conduct an altmetrics analysis of articles published in this known Iranian journal during 1987-2020.

Methods

We used an altmetric approach in this cross-sectional study. As a branch of webometrics, altmetrics is a new evaluative method for scientific publications in Web 2.0. As a new complementary tool for other common measures of scientific influence, altmetrics measures the scientific influence of works in social media by using and analyzing the scientific data on the social web (29).

We collected the needed data in January 2021 from Scopus by applying the search strategy as follows and extracted the bibliographic information of the MJIRI in CSV format:

SRCTITLE (medical AND journal AND of AND the AND Islamic AND republic AND of AND Iran) AND PUBYEAR < 2021

As some papers of MJIRI were not indexed in Scopus, we used the archive of the journal to complement its bibliographic information. In total, 2503 papers were published in the journal from its inception in 1987 to 2020.

The altmetric indicators of these papers were achieved from Altmetric Institute (accessed via https://www.altmtric.com). It is notable that search in Altmetric is fee-based, and its information relates to publications with Digital Object Identifier (DOI), PubMed Record ID, ArXiv ID and/or other standardized IDs.

For extracting Altmetric data, the Digital Object Identifier (DOI) of each document was entered into Altmetric Explorer and the needed information such as the presence score in certain social media websites and tools were provided. The characteristics of tweeters in Twitter and readers in Mendeley were collected and analyzed. Out of a total of 2503 published articles in MJIRI, only 1274 (about 51%) were present in the Altmetric Institute and had an altmetric attention score. Therefore, these articles were included in the study. It is noted that the score for each paper is represented graphically as an altmetric donut or badge where the dissemination of a certain document in each covered medium is represented with a certain color (e.g., light blue for Twitter). The papers with an altmetric attention score were searched in Google Scholar, Scopus and Dimensions for their citation counts, which are needed for conducting some altmetric analyses. Fig. 1 shows the altmetrics indicators of one of MJIRI papers in the Altmetric Institute.

Fig. 1

Altmetric Institute uses link recognition procedures and text-mining techniques for data collection (30). It dedicates a different score for any discussion in blogs and news, any mention in Facebook and Twitter, any citation in Wikipedia, and any readership and bookmarking in Connotea, CiteUlike, and Mendeley. The Sum of these scores results in a total "Altmetric Attention Score". It represents a weighted count of the amount of attention for research output from these social media (31). In other words, the score shows the quantity and quality of attention a document paid in a variety of social media (32).

This institute was established by Euan Adie in 2011, and its first standalone version was released in February 2012. It aims at tracking and analyzing the online activity around scholarly research output (33).

The majority of altmetrics studies were done by using the data of Altmetric Institute (such as (20, 34) in medical sciences) as it is a well-known worldwide altmetrics data provider and measures the rate of the presence of a scientific paper in different social media, such as blogs, news, CiteUlike, Connotea, Facebook, Mendeley, Wikipedia and Twitter, and reference management tools (30). It provides additional helpful statistical data on contributors and readers, as well as their locations and their professional statuses and interested disciplines. Most highly-prestigious scientific journals, known funders, and leading research institutes are of its users.

After fully extracting data on Altmetric Attention Scores, altmetric indicators, as well as Dimensions, Google Scholar, and Scopus citations of the studied articles, data analysis was done in Excell and SPSS-25. Data were analyzed in descriptive and inferential statistics and the Spearman correlation coefficient test was used for testing the possible correlation between articles' altmetric attention scores and their citation counts.

Altmetric indicators of one of MJIRI papers in the Altmetric Institute

Results

Out of all 1274 JMIRI articles included in the Altmetric Institute, 109 papers (8.5%) were shared at least once on online social media. Table 1. shows the presence rate of these papers in different social media websites and tools.

Table 1.

The most used altmetrics sources of MJIRI articles

Sources of Attention	Studied Articles	Articles with this Altmetrics Source (%)	Total Altmetrics Events	Mean Events per Article	Highest Events	Rank
Twitter	1274	91 (7.14)	171	1.88	13	1
Mendeley	1274	87 (6.83)	1597	18.36	140	2
News Outlets	1274	15 (1.18)	16	1.07	2	3
Facebook	1274	9 (0.71)	9	1.00	1	4
Blogs	1274	8 (0.63)	8	1.00	1	5
Video Uploader	1274	4 (0.31)	4	1.00	1	6
Stack Overflow	1274	1 (0.08)	1	1.00	1	7
Policy Source	1274	1 (0.08)	1	1.00	1	7
Google+	1274	1 (0.08)	1	1.00	1	7

As Table 1. depicts, Twitter was the most frequent social medium used for sharing the articles (with 91 articles, 7.14%). These articles were twitted 171 times in total, and the mean rate of twitting these articles was 1.88 per paper. The most highly frequent article with 13 twits was entitled as "Comparison of the effects of Lactobacillus Plantarum versus imipenem on infected burn wound healing", authored by Somayeh Soleymanzadeh-Moghadam in 2020. The second-ranked tool was Mendeley in which 87 articles (6.83%) were read at least once. The most highly-read article with 140 readers was entitled "Telemedicine: A systematic review of economic evaluations", authored by Bahram Delgoshaei in 2017. The third top sharing tool was News Outlets with 15 articles (1.18%) having a total of 16 new stories in different outlets. The mean rate of stories per article was 1.07. The top article with 2 news items was written by Iman Keliddar, Ali Mohammad Mosadeghrad and Mehdi Jafari-Sirizi in 2017 under the title "Rationing in health systems: A critical review".

In total, users from 21 countries in the world tweeted the articles 171 times on Twitter. As Fig. 2 depicts, the top three twitting courtiers/regions were the United States (n=47), the United Kingdom (n=14) and India (n=3), respectively. The twitting countries in 86 twits were unknown.

Fig. 2

Fig. 3 shows twitters' membership statuses and the frequency percentages of their contribution. The top three ranks were dedicated to the members of the public with 137 twits, practitioners (doctors and other healthcare professionals) with 18 twits, and scientists with 16 twits, respectively.

Fig. 3

The professional status of readers of MJIRI articles in Mendeley has been shown in Fig. 4.. Top three ranks were dedicated to master students (n=284), bachelor students (n=240) and PhD students (n=155), respectively. The professional status of 383 and 220 readers was unspecified and non-assigned, respectively.

Fig. 4.

Readers of MJIRI articles in Mendeley were from different disciplines (Fig. 5.). The top three disciplines in this regard were medicine and dentistry (n=335), nursing and health profession (n=190), and biochemistry, genetics, and molecular biology (n=68).

Fig. 5.

Table 2 shows the top 20 articles with high altmetric scores. An article entitled "Rationing in health systems: A critical review" by Iman Keliddar et al. (2017) was the first-ranked one with an altmetric attention score of 26. The article was shared in Twitter (n=6), blogs (n=1), facebook pages (n=1), news outlets (n=2) and Mendeley (n=52). The second rank belonged to the article by Naderyan Fe'li, Shadi (2019) under the title " Metabolic syndrome and 10-year risk of cardiovascular events among schizophrenia patients treated with antipsychotics" with an altmetric score of 11.

It was shared 2 times on Twitter, 1 time in news outlets and 21 times in Mendeley. The third rank belonged to the three articles in common.

Regarding the citation status of the top 20 highly-cited JMIRI articles in Google Scholar, Scopus and Dimensions, the findings showed that the first-ranked article was "epidemiological and clinical characteristics of scorpions in Shiraz (2012-2016); development of a clinical severity grading for Iranian scorpion envenomation", written by Hossein Sanaei-Zadeh and et al. (2017) with 31, 23 and 12 received citations in Google Scholar, Scopus, and Dimensions, respectively. The second rank belonged to the article entitled "Rationing in health systems: A critical review" by Iman Keliddar et al. (2017), which had the highest Altmetric Score and received 13, 7 and 8 citations in Google Scholar, Scopus and Dimensions, respectively. An article by Ghobad Moradi under the title "Catastrophic health expenditure among households with members with special diseases: A case study in Kurdistan" ranked third with receiving 8, 2 and 7 citations in Google Scholar, Scopus and Dimensions, respectively.

For testing the possible relationship between the altmetric attention score and citation performance of MJIRI articles, Spearman correlational test was used. Table 3. shows the correlation coefficients of these two variables. There was a positive significant correlation between the rate of citation counts in Google Scholar, Scopus, and Dimensions (p=.001). However, the relationship between the altmetric attention score and citation performance of MJIR articles was not significant (p>0.05). In other words, the presence of the articles in social media could not result in their received citations.

Table 3.

Correlation between altmetrics scores and citation counts of MJIRI articles

Altmetrics scores and citation counts		Dimensions	Scopus	Google Scholar	Altmetrics Score
Dimensions	Correlation Coefficient	1.000
	p-value	.
Scopus	Correlation Coefficient	0.845**	1.000
	p-value	0.001	.
Google Scholar	Correlation Coefficient	0.958**	0.848**	1.000
	p-value	0.001	0.001	.
Altmetric Attention Score	Correlation Coefficient	-0.056	0.066	0.088	1.000
	p-value	0.836	0.809	0.747	.

**Correlation is significant at the 0.01 level

Twitters' countries of origin twitting MJIRI articles

Membership statuses of twitters of MJIRI articles

Professional status of readers of MJIRI articles in Mendeley

Disciplines of readers of MJIRI articles in Mendeley

Discussion

This study is one of the first in investigating the altmetric indicators of articles published in an Iranian high-prestigious internationally-wide medical journal at its individual aggregate level. As a freely available web tool, data were extracted from Altmetric Institute that has been used in most of the altmetric studied in medicine, allowing a better discussion and helpful analysis of the achieved results.

The most significant finding of this study is that despite the fact that half of MJIRI articles were included in Altmetric Institute, only 8.5% of them were shared at least once in the studied social media. This is a low rate. As noted by Ortega (2017), this may be due to the authors' and readers' unfamiliarity with social media and unawareness of their contribution in disseminating scientific knowledge, as well as not including altmetric indicators in the website of the journal at hand. Some workshops on altmetrics can be of benefit in this regard (35). These results are in line with those found by (30, 36, 37).

Twitter and Mendeley were the most frequently used tools to share MJIRI articles. Twitters were mainly from the USA and the United Kingdom. Iranians were not present on Twitter due to its being filter in Iran, while most researchers worldwide use the tool for disseminating their research findings (38). In addition, Twitter is the most popular social media platform around the world that is widely used in the academic sphere. It has a relatively high coverage of scholarly output compared with the other available altmetric data sources (1). In addition, the presence of MJIRI articles in Mendeley highlights its importance in disseminating scientific output, as noted by (5, 30, 36, 39-41). One reason behind this may be its ability to represent some demographic statuses of readers (42). Besides, Mendeley is used chiefly by science, technology, engineering and mathematics researchers and learners (29).

We found that members of the public pay more attention to the articles. This shows the potentiality of web tools in disseminating scientific information to the public. Students in different academic levels read the studied MJIRI articles in Mendeley. Pooladian and Borrego (2017) found that the majority of readers in Mendeley were Ph.D. students and with the USA as the main country of origin (43).

As we found, review articles tend to be highly regarded than other types in social media. Therefore, altmetrics is able to highlight newly published research items, among which literature and systematic reviews have a higher frequency (44).

No significant correlation was found between altmetric scores and citation counts in this study. Such a relationship is controversial, as noted by Costas, Zahedi and Wouters (2015) (36). They claimed that the analysis of the relationships between altmetrics and citations confirms positive correlations but relatively weak ones, thus supporting the idea that altmetrics do not reflect the same concept of scientific influence as citation does, but can be helpful. One other reason for this finding is the study population in our study who were authors and researchers with low familiarity with and awareness of altmetric tools. It can be said that citation counts and altmetric scores represent unique aspects of evaluating scientific impact. However, many researchers found a positive relationship in some fields and disciplines (e.g., 32, 37, 41, 45-49), including medical journals indexed in Scopus (24). In addition, the fact that these metrics did not correlate with conventional metrics does not mean that they have no sufficient impact. It can be said that the results of the frequent studies measuring correlations between altmetrics and traditional citation indicators should be seen as a first introducing step in deep research on altmetrics. They provide a good opportunity to focus the field on more productive and profitable avenues. Low correlations point to altmetrics which may be of special interest for the broad impact measurement of research (50), i.e., impact on other areas of society than science. Tracing these trends needs further research.

Conclusion

It is needed that altmetric measures are used in research on medical fields and their related journals. Based on Altmetric Institute's data, the low presence of the articles in social media websites and tools can be a sign of low use of these facilities in scientific development. Social media is often underutilized and considered as an untapped resource by many journals in increasing their readership and disseminating research (22). In agreement with Melero (2015), we believe that using social media can certainly promote medical scholars' scientific interactions and make added value for research and innovation(51). Editorial boards can use altmetrics for detecting research trends and publishing approaches and consequent increased citation counts (52). Altmetric indicators can be applied as complementary to other traditional citation-based indicators to make journal articles more visible and broad dissemination in a real-time span.

The Medical Journal of the Islamic Republic of Iran (MJIRI), as a highly-prestigious medical publication, shared little in social media. However, members of the public are of main groups considering the journal in online tools. The journal should go beyond the boundaries of traditional citation-based metrics common in bibliometric studies while paying more attention to newly emerging scholarly tools in order to widen its reach and influence worldwide.

Similar studies can be done on other journals in the field using the data of other related institutions. The interest of the scientific community in these metrics is clearly growing, and this study can be a starting point for developing the application of these indicators in various journals of medical fields. It is needed that journals monitor this interesting direction closely in scientific production and distribution which has revolutionized the conventional approaches to scientific publications of all types.

Conflict of Interests

The authors declare that they have no competing interests.