Technique Articles Are More Effective at Increasing Social Media Attention in Comparison With Original Research Articles: An Altmetrics-Based Analysis.

Purpose: To compare social media attention and citation rates between technique articles and matched original research articles (ORAs) regarding surgical procedures.
Methods: All technique articles published from August 2019 through July 2020 in the free, electronic versions of Arthroscopy Techniques and JBJS Essential Surgical Techniques were matched by topic to articles in the "Original Research" sections of Arthroscopy, Sports Medicine, and Rehabilitation and JBJS Open Access in a 4:1 ratio within this time frame. The primary outcome was the Altmetric Attention Score (AAS). Secondary outcomes included citations, bibliometrics, and social media metrics. Independent t tests were used to compare primary and secondary outcomes between technique articles and ORAs. A multivariate linear regression analysis was performed to determine the association between article type and social media attention while controlling for confounding bibliometric characteristics.
Results: A total of 285 matched research articles (n = 57, 20.0%) and technique articles (n = 228, 80.0%) were included. The mean AAS among all technique articles was 3.63 ± 10.08 (range, 0-96) whereas the mean AAS among all ORAs was 1.30 ± 3.98 (range, 0-25), representing a statistically significant difference (P = .016). The mean citation rate among all technique articles was not significantly different from that among ORAs (P = .73). Multivariate linear regression analysis showed a statistically significant positive association between AAS and article type, with an additional mean increase in the AAS of 2.91 (95% confidence interval, 0.04-5.77; P = .047) for every technique article compared with an ORA. Furthermore, a significant positive relation was noted between the article origin and the AAS, with an increase in the AAS of 3.00 (95% confidence interval, 0.82-5.17; P = .007) for every article published in North America compared with an article originating from another continent. Conclusions: Technique articles resulted in significantly greater AASs and social media attention in comparison with open-access ORAs on similar topics. Publications that described technical procedures in a technique journal and studies from North America were positively associated with greater AASs and greater numbers of citations received by articles. Clinical Relevance: An improved understanding of how much attention is given to technique articles versus matched ORAs by social media may influence the methods authors and journals use for distributing content. The present study suggests that one option to increase the amount of social media attention received for a particular study may be to utilize an accompanying surgical technique video or illustrations as these are easily shareable on social media and offer rapid dissemination of knowledge, similar to that of an infographic. However, physicians who view multimedia within technique articles should be encouraged to review the accompanying articles and the supporting original research as a primary source before making changes in their clinical practice.

The use of the internet has increased substantially since the start of the century, with social media becoming a critical platform for the dissemination of research among health care providers and institutions., Multiple studies have since investigated the impact of social media in the dissemination of critical research as measured by the Altmetric Attention Score (AAS), a tool that enumerates the impact of shared research on various media platforms via an aggregate score accrued by a particular article.,3, 4, 5, 6 Specifically within orthopaedic surgery, the positive association between the citation rate and the AAS has been established, inciting researchers to find innovative methods to share their findings, including graphical or video media that allow for rapid knowledge dissemination and acquisition. Two popular examples of these media include infographics and peer-reviewed surgical technique videos. However, despite the potential for easily digestible, visual representations of knowledge to be disseminated across online media platforms, the extent of their shareability and ultimate impact are not well understood.

Previous studies have shown the enhanced effectiveness of alternative methods of communicating research outcomes. When comparing infographics with original research articles (ORAs), Kunze et al. found a statistically significant difference in the AAS and overall social media attention favoring infographics. Therefore, visual representations of new orthopaedic research and knowledge appear to be well received and easily shareable by those who frequent online media. However, many orthopaedic journals have recently introduced companion technique journals to demonstrate established surgical concepts, as well as to present recently developed techniques. Although the sharing of technique videos on similar online mediums is common, their influence on readership and online sharing remains poorly understood.

An improved understanding of the online activity of technique articles and original research may influence the methods used by journals for distributing this content. The purpose of this study was to compare social media attention and citation rates between technique articles and matched ORAs regarding surgical procedures. We hypothesized that technique articles would have a significantly higher AAS on average in comparison to matched ORAs on the same subject.

Methods

Article Selection Criteria

This study was exempt from institutional review board approval. All articles from August 2019 through July 2020 from the electronic versions of Arthroscopy, Sports Medicine, and Rehabilitation (ASMAR), Arthroscopy Techniques, JBJS Open Access, and JBJS Essential Surgical Techniques were obtained. This time frame was chosen to allow maximal time for articles to be disseminated within the research community and accrual of social media attention and citation rates (the total number of citations that each article accrued since its publication) while maintaining their recency in accordance with prior Altmetric studies. In addition, no articles were published in ASMAR before August 2019, leaving the selected time frame as the best available time frame for analysis of citation rates and AASs as they pertains to our study. These journals were chosen for this analysis based on the following factors: (1) they had the highest companion-journal impact factor in the field of orthopaedic surgery during the queried time frame; (2) all articles within the journal were available to the public on an open-access basis; and (3) the AAS was reported for all articles. Furthermore, the open-access companions from the respective technique journals were chosen for this analysis to minimize the effects that formatting variation, visibility, and publication frequency, as well as differences in the degree and type of self-promotion among multiple journals reviewed by different editorial groups, that may have an impact on the primary outcome.

The citation rate and the AAS for each article were collected simultaneously in July 2021. Control articles in the “Original Research” sections of ASMAR and JBJS Open Access were selected and matched in a 1:4 ratio on the basis that they directly pertained to a surgical technique or its specific outcomes, as identified by the technique articles in Arthroscopy Techniques and JBJS Essential Surgical Techniques, representing an unbiased sample of articles for comparison. This ratio was chosen because it elicits the lowest bias in matching studies and maximizes statistical power as previously confirmed by applied statistical theory and it has been similarly deployed in prior Altmetric studies., We extracted all articles classified as basic science or computer model studies, prospective randomized controlled trials, prospective cohort studies, prospective case series, retrospective cohort or case-control studies, retrospective case series, case studies or case reports, cross-sectional studies, systematic reviews and/or meta-analyses, and current concept reviews. Articles classified as editorials, letters to the editor, editorial commentaries, and acknowledgments of reviewers were excluded from the analysis.

Primary and Secondary Outcomes

The primary outcomes of interest were the AAS, calculated through weighted social media attention scores that a control or technique article receives, and the cumulative article citation rate. The AAS includes activity from multiple media platforms, including but not limited to Facebook, Twitter, Reddit, blogs, news articles, and Mendeley. The overall score—calculated and updated in real time to provide an accurate estimate of the impact of an individual article’s dissemination—is determined by the attention received from each of the considered aforementioned social media outlets and their relative weights (Table 1). There is no upper limit to the score because each article can continue to accumulate attention over time, with the lowest score being 0 (signifying no social media mentions or attention as calculated by the metric).

Table 1

Altmetric Attention Score Calculation,

Social Media Outlet	Weight
News	8.0
Blog	5.0
Policy document	3.0
Patent	3.0
Wikipedia	3.0
Twitter	1.0
Peer review (Publons or PubPeer)	1.0
Google+∗	1.0
F1000	1.0
Weibo†	1.0
Syllabus (Open Syllabus)	1.0
LinkedIn	0.5
Reddit	0.25
Facebook	0.25
YouTube	0.25
Pinterest	0.25
Q&A (Stack Overflow)	0.25
Mendeley Readers	0 (tracked but not considered in calculation)

NOTE. The total score is calculated as a function of the listed social media weights.

Q&A, question and answer.

Not trackable since 2019 but historical data were kept.

Not trackable since 2015 but historical data were kept.

Data Collection

Collected variables, including the AASs and citation rates, were obtained using the Altmetric Bookmarklet, which reports the aggregate AAS along with relevant statistics regarding mentions from all major social media platforms. The number of citations for each study was extracted through the Dimensions citation database, a platform affiliated with Altmetric that provides the total number of times a work is cited and has been deemed appropriate for citation rate analysis. Secondary variables, either article or Altmetric related, were chosen a priori and collected in accordance with previously published Altmetric investigations., Article characteristics included journal, issue, article title, date of publication, highest degree of first author, number of authors, number of academic institutions, geographic region of origin of the publication, disclosure of any conflict of interest (presence or absence of general self-reported conflict of interest), subject of study, study design, number of referenced studies, and level of evidence as defined by Arthroscopy. Altmetric variables considered and further analyzed included the number of Twitter mentions, number of Facebook mentions, number mentions by news outlets, and number of reads on Mendeley.

Statistical Analysis

All statistical analyses were performed with Stata software (version 16.1; StataCorp). Normality was verified with the Shapiro-Wilk test. Bibliometric and Altmetric characteristics were summarized by descriptive statistics using means with standard deviations for continuous variables and frequencies with percentages for categorical variables, compared between groups using 2-way t tests. A multivariate linear regression analysis was conducted to determine (1) the association between article type (technique articles and control research articles) and AAS and (2) the influence of bibliometric characteristics on this relation. The level of statistical significance was defined as P < .05.

Results

Article Characteristics

A total of 285 matched articles published between August 2019 and July 2020 were included. Of these, 57 were control research articles (20.0%) and 228 were technique articles (80.0%). Complete bibliometric and Altmetric characteristics are described in Table 2. Significant differences were noted in the number of references, authors, and institutions, with control research articles having more of each represented on average (P < .01).

Table 2

Bibliometric and Altmetric Characteristics of Included Technique Articles and Original Research Articles

	Original Research Articles (n = 57)	Technique Articles (n = 230)	P Value
Author degrees			.63
Other degree∗	27 (47.4)	98 (42.6)
MD or DO	30 (52.6)	132 (57.4)
Study origin			.97
Continent other than North America	28 (49.1)	111 (48.3)
North America	29 (50.9)	119 (51.7)
General COI			.32
No	29 (50.9)	95 (41.3)
Yes	28 (49.1)	135 (58.7)
Subject of study			.13
Knee	22 (38.6)	94 (40.9)
Shoulder	16 (28.1)	72 (31.3)
Hip	10 (17.5)	17 (7.4)
Other†	9 (15.8)	47 (20.4)
No. of references	28.3 ± 11.0	15.3 ± 7.9	<.001‡
No. of authors	5.1 ± 2.4	4.1 ± 1.8	.006‡
No. of institutions	2.6 ± 1.4	1.9 ± 1.1	.003‡
Twitter mention			.84
No	39 (68.4)	143 (62.2)
Yes	18 (31.6)	87 (37.8)
No. of Tweets	1.9 ± 5.9	3.3 ± 9.5	.19
Facebook mention			.66
No	51 (89.5)	194 (84.3)	.13
Yes	6 (10.5)	36 (15.7)
No. of Facebook mentions	0.1 ± 0.3	0.2 ± 0.5
Mendeley Readers			.32
No	38 (66.7)	136 (59.1)
Yes	19 (33.3)	94 (40.9)
No. of Mendeley Readers	3.1 ± 5.2	3.1 ± 4.8	.96

NOTE. Data are presented as number (percentage) or mean ± standard deviation.

COI, conflict of interest; DO, doctor of osteopathy; MD, doctor of medicine.

Other author degrees included BS (bachelor of science); MS (master of science); PhD (doctor of philosophy); MD, PhD; MD, MBA (master of business administration); MD, MS; MD, MPH (master of public health); and PT (physical therapist).

Other subjects of study included elbow, wrist, imaging, and foot and ankle.

Statistically significant (P < .05).

Social Media Attention

The mean AAS among all technique articles was 3.63 ± 10.08 (range, 0-96) whereas the mean AAS among all control research articles was 1.30 ± 3.98 (range, 0-25), representing a statistically significant difference (P = .016) (Table 3). A multivariate linear regression model controlling for bibliometric characteristics previously found to influence the AAS in prior studies was constructed to determine the influence of article type on AAS (Table 4). This model showed a statistically significant and positive association between AAS and article type, with an additional increase in the AAS of 2.91 (95% confidence interval [CI], 0.04-5.77; P = .047) on average for every technique article relative to an ORA. Furthermore, a significant positive relation was noted between the continent of origin and the AAS, with an additional increase in the AAS of 3.00 (95% CI, 0.82-5.17; P = .007) on average for every article published in North America compared with an article published in another continent.

Table 3

Mean Altmetric Attention Score and Citation Rate Among Technique Articles and Original Research Articles

	Altmetric Attention Score	Citation Rate
Original research articles	1.3 ± 4.0	0.4 ± 0.8
Technique articles	3.6 ± 10.1	0.4 ± 0.8
P value	.016∗	.73

NOTE. Data are presented as mean ± standard deviation.

Statistically significant (P < .05).

Table 4

Multivariate Linear Regression Model for Key Publication Characteristics, Citations, and Altmetric Attention Scores

Variable	Regression Coefficient	95% CI	P Value
Technique (reference: original article)	2.91	0.04 to 5.77	.047∗
COI (reference: no COI)	–0.35	–2.53 to 1.83	.75
North America (reference: other continents)	3.00	0.82 to 5.17	.007∗
Area of study	–0.04	–0.50 to 0.43	.88
No. of authors	0.11	–0.43 to 0.66	.68
Degree of first author (reference: non-MD and non-DO)	1.19	–0.91 to 3.28	.27
Month published (reference: April)
January	–1.76	–6.26 to 2.73	.44
February	–0.24	–5.19 to 4.71	.92
March	8.21	–4.22 to 5.36	.81
May	4.34	–0.45 to 9.13	.08
June	1.44	–2.70 to 5.59	.49
July	0.66	–4.17 to 5.48	.79
August	0.53	–12.15 to 13.22	.93
September	4.28	–6.28 to 14.94	.42
October	3.74	–13.83 to 21.31	.68
November	–1.87	–6.05 to 2.30	.38
December	0.90	–3.34 to 5.13	.68
No. of citations	4.21	2.91 to 5.51	<.001∗

NOTE. Other author degrees included BS (bachelor of surgery); MS (master of surgery); PhD (doctor of philosophy); MD, PhD; MD, MBA (master of business administration); MD, MS; MD, MPH (master of public health); and PT (physical therapist).

CI, confidence interval; COI, conflict of interest.

Statistically significant (P < .05).

Citation Rates

The mean citation rate among all technique articles was 0.35 ± 0.83 (range, 0-4), whereas the mean citation rate among all control research articles was 0.41 ± 0.81 (range, 0-4). This difference was not statistically significant (P = .73). Multivariate linear regression analysis showed that the citation rate was significantly associated with the AAS (β = 4.21; 95% CI, 2.91-5.51; P < .001) (Table 4).

Discussion

The main findings of this study are as follows: (1) Technique articles had greater AASs on average compared with ORAs pertaining to surgical techniques or outcomes of the same topic. (2) After adjusting for key confounding bibliometric characteristics, technique articles were significantly predictive of and positively associated with a higher AAS. (3) Citation rates did not differ between original articles and technique articles at a minimum of 1 year after online publication.

The mean AAS was 3.6 for technique articles and 1.3 for control research articles, representing a statistically significant difference in social media attention in favor of technique articles. The use of orthopaedic technique journals as a platform to highlight established procedures and recently developed surgical innovations has become commonplace within the field, with Techniques in Orthopaedics as one of the first journals publishing such content as early as 1986. However, with the evolution of the internet as the largest contemporary source of health care information and advancements in technology,, many orthopaedic journals began supplementing traditional articles by including videos to enhance viewer understanding of topics or procedures. Technique videos have been shown to be an effective tool to share medical knowledge, as well as surgical demonstrations, and the results of our study suggest that they are highly shared across social media. Similarly, surgical videos are powerful tools to facilitate acquiring new surgical skills and accelerate learning new techniques. In a randomized controlled trial, Pan et al. found that video instruction significantly improved student confidence and performance of venipuncture as compared with students taught without video demonstrations. Therefore, not only do technique articles that contain videos increase social media attention and shareability, but they may also enhance content teaching and knowledge retention.

It is interesting to note that both established and newer journals are migrating from publishing predominately text-based articles to an all-video, online format featuring enhanced abstracts. It is plausible that the visual format of technique videos may garner even greater social media attention than currently contextualized through the results of this study by allowing for the presentation of information in a concise, visually appealing manner while upholding patient privacy and the efficacy of the treatment. Moreover, as social media platforms, such as Instagram, transition their company missions to video sharing rather than photograph sharing, technique videos are poised to become more amenable to sharing and viewing on social media platforms in comparison to links to traditional articles and pictures. For instance, the International Hernia Collaboration, a closed Facebook group dedicated to sharing deidentified surgical videos to discuss how surgeons handle challenging patient presentations, has grown 6-fold since 2016, with over 11,000 members., The ability for surgeons and trainees to learn from these videos may help reduce potential patient morbidity that is associated with learning any new surgical procedure. Ultimately, the use of narrated videos to demonstrate the management of difficult cases may allow journals to increase their readership and maintain a far-reaching presence regardless of decreases in the numbers of print articles they publish.

In this study, technique articles were found to be a significant positive predictor of increased social media attention. Specifically, the mean increase in the AAS from technique articles was 2.91 relative to ORAs. Although articles were matched by subject, this finding reinforces that significant differences exist in social media attention between technique articles and ORAs and that social media attention may translate into increased citations and impact. As such, technique videos have the potential to have an increasingly strong impact throughout online forums. Previous studies have found the AAS to be a significant predictor of a greater number of citations within the orthopaedic literature and in many other fields of medicine.26, 27, 28, 29 Kunze et al. have postulated that researchers commonly cite studies they first encounter when writing their own manuscripts to enhance hypothesis generation and idea generation. Similarly, sharing technique videos may ultimately allow for social media to serve as a platform by which journals can not only optimize the extent of their reach in the scientific community but also garner increased citation rates and academic impact of their research. Furthermore, North America was an independent factor associated with an article achieving increased social media attention, with a mean increase in the AAS of 3.0 compared with articles from other origins. This finding is not surprising, given that most of the publications in the included journals were from North America and previous studies have similarly shown that articles from North America are associated with higher citation rates and AASs., The association among the citation rate, the AAS, and publications from North America has previously been described in the orthopaedic literature and is likely a reflection of high rates of publication in the United States. As such, considering that our study identified a positive association between AAS and citation rate, these findings are plausible and logical within the context of our study.

Although technique videos have been shown to enhance conceptual understanding and retention of content compared with written words alone, while garnering increased social media attention as suggested by our study, researchers should be cautioned on the limitations of using and interpreting content through these formats. For videos longer than 6 minutes, viewership engagement and learning begin to decrease at a rapid rate, with attention decreasing by over 50% after approximately 10 minutes. However, the relative time limit of presenting information through multimedia may require oversimplification of challenging procedures. Consequently, visual presentations may not support the same level of detailed explanation as other methods of surgical demonstration such as laboratory courses and instructional course lectures. Another key concern is that technique articles often lack long-term follow-up, patient outcome data, and control groups for comparison. This represents a low level of evidence that requires support from higher-level original research. Clinicians who view multimedia within technique articles should understand that although they are engaging media to enhance their skills, they should also be encouraged to review the accompanying articles and the supporting original research as a primary source before making changes in their practice.

Limitations

This investigation is not without limitations. First, this study examined a subset number of articles from 4 orthopaedic journals under 2 editorial groups. Therefore, the findings of this study may represent the unique practices of these journals’ readership and may not broadly apply to the greater orthopaedic literature. Second, the time restriction, given the initiation of ASMAR in August 2019, precludes the understanding of longer-term social media attention and citation rates. However, by including articles within a full year past their initial publication, our study allotted and captured an adequate length of time to accrue AASs and citations as implemented in prior Altmetric investigations,, while maintaining article recency. Third, the selective inclusion of ORAs matched by topic to technique articles based on our study purpose may inherently bias the amount of social media attention each article receives. Finally, our analysis examined only a subset of bibliometric characteristics. However, these characteristics have been used in prior literature, and have been deemed feasible for appropriate analytical use. Therefore, the decision was made to select and include these characteristics in our study.

Conclusions

Technique articles resulted in significantly greater AASs and social media attention in comparison with open-access ORAs on similar topics. Publications that described technical procedures in a technique journal and studies from North America were positively associated with greater AASs and greater numbers of citations received by articles.