Correlation Between Quality of Evidence and Number of Citations in Top 50 Cited Articles in Rotator Cuff Repair Surgery.

BACKGROUND: The number of article citations has been used as a measure for the impact of an article in the medical literature, with little emphasis on quality.
PURPOSE: To (1) identify the top 50 most cited articles related to rotator cuff repair surgery, (2) determine whether there was a correlation between the top cited articles and level of evidence, and (3) determine whether there was a correlation between the top cited articles and study quality. STUDY
DESIGN: Cross-sectional study.
METHODS: The Web of Science and Scopus online databases were searched to identify the top 50 cited articles in rotator cuff repair surgery. Methodological quality was analyzed for each article using the Modified Coleman Methodology Score (MCMS), Jadad scale, and Methodological Index for Non-Randomized Studies (MINORS). Correlation coefficients were calculated to determine the degree of correlation between the top cited articles and level of evidence and study quality using each quality score.
RESULTS: The mean number of citations for each article in each of the 2 databases was 319 ± 187 (range, 177.5-1033.5). Twenty-nine articles (58%) were from the United States. The most common level of evidence was level 4 (54%), with 3 (6%) articles being level 1. There was no significant correlation between the mean number of citations and level of evidence (rs = -0.28), the MCMS (rs = -0.29), and the MINORS score (rs = -0.25). There was a weak negative correlation between the mean number of citations and the Jadad score (rs = -0.36).
CONCLUSION: The top 50 cited articles in rotator cuff repair surgery comprise a variety of years, journals, countries of origin, and study types. Despite being the 50 most cited articles, the most common type of article was the level 4 case series with a poor mean quality assessment score. There was no significant correlation between the mean number of citations and level of evidence or methodological quality using a variety of scores.

Article citations by other authors have been used as a measure for the academic impact of an article in the medical literature.[1,7,10,30] Research productivity for authors and the impact factor for journals are calculated based on the number of citations associated with each publication.[25] Impact factor has been defined as the number of citations within a given year of items published by a journal in the preceding 2 years divided by the number of citable items published by the journal in these 2 years.[12]

Recently, it has been suggested that the impact factor has gone from a measure of a journal’s citation influence to a surrogate that assesses the scholarly value of work published in that journal.[20] However, further evidence suggests that journal impact factors are related directly to a few highly cited articles within the journal and are not reflective of the quality of the majority of the articles published in the journal.[6] As such, several new metrics have been created to objectively assign a value to journals based on the number of citations of a journal over a given time period, including the Immediacy Index, SCImago Journal Rank, CiteScore, Source Normalized Impact per Paper, h-index, 2-year impact factor, 5-year impact factor, and ResearchGate Score. This has led to several studies analyzing the top cited articles in their respective fields.[4,18,23,24]

Improved understanding of musculoskeletal basic science and the development of new surgical implants and techniques have led to rapid increases in the orthopaedic literature.[11,21] As such, several authors have attempted to analyze the most cited articles in shoulder, elbow, pediatric, and foot and ankle surgery.[3,5,14,16,22]

As the number of publications on rotator cuff repair continues to rise, it is important to analyze the quality of the most cited articles on this topic. Also, as researchers are increasingly evaluated and compared by their studies’ academic impact, “top 50” or “top 25” lists of articles on a specific topic are of high value to the readership if, upon critical analysis, their methodological quality supports their top ranking.

The purpose of this study was to (1) identify the top 50 most cited articles related to rotator cuff repair surgery, (2) determine if there was a correlation between the top 50 cited articles and level of evidence, and (3) determine if there was a correlation between the top cited articles and study quality. We hypothesized that there would be a weak to no significant correlation between the top cited articles in rotator cuff repair surgery and level of evidence and study methodological quality.

Methods

The Web of Science and Scopus online databases were searched in December 2017 according to previously described methods without date restrictions.[3,16,17,29] The terms “rotator cuff repair,” “rotator cuff surgery,” “arthroscopic rotator cuff repair,” and “arthroscopic rotator cuff surgery” were individually searched in each database. All articles and all journals were included. Articles were sorted by the number of times cited, from highest to lowest. Each article was evaluated to determine whether it was appropriately related to rotator cuff surgery. The number of citations for each article was then averaged between the 2 search engines to create a list of the top 50 cited articles in rotator cuff repair surgery.

Characteristics from each article were recorded: number of times cited, year of publication, name of journal, country of origin, and study type (review, technique, animal study, cadaveric study, case series, retrospective cohort, prospective cohort, case control, and randomized controlled trial). Studies were categorized into therapy/prevention, diagnostic, prognostic, and economic based on the Oxford Centre for Evidence-based Medicine levels of evidence (http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/). Level of evidence for each study was also evaluated based on The Journal of Bone & Joint Surgery–American Volume guidelines.[19] Study methodological quality was analyzed for each article using the Modified Coleman Methodology Score (MCMS), Jadad scale, and Methodological Index for Non-Randomized Studies (MINORS).[8,15,27,28] Technique studies, reviews, animal studies, and cadaveric studies were not evaluated for methodological quality.

Data were tested for normal distribution using the Kolmogorov-Smirnov test (http://www.physics.csbsju.edu/stats/KS-test.n.plot_form.html). The Pearson correlation coefficient (r) was used to determine the degree of correlation between the top cited articles and level of evidence and the degree of correlation between study quality and the top cited articles when the data were normally distributed. The Spearman correlation coefficient (r s) was used to determine the degree of correlation between the top cited articles and level of evidence and the degree of correlation between study quality and the top cited articles when the data were not normally distributed. Correlation was defined as a perfect negative linear relationship (r, r s = –1), strong negative linear relationship (r, r s = –0.70), moderate negative linear relationship (r, r s = –0.50), weak negative linear relationship (r, r s = –0.30), no linear relationship (r, r s = 0), weak positive linear relationship (r, r s = +0.30), moderate positive linear relationship (r, r s = +0.50), strong positive linear relationship (r, r s = +0.70), and perfect positive linear relationship (r, r s = +1). Student t tests were carried out for 2 group comparisons. A P value <.05 was defined as significant.

Results

The 50 most cited articles in rotator cuff repair surgery were published between 1984 and 2011 (Table 1). The decade from 2000 to 2009 accounted for the greatest number of articles (n = 30) (Figure 1). The selected articles were published in 12 journals (Figure 2). Most were published in The Journal of Bone & Joint Surgery–American Volume, with 19 articles (38%). The overall mean number of citations for each article was 319 ± 187 (range, 177.5-1033.5) (Table 1). There were 8 countries represented, with 29 (58%) of the top 50 cited articles originating from the United States (USA) (Figure 3).

TABLE 1

Top 50 Most Cited Articles in Rotator Cuff Repair Surgery

Rank	Article	Country	Type	No. of Citationsb	Level of Evidence	MCMS	Jadad Score	MINORS Score
1	Goutallier D, Postel JM, Bernageau J, Lavau L, Voisin MC. Fatty muscle degeneration in cuff ruptures: pre- and postoperative evaluation by CT scan. Clin Orthop Relat Res. 1994;304:78-83.	France	Case series	1033.5	4	48	1	10
2	Galatz LM, Ball CM, Teefey SA, Middleton WD, Yamaguchi K. The outcome and repair integrity of completely arthroscopically repaired large and massive rotator cuff tears. J Bone Joint Surg Am. 2004;86:219-224.	USA	Case series	912	4	49	1	10
3	Harryman DT 2nd, Mack LA, Wang KY, Jackins SE, Richardson ML, Matsen FA 3rd. Repairs of the rotator cuff: correlation of functional results with integrity of the cuff. J Bone Joint Surg Am. 1991;73:982-989.	USA	Case series	761	4	56	1	10
4	Ellman H, Hanker G, Bayer M. Repair of the rotator cuff: end-result study of factors influencing reconstruction. J Bone Joint Surg Am. 1986;68:1136-1144.	USA	Case series	608.5	4	54	1	12
5	Gerber C, Fuchs B, Hodler J. The results of repair of massive tears of the rotator cuff. J Bone Joint Surg Am. 2000;82:505-515.	Switzerland	Case series	600.5	4	51	1	10
6	Boileau P, Brassart N, Watkinson DJ, Carles M, Hatzidakis AM, Krishnan SG. Arthroscopic repair of full-thickness tears of the supraspinatus: does the tendon really heal? J Bone Joint Surg Am. 2005;87:1229-1240.	France	Case series	591	4	61	1	12
7	Gerber C, Schneeberger AG, Beck M, Schlegel U. Mechanical strength of repairs of the rotator cuff. J Bone Joint Surg Br. 1994;76(3):371-380.	Switzerland	Technique	447	5	N/A	N/A	N/A
8	Werner CM, Steinmann PA, Gilbart M, Gerber C. Treatment of painful pseudoparesis due to irreparable rotator cuff dysfunction with the Delta III reverse-ball-and-socket total shoulder prosthesis. J Bone Joint Surg Am. 2005;87(7):1476-1486.	Switzerland	Case series	433	4	55	1	10
9	Morgan CD, Burkhart SS, Palmeri M, Gillespie M. Type II SLAP lesions: three subtypes and their relationships to superior instability and rotator cuff tears. Arthroscopy. 1998;14:553-565.	USA	Case series	405	4	52	1	10
10	Fuchs B, Weishaupt D, Zanetti M, Hodler J, Gerber C. Fatty degeneration of the muscles of the rotator cuff: assessment by computed tomography versus magnetic resonance imaging. J Shoulder Elbow Surg. 1999;8:599-605.	Switzerland	Case series	400	4	43	1	12
11	Goutallier D, Postel JM, Gleyze P, Leguilloux P, Van Driessche S. Influence of cuff muscle fatty degeneration on anatomic and functional outcomes after simple suture of full-thickness tears. J Shoulder Elbow Surg. 2003;12(6):550-554.	France	Case series	395	4	56	1	10
12	Wall B, Nove-Josserand L, O’Connor DP, et al. Reverse total shoulder arthroplasty: a review of results according to etiology. J Bone Joint Surg Am. 2007;89:1476-1485.	USA	Prospective cohort	389	2	57	1	18
13	Boileau P, Watkinson DJ, Hatzidakis AM, Balg F. Grammont reverse prosthesis: design, rationale, and biomechanics. J Shoulder Elbow Surg. 2005;14(1 suppl S):S147-S161.	France	Review	379	5	N/A	N/A	N/A
14	Gazielly DF, Gleyze P, Montagnon C. Functional and anatomical results after rotator cuff repair. Clin Orthop Relat Res. 1994;(304):43-53.	France	Case series	358.5	4	61	1	12
15	Boileau P, Watkinson D, Hatzidakis AM, Hovorka I. Neer Award 2005. The Grammont reverse shoulder prosthesis: results in cuff tear arthritis, fracture sequelae, and revision arthroplasty. J Shoulder Elbow Surg. 2006;15(5):527-540.	France	Case series	351	4	60	1	10
16	Frankle M, Siegal S, Pupello D, Saleem A, Mighell M, Vasey M. The reverse shoulder prosthesis for glenohumeral arthritis associated with severe rotator cuff deficiency: a minimum two-year follow-up study of sixty patients. J Bone Joint Surg Am. 2005;87(8):1697-1705.	USA	Case series	314	4	63	1	10
17	Jost B, Pfirrmann CW, Gerber C, Switzerland Z. Clinical outcome after structural failure of rotator cuff repairs. J Bone Joint Surg Am. 2000;82(3): 304-314.	Switzerland	Case series	309.5	4	54	1	12
18	Cofield RH, Parvizi J, Hoffmeyer PJ, Lanzer WL, Ilstrup DM, Rowland CM. Surgical repair of chronic rotator cuff tears: a prospective long-term study. J Bone Joint Surg Am. 2001;83(1):71-77.	USA	Prospective cohort	276.5	2	59	1	20
19	Gladstone JN, Bishop JY, Lo IK, Flatow EL. Fatty infiltration and atrophy of the rotator cuff do not improve after rotator cuff repair and correlate with poor functional outcome. Am J Sports Med. 2007;35(5):719-728.	USA	Prospective cohort	276	2	51	1	20
20	Sugaya H, Maeda K, Matsuki K, Moriishi J. Repair integrity and functional outcome after arthroscopic double-row rotator cuff repair: a prospective outcome study. J Bone Joint Surg Am. 2007;89:953-960.	Japan	Case series	276	4	59	1	10
21	Gerber C, Schneeberger AG, Perren SM, Nyffeler RW. Experimental rotator cuff repair: a preliminary study. J Bone Joint Surg Am. 1999;81(9):1281-1290.	Switzerland	Animal study	271	5	N/A	N/A	N/A
22	Gartsman GM, Khan M, Hammerman SM. Arthroscopic repair of full-thickness tears of the rotator cuff. J Bone Joint Surg Am. 1998;80:832-840.	USA	Case series	261.5	4	59	1	12
23	Bishop J, Klepps S, Lo IK, Bird J, Gladstone JN, Flatow EL. Cuff integrity after arthroscopic versus open rotator cuff repair: a prospective study. J Shoulder Elbow Surg. 2006;15:290-299.	USA	Prospective cohort	259.5	2	50	1	18
24	DeOrio JK, Cofield RH. Results of a second attempt at surgical repair of a failed initial rotator-cuff repair. J Bone Joint Surg Am. 1984;66(4):563-567.	USA	Case series	259	4	47	1	10
25	Chandnani VP, Yeager TD, DeBerardino T, et al. Glenoid labral tears: prospective evaluation with MRI imaging, MR arthrography, and CT arthrography. AJR Am J Roentgenol. 1993;161:1229-1235.	USA	Case series	245	4	46	1	10
26	Sugaya H, Maeda K, Matsuki K, Moriishi J. Functional and structural outcome after arthroscopic full-thickness rotator cuff repair: single-row versus dual-row fixation. Arthroscopy. 2005;21(11):1307-1316.	Japan	Case series	242	3	54	1	16
27	Kim DH, ElAttrache NS, Tibone JE, et al. Biomechanical comparison of a single-row versus double-row suture anchor technique for rotator cuff repair. Am J Sports Med. 2006;34:407-414.	USA	Cadaveric study	239	5	N/A	N/A	N/A
28	Burkhart SS, Nottage WM, Ogilvie-Harris DJ, Kohn HS, Pachelli A. Partial repair of irreparable rotator cuff tears. Arthroscopy. 1994;10:363-370.	USA	Case series	231.5	4	48	1	12
29	Franceschi F, Ruzzini L, Longo UG. Equivalent clinical results of arthroscopic single-row and double-row suture anchor repair for rotator cuff tears: a randomized controlled trial. Am J Sports Med. 2007;35:1254-1260.	Italy	Randomized controlled trial	231	1	71	3	20
30	Ma CB, Comerford L, Wilson J, Puttlitz CM. Biomechanical evaluation of arthroscopic rotator cuff repairs: double-row compared with single-row fixation. J Bone Joint Surg Am. 2006;88(2):403-410.	USA	Cadaveric study	227.5	5	N/A	N/A	N/A
31	Green S, Buchbinder R, Glazier R, et al. Systematic review of randomised controlled trials of interventions for painful shoulder: selection criteria, outcome assessment, and efficacy. BMJ. 1998;316:354-360.	Australia	Systematic review	224	1	64	2	22
32	Thomazeau H, Boukobza E, Morcet N, Chaperon J, Langlais F. Prediction of rotator cuff repair results by magnetic resonance imaging. Clin Orthop Relat Res. 1997;344:275-283.	France	Case series	220.5	4	51	1	12
33	Gartsman G, Roddey TS, Hammerman S. Arthroscopic treatment of anterior-inferior glenohumeral instability: two to five-year follow-up. J Bone Joint Surg Am. 2000;82(7):991-1003.	USA	Case series	220	4	60	1	12
34	Burkhart SS, Diaz Pagan JL, Wirth MA, Athanasiou KA. Cyclic loading of anchor-based rotator cuff repairs: confirmation of the tension overload phenomenon and comparison of suture anchor fixation with transosseous fixation. Arthroscopy. 1997;13:720-724.	USA	Cadaveric study	217.5	5	N/A	N/A	N/A
35	Lafosse L, Brzoska R, Toussaint B, Gobezie R. The outcome and structural integrity of arthroscopic rotator cuff repair with use of the double-row suture anchor technique. J Bone Joint Surg Am. 2008;90:275-286.	France	Case series	212	4	63	1	12
36	Castricini R, Longo UG, De Benedetto M, et al. Platelet-rich plasma augmentation for arthroscopic rotator cuff repair: a randomized controlled trial. Am J Sports Med. 2011;39(2):258-265.	USA	Randomized controlled trial	211	1	81	4	24
37	Burkhart SS, Morgan CD, Kibler WB. Shoulder injuries in overhead athletes: the “dead arm” revisited. Clin Sports Med. 2000;19(1):125-158.	USA	Case series	210	4	55	1	10
38	Burkhart SS, Danaceau SM, Pearce CE Jr. Arthroscopic rotator cuff repair: analysis of results by tear size and by repair technique-margin convergence versus direct tendon-to-bone repair. Arthroscopy. 2001;17:905-912.	USA	Case series	210	4	65	1	14
39	Kirkley A, Griffin S, Dainty K. Scoring systems for the functional assessment of the shoulder. Arthroscopy. 2003;19(10):1109-1120.	Canada	Review	204	5	N/A	N/A	N/A
40	Rockwood CA Jr, Williams GR Jr, Burkhead WZ Jr. Debridement of degenerative, irreparable lesions of the rotator cuff. J Bone Joint Surg Am. 1995;77(6):857-866.	USA	Case series	202.5	4	55	1	10
41	Liu SH, Baker CL. Arthroscopically assisted rotator cuff repair: correlation of functional results with integrity of the cuff. Arthroscopy. 1994;10(1):54-60.	USA	Case series	192	4	50	1	10
42	Thomopolous S, Williams GR, Soslowsky LJ. Tendon to bone healing: differences in biomechanical, structural, and compositional properties due to a range of activity levels. J Biomech Eng. 2003;125:106-113.	USA	Animal study	190	5	N/A	N/A	N/A
43	Walch G, Edwards TB, Boulahia A, Nove-Josserand L, Neyton L, Szabo I. Arthroscopic tenotomy of the long head of the biceps in the treatment of rotator cuff tears: clinical and radiographic results of 307 cases. J Shoulder Elbow Surg. 2005;14(3):238-246.	USA	Case series	189	4	56	1	8
44	Iannotti JP, Codsi MJ, Kwon YW, Derwin K, Ciccone J, Brems JJ. Porcine small intestine submucosa augmentation of surgical repair of chronic two-tendon rotator cuff tears: a randomized, controlled trial. J Bone Joint Surg Am. 2006;88:1238-1244.	USA	Prospective cohort	187.5	2	63	3	20
45	Zheng MH, Chen J, Kirilak Y, Willers C, Xu J, Wood D. Porcine small intestine submucosa (SIS) is not an acellular collagenous matrix and contains porcine DNA: possible implications in human implantation. J Biomed Mater Res B Appl Biomater. 2005;73:61-67.	Australia	Animal study	185	5	N/A	N/A	N/A
46	Lo IK, Burkhart SS. Double-row arthroscopic rotator cuff repair: re-establishing the footprint of the rotator cuff. Arthroscopy. 2003;19:1035-1042.	USA	Technique	183.5	5	N/A	N/A	N/A
47	Wirth MA, Rockwood CA Jr. Complications of shoulder arthroplasty. Clin Orthop Relat Res. 1994;307:47-69.	USA	Review	182.5	5	N/A	N/A	N/A
48	Mazzocca AD, Millett PJ, Guanche CA, Santangelo SA, Arciero RA. Arthroscopic single-row versus double-row suture anchor rotator cuff repair. Am J Sports Med. 2005;33:1861-1868.	USA	Cadaveric study	180	5	N/A	N/A	N/A
49	Riley GP, Harrall RL, Constant CR, Chard MD, Cawston TE, Hazleman BL. Tendon degeneration and chronic shoulder pain: changes in the collagen composition of the human rotator cuff tendons in rotator cuff tendinitis. Ann Rheum Dis. 1994;53: 359-366.	UK	Cadaveric study	180	5	N/A	N/A	N/A
50	Park MC, Tibone JE, ElAttrache NS, Ahmad CS, Jun B-J, Lee TQ. Part II: biomechanical assessment for a footprint-restoring transosseous-equivalent rotator cuff repair technique compared with a double-row repair technique. J Shoulder Elbow Surg. 2007;16:469-476.	USA	Cadaveric study	177.5	5	N/A	N/A	N/A

MCMS, Modified Coleman Methodology Score; MINORS, Methodological Index for Non-Randomized Studies; N/A, not applicable.

Number of citations for each article averaged between the Web of Science and the Scopus online databases.

Figure 1.

Number of most cited articles and decade of publication.

Figure 2.

Number of top 50 cited articles in each journal. AJR, American Journal of Roentgenology; AJSM, The American Journal of Sports Medicine; ARD, Annals of the Rheumatic Diseases; BMJ, The British Medical Journal; CORR, Clinical Orthopaedics and Related Research; CSM, Clinics in Sports Medicine; JBE, Journal of Biomechanical Engineering; JBJS-Am, The Journal of Bone & Joint Surgery–American Volume; JBJS-Br, The Journal of Bone & Joint Surgery–British Volume; JBMR, Journal of Biomedical Materials Research; JSES, Journal of Shoulder and Elbow Surgery.

Figure 3.

Countries represented in top 50 most cited articles.

The most common type of article was the retrospective case series, with 28 (56%) (Figure 4). There were studies of all levels of evidence (1-5), with the most common being level 4 (54%) (Figure 5). There were 3 studies (6%) that were level 1 evidence in the top 50 cited articles. Forty-three (86%) of the studies were prognostic, with the remaining being diagnostic. There was no significant correlation between the mean number of citations and level of evidence (r s = –0.28, n = 50, P = .047) among the studies included in the review.

Figure 4.

Top 50 cited articles by article type. RCT, randomized controlled trial.

Figure 5.

Top 50 cited articles by level of evidence.

Thirty-six studies were available for an analysis of methodological quality. The 14 studies excluded were 6 cadaveric studies, 3 animal studies, 3 reviews, and 2 technique studies. The mean MCMS was 56.3 ± 7.5 (poor; range, 43-81). There was no significant correlation between the mean number of citations and the MCMS (r s = –0.29, n = 36, P = .087). The mean Jadad score was 1.2 ± 0.7 (range, 1-4). There was a weak negative correlation between the mean number of citations and the Jadad score (r s = –0.36, n = 36, P = .027). The mean MINORS score was 13.1 ± 4.2 (range, 8-24). There was no significant correlation between the mean number of citations and the MINORS score (r s = –0.25, n = 36, P = .147).

Studies were analyzed for change over time. There was no significant correlation between year published and level of evidence (r s = –0.20, n = 50, P = .175). There was a moderate positive correlation between year published and the MCMS (r s = 0.51, n = 36, P < .01). There was a weak positive correlation between year published and the Jadad score (r s = 0.32, n = 36, P = .054). There was a weak positive correlation between year published and the MINORS score (r s = 0.36, n = 36, P = .032).

Differences between level of evidence, the MCMS, the Jadad score, and the MINORS score were evaluated by country of origin. Countries were defined as either USA (n = 29) or non-USA (n = 21), which comprised France, Switzerland, Japan, Italy, Australia, Canada, and the United Kingdom. The mean level of evidence was 3.8 ± 1.1 for USA studies and 3.9 ± 1.1 for non-USA studies (P = .701). There was no significant difference (P = .776) between the mean MCMS for USA studies (56.0 ± 7.9) and the mean MCMS for non-USA studies (56.7 ± 7.1). There was no significant difference (P = .871) between the mean Jadad score for USA studies (1.2 ± 0.8) and the mean Jadad score for non-USA studies (1.2 ± 0.6). There was no significant difference (P = .648) between the mean MINORS score for USA studies (13.3 ± 4.6) and the mean MINORS score for non-USA studies (12.7 ± 3.8).

Discussion

This study identified the top 50 most cited articles related to rotator cuff repair surgery. This study demonstrated that there was no significant correlation between the top 50 cited articles and level of evidence, confirming our hypothesis. This study also demonstrated that there was no significant correlation between the MINORS score and MCMS and the top 50 cited articles. However, there was a weak negative correlation between the mean number of citations and the Jadad score, which partially confirmed our hypothesis.

Two previous studies have evaluated the top cited articles in shoulder surgery. Namdari et al[22] demonstrated that the majority of the top 50 cited articles in shoulder surgery were published in the 1990s (n = 18) and 1980s (n = 17). Additionally, a study by Kraeutler et al,[16] which assessed the top 50 cited articles in rotator cuff surgery, showed 29 articles published in the 2000s and 16 published in the 1990s.[22] Kraeutler et al[16] had similar results as in the present study, with most articles published in the 2000s (n = 30) and 1990s (n = 17). Neither of these studies evaluated the quality of the top articles currently in the literature. This current study furthers the argument of the need for an emphasis on higher quality studies in the top orthopaedic journals.

These results were interesting, as time since publication intuitively would provide an advantage by increasing the likelihood of citations. However, our findings may be related to an increase in the overall number of publications (“inflation”) over the past decade.[11,21] Additionally, it may reflect a paradigm shift in rotator repair surgery over the past decade from open to arthroscopic surgery, with 43% of articles cited in the past decade evaluating outcomes after arthroscopic repair. This is further supported by Colvin et al,[9] with arthroscopic rotator cuff repairs increasing 600% compared with 34% for open repairs from 1996 to 2006.

In the current study, the majority of articles were published in The Journal of Bone & Joint Surgery–American Volume (38%), Arthroscopy (16%), and Journal of Shoulder and Elbow Surgery (14%). This same trend was seen in prior studies assessing shoulder surgery and rotator cuff surgery, with 57% and 40% of the top cited articles from The Journal of Bone & Joint Surgery–American Volume, respectively.[16,22] Similar to the current study, these articles also had Journal of Shoulder and Elbow Surgery and The American Journal of Sports Medicine as major sources of top cited articles. This is to be expected, as these journals consistently have the highest impact factors in the orthopaedic surgery literature.[26]

Most (58%) of the authors of the top 50 cited articles were from the USA. This trend has also been seen in previous studies in shoulder surgery, plastic surgery, and general surgery and in the top 100 articles in orthopaedic surgery.[16-18,22,24] This indicates a possible bias toward American authors, as several of the top journals in orthopaedic surgery are based in the United States and published in the English language.

Previous studies have shown that the majority of the top cited articles in orthopaedic and shoulder surgery are level 4 case series.[16,17,22] This was similar to the present study on rotator cuff repair, in which 28 (56%) of the articles were level 4 case series. There has been a recent focus on evidence-based medicine, as reflected in the positive correlation seen in this study between year of publication and methodological quality, with more recent studies being of higher quality (higher Jadad score, MCMS, and MINORS score). However, low-level studies still dominate the literature and should serve as an impetus to improve the methodological quality of studies investigating this topic.

It is possible that if this study were to be repeated at a later date, the number of investigations with low levels of evidence would decrease. This trend is apparent when comparing the study by Namdari et al[22] published in 2012 (0 level 1 studies) to a more recent study by Kraeutler et al[16] published in 2016 (3 level 1 studies) and the current study (3 level 1 studies).[17] In addition, analogous to the calculation of the impact factor, more recent citation data (2 or 5 years rather than the past 30 years as used in the current study) would be highly likely to change the composition of the current investigation’s top 50 cited list.

In contrast to previous studies, the current study analyzed the methodological quality of the top cited articles. According to the MCMS, articles ranged from poor (scores <55) to good (scores of 70-84), with the overall mean MCMS classified as poor (56.3). Although not investigated in other articles on top citations within the field, a prior study by Harris et al[13] found similar results in articular cartilage studies, with poor methodological quality overall. The methodological quality deficiencies identified in this study should guide the design, conduct, and reporting of future rotator cuff repair studies.

Additionally, the current study aimed to determine if there was any correlation between level of evidence and methodological quality with number of citations. As expected, there was no significant correlation (r s = –0.28) between level of evidence and number of citations. This is in contrast to a prior study by Arshi et al[2] that evaluated the top cited articles in cartilage surgery, in which the number of citations was correlated with a stronger level of evidence. However, in that investigation, there was an increased number of level 1 and 2 studies compared with the present study. We found no significant correlation between the mean number of citations and the MCMS (r s = –0.29) and MINORS score (r s = –0.25) and only a weak negative correlation between the mean number of citations and the Jadad score (r s = –0.36). The poor correlation between level of evidence and number of citations is because of the few number of level 1 and 2 studies compared with the lower level studies. Additionally, the poor correlation between number of citations and methodological quality is likely secondary to the MCMS and Jadad scale’s favoring randomized controlled trials (only 3 are present in this study) while the MINORS favors nonrandomized controlled trials.

This study emphasizes the importance of critically evaluating the literature and encourages the orthopaedic community to rely more on the results from higher quality studies with a high level of evidence as opposed to just the top cited articles or highest impact journals in the literature. The methodological quality deficiencies identified in this study should guide future rotator cuff studies to improve their methodology and design and achieve higher quality articles. The current study should also serve as a template for future investigations that attempt to analyze the quality of the literature in orthopaedics and sports medicine–related topics.

The current study has a number of strengths and limitations. This was the first study to analyze the quality of the most cited articles in rotator cuff repair surgery. The number of articles (N = 50) was arbitrarily chosen and may have eliminated other influential articles. However, this number is based on several previously published studies.[3,14,16,22] This study also only utilized 2 databases: Web of Science and Scopus. However, Scopus is the world’s largest scientific database, and by combining with Web of Science, we feel that few to no citations were missed.

Additionally, the search protocol in this study was not limited to “known” orthopaedic journals of previously published articles, as we did not want to limit the potential for missed relevant articles. The number of citations can be influenced by several factors. One possible factor is high-volume authors citing their own work (“self-citation”), which was not accounted for in this study. Also, there appears to be a “snowball effect” to citations, as other authors are more likely to cite an article because of previous citations rather than for its content or quality.[17] There is also a disadvantage to newer published articles having less time to accrue citations, although this did not seem to be a factor in this study. Using a 2-year or 5-year (or any arbitrarily chosen time duration) selection eligibility period could change the composition of the top 50 studies in the current investigation’s list. There were 14 studies (28%) eliminated from the quality evaluation because of no available methodological quality of the evidence scoring system for nonclinical studies. A final limitation is that the number of times that an article is cited is always changing and is dependent on shifts in the field that change over time rather than the quality or content of the article.

The top 50 cited articles in rotator cuff repair surgery comprise a variety of years, journals, countries of origin, and study types. Despite being the 50 most cited articles, the most common type of article was the level 4 case series with a poor mean quality assessment score. There was no significant correlation between the mean number of citations and level of evidence or methodological quality using a variety of scoring measures.