Assessment and Trends in the Methodological Quality of the Top 50 Most Cited Articles in Shoulder Instability.

BACKGROUND: Citation counts have often been used as a surrogate for the scholarly impact of a particular study, but they do not necessarily correlate with higher-quality investigations. In recent decades, much of the literature regarding shoulder instability is focused on surgical techniques to correct bone loss and prevent recurrence.
PURPOSE: To determine (1) the top 50 most cited articles in shoulder instability and (2) if there is a correlation between the number of citations and level of evidence or methodological quality. STUDY
DESIGN: Cross-sectional study.
METHODS: A literature search was performed on both the Scopus and the Web of Science databases to determine the top 50 most cited articles in shoulder instability between 1985 and 2019. The search terms used included "shoulder instability," "humeral defect," and "glenoid bone loss." Methodological scores were calculated using the Modified Coleman Methodology Score (MCMS), Jadad scale, and Methodological Index for Non-Randomized Studies (MINORS) score.
RESULTS: The mean number of citations and mean citation density were 222.7 ± 123.5 (range, 124-881.5) and 16.0 ± 7.9 (range, 6.9-49.0), respectively. The most common type of study represented was the retrospective case series (evidence level, 4; n = 16; 32%) The overall mean MCMS, Jadad score, and MINORS score were 61.1 ± 10.1, 1.4 ± 0.9, and 16.0 ± 3.0, respectively. There were also no correlations found between mean citations or citation density versus each of the methodological quality scores.
CONCLUSION: The list of top 50 most cited articles in shoulder instability comprised studies with low-level evidence and low methodological quality. Higher-quality study methodology does not appear to be a significant factor in whether studies are frequently cited in the literature.

Primary shoulder dislocations are one of the most frequent joint dislocations seen in orthopaedics, with an incidence of 23.9 dislocations per 100,000 person-years in the United States.[46] In the past few decades, the paradigm of recurrent shoulder instability management has shifted toward surgical stabilization of the glenohumeral joint, as nonoperative treatment has been shown to lead to higher rates of recurrence.[17,18] However, there is still no clear consensus on the surgical management of glenoid and humeral bone loss.[21] Current debate is ongoing about the use of the Latarjet procedure and allograft bone blocks for glenoid defects, as well as the remplissage procedure and humeroplasty for humeral defects.[21] Owing to the amount of research on shoulder instability, it is important to analyze the types and quality of these investigations to determine the driving factors for current and future clinical practice.[19,36]

The number of citations that an academic research paper receives is often regarded as a simple way to measure the impact it has on its field.[1,13] The journal impact factor, which is based on the average number of citations received by articles, has been widely regarded as an index for the productivity and value of studies published within a certain journal.[10] However, it has been shown that high impact factors do not necessarily correlate with high-quality studies.[2,10] In orthopaedics, studies have recently been published regarding the quality of research on rotator cuff,[41] elbow,[24,25] pediatrics,[3] spine,[43] and foot and ankle[30] surgery. These studies have sought to stratify differences in quality and methodology and provide a summary for the various types of research that have been crucial in driving the development within their respective fields.[25,41]

The purpose of this investigation was to (1) identify the top 50 most cited articles regarding shoulder instability and (2) determine whether there was a correlation between the mean citations and citation density of these articles and the level of evidence or their methodological quality. We hypothesized that there is a weak or insignificant correlation between the top 50 articles in shoulder instability and their level of evidence or methodological quality.

Methods

Search Strategy

A search of the Scopus and the Web of Science databases was performed on February 26, 2019, by 2 authors (K.Y. and A.A.). The databases included studies published between 1985 and 2019. The starting year of 1985 was selected because it is the earliest year accessible on these databases, and it captures any “classic” literature that was published before the most recent decade. The following Boolean search was conducted in each database using the advanced search function: “Glenoid Defect” OR “Bankart Lesion” OR “Osseous Bankart” OR “Bony Bankart” OR “Humeral Defect” OR “Shoulder Instability” OR “Bipolar Instability” OR “Primary Instability” OR “Recurrent Instability” OR “Glenoid Bone Loss” OR “Bony Defect” OR “Humeral Bone Loss” NOT “Arthroplasty” NOT “Replacement” AND “Surgical Procedures,” “Operative” OR “Orthopedics” OR “Orthopaedics.”

Selection Criteria

All relevant articles and journals that were published in the English language were included. The articles were then filtered by their number of citations, from highest to lowest. The number of citations was averaged from each database to formulate the top 50 most cited articles in shoulder instability according to both databases. There were 21 articles that appeared in only 1 of the 2 databases, and they were excluded. All titles and abstracts were then evaluated for their relevance to shoulder instability and selected from each database. The citation density of each article was calculated by dividing the average number of citations between the 2 databases by the number of years since publication, in order to account for older articles having more citations.

Data Extraction

The following characteristics for each article were recorded: title, author(s), journal, publication information, country of origin, and study type. We also recorded the level of evidence, based on the guidelines established by Marx et al.[31] Additionally, each article was evaluated for methodological quality by 2 authors separately (K.Y. and A.A.). Consensus was achieved when discrepancies between scores were found. The reviewers used 3 scoring systems, selected to maintain consistency with similar bibliometric studies in other areas of orthopaedics.[25,41] The range of scores for each scoring system is listed in parentheses, with higher scores indicating higher methodological quality: the Modified Coleman Methodology Score (MCMS)[37] (0-100), the Jadad scale[26] (0-5), and the Methodological Index for Non-Randomized Studies (MINORS)[39] (0-24). The MCMS is a tool that accounts for the study design, methodology, and quality of outcome reporting, and its reliability has been validated with 99% reproducibility.[14,37,42] The Jadad scale favors studies that are well randomized and blinded and also assesses the appropriateness of individual randomization and blinding methods.[26] The MINORS score is geared toward nonrandomized studies and was originally developed for surgical fields because of the large number of observational studies in that field.[39] Scores were not calculated for cadaveric, review, or technique studies.

Statistical Analysis

The data were assessed for a normal distribution using a Kolmogorov-Smirnov test. Based on the results, the Spearman rank coefficient (R S) was used to determine the degree of correlation if the data did not follow a normal distribution. The Pearson correlation coefficient (R) was used to determine the degree of correlation if the data were normally distributed. The strength of correlation for both tests was defined as shown in Table 1. Two-group comparisons were analyzed with the Student t test. A value of P < .05 was defined as statistically significant. The primary outcome measure for this study was to determine if there is a correlation between the mean citations and citation density and the methodological quality based on 3 separate scoring systems. The secondary outcome measure was to analyze trends in methodological quality over 3 separate decades.

TABLE 1

Classification of Correlation Strength for the Pearson Correlation Coefficient (R) and Spearman Rank Coefficient (R S)

Negative Correlation	Strength	Positive Correlation	Strength
Perfect negative linear correlation	R, R S = –1	Perfect positive linear correlation	R, R S = 1
Strong negative linear correlation	–0.70 > R, R S > –1	Strong positive linear correlation	0.70 < R, R S < 1
Moderate negative linear correlation	–0.50 > R, R S > –0.70	Moderate positive linear correlation	0.50 < R, R S < 0.70
Weak negative linear correlation	–0.30 > R, R S > –0.50	Weak positive linear correlation	0.30 < R, R S < 0.50
No negative linear correlation	0 > R, R S > –0.30	No positive linear correlation	0 < R, R S < 0.30

Results

Article Data

The top 50 most cited articles on shoulder instability between 1985 and 2019 are shown in Appendix Table A1. Seven different journals were represented on the list, with the Journal of Bone and Joint Surgery–American Volume (n = 17; 34%), Arthroscopy (n = 15; 30%), and the American Journal of Sports Medicine (n = 10; 20%) having the top 3 most publications (Figure 1). The mean (±SD)  number of citations of all 50 articles was 222.7 ± 123.5 (range, 124-881.5). The mean (±SD) citation density overall (mean number of citations ÷ years since publication) was 16.0 ± 7.9.

APPENDIX TABLE A1

Top 50 Most Cited Articles in Shoulder Instability.

Rank	First Author (Year), Journal, Country	Study Type	Average No. of Citations	Citation Density	LOE	MCMS	Jadad Score	MINORS Score
1	Burkhart (2000), Arthroscopy, US	Case series	881.5	48.97	4	68	1	18
2	Boileau (2006), JBJS, France	Case series	46.95	39.13	4	69	1	18
3	Itoi (2000), JBJS, US	Cadaveric	447	24.83	5	N/A	N/A	N/A
4	Allain (1998), JBJS, France	Case series	386.5	19.33	4	53	1	15
5	Sugaya (2003), JBJS, Japan	Case-control	353.5	23.57	3	60	0	12
6	Kirkley (1998), AJSM, Canada	Review	339.5	16.98	5	N/A	N/A	N/A
7	Balg (2007), JBJS, France	Case-control	321.5	29.23	3	55	1	18
8	Burkhart (2007), Arthroscopy, US	Case series	287	26.09	4	58	1	11
9	Taylor (1997), AJSM, US	Prospective observational	281.5	13.40	2	43	1	17
10	Bigliani (1998), AJSM, US	Case series	281	14.05	4	48	1	13
11	Chandnani (1993), AJR, US	Case series	245	9.80	4	46	1	10
12	Cole (2000), JBJS, US	Case series	238	13.22	4	46	1	18
13	Gartsman (2000), JBJS, US	Case series	237.5	13.19	4	60	1	12
14	Burkhart (2002), Arthroscopy, US	Cadaveric	234	14.63	5	N/A	N/A	N/A
15	Yamamoto (2007), JSES, Japan	Cadaveric	224.5	20.41	5	N/A	N/A	N/A
16	Lo (2004), Arthroscopy, US	Cadaveric	219.5	15.68	5	N/A	N/A	N/A
17	Gerber (2002), CORR, Switzerland	Review	216.5	13.53	5	N/A	N/A	N/A
18	Bottoni (2002), AJSM, US	RCT	215	13.44	1	60	3	16
19	Kim (2003), JBJS, South Korea	Case series	209	13.93	4	67	1	18
20	Robinson (2006), JBJS, UK	Prospective cohort	204	17.00	2	72	1	18
21	Zacchilli (2010), JBJS, US	Review	196	24.50	5	N/A	N/A	N/A
22	Fabbriciani (2004), Arthroscopy, Italy	RCT	193.5	13.82	1	72	3	18
23	Mazzocca (2005), AJSM, US	Case series	189.5	14.58	4	59	1	18
24	Warner (2006), AJSM, US	Case series	189	15.75	4	51	1	17
25	Kirkley (1999), Arthroscopy, Canada	RCT	188.5	9.92	1	72	4	20
26	Purchase (2008), Arthroscopy, US	Technique	185.5	18.55	5	N/A	N/A	N/A
27	Kirkley (2005), Arthroscopy, Canada	RCT	176	13.54	1	78	3	20
28	Gill (1997), JBJS, US	Case series	175	8.33	4	67	1	17
29	Bacilla (1997), Arthroscopy, US	Prospective cohort	171	8.14	2	68	1	18
30	Itoi (2007), JBJS, Japan	RCT	171	15.55	1	77	3	18
31	Hovelius (2008), JBJS, Sweden	RCT	170.5	17.05	1	81	3	20
32	Tauber (2004), JSES, Austria	Prospective cohort	166	11.86	2	64	1	16
33	Provencher (2010), JBJS, US	Review	162	20.25	5	N/A	N/A	N/A
34	Lafosse (2007), Arthroscopy, France	Case-control	160	10.67	3	66	1	14
35	Lee (2000), JBJS, US	Cadaveric	158.5	8.81	5	N/A	N/A	N/A
36	Kim (2002), Arthroscopy, South Korea	Case-control	158.5	9.91	3	64	1	18
37	Chuang (2008), Arthroscopy, US	Case-control	155.5	15.55	3	44	1	13
38	Kwon (2005), JSES, US	Cadaveric	155.5	11.96	5	N/A	N/A	N/A
39	Hovelius (2001), JSES, Sweden	Review	153.5	9.03	5	N/A	N/A	N/A
40	Bottoni (2006), AJSM, US	RCT	152.5	12.71	1	68	3	20
41	Wilk (1997), JOSPT, US	Review	147.5	7.02	5	N/A	N/A	N/A
42	Mologne (2007), AJSM, US	Retrospective cohort	147	13.36	3	49	1	15
43	Itoi (2001), JBJS, Japan	Case series	146	8.59	4	56	1	15
44	Yiannakopoulos (2007), Arthroscopy, Greece	Case series	144.5	13.14	4	65	1	13
45	Yamamoto (2009), AJSM, US	Cadaveric	143.5	15.94	5	N/A	N/A	N/A
46	Saito (2005), AJSM, Japan	Prospective cohort	141.5	10.88	2	58	1	10
47	Griesser (2013), JSES, US	Review	140	28.00	1	N/A	N/A	N/A
48	Sugaya (2005), JBJS, Japan	Case series	138.5	10.65	4	62	1	18
49	Wolf (1998), Arthroscopy, US	Case series	137	6.85	4	52	1	13
50	Provencher (2009), Arthroscopy, US	Technique	124	140	132	1467	5	N/A

a AJR, American Journal of Roentgenology; AJSM, American Journal of Sports Medicine; CORR, Clinical Orthopaedics and Related Research; JBJS, Journal of Bone and Joint Surgery–American Volume; JOSPT, Journal of Orthopaedic & Sports Physical Therapy; JSES, Journal of Shoulder and Elbow Surgery; LOE, level of evidence; MCMS, Modified Coleman Methodology Score; MINORS, Methodological Index for Non-Randomized Studies; N/A, not applicable; RCT, randomized controlled trial.

Figure 1.

The top 50 most cited articles according to journal title.

The top 50 studies were from 11 different countries, with the United States (n = 28; 56%), Japan (n = 6; 12%), and France (n = 4; 8%) (Figure 2) producing the most articles. There was no significant difference in the average citations (P = .99) between US (n = 29; mean, 222.9 ± 142.9) and non-US (n = 21; mean, 222.5 ± 93.9) studies. There was also no significant difference (P = .97) in citation density between US (n = 29; mean, 16.0 ± 8.4) and non-US (n = 21; mean, 16.0 ± 7.3) studies.

Figure 2.

The top 50 most cited articles according to country of origin.

The most common type of study represented was the retrospective case series (n = 16; 32%) (Figure 3), which corresponds to an evidence level of 4 (Figure 4). Randomized controlled trials (RCTs), reviews, and cadaveric studies each had 7 publications (14%), corresponding to evidence levels 1, 5, and 5, respectively. The majority of articles were published between 2000 and 2009 (n = 37; 74%) (Figure 5). A breakdown of topics by decade is also shown in Figure 5. The bulk of the research conducted in the 2000s was primarily about glenohumeral bone loss (n = 17) and arthroscopic Bankart repair (n = 15).

Figure 3.

The top 50 most cited articles according to study type. RCT, randomized controlled trial.

Figure 4.

The top 50 most cited articles according to level of evidence.

Figure 5.

The top 50 most cited articles according to decade published and topic breakdown.

Methodological Quality

When analyzing methodological quality, 7 review articles, 7 cadaveric studies, and 2 technique studies were excluded (n = 16 total excluded). From the remaining 34 papers, the overall mean MCMS, Jadad, and MINORS scores were 61.1 ± 10.1 (range, 43-81), 1.4 ± 0.9 (range, 1-4), and 16.0 ± 3.0 (range, 10-20), respectively. In total, 21 studies (61.8%) had a MINORS score of higher than 16, which has widely been regarded as the cutoff for a high-quality study.[40] Methodological scores in US studies (n = 16) versus non-US studies (n = 18) were also analyzed. There was a significant difference in the MCMS in US versus non-US studies (56.1 ± 9.2 vs 66.2 ± 8.4, respectively; P = .002). However, there was no significant difference between US and non-US studies in Jadad scores (1.2 ± 0.7 vs 1.6 ± 1.1, respectively; P = .31) or MINORS scores (15.3 ± 2.9 vs 16.8 ± 2.9, respectively; P = .15).

Methodological Quality Versus Citations

Using the Kolmogorov-Smirnov test, the data were not found to be normally distributed. Based on the Spearman coefficient, there was no positive correlation found between mean citations or citation density versus level of evidence (R S = 0.20, P = .20, and R S = 0.02, P = .39, respectively). There was no correlation found between mean citations or citation density versus each of the methodological quality scores (Table 2).

TABLE 2

Correlation Between Mean Citations and Citation Density Versus Level of Evidence and Methodological Quality Scores

Comparison	Pearson Coefficient (R) or Spearman Coefficient (R S)	Sample Size (n)	P
Mean citations vs level of evidence	0.24	34	.15
Citation density vs level of evidence	0.06	34	.37
Mean citations vs MCMS	–0.12	34	.31
Citation density vs MCMS	0.03	34	.39
Mean citations vs Jadad score	–0.18	34	.22
Citation density vs Jadad score	–0.07	34	.36
Mean citations vs MINORS	–0.06	34	.37
Citation density vs MINORS	0.06	34	.38

 MCMS, Modified Coleman Methodology Score; MINORS, Methodological Index for Non-Randomized Studies.

Methodological Quality by Decade

When analyzing trends in methodological quality over time, we found no significant difference between the mean level of evidence from 1990 to 1999 (35 ± 14) and that from 2000 to 2009 (3.4 ± 1.4; P = .98). There was no significant difference between the mean MCMS from 1990 to 1999 (56.1 ± 11.2) and that from 2000 to 2009 (62.7 ± 9.4; P = .11), no significant difference between the mean Jadad from 1990 to 1999 (1.4 ± 1.1) and that from 2000 to 2009 (1.4 ± 0.9; P = .90), and no significant difference between the mean MINORS from 1990 to 1999 (15.4 ± 3.2) and that from 2000 to 2009 (16.2 ± 2.9; P = .48). The sample size for the years 2010 to 2019 was too small (n = 3) to run a proper analysis.

Discussion

This investigation determined the top 50 most cited articles regarding shoulder instability and demonstrated that there was no significant correlation between the mean number of citations or citation density and their respective level of evidence or methodological quality. The most common type of study was the retrospective case series, published in the Journal of Bone and Joint Surgery–American Volume and conducted in the United States. Arthroscopic procedures and glenohumeral bone loss were the most frequently discussed topics and were primarily published between 2000 and 2009.

A bibliometric study such as this is able to estimate trends and interest in shoulder instability worldwide based on the frequency of citations a particular article has accrued. The number of citations an article receives is based on, but not limited to, the length of time since publication, the quality of the work, the interest of the primary topic, and the application of the article to help advance research.[34] When analyzing bibliometrics, it is important to keep in mind a few biases that occur in the academic community. First, articles that are published in the early phases of a new focus may be more cited as they become foundational to the advancement of the field.[34] Additionally, a “snowball effect” has been described in the scientific community, in which authors are more likely to reference heavily cited articles in their own work.[27] Finally, the “obliteration by incorporation” phenomenon describes the process in which information from previously popularized studies becomes less frequently referenced as the pertinent findings are incorporated into present knowledge.[22] Although it is difficult to eliminate these biases in any bibliometric study, the use of a citation density measure helps provide a more accurate picture of citation counts over the time interval of interest in this study.

Our results reflect an increasing interest in referencing articles on arthroscopic and minimally invasive techniques in the treatment of shoulder instability. There were a total of 15 articles (30%), mostly published between 2000 and 2009, that analyzed arthroscopic repair within the top 50 list. As arthroscopic techniques improved over recent decades, it has been shown that arthroscopic repair of shoulder instability has similar efficacy to open Bankart repair when comparing long-term stability and complications.[8,20] Bonazza et al[7] reported that 90.5% (n = 60,248) of shoulder stabilization procedures were performed arthroscopically between 2008 and 2012.

Interestingly, 5 of the 7 (71.4%) RCTs on the current list primarily discuss arthroscopic shoulder stabilization. The most cited of these RCTs was conducted by Bottoni et al[9] in 2002 (rank, 18; mean citations, 215; citation density, 13.44), which found arthroscopic stabilization to be superior to nonoperative management in first-time anterior shoulder dislocations in reducing the rate of recurrence. The prospective RCT conducted by Bottoni et al[8] in 2006 (rank, 40; mean citations, 152.5; citation density, 12.71) directly compared the efficacy of arthroscopic stabilization versus open stabilization of recurrent shoulder instability and found no significant difference in outcomes between the 2 methods. Higher-evidence studies such as these have played a role in driving the arthroscopic Bankart repair as the procedure of choice for most cases of shoulder instability since the mid-2000s. It is important to note that there were no articles on arthroscopic Bankart repair within the top 50 most cited in the most recent decade (2010-2019). This may reflect the “obliteration by incorporation” phenomenon described above.

More recent studies have suggested an increasing trend toward addressing glenoid and humeral bone loss when treating recurrent anterior shoulder instability, as soft tissue–only procedures have unacceptably high failure rates.[11,15,21,32] In the most cited article on the list, Burkhart and De Beer[12] (rank, 1; mean citations, 881.5; citation density, 48.97) in 2000 described the necessity of addressing bony deficits in patients presenting with inverted-pear glenoid lesions to prevent recurrence. Although this study has a low level of evidence (level 4), it is heavily cited because of the introduction of a system to help identify effective treatment options based on the degree of bone loss found in the patient.[12]

Classically cited literature from the early 2000s, including the Burkhart study, suggests that glenoid augmentation procedures are typically indicated if bone loss is greater than 25%.[4,21] These bone block procedures, including the Latarjet and iliac crest, femoral head, or osteochondral grafting, have shown promising outcomes as a solution for reducing the recurrence rate over traditional arthroscopic soft tissue repair.[7,21,44,45] A systematic review of primary acute anterior shoulder dislocations found 1245 shoulders with soft tissue lesions compared with 1977 shoulders with bony defects.[29] In our list, 22 articles (44%) are directly related to glenohumeral bone loss, which was far more common than failure of soft tissue restraints, indicating a growing interest in the field to address the techniques described above to improve upon traditional soft tissue procedures.[15]

However, it is important to note that the evidence regarding these topics is found predominantly in case series, and treatment algorithms must weigh multiple risk factors for recurrence.[35] Interestingly enough, all basic science studies in our series were cadaveric studies, with 7 out of the 7 studies analyzing the glenohumeral bony anatomy. Only 1 article investigated the biomechanics of the rotator cuff musculature as a dynamic stabilizer of the glenohumeral joint. While classic literature has laid out a sturdy foundation for describing these risk factors, including the study conducted by Boileau et al[5] in 2006 (rank, 2; mean citations, 469.5; citation density, 39.13), current literature warrants an update to this treatment algorithm.[23,28,38] However, the infrequency of these lesions, the complex interaction of risk factors, and variations in surgical technique make this topic particularly challenging to study within a controlled clinical trial. Nevertheless, the accumulation of heavily cited case series in recent decades on this trending subject provides the foundation for higher-evidenced studies in the future to elucidate the roles of each of these treatment schemes in standard clinical practice.

Not surprisingly, the retrospective case series (evidence level, 4) was the most common type of study (n = 16) in our series. Most of these low-evidence articles discuss glenohumeral bone loss and signify an increasing amount of interest in bone block procedures. There was no correlation between the mean number of citations and citation density versus the MCMS, Jadad score, or MINORS score. This concurs with previous bibliometric studies done for rotator cuff repair[41] and ulnar collateral ligament surgery.[25] The majority of publications also originated from the United States, similar to trends found in rotator cuff repair (58% US),[41] spinal oncology (68% US),[16] and ulnar collateral ligament surgery (92% US).[25] Our list only had 7 RCTs (14%) that were scored for methodological quality, 6 of which investigated arthroscopic procedures. Only 2 RCTs originated from the United States, which may be because of the stricter regulations and higher costs for developing RCTs in the United States compared with other countries.[6,33,47] This was also reflected by significantly lower (P = .002) mean MCMS outcomes for US (56.1 ± 9.2) versus non-US (66.2 ± 8.4) articles.

Smeeing et al[40] used a cutoff of 16 points on the MINORS scale to delineate high-quality studies from low-quality studies in comparing randomized versus nonrandomized studies on clavicular fractures. However, multiple nonrandomized studies in our series scored above 16 points, dispelling the notion that observational studies are inherently of low evidence and cannot have high methodological quality. Most articles were published in the Journal of Bone and Joint Surgery–American Volume (36%), Arthroscopy (28%), and the American Journal of Sports Medicine (20%). These results are similar to those of bibliometric studies conducted on rotator cuff repair[41] and are congruent with the fact that these are the most-read journals regarding shoulder instability.

There were several limitations to this bibliometric review. First, our sample size was arbitrarily limited to 50 articles. Had a larger list been generated, it is possible that we could have been able to compare data from the 1990s and 2000s to the 2010s. This became more apparent when only 34 studies were able to be assessed for methodological quality. Additionally, our study only took into consideration the search results from the Scopus and Web of Science databases. Scopus is the largest database in the world, and many of our articles that were in the top 50 from Scopus were not listed in Web of Science and therefore were not included in this analysis. A final limitation to our study is the subjectivity of some of the items on the methodological grading scales. For instance, in the MCMS, there is no clear distinction between a “well-described” rehabilitation protocol and one “described without complete detail,” which is a difference of 5 points versus 3 points for that category.[37] In the MINORS scale, the first item is “a clearly stated aim” (2 points), which may be interpreted differently by various investigators.[39] While it is difficult to create a scale that can be completely objective, this ambiguity could have led to differences in scoring of the MCMS, Jadad, or MINORS between the different reviewers.

Conclusion

The top 50 most cited articles in shoulder instability comprise low-level evidence and low methodological quality. There was no significant relationship between the number of citations and the quality of the study based on 3 separate indices for assessing study methodology. Although higher-quality studies on this topic exist, they are not necessarily cited as frequently as lower-level studies.