Effectiveness of 3-dimensional shoulder ultrasound in the diagnosis of rotator cuff tears: A meta-analysis.

BACKGROUND: Numerous quantitatively based studies measuring the accuracy of 3D shoulder ultrasound (US) for the diagnosis of rotator cuff tears remain inconclusive. In order to determine how effective 3D shoulder US is for detecting rotator cuff tears, a meta-analysis was performed systematically.
METHODS: Six electronic databases, PubMed/Medline, Embase, Cochrane Library, CNKI, VIP data, and Wanfang data, were utilized to retrieve articles praising the diagnostic value of 3D shoulder US for use in detecting rotator cuff tears. After screening and diluting out the articles that met inclusion criteria to be used for statistical analysis, the pooled evaluation indexes including sensitivity, specificity, and diagnostic odds ratio (DOR) as well as the summary receiver operating characteristic curve (SROC) were calculated utilizing Meta-Disc v.1.4.
RESULTS: Screening determined that out of 4220, 7 studies involving a total of 282 patients were deemed viable for inclusion in the meta-analysis. The results of the analysis showed that the sensitivity and specificity were at 94% and 83%, respectively, with a DOR of 60.06, Q* index of 0.9058 and the area under SROC of 0.9609. Additionally, a satisfactory accuracy of 3D shoulder US was observed in detecting full- and partial-thickness rotator cuff tears.
CONCLUSION: This meta-analysis suggests that 3D shoulder US is very effective and highly accurate to detect full-thickness rotator cuff tears, but may lack accuracy in the diagnosis of partial tears.

Introduction

Rotator cuff injuries, tears, and complications lead to discomfort, pain, and motor dysfunction. Open repairs or arthroscopic surgeries are often required to treat rotator cuff tears, and more particularly full-thickness injuries.[ With an aging population, it is expected that the prevalence and severity of rotator cuff tear occurrence will increase.[ As such, diagnosis of shoulder injuries will ultimately play a crucial role for implementing appropriate treatment strategies.

Traditional diagnostic methods such as clinical physical examination tests, shoulder plain film, magnetic resonance (MRI), and magnetic resonance arthrography (MRA) are commonly used to detect the rotator cuff injuries. A recent meta-analysis, including 11 studies, was performed to assess the performance of physical examination tests of the shoulder (PETS) as a diagnostic tool for rotator cuff tears.[ This research showed that single PETS possessed a low clinical performance at diagnosing rotator cuff tears. However, when the different PETS for superior labral anterior posterior lesions were pooled, a statistical significant change in post-test probability indicating an overall statistical validity was found. Contrary to this, Brockmeyer et al[ reported that the diagnostic accuracy and clinical tests including the Jobe test (sensitivity 64.1%, specificity 43.2%) and the impingement-sign (sensitivity 76.7%, specificity 46.6%) on their own are insufficient for detecting partial-thickness tears. These conflicting reports of the various physical examination assays used to define rotator cuff tears have added confusion to shoulder examinations performed within a routine clinical setting.

In the last decade, shoulder plain film and various physical examination tests have been shown to be insufficient at effectively diagnosing rotator cuff tears.[ Through various technological advances, noninvasive imaging techniques, such as ultrasound (US) and MRI, have significantly improved the diagnostic accuracy of rotator cuff tears.[ Historically, MRI has been suggested to be extremely effective in diagnosing full-thickness rotator cuff tears. It has been considered as the favored imaging tool for preoperative examinations but its high cost and time consumption make this imaging modality not readily available, clinically.[ At the moment, because of a high sensitivity and specificity of approximately 90%, MRI is emerging as the primary choice for the presurgical diagnosis choice for patients where rotator cuff injuries are suspected.[ However, since MRIs are expensive and time consuming for revealing partial-thickness tears,[ this imaging modality is still not routinely used clinically. Indeed, MRIs have been shown to be ineffective at revealing partial-thickness tears.[ In fact, multiple studies have reported that MRA is far more diagnostically effective for detecting full- and partial-thickness rotator cuff but especially small full-thickness tears.[ A previous meta-analysis[ that utilized 65 articles suggested that MRA could provide the accuracy in detecting full-thickness tears; however, Co et al[ contradicts this finding indicating that MRA is only applied in the cases where patients possessed labral abnormalities, as MRA only improves the sensitivity and specificity by 3% to 4% when compared with either US or MRI.

US, on the other hand, has been suggested to be a more suitable and reliable diagnostic tool option at detecting rotator cuff injuries as it is economical, time saving, and readily available,[ while being on par with and perhaps even more accurate than MRI.[ Roy et al[ in a meta-analysis recently suggested that US has a high pooled sensitivity and specificity (0.96 [95% confidence interval [CI] 0.91–0.99] and 0.92 [95% CI 0.87–0.96], respectively) at identifying patients with rotator cuff tears. Subsequently, 3D US has emerged, which is showing even more reliability at diagnosing rotator cuff tears than simple US[ and also possessing a great effectiveness in the detection of other diseases.[

While various studies attempted to quantitatively measure the accuracy of 3D shoulder US for use in the diagnosis of rotator cuff tears,[ results remain inconclusive. As such, the aims of the present study were to more critically and accurately evaluate all available scientific published material, using strict inclusion and exclusion criteria, to determine the effectiveness and reliability of 3D shoulder US diagnosis for detecting rotator cuff tears when operation or arthroscopy was regarded as the gold standard.

Materials and methods

Selection, inclusion, and exclusion criteria

Published articles were retrieved utilizing 2 methods. First, 6 scientific databases were searched (PubMed/Medline, Embase, the Cochrane Library, CNKI, VIP database, and Wanfang data) utilizing the keywords “three-dimensional Ultrasound,” “3-dimensional Ultrasound,” “3D Ultrasound,” “Three-dimensional sonography,” “3-dimensional sonography,” “3D sonography AND “rotator cuff,” “supraspinatus,” “subscapular muscle,” or “subscapularis.” No language limitation was imposed. The last search was updated on June 25, 2018. Subsequently, all reference lists of relevant articles (reviews, systematic reviews, meta-analyses, included studies) were further screened, manually, to retrieve additional studies that were not listed in the databases.

Clinical studies evaluating the performance of 3D US in the diagnosis of patients with rotator cuff tears were eligible for inclusion in this meta-analysis. The articles that reported data to calculate the sensitivity and specificity were further included into this statistical analysis. For articles containing overlapping data, the one presenting the most comprehensive data or that was published recently was selected.

The exclusion criteria were letters, reviews, editorials, and other nonoriginal studies; congress proceedings; animal experiments; and articles providing no data to calculate. Two independent and blinded investigators (investigators A and B) scanned the retrieved studies using the aforementioned criteria. The articles title and abstracts were looked at and excluded if the topic was not relevant to the study. All remaining were then subsequently retrieved and reviewed in detail.

Data extraction and quality assessment

To reduce potential bias, the process of data extraction and quality assessment was conducted by again the 2 blinded and independently working investigators (FL and AT). Each article deemed appropriate for inclusion the first author's surname, year of publication, country of origin, number, age and gender of participants, ultrasonic instrument type, frequency of ultrasonic instrument use, and types of rotator cuff tears and gold standard were noted. With respect to studies eligible for inclusion in the meta-analysis, full- and partial-thickness rotator cuff tears were classified as true positive (TP), false positive (FP), true negative (TN), or false negative (FN) cases according to the true outcome verified by the gold standard. The numbers of TP, FP, TN, and FN were entered into a standardized Excel (Microsoft Corporation, Redmond, WA) sheet by a further 2 blinded investigators independently.

A quality assessment tool for the diagnostic accuracy studies (QUADAS) was used to evaluate the methodological quality of the included studies.[ This tool contains 14 items, and each one was described for 1 score. Any discrepancies were resolved by consensus from a third investigator (DZ).

Statistical analysis

This investigation was conducted based on “the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)” statement.[ Ethical approval and patient consent were not necessary, as the analysis was performed on data available within already published articles. All statistical analyses were calculated with Meta-Disc software v.1.4.[ The threshold effect was assessed using a Spearman correlation coefficient with the heterogeneity being evaluated across studies using the I2 test. An I2 < 50% was considered not significant. In order to quantitatively evaluate the value of 3D shoulder US in the diagnosis of rotator cuff tears, a random-effect model was applied to calculate the following pooled outcome estimates: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) with 95% CI. Moreover, the summary receiver operating characteristic curve (SROC) with sensitivity as the x-coordinate and 1-specificity as the y-coordinate was constructed. The Q∗ index (the point where sensitivity and specificity are equal on SROC) and AUC (the area under SROC) reflected the diagnostic accuracy of 3D shoulder US.

Results

Selection process

The detailed article search and study selection process are listed in Fig. 1. A total of 4205 articles were retrieved after the initial search of the chosen electronic databases, with 15 additional articles being identified as originating from the references lists from the relevant studies scanned for in the databases. Of the 4220 articles scanned, 4186 failed the selection criteria. Out of the 34 articles resulting from the selection process (Fig. 1), 1 article[ was a duplicate publication, another article[ was referred to an experimental study and 21 articles lacked sufficient data to calculate sensitivity and specificity and were excluded from the meta-analysis. After careful selection, eventually, 7 articles[ involving 282 participants were used for the meta-analysis.

Figure 1

Selection flow chart for studies included in the systematic review and meta-analysis.

Study characteristic and quality assessment

The detailed characteristics of the studies utilized in the meta-analysis are represented in Table 1. Articles were published in the time span between 2000 and 2015 with sample sizes ranging from 30 to 50 specimens. A total of 7 studies[ assessed 3D shoulder US for diagnosing rotator cuff tears including full- and partial-thickness rotator cuff tears. Only 1 study[ reported on 3D shoulder US for diagnosing full-thickness rotator cuff tears based on the size of tears. Three studies[ reported that they used ultrasonic instruments and produced at the GE company, and the frequency of 3D shoulder US for diagnosing rotator cuff tears was 8 to 15 MHz. For all included studies, the gold standard of diagnosing rotator cuff tears was used, which was arthroscopy or surgery. Several studies sharing overlapping participants presented data for different subgroup analyses. All diagnostic data including TP, FP, FN, and TN from original included studies are represented in Table 2.

Table 1

Main characteristics of the included studies.

Table 2

Diagnosis accuracy data of 3D ultrasound for rotator cuff tears.

The methodological quality resulted in 3 studies[ receiving a score of 11, 1 study[ receiving a score of 12, with the remaining 2 studies[ achieving an overall score of 13 according to QUADAS (Table 1).

Rotator cuff tears

Results assessing the diagnostic performance of the 3D shoulder US method for detecting rotator cuff tears in patients as generated from the 7 studies,[ involving 287 shoulders, included in the present meta-analysis showed that a sensitivity of 94% (95% CI 90–97), a specificity of 83% (95% CI 74–90), a PLR of 4.83(95% CI 1.42–16.36), an NLR of 0.11 (95% CI 0.06–0.21), and a DOR of 60.06 (95% CI 13.54–266.34) (Fig. 2). The threshold effect was not found in the provided lesion-based data (Spearman correlation coefficient = 0.270; P = .558). There was no significance between the study heterogeneity for the sensitivity (I2 = 0.00%), whereas the Q∗ index and AUC were both 0.9058 and 0.9609, respectively (Fig. 2).

Figure 2

Results of 3D shoulder ultrasound for detecting rotator cuff tears: (A) pooled sensitivity, (B) pooled specificity, (C) pooled diagnostic odds ratio, and (D) summary receiver operating characteristic curve (SROC) with the Q∗ index.

Full-thickness rotator cuff tears

Performance assessing the diagnostic performance of the 3D shoulder US method for detecting full-thickness rotator cuff tears in patients as generated from the 7 studies,[ involving 287 shoulders, included in the present meta-analysis, showed a sensitivity of 91% (95% CI 85–95), a specificity of 96 (95% CI 91–98), a PLR of 13.64 (95% CI 6.34–29.35), an NLR of 0.12 (95% CI 0.07–0.19), and a DOR of 146.58 (95% CI 49.17–436.91) (Fig. 3). The threshold effect was not found in the provided lesion-based data (Spearman correlation coefficient = 0.054; P = .908). There was no significant difference between-study heterogeneity for the sensitivity (I2 = 0.00%), whereas the Q∗ index and AUC were both 0.9202 and 0.9702, respectively (Fig. 3).

Figure 3

Results of 3D shoulder ultrasound for detecting full-thickness rotator cuff tears: (A) pooled sensitivity, (B) pooled specificity, (C) pooled diagnostic odds ratio, and (D) SROC with the Q∗ index. SROC = summary receiver operating characteristic curve.

Partial-thickness rotator cuff tears

Results of the 3D shoulder US for detecting partial-thickness rotator cuff tears analysis was conducted. A total of 7 studies,[ involving 287 shoulders, were available to investigate the diagnostic performance of 3D shoulder US in partial-thickness rotator cuff tears. The threshold effect was not found in the provided lesion-based data (Spearman correlation coefficient = −0.037; P = .937). The pooled results were as follows: sensitivity of 82% (95% CI 69–91), specificity of 88 (95% CI 83–92), PLR of 6.30 (95% CI 3.31–11.99), NLR of 0.26 (95% CI 0.12–0.56), and DOR of 27.56 (95% CI 10.27–73.98) (Fig. 4). There was no significant difference between-study heterogeneity for the sensitivity (I2 = 0.00%). Meanwhile, the Q∗ index and AUC were 0.9203 and 0.8537, respectively (Fig. 4).

Figure 4

Results of 3D shoulder ultrasound for detecting partial-thickness rotator cuff tears: (A) pooled sensitivity, (B) pooled specificity, (C) pooled diagnostic odds ratio, and (D) summary receiver operating characteristic curve (SROC) with the Q∗ index. SROC = summary receiver operating characteristic curve.

Based on size of full-thickness rotator cuff tears

Performance assessing the diagnostic performance of the 3D shoulder US method for detecting sizes of rotator cuff tears was performed as generated from only 1 study,[ involving 50 patients, included in the present meta-analysis demonstrated that the sensitivity was 46.7% (7/15) for large rotator cuff tears, 77.8% (14/18) for medium rotator cuff tears, and 100% (2/2) for small rotator cuff tears, respectively. As the number of study involved a small amount of patients, this result should be interpreted cautiously.

Discussion

Our meta-analysis from multiple studies has shown that US has a sensitivity of 86% to 100% and a specificity of 67% to 98% in the detection of full-thickness rotator cuff tears.[ Various diagnostic values of US for rotator cuff tears are highly dependent on operator experience and image quality of machines with high resolution. A prospective comparative study[ demonstrated that high sensitivity and specificity of high-resolution US, when compared with MRI for diagnosis of both full- and partial-thickness rotator cuff tears, had an accuracy of 98.4% for full-thickness and 95.9% for partial-thickness rotator cuff tears.

Currently, the 3D tomographic technique of 3D US presents both the relevant information of US and the relative position of sections, which contribute in decreasing the subjectivity of the operator. Three-dimensional image reconstructions sharpen quality of the tear anatomy via multiplanar display making this technique very helpful for evaluating complex anatomical situations of rotator cuff tears. Numerous studies[ reported that the sensitivity and specificity of 3D US for detecting rotator cuff tears were 77% to 100% and 50% to 90%, respectively. A study by Kijima et al[ was the first to quantitatively demonstrate the reproducibility of 3D US in evaluating the configuration of rotator cuff lesions with a concordance rate of 91.4%. Experimental studies have also shown that rotator cuff lesions, especially partial-thickness rotator cuff tears, were more often correctly diagnosed with 3D US than with conventional 2D US, where the sensitivities of each method was at 77% and 85%, respectively.[ In this meta-analysis involving the 287 3D US examinations of shoulders that received subsequent confirmation from MRI examination, surgical operation or shoulder arthroscopy for rotator cuff tears injuries, the sensitivity and specificity of 3D US for full-thickness rotator cuff tears was at 91% and 96%, with partial-thickness tears at 82% and 88%, respectively.

Since the size of the rotator cuff tear affects the choice of surgical repair,[ it is important to classify tears as small, medium, large, or massive. A clinical study[ with 85 patients (4 small, 56 moderate, 17 large, and 8 massive) was performed to examine the accuracy of external rotation in neutral (0° external position) and in shortened position (45° external position) in relation to rotator cuff tear size using MRI and arthroscopic surgery as the gold standards. Here results showed that the isometric external rotation is an accurate test in diagnosing rotator cuff tears especially tears of the infraspinatus muscle. The isometric strength at the shortened position was a better predictor of clinical, surgical, and imaging findings. Only a few studies have compared the accuracies with which US and MR quantify the size of examinations and 8 FN examinations were included in this study. The results showed a sensitivity of 46.7% (7/15), a specificity of 100% (19/19), and an accuracy of 74.3% (26/35). In the current presented study, based on the sizes of full-thickness rotator cuff tears, only 1 study demonstrated a sensitivity of 65.7% (23/35), a specificity of 100% (15/15), a PLR of 65.7%, an NLR of 34.3%, and an accuracy of 74%. It is unclear if more studies of this nature exist and if they do, our detailed screening could not localize them. As such, these results are not decisive and must be substantiated by more research.

Although satisfactory results have been demonstrated, considering the operators of 3D shoulder US, FP, and FN cases are unavoidable. Numerous factors may bring misdiagnoses of 3D US for rotator cuff tears. First, methodological variability, such as different linear arrays administered for 3D US, may affect the conclusions of our analysis. Besides the diagnostic criteria for the evaluation of rotators cuff tears and types of rotator cuff tears varied in these studies meaning that heterogeneity was unavoidable. Second, some characteristics of the participants, such as gender, age, inclusion intervals, were not provided by the literature. Third, because some measures such as partial-thickness rotator cuff tears were provided in only a few studies, this could result in an overestimation or/and underestimation of the related data. Finally, several subgroup analyses were based on a small number of studies or were impossible because of incomplete data, in particular for size-based analysis of rotator cuff tears, which might influence statistical algorithms and their deductions.

Conclusion

In conclusion, the present investigation indicated that 3D shoulder US has a high sensitivity for identifying full-thickness rotator cuff tears in patients. However, the diagnostic accuracy of 3D US remains limited in detecting partial-thickness rotator cuff tears. This suggests that, while 3D US can be used to detect full-thickness tears, the occurrence of partial tears must still be accessed through MRI examination or arthroscopy when such injuries are suspected. In this regard, large-scale, randomized, prospective trials are required to determine the effectiveness and efficiency of 3D US diagnostic systems in detecting all forms of rotator cuff tears, thereby substantially improving present diagnostic protocols.

Author contributions

Conceptualization: Aiping Teng, Fanxiao Liu.

Data curation: Aiping Teng, Fanxiao Liu, Dongsheng Zhou.

Formal analysis: Fanxiao Liu, Dongsheng Zhou.

Investigation: Fanxiao Liu.

Methodology: Aiping Teng, Fanxiao Liu.

Project administration: Aiping Teng, Fanxiao Liu.

Resources: Aiping Teng, Fanxiao Liu.

Software: Aiping Teng, Fanxiao Liu, Dongsheng Zhou, Tao He.

Validation: Aiping Teng, Fanxiao Liu.

Visualization: Fanxiao Liu.

Writing – original draft: Aiping Teng, Fanxiao Liu, Yan Chevalier, Roland M. Klar.

Writing – review & editing: Fanxiao Liu, Yan Chevalier, Roland M. Klar.