Diagnostic Performance of Diffusion-weighted Magnetic Resonance Imaging in Bone Malignancy: Evidence From a Meta-Analysis.

Current state-of-the-art nuclear medicine imaging methods (such as PET/CT or bone scintigraphy) may have insufficient sensitivity for predicting bone tumor, and substantial exposure to ionizing radiation is associated with the risk of secondary cancer development. Diffusion-weighted MRI (DW-MRI) is radiation free and requires no intravenous contrast media, and hence is more suitable for population groups that are vulnerable to ionizing radiation and/or impaired renal functions. This meta-analysis was conducted to investigate whether whole-body DW-MRI is a viable means in differentiating bone malignancy. Medline and Embase databases were searched from their inception to May 2015 without language restriction for studies evaluating DW-MRI for detection of bone lesions. Methodological quality was assessed by the quality assessment of diagnostic studies (QUADAS-2) instrument. Sensitivities, specificities, diagnostic odds ratio (DOR), and areas under the curve (AUC) were used as measures of the diagnostic accuracy. We combined the effects by using the random-effects mode. Potential threshold effects and publication bias were investigated. We included data from 32 studies with 1507 patients. The pooled sensitivity, specificity, and AUC were 0.95 (95% CI, 0.90-0.97), 0.92 (95% CI, 0.88-0.95), and 0.98 on a per-patient basis, and they were 0.91 (95% CI, 0.87-0.94), 0.94 (95% CI, 0.90-0.96), and 0.97 on a per-lesion basis. In subgroup analysis, there is no statistical significance found in the sensitivity and specificity of using DWI only and DWI combined with other morphological or functional imaging sequence in both basis (P > 0.05). A b value of 750 to 1000 s/mm enables higher AUC and DOR for whole-body imaging purpose when compared with other values in both basis either (P < 0.01). The ROC space did not show a curvilinear trend of points and a threshold effect was not observed. According to the Deek's plots, there was no publication bias on both basis. Our results support the use of DWI as an effective means for distinguishing malignant bone lesions; however, various imaging parameters need to be standardized prior to its broad use in clinical practice.

INTRODUCTION

Malignant tumors are associated with poor clinical outcomes and high morbidity and mortality,[1-4] as compared with benign tumors. Hence, an effective means for differentiating between malignant tumors and benign tumors is crucial for accurate diagnosis. Whole-body technetium-99m methyldiphosphonate (Tc-99m MDP) bone scintigraphy (BS) is one of the most commonly practiced methods for suspected bone lesions, especially for patients with pain symptom in follow-up visits, and it remains a reference for oncologists.[5] However, the main limitation of BS is the fact that detection for new lesion becomes difficult for patients already exhibiting elevated Tc-99m MDP uptake, which subsequently affects its accuracy in predicting bone tumor. Magnetic resonance imaging (MRI) and positron emission tomography (PET) have shown potential in bone tumor diagnosis, but their replacement of BS is still debatable.[6]

PET/CT has combined benefit of PET's sensitivity and CT's anatomical information, and metabolic tracers such as [18F]2-fluoro-2-d-glucose (FDG) can show the elevated glucose uptake and cellular metabolism within tumors. PET/CT has superior sensitivity in detecting bone metastases than other imaging modalities,[7] however is insensitive in detecting bone marrow involvement.[8] Recently, European guidelines addressing PET/CT in bony tumors has concluded that “the role of PET-CT in monitoring bone lesions has been reported in a few small studies and appears potentially promising; however, prospective trials are needed to establish its true clinical utility.”[9] The major pitfall for PET/CT is associated with ionizing radiation exposure. With the recent decision to end the National Oncology PET Registry, use of PET/CT for routine surveillance is now clearly not recommended by Centers for Medicare and Medicaid Services (CMS).[10] American Society of Clinical Oncology (ASCO), the European Union of Urology, the European Society for Medical Oncology (ESMO),[11] and NCCN have all declined to include surveillance PET examinations in disease-specific guidelines. Moreover, ionizing radiation can also be produced by bone scintigraphy which is the same as the PET/CT. As for the amount of ionizing radiation, US annual per capita radiation dose increased from 0.1 mSv in 1980 to 0.77 mSv in 2006 from the source of nuclear medicine.[12] According to the BEIR report VII, exposure to ionizing radiation causes roughly a tripling effect in lifetime cancer risk by comparison with a person without exposure above the age of 30 years.[13]

Diffusion-weighted imaging (DWI) is a technique that probes the level of water molecule diffusion within the microstructures in tissues, and is sensitive to local pathological alterations. Over years, DWI has found a broad range of applications both in neuro imaging and in body imaging; moreover, it has gained much attention in tumor imaging due to both its outstanding sensitivity and specificity as well as the absence of contrast media administration, which is important for patients with impaired renal functions. The level of diffusion is controlled by the diffusion-sensitizing coefficient so called b-value, and diffusion acquisition at two distinctive b-values (zero or nonzero) allows the derivation of the apparent diffusion coefficient (ADC, in the unit of s/mm2). ADC value is a quantitative measure of water movement in tumors: low ADC values indicate an abundance of cell membranes, whereas high ADC values are indicative of cellular regions. Hence, DWI may be either qualitatively inspected or quantitatively assessed based on calculated ADC values.[14] Whole-body DW-MRI has recently become practical due to technological advances, and it emerged as a promising bone marrow assessment tool for detection of both primary cancer or distant metastasis of bone.[15,16] The added diagnostic value of DWI has been reported in several studies; there are consistent findings showing that it could have a comparable or better performance in diagnosing bone tumors, as compared with BS[17-22] or PET/CT[23-25].

Since various studies predicting the accuracy of DWI in detecting bone tumors have been published, results of these studies are drastically diverse because of the differing DWI protocol used. Here, we performed an updated meta-analysis to investigate the diagnostic value of DW-MRI as a standalone method in bone lesions screening. Moreover, we intended to compare the average adjusted accuracies of DWI between different DWI sequences, analysis methods, b values, and covariate that may affect the effectiveness of modalities.

METHODS

We did a meta-analysis in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [26] and the guidelines described in the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.

Search Strategy and Selection Criteria

We searched for the relevant studies (Table S1, http://links.lww.com/MD/A514) in the online database of EMBASE, PubMed from the date of their inception up to May 2015 with the assistance of a librarian. No language restriction was placed. The reference in all the retrieved articles was also searched for any additional relevant studies. A radiologist and an oncologist were asked to look through these literatures and assess their eligibility for analysis.

The inclusion criteria included: studies that assessed the sensitivity, specificity, and other metrics assessing the diagnostic performance of DWI, among which systematic reviews and meta-analyses were used only as a source of references; studies that validated the performance of DWI in cancer diagnosis and showed that all participants had the reference tests; studies that assessed primary bone tumors or bone metastasis; and studies based on which the true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) were able to be calculated on the basis of sensitivity and specificity in respective publications. Conference abstracts were also included when they contained relevant published data or relevant unpublished data if could be traced from the authors. We excluded all studies that could be classified as narrative reviews, letters, editorials, comments, and case reports, and surveillance of the response of chemoradiotherapy in patients with cancers. A total of 32 studies were finalized, any disagreement between them was resolved by discussing with a third party. The inclusion of all the studies based on the above criteria was done in 2 stages: in the first stage the inclusion was based on title and abstract; and in the second stage, the full texts were considered. The literature flow diagram is shown in the Appendix as PRISMA flowchart.

Quality Assessment

The quality of the selected studies and the potential bias were assessed using the prespecified QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) guideline, including additional items as recommended by the Cochrane Collaboration.[27] This quality assessment procedure was independently performed by 2 pairs of reviewers and was checked by a fifth reviewer. Any disagreements were resolved by discussion involving all researchers when necessary. The reference standard was validated by a clinical review committee consisting of 3 researchers.

Data Extraction

Two reviewers independently extracted relevant data from the selected studies in a standard form, a third investigator checked the extracted data, and a fourth investigator arbitrated on discrepancies between the first 2 investigators. Any identified discrepancies were discussed and corrected. Two-by-two contingency tables were constructed based on the data published, summarizing TP, FP, TN, and FN on the basis of sensitivity and specificity in respective papers. Moreover, if various kinds of sequences (DWI sequence only vs. DWI combined with other functional imaging sequences) were available in same papers, we placed them in our study separately and made a subgroup analysis of each type in all studies. In the publications, either the number of patients or the number of lesions was used for the statistical analyses; we conducted separate analyses for each category to avoid any potential inconsistency.

Statistical Analysis

A random effects model was performed for the primary meta-analysis using a nonlinear mixed model approach. The primary objective was to estimate the sensitivity and specificity, positive and negative likelihood ratios (LR+ and LR–), and diagnostic odds ratio (DOR) with 95% confidence intervals (CIs) of DW-MRI for the diagnosis of bone lesions. We assumed bivariate normal distributions for sensitivity and specificity and presented a forest plot. LR+ and LR– are metrics derived from the summarized sensitivity and specificity for assessing the discriminating ability of the imaging modality.[28,29] If the LR+ is >5.0 and the LR– is <0.2, then the test can both rule in and rule out the disease. DORs were calculated for the discriminating ability of the imaging methods. The value ranged from 0 to infinity, higher values indicate better discriminatory test performances. The receiver-operator characteristic (SROC) graph analyzes the pooled accuracy, and each data point comes from the different studies. SROC curve is then formed based on these points to form a smooth curve. The area under the SROC curve (AUC) was estimated for the diagnostic accuracy of each imaging method. An AUC that is >0.5 and closer to 1.0 implies better accuracy.

The presence of heterogeneity was assessed using a fixed-effect meta-regression and I2 statistics.[30]I2 over 0.50 indicates heterogeneous, while P < 0.05 was considered having heterogeneity in likelihood ratio χ2 test. We assessed publication bias by Deeks’ plots.[31] We analyzed data separately on a per-lesion and on a per-patient basis. We also performed separate analyses in which MR was performed with DWI only or DWI combined with other functional sequences. Furthermore, we performed separate analyses for the subset of b value (750–1000 s/mm2 or other values). We also performed meta-analyses for within-study comparisons on the reference standard, factors related to study design (prospective or retrospective), consecutive enrollment and operation interval. All tests were 2-sided with a type I error of 0.05. All analyses were performed using the software StataSE version 12 (StataCorp) and MetaDisc (Version 1.4).

Ethics

All the data involved in this meta-analysis study were from sourced respective publications, which had their own ethic approval in accordance with the local ethic committee's guideline. Hence, no separate ethical committee approval is needed for this study.

RESULTS

Literature Searches

A total of 491 publications were reviewed. The filtering process for the publications is shown in the flowchart in Figure 1A, 459 publications were excluded after primary and subsequent reviewing. In the end, 32 papers involved 1507 patients, were included in this meta-analysis.[17-25,32-54] In addition, 8 papers[17,18,34,39,43,44,48,53] consisted of both per-lesion and per-patient based analyses, contributing additional 10 studies. Finally, 23 studies reported on a per-lesion basis and 19 studies on a per-patient basis met the inclusion criteria of our research. The detailed baseline characteristics are listed in Table 1.

FIGURE 1

Summary of methodological quality of included studies on the basis of QUADAS-2 checklist for each study.

TABLE 1

Baseline Characteristics of the Included Studies

Quality Assessment of Published Studies

The quality of the included studies was assessed by the QUADAS-2 tool (Fig. 1B). Discriminations were primarily found in the domain of “Reference Standard” and “Flow and Timing” for all studies. Consequently, we selected these domains (reference standard and operation interval) as covariate in meta-regression and performed separate analyses on the subset of studies.

Overall Sensitivity, Specificity, LRs, and SROC Curves

For the assessment of efficacy of DWI in bone neoplasms (with a 95% CI reported in the included individual studies), the detailed sensitivity and specificity values on a per-lesion and per-patient basis are illustrated by the forest plot as shown in Figure 2A and B. On a per-patient basis, the pooled sensitivity, specificity were 0.95 (95% CI, 0.90–0.97) and 0.92 (95% CI, 0.88–0.95) respectively. Summary estimates indicated DOR value was 207 (95% CI, 82–523), LR+ was 11.8 (95% CI, 7.5, 18.6), and LR– was 0.06 (95% CI, 0.03, 0.11). The SROC curve was symmetric, and the AUC value was 0.98 (95% CI, 0.96–0.99) (Fig. 3A). These results showed that DWI provides excellent diagnostic accuracy in differentiating bone lesions on a per-patient basis.

FIGURE 2

Forest plot of sensitivities and specificities on per-patient basis and per-lesion basis for the diagnosis of bone maligancy. A, Sensitivity and specificity for per-patient basis. B, Sensitivity and specificity for per-lesion basis. Each solid square represents an eligible study (error bars represent 95% CI).

FIGURE 3

SROC curves and area under the curve of per-patient basis (A) and per-lesion basis (B) in the diagnosis of bone malignancy.

On a per-lesion basis, the pooled sensitivity and specificity were 0.91 (95% CI, 0.87–0.94) and 0.94 (95% CI, 0.90–0.96). Summary estimates showed that DOR value was 149 (95% CI, 88–251), LR+ was 14. 4 (95% CI, 9.1, 26.6), and LR– was 0.10 (95% CI, 0.07, 0.14). The SROC curves were symmetric and the AUC value was 0.97 (95% CI, 0.96–0.99) (Fig. 3B). According to evaluation mentioned above, eDWI has excellent ability to both confirm and exclude presence of bone malignancy on a per-lesion basis.

The ROC space did not illustrate a curvilinear trend of points and Spearman's correlation coefficient was −0.04 (P = 0.87) on per-patient and 0.4 (P = 0.06) for a per-lesion basis. It was suggested that there was no presence of a threshold effect.

Heterogeneity and Publication Bias

eHeterogeneity was observed in sensitivity (I2 = 63.8 P < 0.01) and specificity (I2 = 58.3 P < 0.01) on a per-patient basis; also in sensitivity (I2 = 84.2 P < 0.01) and specificity (I2 = 92.8 P < 0.01) on a per-lesion basis. This result was validated by the I2 and Cochran Q tests. Hence, the diagnostic indices were calculated using a random effect model. The results of multivariate meta-regression analysis (Table 2) demonstrated that there was significant heterogeneity in covariate of b value on per-lesion basis, while no significant heterogeneity in any covariate on per-patient basis.

TABLE 2

Results of the Multivariable Meta-Regression Model for the Characteristics With Backward Regression Analysis (Inverse Variance Weights; Variables Were Retained in the Regression Model if P < 0.05)

I2 tests also enabled us to detect heterogeneity caused by covariate of b value (I2 = 60.0) on a per-patient basis, while covariates of standard reference (I2 = 52.3) and b value (I2 = 76.7) were responsible for the heterogeneity on a per-lesion based analysis.

The Deeks’ funnel plots were generated to assess the evidence of bias toward studies (Fig. 4). According to the plot, there was no conclusive evidence of publication bias on per-patient basis (P = 0.30) and per-lesion basis (P = 0.24).

FIGURE 4

Linear regression test of funnel plot asymmetry on per-patient basis (A) and per-lesion basis (B). Each solid circle represents a study in this meta-analysis. The statistically nonsignificant P values of 0.30 (A) and 0.30 (B) for the slope coefficient suggest symmetry in the data.

Subgroup Analysis

We performed subgroup analysis to estimate the level of the effect by classifying studies in each covariate. The values of average adjusted sensitivity, specificity, LR+, LR−, DOR, and AUC of SROC curve were calculated from meta-mathematical models, which are shown in Tables 3 and 4.

TABLE 3

Quantitative Subgroup Analysis of All Available Covariate on a Per-Patient Basis

TABLE 4

Quantitative Subgroup Analysis of All Available Covariate on a Per-Lesion Basis

According to Table 3, the results showed that the DOR and AUC in studies by qualitative analysis was significantly higher than studies that underwent quantitative analysis (combined with ADC value) on per-patient and per-lesion basis (both are P < 0.01).

DWI is now regarded as an adjunct to conventional MRI protocol, but would other functional imaging sequences help to further improve accuracy? The average adjusted sensitivity and specificity for MR with DWI only was 0.94 (95% CI, 0.89–0.97) and 0.89 (95% CI, 0.85–0.92), while those for DWI combined with other sequences were 0.91 (95% CI, 0.87–0.95) and 0.91 (95% CI, 0.88–0.94), the average adjusted AUC were 0.9701 and 0.9545 for DWI only and DWI combined with other sequences on a per-patient basis. When it comes to per-lesion basis, average adjusted sensitivity and specificity for DWI only were 0.90 (95% CI, 0.87–0.93) and 0.96 (95% CI, 0.95–0.97), while those for DWI combined with other sequences were 0.87 (95% CI, 0.85–0.90) and 0.94 (95% CI, 0.93–0.95), the average adjusted AUC were 0.9613 and 0.9719 for DWI only and DWI with other sequences on a per-lesion basis.

According to previous studies, we summarized b values that may be used as a guide when performing DWI for qualitative assessment and a b value in the range of 750 to 1000 s/mm2 may be optimal for whole-body DWI. Thus, we conducted a subgroup analysis to compare the b value of 750 to 1000 s/mm2 with other b values, pooled sensitivity, specificity, and AUC for b value of 750 to 1000 s/mm2 were 0.91 (95% CI, 0.87–0.94), 0.90 (95% CI, 0.87–0.92), and 0.9535 on a per-patient basis, while those for other b values were 0.99 (95% CI, 0.94–1.00), 0.91 (95% CI, 0.84–0.96), and 0.9968, respectively. On a per-lesion basis, pooled sensitivity, specificity, and AUC of b value for 750 to 1000 s/mm2 were 0.90 (95% CI, 0.88–0.92), 0.96 (95% CI, 0.95–0.97), and 0.9770, while those for other values were 0.85 (0.81–0.88), 0.83 (0.80–0.86), and 0.9279.

DISCUSSION

This updated study, to our knowledge, involves the largest number of patients and most comprehensive subgroup analysis in the field of whole-body DWI in differentiating bone tumors. Our findings suggest that radiation-free DWI showed a good diagnostic value on both per-patient and per-lesion-based analysis; hence, it may function as an alternative to the current nuclear medicine approach in differentiating benign from malignant bone tumors.

Despite the wide use of Tc-99m MDP BS, its use as an independent method for bony lesions is far from ideal due to lack of accuracy.[6] Another currently practiced method PET/CT also has its limitation in the detection of bone-marrow disease, as the high cellularity of normal bone marrow can be misdiagnosed as diffuse tumor infiltration or mask tumor deposits.[55] Hence, a radiation-free imaging modality with validated diagnostic accuracy is much needed for diagnosis of bone tumor.

Two previous studies indicated that the performance of DWI is similar to PET/CT, both being significantly accurate than BS in detection of bone lesion on both per-patient and per-lesion basis.[56,57] A recently published meta-analysis by Li et al showed that whole-body DWI featured similar level of sensitivity (0.897 vs. 0.895) and specificity (0.954 vs. 0.957) to PET/CT for osseous lesions detection.[56] In their study, diagnostic accuracy of DWI was compared with PET/CT in various kinds of primary and metastatic malignancies, but only a few cases were available in bone lesions. Limited sample size may impair the validity and accuracy; furthermore, only studies published in English were included which might also induce the “Tower of Babel” bias of their results. An earlier meta-analysis, which was published in 2011, included 11 studies with 495 patients, and the results indicated that whole-body DWI had a pooled sensitivity of 0.899 and a pooled specificity of 0.918.[57] Our updated study, by contrast, included a larger number of samples and reached similar conclusion to theirs.

The accuracy of PET/CT and scintigraphy in bone tumors differentiation can be learned from recent publications. A meta-analysis performed by Shen GH et al, including 12 published studies, provided an overview of the literature on the value of BS and PET/CT for monitoring the bone lesions.[58] The pooled sensitivity, specificity, and AUC for BS were, respectively, 0.79, 0.82, and 0.89 on a per-patient analysis, and respectively 0.59, 0.75, and 0.77 on a per-lesion analysis. The results of sensitivity, specificity, and AUC for PET/CT were 0.87, 0.97, and 0.95 on a per-patient analysis, while 0.83, 0.95, and 0.9494 on a per-lesion analysis. More recently, another meta-analysis conducted by Shen CT et al, including 20 articles and 1170 patients, indicated that pooled sensitivity, specificity, and AUC for BS were 0.88, 0.80, and 0.90 on a per-patient basis analysis (accuracy on a per-lesion level was not documented). Meanwhile, those for PET/CT were 0.92, 0.93, and 0.98 on a per-patient basis, and 0.87, 0.95, and 0.98 on a per-lesion analysis.[59] Both of the studies indicated a considerably lower sensitivity, specificity, and AUC of BS when compared with DWI according to our finding (P < 0.01). Although the specificity of DWI may be inferior to that of PET/CT, DWI featured a more superior sensitivity to PET/CT in screening bone tumor (both P < 0.01), and current state-of-the-art nuclear medicine imaging methods may feature insufficient sensitivity for bone tumor in general. Our results indicated that DWI may be considered a potential alternative to BS or PET/CT in screening suspicious bone malignancy for its superior sensitivity.

A factor that may affect the diagnostic accuracy of DWI and hence deriving away from the current conclusion is the different imaging protocol setup and equipment used in the different studies surveyed. Hence, we attempted to take into consideration of the varying DWI imaging conditions and to assess the level of subsequent impacts by classifying studies in each covariate with the aim of refining the validity of our research.

We assessed whether DWI combined other functional imaging sequences showed higher diagnostic accuracy than DWI alone in our stratified analysis. After adjusting for different subgroups, results showed that both of types of methods showed a similar high DOR value. Hence, DWI could function as an independent method in detecting bone malignancy.

At the present, only very limited number of research had demonstrated the predictive value of quantitative analysis with ADC value. Padhani et al found that ADC values in osseous metastasis at a cutoff greater than 0.77 × 10−3 mm2/s resulted in a sensitivity of 0.85 and specificity of 0.90 for bone metastases differentiating from benign lesions, which was similar to our finding in quantitative analysis subgroup.[60] Compared with osseous metastasis, primary malignant bone tumors are rare and traditionally best assessed with conventional radiography for initial characterization and determination of the location of a biopsy. According to the study performed by Hayashida et al, functional imaging with ADC map alone may not be helpful for differentiating malignant tumors from benign lesions in the diagnosis of primary bone tumors.[61] According to the result of subgroup analysis, we observed that quantitative approach (based on ADC value) was slightly inferior to qualitative approach in diagnosing bone tumors. However, we noted that there were a relatively small number of quantitative approach-based studies compared with the qualitative approach-based counterparts, which may render the value of quantitative approach as assessed in this study.

In addition, DWI should be performed with appropriate choices of b values taking considerations of factors including anatomic region, tissue composition, and pathologic processes. Our meta-analysis showed that the b value in the range of 750 to 1000 s/mm2 features higher accuracy, which is in agreement with consensus on International Society for Magnetic Resonance in Medicine Meeting.[62]

Although most of the studies included in this meta-analysis study were of good quality on the basis of QUADAS-2 criteria, heterogeneity was observed in the sensitivity and specificity in our study. A detailed meta-regression and subgroup analysis was performed to identify the potential source across studies. We expect that identification of the factors that led to the heterogeneity may strengthen the validation of our results, and the factors that influence diagnostic accuracy may help to optimize the design of future research.

We reviewed the diagnostic accuracy of DWI according to the updated methods for diagnostic meta-analyses. Despite our findings supporting high accuracy of DWI, limitations of the study affect the current strength of the evidence due to various sources of the data involved.

First, heterogeneous results of our study may affect the reliability of the conclusions. According to meta-regression analysis, although covariates of b value had been found as the source of the heterogeneity, we could not specify the source on a per-patient basis. Second, we compared diagnostic accuracy of DWI in our research to those of BS and PET/CT in other meta-analysis; it remains relatively inconclusive that DWI is superior to BS or PET/CT in diagnosing bone tumors. Third, most of the studies could not apply the histopathological diagnosis of bone lesions for ethical reasons because biopsies of suspected bone lesions are not part of routine examination. Therefore, part of studies used multiple imaging modalities and/or follow-up as the standards of reference instead of histopathology. Other potential limitations may be attributed to the optimization of parameters of the imaging modalities that are lack of consensus in our study.

In conclusion, our results potentially support the use of DWI as an effective method for distinguishing malignant from benign bone lesions, and utilization of radiation-free DWI may dramatically benefit population that is vulnerable to ionizing radiation. Ability to provide morphological and functional information in a single scan makes DWI attractive and promising in the diagnosis of bone tumor. Additional effort is needed for the imaging protocol standardization of DWI to achieve quality assurance.