Agreement and differentiation of intradural spinal cord lesions in dogs using magnetic resonance imaging.

BACKGROUND: Magnetic resonance imaging is the method of choice for diagnosing spinal cord neoplasia, but the accuracy of designating the relationship of a neoplasm to the meninges and agreement among observers is unknown.
OBJECTIVES: To determine agreement among observers and accuracy of diagnosis compared with histology when diagnosing lesion location based on relationship to the meninges. ANIMALS: Magnetic resonance images from 53 dogs with intradural extramedullary and intramedullary spinal neoplasms and 17 dogs with degenerative myelopathy.
METHODS: Six observers were supplied with 2 sets of 35 images at different time points and asked to designate lesion location. Agreement in each set was analyzed using kappa (κ) statistics. We tabulated total correct allocations and calculated sensitivity, specificity, and likelihood ratios for location designation from images compared with known histologic location for lesions confined to 1 location only.
RESULTS: Agreement in the first set of images was moderate (κ = 0.51; 95% confidence interval [CI], 0.43-0.58) and in the second, substantial (κ = 0.69; 95% CI, 0.66-0.79). In the accuracy study, 180 (75%) of the 240 diagnostic calls were correct. Sensitivity and specificity were moderate to high for all compartments, except poor sensitivity was found for intradural extramedullary lesions. Positive likelihood ratios were high for intradural extramedullary lesions and degenerative myelopathy. CONCLUSIONS AND CLINICAL IMPORTANCE: Overall accuracy in diagnosis was reasonable, and positive diagnostic calls for intradural extramedullary lesions and negative calls for intramedullary lesions are likely to be helpful. Observers exhibited considerable disagreement in designation of lesions relationship to the meninges.

confidence intervals

degenerative myelopathy

extradural

intradural‐extramedullary

intramedullary

kappa

magnetic resonance imaging

INTRODUCTION

Neoplasms affecting the spinal cord may be classified based on their anatomic relationship with the meninges and can be defined as intradural or extradural (ED); intradural tumors can be further classified as intradural extramedullary (IDEM) or intramedullary (IM). In dogs, approximately 50% of neoplasms affecting the spinal cord are intradural: approximately 35% IDEM and approximately 15% IM. The remaining 50% of neoplasms originate in the ED space.

Magnetic resonance imaging (MRI) plays a key role in diagnosis of spinal cord neoplasia because of its excellent contrast resolution. Signal intensity, degree of contrast enhancement, and presence of fluid‐filled compartments are important characteristics used to differentiate various spinal cord neoplasms. , , , Moreover, the location of a neoplasm in relation to the meninges also helps predict the histological type of spinal cord neoplasm.

Intradural extramedullary tumors include meningiomas, nerve sheath tumors, and nephroblastomas. , , Intramedullary tumors include oligodendrogliomas, astrocytomas, and ependymomas. , Some neoplasms, such as nephroblastoma, have been reported to occupy both IDEM and IM locations. Cytoreductive surgery is considered the primary treatment for patients with spinal cord neoplasia and may be used alone or in conjunction with radiation therapy and chemotherapy. Long‐term prognosis for spinal cord neoplasms is variable and depends on histological type, extent of neoplastic infiltration, degree of cytoreduction, neurological function of the patient before and after surgery, and surgeon experience. Cytoreductive surgery is more commonly pursued for ED and IDEM neoplasms. However, cytoreductive surgery is uncommonly performed on IM neoplasms, which likely reflects the technical expertise needed to resect neoplasms within the spinal cord parenchyma without causing iatrogenic injury. , As a result, it is useful to be able to differentiate between IDEM and IM neoplasms preoperatively. Although it has excellent contrast resolution, the relatively low spatial resolution of MRI may negatively impact differentiation of IDEM and IM neoplasms. In humans, the reported sensitivity of MRI in the diagnosis of IDEM tumors has been reported to be approximately 83% with 31 of 187 misdiagnosed as IM tumors.

To our knowledge, no studies in veterinary medicine have investigated the ability to distinguish IDEM from IM neoplasms using MRI. Our purpose was to determine interobserver agreement among a diverse group of board‐certified veterinary neurologists and radiologists in the ability to distinguish IDEM from IM neoplasms using MRI in a population of dogs reported in a previous study. Additionally, we aimed to determine the overall ability of the readers to correctly identify a subset of lesions and their relationship to the meninges compared to histopathologic diagnosis.

MATERIALS AND METHODS

Magnetic resonance images of dogs with intradural (IDEM or IM) neoplasia and degenerative myelopathy (DM) were retrospectively selected by medical record review from a previously described cohort originating from 5 institutions (Texas A&M University, University of Georgia, Washington State University, University of Tennessee, and Virginia Maryland College of Veterinary Medicine). Dogs with intradural primary or metastatic neoplasia were included if spinal cord MRI was available for review and necropsy or biopsy confirmed the diagnosis. Cases were excluded if extradural lesions were present, no histopathology report was available, or if the MR images were unavailable or incomplete. A control group of images from previously selected dogs with suspected DM also was included. A diagnosis deemed consistent with DM included a homozygous positive superoxide dismutase (SOD)‐1 mutation, clinical signs consistent with chronic progressive thoracolumbar (T3‐L3) myelopathy and a previously‐interpreted normal MRI.

All MRI studies were required to meet specific inclusion criteria to enhance the homogeneity of image data for readers, as follows: (a) field strength ≥1.0 T, (b) sagittal and transverse image planes for T2‐weighted image sequences through the lesion area, (c) transverse image planes for T1 transverse pre‐ and postcontrast images, and (d) images in digital imaging and communications in medicine (DICOM) format. Postcontrast sequences were acquired after IV administration of gadolinium‐based contrast agent (gadopentetate dimeglumine, Magnevist, Bayer Healthcare Pharmaceuticals, Wayne, New Jersey) at a dosage of 0.1 mmol/kg. Any additional image planes or sequences were excluded from analysis because other sequences were not considered to be standard by all institutions and were not available for all cases.

Institution, age (in years), weight (kilograms), breed, sex, and results of histopathology from necropsy or biopsy were retrieved from the medical records of all included cases. The MRI readers recruited to participate were not the same as the readers recruited for the previous study. The readers included both board‐certified veterinary neurologists and radiologists from various backgrounds and training institutions in the United States and United Kingdom. The MR images were anonymized and divided into 2 groups each containing 35 MRI cases. The review of each group was separated temporally to assess for reproducibility. Each group of cases was independently reviewed and interpreted by readers. Readers were provided with the digital images and an Excel spreadsheet but were blinded to all clinical data.

For the first part of the study, which was designed to assess interobserver agreement in lesion location, readers were asked to record: case number, intradural disease (Y/N), lesion within the IM compartment (Y/N), or the IDEM compartment (Y/N) and also were instructed to record if a lesion was both IDEM and IM. , Additional characteristics of lesions such as spinal nerve enlargement, “golf‐tee” sign, and dural tail were recorded if noted to draw attention to various imaging features, but were not analyzed independently. An example score sheet and instructions provided to the readers are included in the Supporting Information Data S1. Readers were instructed to return completed responses within 8 weeks of receipt. The second set of 35 MR images was provided to the readers 3 months after completion of the first set and they were instructed to follow the same guidelines as previously described.

The second part of the study was designed to determine how often lesion location was called correctly by the observers. For this analysis, and in contrast to the previous analysis, tumors that can occupy both IM and IDEM locations were excluded (eg, nerve sheath tumors); the remaining tumors included meningiomas (IDEM localization) and gliomas and ependymomas (IM localization). For this study, image reading was restricted to neoplasms known from histology to occupy only 1 location to ensure that readers were making unequivocal calls.

Images of dogs diagnosed with DM also were included in the analysis to provide greater scope for incorrect calls and permit more realistic assessment of sensitivity and specificity of diagnostic calls.

Statistical analysis

All imaging response data were entered into a spreadsheet (Microsoft Excel) and analysis was conducted using Stata 14 (StataCorp, College Station, Texas) and MedCalc (www.medcalc.org). The 2 initial datasets were analyzed separately for reader agreement. Agreement between readers for presence or absence of a lesion and localization of the lesion when present was calculated using an unweighted kappa (κ) score for each dataset, with results interpreted as no agreement (≤0), none to slight (0.01‐0.2), fair (0.21‐0.40), moderate (0.41‐0.60), substantial (0.61‐0.80), or almost perfect agreement (0.81‐1.00). Positive agreement was defined as readers concluding the absence of a lesion or the same localization of the lesion in relationship to the meninges for each set of MR images. Estimates of agreement are provided with 95% confidence intervals (CI).

In the second part of the study, the ability of readers to correctly identify lesions within known anatomic locations was evaluated by tabulating results for individual readers and by sensitivity and specificity across readers as a group. Sensitivity was calculated as the proportion of cases with a known anatomic location correctly identified as occupying that location or correctly identifying the absence of a lesion for DM cases. Specificity was calculated as the proportion of cases within the other categories correctly identified as not having the incorrect anatomic localization. We also calculated positive and negative likelihood ratios and positive and negative predictive values to provide summary estimates of the diagnostic efficacy of MRI examination.

RESULTS

The study population consisted of MRI studies from 70 dogs and these were used to conduct 2 investigations on the diagnosis of intradural lesions based on their relationship to the meninges: to assess agreement among individual readers and to assess the ability of readers to make correct localization calls (against a gold standard of histologic diagnosis).

For the agreement studies, the 70 images were divided randomly into 2 groups of 35 (see Supporting Information for demographic data). Two cases from the first group and 8 cases from the second group were excluded before statistical analysis because of unforeseen technical difficulties with the transfer of DICOM images.

Group 1: Histopathology‐confirmed diagnoses included meningioma (n = 11), DM (n = 5), nephroblastoma (n = 6), malignant nerve sheath tumor (n = 5), glioma (n = 5), and ependymoma (n = 1).

Group 2: Histopathology and confirmed diagnoses included meningioma (n = 7), DM (n = 10), and other—undifferentiated mesenchymal tumor (n = 2), ependymoma (n = 2), glioma (n = 1), hemangioblastoma (n = 1), primitive neuroectodermal tumor (n = 1), nephroblastoma (n = 1), malignant nerve sheath tumor (n = 1), and lymphoma (n = 1).

Interobserver agreement

There were 6 readers including 4 board‐certified veterinary neurologists and 2 board‐certified radiologists. Two of the 6 readers were practicing at institutions from which images were retrieved. Within the first set of images, agreement among observers for anatomic localization varied considerably. For 7 studies (of a total of 33) there was complete agreement among all 6 observers and there were 13 studies for which 5 observers were in agreement. There were 2 studies in which the lesion was allocated to each of the 4 categories (IM, IDEM, DM, or both IM and IDEM) by at least 1 observer and 5 studies in which lesions were designated to 3 categories by the 6 observers. Similar outcomes were found in the second dataset, but with slightly higher agreement. There were 13 studies (of 27 total) for which all 6 observers agreed about lesion anatomic location and another 6 studies for which lesion location was agreed upon by 5 observers. In this second set, there was no study in which observers allocated the lesion to each of the 4 available categories, but there were 3 studies for which observers allocated the lesion to each of 3 categories.

Overall, agreement among readers for the first dataset was moderate (κ = 0.51; 95% CI, 0.43‐0.58). Agreement values varied little among individual categories (Table 1) but were marginally higher for IDEM and lowest for the category “both.” In the second dataset, agreement was substantial (κ = 0.69; 95% CI, 0.66‐0.79). This overall higher agreement was associated with higher κ values for the 3 categories of IM, IDEM and “no lesion,” but a much lower κ value for the category of “both” in this dataset (Table 2).

TABLE 1

Agreement for first dataset

Outcome	κ
IM	0.5064
IDEM	0.5418
None	0.5356
Both	0.4610
Combined	0.5117

Abbreviations: IDEM, intradural‐extramedullary; IM, intramedullary.

TABLE 2

Agreement for second dataset

Outcome	κ
IM	0.7436
IDEM	0.6520
None	0.8823
Both	0.1556
Combined	0.6846

Abbreviations: IDEM, intradural‐extramedullary; IM, intramedullary.

Ability to correctly determine lesion location

Of 40 studies for which a definitive single anatomic location of a tumor (IDEM vs IM) was known from necropsy and histopathology or in which DM had been diagnosed, 15 had DM (ie, no imaging mass lesion), 7 were histologically diagnosed with IM neoplasms, and 18 were diagnosed with meningiomas (Table S3). There were 16 studies for which all observers correctly identified the anatomic location of the lesion and 12 for which 5 of 6 observers correctly identified lesion location. Twelve of the images in which correct calls were made by all observers were those without a mass lesion (ie, diagnosed with DM). Of the 18 dogs with histologically‐confirmed IDEM neoplasms (Figure 1), 4 were diagnosed correctly by all observers. In each of the 7 studies in which lesions were confined to the IM space (Figure 2), 5 readers correctly designated this location.

FIGURE 1

Eleven‐year‐old female spayed Greyhound with progressive clinical signs and a confirmed meningioma. Six out of 6 evaluators agreed that this was an intradural‐extramedullary lesion. The large white arrow on the sagittal plane image denotes the position of the transverse slices. There is widening of the subarachnoid space caudal to the lesion (small white arrow). The mass (white arrowheads) results in compression and displacement of the spinal cord. Compared to the spinal cord, the mass is hyperintense on T1‐ and T2‐weighted images with faint, diffuse contrast enhancement. (3‐T MRI). T1, T1‐weighted image; T1c, T1‐weighted image following gadolinium contrast medium administration, FS fat saturation applied; T2, T2‐weighted image

FIGURE 2

Three‐year‐old intact male Boxer with progressive clinical signs and a confirmed glioma. Five out of 6 evaluators agreed that this was an intramedullary mass. The large white arrow on the sagittal plane image denotes the position of the transverse slices. There is widening and T2‐hyperintensity of the spinal cord (small white arrow). The mass (white arrowheads) results in expansion of the spinal cord without displacement. Compared to normal‐appearing spinal cord, the mass is hyperintense on T2‐weighted images, isointense on T1‐weighted images, and has diffuse contrast enhancement. There is a thin rim of normal‐appearing spinal cord surrounding the lesion. (1‐T MRI). T1, T1‐weighted image; T1c, T1‐weighted image following gadolinium contrast medium administration; T2, T2‐weighted image

Diagnostic sensitivity and specificity varied from moderate to high among the lesion locations and the 95% CI were wide for many of these (Table 3). Sensitivity was lowest for the IDEM category (meningioma) at 62.0% (95% CI, 52.2‐71.2). Positive likelihood ratio and positive predictive value were high for IDEM cases. Similarly, for IM lesions, the negative predictive value was high, but the intermediate value of the negative likelihood ratio suggests that this finding reflects the low prevalence of lesions in this compartment. Positive and negative likelihood ratios and predictive values were intermediate for DM cases.

TABLE 3

Sensitivity and specificity of the combined calls from the 6 observers on 40 lesions with histologic diagnoses (n = 240 calls)

(a) Intradural/extramedullary lesions
	Test result
	IDEM diagnosed	IDEM not diagnosed
Histological IDEM	67	41
Non‐IDEM	5	127

Abbreviations: DM, degenerative myelopathy; IDEM, intradural‐extramedullary; IM, intramedullary.

Indicates the value is dependent upon disease prevalence in a study population.

DISCUSSION

Overall, our results imply the need for some degree of caution in decision‐making in the diagnosis of intradural neoplasia in dogs using standard MRI sequences. First, although there was moderate to substantial agreement among readers, there were few studies for which all observers agreed on lesion localization and, of these, the majority were diagnosed with DM and lacked any imaging lesion. Most notably, agreement in recognition of lesions classified as both IM and IDEM was inconsistent. In health care and clinical research, an interrater reliability (κ score) of 0.8 is considered minimally acceptable. High interrater reliability would imply that similar treatment recommendations and prognoses would be provided to all owners before treatment. In our study, there was moderate (κ = 0.54) or substantial (κ = 0.65) agreement among observers for IDEM lesions. This observation implies that a patient's presumptive diagnosis based on interpretation of an MRI study will be dependent on the neurologist or radiologist reading the study. Given the interrater reliability defined here for classifying IDEM vs IM neoplasms, it is anticipated that the presumptive histological diagnosis assigned to a lesion affecting the spinal cord in an individual patient based on interpretation of an MRI study would differ among clinicians and ultimately result in different treatment recommendations and prognoses provided to owners. Furthermore, considerable difference was found in the agreement in location classification between our 2 test datasets, indicating that the level of agreement we report here may be different when applied to the datasets in different clinics, which may have different prevalences of specific diagnoses.

Although observers may agree or disagree, it is also important to know how often they are correct in their diagnosis. To address this question, we examined the designations made by the observers for studies in which we had definitive diagnoses of lesions that were either IDEM or IM but in which a category of “both” was not possible (Table S3). In general, the observers achieved moderate to high levels of accuracy (Table 3), but accuracy varied among individuals. Nevertheless, of the 18 cases with histologically‐confirmed meningioma (and therefore classified as IDEM), only 5 were assigned the correct anatomic location by all observers. Sensitivity and specificity were moderate to high for all locations, except for a low sensitivity (62%) for IDEM lesions (Table 3). However, these results give information about the test itself, rather than the usefulness of the test when applied in practice. Negative and positive predictive values can be helpful in this regard but they are affected by the prevalence of the disease within the sample population. In our sample, the prevalence of both IDEM and IM lesions was generally consistent with their reported prevalence within cases of spinal neoplasia as a whole, although in our population the remainder was made up of DM cases. Analysis of diagnostic accuracy suggests that a positive diagnostic call for IDEM would likely be helpful in diagnosis (positive likelihood ratio > 10). Although the negative predictive value for IM lesions was high, suggesting a negative call might be helpful in diagnosis, the intermediate negative likelihood ratio suggests that this is also a consequence of the low prevalence of lesions in this category (as is also thought to be true more generally). , For DM cases, the likelihood ratios and predictive values were intermediate, suggesting that both positive and negative calls for this condition should be treated with some caution.

Accurate recognition of IM neoplasms is potentially important for surgical approaches, because IM neoplasms often are not considered for cytoreductive surgery, although some patients with IM neoplasms will benefit. , , In those patients with IM neoplasia when cytoreductive surgery is considered, setting appropriate client expectations as to the potential for iatrogenic injury is important before surgery. In our study, κ values for IM neoplasms varied between 0.51 and 0.74 in the 2 datasets, respectively, indicating a wide range of reliability in assigning the correct classification of an IM neoplasm. Care is required when considering the possibility of lesions in this location, because there was a great deal of variability in their accurate recognition (Table 3).

Altogether, our results suggest the possibility that a minority of dogs might have incorrect lesion localization on MRI, implying an inaccurate assumed histopathologic diagnosis that could lead to inappropriate treatment recommendations and prognoses. Of the lesions we examined, the most important to identify are those in the IDEM category, because dogs with such lesions may be more likely to be considered for cytoreductive surgery. Neoplastic lesions classified as IDEM are most likely to be meningioma or nerve sheath tumor, and affected dogs are likely to receive the most benefit from surgical excision. , ,

Increasing the number of patients with intradural neoplasms that are treated by cytoreductive surgery may provide more information regarding optimal treatment approaches and prognostic information. In humans, no difference is found in postoperative improvement of neurologic status between patients with IM and IDEM neoplasms. Macroscopic total resection, which is known to be associated with prognosis, also was achieved in the majority of patients with intradural neoplasms, regardless of compartment.

Our study provides new information relative to the original study performed on this population of dogs. The purpose of the original study was to estimate the sensitivity, specificity, and agreement of 3 radiologists in broadly categorizing dogs as affected vs control and by etiology (inflammatory, neoplastic, and vascular). Here, we set out to answer a different question: Will a diverse set of image evaluators agree on lesion location in dogs with spinal cord neoplasia? We chose this question because dogs with IM and IDEM neoplasia typically are treated differently. We excluded dogs with inflammatory and vascular spinal cord disease a priori because we were concerned that the low numbers could cause misleading results. The original study only had 3 image evaluators from 2 institutions in a single country. We followed published recommendations and increased the number and geographic location of image evaluators in hope that our findings would be more broadly applicable. To further increase the generalizability of our findings, we divided the population into 2 random groups, which differed in composition and were assessed at different time points, separated by at least 3 months.

Our study had several limitations. The number of patients with IM neoplasms was much smaller than the other groups, owing to the low prevalence of these neoplasms in the canine population. Additionally, most IM neoplasms in our study were ependymomas and gliomas. Other types of tumors were not specifically excluded but were not encountered in the group sampled. A larger sample size would help avoid this limited heterogeneity of tumor types, although our sample included the most commonly encountered lesions. , There also may have been inherent bias in case selection because a histopathologic diagnosis was necessary for inclusion in the study. Postcontrast T1‐weighted images were only available in a transverse plane. In practice, postcontrast images in multiple planes typically are acquired. The ability to examine multiplanar postcontrast images adds value to a study in many ways, including increasing the visibility of the margins of a mass relative to the subarachnoid space. The MRI examinations were performed at different institutions resulting in variable image parameters and quality. This design should make our results more clinically applicable because off‐site image evaluation is now commonplace in veterinary medicine. Additionally, extradural lesions were excluded in our study because they were not included in the original dataset. Inclusion of extradural neoplasms in our study may have resulted in a more complete comparison and additional studies will be needed to determine the utility of MRI in diagnosing extradural neoplasia. Lastly, 2 of the readers were from institutions where images were acquired. It is unknown whether the clinicians were involved directly with these cases, although the images were collected from 2007 to 2014 and it was considered improbable that the readers would recall individual cases from so long before.

One option to improve diagnostic capability might be to use a wider range of MRI sequences, some of which can reproduce more accurately the myelographic views that were traditionally used for categorizing lesions in relationship to the meninges. For instance, the use of 3‐dimensional field echo steady state free procession (FE3D‐SSFP) images in conjunction with fast advanced spin echo (FASE) or half‐Fourier acquisition single‐shot turbo spin Echo (HASTE) images may help. Computed tomography and myelography were not evaluated in our study, and it is unknown if other imaging modalities would have resulted in improved agreement scores. Without this information it would be difficult to recommend to owners that their dogs should undergo more invasive imaging after the initial MRI investigation.

Future work should evaluate the relationship between image quality (such as spatial resolution) in diagnostic accuracy and interrater agreement of intradural spinal cord lesions. It is widely assumed that increased field strength and spatial resolution will translate to improved accuracy, but the results in neurological and musculoskeletal diseases of humans do not always support this assumption. , , Furthermore, emerging MRI techniques (including spectroscopy and diffusion tensor imaging) , should be developed to improve diagnostic accuracy and surgical planning in dogs.

In conclusion, our results indicate that classifying lesions on MRI as IDEM, IM, or both should be treated with caution because differences of opinion exist among specialists, and failure to designate the correct classification is common. On the other hand, our data provide additional evidence to caution against drawing too strong a conclusion regarding a presumptive histologic diagnosis of a specific neoplasm and decision to pursue cytoreductive surgery based solely on the use of MRI to classify lesion location relative to the meninges.

CONFLICT OF INTEREST DECLARATION

Authors declare no conflict of interest.

OFF‐LABEL ANTIMICROBIAL DECLARATION

Authors declare no off‐label use of antimicrobials.

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEE (IACUC) OR OTHER APPROVAL DECLARATION

Authors declare no IACUC or other approval was needed.

HUMAN ETHICS APPROVAL DECLARATION

Authors declare human ethics approval was not needed for this study.

Table S1: Demographic data for group 1.

Table S2: Demographic data for group 2.

Table S3: Correct identification of histologically confirmed IDEM and IM neoplasms and degenerative myelopathy.

Click here for additional data file.

Data S1: MRI project instructions and spinal MRI workbook.

Click here for additional data file.