Centralized and Local Color Doppler Ultrasound Reading Agreement for Diagnosis of the Chronic Cerebrospinal Venous Insufficiency in Patients with Multiple Sclerosis.

BACKGROUND: An impaired cerebrospinal venous drainage was postulated to be a cofactor in the multifactorial pathogenesis of multiple sclerosis (MS). Chronic cerebrospinal venous insufficiency (CCSVI) is characterized by abnormalities of the main extracranial cerebrospinal venous outflow routes, which can be detected by color Doppler Ultrasound (CDUS) using 5 venous hemodynamic (VH) criteria. Discrepant results between different investigators were reported in the past, therefore the usefulness and applicability of the CCSVI CDUS-based diagnosis in clinical research and practice has been questioned. The reproducibility of proposed criteria for CCSVI detection depends on the blinding, training level, skills of the operator and interpretation of VH criteria.
OBJECTIVES: To assess agreement between centralized and local reading of CDUS examination for diagnosis of CCSVI in trained Doppler sonologists.
METHODS: This study was performed in 78 MS patients and 28 age- and sex-matched healthy controls (HCs). Extracranial and transcranial CDUS venous hemodynamic assessment was conducted, according to International Society of Neurovascular Disease (ISNVD) recommended criteria, by a single CCSVI-trained expert sonologist blinded to the subject disease status. After the local Doppler sonologist performed the investigation, all images and video clips of the CDUS examination were sent to the centralized reading center, where a second blinded reading was performed by two CCSVI-trained expert sonologists. Statistical analyses were performed comparing accuracy of CCSVI diagnosis (≥2 VH criteria) and each of the 5 individual VH criteria using Cohen kappa statistic, along with positive/negative agreement and Odds ratio (OR) with 95% confidence intervals (95% CI).
RESULTS: Diagnosis of CCSVI was obtained in 59.7% of local and 64.3% centralized readers (Kappa, 0.67, p<0.001). Similar Kappa values were obtained for CCSVI diagnosis and individual CCSVI criteria in both MS patients and HCs. The highest Kappa between local and centralized readers was observed for VH criteria 5 (0.93) followed by VH criteria 4 (0.70), VH criteria 1 (0.66), VH criteria 2 (0.64) and VH criteria 3 (0.58). The positive predictive value (PPV) and negative predictive value (NPV) for CCSVI diagnosis were 82.7% and 86,7%, respectively with an OR of 31.1 (95% CI 11.1-87.5, p<0.001). The highest agreement between local and centralized readers was observed for VH criteria 4 (OR 98.7, 95% CI 17.1-569.9, p<0.001) with 72.7% PPV and 97.3% NPV followed by VH criteria 5 (53, 95% CI 13.4-209.2, p<0.001) with 98.1% PPV and 100% NPV value.
CONCLUSION: Centralized reading of the CDUS examination for the diagnosis of CCSVI is feasible with high accuracy in CCSVI-trained Doppler sonologists. The most reproducible VH criteria between local and centralized readers were VH criteria 4 and 5. Copyright© Bentham Science Publishers; For any queries, please email at epub@benthamscience.org.

Introduction

In the last few years, a vascular condition, named chronic cerebrospinal venous insufficiency (CCSVI) [1], has generated an intense interest in better understanding the role of extracranial venous anomalies and developmental variants, particularly in relation to central nervous system (CNS) 
pathology [2-9]. CCSVI has been described as anomalies of the main extra-cranial cerebrospinal venous outflow routes that interfere with normal venous outflow, that is more 
frequently observed in patients with multiple sclerosis (MS) [1, 10, 11].

Color Doppler Ultrasound (CDUS) is a non-invasive imaging technique that provides high-resolution images with real-time, dynamic interrogation of structural/morphological and hemodynamic/functional venous abnormalities at relatively low cost, therefore it was proposed as a method of choice for the screening of CCSVI [1, 10].

The outflow anomaly can be detected by using 5 venous hemodynamic (VH) criteria. A cut-off for CCSVI diagnosis classification consists of ≥2 abnormal VH criteria [1].

Recent meta-analyses [12-15] have suggested an independent association between an ultrasound-based diagnosis of CCSVI and MS with OR ranging between 1.9 and 13.5. However, considerable heterogeneity (I squared statistic >50%) across included studies, was documented. Therefore the usefulness and applicability of the CCSVI CDUS-based diagnosis in clinical research and practice has been questioned [16, 17]. It should be noted that most studies were monocentric and blinding of sonographer was sub-optimal. [18].

A multimodal approach that uses Ultrasound, MRI and Contrast-enhanced MR venography could overcome the limitations of individual methods because each imaging technique has its own strengths and weakness and therefore provide complementary information on cerebrospinal venous vasculature [19]. However, this approach is difficult to achieve in clinical practice.

When comparing the MRI and US, the US found 5 times more IJV narrowing (CSA <0.3 cm2) than MRI [20]. Therefore, the MRI and US measurements for IJV CSA are not comparable. One explanation could be that the compression of the neck by the US probe can artifactually decrease the CSA of IJV as well as an extrinsic compression of omohyoid muscle due to the neck position [21], which may be different when using different examination methods.

The reproducibility of the categorical CCSVI - CDUS-based diagnosis depends on the training level and skills of the operator, blinding and reading criteria, [22-29]. The Ultrasound examination requires some degree of subjectivity in performing and interpretation of exams and the best method to avoid potential bias is blindness of the sonographer.

Aim of the study was to evaluate the reproducibility of CCSVI-CDUS diagnosis performed by trained Doppler sonologists, using a blinded centralized protocol.

Methods

This study was performed in 78 multiple sclerosis (MS) patients and 28 age- and sex-matched healthy controls (HCs) who obtained CDUS examination by a CCSVI-trained expert sonologist blinded to the subject disease status.

The study was approved by the local Institutional Review Board of the University of Buffalo, USA and University of Naples, Italy, and written informed consent was obtained from each participant.

The inclusion criteria for this study were: a) age 18-75 years, and b) being MS patients or HCs with unknown history of neurological disease. Exclusion criteria were: a) presence of relapse and steroid treatment within 30 days preceding their enrollment in the study (for MS patients), b) pre-existing medical conditions known to be associated with cerebral pathology (cerebrovascular disease, positive history of alcohol abuse), c) evidence of brain ischemic or hemorrhagic infarcts, or space-occupying lesions on MRI exam performed within 30 days of physical/neurologic examination with the standardized study protocol, and d) pregnancy. Additional contraindications for CDUS assessment included the following:

Subjects with short, thick, muscular necks

Subjects who have had recent surgery (penetration and visualization may be limited secondary to the presence of edema, hematoma, surgical staples, dressings)

History of chronic obstructive pulmonary disease (COPD) and arthritic necks (may not be able to lie flat)

Subjects who are unable to cooperate with the evaluation due to changes in mental status (advanced dementia, advanced Alzheimer’s, mental retardation, etc.) and involuntary movement.

For this examination an ultrasound machine (Esaote-Biosound MyLab25 GOLD) with Quality Doppler Profiles (QDP) software, 2.5 and 7.5-10 MHz transducers was used. The subjects were instructed not to reveal their disease status during the examination. All study subjects were positioned and draped (covered with a blanket, leaving only the head and neck exposed) on the Hydraulic chair capable of tilting between 90° to 0° by the unblinded study coordinator who also removed any assistive device used by the patients from the room before the Doppler sonographer entered, to avoid any visual cues of the presence of disease.

Water-soluble, hypoallergenic, medium density ultrasound gel was used for sound transmission. Gel maintained at a warm temperature in a thermasonic Gel Warmer as cold gel may cause the vein to contract. Subjects drank minimum 16 oz H2O within 2 hours of scan time.

Phantom, GAMMEX Model 1430GS Mini Doppler Flow System, Middleton, WI, USA, was used for annual calibration Quality Control. The quality control procedure with the phantom examines the Doppler for signal sensitivity, color flow sensitivity, flow sensitivity at depth, color flow B-mode image congruency, directional discrimination, accuracy of flow velocity readout, and sample gate positioning accuracy.

Venous Hemodynamic (VH) Criteria

The VH CDUS criteria, according to the recommendations of the International Society for Neurovascular Disease [30], included: (1) reflux present in an outflow pathway [internal jugular vein (IJV) and/or vertebral vein (VV)] with the head positioned at 0° and 90°; (2) high resolution B-mode evidence of proximal IJV narrowing and/or other B-mode anomalies (Flap, Septum, Web, Annulus, Non-compliance); (3) flow not detectable in the IJVs and/or VVs despite numerous deep inspirations; (4) abnormal posture control of IJV flow and (5) reflux in the intracranial veins/deep cerebral veins in any position. After the local Doppler sonologist performed the investigation, all images and video clips of the CDUS examination were sent to the centralized reading center, where a second blinded reading was performed by a CCSVI-trained expert sonologist (Fig. ).

Statistical Analysis

Statistical analysis was performed comparing accuracy of CCSVI diagnosis (≥2 VH criteria) and each of the 5 individual VH criteria using Cohen kappa statistic, along with positive/negative agreement and Odds ratio (OR) with 95% confidence intervals (95% CI). A nominal p-value of ≤0.05 was considered statistically significant, and p<0.1 was considered a trend, using two-tailed test.

Results

Table provides demographic and clinical characteristics of the study sample.

A nominal p-value of ≤0.05 was considered statistically significant.

There are no significant differences between the two groups.

Diagnosis of CCSVI was obtained in 60.4% of local and 67.0% centralized readers (K 0.61). The highest Kappa between local and centralized readers was observed for VH criteria 5 (K 0.93) followed by VH criteria 1 (K 0.80), VH criteria 4 (0.70), VH criteria 2 (K 0.60) and VH criteria 3 (K 0.58). Criterion 4 that was positive in only 11 subjects (10.4%) (Table ), showed the highest specificity for the CCSVI diagnosis (Table ).

Criterion 5 that had higher sensitivity was demonstrated in more than 80% patients, but with low specificity (88.2%) for CCSVI diagnosis (Table ). The positive predictive value (PPV) and negative predictive value (NPV) for CCSVI diagnosis was 81.6% and 82.9%, respectively with an OR of 21.6 (95% CI 7.4-62.6, p<0.001). The best diagnostic accuracy between local and centralized readers was observed for VH criteria 4 (OR 81.8, 95% CI 14.1-473.3, p<0.001) with 72.7% PPV and 96.8% NPV followed by VH criteria 5 (OR 45.0 CI 11.5-179.2, p<0.001) with 97.8% PPV and 100% NPV value (Table ).

CDUS demonstrated to be highly reproducible and capable of detecting patients with abnormal posture control of IJV flow and reflux in the intracranial veins/deep cerebral veins whereas it is poorly reproducible for the VH criteria 1-3.

The highest Kappa between local and centralized readers was observed for VH criteria 5 (K 0.93) followed by 
VH criteria 1 (K 0.80), VH criteria 4 (0.70), VH criteria 2 
(K 0.60) and VH criteria 3 (K 0.58).

K are derived using Cohen kappa statistic. A nominal 
p-value of ≤0.05 was considered statistically significant.

Discussion

Considering the current studies that postulate an 
influence of vascular factors on neurological disorders [31], it is increasingly necessary to develop diagnostic methods capable of evaluating and quantifying vascular anomalies.

Ultrasonography has shown to be a practical, effective and low-cost method, not requiring ionizing radiation. Moreover, CDUS is the only one method that can explore cerebral hemodynamics and vein valves movement in real-time.

Ongoing clinical trials seek to assess the presence of cerebral outflow anomalies by ultrasound but the accuracy of this technique for investigating patients with MS has not been systematically assessed.

Our study based on off-line and centralized readings of CDUS images digitally recorded, thus bypassing the influence of the sonographer on measurement variability, demonstrates that blindness of the operator is crucial to ensure unbiased ascertainment of diagnosis. Therefore, researchers should make every effort to incorporate blinding into their trial designs and readers should look for descriptions in the published reports.

CDUS demonstrated to be highly reproducible and capable of detecting patients with abnormal posture control of IJV flow and reflux in the intracranial veins/deep cerebral veins whereas it is poorly reproducible for the VH criteria 1-3.

Finding reflux in an outflow pathway, flow not detectable in the IJVs and/or VVs and B-mode evidence of proximal IJV stenosis or other anomalies are too subjective parameters that do not allow to achieve sufficient uniformity of diagnosis.

Although the agreement on CCSVI diagnosis was acceptable in the present study, seven patients were differently classified by a different reader. (CCSVI positive if > 2VH criteria are fulfilled).

In recent years, some studies analyzed agreement for CCSVI diagnosis, with different results.

Comi et al. [18] performed a multicenter CoSMo study that involved 35 centers in Italy and evaluated 1,767 subjects, including 1,165 MS patients, 226 patients with other neurologic diseases and 376 healthy controls. The primary endpoint was to compare the prevalence of CCSVI in patients with MS versus patients affected by other neurodegenerative diseases and healthy volunteers. Local Color Doppler examination was carried out by a certified sonologist and the central image readings performed by experts in the field. In that study, there were substantial differences between central and local readers. The overall CCSVI prevalence in the local readings was significantly higher, as compared to the first centralized reading (14.9% versus 3.2%; P <0.001). Agreement between the local sonologists’ CCSVI diagnosis and the central expert reader diagnosis was very low. Kappa statistics test was 13% (standard error = 3%). The negative agreement was 92% and the positive agreement was 18%. Of 264 CCSVI positive at local color Doppler examination the central reader confirmed only 28.

The two groups of readers (central and local) identify within the same set of images two different proportions of positive results. Assuming that the two operators evaluate the images in the same way, the two proportions should be equal to each other. Conversely assuming that the two operators evaluate the images in a different way the two proportions will be different. In this study, the two proportions were statistically different, therefore could be a bias in the study: the two readers read differently the same images. What is revealed in the CoSMo study is “systematic error”. The high discrepancy between the measurements by the two groups of reading could indicate a prejudice in one of the groups because the central reader systematically has contradicted the result of the local reader, providing a negative result systematically. In conclusion, the authors of the CoSMo study can state that the interpretation of the images by the two groups is different; but they cannot state which group performs the correct diagnosis because the true prevalence of the condition is unknown.

Leone et al. [32] evaluated inter-rater agreement in a color-Doppler sonography venous examination carried out in accordance with Zamboni’s five criteria by eight sonographers with different expertise, in 38 patients with MS and 55 controls age-matched. They concluded that the agreement was unsatisfactory for the diagnosis of CCSVI as a whole, for each of its five criteria.

Laukontaus et al. [33] evaluated inter-observer agreement between two ultrasound examiners. The inter-observer agreement for all parameters was poor, except for CSA of IJV at the thyroid level. Criteria 1, 4 and 5 were found in less than 10% of both patients and controls. Limitations of the study were partial blindness, inability to perform transcranial Doppler and the use of two different ultrasound system.

Tsivgoulis et al. established the intra-rater and interrater reliability of ultrasound criteria of CCSVI in a pilot study of 15 individuals examined by 2 sonographers who were blinded to the patients’ clinical features. They documented excellent intra-rater and interrater agreement (kappa values ranging from 0.82 to 1.00) regarding 3 out of 5 ultrasound criteria: reflux in cervical veins, high-resolution B-mode evidence of proximal IJV stenosis, and flow not-Doppler detectable in IJV [26].

Menegatti [24] demonstrated that the reproducibility was depended on the level of experience between trained and not trained operators. The inter observer agreement between trained operators was high (K 0.80) as well as the intra observer variability rate (K 0.93).

We have identified which of the CDUS criteria that have been proposed [30] enables reproducible diagnosis and therefore, on condition of uniformity of execution protocol and accurate blindness, is suitable to perform multi-center projects always and for monitoring the progression/
regression of neck vein abnormalities.

Typically, the most common conditions of change included in a reproducibility statement were different locations, operators, and measuring systems.

The dependence of B-mode image quality on interrogation angle and fine adjustment of instrument controls makes the role of the sonographer crucial to the measurement process. It may often be difficult to decide whether a minor wall irregularity represents a septum/flap or only a reverberation artifact.

High-resolution ultrasound allows morphological characterization of vessel walls and valves that matches reasonably well with histological features of specimens. The potential methodological shortcomings are that many vein valves are located in deep, poorly reflect emitted ultrasound, and rapid valve movement requires very high frame rate and an experienced operator. Moreover, there is no consensus on ultrasonographic criteria for morphological characterization of jugular valves and actual classification is based on subjective judgment.

The deep location of the valve plane of the Jugular/
subclavian confluence, the impossibility to obtain the scan planes perpendicular to the major axis of the vessel, the scarce remaining ultrasound energy after passage of the high-frequency ultrasound signals through the tissues in the neck are probably a major source of error in evaluating valve morphology. There are many technical limitations, that pose some difficulty to the performance of the US scan, limiting its reliability: the artificial compression of cervical veins by the ultrasound probe, or contraction of cervical musculature, inappropriate pulse repetition frequencies, misinterpretation of pulsation artifact from the adjacent carotid artery as venous reflux, reverberation artifact, variability in the hydration status of patients, non-cooperation during evaluation at different body positions, respiratory maneuvers, and the variations in normal patterns of cerebral venous drainage within the healthy population. Interpretation of different patterns of venous drainage is not standardized, can be highly subjective, depending upon the expertise of the sonographers.

The strengths of our study are the choice of trained Doppler sonographers and the complete study blindness. We confirm the need for training in application of the CDUS CCSVI protocols, previously emphasized in other studies [24, 25] as well as the need to blinded study [6].

conclusion

In conclusion, if an adequate blindness of the operator is obtained, centralized reading of the CDUS examination for the diagnosis of CCSVI is feasible in CCSVI-trained sonologists. The most reproducible VH criteria was abnormal posture control of IJV flow and reflux in the intracranial veins/deep cerebral veins.

STUDY DISCLOSURE

This study was funded by the The Annette Funicello Research Fund for Neurological Diseases and internal resources of the Buffalo Neuroimaging Analysis Center. In addition, we received support from the Jacquemin Family Foundation.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

Not applicable.

HUMAN AND ANIMAL RIGHTS

No Animals/Humans were used for studies that are base of this research.

CONSENT FOR PUBLICATION

Not applicable.

Table 1

Demographic and clinical characteristics of multiple sclerosis patients and healthy controls.

-	HC (n=28)	MS (n=78)	p
Female, n (%)	17 (60.7%)	56 (71.8%)	.277
Age, mean (SD)	50.3 (13.1)	52.8 (9.9)	.307
MS type   RR (Relapsing-remitting) SP (Secondary Progressive) PP (Primary Progressive)	---	42 (53.8%)31 (39.7%)4 (5.1%)	-
EDSS (Expanded Disability Status Scale), median interquartile range (IQR)	-	3.5 (2.0-6.0)	-
Disease duration, mean Standard Deviation (SD)	-	21.7 (10.1)	-
Relapses past 5 years, mean Standard Deviation (SD)	-	1.03 (2.3)	-
BMI (Body mass index), mean Standard Deviation (SD)	26.5 (5.4)	27.9 (5.9)	.267
Hypertension, n (%)	5 (17.9%)	10 (12.8%)	.466
Hyperlipidemia, n (%)	7 (25%)	13 (16.7%)	.291
Diabetes, n (%)	1 (3.6%)	3 (3.8%)	.955
Heart disease, n (%)	2 (7.1%)	14 (17.9%)	.194
Smoking†, n (%)	9 (32.1%)	40 (51.3%)	.095

Note: †Denotes whether subject has ever smoked. Disease modifying therapy use among MS subjects consisted of: non-therapy: 9, interferon-beta: 24, glatiramer acetate: 27, natalizumab: 4, other: 14.

P-values are derived from the Pearson chi-square test and student t-test.

Table 2

Accuracy of CCSVI diagnosis and individual positive criteria between local and centralized reading center.

Total Sample (n=106)	Local	Centralized Readers	Kappa	p
Abnormal	64 (60.4%)	71 (67.0%)	.614	<.001
VH 1	2 (1.9%)	3 (2.8%)	.795	<.001
VH 2	70 (66.0%)	78 (73.6%)	.600	<.001
VH 3	21 (19.8%)	16 (15.1%)	.576	<.001
VH 4	11 (10.4%)	11 (10.4%)	.696	<.001
VH 5	89 (84.0%)	91 (85.8%)	.926	<.001
MS (n=78)
Abnormal	51 (65.4%)	54 (69.2%)	.622	<.001
VH 1	2 (2.6%)	3 (3.8%)	.794	<.001
VH 2	49 (62.8%)	57 (73.1%)	.651	<.001
VH 3	17 (21.8%)	12 (15.4%)	.537	<.001
VH 4	11 (14.1%)	11 (14.1%)	.682	<.001
VH 5	68 (87.2%)	69 (88.5%)	.940	<.001
HC (n=28)
Abnormal	13 (46.4%)	17 (60.7%)	.578	.001
VH 1	0 (0%)	0 (0%)	1	<.001
VH 2	21 (75.0%)	21 (75.0%)	.429	.034
VH 3	4 (14.3%)	4 (14.3%)	.708	.001
VH 4	0 (0%)	0 (0%)	1	<.001
VH 5	21 (75.0%)	22 (78.6%)	.900	<.001

Table 3

Diagnostic accuracy (sensitivity, specificity, positive and negative predictive values, and the Odds Ratio) of Local (considered Gold Standard) vs centralized reader center.

Total Sample	Sensitivity	Specificity	PPV	NPV	Odds Ratio	p
Abnormal	90.6%(80.1, 96.1)	69.0%(52.84, 81.9)	81.6%(70.4, 89.5)	82.9%(65.7, 92.8)	21.6(7.4, 62.6)	<.001
1 or more*	91.9%(82.6, 96.6)	56.3%(37.9, 73.2)	82.9%(72.7, 90.2)	75.0%(52.9, 89.4)	14.6(4.9, 43.3)	<.001
2 or more*	72.0%(50.4, 87.1)	93.8%(85.5, 97.7)	78.3%(55.8, 91.7)	91.6%(82.9, 96.3)	39.9(11.1, 137.7)	<.001
1 positive**	85.1%(71.1, 93.3)	62.3%(47.8, 74.8)	66.7%(53.2, 77.9	82.5%(66.7, 92.1)	9.4(3.6, 25.0)	<.001
2 positive**	72.0%(50.4, 87.1)	95.1%(87.2, 98.4)	81.8%(58.9, 94.0)	91.7%(83.0, 96.3)	49.5(13.0, 87.1)	<.001
3 positive**	33.3%(1.8, 87.5)	100%(95.5, 100)	100%(5.5, 100)	98.1%(92.6, 99.7)	0.02(0.01, 0.1)	<.001
Criterion 1	100%(19.8, 100)	99.0%(93.9, 99.9)	66.7%(12.5, 98.2)	100%(95.5, 100)	3.0(0.6, 14.8)	<.001
Criterion 2	92.8%(83.4, 97.3)	62.8%(46.2, 78.7)	83.3%(72.8, 90.4)	82.1%(62.4, 93.2)	23.0(7.4, 71.6)	<.001
Criterion 3	57.1%(34.4, 77.4)	95.3%(87.7, 98.5)	75.0%(47.4, 91.7)	90.0%(81.4, 95.0)	27.0(7.2, 101.5)	<.001
Criterion 4	72.7%(39.3, 92.7)	96.8%(90.4, 99.1)	72.7%(39.3, 92.7)	96.8%(90.4, 99.2)	81.8(14.1, 473.3)	<.001
Criterion 5	100%(94.8, 100)	88.2%(62.2, 97.9)	97.8%(91.5, 99.6)	100%(74.6, 100)	45.0(11.5, 179.2)	<.001

Criterion 5 that had higher sensitivity was demonstrated in more than 80% patients, but with low specificity (88.2%) for CCSVI diagnosis.

The best diagnostic accuracy between local and centralized readers was observed for VH criteria 4 followed by VH criteria 5.

Note: 95% confidence intervals are reported in parentheses.

*excludes Criteria 2. ** excludes criteria 1 and 2.