Validity and reliability of pocket-sized ultrasound devices in measurement of optic nerve sheath diameter in ICU patients.

The measurement of the optic nerve sheath diameter (ONSD) by ultrasonography (USG) is particularly important for intracranial pressure (ICP) monitoring when invasive measurements are not possible or are contraindicated. Standard USG (SUDs) devices can be bulky and may break down the workflow. The validity and reliability of pocket-sized USG devices (PSUDs) compared to SUDs for ONSD measurement has not been investigated, yet. We compared the reliability and validity of PSUDs with SUDs for ONSD measurement. 35 patients were included in the study. ONSD measurements with PSUDs and SUDs were performed by two clinicians as three different measurements in both sagittal and transverse axis of the eye. There was agreement between mean transverse ONSD (mtONSD), mean sagittal ONSD (msONSD), and mean ONSD(mONSD) values measured with each device according to Bland-Altman test (p = 0.166, p = 0.135, p = 0.066, respectively) with no proportional bias (p = 0.544, p = 0.868, p = 0.929, respectively). Intraclass correlation coefficients (ICC) were found to be greater than 0.7 for mtONSD, msONSD, and mONSD values measured by SUD (ICC = 0.790, ICC = 0.817, and ICC = 0.844, respectively). Similarly, for mtONSD, msONSD, and mONSD values measured by PSUD, ICC were found greater than 0.7 (ICC = 0.763, ICC = 0.814, and ICC = 0.843, respectively). There was no statistically significant difference between mtONSD and msONSD (p = 0.441 for SUD and p = 0.893 for PSUD). There was a good correlation between mtONSD and msONSD (r = 0.767 for SUD and r = 0.816 for PSUD). The total variance between three different measurements in the transverse and sagittal axis was ± 0.6 mm. PSUDs can be used with similar validity and reliability as SUDs for ONSD measurement.

Introduction

Increased intracranial pressure (ICP) is a common problem in intensive care unit (ICU) patients with or without primary neurological pathology [1]. Early detection of its presence and determination of the main cause of increased ICP are very important in the prevention of potentially devastating secondary brain injuries [2]. Indications of intracranial pressure monitoring in traumatic brain injury (TBI) has been defined in the 2007 guidelines [3]. Although there are several ways of ICP monitoring, monitoring through an intraventricular catheter is the gold standard nowadays [2]. However, invasive ICP monitoring methods are associated with complications such as ventriculitis, meningitis, intracranial hemorrhage, and neurologic deficits [1, 4–6]. In addition to these complications, there are some disadvantages of invasive ICP monitoring; for example, catheter occlusion, need for re-calibration with the change of head position, obtaining inaccurate values in case of cerebrospinal fluid leakage, or need for catheter replacement after 5 days. Furthermore, the neurosurgeon may not always be available for the invasive ICP monitoring process or health facilities may not have adequate qualifications for ICP monitoring. For these reasons, non-invasive ICP monitoring has become an alternative option today. Also, noninvasive methods can be helpful when invasive tools are not indicated (i.e., milder than severe degrees of traumatic brain injury), are contraindicated (patients with hemostatic disorders) or are unavailable (such as in general intensive care).

Numerous methods have been described for non-invasive ICP monitoring; for instance, transcranial doppler ultrasonography (USG), magnetic resonance imaging/computed tomography (CT), optic nerve sheath diameter (ONSD) measurement by USG and fundoscopy [5, 6]. Among the alternatives, ultrasonographic measurement of ONSD can be preferred because it is easy to learn especially by emergency and intensive care physicians, it is easy to apply at bedside, it does not cause complications, and the patient is not exposed to radiation. Optic nerve sheath is anatomically continuous with dura mater, and optic nerve, pia mater and subarachnoid space can be distinguished easily with proper technique [7]. Ultrasonographic images should be obtained with appropriate techniques according to new Optic Nerve Sonography Quality Criteria [8]. Recent studies suggest that daily monitoring of ONSD measurement by USG can be useful not only for the determination of increased ICP but also for evaluating the neurological prognosis of the patients [9-11].

With the development of technology, the USG devices are getting smaller and handheld or pocket-sized USG devices (PSUD) are in use. The number of studies investigating their accuracy and reliability in the evaluation of various organ systems in ICU patients has been increasing day by day [12-14]. Their validity has been tested for cardiac, abdominal, and vascular (IVC, lower extremity venous Doppler, jugular cannulation e.g.) structures. We also think that conducting studies on the comparison of ONSD measurements by PSUD and by standard USG devices (SUD) are important for finding a more practical way of performing ONSD measurement; however, this subject has not been studied in the literature, yet. For this reason, this observational prospective study was carried out to compare PSUD to SUD in the measurement of ONSD and determine the applicability of PSUD in ONSD measurement.

Materials and methods

Patient population

This prospective observational study was performed in Gazi University School of Medicine, Department of Pulmonary Critical Care Medicine between May 2018 and October 2018. Exclusion criteria were being younger than 18, presence of cataract, artificial lenses, and history of any mass lesions inside the eye globe. The study protocol was approved by the ethics committee of the related institution (28.12.2015/166), and written informed consent was obtained from the patients or their close family members.

Clinical data

Admission Acute Physiology and Chronic Health Evaluation (APACHE II) scores were recorded. SOFA (Sequential Organ Failure Assessment) scores and Encephalopathy and Kelly Matthay scores were calculated and recorded for the evaluation of organ and cognitive functions on the ONSD measurement day. Vital signs, arterial blood gases, laboratory analysis, mechanical ventilator settings, mechanical ventilation requirement, drugs, use of sedative/neuromuscular agents were recorded during admission and ONSD measurement days. Intracranial images, if available, were recorded to detect any possible clinical conditions related to ONSD measurement.

USG devices

In this prospective observational study we used Vivid-Q (full range of standard modalities and measurements: MM, 2D, PW, CW, Colour, TVI, and TEE, GE Systems) as SUD and VScan (with a dual probe, GE Systems) as PSUD (2D and Colour Doppler). ONSD measurements were obtained with 7–13 MHz and 4–8 MHz linear probes of SUD-Vivid-Q and PSUD-VScan, respectively. We used the superficial option on the V-Scan device and carotis option on the Vivid-Q device.

Measurement technique

Each measurement was performed by two clinicians with 3 years of clinical experience. Measurements were performed in the supine position with the head elevated to 30°. The video images to be used for the measurement of ONSD were recorded as six-second shots repeated at least two times in transverse and sagittal axis for one eyeball and with each USG devices. Then the two clinicians independently performed three measurements from the transverse axis and three measurements from the sagittal axis of one eye using previously recorded videos. We used special software for V-Scan devices to perform the measurements of previously recorded ONSD footage. The ONSD measurement was performed according to the new Optic Nerve Sonography Quality Criteria. [8]. While recording proper images for measurement, we considered the following matters: clear discrimination of optic nerve, internal and external border of the subarachnoid space; capturing close proximity of the hypoechogenic regions which belong to optic nerve fibers and vitreous fluid inside the eye globe and lens-free images. Measurements were made perpendicularly to the longitudinal axis of the nerve 3 mm behind the vitreoretinal conjunction where the optic nerve is the thickest (Fig. 1). Mean transverse ONSD (mtONSD was calculated as the average of three transverse measurements), mean sagittal ONSD (msONSD was calculated as the average of three sagittal measurements) and mean ONSD (mONSD was calculated as the average of mtONSD and msONSD) were calculated from the ONSD measurement values obtained with each device for one eye.

Fig. 1

Picture of optic nerve sheath diameter measurements with different devices. a Measurements with Pocket Sized Ultrasonography Device (VScan). b Measurements with Standart Ultrasonography Device (VividQ). Created by MicroDicom Version 2.9.0 Computer Program. mm: millimeter

Statistical analysis

The necessary sample size was found to be 32 patients according to power analyses (α = 0.05; β = 0.8) conducted on GPower 3.1.9.2 software package. Continuous variables were described as mean ± standard deviation or median (interquartile range) depending on whether the distribution was normal or not. P < 0.05 was taken as a measure of statistical significance. Bland–Altman analysis was used to assess if there were agreement and any significant proportional bias between the measurements. Interobserver reliability of ONSD measurements was assessed using the intraclass correlation test. ICC > 0.7 (intraclass correlation coefficient) was taken as a measure of strong correlation. One-way analysis of variance (ANOVA) test was used for the comparison of more than two group means and it was also used to estimate the variance between three different measurements performed on each axis with each USG device. The Paired Samples T test was used to investigate the difference between two dependent variables that fit the normal distribution. Statistical analyses were carried out with the IBM SPSS statistical program version 22.

Results

The total number of patients included in this study was 35. Appropriate images couldn’t be taken in one patient from the sagittal axis, and in one patient from the transverse axis. Demographic data, arterial blood gas analysis, and admission diagnoses were presented in Table 1.ICU duration of the patients was 17.25 ± 16.01 days. Eleven (31%) patients died during their follow-up in the ICU. For the evaluation of the agreement between PSUD and SUD relating to the measurement of ONSD, the difference and means of mtONSD, msONSD, and mONSD values were calculated. According to the Shapiro–Wilk test, mtONSD, msONSD, and mONSD were observed to fit a normal distribution. There was agreement between mtONSD, msONSD, and mONSD values measured with each device according to Bland–Altman test (p = 0.166, p = 0.135, p = 0.066, respectively) without proportional bias (p = 0.544, p = 0.868, p = 0.929, respectively) (Table 2). The Bland–Altman Test graphics were shown in Fig. 2. Similar favorable results were also obtained between each ONSD measurement with each USG device in the sagittal and transverse axis (Fig. 3). According to ANOVA test results, the total variance values between the three different measurements obtained using SUD in the transverse and the sagittal axis were ± 0.60 mm and ± 0.61 mm, respectively. The total variance values between the three different measurements in the transverse and sagittal axis with PSUD were ± 0.61 mm and ± 0.62 mm, respectively. During the assessment of interrater reliability, Intraclass correlation (ICC) coefficients for mtONSD, msONSD, and mONSD values obtained using SUD were found to be greater than 0.7 (ICC coefficient = 0.790, p = 0.0001, ICC coefficient = 0.817, p = 0.0001 and ICC coefficient = 0.844, p = 0001, respectively). Similarly, for mtONSD, msONSD, and mONSD values obtained using PSUD, ICC coefficients were found to be greater than 0.7 (ICC coefficient = 0.763, p = 0.0001, ICC coefficient = 0.814, p = 0.0001 and ICC coefficient = 0.843, p = 0.0001, respectively). mtONSD was 5.40 ± 0.58 mm, msONSD was 5.43 ± 0.61 mm, and mONSD was 5.43 ± 0.57 mm, all of which were obtained using SUD. mtONSD was 5.52 ± 0.62 mm, msONSD was 5.53 ± 0.59 mm, and mONSD was 5.53 ± 0.57 mm, all of which were obtained using PSUD. There was no statistically significant difference between mtONSD and msONSD measurements according to the Paired Samples T-test results for each USG device (mean difference = − 0.05 ± 0.36 mm, p = 0.441 for SUD and mean difference = 0.00 ± 0.41 mm, p = 0.893 for PSUD). In addition, there was a good correlation between mtONSD and msONSD for each USG device (r = 0.767 for SUD and r = 0.816 for PSUD).

Table 1

Demographic and clinical findings of the patients

N = 35
Age (years)	74 ± 13
Gender, F/M (n)	16/19
BMI (kg/m2)	26 ± 5
APACHE II—Score [0–71]	20 ± 8
Encephalopathy score [0–4]	2 [0–3]
Kelly Matthay score [0–6]	2 [1–4]
SOFA score [0–24]	4 [3–7]
GCS [0–15]	14 [7–15]
No mechanical ventilation, n (%)	20 (57)
Invasive mechanical ventilation, n (%)	11 (31)
Non-invasive mechanical ventilation, n (%)	4 (11)
Arterial blood gas analysis on the ONSD measurement day
PH	7.4 ± 0.7
PaO2 (mmHg)	76 ± 27
PaCO2 (mmHg)	43 ± 13
SpO2 (%)	91 ± 8
HCO3 (mmol/L)	29 ± 12
Lactat (mmol/L)	1.5 ± 0.5
Admission diagnosis
Pneumonia, n (%)	21 (60)
Cardiac, n (%)	21 (60)
COPD attack, n (%)	16 (45)
Neurologic, n (%)	15 (42)
Sepsis, n (%)	14 (40)
Renal, n (%)	12 (34)
Other infections, n (%)	9 (25)
Metabolic, n (%)	7 (20)
Septic shock, n (%)	4 (11)
Interstitial lung disease, n (%)	3 (8)
ARDS, n (%)	2 (0.5)

N, n number, F female, M male, BMI Body Mass Index, kg/m kilogram per square meter, APACHE-II Acute Physiology and Chronic Health Evaluation Score, SOFA sequential organ failure assessment, GCS Glasgow Coma Scale, PaO partial arterial oxygen pressure, PaCO partial arterial carbon dioxide pressure, SpO Oxygen saturation, COPD chronic obstructive pulmonary disease, ARDS acute respiratory distress syndrome

Table 2

Bland Altman Test for ONSD measurement of transverse and sagittal section

	Mean value (mm)	%95 CI of mean value (mm)	AgreementP value	Proportional biasP value
Difference between mtONSD	− 0.113 ± 0.466	(− 0.276)–(0.049)	0.166*	0.544**
Difference between msONSD	− 0.096 ± 0.366	(− 0.224)–(0.031)	0.135*	0.868**
Difference between mONSD	− 0.101 ± 0.315	(− 0.210)–(0.007)	0.066*	0.929**

CI Confidence Interval, mm millimeter, mtONSD average of three ONSD measurements obtained from the transverse section of the eye with each USG device, msONSD average of three ONSD measurements obtained from the sagittal section of the eye with each USG device, mONSD average of mtONSD and msONSD

*There is agreement

**There is no proportional bias

Fig. 2

Bland–Altman test graphics of mtONSD, msONSD and mONSD for pocket-sized USG device, and standard USG device. mtONSD: Average of three ONSD measurements obtained from the transverse section of the eye with each USG device; msONSD: Average of three ONSD measurements obtained from the sagittal section of the eye with each USG device; mONSD: Average of mtONSD and msONSD; mm: millimeter

Fig. 3

Bland–Altman test graphics for each ONSD measurement. ONSD: optic nerve sheath diameter; mm: millimeter

Discussion

According to the results of this study, the agreement between PSUD and SUD relating to ONSD measurements was very good, and there was no proportional bias between the multiple measurements obtained from each USG device. The measurements performed by each clinician with each device overlapped through high ICC coefficients. Our results suggest that PSUDs are highly reliable in ONSD measurements compared to SUDs. Therefore, PSUDs may be suggested as useful as SUDs in evaluating ONSD in in vivo environment. When we look at the Bland–Altman test graphics of our study, we can see that there are 0.80 mm positive and -1.02 mm negative limits of agreement for transverse axis, there are 0.62 mm positive and − 0.81 mm negative limits of agreement for sagittal axis, and lastly there are 0.51 mm positive and − 0.72 mm negative limits of agreement for both axis in average. Moreover, when all Bland–Altman graphics in Fig. 3 are examined one by one, larger limits of agreement values can be seen. These limits of the agreement are not only affected by the difference in measurement capability between the two devices but also by the intraobserver variability. When we look at the literature for intraobserver variability values of ONSD measurements, we can see that our values for agreement limits are similar to those in the literature. For example, in a study evaluating the reproducibility of SUD in ONSD measurement, the limits of agreement were found as 0.62/− 0.77 mm [14]. In another study, a maximal difference between 0.43 and 1.19 mm was detected between the repeated ONSD measurements of the observers [15]. Finally, Baurlee et al. found that the limits of agreement between the repeated measurements of the observers were 0.85/− 0.65 mm for the right eye and 0.77/− 0.69 mm for the left eye [16]. On the other hand, in our study, we found that there was ± 0.6 mm variance between all measurements of the observers. This finding was in line with those in the literature. Therefore, we think that these agreement limit values between each USG device obtained in our study according to Bland–Altman test statistics can be explained by the intraclass variation between the observers. Also, there was a clinically acceptable difference between the measurements of each device in the transverse and sagittal axis according to the mean difference and 95% confidence interval of mean difference (Table 2).

Although SUDs are valuable non-invasive assessment tools, they are expensive and may not be useful for daily clinical practice in ICUs and emergency departments. Consequently, smaller and more economic USG devices have been developed and their image qualities are getting better every other day. PSUDs allow clinicians to evaluate physical examination findings quickly and they are quite important for physicians especially in emergency services in terms of avoiding disruption of workflow and reducing the waiting time of patients [17]. PSUDs have been shown to be suitable devices compared to SUDs for demonstrating several abdominal pathologies. On the other hand, assessment of solid organ pathologies by these devices is still a problem because of their narrow field of vision and poor penetration property. As a result, in wider and/or deeper areas of interest, the yield and viewing capacity of PSUD is reduced [17]. However, in the context of the assessment of ONSD, penetration requirement is low and the investigation area is quite narrow. It is usually possible to get good quality images of the optic nerve behind the hypoechogenic vitreous in the absence of cataract, mass or the artificial lens inside the eye globe [8]. In relation to this information, PSUDs (VScan) have a similar degree of accuracy and yields consistent results with SUDs in an ultrasound study on a simulation model for ONSD measurement and these results need to be confirmed by further in vivo studies [18]. PSUDs have also been found beneficial for the assessment of other organs and systems. For example, the use of PSUDs (for lung, heart and inferior vena cava USG) in the evaluation of patients who are admitted to emergency services with acute respiratory problems is feasible, reliable, and beneficial for high yield and speed [12]. Similarly, PSUDs are also found to be as efficient as SUDs in the evaluation of thickening and excursion of the diaphragm in ICU patients, but the quality of the image is low in obese and edematous patients [13]. Also, PSUDs are reliable in demonstrating proximal deep venous thrombosis compared to venous duplex USG [19].

Many medical conditions may lead to increased ICP, such as hypercarbia, hypoxemia, hepatic encephalopathy, shock, hypertension, hyperpyrexia, central nervous system infections, which are frequently seen in ICU patients. Also, more importantly and, maybe, more frequently, ventilation strategies commonly used in patients with ARDS are potentially associated with an increased risk of intracranial hypertension. In particular, the use of higher level of positive end-expiratory pressure, recruitment maneuvers and prone positioning can have undesirable effects on ICP. In a recent study, ONSD was tested as a surrogate non-invasive measures of ICP changes during prone position and/or PEEP application in non-brain-injured patients who underwent spine surgery [20]. In this study, mean ONSD increased significantly upon prone positioning and upon a PEEP application of 8 cm H2O [20]. For this reason, if a patient has brain injury and requires mechanical ventilation, particular care is recommended not only for lung protection but also for brain protection [11]. On the other hand, hypercarbia is frequently seen in patients in our intensive care practices. Hypercapnia is widely known to increase the cerebral blood flow, cause cerebral edema and therefore increased ICP [21]. In healthy individuals, hypercarbia (6.5 kPa = 48.7 mmHg) increased the ONSD measurement values from 4.2 to 4.7 mm and it quickly returned to basal values with normocarbia [22]. But in ICU patients, once quite high carbondioxide levels are observed, greater ONSD values can be expected. Also, hypoxemia just as hypercarbia leads to increased cerebral blood flow and increased ICP. However, ONSD measurement values which are correlated with raised ICP have a wide range and are not the same for each study. For example, in a recent meta-analysis, the cut-off values of ONSD increased from 5 to 6.3 mm [5], while changing from 5 to 5.3 mm in another [23]. This wide range of cut-off values is a problem for proper use of USG for this purpose. Cumulative data related with intracranial pressure and ultrasonographic measurement of ONSD have shown that an increase in ONSD above a cut-off value of 5.7–6 mm strongly suggests an increased ICP > 20 mmHg [24]. In our study, mean measurements of ONSD with both devices were close to the upper limits of normal values given in the literature [24, 25]. We can think that our measurements are close to the upper limit as a result of many situations described above which lead to increased ICP. But it is clear that we need more patients to clarify this hypothesis.

Since the aim of our study was to compare the performances of the devices but was not to compare the patient groups for ONSD measurements, we did not perform detailed statistical analysis for the comparison of patient groups. In addition, the number of patients ventilated with high pressure and volumes and treated with recruitment maneuvers in the prone position was very low in this study population. Because of these reasons, we could not divide our patients into clinical groups, and thus we could not compare their ONSDs. Additionally, it is not possible to say if the results could have changed with increasing ICP. Clinical information such as the length of stay in the ICU, the percentage of mortality and the underlying disease variability in the patient group included in this study is consistent with the general information of our ICU, where the study was conducted. However, our study showed that PSUDs might be useful to measure ONSD as much as SUDs. In our opinion, this is important in ICU and emergency settings. Recent developments have shown that the use of PSUDs is increasing. Some developed countries are using them in their ambulance services. Cardiologists, internists, and general practitioners prefer them in the examination of the heart instead of physical examination. Finally, they may be more practical and available for resource-limited emergency settings and countries. European Association of Echocardiography recommends them in the completion of physical examination, fast initial screening in an emergency setting, first cardiac evaluation in the ambulance, and determination of complete echocardiography requirement [26].

As a result, in our study, there was good agreement, but no bias, and a good interrater correlation between the operators performing transbulbar sonography for measuring ONSD with two devices. This result suggests that pocket-sized devices can be used reliably for this aim. On the other hand, despite some studies reporting that learning how to measure ONSD is simple and easy and the learning curve is steep, some authors have concerns and hesitations about the risk of a high probability of bias in the measurements [27, 28]. For these reasons, consideration of quality criteria and experience of physicians are very important factors for a reliable measurement of ONSD. Finally, in some PSUD devices, there are two choices for performing a measurement. One of them is to do measurements on the device itself and the other is to use their special software. In our study, we used V-Scan Gateway software in order to increase our chances to do more precise measurements. This software can be easily used after it is set up on any computer in ICUs.

Conclusion

The results of this study suggest that PSUDs have similar reliability and agreement in ONSD measurements with SUDs, and therefore can be preferred especially in the emergency and intensive care settings when invasive ICP monitoring is necessary but not available or applicable.