Novice emergency physician ultrasonography of optic nerve sheath diameter compared to ophthalmologist fundoscopic evaluation for papilledema.

OBJECTIVES: The purpose of this study is to determine the sensitivity and specificity of novice emergency physician-performed point-of-care ultrasound diagnosis of papilledema using optic nerve sheath diameter (ONSD) against ophthalmologist-performed dilated fundoscopy. This observational study retrospectively analyzed results of ultrasound-measured ONSD of emergency department (ED) patients with suspected intracranial hypertension from a period spanning June 2014 to October 2017.
METHODS: This study concerns a population of ED patients at a large, tertiary-care urban academic medical center from June 2014 to October 2017 over the age of 18 years with primary vision complaints evaluated for papilledema both by an emergency physician-performed ultrasound and an ophthalmologist-performed fundoscopic examination during their ED stay. Sensitivity and specificity of emergency physician-performed ultrasound measurement of optic nerve sheath diameter in the diagnosis of papilledema were primary outcomes for this study.
RESULTS: A total of 206 individual patients (male 49%, female 51%; median age 45 years) were included in the study with a total of 212 patient encounters. Calculated sensitivity for the ocular ultrasound examination performed by emergency physicians to diagnose papilledema was 46.9% (95% confidence interval [CI], 32.5% to 61.7%), and specificity was 87.0% (95% CI, 82.8% to 90.5%). Positive predictive value and negative predictive value were calculated to be 35.4% (95% CI, 23.9% to 48.2%) and 91.5% (95% CI, 87.8% to 94.4%), respectively.
CONCLUSIONS: Sonographic measurement of ONSD by emergency physicians has low sensitivity but high specificity for detection of papilledema compared to ophthalmologist-conducted fundoscopy.

BACKGROUND

Although the use of point‐of‐care ultrasound (POCUS) has expanded considerably in the last few years, one ultrasound application of particular interest is the evaluation of patients for papilledema when increased intracranial pressure (ICP) is clinically suspected. Various etiologies may contribute to increased ICP. Of these, idiopathic intracranial hypertension (IIH) is a frequent indication for papilledema evaluation at the bedside. These patients often present to the emergency department with acute onset headache with or without vision changes. Therapeutic lumbar puncture is indicated for those IIH patients in whom papilledema is detected. Traditionally, a dilated funduscopic examination by an ophthalmologist would be used to detect papilledema, but in settings with limited access to an ophthalmology consult, the use of ocular ultrasound to detect papilledema may expedite diagnosis and therapy. The criterion standard for increased ICP detection is direct measurement via opening pressure during lumbar puncture or with external ventricular drainage, which is both time consuming and brings inherent risk. Thus optic nerve sheath diameter (ONSD) and subsequent papilledema may serve as a surrogate for increased ICP and these alternative modalities have been explored recently.

IMPORTANCE

Measurement of the ONSD by head computed tomography (CT) or ultrasonography is one such emerging method. Both CT and ultrasound measurements of ONSD have been correlated with measured ICP via ICP monitoring, and a meta‐analysis of literature concerning CT and ocular ultrasound for detection of ICP showed that ultrasound performed favorably when compared to CT for diagnostic accuracy of increased ICP. Notably, studies have shown that emergency physicians can perform POCUS measurements of ONSD accurately when compared to CT measurements in patients suspected of having elevated ICP. , Despite the availability of literature describing the use and accuracy of POCUS in the setting of suspected increased ICP, there is a paucity of data comparing the detection of papilledema by performing ocular ultrasonography and ophthalmologists performing fundoscopic examination, especially when the emergency physicians are considered ultrasound novices. Oftentimes, emergency physicians lack confidence in their ability to clearly see the fundus in a non‐dilated eye, and to our knowledge no ultrasound study compared POCUS to an ophthalmologist's dilated fundoscopic examination.

Goals of this investigation

This study aims to assess sensitivity and specificity of emergency physician‐performed ocular ultrasound for diagnosis of papilledema compared to ophthalmology evaluation.

METHODS

Study population and setting

Patients over the age of 18 years presenting to the ED with primary vision complaints who received both an ED formal ophthalmology consultation and bedside ocular ultrasound performed by an emergency physician in a tertiary care, > 100,000 annual volume, urban academic medical center from 2014–2017 were included in the study. Operators were novice resident emergency physicians, most of whom were on their intern emergency ultrasound elective and had varying levels of POCUS experience entering residency. Images were collected on a Phillips Sparq or a Sonosite Xporte ultrasound machine using a high‐frequency linear transducer with patients in an upright position. Optic‐nerve sheath diameter was measured 3 mm posterior to the papilla with calipers placed at the interior border of the Hypoechoic nerve, measured in two planes with the average of the measurements recorded. Patients were excluded if they did not receive both formal ophthalmology evaluation and bedside ocular ultrasound performed by an emergency physician.

The Bottom Line

This study evaluated use of ocular point‐of‐care ultrasound (POCUS) by novice emergency physicians to diagnose papilledema detected by ophthalmologists. Contrary to prior studies, POCUS sensitivity was low ‐ only 47%. This study highlights the need for proper training, standard examination techniques, and monitoring to ensure emergency physician proficiency with this technique.

Data collection

Retrospective chart review was conducted for adult patients presenting to the emergency department with primary vision complaints assessed for papilledema via ophthalmologic examination and POCUS by emergency physicians. All POCUS ocular examinations stored within the centralized image repository in the aforementioned date range were queried for concomitant ED visit ophthalmology consultation via electronic health record database query. Data collection from the electronic health record used in this ED targeted results of emergency physician‐performed transorbital ocular POCUS as well as ophthalmologist‐performed fundoscopic examination. Additionally, ocular ultrasound results were reviewed in QPath via a secure ultrasound database where the emergency medicine ultrasound images are stored. This study received approval from the institutional review board.

Data analysis

During retrospective chart review, the results of the ophthalmologist‐performed fundoscopic examination including detection of papilledema and grading of the papilledema (if applicable) were recorded alongside emergency physician‐measured ONSD. ONSD measurement greater than 5 mm on ocular ultrasound was diagnosed as papilledema. Additionally the demographic factors, including age and sex, were recorded for each patient. Demographic variables were compared between patients with and without funduscopic‐diagnosed papilledema using a chi‐square test (gender) and Mann‐Whitney U test (age) in the Statistica software package (TIBCO, Inc.). Statistical significance was determined using α = 0.05.

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the emergency physicians’ evaluation for papilledema based on ocular ultrasound were calculated based on the following definitions. Opinions of the ophthalmologist and emergency physicians were considered to agree in the cases of true positives and true negatives but to diverge in the cases of false positives and false negatives. Sensitivity, specificity, PPV, and NPV were calculated using R statistical software (R Foundation for Statistical Computing, Vienna, Austria) and the “epiR” package.

A receiver operating characteristic (ROC) curve analysis was performed using R statistical software and the “optimal cut points” package. An optimal ONSD cutoff value was determined based on the method that maximized sensitivity and specificity. The area under the curve (AUC) was calculated with 95% confidence intervals (CIs).

RESULTS

Of 215 unique patients comprising 222 distinct ED visits (encounters) originally included in the study, 9 received an ocular ultrasound performed by an emergency physician, but the ONSD was not documented; these were thus excluded from the study. An additional 4 patients who received both an ophthalmologist‐performed fundoscopic examination and an emergency physician‐performed ocular ultrasound had only 1 ONSD measurement recorded despite 2 or more ophthalmologist evaluations. A total of 206 individuals (51% female, 49% male) were included in the analysis for a total of 212 considered encounters performed by 55 unique resident physician operators ranging from PGY1–PGY4. The operator experience varied as such: 10 operators had only 2 ocular ultrasound meeting inclusion criteria, the other 45 operators performed between 2 and 20 ocular ultrasounds meeting study inclusion criteria. Of those 212 encounters included in the final analysis, 28 (13.2%) resulted in a diagnosis of papilledema by funduscopic examination for at least 1 eye. There was no significant difference in sex among all encounters (female: 53.3%, male: 46.7%), although there was a significant difference (χ2 = 8.28, df = 1, P = 0.004) in sex between encounters with (female: 78.6%; male: 21.4%) and without (female: 49.5%; male: 50.5%) fundoscopic‐diagnosed papilledema. There was also a significant difference (P = 0.003) in age between individuals with (median: 37.0 years, interquartile range: 27.0‐46.5 years) and without (median: 48.0 years, interquartile range: 33.0‐60.0 years) fundoscopic‐diagnosed papilledema.

A total of 372 instances of ophthalmologist fundoscopy and emergency physician‐performed ONSD measurement were used to calculate the sensitivity and specificity as well as positive and negative predictive value of the emergency physicians’ ocular ultrasound evaluation for papilledema (Table 1). Papilledema was diagnosed by fundoscopy for 49 (13.2%) eyes examined, whereas 17.5% (N = 65) of eyes examined had ONSD measurements exceeding 5.0 mm (Table 1).

TABLE 1

Diagnosis of papilledema by the emergency physician‐performed ocular ultrasound versus ophthalmologist‐performed fundoscopy

	Papilledema on fundoscopy	No papilledema on fundoscopy	Total
Papilledema on ultrasound (ONSD >5.0)	23 (true positive) (6.2%)	42 (false positive) (11.3%)	65 (17.5%)
No papilledema on ultrasound (ONSD ≤ 5.0)	26 (false negative) (7.0%)	281 (true negative) (75.5%)	307 (82.5%)
	49 (13.2%)	323 (86.8%)	372 (100%)

ONSD, optic nerve sheath diameter.

Sensitivity for the ocular ultrasound (cutoff = 5.0 mm) performed by emergency physicians to diagnose papilledema was calculated to be 46.9% (95% CI, 32.5% to 61.7%), and specificity was 87.0% (95% CI, 82.8% to 90.5%). The PPV was 35.4% (95% CI, 23.9% to 48.2%), and the NPV was 91.5% (95% CI, 87.8% to 94.4%).

ROC analysis identified an ONSD cutoff value of 4.6 mm to maximize sensitivity (67.3%) and specificity (67.5%) for use as a diagnostic test (Figure 1). The AUC associated with a cutoff value of 4.6 mm was 73.7% (95% CI, 65.5% to 82.0%). PPV and NPV for an ONSD cutoff value of 4.6 were 23.9% (95% CI, 19.9% to 37.9%) and 93.2% (95% CI, 87.9% to 94.6%), respectively.

FIGURE 1

Receiver operating characteristic (ROC) curve to evaluate diagnostic capability of optic nerve sheath diameter (ONSD) and measurements to predict papilledema. AUC, area under the curve; MaxSpSe, maximised specificy and sensitivity

LIMITATIONS

Other than the limitations inherent with a retrospective study, this research was affected by potential bias owing to setting, varying levels of ultrasound operator experience, lack of uniform criteria for ophthalmology consult and subsequent fundoscopic examination as well as for emergency physician‐performed POCUS examination, failure to document final diagnoses or further testing to evaluate the presence or absence of increased ICP, potential complications in using fundoscopy as the “criterion standard,” and few instances of failure to record measurement data, which led to exclusion of otherwise useful data points. This research concerned patients at a large, urban tertiary care facility; therefore, referral bias may have played a role as well as selection bias owing to undocumented phantom examinations not being reviewed as part of the study. As mentioned previously, the ultrasound operators were trainees rather than experienced clinicians, which may have affected their ability to accurately measure ONSD by POCUS. Furthermore, although ophthalmologists are certainly experts, the reliability of a fundoscopy examination for increased intracranial pressure is not absolute. A recent study has shown that the sensitivity and specificity of fundoscopic examination for increased intracranial pressure are 100.0% and 35.4%, respectively. However, this study does not detail blinding of the emergency physicians performing direct ophthalmoscopy to final diagnosis or CT findings supportive of increased ICP. The findings of Phase I of the Fundus Photography vs Ophthalmoscopy Trial Outcomes in the Emergency Department (FOTO‐ED) study suggest emergency physicians frequently miss fundoscopic evidence of increased ICP except when such a fundus finding has already been diagnosed. The current literature reports variable accuracy of ophthalmologist‐performed fundoscopy for the detection of true papilledema, including a study that cites an 85.3% sensitivity for neuro‐ophthalmologists interpreting fundus photographs and another that reports a considerable 30% rate of misinterpretation of papilledema , ; thus accuracy of ophthalmologist‐performed fundoscopy for papilledema is likely unknown. It is also important to note that in acute settings such as trauma, increased intracranial pressure does not result in papilledema and may take days to develop. It is also known that a subset of pseudotumor patients will have elevated opening pressures on lumbar puncture, without evidence of papilledema, making ONSD an imperfect screening tool. Finally, as noted in the Results section, there were 9 instances of patients undergoing ocular ultrasound without a measurement being recorded and another 4 in which the ocular ultrasound measurement was documented for 1 eye rather than bilaterally.

DISCUSSION

Compared to the standard of care of formal ophthalmology evaluation via fundoscopy, the ocular ultrasound performed by novice emergency physicians has low sensitivity and high specificity, indicating that although it may be a useful tool for ruling in a diagnosis of papilledema, it is not sufficiently sensitive to rule out papilledema.

The use of POCUS has seen considerable growth in the last decades, so much so that current emergency medicine trainees may feel more comfortable with ocular ultrasound than funduscopic exam. Compared to more traditional imaging modalities, POCUS is non‐invasive and inexpensive and requires no radiation exposure to provide rapid and accurate diagnoses. POCUS is not limited to the viscera of abdominal and thoracic cavities and has shown to be useful for examination of other complaints in the ED including musculoskeletal and ocular complaints. As a superficial fluid‐filled structure, the eye is an ideal candidate for examination by ultrasound at the bedside. Although slit lamp examination and CT are common and effective modalities for diagnosing eye conditions in the ED, examination by POCUS is immediately accessible and can facilitate the lack of a dilated fundoscopic examination; though it does not supplant the need for bedside funduscopic examination. Blavias et al showed that bedside ocular ultrasound examination for various ocular pathologies by emergency physicians largely agreed with confirmatory studies (either CT, complete ophthalmologic evaluation, or both) with a resulting sensitivity of 100% and a specificity of 97.2%.

We demonstrated that bedside ocular ultrasound performed by emergency physicians to measure the optic nerve sheath diameter for diagnosis of papilledema was a low‐sensitivity, high‐specificity test when compared to evaluation by ophthalmologists performing fundoscopic examination. This is based on the currently accepted ONSD cutoff of 5.0 mm, but our ROC analysis suggests that a lower cutoff may improve sensitivity, making this a more desirable screening modality in the ED. Papilledema is a sign of elevated intracranial pressure, a potentially disastrous diagnosis if unrecognized or unaddressed. The FOTO‐ED study found that over 80% of unknown ocular findings were missed by emergency physicians but captured by non‐mydriatic fundus photography. We believe our study is the first to compare the ability of novice emergency physicians to accurately detect papilledema on ocular ultrasound against fundoscopic examination by an ophthalmologist.

The observed combination of sensitivity and specificity for detecting increased ICP via ocular ultrasound examination diverges from that reported in the literature. We explored a potential explanation for this finding in the relatively conservative cutoff of ONSD > 5 mm as a pathologic finding. A meta‐analysis of literature regarding the correlation of ONSD measurement with ICP showed good accuracy of ultrasonography‐based detection of intracranial hypertension; however, only one study included in this meta‐analysis used an ONSD cutoff of > 5 mm for pathology, whereas the other included studies used a pathologic threshold of 5.2‐5.9 mm. Interestingly, in our calculations of sensitivity and specificity, we failed to achieve the same high‐sensitivity, high‐specificity combination reported by other investigators, largely attributed to non‐uniform or incorrect ONSD measurements by novices. ROC analysis did show that an ONSD cutoff of 4.6 mm maximized sensitivity (67.3%), which is potentially useful in screening for papilledema (Figure 1), though it should be recognized that no strict cutoff has been widely accepted.

There was a high proportion of false negatives among the patient encounters included in the study with an average ONSD measurement of 4.25 ± 0.65 mm. No current literature supports this smaller ONSD as being indicative of increased intracranial pressure or papilledema. Therefore, we explored the limitation of the performing emergency physicians’ training level, as there was not any standardized ultrasound training for the operators to better stratify experience before this study. All of the operators who contributed ONSD measurements to the study were ultrasound novices and resident trainees and did not include anyone with advanced or fellowship training. Notably, much of the published literature exploring emergency physician use of ONSD measurements on ocular ultrasound as an indicator of raised ICP report using experienced ultrasonographers with ultrasound training anywhere from an hour‐long course to years of fellowship and directorship‐level experience. , , , A recent comparison of ONSD measurements by ultrasound fellowship‐trained emergency physicians to ONSD measurements by resident emergency physicians (with varying levels of training) interestingly showed greater variation in agreement of measurements by resident emergency physicians (intraclass correlation coefficient 0.50; 95% CI, 0.25–0.89) versus their counterparts who had completed ultrasound fellowship training (intraclass correlation coefficient 0.73; 95% CI, 0.44–0.96). A 2019 meta‐analysis showed that bedside ONSD measurements using POCUS were an accurate means for detecting increased intracranial pressure for all operators but also reported a slightly higher sensitivity for trained operators (95%) versus novices (93%). This finding together with our results perhaps presents an opportunity for further education in POCUS ocular examinations for resident emergency physicians that will afford more confidence in their diagnostic abilities using ocular ultrasound. Despite not correlating well with fundoscopy, POCUS may still be the most comfortable tool in an emergency physician's armamentarium for non‐invasive assessment of increased intracranial pressure.

Sonographic measurement of ONSD by novice emergency physicians has low sensitivity but high specificity for detection of papilledema compared to ophthalmologist‐conducted fundoscopy, limiting its use as a screening tool for elevated ICP. Despite not correlating well with fundoscopy, POCUS may still be the only comfortable tool that an emergency physician adopts for non‐invasive evaluation of increased intracranial pressure. Future studies are needed to assess the effect of training, uniform procedures/measurements as well as experience over time on ONSD measurements. The authors would like to thank Dr. Tiffany Fong and the Department of Emergency Medicine at Johns Hopkins Hospital for their gracious support of this research endeavor.

CONFLICT OF INTEREST

The authors have no conflicts of interest to report.

AUTHOR CONTRIBUTIONS

Study design and data collection: CLW, DS. Data interpretation: CLW, SML, DS, LH. Manuscript preparation: CLW, SML, DS, DJ. Manuscript review: CLW, SML, DS, LH, DH, CO.