Accuracy of Remote Diagnosis of Acute Posterior Segment Pathology by Residents and Attendings Captured with a Smartphone and Standard 20/28D Lens.

Background and Objective: To assess the use of a smartphone-based fundus camera in diagnosing acute posterior segment disease in the emergency room (ER) of an academic center, where standard fundus cameras are currently unavailable. Study Design/Patients and
Methods: Images were obtained on 10 eyes with various types of posterior segment pathology. These were analyzed retrospectively by 35 reviewers (21 residents and 14 attending physicians). Accuracy of diagnosis was compared to in-office fundus photography.
Results: The average probability of correct diagnosis was over 78%. There was no difference between resident and attending reviewers, except for one of the vitreous hemorrhage cases where residents performed better (p=0.039).
Conclusion: Smartphone-based cameras are effective in screening for ocular diseases in the ER of academic centers, where residents and fellows are often the first providers to see various types of vision and life-threatening pathology.

Background and Objective

Fundus photography is a fundamental tool used to diagnose and monitor posterior segment pathology. However, it is typically unavailable in many situations such as emergency departments, hospital floors, or underserved areas, where urgent ophthalmic consultations may be necessary yet difficult to obtain. Furthermore, general healthcare providers in these settings typically have not had the training needed to examine the posterior segment. This can lead to life and vision threatening diseases going undiagnosed or having a delayed diagnosis. Indirect ophthalmoscopy and handheld contact fundus cameras provide an accessible and reliable method for physicians to determine the need for and urgency of an ophthalmic consult. However, their availability and high cost limit their use in certain settings.1 New smartphones equipped with high-resolution cameras are increasingly being used to screen for anterior and posterior segment pathology to circumvent this problem.2

Smartphone-based fundoscopy has proven its efficacy in diagnosing diabetic retinopathy, macular degeneration, and retinopathy of prematurity. However, its application in diagnosing other posterior segment pathologies, especially in the emergency care setting, has been largely limited.3 This method of imaging could help increase access to necessary ophthalmic intervention and reduce unnecessary cost on the healthcare system for non-urgent patient presentations. A study by Padovani-Claudio et al analyzed information on over 100,000 subjects who were seen across multiple emergency departments for eye-related complaints. They found that only 8.6% of these patients had clearly urgent ocular pathology.4 Furthermore, the Fundus Photography versus Ophthalmoscopy Trials Outcomes in the Emergency Department (FOTO-ED) study showed that only 14% of the patients with complaints and conditions in which fundus examination is considered important had direct ophthalmoscopy performed by an emergency department physician during their evaluation.5–7 Alternatively, ultrasound has been explored as an imaging modality to examine ophthalmic complaints in the emergency care setting. While it may offer advantages for diagnosing some emergencies such as retinal detachments, the diagnostic utility varies widely with emergency practitioner experience. Additionally, many acute retinal and vitreous abnormalities without ultrasonographic findings may be missed, especially those that cause no significant structural changes to the retina, choroid, or sclera.8

These studies highlight the challenges that emergency departments face in receiving large volumes of non-urgent eye complaints while also mitigating the possibility of missing critical and urgent ocular disease. Smartphone-based fundoscopy could provide an innovative and inexpensive option for hospitalists, primary care physicians, and emergency physicians to triage, document, and communicate with ophthalmologists in order to treat patients in a more appropriate and timely manner, thus reducing the burden of vision threatening disease.

In academic centers, where residents and fellows are often the first ophthalmic provider to see the patient, providing retinal specialists with high-quality images of the posterior segment can improve patient care. Recently, there has been evidence of diagnostic utility for mydriatic smartphone images and fundus photography amongst both experienced and inexperienced trainees.9 However, there is a lack of data that compares how post-graduate training affects the accuracy of the diagnosis and triage of posterior segment disease. Moreover, previous studies on acute retinal pathology have mostly focused on retinal detachments. The purpose of this study is to assess the use of a smartphone-based fundus camera in the diagnosis of acute posterior segment pathology in the emergent setting. Unlike previous studies, we compare the accuracy between residents and attendings, and include a wider breadth of posterior segment pathology, from ocular trauma to neuro-ophthalmic disease.

Patients/Materials and Methods

Population

This is a retrospective, non-randomized study of 10 eyes with acute posterior segment pathology that presented to the emergency room at an academic medical center. This study was approved by the institutional review board of Oakland University William Beaumont School of Medicine and adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from patients to access their medical records and was required by IRB protocol before enrollment into the study. The study was compliant with Health Insurance Portability and Accountability Act. Funduscopic examination findings were obtained by one examiner (S.B.) after dilation with 1% tropicamide and 2.5% phenylephrine. S.B. was an ophthalmology resident at the time of this study. He went through a short training process of acquiring images on 10 control subjects then on patients with evident acute posterior segment pathologies. Objective metrics for quality of the images were that the details of the optic nerve, macula, vessels, and midperiphery could be seen. If media opacities in the lens or vitreous precluded a view of the posterior segment, then they were excluded. Images with vitreous opacities were only included if they were a part of the acute posterior segment pathology being ascertained, such as proliferative diabetic retinopathy. These images were still excluded if there was no view of posterior pole and midperipheral structures. Patients were excluded if they were below the age of 18, if there were any media opacities excluding the view of the posterior segment, or if they were unable to participate in a dilated fundus examination.

Image Acquisition

Posterior segment pathology was photographed using an iPhone camera with standard 20 or 28 diopter handheld Volk® lens. Sample images can be seen in Figure 1. Images were acquired over a timespan of around 3 months.

Figure 1

Sample fundoscopic photos taken with the smartphone camera. Top left image is from an eye with an optic nerve avulsion, top right is from a patient with hypertensive optic neuropathy, bottom left is from a patient a superior macula off retinal detachment, and the bottom right image is from an eye with a vitreous hemorrhage secondary to proliferative diabetic retinopathy.

Image Analysis

Images were presented to 35 blinded reviewers (14 attending and 21 resident physicians) to be retrospectively analyzed from the electronic medical record. Reviewers were provided with a short history of less than 200 characters. They were asked to make a presumed diagnosis, rate their confidence in their assessment of the photo, and grade the quality of the photo on a 5-point scale. These responses would be compared to the patient’s final in-office diagnosis in order to determine the accuracy. Reviewer responses were collected over a period of around 2 years.

Statistical Analysis

Statistical analyses were performed with SAS 9.4 (SAS Institute Inc., Cary, NC, USA). Categorical data were summarized into counts and proportions. and continuous data were summarized into means. Comparisons of accuracy between attendings and residents and accuracy between each individual case pathology were ascertained using Pearson chi-squared tests or Fisher’s exact tests where cell sizes were too small. P-values for both of these comparisons were considered significant if they were <0.05. All evaluations were done independently for each case pathology, regardless of whether or not they had the same pathology type. Comparisons of confidence in diagnosis between attendings and residents, associations between accuracy and quality of photo, as well as accuracy and confidence in diagnosis were assessed using Wilcoxon rank sum test. We used p-values <0.005 for statistical significance in these circumstances since they were being made across multiple case pathologies.

Results

Ten images were obtained from 10 eyes with acute posterior segment pathology. Table 1 presents the accuracy of reviewer diagnosis, based on the percentage of posterior segment pathology that they correctly identified. With regards to pathology type, macula off retinal detachments (RD), fungal chorioretinitis (FC), and hypertensive optic neuropathy were diagnosed with 100% accuracy. Horseshoe retinal tear case 2 and optic nerve avulsion were diagnosed with the lowest average accuracy. There were no statistically significant differences between the accuracy for residents and attendings except for vitreous hemorrhage case 2 (p=0.039), where residents performed better than attendings. In addition, there was no statistically significant difference between resident and attending physicians in the confidence of their diagnoses.

Table 1

The Accuracy of the Reviewers in Diagnosing Posterior Segment Pathology, as Determined by the Percentage of the Time That They Were Correct

Pathology	% Residents Correct	% Attendings Correct	P value Comparing Attendings to Residents*
Macula-off Retinal Detachment Case 1	100	100	1
Macula-off Retinal Detachment Case 2	100	100	1
Fungal Chorioretinitis	100	100	1
Hypertensive Optic Neuropathy	95.2	92.9	1
Vitreous Hemorrhage Case 1	100	100	1
Vitreous Hemorrhage Case 2	90.5	57.1	0.039
Non-Proliferative Diabetic Retinopathy with Diabetic Macular Edema	81	78.6	1
Optic Nerve Avulsion	38.1	50	0.486
Horseshoe Retinal Tear Case 1	95.2	92.9	1
Horseshoe Retinal Tear Case 2	9.5	14.3	1

Note: *Chi-squared test.

When comparing the accuracy for grading each individual case pathology amongst all 35 reviewers, there were only a few statistically significant differences. Horseshoe retinal tear case 2 had a lower average than all of the cases except for the optic nerve avulsion at p<0.005. Likewise, the average for optic nerve avulsion was lower than all of the cases except for horseshoe retinal tear case 2, vitreous hemorrhage case 2, and mild NPDR with DME. There were no other statistically significant differences between the accuracy for each of the 10 cases.

Table 2 depicts a correlation between the quality of the photo and confidence in diagnosis for each of the 10 cases. All of these correlations were significant at p<0.005, except for fungal chorioretinitis. There was a statistically significant correlation between confidence in diagnosis and accuracy of diagnosis (p=0.004) for optic nerve avulsion but none for the other associations. These associations were not applicable for the two macula-off retinal detachment cases and vitreous hemorrhage case 1 where the reviewers had 100% accuracy.

Table 2

A Correlation Between the Quality of the Photo and Confidence in Diagnosis for Each Reviewer in the 10 Cases

Case	Confidence in Diagnosis	Quality of Photo	Correlation Coefficient	P value for Associations
Macula-off Retinal Detachment Case 1	4.1	4.2	0.64	<0.001
Macula-off Retinal Detachment Case 2	3.9	3.7	0.55	0.001
Fungal Chorioretinitis	3.3	2.3	0.40	0.18
Hypertensive Optic Neuropathy	3.7	3.9	0.58	<0.001
Vitreous Hemorrhage Case 1	3.7	3.8	0.69	<0.001
Vitreous Hemorrhage Case 2	3.3	3.6	0.54	0.001
Non-Proliferative Diabetic Retinopathy with Diabetic Macular Edema	3.2	2.7	0.64	<0.001
Optic Nerve Avulsion	3.1	3.2	0.60	<0.001
Horseshoe Retinal Tear Case 1	3.5	2.8	0.82	<0.001
Horseshoe Retinal Tear Case 2	2.7	2	0.70	<0.001

Note: *Wilcoxon rank sum test.

Discussion

This study has demonstrated near equivalence of quality and diagnostic utility of mydriatic smartphone-based images obtained in the emergency department as compared to fundoscopic photographs. We were uniquely able to demonstrate that residents did just as well as attendings in grading fundus photos, if not better. This is important considering that many of these consults are placed in academic centers where residents or fellows are the first provider to see the patient and have to decide about whether attending input is needed. Furthermore, we demonstrated that this technique can be applied to diseases from multiple disciplines like neuro ophthalmology or uveitis. This is crucial as many neuro-ophthalmic or uveitic conditions can lead to potentially life-threatening pathology that may otherwise have been missed or experience a delay in treatment.

According to previous studies, handheld fundoscopic cameras can be effective in screening for ocular disease.10 While increasing evidence supports its use in the emergency care setting,7 several of these cameras are not cost effective and provide challenges with portability, interface difficulty, and storage. Mobile technology can provide a potential solution as its use in the healthcare setting has experienced an exponential rise over the past decade. Device costs have precipitously dropped, and the latest generations of smartphone camera resolution allow for accurate documentation and review of ophthalmic imaging.3 Paired with ease of use in obtaining high-quality images as demonstrated by the cross-sectional study by Adam et al, smartphone-based fundoscopy can change the way imaging is obtained and shared amongst ophthalmologists and consulting services alike.9

It has been postulated that photographer experience in standard fundoscopic imaging is important.5–7 Although photographers should be trained before acquiring smartphone-based imaging, the process of training appears to be more straightforward and can be generalized to even those who are not photographic technicians. One study indicated that medical students, who received a brief standardized training and experience, were able to capture retinal images on a smartphone that were accurately read and diagnosed by ophthalmologists.9 In our study, all the images were obtained by one individual with minimal training, yet the images were of sufficient quality to lead to accurate diagnosis and management in most cases. This suggests that smartphone-based imaging is easy to use and more accessible than standard fundus photography. Despite the higher resolution and ease of use of traditional fundoscopic imaging, advantages of smartphone imaging include easy access, high portability, low cost, and wireless connection to secure networks for data transmission. Recent work involving funduscopic imaging with smartphones reflects increasing interest in harnessing the potential of smartphone camera technology while maximizing ease of use for both ophthalmologists and non-ophthalmologists alike.9

Limitations of this study were the retrospective nature of the study, the lack of control subjects, the small sample size, the need for pharmacologic mydriasis, the inability to assess repeatability across multiple photographers, and inexperience of both the photographer and reviewers with smartphone-based photography. Furthermore, we only used one photographer in this study in order to eliminate variability in the quality of the images as a confounder in our analysis. The photographer was also a resident at the time of the study who had been well trained in indirect ophthalmoscopy. This meant that he had a short training process in obtaining high-quality photographs. The training process may be more difficult for technicians and medical students who are less experienced with positioning and skill required to take photographs with a 20D lens. However, we suggest that future studies assess the repeatability of image acquisition amongst multiple photographers. In addition, the accuracy for horseshoe retinal tear case 2 and optic nerve avulsion were fairly low, but the grade for quality of these photos was also low. However, the quality of photograph correlated with the confidence of diagnosis in both of these cases and with the accuracy of diagnosis of optic nerve avulsion. Thus, in a clinical setting, reviewers can avoid making clinical decisions based off of unclear photos and uncertain diagnoses that may inevitably lead to medical errors. The lower accuracy and quality of imaging of peripheral retinal tears argues for less utility in peripheral retinal pathology when compared to posterior pole/media pathology at this time. Future studies will need to examine how the sensitivity can be increased to detect various types of posterior segment pathology and reduce the number of false negatives. We also recommend that future studies investigate the reproducibility of quality smartphone-based fundus photographs in inpatient settings, telemedicine consults, and in resource poor areas and their utility in patient care.

Summary

In our study, we found that a smartphone-based fundus camera can accurately diagnosed a variety of acute posterior segment pathologies in the emergency room in up to 78% of the cases. There was no significant difference between resident and attending reviewers, suggesting that this technology can be used in an academic setting.