Prospective Comparative Study Investigating Agreement between Tele-Ophthalmology and Face-to-face Consultations in Patients Presenting with Chronic Visual Loss.

INTRODUCTION: This study aims to investigate the diagnostic accuracy of store-and-forward tele-ophthalmology consultations for non-diabetic patients, aged 40 and above, presenting with vision impairment of 3 months or more, in terms of cataracts, glaucoma, and age-related macular degeneration.
METHODS: This is a prospective comparative study. Enrolled subjects were independently assessed by both tele-ophthalmology and face-to-face assessment. Agreement level between the two modalities for diagnosis and severity were compared using kappa statistic. Diagnostic accuracy of tele-ophthalmology was determined using the face-to-face consultation serving as the gold standard. Costs were compared by calculating the downstream costs generated by each modality in terms of investigations and treatment.
RESULTS: A total of 860 eyes of 430 patients were assessed during the study period. Tele-ophthalmology consultations had significantly high agreement with face-to-face consultations in the diagnosis and grading of all three ocular conditions; cataracts, glaucoma, and AMD. Diagnosis and grading of cataracts and AMD reached [Formula: see text] values of > 0.8, while diagnosis and grading of glaucoma reached [Formula: see text] values between 0.61 and 0.8. In terms of diagnostic accuracy, tele-ophthalmology consultations were highly sensitive and specific for AMD with greater than 99% sensitivity and specificity achieved by tele-ophthalmology. There was high specificity when diagnosing cataracts, but lower sensitivity at 87.8%. Conversely, there was high sensitivity for diagnosing glaucoma, but lower specificity at 76.5%. Downstream costs were similar between groups.
CONCLUSIONS: Store-and-forward tele-ophthalmology consultations are accurate and comparable to face-to-face consultations for diagnosis and grading of cataracts, glaucoma, and AMD.

Key Summary Points

Introduction

With recent developments in computer processing power, communications technology, and imaging capabilities, telemedicine is increasingly seen as a viable alternative to traditional face-to-face consultations. There is potential for telemedicine to be the great equalizer in medical care. It improves access to specialist health care services for underprivileged patients, providing coverage to areas with limited expertise. Telemedicine also has the potential to reduce patient travel and hospital crowding, advantages that became very important worldwide during the coronavirus 2019 (COVID-19) pandemic.

COVID-19 had a major impact on healthcare systems worldwide, paralyzing specialist clinic services and elective surgical lists while medical professionals and resources were diverted towards combating the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). In healthcare systems where local spread of COVID-19 remained relatively low, such as in Hong Kong, specialist clinical services and elective surgical lists could still be maintained, albeit at a lower volume, as demonstrated by published work from our group [1, 2]. Despite continuation of healthcare services, a significant percentage of patients deferred their specialist consultations and elective surgery, especially in the early stages of the pandemic. The resulting interest in telemedicine services has greatly impacted the field of ophthalmology, with tele-ophthalmology solutions, from triage and acute care to follow-up care, surging worldwide during the pandemic [3-6].

Despite the recent upsurge in interest being related to COVID-19, telemedicine services have clear benefits in ophthalmology beyond the pandemic [7]. As a specialty that relies on image-based diagnostics and investigations, ophthalmology is a perfect fit for store-and-forward telehealth solutions [8]. This is where clinical information is collected and then transferred to another site for subsequent evaluation. The integrity of a store-and-forward tele-ophthalmology consultation is highly dependent on the availability of specialist imaging equipment [9]. For sight-threatening ocular diseases, timely diagnosis and treatment is required in order to prevent permanent blindness. Furthermore, disabled or visually impaired patients have difficulty in reaching clinics independently and may benefit from consultations via telecommunications [10]. However, there are several important considerations for use of tele-ophthalmology in clinical care. First, much of the current evidence for tele-ophthalmology effectiveness has come from single-disease screening programs. Diabetic retinopathy screening is the first and most successful implementation of telemedicine in ophthalmology worldwide. In Hong Kong, the risk assessment and management program (RAMP) designed by the Hospital Authority arranges for diabetic retinopathy screening via retinal photography at Family Medicine and General Outpatient Clinics [11]. Retinal images are then transmitted to the reading center where they are graded by optometrists. Our group previously also demonstrated that glaucoma and age-related macular degeneration screening could be done effectively within the RAMP program [12, 13]. We further documented how even during the COVID-19 pandemic, diabetic retinopathy screening services could be safely maintained in general outpatient clinics without patients needing to visit hospitals, where the risk of infection would be high [14]. Clearly, Hong Kong already has the necessary imaging equipment and assessment capabilities in the primary care setting. However, it remains to be seen whether the success in the screening service can be translated to a tele-ophthalmology consultation. Using the existing available technologies, we reported an initial framework for telemedicine care in ophthalmology for patients who could not attend face-to-face visits due to mandatory testing or quarantine requirements as a basis for [15]. Secondly, the accuracy and effectiveness of tele-ophthalmology platforms across multiple diseases, and especially for consultation purposes, have not been established in the published literature [16]. Existing studies comparing tele-ophthalmology with face-to-face care have generally shown comparable outcomes between the two consultation methods [17, 18]. However, the methodologies and outcomes investigated vary greatly between studies and may not be applicable directly clinical practice. For example, some studies were designed to use deliberately vague outcomes to facilitate agreement between the two techniques, such as ‘keratopathy’, ‘neuropathy’, ‘lens opacity’. A diagnosis and management plan, however, cannot be made with these outcomes. Other limitations of existing tele-ophthalmology systems include the lack of robust clinical trials, small sample sizes, and the absence of quality data, making an assessment of its economical and clinical benefits in existing healthcare systems difficult.

Methods

This was a prospective comparative study investigating (1) the diagnostic and decision-making accuracy of tele-ophthalmology consultations and (2) their agreement with face-to-face consultations in non-diabetic subjects, aged 40 and over, who were referred to the eye clinic of Grantham Hospital, Hong Kong, for blurring of vision for 3 months or more. The study also compared downstream costs generated by each consultation technique in terms of investigations, procedures, and therapy. Subjects were excluded if they were unable to clearly communicate with the physician verbally or if the captured retina and slit-lamp photos were not readable by the study ophthalmologist. This study, including its protocol, information sheets and consent forms, was approved by the University of Hong Kong/ Hospital Authority Hong Kong West Cluster Institutional Review Board (HKU/HKWC IRB Protocol No. 19-101).

Consecutive patients, aged 40 and above, who were referred to the Specialist Ophthalmology Clinic of Grantham Hospital, Hong Kong SAR for blurring of vision for 3 months or more were recruited. The study period was between August 1, 2019 and July 31, 2021. Eligible subjects were then contacted via phone for study enrolment. The study workflow is outline in Fig. 1 and all components, including face-to-face assessment, photo-assessment and tele-ophthalmology assessment, were completed within 1 day. After informed written consent was given for study enrolment, the recruited subjects underwent face-to-face consultation and examination by an ophthalmologist in the eye clinic. A qualified optometrist performed a Snellen chart distant visual acuity test with pinhole. The ophthalmologist took a medical history, performed a slit-lamp biomicroscopy examination with intraocular pressure measurement via applanation tonometry, and a mydriatic retinal examination with an indirect ophthalmoscope, in exactly the same manner he/she practices in the eye clinic. At the end of the consultation and before any ophthalmic investigation was done, the ophthalmologist completed the assessment form (labeled as Form A).

Fig. 1

Study workflow. AMD age-related macula degeneration, PPV positive predictive value, NPV negative predictive value

For the purpose of this study in which ocular images were relied on in the tele-ophthalmology arm, cataract was defined as cloudiness and or yellow discoloration of the crystalline lens; glaucoma was defined as pathological cupping of the optic disc; and AMD was defined by the presence of drusen, geographical atrophy (dry AMD) and or evidence of choroidal neovascularization in the macular area (wet AMD). Other ophthalmic diseases were noted and appropriately managed by the face-to-face consultation ophthalmologist. However, for the purposes of the study, other ophthalmic conditions were not recorded in the assessment forms by the ophthalmologists in each arm. For the tele-ophthalmology arm, we employed a store-and-forward consultation model, where the subject is indirectly assessed using collected clinical information. The optometrist captured color photo(s) of the anterior segment of the eye through a digital compact camera (Canon 1D Mark III, Canon Inc, Japan) attached to a slit-lamp biomicroscope. Anterior segment photographs were captured under low magnification (7.5x) with diffuse illumination of the cornea and the bulbar conjunctiva and parallelepiped section of the corneal stroma and central anterior chamber. Digital photographs of the retina covering two standard fields, focused on the optic disc and macula, respectively, were taken using a mydriatic fundus camera (Topcon TRC-50DX: Type IA, Topcon Corporation, Japan). The imaging equipment used for the study matched that already available at Family Medicine Clinics and General Outpatient Clinics of Hong Kong. The images captured together with the recorded pinhole visual acuity, intraocular pressure (by applanation tonometry), and patient’s referral letter from the family physician were transmitted to a different ophthalmologist in the Specialist Ophthalmology Clinic of the Grantham Hospital through broadband connection to the University of Hong Kong server. The ophthalmologist in the tele-ophthalmology arm sat in a separate room and did not meet the patient. The images, together with the recorded pinhole visual acuity, intraocular pressure (by applanation tonometry), and patient’s referral letter, were viewed by the ophthalmologist using a personal computer connected to a server. After examination, the ophthalmologist in the tele-ophthalmology arm completed the assessment form (labeled as Form B). The tele-ophthalmology arm had no real impact on the patient’s clinical management. The two ophthalmologists who participated in this study were registered ophthalmic specialists in Hong Kong and had similar clinical experience at 3–5 years of post-fellowship work. An assessment of inter and intra-reader reliability, using the tele-ophthalmology workflow on 20 sample eyes, demonstrated high levels of agreement > 90% between the two ophthalmologists and between repeat assessments for cataracts, glaucoma, and AMD (see Supplementary Table 1). They were blinded to the diagnosis and plan of management offered to the patient by the other ophthalmologist.

Comparison of cost was conducted using the investigations, procedures, and treatments each consultation modality generated per patient. The severity classification was used to estimate the plan of management. A timeframe of the next 6 months was chosen for calculation of treatment and investigation costs as the maximum next follow-up date. For cataracts, phacoemulsification and intraocular lens implantation were offered to those with moderate-to-severe disease. For possible glaucoma, those who were classified as glaucoma suspects were offered perimetry and optical coherence tomography nerve fiber layer analysis (OCT NFL) for workup. For those with definite glaucoma, they were offered workup and treatment, with first-line therapy in the form of latanoprost (a prostaglandin analogue) applied daily to the diseased eye. For AMD, those classified as wet AMD were offered angiography and OCT macula workup and three monthly injections of ranibizumab. The price in Hong Kong dollars (HKD) of each investigation and treatment is derived from the Hospital Authority Ordinance (Chapter 113) List of Private Charges.

For the study, the sample size calculation was based on the number of subject needed to estimate the level of agreement, with a reasonable degree of precision. We tested different levels of prevalence of diseases from 5 to 30% to reflect the variation of the prevalence of each eye disease in the studied group. Cohen’s statistic was used in order to assess the level of agreement for the two groups. With an estimated disease prevalence of 10%, a value of 0.5, the minimum sample size needed is 355 subjects.

Data Analysis

Statistical analysis was performed using SPSS statistics 27.0 (IBM SPSS Inc., Chicago, IL, USA). Cohen's statistic was run to assess the level of agreement between face-to-face consultation and tele-ophthalmology in the diagnosis and grading of cataract, AMD, and glaucoma. The interpretation of kappa values were as follows: < 0.20, poor agreement; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1 as almost perfect agreement [19]. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated to investigate the diagnosis accuracy of telemedicine in comparison with face-to-face consultation as the gold standard. A two-tailed p value less than 0.05 was considered as statistically significant. Both eyes of each subject were enrolled in the study as the focus was on evaluating accuracy of tele-ophthalmology consultations compared to face-to-face consultation rather than on the prevalence and distribution of ophthalmic diseases themselves. Each eye was independently assessed for the study for diagnosis, severity, and management decision-making.

Results

During the study period, 446 consecutive subjects who met the inclusion criteria were considered for enrolment (see Fig. 2). Of the 446 subjects, one declined to participate in the study on the day of the examination. Of the remaining 445 subjects, a further 15 subjects had un-assessable retina photographs due to poor visibility. These subjects were therefore excluded from analysis. Finally, 860 eyes from 430 subjects were assessed for the comparison study. The mean age of subjects at enrolment was 67 years (± 11). 57.6% of recruited subjects were female, compared to 42.3% being male. Mean pinhole LogMAR visual acuity was 0.35 (± 0.12) for right eyes and 0.36 (± 0.11) for left eyes. (Average IOP was 16.3 mmHg (± 3.0) for right eyes and 16.5 mmHg (± 3.1) for left eyes. The two ophthalmologists took turns serving as the face-to-face and tele-ophthalmology physician 44.9 and 55.1% of the time, respectively.

Fig. 2

Patient enrolment flowchart

Disease Prevalence and Severity Distribution

Overall disease prevalence, based on each assessment technique, was determined for each of the three pathologies cataracts, glaucoma and AMD (see Table 1). Using face-to-face consultation as the gold standard for diagnosis, the prevalence of each pathology was as follows: 75.3% of eyes had cataracts, 31.6% of eyes were classified as possibly with glaucoma, and 12.4% of eyes had AMD. In terms of distribution of severity, for those with cataracts, 64.4% were graded as early, 31.7% as moderate, and 3.6% as severe. For those who had possible glaucoma, 84.6% were graded as glaucoma suspects and 17.6% were graded as definite glaucoma on presentation. For eyes with AMD, 53.3% had drusen only, 22.4% were classified as having dry AMD, 21.7% of eyes were classified as having dry AMD with geographical atrophy, and only 2.80% had wet AMD. Overall, disease prevalence was high in our study cohort, reflecting the high likelihood of the presence of sight-threatening pathology in elderly patients presenting with visual impairment for 3 months or more. For the same cohort of subjects, when evaluated by tele-ophthalmology, the prevalence of cataracts increased to 80.2%, but this was not statistically significant. The prevalence of possible glaucoma however decreased significantly to 25.1% in the tele-ophthalmology arm. Post hoc comparisons revealed that face-to-face consultation was associated with a higher prevalence of possible glaucoma as well as definite glaucoma (see Table 2). The prevalence of AMD remained largely unchanged in the tele-ophthalmology arm.

Table 1

Disease prevalence between consultation techniques

	Prevalence from face-to-face consultation	Prevalence from tele-ophthalmology consultation	χ2 statistic	p value
Cataracts	671/860 (75.3%)	690 (80.2%)	1.8	0.400
Early	432/671 (64.4%)	444/690 (64.4%)
Moderate	213/671 (31.7%)	224/690 (32.5%)
Severe	24/671 (3.6%)	22/690 (3.19%)
Possible glaucoma	272/860 (31.6%)	216 (25.12%)	12.6	0.002
Glaucoma suspect	230/272 (84.6%)	193/216 (90.2%)
Definite glaucoma	48/272 (17.6%)	23/216 (10.7%)
Age-related macular degeneration	107/860 (12.4%)	106 (12.3%)	0.1	1.000
Drusen only	57/107 (53.3%)	62/106 (58.5%)
Dry AMD	24/107 (22.4%)	15/106 (14.2%)
Dry AMD with GA	23/107 (21.5%)	25/106 (23.6%)
Wet AMD	3/107 (2.80%)	4/106 (3.70%)

VA visual acuity, IOP intraocular pressure, AMD age-related macular degeneration, GA geographical atrophy

Table 2

Post hoc comparisons of the prevalence of glaucoma between techniques

Methods	Statistical values	Possible glaucoma	Normal
FTF consultation	Number (adjusted residual)	272 (3.5)	618 (− 3.5)
	Bonferroni-adjusted p value	0.0004	0.0004
Telemedicine consultation	Number(adjusted residual)	673 (1.7)	217 (− 1.7)
	Bonferroni-adjusted p value	0.09	0.09

Bonferroni-adjusted p value for 3*2 Chi-square test: With alpha set at 0.05, and 6 comparisons, the p value required for significance would be 0.05/6 = 0.008

FTF face to face, IOP intraocular pressure

Agreement Level Between Telemedicine and Face-to-Face Consultation

Agreement levels between tele-ophthalmology and face-to-face consultations are summarized in Table 3, and segregated according to disease and severity. Overall, agreement levels for both modalities were high across all diseases and all severities. Tele-ophthalmology had “almost perfect” agreement with face-to-face consultation in establishing the presence of cataract or AMD, and “substantial” agreement in establishing the diagnosis of glaucoma. However, compared with establishing a diagnosis, telemedicine and face-to-face consultation had lower kappa values for severity grading for all three diseases, although the reduction in agreement level was not statistically significant.

Table 3

Agreement between tele-ophthalmology and face-to-face consultation in the diagnosis and grading of cataract, AMD, and glaucoma

	AgreementN (%)	Kappa value (95% CI)	Agreement level	p value
Cataract present
FTF vs. telemedicine	833/860 (96.9%)	0.91 (0.87–0.94)	Almost perfect	0.000
AMD present
FTF vs. telemedicine	853/860 (99.2%)	0.96 (0.93–0.99)	Almost perfect	0.000
Glaucoma present
FTF vs. telemedicine	790/860 (91.9%)	0.80 (0.75–0.84)	Substantial	0.000
Cataract severity
FTF vs. telemedicine	800/860 (93.2%)	0.89 (0.86–0.92)	Almost perfect	0.000
AMD severity
FTF vs. telemedicine	837/860 (97.3%)	0.88 (0.84–0.93)	Almost perfect	0.000
Glaucoma severity
FTF vs. telemedicine	765/860 (88.9%)	0.74 (0.69–0.78)	Substantial	0.000

VA visual acuity, IOP intraocular pressure

Diagnostic Accuracy of Tele-Ophthalmology in Eyes with Cataracts, Glaucoma, and AMD

Taking face-to-face consultation as the gold standard for comparison, the sensitivity, specificity, PPV, and NPV of tele-ophthalmology consultations in diagnosing cataracts, glaucoma and AMD were calculated and are summarized in Table 4. The accuracy for diagnosing AMD was highest with greater than 99% sensitivity and specificity achieved by tele-ophthalmology. Tele-ophthalmology achieved a high specificity when diagnosing cataracts, but had a lower sensitivity at 87.8%. Conversely, tele-ophthalmology had high sensitivity for diagnosing glaucoma, but had a lower specificity at 76.5%. Positive and negative predictive values were above 90% across the board. Taking the overall results into consideration, tele-ophthalmology was an accurate diagnostic tool.

Table 4

Diagnosis accuracy of telemedicine in the diagnosis of cataract, AMD, and glaucoma

	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Cataract	87.8	99.4	97.6	96.7
AMD	99.7	99.5	99.3	98.1
Glaucoma	98.7	76.5	90.5	96.3

AMD age-related macular degeneration, PPV positive predictive value, NPV negative predictive value, VA visual acuity, IOP intraocular pressure

Generated Downstream Costs of Each Consultation Modality

From the prevalence data in Table 1, tele-ophthalmology consultations diagnosed a higher number of cataracts, especially at moderate severity. This in turn generated a higher number of cataract procedures. Conversely, tele-ophthalmology consultations diagnosed a significantly lower number of glaucoma suspects and definite glaucoma. This resulted in fewer subjects undergoing investigations and treatments. For AMD, the number of wet AMD cases were similar between groups and thus generated workload was not significantly different. Overall, the estimated cost, per patient, per 6 months, was higher for tele-ophthalmology consultations and compared to face-to-face consultations, with a difference of HKD 99.86/patient/6 months (or USD 12.82/patient/6 months) between face-to-face consultation and tele-ophthalmology (see Table 5).

Table 5

Estimated 6-month downstream costs resulting from each consultation modality

	Generated investigations and treatment in next 6 months	Downstream costs of face-to-face consultation	Downstream costs of tele-ophthalmology consultation (with VA, IOP)
Cataracts
Early	None	–	–
Moderate	Phacoemulsification and IOL implantation (HKD 15,100)	HK$3,216,300.00	HK$3,382,400.00
Severe	Phacoemulsification and IOL implantation (HKD 15,100)	HK$362,400.00	HK$332,200.00
Possible glaucoma
Glaucoma Suspect	HVF 24-2 (HKD 600) and OCT NFL Assessment (HKD 600)	HK$138,000.00	HK$115,800.00
Definite glaucoma	HVF 24-2 (HKD 600) and OCT NFL Assessment (HKD 600) and 6 months of topical Latanoprost (HKD 180 per bottle)	HK$109,440.00	HK$52,440.00
Age-related macular degeneration
Drusen only	None	–	–
Dry AMD	None	–	–
Dry AMD with GA	None	–	–
Wet AMD	FFA and ICG, 2 × OCT macula (HKD 600) and 3 × intravitreal ranibizumab injection and drug fee (HKD 2190 + HKD 6350)	HK$87,540.00	HK$116,720.00
Total added costs for 860 patients/6 months		HK$3,913,680.00	HK$3,999,560.00
Average added cost per patient/6 months		HK$4550.79	HK$4650.65

VA visual acuity, IOP intraocular pressure, AMD age-related macular degeneration, GA geographical atrophy, IOL intraocular lens implant, HVF 24-2 Humphrey Visual Field SITA Standard 24–2, FFA fundus fluorescein angiography, ICG indocyanine green angiography, OCT optical coherence tomography, NFL nerve fiber layer analysis

Discussion

In our study, we compared a store-and-forward tele-ophthalmology consultation with a traditional face-to-face consultation for subjects, aged 40 and above, referred for chronic visual loss. Our study only covered the first consultation and did not include ophthalmic investigations. The study only used imaging tools that were already available in public Family Medicine Clinics in the tele-ophthalmology consultation arm. The strict inclusion criteria helped to generate a high prevalence of cataracts, glaucoma, and AMD, allowing for a robust comparison between the two study arms.

We demonstrated that tele-ophthalmology consultations had significantly high agreement levels with face-to-face consultations in both diagnosis and severity classification for cataracts, glaucoma, and AMD. For the diagnosis and severity grading of cataracts and AMD, this reached ‘near-perfect’ levels of agreement, while glaucoma diagnosis and severity grading were at ‘substantial’ levels of agreement. These findings show that in the hands of an experienced ophthalmologist, a diagnosis and classification can be accurately made be through the use of slit-lamp and fundus images alone. To explain the lower agreement levels for glaucoma and cataracts compared to AMD, it is likely that current imaging techniques do not allow for differentiation of subtle differences in disease stages and for identification of early disease forms.

In terms of diagnostic accuracy, tele-ophthalmology was highly sensitive and specific for AMD at greater than 99% for all parameters. This is expected as the diagnosis is completely based on retina findings, and thus can be determined from a clearly taken fundus photograph. In the diagnosis of cataracts, tele-ophthalmology was less sensitive at 87.3% but was highly specific. A lower sensitivity level here means that there are a number of false-negatives generated. This is likely due to the relatively inconspicuous appearance of early cataracts, in that the ‘yellowing’ of the lens can be very subtle and may not be adequately captured on a slit-lamp photo. Conversely, in the diagnosis of glaucoma, tele-ophthalmology was highly sensitive but less specific at 76.5%, meaning that the consultation was generating a number of false-positives. This may be due to the fact that our study used a two-dimensional retinal photograph for the assessment of optic disc cupping. Without stereopsis, the cupping is less adequate evaluated. A number of tele-ophthalmology studies have used stereoscopic images of the optic nerve head for accurate remote diagnosis and monitoring of glaucoma [20]. While this would definitively improve the accuracy of glaucoma diagnosis, this type of fundus camera would not be a commercially viable solution for the primary care setting at the moment. Furthermore, there was inter-reader variability when determining cup-to-disc ratio and making a glaucoma diagnosis. The introduction of more objective parameters, including OCT NFL analysis would likely help in reducing this variability.

In terms of generated downstream costs, overall the tele-ophthalmology consultations were more costly per patient than the face-to-face ones. This was largely due to the tele-ophthalmology consultations over-diagnosing cataracts and classifying more as at least moderate in severity. The differences were small, however, at USD 12.82/patient/6 months. The difference in costs demonstrates that diagnostic accuracy is the most important determinant on the cost-effectiveness of tele-ophthalmology in the long run. Although not shown by our study, it is important to consider the potential damaging impact, in terms of medico-legal costs and loss of quality of life to the patient, that unnecessary cataract surgeries and missed glaucoma may generate. There are important possible reasons accounting for the differences in downstream costs between the tele-ophthalmology and face-to-face consultations in our study. Firstly, potential downstream costs were calculated based on disease severity alone without taking into consideration other patient and ocular factors. This was to ensure that both arms used the same threshold for booking procedures or investigations. Our assumption likely significantly impacted the downstream costs for cataracts. In reality, a decision for cataract surgery would take into consideration the patient’s occupational needs, activities of daily living, and other visual symptoms including glare. In the public healthcare system of Hong Kong, adequate justification would be needed for cataract listing where visual acuity is 20/40 or better. As the tele-ophthalmology arm did not allow direct communication between the ophthalmologist and the subject, the information could not be ascertained. Secondly, a decision to be listed for cataract surgery requires patient consent after a thorough discussion of risks and benefits. Patients with moderate-to-severe cataracts may not always opt for cataract surgery depending on their own circumstances.

There were a number of limitations in our study that must be considered to fully understand the context of our results. First, the study was limited by its specific nature in addressing the clinical scenario of non-diabetic subjects, aged 40 and above, presenting with chronic blurring of vision. This was to allow the focus to be on cataracts, glaucoma, and age-related macular degeneration. These are the three leading causes of blindness worldwide, but do not form a comprehensive list of possible sight-threatening conditions. Thus, we do not know how this tele-ophthalmology platform performs in assessing other common causes of chronic visual loss, such as epiretinal membranes and diabetic macula edema. Secondly, the store-and-forward methodology for teleconsultations did not allow the ophthalmologist to converse with the study subject or to examine the subject directly. This would otherwise be feasible in live tele-ophthalmic practices. Thus our ophthalmologist had to rely on high-quality photographs and VA, IOP, and the referral letter for relevant context. Thirdly, the lack of OCT assessment in the study meant that investigations were still required for disease confirmation. This may limit the application of our results in specialty ophthalmic practices. However, our study’s use of only simple equipment and examination techniques allows our model to be adopted for the primary care setting. Lastly, the study had a high prevalence of cataracts, possible glaucoma and AMD, far higher than was originally expected from the community, where cataracts, glaucoma, and AMD accounted for 37, 11, and 26% of all causes of blurring of vision our group’s published Hong Kong Eye Study [21]. A logical reason for this difference was that the teleophthalmology study was conducted during the COVID-19 pandemic. There was a significant drop in specialist outpatient attendance, owing to fears of being infected with SARS-CoV-2, especially in 2020. This is especially true for our target age group, which has low vaccination rates to this day and are most susceptible to COVID-19-related morbidity and mortality. It is likely that those who still wished to be seen in the public healthcare system during this study period had poorer vision and more serious sight-impacting ocular diseases, in particular cataracts. This is evidenced by the low average presenting visual acuity of our study subjects. Furthermore, in our study, we could only classify patients as having possible glaucoma after the initial consultation, as confirmation requires evidence of functional visual field loss or structural damages to the optic nerve and may not be immediately apparent on the first visit unless severe.

To date, our study is the largest prospective comparative study investigating levels of agreement between tele-ophthalmology and face-to-face consultations across multiple eye diseases. A number of published studies have investigated the diagnostic accuracy of store-and-forward tele-ophthalmology consultation programs across multiple eye diseases. However, unlike our study, these publications only examined the presence or absence of eye pathology, but not its severity. The largest of these studies was published in 2017 by Maa et al. from Georgia, USA [22]. This was a retrospective quality assurance study. Here, 2690 subjects in rural areas underwent eye screening using a store-and-forward telehealth solution. Patients with eye pathology, including cataracts, glaucoma or glaucoma suspect, diabetic retinopathy, and AMD were referred to the Atlanta Veterinary Affairs Eye Clinic for face-to-face ophthalmological assessment. Agreement level was assessed between the two assessment modalities for the 613 referred cases. Like our study, agreement level between tele-ophthalmology and face-to-face consultations was high for AMD, glaucoma/glaucoma suspects and cataracts. Maa et al. subsequently conducted a prospective comparative study to examine the diagnostic accuracy of the described telehealth solution for 256 subjects without known eye diseases [23]. Same-day face-to-face assessment was performed for comparison. The study found that while there was substantial agreement between the two modalities for cataracts and diabetic retinopathy, agreement levels were lower for glaucoma suspect and AMD. Interestingly, further analysis by Maa et al. found that the addition of OCT did not improve diagnostic accuracy of the tele-ophthalmology service [24]. It is important, however, to note that the study may not have been adequately powered to detect this effect. Another study, by Chow et al. from Massachusetts, USA, compared the diagnostic accuracy of non-mydriatic digital retinal imaging versus face-to-face dilated ophthalmic examination for non-diabetic eye diseases in a cohort of 200 diabetic subjects [25]. In this study, face-to-face consultation was performed an average of 39.6 days after digital imaging assessment. Remote assessment by digital imaging was similarly found to be accurate with high levels of agreement to face-to-face assessment across multiple eye diseases, chiefly cataract, glaucoma suspect, and choroidal lesions. A similar study was reported by Conlin et al., in 2015, where the accuracy of a store-and-forward tele-ophthalmology consultation used in diabetic retinopathy screening purposes was compared to a same-day face-to-face comprehensive eye examination across multiple eye diseases. A total of 317 diabetic patients were prospectively recruited for the comparative study. Agreement levels were substantial between the two modalities for cataracts, glaucoma, and diabetic retinopathy, and moderate for AMD. In comparison, agreement levels between the two modalities in our study were substantial to near perfect for cataracts, glaucoma, and AMD for both presence and severity. This may reflect the very strict criteria that we used for grading each pathology and also the strict criteria for selection of grading ophthalmologists. In our study, the two ophthalmologists were both qualified specialists in Hong Kong of similar post-fellowship experience. Additionally, both of our study ophthalmologists happened to receive training from the same eye center under the same supervisors. Thus we may need to bear in mind that lower levels of agreement may be expected in a real-world setting where greater variations in training would be expected between ophthalmologists.

In summary, tele-ophthalmology is a highly accurate and viable consultation method. However, there are still diagnostic and grading limitations, particularly for cataracts and glaucoma, that prevent it fully from being an alternative to face-to-face consultation with current technology. If the alternative to the proposed tele-ophthalmology model is no consultation, especially during periods of pandemic-related lockdown, or in remote places of the world, then it would be an acceptable form of service provision. However, in the context of a small, well-connected city with excellent healthcare infrastructure, like the Hong Kong SAR, tele-ophthalmology with current technology is still inferior to face-to-face consultations. The accuracy of tele-ophthalmology consultations can be greatly improved through further technological advances. If high-resolution slit-lamp and fundus cameras with stereopsis functions are commercially viable and implemented, we will likely see improvements in diagnosis and grading of cataracts and glaucoma. Another viable method is to introduce machine learning as an aid to diagnosis and management. As it stands though, tele-ophthalmology may serve as a better triage tool than an alternative, with limitations, to face-to-face consultations in Hong Kong.

Conclusions

Our study demonstrated high agreement levels in diagnosis between tele-ophthalmology and face-to-face consultations. While tele-ophthalmology is an accurate consultation method, there are still discrepancies between the two modalities in determining disease severity for cataracts and glaucoma, which in turn would guide management decisions.

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 259 KB)

Despite the recent upsurge in interest due to COVID-19, telemedicine services have clear benefits in ophthalmology beyond the pandemic.

It is thus important to determine the diagnostic accuracy of store-and-forward tele-ophthalmology compared to traditional face-to-face consultations in a prospective study.

In our study, tele-ophthalmology consultations had high agreement with face-to-face consultations in the diagnosis and grading of cataracts, glaucoma, and age-related macular degeneration (AMD).

In terms of diagnostic accuracy, tele-ophthalmology consultations were highly sensitive and specific for AMD, highly specific but less sensitive for cataracts and highly sensitive but less specific for glaucoma.

However, tele-ophthalmology generated higher downstream costs than face-to-face consultations.