Evaluation of the effectiveness of diagnostic & management decision by teleophthalmology using indigenous equipment in comparison with in-clinic assessment of patients.

BACKGROUND &
OBJECTIVES: There is a concern on the quality and the usefulness of teleophthalmology images, particularly those using indigenous equipment, in making a diagnosis and treatment decisions in ophthalmology. The present study was done to compare the level of agreement and sensitivity and specificity of diagnosis and management decisions of various eye diseases by teleophthalmology using indigenous equipment, compared to the in-clinic assessment.
METHODS: Patients having different eye diseases were evaluated by two ophthalmologists - one ophthalmologist examined the patient in clinic setting while the other ophthalmologist made the diagnosis and management decision based on images sent by teleophthalmology. The images were taken by the ophthalmic technician using digital imaging system and fundus camera. The clinical findings and management decisions by the two ophthalmologists were masked to each others.
RESULTS: In diagnosis of anterior segment eye diseases such as cataract and corneal diseases there was good to very good agreement (kappa values of 0.68 and 0.91 for cataract and corneal diseases respectively) between in-clinic assessment and assessment by teleophthalmology. There was moderate agreement (kappa values of 0.52 and 0.48 for glaucoma and retinal diseases respectively) between in-clinic assessment and assessment by teleophthalmology for the diagnosis of glaucoma and retinal diseases. For the management decisions of patients, there was moderate level of agreement in all groups of eye diseases. INTERPRETATION &
CONCLUSIONS: Teleophthalmology, using indigenous equipment was found to be effective in diagnosis and management decision of anterior segment eye diseases such as cataract and cornea, and with some modification and continuous training to the technicians could become an effective tool for screening and referral of glaucoma and retinal diseases.

In India approximately 70 per cent of the population resides in rural areas1 and at the same time there is a wide variation in the accessibility to ophthalmic care and availability of ophthalmic experts. Most of the ophthalmologists in India are concentrated in urban areas and there is non-availability of ophthalmologists in the rural areas23. India has witnessed rapid development in the field of telecommunication making tele-health feasible in most of the places. Teleophthalmology offers great potential for accessibility and affordability of quality care in remote and rural areas in India.

There are concerns on the quality and the usefulness of teleophthalmology images in making a diagnosis and treatment decisions in ophthalmology. Some of the studies conducted in western settings found good agreement between the diagnosis and management decisions by teleophthalmology compared to the diagnosis and management decisions in eye clinics4567. There was overall good sensitivity and specificity for the teleophthalmology compared to in-clinic assessment. Bahaadinbeigy and Yogesan8 in their summary of published paper on teleophthalmology projects concluded that most of the teleophthalmology projects to date have been focused on the treatment of diabetic retinopathy. At the same time most of the studies have been done using equipment not affordable for most of the primary care setting in India. With the availability of low cost indigenous equipment for teleophthalmology, studies are needed to assess their effectiveness particularly in primary settings such as vision centres. Also, the effectiveness of teleophthalmology for other common blinding diseases in India such as cataract, glaucoma and corneal diseases requires to be evaluated.

The present study was done to compare the level of agreement and sensitivity and specificity of diagnosis and management decisions of various eye diseases by teleophthalmology using indigenous equipment, compared to the in-clinic assessment.

Material & Methods

The study was carried out in various superspeciality clinics of Venu Eye Institute & Research Centre, New Delhi, India, between June to December 2010. The study was approved by the Ethics committee of the Institute. Written informed consent from the all study subjects was taken prior to including them in the study. Participants for this study were recruited randomly from the patients visiting outpatient department of Venu Eye Institute & Research Centre. Two ophthalmologists in each speciality (cataract, cornea, glaucoma and retina) were involved in the study.

In the study, comparisons were made between the diagnosis and management decisions by teleophthalmology with in-clinic assessment of the same patients. Each patient was evaluated by both ophthalmologists - one ophthalmologist examined the patient in clinic setting while the other ophthalmologist made the diagnosis and management decision based on images captured by indigenous equipment. The images were taken by the ophthalmic technicians. The clinical findings and management decisions by the two ophthalmologists were masked to each other. Also, to avoid any measurement bias both ophthalmologists were masked to the hospital records of the patients.

The sample size calculation was based on an estimated 20 per cent disagreement in diagnosis by teleophthalmology and in-clinic diagnosis with 95% confidence interval of the width 10 per cent (i.e. 10 to 30%). Sixty two patients in each superspeciality were needed to be examined both by one consultant through teleophthalmology and another consultant directly in the clinic. Before the start of the study, the interobserver agreement between the two consultants of each superspecility on images of 50 eyes of 25 patients randomly selected from these clinics were assessed.

During the study, the patients were selected randomly in the superspeciality clinics by the resident doctors. Every 5th patient visiting the concerned superspecility clinic was included in the study. In situation of patient not willing to participate in the study, next patient visiting the clinic was counseled to participate in the study. After documenting the patient's complaints and relevant history in the specially designed data recording form by the resident, the anterior segment photography for the patients with cataract and corneal diseases was done by a trained technician using Appasamy Digital Imaging System (Model AIA- 11 5 S, Appasamy associates, Puducherry, India). The cataract patients were assessed using observation with an optical section or direct focal illumination after dilating the pupil. The posterior segment photography (approximately 20 degree posterior pole images for retinal diseases and optic disc images for glaucoma patients) in patients with retinal diseases and glaucoma was done by indigenously developed fundus camera (Portacam II, Madhu Instruments, New Delhi, India) after dilating the pupils with tropicamide (1%) and phenylephrine (5%) eye drops. In patients with history of hypertension and cardiac diseases only tropicamide eye drops were used. These images were transferred and stored in a marked computer in the teleophthalmology unit of Venu Eye Institute & Research Centre for analysis by the 1st consultant in the concerned superspeciality. The consultant recorded the diagnosis and management decisions in the data recording form based on the assessment of the images stored in the computer. The management decisions included observation, refraction, medical management and referral for further investigations or surgery. The patients were examined in the respective superspeciality clinics by the 2nd consultant and the findings were noted in data recording form. The diagnosis of glaucoma was made on the basis of disc findings i.e. a vertical cup disc ratio of ≥0.7 or focal neuroretinal rim defect. Simple outcome measures such as presence or absence of diseases and severity of diseases were recorded. Both consultants were masked for the patient case sheet to avoid any bias in the diagnosis and management decisions.

Data were entered and cleaned by the researcher using MS Excel database. The confidentially of the study subjects was maintained. The data were exported and analysis was done using STATA v.10 (Statacorp, College Station Texas, USA). Kappa statistics were used to assess agreement on nominal variables such as diagnosis and management decisions of different diseases.

The kappa values were interpreted as9, <0.20 as poor strength of agreement, 0.21-0.40 as fair, 0.41-0.60 as moderate, 0.61-0.80 as good and 0.81-1.0 as very good.

Results

The interobserver agreements between the two ophthalmologists of each speciality were measured before the start of the study to avoid any measurement bias arising due to the difference in level of skill and experience. The results showed good agreement between ophthalmologists with Kappa values ranging between 0.58 to 0.87 for the diagnosis of eye diseases and 0.59 to 0.68 for the management decisions. (Table I).

Table I

Kappa values - Interobserver variations between ophthalmologists, specialty-wise

A total of 247 patients were seen in various clinics between June to December 2010 (approximately 60 study subjects from each speciality) were included in the study. The demographic characteristic of the study subjects are summarized in Table II. The mean age of retina and cornea patients in this study were lower compared to those of cataract and glaucoma patients explained by the fact that cataract and glaucoma are mostly age related conditions. Other common retinal diseases such as diabetic and hypertensive retinopathies, macular degenerations, choroiditis, vaso-occlusive disorders, etc., and common corneal diseases such as corneal ulcer, corneal opacities, pterygium, and corneal foreign body were included in the study.

Table II

Demographic characteristic of the study subjects

The levels of agreements of the diagnosis and management decisions of different eye diseases between in-clinic assessment and those made by teleophthalmology are summarized in Table III. In diagnosis of anterior segment eye diseases such as cataract and corneal diseases good to very good agreement was seen between in-clinic and teleophthalmology assessment. There was moderate agreement between in-clinic assessment and teleophthalmology for the diagnosis of glaucoma and retinal diseases. For the management decisions of patients, there were moderate levels of agreement in all groups of eye diseases.

Table III

Agreements on the diagnosis and management decisions of different eye diseases

The diagnostic accuracy of the teleophthalmology in detecting eye diseases was high in sensitivity for cataract and corneal diseases, moderate in retinal diseases and lower for glaucoma. The specificity of teleophthalmology diagnoses was lower for cataract and retinal diseases, moderate for glaucoma and high for corneal diseases. For the management decisions, the sensitivity of teleophthalmology was high for cataract but moderate to low for glaucoma, retinal and corneal diseases. The specificity of management decisions by teleophthalmology was moderate for corneal and retinal diseases but low for cataract and glaucoma (Table IV).

Table IV

Sensitivity and specificity of teleophthalmology diagnosis and management decisions of different eye diseases

Discussion

Simple criteria such as presence or absence of diseases, simple grading of diseases and referral decisions were chosen keeping in mind the potential applicability of the teleophthalmology services in primary eye care settings. These criteria are enough for taking a decision whether or not a patient needs to be referred to higher eye care centres (secondary/tertiary/centre of excellences) for any further management.

High level of agreement between teleophthalmology and in-clinic assessment and high sensitivity of teleophthalmology in the diagnosis of anterior segment diseases was found in this study. Threlkeld et al10 in their study on telemedical evaluation of ocular adnexa and anterior segment found highest sensitivity and specificity for clinical findings with high contrast cues for colour and depth. These findings make teleophthalmology a potential tool for the screening of anterior segment diseases in the primary eye care settings where the availability of ophthalmologists and trained paramedics are limited. Since anterior segment diseases are major causes of blindness in India11, this technology can help to control avoidable blindness.

There was moderate level of agreement in the diagnosis of posterior segment diseases such as glaucoma and retinal diseases. The sensitivity of teleophthalmology in the diagnosis of glaucoma and retinal diseases was also moderate. This could be partly explained by suboptimum focusing of some of the images in the early part of the study. The image quality can be improved by using real time teleophthalmology. Continuous effort can also be made to further improve the quality of optical system of indigenously made teleophthalmology equipment. These factors will also increase the agreement and sensitivity of management decisions by teleophthalmology compared to in-clinic assessment. Based on personal experiences, feedback and personal communications from other ophthalmologists it has been seen that the ophthalmologists interpreting the tele-images usually have the tendency to make a positive diagnosis (in borderline cases of retinal diseases and early cataractous changes). This could be a possible reason for low specificity of tele-ophthalmology diagnosis for cataract and retinal diseases.

Management decisions depend a lot on the direct interaction of the patient with the ophthalmologist. Some of the direct questions such as severity of loss of vision and associated other visual symptoms (metamorphopsia, dark spots, etc.) may play important role in making management decisions such as need for surgery in cataract patients as well as further requirement of investigation and appropriate management of retina patients. In the absence of direct communication, the ophthalmologists evaluating the photographs prefer to advise for referral and further investigations in borderline cases. This could the possible reason for low specificity for management decisions. Moderate to low specificity in the diagnosis and management decisions means some of the patients not having eye disease can be referred for further management. This may cause some extra burden to higher centres and inconvenience to the patients. But, considering the blinding nature of these diseases it is better to make error on side of over-diagnosis rather than under diagnosis. Real time teleophthalmology with video conferencing facilities may play a significant role in minimizing the referrals.

The study had certain limitations. It was done in a tertiary hospital setting where the patients usually visit with more advanced stages of disease compared to primary eye care settings. The agreement level between the two modalities, and sensitivity and specificity of diagnosis and management decisions by teleophthalmology compared to in-clinic assessment could be different for primary care setting patients. The study evaluated the store and forward type of teleophthalmology, therefore, the current study is applicable only for non-real time teleophthalmology practices. The agreement level, sensitivity and specificity of real time teleophthalmology could be better as the ophthalmologist evaluating the images can guide the slit-lamp operator in the remote site to make fine adjustments and to obtain appropriate angles. The quality of diagnostic equipment available in various superspeciality clinics can also affect the results. Information on the quality of images, such as proportion of gradable images could have been gathered in this study.

In conclusion, teleophthalmology, using indigenous equipment was effective in diagnosis and management decision of anterior segment eye diseases such as cataract and corneal diseases. With some modification and continuous training to the technicians it could be an effective tool for screening and referral of glaucoma and retinal diseases. Teleophthalmology using indigenous equipment has a potential to deliver quality eye care, particularly primary eye care, to people living in remote and rural areas where access to eye care is limited. Further studies are needed to ascertain the cost effectiveness of teleophthalmology services in the rural and remote areas.