Development of a store-and-forward telescreening system of diabetic retinopathy: lessons learned from Iran.

BACKGROUND: The present study describes the development and identity phases of a teleophthalmology system used for screening of diabetic retinopathy.
METHODS: A questionnaire was used to identify the main factors necessary for diagnosis of diabetic retinopathy and the features required for a teleophthalmology system. In the second phase, a web-based prototype of the system was designed using the data collected in the previous phase. In the final phase, the system was optimized based on the users' ideas and comments; then, it was evaluated through a standard usability testing questionnaire.
RESULTS: The results showed that the lowest average percentages were related to ethnicity (61%), optometrist's office address (61%), and consultant physician's office address (65%). A web-based prototype was designed using the Visual Studio and Dreamweaver programming tools. This system comprised patient identity data, medical history, clinical data, and retinal images of the patient. The mean score of usability testing and user satisfaction including specialists, residents, and optometrist was 7.3, 7.1 and 7.3 (out of a total 9), respectively. The evaluation results revealed that the system was classified as good.
CONCLUSION: The telescreening system suggested in the current study could be helpful in timely diagnosis. Moreover, it would reduce the treatment costs and complexities. Regardless of the positive points of telemedicine systems, one of the most challenges in this study was the Internet infrastructure to design and apply the system. The future studies, therefore, could focus on the application of cell phone technology for rendering teleophthalmology.

Background

According to the World Health Organization (WHO) estimates, about 40–45 million people were blind in 2004, and more than three times the figure suffered from visual impairment around the world, most of whom lived in low-income countries. With the current process and without effective interventions, the number of blind people is predicted to reach 76 million by 2020 [1, 2]. Cataract, glaucoma, macular degeneration, diabetic retinopathy, trachoma, and corneal opacity are among the most important reasons of blindness [3-6]. Diabetic retinopathy is a vascular complication of diabetes that causes damage to the retina. The disorder is one of the most debilitating chronic diseases and also one of the causes of preventable blindness in the world. In the early stages of the disease, there are usually no visible symptoms; however, the severity of the disorder increases over time and eventually leads to complete loss of vision [7, 8].

Based on the present findings, about 30% of diabetic patients around the world are affected by diabetic retinopathy [9]. Since the disease has no symptoms in the early stages, annual retinal examination has been introduced as a screening standard for diabetic retinopathy to prevent vision loss and blindness in diabetic patients [7, 10]. Due to inadequate knowledge about the necessity of examination, lack of facilities needed for medical visits, lack of access to clinics, lack of eye care professionals, and shortage of equipment, most people suffering from diabetes do not receive the suggested screening for diabetic retinopathy [11-14]. In recent years, information and communication technologies have had a significant growth in different fields, and health environments have experienced great developments in providing health services using these technologies. In this regard, the field of telemedicine has experienced significant changes [15-18]. Telemedicine refers to the use of information technology to provide data related to health care and different medical services, including diagnosis, treatment, and distance learning [19]. Telemedicine is a comprehensive system that includes several applications and is used within and across organizations to provide health services. It is also used by healthcare professionals to offer services such as assessment of remote radiology images and distribution of lab results or consultation with clinical specialists [20, 21]. Some of its subset topic include teleoncology, telediagnosis of heart diseases, telediagnosis of skin diseases, teleradiology, and teleophthalmology [22, 23]. Tele-ophtalmology is a branch of telemedicine that deals with provision of eye care by digital medical equipment and telecommunication technology [24]. In many countries, access to specialized counselling services including ophthalmic consultation services is limited. Even in areas where the number of medical ophthalmologists is sufficient, limitations such as the cost of specialized services and geographic differences between towns and villages may limit access to specialists [25, 26]. The objectives of this study were to identify information requirements of a teleophthalmology system for remote screening of DR and also to develop and evaluate a system.

Methods

In the first phase of this developmental applied study, a questionnaire was used to identify the factors necessary for diabetic retinopathy screening and to identify the features required for a teleophthalmology system. In addition, the information needs of clinical specialists including retina specialists, residents, and optometrist working in hospitals and ophthalmic research centers were extracted for screening of diabetic retinopathy. The questionnaire comprised three sections with a total of 68 questions, including personal information (4 questions), data items needed for the system (52 questions), and features required in the system (12 questions). In front of each question, two options of “necessary” (one point) and “unnecessary” (zero points) were considered. At the end, there was an empty space for participants to write their suggestions and other comments. The validity of the questionnaire was confirmed by six medical informatics, health information management, and eye specialists (faculty members with three years of experience). The reliability of the questionnaire was measured using the Cronbach’s alpha (a = 0.80).

In the second phase, the data obtained via the information needs assessment questionnaire from retina specialists, residents, and optometrists were analyzed using descriptive statistics and frequency distribution. In this phase, each of the required data items was considered essential only if it was considered necessary in the system by an average of at least 50% (median) of the respondents. Besides using the data of the questionnaires, a web-based prototype for the teleophthalmology system was designed and given to users for optimization and receiving their comments. In order to design the prototype, ASP.NET and html in Visual Studio, and Dreamweaver programming tools were used for patient information and data storage, and the users were temporarily provided with the designed system at this stage.

Then, while the designed prototype was being used by specialists, residents, and optometrists, the researcher visited the users in person and received their comments and feedback for optimization. If a user was absent, the researcher contacted him/her through e-mail and received his/her comments to make changes or add necessary features. According to the comments received from the system users, the following changes were done: changing the location and arrangement of data elements, improving the process and methods of entering data elements, and changing the priority of entering data elements and number of required tabs.

The usability of the prototype was evaluated through the think-aloud protocol and the presence of five users in different stages of evaluation process was considered sufficient. Based on the feedback and comments collected from the users in the evaluation stage, the final design of the teleophthalmology system was done using programming languages together with CSS, HTML, and ASPN.NET technologies in Visual Studio and Dreamweaver environment. Finally, the system was provided to users through the Internet following purchasing a host and a permanent domain at www.teleophdr.ir.

In the final phase of the study, the standard Questionnaire for User Interaction Satisfaction (QUIS) was used to evaluate the usability of the final version of the system [27]. The questionnaire has six parts, including identity information of the participant (3 questions), overall reaction to the software (6 questions), screen (4 questions), terminology and system information (6 questions), learning (6 questions), and system capabilities (5 questions). This questionnaire is designed based on a 9-point Likert scale. A response with a score of zero (the lowest) to 9 (the highest) is considered to answer each question. For data analysis, the scores were classified into three groups: 0–3 (weak), 3.1–6 (moderate), and 6.1–9 (good).

The validity of the questionnaire was confirmed by five faculty members of medical informatics and health information management (with at least three years of work experience). Based on the existing studies, the reliability of the questionnaire has been reported 0.94 [28].

Data availability

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Results

For information needs analysis of teleophthalmology system users, a questionnaire was given to 13 retina specialists, 10 residents, and 15 optometrists working in eye hospitals and ophthalmic research centers. Of them, 8 retina specialists (61.54%), 7 residents (70%), and 8 optometrists (53.33%) completed the questionnaire.

The data elements required for the system included five main groups: patient’s identity data, patient’s clinical data, data related to optometrist, data related to consultant physician, and features required for the system. In this regard, after analyzing the questionnaires, the mean percentage of specialists’, residents’, and optometrists’ responses regarding the necessity of the presence of required data elements in the system was as follows:

The majority mean percentage of participants’ response to the data elements of patient identity were age (95%), sex (87%), patient’s contact number (91%) and job (82%). However, other elements such as weight (78%), first name (78%), last name (78%), national code (74%), patient’s residential address (74%), ethnicity (61%) and height (69%) also approved. The data elements related to optometrists were first name (69%), last name (69%), work address (61%), and phone number (78%).

Data elements related to consultant physician including first name, last name, medical council number, work address and phone number also approved with the mean percentage of 95%, 95%, 87%, 65%, and 95% respectively. The mean percentage of participants’ response to the features needed in the system were (100%) for data elements. Data elements of patient’s information also approved by 100%. Additionally, the mean percentage of other data elements is shown in Table 1.

Table 1

Patient’s clinical information

Users	Specialist %	Resident %	Optometrists %	Average %
Data Elements	Necessary	Necessary	Necessary	Necessary
Level of urgency	100	71	87	86
Current disease symptoms	87	100	100	95
Drugs used for diabetes	87	100	100	95
Smoking (packs per day)	87	85	100	91
History of anemia	87	85	75	82
Family history of diseases	75	100	100	91
Past medical history	62	100	100	86
Allergies (including drugs)	62	85	87	78
Other current diseases	75	100	62	78
Refractive error	87	100	100	95
Pictures taken from patient’s eye (direct view) (right-left)	87	100	100	95
Pictures taken from patient’s eye (left view) (right-left)	87	100	100	95
Pictures taken from patient’s eye (right view) (right-left)	87	100	100	95
Pictures taken from patient’s eye (up view) (right-left)	75	100	100	91
Pictures taken from patient’s eye (down view) (right-left)	75	100	100	91
Impossibility of shots	75	85	75	78
Clinical examination by optometrist	87	100	100	95
Prescription of hyperopic glasses	50	100	100	82
Prescription of myopic glasses	37	85	87	69
Optometrists ‘s primary diagnosis	75	71	100	82

According to Table 1, most data elements presented in the five main groups of patient’s identity data, patient’s clinical data, data related to the optometrist, data related to the consultant physician, and features needed in the system were considered necessary by users to a large percentage. In fact, the lowest percentage (61%) was related to two data elements: ethnicity and optometrist’s office address.

At the end, the usability and the level of user satisfaction with the final version of the system were evaluated by users. The evaluation results are presented in Table 2.

Table 2

Evaluation of system by users

Part	Users	Mean	Standard deviation
Overall reaction to the software	Specialist	7.42	.82
	Resident	6.78	.75
	Optometrists	7.08	.79
Screen	Specialist	8.25	.36
	Resident	7.7	.58
	Optometrists	8.22	.97
Terminology and system information	Specialist	7.25	.59
	Resident	7.42	.62
	Optometrists	7.25	.72
Learning	Specialist	7.3	.47
	Resident	6.9	.55
	Optometrists	7.25	.75
System capabilities	Specialist	6.4	.56
	Resident	6.5	.75
	Optometrists	6.9	.66

According to Table 2, the mean and standard deviation of the scores obtained by specialists, residents, and optometrists are presented for different parts of the questionnaire. The mean score of usability and satisfaction of the users with the system given by specialists, residents, and optometrists was 7.3, 7.1, and 7.3 (out of 9), respectively. Since all the three means were between 6.1 and 9 in different parts of system evaluation, it can be concluded that all the three groups participating in the study evaluated the different parts of the system as “good”.

Discussion

One of the important aspects of teleophthalmology is the swift transmission of data and images, through which physicians can be in contact with their patients in remote areas [29]. Due to the visual nature of eye diseases, the application of one of these three methods is common in remote diagnosis: store-and-forward, synchronous or video-conferencing, and hybrid methods [22, 23, 29]. The difference between store-and-forward and video-conferencing method is that contrary to the latter, there is no direct interaction between the patient and the doctor in the former, and virtual examination is not possible. Meanwhile, the doctor’s response has a delay in this method, but it is prompt in video conferencing [22, 23]. In video-conferencing method, a wider band is required and the cost of equipment is also higher. Moreover, the quality of images is lower in this method; therefore, it is not suitable for the diagnosis of eye diseases that require high-quality color images [30].

In a study entitled “OPHDIAT project” conducted by Massin, et al. (2008) for telediabetic retinopathy, 2052 patients were screened. The patient’s identity information included identity number, birth date, and sex, and their clinical data comprised the type of diabetes, duration of diabetes, therapies performed, and retinal images of their eyes. The information was sent to the specialist to diagnose diabetic retinopathy and provide a necessary treatment plan [31].

A similar study entitled “Joslin Vision Network (JVN)” was conducted by Antony, et al. for remote screening of diabetic retinopathy. The patient’s identity information that were sent to the specialist included the patient’s identity number, sex, and age, and their clinical data consisted of duration of diabetes, type of diabetes, and patient’s retina images. Additional images and information were sent to the specialist if the optometrist considered them to be useful in diagnosis [32].

In another study performed by Olayiwola, et al. to improve diabetic retinopathy screening through telemedicine, the patient’s identity information included age, sex, insurance data, and race, and their clinical data comprised the duration of diabetes, treatments done for diabetes including insulin therapy, systolic blood pressure, diastolic blood pressure, and patient’s disease records [33].

Most of the points mentioned in the above studies were also considered in the present research. However, although the software was designed based on the usesr’ needs assessment, some comments and needs, especially those mentioned by the retinal subspecialist, were not considered in the system. It should be mentioned that no information needs analysis was done for specialists, residents, and optometrists in the above studies to determine data items needed for teleophthalmology, and only physical examination standard forms found in hospitals and/or technical centers were used. In the present study, however, needs assessment was done for retina specialists, residents, and optometrists to identify data items.

Some teleophthalmology-related studies conducted for management, screening, and diagnosis of diabetic retinopathy did not evaluate the users’ comments on usability and satisfaction rate through questionnaires, and the users’ comments and feedback were only used to change the software or enhance it [34-36].

Among the studies that examined users’ comments is the Ophdiat conducted by Schulze, et al. in France in 2004. The screening centers that formed this network were connected to an ophthalmology center through a central server. The network was managed by a committee that was responsible for organization, evaluation of the results obtained from the network, expansion, monitoring the effectiveness of screening programs, communications, budget provision, evaluation of cameras, and software upgrades. In this study, different parts of the designed software were upgraded based on the feedback given to the committee by users [37].

In a pilot study in 109 volunteers conducted in China as the first teleophthalmology health system for screening and diagnosis of diabetic retinopathy and also for evaluation and review of the present system, items such as trained people’s qualification, hardware required for taking images, image analysis software, and digital data storage, retrieval, and transmission system were evaluated and finally approved [38].

Lee, et al. who established a web-based system for diabetes management training also examined users’ comments. After implementation of the system, the researchers used a 20-item questionnaire to assess user satisfaction with the user interface (UI). The questionnaire evaluated user satisfaction with the system based on a five-point Likert scale. According to the results, 45% of the users were satisfied with the user interface of the system and had a positive opinion about its capabilities, 40% had no idea about it, and the remaining 15% had a negative view [39]. Moreover, the assessment results of the present study revealed that this system had an easy and user-friendly interface through which the users could easily insert the necessary data, and have access to the existing facilities through their user panel to do the necessary proceedings.

In another study conducted by Jennings, et al. to assess the usability of a web-based system, the authors examined the number of times it was visited in a six-month period, and also studied the system’s acceptability and performance through a 16-item questionnaire. Their suggestion for the system’s enhancement was not used in their study [40].

Similarly, in a study titled “usability testing and improvement of telemedicine websites” in 2010, Alexandru used a 27-item questionnaire - with both open-ended and closed-ended questions - to assess the ability of a web-based telemonitoring system for patients with chronic obstructive pulmonary disease. The questionnaire was answered by the system’s users electronically. In this questionnaire, 10 items were related to the general application of the system, another 10 to the design features and user interface, and 7 to the measurement tools usability and taking reports from the system [41]. Due to the results of the present study, the possibility of sending information to more than one specialist at a time was added to the system.

It can be stated, therefore, that there are different methods and standard questionnaires for the evaluation of the system’s usability that are used according to the type of research and researcher’s views.

Conclusion

Using the suggested system, diabetic retinopathy can be screened in time, and the risk of vision loss and blindness as well as the likelihood of complexities and treatment costs will reduce considerably. Such systems could be useful for physicians and save their time. This shows technological intervention in different aspects of medicine as well as ophthalmology and screening specially. Regardless of the advantages of telemedicine systems, the internet connection and its bandwidth are the most important challenge for users. Therefore, the focus of subsequent studies should be on the use of mobile technology in rendering teleophthalmology services.