The clinical utility of eye exam simulator in enhancing the competency of family physician residents in screening for diabetic retinopathy.

OBJECTIVES: To evaluate the utility of eye exam simulators in the training and assessment of family medicine residents for screening diabetic retinopathy (DR) utilizing direct ophthalmoscopy (DO).
METHODS: This prospective, single arm, cross-sectional study was conducted at King AbdulAziz University Hospital, Jeddah, Kingdom of Saudi Arabia in April 2013, wherein the final year family medicine residents of the Saudi Board family medicine training program, underwent a practical session on DO using an eye exam simulator. The cognitive and motor skills of the participating family residents in performing DO, and their competency at diagnosing DR was assessed before, and after a practical session with the eye simulator.
RESULTS: A total of 14 out of total 20 final year residents consented to join the study. Of these, 57.1% were females. A total of 42.9% (6/14) showed initial motor skill competency, and 35.7% showed cognitive skill competency to diagnose DR. Before the session on the eye simulator, merely 7.1% of the residents expressed confidence in performing DO. After the practical session, 78.6% (11/14) showed motor, and 64.3% (9/13) showed cognitive skill competency, in diagnosing DR. A total of 50% were adequately confident in performing DO. A total of 71.4% (10/14) of the residents preferred learning DO via simulation practical sessions than clinical rotation in ophthalmology clinics.
CONCLUSION: Eye exam simulators are good tools in learning and assessment of DO skills leading to significant improvement in the efficiency and confidence of family physicians in screening for DR.

Diabetes mellitus (DM) is a widespread chronic disease with an estimated global prevalence of 2.2%, which is expected to double by the year 2030.1 Epidemiological studies have shown a constant rise in the prevalence of DM in the Saudi population within the last decade, escalating from 3.8% in the year 20002 to 27.3% in 2004,3 and finally to 30% as reported in 2011.4 Diabetic retinopathy (DR) is a known systemic complication of DM, and it is listed by the World Health Organization (WHO) as one of the top 5 leading causes of blindness in the world.5 It has been found to affect 30% of diabetic patients in Saudi Arabia,6,7 and the cause of blindness in 20.9% of patients, making it the third most common cause of visual impairment in the Saudi population with refractive errors and cataract8 being the leading causes. Hence, it is of utmost clinical importance to screen diabetic patients for the presence of DR as early as possible following a diagnosis of DM.9,10 Primary health care practitioners and family physicians represent the very first level of care, most diabetic patients receive. Patients can be accurately screened for DR in primary care clinics.11 However, ensuring that the primary health care and family physician can optimally and competently performing direct ophthalmoscopy (DO) is vital for timely referral of patients in need for urgent treatment. Clinical studies have shown that 23% of diabetics present with DR during their first visit to the ophthalmologist.12 A significant correlation has been reported between the rate of detection of DR and the delay in screening diabetic patients for DR.13 Although guidelines proposed by the major international ophthalmic societies,14 emphasize the need to acquire the basic skills, to perform DO by all undergraduate medical students and ophthalmologists, and other physicians believe that learning the skill of performing DO is an essential part of medical education this need has not been addressed by most medical schools, as studies in different countries demonstrated a lack of uniformity in the implementation of the ophthalmology curriculum,15 ophthalmology attachment is not compulsory in all the medical schools within the same country,16 and presence of gaps between teaching, knowledge, and clinical skills in several medical schools.17-19 This deficiency in the undergraduate ophthalmology education adversely affects the competency of family physicians resulting in lack of their ability to detect important ocular pathologies.20 Direct ophthalmoscopy is classically learnt on fellow students. However, the limitation of practicing DO on fellow students is that most often ocular pathologies cannot be encountered during this approach. This leads to the understanding that learning and practicing DO on patients with ocular pathologies is a more efficacious method of learning. Moreover, patients with ocular pathologies would be easily willing to participate in sessions, wherein their eyes would be examined via an ophthalmoscope.21 However, the flip side of this approach is the fact that learning DO on patients might be an intimidating experience for students. In addition, the absence of standardized setting is a challenge while choosing the best education method to be used for skill acquisition and assessment. Hence, the situation presents an unmet medical need: the need to adopt alternative methods, which are feasible for teaching DO like the use of eye exam simulators. This study was conducted to evaluate the efficacy of workshops using an eye exam simulator in improving competency and confidence level of the final year family medicine residents in performing DO.

Methods

This prospective, single arm, cross-sectional study was conducted at King AbdulAziz University Hospital, Jeddah, Kingdom of Saudi Arabia in April 2013 wherein the final year family medicine residents of the Saudi Board family medicine training program underwent a practical session on DO using an eye exam simulator. After obtaining approval from the ethics committee at King AbdulAziz University, all final year family medicine residents were invited to participate in the study. The study included those residents who had completed their ophthalmology rotation in their previous year according to their curriculum, which included didactic lectures and clinical rotation in ophthalmology clinics. All residents underwent an initial evaluation using a 5-minute objective structured clinical examination (OSCE) station with the eye exam simulator (Kyoto Kagaku Co. Torrance, CA, United States,22 with 3 mm pupil size, showing a photo of pre-proliferative DR. Six colored fundus photographs showing different retinal diseases printed on an A4 size white paper including the photo shown in the simulation model, and labeled A-F were also shown. Residents were asked to identify the photo seen in the simulation model to assess their DO motor skills. Also, they were instructed to write the diagnosis of the pathology observed, in order to assess their cognitive skills in diagnosing DR (Appendix 1 shows part 1 assessment sheet). After the conclusion of the first part of the study, all the residents received a 90-minute intensive practical session on DO that included a brief lecture, and a hands-on practice session using eye exam simulators. Subsequently, all the residents were re-evaluated using the same OSCE station to detect any improvement in their motor and cognitive skills in DO and DR screening (Appendix 2 shows part 2 assessment sheet).

Statistical analysis

Data was analyzed using the Statistical Package for Social Sciences software version 20 (IBM Corp, Armonk, NY, USA). Differences were considered statistically significant at p<0.05.

Results

Fourteen out of the total 20 (70%) fourth year family medicine residents consented to participate in this study. A total of 57.1% (8/14) of them were females. All the participating residents underwent a month long clinical ophthalmology rotation in the previous academic year, and had successfully passed the end-of-rotation exam. None had undertaken any previous special course on ophthalmoscopy, and 71.4% (10/14) of them graduated from King AbdulAziz University. A total of 42.9% (6/14) of the residents showed initial motor skill competency in DO by correctly identifying the photo shown in the eye exam simulator, and 35.7% (5/14) showed cognitive skill competency by being able to accurately diagnose the ocular condition shown in the simulator model. After the DO simulation practical session, 78.6% (11/14) showed motor skill competency in DO, and 64.3% (9/14) were able to arrive at the accurate diagnosis. shows the difference in motor and cognitive competency level before and after the simulation practical session. Before the session on the eye simulator, none of the resident felt adequately confident in DO, and approximately one third of them (5/14) reported low confidence. Immediately after the practical session using the eye simulator, 50% (7/14) felt quite confident to perform DO (). There was no statistically significant correlation between male and female residents with respect to the motor or cognitive competency in performing DO. A total of 71.4% (10/14) of the residents reported that the experience of using simulated eye trainer in learning DO was satisfactory, and 57.1% found it satisfactory as an assessment method (). A total of 71.4% (10/14) of the residents preferred using simulation in ophthalmology learning as they realized it would provide them sufficient time for adequate practice without causing any harm, or discomfort to actual patients. On the other hand, 78.6% (11/14) of them preferred practicing with patients for assessment as they believed it mirrored the actual real time clinical scenario. A total of 57.1% (8/14) of the residents, mostly females (75%, p=0.043) preferred the option of learning ophthalmology via simulation workshops as compared with any other method.

Comparison of the effect of simulation teaching session on the competency level for direct ophthalmoscop among medical students.

Level of confidence in direct ophthalmoscopy before and after simulation teaching among medical students.

Residents satisfaction from using simulation as a learning and assessment method.

Discussion

This study evaluated the impact of the use of simulation methods in learning, assessment, skill acquisition, and confidence building in conducting DO by family medicine residents. Our results showed that with the current curriculum, 42.9% of our graduating family medicine residents have motor skill competency, and 35.7% have cognitive skill competency in diagnosing DR, and 57.1% of them have little, or no confidence in performing DO. Those results are in line with the studies of Chan,20 Moercke,23 and Eze24 who also reported low competency and confidence levels by non-ophthalmologists in conducting DO in spite of receiving an ophthalmology curriculum that meets the international standards.

Following a single 90-minute practical session using eye exam simulator, our residents showed great improvement in all the tested parameters with doubling of both motor and cognitive skills in comparison with the competency levels detected before the session. In addition, all the residents reported a notable increase in their confidence level in performing DO rising from 7.1% before the practical session to a whopping 50% in the final evaluation. This can be explained by the beneficial effect of learning by simulation, which is known to have a positive correlation with the level of confidence in conducting various clinical skills in both novices, and advanced practitioners.25,26 Moreover, our residents have also benefited from the intense training courses that have been shown in previous studies to have a positive effect on trainees’ self perceived level of confidence.27

In the present study, 71.4% of the residents favored learning by simulation compared with the traditional method of learning in clinical rotations with real patients. The rationale for this preference can be best explained by the fact that simulation-based learning provides flexibility in choosing when, and how long the learner needs to acquire and retain the needed skill, without being conscious, or concerned regarding annoying, or harming patients. However, residents participating in the study preferred the use of real patients for assessment as they believed actual patients better mirror real time clinical situations. This impression supports the reason behind using the term simulation-enhanced education, instead of simulation-based education, as simulation should act as a supplementary academic tool, and can never replace the learning obtained by handling patients.

The limitation of this study is that it has been conducted once on freshly trained residents. Future studies need to investigate when skill decay starts to guide programs directors in planning the best schedule for reinforcement sessions to achieve the desired goal.

In conclusion, based on the results of this study, we recommend that the ophthalmology curriculum for the family medicine postgraduate training program must be re-evaluated. The introduction of frequent simulation practical sessions will allow the trainees not only to acquire the basic eye examination skills, but also further expertize in ophthalmic examination. This initiative can also help prevent decay of ophthalmology diagnostic skills among general practitioners, and will augment their ability to screen and identify patients with DR. In turn, general physicians will be able to refer cases of DR to the ophthalmologist in a timely manner, which could be instrumental in controlling the rising prevalence and incidence of visual disabilities in the country.