Direct ophthalmoscopy on YouTube: analysis of instructional YouTube videos' content and approach to visualization.

BACKGROUND: Direct ophthalmoscopy is well-suited for video-based instruction, particularly if the videos enable the student to see what the examiner sees when performing direct ophthalmoscopy. We evaluated the pedagogical effectiveness of instructional YouTube videos on direct ophthalmoscopy by evaluating their content and approach to visualization.
METHODS: In order to synthesize main themes and points for direct ophthalmoscopy, we formed a broad panel consisting of a medical student, junior and senior physicians, and took into consideration book chapters targeting medical students and physicians in general. We then systematically searched YouTube. Two authors reviewed eligible videos to assess eligibility and extract data on video statistics, content, and approach to visualization. Correlations between video statistics and contents were investigated using two-tailed Spearman's correlation.
RESULTS: We screened 7,640 videos, of which 27 were found eligible for this study. Overall, a median of 12 out of 18 points (interquartile range: 8-14 key points) were covered; no videos covered all of the 18 points assessed. We found the most difficulties in the approach to visualization of how to approach the patient and how to examine the fundus. Time spent on fundus examination correlated with the number of views per week (Spearman's ρ=0.53; P=0.029).
CONCLUSION: Videos may help overcome the pedagogical issues in teaching direct ophthalmoscopy; however, the few available videos on YouTube fail to address this particular issue adequately. There is a need for high-quality videos that include relevant points, provide realistic visualization of the examiner's view, and give particular emphasis on fundus examination.

Introduction

Direct ophthalmoscopy is a fast and cheap method for diagnosing life- and sight-threatening conditions1 that medical students and physicians encounter frequently.2,3 It is taught to medical students, and frontline physicians are expected to be familiar with the skill.4,5 However, reports show that in reality, this skill is performed poorly and avoided due to lack of adequate training and low self-confidence.6,7 Direct ophthalmoscopy may be extremely well-suited for online video-based instruction for two reasons: videos may enable the student to see exactly what the examiner sees, providing realistic images of what to expect, and timely access that enables recall of the technique right before it needs to be performed. YouTube (Google Inc., Mountain View, CA, USA) was launched in May 2005 as a user-created video-sharing website and has since gained extreme popularity with presently more than 1 billion users, including medical students and physicians who use it regularly for instructions on clinical skills.8,9 Apart from a few good examples, contents of these instructional videos are reported to be of mediocre or poor quality in important areas such as basic life support,10 physical examination,11 basic clinical skills,12–14 anatomy and physiology,15,16 and surgery.17 Medical students, young physicians, and faculty teaching direct ophthalmoscopy need an overview of what is available and what is needed.

Our aim with this descriptive and exploratory study was to systematically evaluate instructional YouTube videos on direct ophthalmoscopy in terms of content and approach to visualize instructional themes and points.

Methods

We synthesized the main themes and points within themes for direct ophthalmoscopy by reading book chapters targeted to medical students and physicians in general18–20 and formed a panel consisting of one medical student, two junior physicians, one young ophthalmologist, one senior consultant and research associate professor in medical education and simulation, and one senior consultant and professor in ophthalmology who regularly teaches direct ophthalmoscopy (Table 1).

Table 1

Main themes and points within themes for direct ophthalmoscopy

Themes	Points
Handling of the ophthalmoscope	Light control
	Lens selection
	Filter and aperture selection
Optimizing the environment	Dimmed room light
	Dilation using eye drops
	Taking of glasses unless extreme astigmatism
Approaching the patient	Informing the patient about the examination
	Instructing the patient where to look
	Orientation (the examiner and the examinee)
	Gently holding the patient’s upper lid
	Identifying the red reflex
	Approaching from a temporal position
	Adjusting focus to anterior segments
	Adjusting focus to fundus
Fundus examination	Identification of retinal structures and areas
	Structure and sequence of the examination
	How to interpret findings
	Limitations

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to ensure quality in the selection process and in data extraction.21 We searched YouTube on February 15, 2016 using the search term: “ophthalmoscopy” OR “ophthalmoscope” OR “fundoscopy” OR “fundoscope”. We included videos that gave instructions for direct ophthalmoscopy on human eyes. We did not restrict on upload date, video duration, quality, or rating. For practical reasons, we excluded videos with less than ten views and non-English videos.

All videos were screened based on title and description by one author who excluded obviously irrelevant videos. Two authors reviewed the remaining videos to assess eligibility and extract data. In the case of video duplicates, data were extracted from the copy with most views. We extracted data on the upload date, video duration, highest available streaming quality, number of positive and negative ratings, views, and any details on uploader credentials. We evaluated the content of the included videos by assessing whether themes and points were addressed, and noted the time spent on these key points and the sequence at which these points were presented. We also investigated how the contents of the video were visualized.

Due to the nonparametric nature of our data, continuous values were presented as median and interquartile range (IQR). Correlations between video statistics and contents were investigated using two-tailed Spearman’s correlation. Number and percentage were reported for categorical values. We used Excel 2016 (Microsoft Corporation, Redmond, WA, USA) for data management and Statistical Package for the Social Sciences 23 (IBM Corporation, Armonk, NY, USA) for data analysis, and considered a P-value <0.05 as an indication of statistical significance. Ethical approval and patient consent were not sought for this study, since no patients or participants were involved, and since this study is a review of instructional YouTube videos.

Results

We screened a total of 7,640 videos, of which 101 (1%) were found relevant for detailed eligibility assessment. Of the 101 videos, 74 (73%) were excluded, as 40 (54%) were not instructional (eg, presentation of ophthalmoscopy products), 22 (30%) were duplicates, eight (11%) were not videos of direct ophthalmoscopy, and four (5%) were not in English (Figure 1). The remaining 27 (27%) videos were included in this study (Table 2). Eligible videos were uploaded between May 4, 2007 and October 28, 2015. In total, these videos were viewed 903,105 times. Video statistics with medians, IQRs, and ranges are summarized in Table 3. Only eight (30%) videos were of high-definition quality (720p and above) and newer videos were uploaded at a higher quality (ρ=−0.73; P<0.001; Spearman’s correlation). All 27 videos were included in the evaluation of content and visualization. Five of the videos included were originally one video that was separated into five different smaller videos by the uploader, and these five videos were evaluated in combination.

Figure 1

Selection of eligible YouTube videos for the study.

Table 2

YouTube videos included in this study

https://www.youtube.com/watch?v=NE_epHjNpfo

https://www.youtube.com/watch?v=leMexvs9HVU

https://www.youtube.com/watch?v=PUz2HLromxY

https://www.youtube.com/watch?v=wPzCA9k8GRQ

https://www.youtube.com/watch?v=-7Sh2H1jgUU

https://www.youtube.com/watch?v=AzxNGz1cjgI

https://www.youtube.com/watch?v=bHP2rn4W-pM

https://www.youtube.com/watch?v=D7Lyv0oTjV0

https://www.youtube.com/watch?v=9xxTNINSrFs

https://www.youtube.com/watch?v=7lhvhKvK_iM

https://www.youtube.com/watch?v=AutUi09JIXY

https://www.youtube.com/watch?v=tTU2jiWHpJQ

https://www.youtube.com/watch?v=7UhA40RUIpM

https://www.youtube.com/watch?v=czw_8fWLibE

https://www.youtube.com/watch?v=2S8NUBdS8Kc

https://www.youtube.com/watch?v=KyrfE4ub2qg

https://www.youtube.com/watch?v=yhDm8lLF-z4

https://www.youtube.com/watch?v=-IVsQ1zaoaQ

https://www.youtube.com/watch?v=-iuumsGWo6k

https://www.youtube.com/watch?v=5LvUmOFOsGA

https://www.youtube.com/watch?v=xomOEqw-LDk

https://www.youtube.com/watch?v=KidtiY7mwjk

https://www.youtube.com/watch?v=4zAfHf5Y4EM

https://www.youtube.com/watch?v=vG8n3ZCr80Q

https://www.youtube.com/watch?v=34z5Q0z8ocg

https://www.youtube.com/watch?v=Um8cSYmoyw8

https://www.youtube.com/watch?v=Sjgy4TwT5Vk

Table 3

Summary of video statistics based on YouTube data

	Median	Interquartile range	Range
Duration, seconds	213	98–357	23–717
Ratings1
Positive, n	20	1–56	0–640
Negative, n	0	0–2	0–14
Total views, n	11,636	579–54,022	19–196,519
Available online, days	1,634	577–2,783	30–3,176
Views per week, n	43	12–213	2–857

Note:

Ratings were disabled by the uploader in one video.

Video contents

Handling of the ophthalmoscope was included in 17 (74%) videos with a median duration of 66 seconds (IQR: 44–114 seconds) and was presented at the beginning in 16 videos (94%). Optimizing the environment was included in 17 (74%) videos and had a median duration of 10 seconds (IQR: 3–26 seconds). This part was presented at the beginning in six (35%) videos and as the second part in eleven (65%) videos. Approaching the patient was included in all videos with a median duration of 79 seconds (IQR: 55–99 seconds). This was mostly presented as the second (43%) or the third (43%) part of the video. Fundus examination was included in 17 (74%) videos with a median duration of 58 seconds (IQR: 40–94 seconds), and was always presented as the final part of the video. No videos covered all of the 18 points assessed. Overall, a median of 12 points (IQR: 8–14 key points) were covered. Detailed summary is available in Table 4. Videos were uploaded by medical schools (n=6), unspecified users (n=6), medical doctors (n=4), ophthalmoscopy distributors (n=3), hospitals (n=2), and university students (n=2). We found a few examples of poor instructions not recommended by experts within the field: one video examined the patient’s left eye with right eye and another video recommended to start fundus examination by examining the macula. We found a significant correlation between the time spent on reviewing fundus examination and views per week, but not total views, indicating that the popularity of the video may be more due to the video reviewing fundus examination than the video being popular because of longer exposure time on YouTube (Table 5).

Table 4

Themes and points on direct ophthalmoscopy included in YouTube videos

Themes	Points	Yes n (%)	Partly n (%)	No n (%)
Handling of the ophthalmoscope	Light control	16 (94)	1 (6)	0 (0)
	Lens selection	15 (88)	1 (6)	1 (6)
	Filter and aperture selection	12 (70)	1 (6)	4 (24)
Optimizing the environment	Dimmed room light	14 (82)	0 (0)	3 (18)
	Dilation using eye drops	7 (41)	0 (0)	10 (59)
	Taking of glasses unless extreme astigmatism	5 (29)	3 (18)	9 (53)
Approaching the patient	Informing the patient about the examination	8 (35)	1 (4)	14 (61)
	Instructing the patient where to look	21 (91)	0 (0)	2 (9)
	Orientation of the examiner and the examinee	19 (83)	1 (4)	3 (13)
	Gently holding the patient’s upper lid	5 (22)	4 (17)	14 (61)
	Identifying the red reflex	19 (83)	0 (0)	4 (17)
	Approaching from a temporal position	14 (61)	1 (4)	8 (35)
	Adjusting focus to anterior segments	3 (13)	2 (9)	18 (78)
	Adjusting focus to fundus	10 (43)	3 (13)	5 (22)
Fundus examination	Identification of retinal structures and areas	14 (82)	3 (18)	0 (0)
	Structure and sequence of the examination	13 (76)	2 (12)	2 (12)
	How to interpret findings	5 (30)	2 (12)	10 (58)
	Limitations	2 (12)	1 (6)	14 (82)

Table 5

Correlations between the time spent on different themes on direct ophthalmoscopy and YouTube video statistics

	Handling of the ophthalmoscope (n=17)	Optimizing the environment (n=17)	Approaching the patient (n=23)	Fundus examination (n=17)
Ratings1
Positive	ρ=0.18	ρ=0.36	ρ=0.16	ρ=0.38
	P=0.51	P=0.18	P=0.47	P=0.14
Negative	ρ=−0.10	ρ=0.29	ρ=0.00	ρ=0.24
	P=0.71	P=0.28	P=1.00	P=0.36
Total views	ρ=−0.09	ρ=0.15	ρ=−0.04	ρ=0.25
	P=0.33	P=0.58	P=0.85	P=0.33
Views per week	ρ=0.06	ρ=0.36	ρ=0.07	ρ=0.53
	P=0.83	P=0.15	P=0.76	P=0.03

Notes: All correlations are based on Spearman’s method for nonparametric data. ρ is the Spearman’s correlation coefficient.

Ratings were disabled by the uploader in one video.

Content visualization

Handling of the ophthalmoscope was demonstrated as seen by the examiner in 15 videos (88%) and not demonstrated in one video (6%). In one video (6%), the viewer could only see the effect of adjusting the ophthalmoscope on the light projected, but without seeing how the ophthalmoscope was actually handled. Optimizing the environment was demonstrated as seen by the examiner in ten videos (41%) and not demonstrated in six videos (35%).

We found most difficulties in video demonstration of how to approach the patient and how to examine the fundus. Approaching the patient was demonstrated as seen by the examiner in two videos (9%) and not demonstrated in one video (4%). The remaining 20 videos (87%) were seen from a bystander’s point of view in which one would see the examiner get closer to the patient without seeing what the examiner actually sees through the ophthalmoscope. Three of these videos compensated for this by including a picture of what the red reflex looks like. Fundus examination was demonstrated as seen by the examiner in four videos (22%), of which two were an approximation achieved by using a fundus photograph but limiting the visible field to a small circular area. Fundus examination was not demonstrated at all in four videos (22%). In the remaining ten videos (56%), a fundus photograph was shown without limiting the visible field.

Discussion

One of the fundamental problems in teaching direct ophthalmoscopy is that the student cannot see what the teacher sees in the direct ophthalmoscope and vice versa.1 Hence, it is difficult to both show what it looks like when the teacher performs the skill correctly and to give relevant feedback to the student. Therefore, it seems that direct ophthalmoscopy may be well-suited for video-based instruction, particularly if videos enable the student to see what the examiner sees when performing direct ophthalmoscopy. This study showed that the number of instructional videos on direct ophthalmoscopy is small with varying content and that the possibilities of video instruction were not utilized in such a way that the examiner’s view was realistically visualized. Also, we found that more time spent on fundus examination correlated with more views per week, which may suggest that this part is of particular interest to the viewers.

There is an ongoing discussion on whether or not direct ophthalmoscopy should be a part of the pregraduate medical curriculum and be expected as a skill every physician should be proficient in.22,23 Eye disorders are a common cause for medical attention in the primary sector and their prevalence is only expected to increase with an increasing elderly population.2,3,24 An incorrect or delayed diagnosis can result in otherwise preventable loss of vision, which impacts the patient’s ability to drive, work, and function in general.25,26 While these facts pull in the direction of keeping direct ophthalmoscopy in the curriculum, some argue that the general proficiency in direct ophthalmoscopy is poor, which by itself may lead to an incorrect or delayed diagnosis.23 We believe that improvements in educational initiatives are yet to be explored and that better Internet resources including YouTube videos, innovative instructional methods,27–30 and evidence-based simulation training30,31 may contribute to optimizing training in direct ophthalmoscopy in the future.

Internet resources are of variable quality, as anyone with or without credentials can contribute.32,33 We found a few poor examples in YouTube videos of direct ophthalmoscopy. Such examples of poor videos have also been demonstrated in, for example, cardiopulmonary resuscitation and lumbar puncture, which may lead to severe consequences on the patients.10,34 How best to navigate as a clinical instructor with these types of resources is a matter of ongoing discussion.32,33 One solution is that the clinical instructor develops his/her own educational videos or distributes selected quality-verified educational material through their own developed apps to achieve higher quality when the users access these materials on-the-go.35–38

A plethora of studies exist on how to implement Internet technologies for medical education and they show that it may be feasible on many occasions, but paucity remains when it comes to studies on why and when it works.39 Mayer and Moreno lay the ground work for the Cognitive Theory of Multimedia Learning, which is a framework for understanding multimedia-assisted learning from the learner’s perspective.40 According to the theory, images and sounds are channeled into the working memory, which then integrates and organizes information for short- and long-term memory.40 Mayer also suggested that the capacity of this system is limited and that learning outcome may depend on the presentation because learners can be overloaded with irrelevant details.41 For example, highlighting essential material, eliminating extraneous material, and presenting words and pictures in combination rather than as words alone are all shown to significantly benefit learning.41 Our experience and findings of this study indicate that the available YouTube videos on direct ophthalmoscopy were not designed with these considerations in mind, which if otherwise could provide an important improvement to the learning experience that suffers from a nonvisual instructive nature of a procedure that is fundamentally visual. When designing instructional videos on direct ophthalmoscopy, we suggest the inclusion of the spectrum of relevant points (Table 1), realistic visualization of what the procedure actually looks like from the examiner’s view, and giving particular emphasis on how to examine the fundus, which we find is correlated with video popularity since this may reflect what the users are demanding. Also, we suggest that Mayer’s findings41 on how to optimize the pedagogical effectiveness by considering the design of the instructional videos, for example, highlighting essential material, eliminating extraneous material, and presenting words and pictures in combination, are considered when designing these videos, as these considerations may further improve the instructional value.

How best to utilize video instruction of clinical procedures and their role in timely learning in clinical practice is a field yet to be fully explored. One study found that medical students could perform volar splinting better and faster when they had timely access to an instructional video.42 Another study found that medical students and resident physicians performed better at chest tube insertion when they could use a mobile device for timely access to an instructional video.43 Exploratory studies have looked into how this knowledge is appreciated and suggest that timely access to instructional videos cannot stand alone, but rather augment the overall learning experience.44,45 Learning technical skills may need hands-on experience and relevant feedback, which requires on-site teaching on patients or simulation training.

Strengths and limitations of this study should be considered when interpreting our findings. One strength is the inclusion of a panel representing a broad range of expertise levels, which enabled different views on the matter. Another strength is our systematic approach to inclusion of videos and extraction of data. Inter-rater agreement of our evaluations was 0.95 for evaluating themes and points and 0.87 for the sequence of presentation (Cohen’s kappa), which demonstrates good consistency. We have experienced that systematic searches on YouTube yield a very large number of hits with videos that are irrelevant, which is also reflected by the low number of eligible videos (1%) obtained from the screening process. An important limitation is that we only assessed videos with ten views or more and in English for practical reasons and to provide results that are relevant for a broad community. However, this approach may miss videos that are released very recently or the videos that may be well-designed but are not popular for other reasons. Videos may lack important content or may be designed poorly, but still serve their purpose. Another important limitation of our study design is that we did not investigate the learning effect of the included videos and, therefore, cannot say whether these videos actually work well in practice.

In conclusion, videos may help overcome the pedagogical issues in teaching direct ophthalmoscopy, but current videos on YouTube fail to address this particular issue adequately. There is a need for high-quality videos that fully utilize the possibilities of video as an instructional tool (Table 6).

Table 6

Suggestions for videos instructing in direct ophthalmoscopy

Illustrate the key themes and points essential for performing direct ophthalmoscopy

Consider how to illustrate the key concepts so that the learner sees what he/she should expect to see

Put emphasis on how to examine the fundus and interpret findings

Consider omitting irrelevant details, highlighting essential information, and presenting words and pictures in combination