Ability of Head-Mounted Display Technology to Improve Mobility in People With Low Vision: A Systematic Review.

Purpose: The purpose of this study was to undertake a systematic literature review on how vision enhancements, implemented using head-mounted displays (HMDs), can improve mobility, orientation, and associated aspects of visual function in people with low vision.
Methods: The databases Medline, Chinl, Scopus, and Web of Science were searched for potentially relevant studies. Publications from all years until November 2018 were identified based on predefined inclusion and exclusion criteria. The data were tabulated and synthesized to produce a systematic review.
Results: The search identified 28 relevant papers describing the performance of vision enhancement techniques on mobility and associated visual tasks. Simplifying visual scenes improved obstacle detection and object recognition but decreased walking speed. Minification techniques increased the size of the visual field by 3 to 5 times and improved visual search performance. However, the impact of minification on mobility has not been studied extensively. Clinical trials with commercially available devices recorded poor results relative to conventional aids. Conclusions: The effects of current vision enhancements using HMDs are mixed. They appear to reduce mobility efficiency but improved obstacle detection and object recognition. The review highlights the lack of controlled studies with robust study designs. To support the evidence base, well-designed trials with larger sample sizes that represent different types of impairments and real-life scenarios are required. Future work should focus on identifying the needs of people with different types of vision impairment and providing targeted enhancements. Translational Relevance: This literature review examines the evidence regarding the ability of HMD technology to improve mobility in people with sight loss. Copyright 2020 The Authors.

Introduction

According to the World Health Organization, more than a billion people have vision impairment or blindness that cannot be treated. Globally, the leading causes of low vision are uncorrected refractive error and cataract. In the developed world, the leading causes are macular degeneration, glaucoma, cataract, diabetic eye disease, and retinitis pigmentosa (RP)., These eye conditions can result in central vision loss (CVL), peripheral field loss (PFL), or a combination of both with approximately 74%, 13%, and 11% of people with low vision suffering from each type of loss, respectively. One of the problems faced by people with low vision (LV) that has an impact on their overall quality of life and limits their participation in day-to-day activities is reduced mobility.– Mobility is the key dimension of generic health-related quality-of-life, visual disability and vision-related quality-of-life.,

Mobility is defined as the act or ability to move from one's current location to one's desired location in another part of the environment safely, gracefully, efficiently, and comfortably. Having good orientation skills, defined as the ability to use one's residual vision and/or other senses to understand the local environment at any given time, is critical to achieving good mobility performance.

Mobility performance in people with LV is influenced by a range of visual factors including the visual field (VF),– contrast sensitivity (CS), visual acuity (VA),, and visual scanning ability., It has been proposed that people with LV have critically impaired mobility when their VF is smaller than 15 degrees diameter, and they are at risk of having inadequate mobility when the VF is reduced to 31 degrees to 52 degrees. It is widely accepted that the size of the VF together with CS are the most important predictors of mobility performance. Although there is no universal agreement on the impact of acuity on mobility performance, some studies found an association between reduced acuity and mobility impairment. Additionally, acuity and CS are necessary for detecting obstacles from a distance.

The mobility problems experienced by people with LV depends on the nature of the underlying vision loss. For those with central vision loss, mobility problems may stem from difficulty reading signs, coping with public transport, and finding the desired destination in unfamiliar places. In contrast, people with PFL tend to have problems navigating stairs, detecting and avoiding people and obstacles, and avoiding trip hazards., Regardless of the type of VF loss, mobility problems become more pronounced when the level of illumination is reduced., Many visual conditions also result in reduced depth perception, which is also important for mobility.

A fundamental concept in low vision rehabilitation is the notion of making things bigger (e.g. larger signs), bolder (e.g. use of contrasting colors to highlight obstacles), and/or brighter. However, making these modifications in the built environment is not practical in most instances. Sight substitution approaches, such as the use of long canes or guide dogs, are popular approaches in improving mobility. Guide dogs are effective mobility aids, but their high cost and associated maintenance issues are a limitation. Canes are inexpensive and effective aids that help people survey their immediate environment (e.g. obstacles and changes in surface heights). However, their usefulness is limited by the length of the cane and to low-lying obstacles. Telescopes can be used for orientation tasks, such as reading signs or identifying features of the environment. However, their effectiveness is limited by a restricted field of view (FOV) and the need for good physical coordination and dexterity.

In recent years, rapid technological advances have led to an increase in the number of electronic mobility aids. These devices use various sensors (depth and ultrasonic sensors, and cameras) to capture the environment. Using computer vision and signal processing techniques, the visual information is translated into alternative modalities, such as auditory and vibrotactile. Audio electronic travel aids provide feedback (warnings and guidance) with either verbal descriptions or sonification. The vibrotactile aids provide feedback through small vibrators embedded in various places, such as in the handle of an augmented cane, soles inside shoes, wrist bracelet, and belt, and so on.

Although vision substitution techniques would be essential for people who are completely blind, the majority of people with LV have useful residual vision and prefer to use it to observe the environment. As smartphones become more ubiquitous and powerful, the number of mobile applications designed for people with vision loss is increasing. Utilizing complex computer vision algorithms, mobile applications can apply filters to modify the brightness range or increase the contrast of edges in images or videos. Mobile devices can also use deep learning techniques to understand the environment to detect and recognize text or people and obstacles., Similar computer vision techniques can be applied to head-mounted display (HMD) systems with the added benefit of them being hands-free. More than 25 years ago, Massof et al. developed the first head-mounted low vision aid, the Low Vision Enhancement System, which provided improved VA and CS. Since then, advances in sensors, cameras, displays, and computational hardware have led to the availability of various consumer HMD systems, such as Google Glass, Microsoft HoloLens, and Oculus Rift. These powerful devices facilitate the development and implementation of complex computer vision algorithms that may improve mobility and orientation in people with LV.

To make the best use of these emerging technologies as LV mobility aids, it is useful to understand the current state of HMD technology, and what types of vision enhancements and information processing have already been identified as being helpful in improving mobility performance. Harper et al. reviewed the earliest HMD devices and suggested potential future developments in image enhancements. The narrative review paper by Deemers et al. provides an excellent overview of currently available HMD devices and flags strategies that can be used to help people with LV in general. In addition, Ehrlich et al. have recently provided an expert perspective on different types of HMD technologies, identifying important optical and human factor considerations. This paper complements this work by providing a systematic review of the current state of research on how image enhancements and processing techniques implemented on HMDs affect orientation and mobility in people with LV.

Methods

Search Method

A systematic review of the literature was undertaken. Potentially relevant articles published any time before November 2, 2018, were identified via a keyword search of the following databases: MEDLINE, CHINL, Scopus, and Web of Science. A wide range of keywords, developed in conjunction with a subject librarian, was used to capture the potentially relevant literature (see the Table). Articles were screened in three stages: (1) based on the title, (2) based on the title and abstract, and (3) based on the full text of the article.

Table.

Keywords – Items Within Each Column were Joined by the ‘OR’ Operator and Between Columns were Joined by the ‘AND’ Operator.

Technology	Visual Impairment	Functionality
“human computer interaction”	“sight loss”	Mobility
“emerging tech*”	“visual* impair*”	Orientation
“image enhanc*”	“low vision”	Navigation
“head mounted”	blind*	Wayfinding
“object recogn*”	“vision loss”	“obstacle avoidance”
“object detect*”		“obstacle detection”
“augmented reality”		“visual function”
“virtual reality”		“vis* enhancement”
“electronic travel aid*”		“enhanced vision”
wearable*
“computer vision”

Study Selection

Articles reporting vision enhancement techniques, implemented using HMDs aimed at improving mobility or associated visual functions were included. Articles describing physical implants, such as retinal prosthesis, were excluded. Studies of image enhancements implemented on displays other than HMDs, mobility aids with alternative modalities other than visual, and vision enhancements for near-vision tasks were also excluded as were studies that did not involve participants.

Data Collection

Information about the HMD devices used, type of enhancements/intervention, participant characteristics, study design, and outcomes were extracted.

Results

The initial literature search identified 2474 potentially relevant articles, but this included 856 duplicates. After removing duplicates, the remaining 1618 articles were screened according to the methods described previously, leaving just 28 articles in the review (see Fig. 1).

Figure 1.

Flowchart of the literature search process

The results presented in the following paragraphs are grouped based on the type of enhancement used in the studies. Six studies evaluated scene simplification, seven studies explored the effect of scene minification, and five studies assessed the usefulness of off-the-shelf devices. One study explored scene minification and digital zooming. A further nine studies explored various enhancements, such as digital zooming, edge enhancement, and the use of visual cues to attract attention.

The studies in this review used different types of augmented reality (AR) technology to provide visual enhancements. Some of these enhancements were aimed at improving mobility directly, whereas the rest improved aspects of visual function that are important for orientation, such as visual search and reading signs.

Most investigations used an observational, cross-sectional, within-subject study design. Others used a randomized placebo-controlled study design, case-controlled study design,– a longitudinal within-subject design,, and a within-group design. To evaluate visual factors related to orientation and mobility, a range of clinical vision tests, recognition tests, and visual search tests were used. Clinical vision tests included VA, CS, and VF. Recognition tests involved participants locating and recognizing specific items (e.g. objects, poses, and obstacles). Visual search tests involved locating letters and symbols on different backgrounds in both stationary and mobile scenarios as well as following visual cues to improve scanning and search performance. To quantify mobility performance, the studies used indoor mobility courses with varying levels of difficulty. One study used a real-world outdoor environment at night. Time taken to traverse mobility courses and the number of mobility errors were predominantly used as outcome measures. Mobility errors included unintended contact with obstacles and walls, deviation from the intended path, and problems with orientation (e.g. being unable to find the path after colliding with an obstacle). Only three studies,, used percentage of preferred walking speed (PPWS), a useful mobility efficiency outcome measure that can be used as a between-participant walking efficiency measure, in addition to assessing mobility changes in a single participant. Two studies used raw walking speed as the outcome measure., Van Rheede et al. used additional outcome measures: deviation distance (distance from which participants deviated from the collision course) and hesitation scores (changes in walking speed), calculated from the video data of mobility trials. One study used an orientation and mobility expert to grade the mobility performances.

Technology

This section introduces the AR technologies used in the studies.

AR superimposes the computer-generated virtual world in real-world environments. AR, also known as mixed reality, enhances the natural environment of a person in the real world by adding virtual elements or holograms that are dynamic and interactive. It incorporates mainly visual feedback and uses spatial or 3D sound to provide an immersive real-world experience.

The hardware components of AR systems include a display, processor, sensors, and input devices. These systems primarily use small displays positioned close to the eyes and attached to a headset that rests on the forehead. These display systems are known as HMDs. Displays may use both occluded and see-through displays. The processor processes the state of the real-world environment and generates virtual objects and/or environments. The processor can be built-in (allows better mobility) or tethered (conventional computers attached to the display via a cable). Various sensors (e.g. cameras, infrared depth sensors, accelerometer, and global positioning system [GPS]), and input devices (e.g. keyboard, mouse, game controllers, and eye trackers) allow the AR systems to read the state of the environment and facilitate user interaction. The type of sensors used to acquire information about the environment largely determines the usability of the HMDs. The studies included in this paper used different types of cameras (CMOS and CCD) and depth sensors (based on stereo vision or infra-red). The stereo vision-based camera's usability is limited by environmental factors, such as bright intensity lights and nontextured surfaces. In contrast, infra-red sensors have a limited range and do not perform well with transparent or translucent objects. The sensitivity of CMOS and CCD sensors determines the HMD's usability under low light conditions.

AR is used as the primary platform for visual enhancements to improve mobility in people with LV due to its ability to combine the real-world and holograms that make things easier for users to see. The two different types of HMD technologies used in AR systems are video see-through displays (VSTs) and optical see-through displays (OSTs; see Fig. 2). Optical systems for these HMD display systems can be classified based on the image formation (i.e. pupil forming and non-pupil forming designs). Pupil-forming displays, such as that used in the HoloLens device, are relatively complex and involve an array of lenses and an intermediary image. In non-pupil forming designs, such as the HTC Vive, there is no intermediate image and a single convex lens system is used to ensure that images formed on the display are in focus. Pupil forming displays allow flexibility in the location of the image source, which enables better ergonomics. Non-pupil forming devices are relatively easier to design and fabricate with the disadvantages of adding extra weight over the eyes.

Figure 2.

Example of VST and OST devices. (a) HTC VIVE with occluded display. (b) Microsoft HoloLens with transparent display.

Video See-Through Displays

VSTs, are the occluded display systems that are primarily associated with virtual reality environments (e.g. Oculus Quest, Oculus Rift, and HTC Vive). When used in AR systems, VSTs block the natural view of the environment and allow users to see the real-time environment via a video feed captured by camera(s). The video feed that is displayed to the users could be modified in real-time to provide vision enhancements. The main advantage of these systems is the wide range of image manipulations possible, such as color inversion to enhance contrast and object deletion. Compared with OSTs, these displays also have larger screen sizes. One of the main disadvantages of this type of display is the inability to use peripheral vision, and in case of system failure, the occluded display will leave users completely unable to see until they are taken off. Another drawback is the size and bulkiness of the devices. They tend to protrude in front of the face and are not comfortable for long time usage. They also need to be tethered to a more powerful external hardware (e.g. laptop) to perform computationally expensive tasks, such as running computer vision algorithms. In such a situation for VSTs, being tethered is a major limitation for mobility.

Optical See-Through Displays

OSTs, are the most commonly used type of displays in AR environments as they allow users to see the real-world enhanced with virtual objects or holograms (e.g. Magic Leap, Microsoft HoloLens, Google Glass, and Epson Moverio BT-300). The main advantage of OSTs is unimpeded peripheral vision, and in case of power failure or system failure, they do not impede the user's view. The main disadvantage is the smaller screen size compared with VSTs. In bright daylight, the visibility of holograms or virtual objects is also relatively poor due to the limited dynamic range of the display. However, visors or liquid crystal filters can be used to dim the environment to a certain extent. Precise image registration between virtual and real-world objects is essential to maintain the illusion that they coexist in the same environment. However, with careful design and implementation, modern OSTs can minimize image registration errors (e.g. to a level precise enough to be used as visual guides in surgery)., Due to the see-through nature, certain types of image manipulations, such as color inversion and object deletion, cannot be performed in OSTs.

Types of Enhancements

This section will discuss various types of visual enhancement techniques used to improve mobility.

Scene Simplification

Scene simplification is a process where objects or information that is of little importance to the current task is removed from the scene. By reducing visual clutter, users can focus on essential features, such as obstacles in the scene. Six studies,–,, summarized in Appendix Table A1, assessed the effect of scene simplification on mobility and mobility-related visual tasks.

Table A1.

Summary of Scene Simplification

Author, Year	Technology	Intervention	Participants	Study design	Outcomes
Everingham et al.65 1999	• Simulated HMD • Camera – unspecified • HMD - 50cm diagonal screen placed close to participants at a distance to allow 54.6° of visual field.	• Images were segmented into regions corresponding to 7 different groups of objects. The regions in images were colour-coded based on the type of object (e.g. blue for sky, red for obstacle)	• 16 LV participants (registered blind) • Age: 38-87, mean = 69 • Gender: unspecified • Disease: range of impairments including AMD, RP, and optic atrophy (exact numbers not given)	• Observational. • Within-subject design. • Orientation study. • Training: None. • Method: Three groups of 15 images shown to participants and asked to point out obstacles, vehicles, and tracing region between pavement and road for each group. • Outcome measures: no. of correct identification.	• Recognising obstacles improved from 40% with original-image to 87.5% with colour-encoded images (p<0.001).
Jones et al.46 2006	Custom hardware.• Processor – Dell Inspiron 8200 (Intel P4) • Camera - Marshall Electronics CCD, f = 6.0mm, FoV: 40°H, 33°V, 57°Diagonal • HMD - Olympus Eye-Trek 250 W, binocular, opaque, FoV: 37.5°H, 21.7°V	• Images captured from the camera are segmented in each frame (8 fps) into separate objects. Seven object types (vehicle, vegetation, buildings, sky, road, pavement and obstacles) were recognised and encoded into 7 different colours. The images with coloured objects were presented to the user via HMD. • users shown 3 modes: original images, fully labelled coloured images, and combination of coloured images and original images (blend mode).	• 8 LV participants • Age: 46-83, mean = 64.1 • Gender: 2M, 6F • Disease:▪ 5 RP ▪ 3 RP + cataract • 8 normal vision (NV) control • Age: 48-76, mean =62.42	• Observational. • Case-control study. • Mobility study. • Training: None. • Mobility course – 12.54m x 1m, 3 turns, 30 traffic cones as obstacles. • Method: Traversed the mobility course twice (in each direction) without the headset and average was calculated as the participant's preferred walking speed. Then the participants completed the course in each of the headset conditions. • Outcome measures: PPWS, and number of errors (deviation, orientation, contact)	• LV walking efficiency reduced significantly when wearing headset in original image and coloured image view (mean 40%, 36.01% respectively, p< 0.05). • Blend view improved PPWS for 2 LV participants, overall reduced mean PPWS to 67.92%. • LV participants in blend mode made less errors (p<0.05), no significant change in other views. • Control group showed reduced walking efficiency and made more errors with headset (p<0.05).
Lieby et al.53 2011	Custom hardware.• Processor – unspecified • Camera – Point Gray Bumblebee stereo camera • HMD – eMagin Z800 3D Visor, binocular, opaque, FoV: unspecified.	• Displayed low resolution images (35x30 pixel) of original depth and colour images of size 320x240 pixels. The pixel value in the low-resolution image is calculated based on the corresponding area's depth or colour intensity value in higher resolution image.	• 4 NV participants • Age: 21-24 • Gender: 1M, 3F	• Observational study. • Within-subject design. • Mobility study. • Training: None. • Mobility course – maze made up of 3x6 cubicles of 1.5x1.5 m with obstacles (low-lying and overhanging). • Method: traversed the mobility course with colour and depth-based representations with or without obstacles on the course. • Outcome measures: PPWS, and no. of errors (contact with walls and obstacles)	• Depth-based representation performed better than colour-based images with hanging obstacles (p=0.01). • Colour-based representation performed better than depth without obstacles (p<0.001). • Both representations had over 50% of PPWS, p<0.05. • No statistics given re: no. of contacts.
Hicks et al.47 2013	Custom hardware.• Processor – unspecified laptop (Core2Duo 2 Ghz) • Camera - ASUS Xtion Primesense 1080 infrared camera, FoV: 58°H, 45°V • HMD – 24x8 LEDs attached onto a ski goggle with effective resolution of 18x8 pixels, binocular, opaque, FoV: 58°H, 26°V	• Represents the distances of objects in the world as scale of brightness.	• 7 NV participants • Age: 22-36	• Observational. • Within-subject design. • Mobility study. • Training: 2-3 minutes. • Mobility course – 6x2m, 5 obstacles (wet floor signs). Obstacles arranged in different orders for each trial. • Method: Traverse the course 10 times. • Outcome measures: time taken, median velocity, no. of collisions.	• Time taken decreased significantly across trials (p<0.01) (average 112s to 52s from 1st to 10th trial). • Median velocity: increased significantly across trials (p<0.01), (17 to 31 cm/s from 1st to 10th trial). • No. of collisions: reduced across trials (avg 3.9 1st trial to 1.7 10th trial), but not statistically significant.
			• 18 LV participants • Age: 28–90, mean =53.83 • Gender: 7M, 11F • Disease:▪ 5 RP ▪ 3 AMD ▪ 2 Glaucoma ▪ 3 Retinal dystrophy ▪ 1 cataract ▪ 1 Hemianopia ▪ 1 leber optic neuropathy ▪ 1 Stargaardt disease ▪ 1 sarcoidosis uveitis • 1 NV control.	• Observational. • Case-controlled study. • Orientation study. • Training: None. • Method: Search task. Objects represented in 2x2 pixel of light located within ±60 degrees from middle of the view. Participants physically move the head towards the light stimuli until they are at the middle of the view. Each participant completed approximately 30 trials. • Outcome measures: time taken to complete the action described above.	• Able to see the light on display when it's placed within +- 30 degree from the middle mostly under 2 seconds. Missed increasingly if placed outside of 30-degree zone. • Sighted control had the same response time as those with LV. • Participants can detect people from 4m distance with this view. Within 10 mins, can recognise objects, walls, chairs, and their own limbs.
van Rheede et al.482015	Custom hardware.• Processor – Lenovo Thinkpad X220 • Camera - ASUS Xtion infrared camera, FoV: 58°H, 45°V • HMD – 2 4x4cm OLED panels attached onto a head-mount with the resolution of 20x16 pixels, binocular, opaque, FoV: 60°H, 50°V	• Represents the distance of objects in the real world to participants as scale of brightness • Max brightness for distance of 0.7 m – 1 m and min brightness (dark) for 3.5 m or more.	• 11 LV participants • Age: 26-85, mean=54.73 • Gender: 3M, 8F • Disease:▪ 5 RP ▪ 1 cone dystrophy ▪ 1 Stargaardt disease ▪ 1 genetic retinal dystrophy ▪ 1 AMD ▪ 1 Coloboma with cataracts ▪ 1 congenital hypotrichosis with juvenile macular dystrophy • 5 NV control • Age: 25-28, mean = 27.6 • Gender: 2M, 3F	• Observational. • Case-controlled study • Mobility study. • Training: a few minutes • Mobility course – dimly lit 15m x 5m course with 1.2m x 0.5m cylindrical obstacles (max 6) • Method: Participants traversed average 28 times with or without glasses with varying no. of obstacles • Outcome measures: time taken, no. of collisions, deviation distance, and hesitation score calculated based on changes in walking speed.	• No. of Contacts: 6 participants navigated without contacts. For the rest no. of contacts reduced with glasses from 30.5 ± 15 % to 11.2 ± 14.1% (p=0.031) • Trial completion time increased in all participants. • Deviation distance increased for participants with poorer vision but decreased in control participants and LV participants with better vision. • Hesitation increased in all but 2 participants.
Kinateder et al.45 2018	Hololens• Processor - built-in processor • Camera – built-in infrared sensor • HMD - binocular, transparent display, FoV: 30°H x 17.5°V	High-contrast visual patterns are overlaid on objects in real-world based on the distance of the objects.	• 4 LV participants • Age: 54-65, mean=61.5 • Gender: 3M, 1F • Disease:▪ 2 RP ▪ 1 Cataract and glaucoma ▪ 1 Leber heredity optic neuropathy	• Observational. • Within-subject design. • Orientation and mobility study. • Training: Few minutes. • Method: 4 different tasks. Compared performances between with and without glasses.▪ Person localisation: locate life-sized figurine placed 1.8 m from seated participant. Indicate the location by using laser pointer. ▪ Pose recognition: recognise experimenter with 5 different poses at 1.5 m and mimic the pose with participants’ arms. ▪ Object recognition: recognise objects placed on table at 1.5m away and identify them verbally. ▪ Mobility: walk forward until an obstacle on the path was recognised, compared the performance with AR enhancements to performance without AR glasses and performance with cane. • Outcome measures:▪ Person localisation: scored based on the indicated location with laser-pointer (1 - hit, 0.5 - near miss, 0 - miss). ▪ Pose recognition: scored based on the mimicked pose (1 – correct, 0.5 - partially correct, 0 – incorrect) ▪ Object recognition: responses scored correct or incorrect. ▪ Mobility – distance between stopped position and obstacle. ▪ Confidence – subjective confidence of participants for their responses in all tasks except mobility (rated 1 – a guess to 3 – very certain).	• No significant change in person localisation and pose-recognition. • Object recognition improved in 3 participants with improved confidence, but one had decreased performance and confidence • Mobility: all participants detected and stopped before the obstacles further away than when not wearing glasses.
		Red to blue colour patterns and high-low opacity patterns are overlaid on objects in real-world based on the distance of the objects.	• 48 NV participant with simulated low vision (occlusion foils). • Age: mean=21.15 • Gender: 14M, 34F	• Randomised controlled trial▪ 1 group with AR turned off ▪ 1 group with opacity patterns ▪ 1 group with colour patterns. • Orientation and mobility study. • Training: Few minutes. • Mobility course: 5.3 x 3.6 m room with 3 different layouts of obstacles (desks, chairs, books, fan, basket) • Method: 4 different tasks.▪ Pose recognition, object recognition, and gesture recognition followed methods described above. ▪ Mobility – explore the mobility course with 3 different views. • Outcome measures: recognition tasks scored as describe above in previous experiments and recorded subjective feedback for mobility task.	• Pose recognition: no significant change in • Object recognition: performance and confidence improved significantly in both colour and opacity groups, P<0.001 • Gesture recognition: opacity group improved performance (p<0.05), Confidence improved in both colour and opacity groups (p<0.001) • Participants felt AR was more useful and more confident in exploring the room.

Using a low-resolution occluded display, Hicks et al. used a scene simplification approach to convey depth information by making nearer objects brighter than more distant ones. Assessing the feasibility of this representation for visual navigation involved seven participants with normal vision. The average time taken and number of collisions decreased and the average median velocity increased over 10 mobility trials. The study did not include a control condition. Eighteen people with severe sight-impairment were also able to detect 2 by 2 bright lights representing objects located within 30 degrees from the center of the VF in less than 4 seconds leading the authors to conclude that scene simplification may help people with sight-impairment avoid obstacles. However, when this scene simplification approach was evaluated in a separate study involving participants with sight-impairment and a mobility course, although obstacle avoidance was improved, the time to complete the course was increased, and participants were more hesitant while using the HMDs compared to the performance without them. Another study also simplified the scene by using low-resolution pixel intensity to represent distance (depth) and found depth-based representation to have less reduction in walking speed compared to color-based representation in a mobility course with overhanging obstacles. Using the HoloLens device, Kinateder et al. represented depth in different high-contrast color and opacity patterns. The high-contrast visual patterns improved object recognition and allowed the four participants with sight impairment to detect obstacles from further away compared to using a cane or residual vision. These studies suggest that scene simplification can help people with obstacle detection by encoding depth information.

Another simplification technique is to color-code different categories of objects., However, although this color coding technique worked for object recognition in static photographs, it caused a significant reduction in PPWS in eight participants with PFL and eight age-matched controls when evaluated using a mobility course compared to performance without HMDs.

Although the depth-based scene simplification techniques showed potential, especially in obstacle detection, the experiments were carried out in controlled laboratory environments with large obstacles, and it is not clear if this will translate to real-life scenarios where obstacles are often smaller and spaced much more closely.

Minification

Minification aims to help people with PFL by shrinking the image so that more of the visual field is projected onto the center of the FOV. Expanding VF size is potentially useful as it is an important predictor of mobility performance.– Eight studies,,– summarized in Appendix Table A2, assessed the effects of minification on orientation and mobility.

Table A2.

Summary of Scene Minification Studies

Author, Year	Technology	Intervention	Participants	Study design	Outcomes
Vargas-Martín et al.66 2002	Various combinations of off-the-shelf cameras and HMDs.• Cameras: Mitsubishi Artificial Retina • HMDs:▪ Sony Glasstron, Binocular, transparent, FoV: 22°H ▪ MicroOptical EyeGlass, Monocular, transparent, FoV: 17°H	Using the software provided by hardware manufacturers, extracted edge information and overlaid the minified scene using edges in the central field of view.	• 2 LV participants • Age: unspecified • Gender: unspecified • Disease: 2RP	• Observational. • Within-subject design. • Orientation study. • Method: Using different combinations of camera and HMDs, presented the minified view to the participant, for clinical vision test. • Outcome measures: visual field using Auto-Plot Perimeter in dim room.	3 times expansion of visual field measured.
Bowers et al.54 2004	Custom device – LV3• Camera: unspecified, CMOS sensor, 2.9mm ff/2 micro lens. • HMD: Monocular, transparent, FoV: 16°H x 12°V Multi vision (MV) (Commercial night vision device)• HMD: Binocular, opaque, FoV: 32°H x 24°V	Minified grey-scale image of the view of the environment is superimposed onto the central field of view.	• 6 LV participants • Age: 26-68, mean=48.67 • Gender: 4M, 2F • Disease:▪ 5 RP ▪ 1 Chloridaemia	• Observational. • Within-subject design. • Orientation and mobility study. • Training: short training, exact figure not given. • Mobility course: 44m long course divided into two sections lit at 16 lux and 2 lux on average, 41 low lying obstacles (cupboard boxes), and 16 overhanging obstacles (plastic cups and paper bags). 32 m long obstacle-free dark course (<0.1 lux). • Outcome measures:▪ Clinical vision tests: with and without devices in VA, CS at standard illumination levels of lighting cabinet and VF with overhead fluorescent illumination. ▪ Mobility test: PPWS, number of contacts (obstacles and walls), and subjective score of perceived difficulty.	• VA decreased significantly with LV3 (P = 0.02). • CS reduced by 0.6 log units with LV3 (P = 0.004) • VF increased significantly with LV3 by 178% at 16lux (P = 0.002) and 287% at 2 lux (P = 0.002). • LV3 decreased PPWS in both mobility courses (P = 0.01). Increased number of contacts with LV3 (3.3 more contacts, P = 0.03).
Luo et al.71 2006	Custom device by MicroOptical,• Camera: unspecified. • HMD: Monocular, transparent, FoV: 16°H x 12°V	6x and 3x minified edge images of the wider view of field are superimposed onto the centre of the visual field.	• 12 LV participants • Age: 52 ± 9 • Gender: unspecified • Disease:▪ 11 RP, ▪ 1 Choroideraemia (CHM) ss	• Observational. • Within-subject design • Orientation study • Training: < 1hr • Search areas: 2 areas (studies A and B) 90°x70° and 66°x54° degrees with participants sat 32” and 50” away respectively with 3 and 9 participants for respective search areas. • Method: Visual search task to identify random low-contrast letter surrounded by black frame, projected onto a grey background, outside the visual field of the participants. Compared the results w/ and w/o devices and with auditory cue beeping at the direction of the target. • Outcome measures: search time, gaze directness, and gaze speed.	• Search time:▪ Auditory cue: faster (39%∼58%, P < 0.001) and (54% average, P < 0.001) in studies A and B respectively ▪ With minified view: (28%∼74%, P ≤ 0.016) faster search time in study A. The search time of the group VF ≥ 10° (n = 6) reduced by 22% but increased by 177% for small VF group VF <10° (n = 3) by 177% (F1.7 = 7.3; P = 0.03) in study B. • Gaze directness: Minified view improved directness by more than double in study A (P ≤ 0.015), and by 62% in study B (P = 0.014). No significant change with auditory cues. • Gaze speed: No significant change.
Peláez-Coca et al.67 2009	• Processor: FPGA-based board (Field programmable gate array) • Camera: PC180XS, 1/3” CCD, with microlens for 78° H • HMD: Nomad 200D, monocular, transparent, FoV: 13° H	Minified edge image of the view of the environment is superimposed onto the central field of view.	• 8 LV participants • Age: 22-53, mean = 37 • Gender: 3M, 5F • Disease: 8 RP	• Observational study. • Within-subject design. • Orientation study. • Clinical vision test while using the HMD. • Outcome measures: VA, CS, and VF.	• VF expanded by a factor between 3 and 4 • CS: no significant change • VA: no significant change
Guo et al.702009	Custom device• Processor: FPGA-based board • Camera: PC182XS, Ex-View CCD, FoV: 80° • HMD: monocular, opaque, FoV: 16°H x 12°V	Superimpose the 5x minified view of the grayscale or edge images of the wider scene into the visible part of the retina. And the see-through view of the world is blocked.	• 7 LV participants for both intended walk and simulated walk • Age: 40-60 • Sex: 7M • Disease: 7 tunnel vision due to RP and CHM • 12 NV participants for simulated walk • Age: 22-58 • Sex: 6M, 6F • 12 NV participants for intended walk • Age: 24-62 • Sex: 8M, 4F	• Observational study. • Within-subject design. • Orientation study. • Training:▪ less than 10 mins for NV participants ▪ 30 mins for LV participants. • Method: Photo realistic video of shopping mall corridor shown to simulate the environment. Combinations of two image types (grey-scale or edge image) and two image scales (5× minified (80°x60° FOV) or 1:1) were tested. Participants report yes/no if they would collide with obstacles if continue current path interpreted based on the motion cues from the simulation.▪ Simulated walk: Participants are simulated to be walking 1.5 m/s while watching the video. ▪ Intended walk: test the impact on collision judgment when participants first begin to walk, intended direction of travel indicated by a vertical line on screen. • Outcome measures: Perceived safe passage distance (PSPD): calculated by finding transition point from reports of yes to no. Uncertainty: quantified by sharpness of transition. Collision envelope (CE) size, an area where participants perceived the potential collision with obstacles, calculated by summing the PSPD for both sides.	• PSPD: significantly smaller on the viewing side with display in simulated walk (F1,17 = 14.9, P = 0.001), otherwise no significant difference. • Uncertainty: No significant difference. • CE: No significant difference.
Peli et al.69 2009	Custom device• Processor: FPGA-based board • Camera: PC182XS, Ex-View CCD, FOV: 80° • HMD: monocular, opaque, FoV: 16°H x 12°V	As above.	• 10 NV participants • Age: Unspecified. • Gender: Unspecified.	• Observational study. • Within-subject design. • Orientation study. • Training: Unspecified. • Method: Photo realistic video of shopping mall corridor shown to simulate the environment. Combinations of two image types (grey-scale or edge image) and two image scales (5× minified (80° x 60° FOV) or 1:1) were tested. Participants are simulated to be walking 1.5 m/s while watching the video. Participants report yes/no if they would collide with obstacles if continue current path interpreted based on the motion cues from the simulation. • Outcome measures: Perceived safe passage distance (PSPD): calculated by finding transition point from reports of yes to no. Uncertainty: quantified by sharpness of transition.	• PSPD: A small increase in 18% on the side the camera is worn (p=0.004), but not on the display side. • Uncertainty: No significant change. • Minified images didn't seem to have much impact on collision judgment.
Yitzhaky et al.72 2010	• Exact setup unidentified.	Superimposed the minified environment represented in different types of edges (single-level black or white contour, double-line black or white contour, adaptive contour colour that provides highest contrast based on the background image, original color contour) using different edge detection algorithms (Canny, Gergholm, Prewit, Peli) onto the central part of visual field.	• 48 NV participants. • Age: unspecified. • Gender: unspecified. • Simulated tunnel vision by truncating the central part of scene view.	• Between-participants design. • Orientation study. • Method: Shown two videos (with variations in edge types) via VST simulating walking, taken inside home and outside environment. • Outcome measures: Participants asked to recognize objects, human, and to identify person and obstacles.	• Colour representations performed better outdoor. • Canny edge detector best for indoor obstacle recognition.
	• Simulated transparent HMD with 20°H FOV.			• Between-participants design. • Orientation study. • Method: Shown 128 images with different types of contour images. • Outcome measures: Object identification performance.	• 80.6% - objects found out of which (70% certain identification) • Difference between different edge detection methods is small. • Color contours yields best search performance.
Ikeda et al.68 2014	Custom device• Processor: unspecified. • Camera: WAT704-R, CCD, FoV: 53°H x 40°V, Sensitivity: 0.08lux. • HMD: monocular, transparent, FoV: 12.4°H x 9.5°V, 3 levels of brightness: 300, 150, and 75 cd/m2	Overlay 4x minified black and white image of the environment at the centre of the field of view.	• 8 LV participants • Age: 48-78, mean = 59.75 • Sex: 2M, 6F • Disease: 8 RP	• Observational. • Within-subject design. • Mobility study. • Training: 2 practise walks in the mobility course with the device. • Mobility course: 17x16m hall, illuminated at 1.2 and 0.2 lux at brightest and dimmest points. Four 1.5m wide gates and black square carpet pieces, white poles, red and white traffic cones and cardboard boxes to represent obstacles. • Method: Traverse the mobility course in darkened condition. • Outcome measures: No. of collisions and time taken.	• Number of collisions: without glasses, at least one collision in all participants. with glasses, no collision for participants except one participant who had one collision (p< 0.05). • Walking time: No change in walking time.

Minification has been used to expand of the VF of participants with RP and PFL by 2 to 4 times.,, Overlaying a minified contour image, using a multiplexing approach, did not reduce VA and CS in one study but did in another where the minified image was a grayscale.

Comparing performance with and without HMDs, minification using grayscale images did not significantly improve mobility in an obstacle-free course under very dim conditions (< 0.1 lux) in people with night blindness and PFL. However, in obstacle-filled courses at 16 lux and 2 lux, HMD use significantly decreased PPWS for the same participants and increased numbers of obstacle contacts. Conversely, using four times minified black and white image overlays, another study observed a significant reduction in the number of collisions without a reduction in walking speed in eight people with PFL and night-blindness caused by RP compared to the performance without the HMD. This study did not provide clinical information about the participants, and the apparent contradiction could result from differences in participants or differences in the complexity of the experimental environments.

Although it was expected that overlaid minified images in the central FOV would be a source of distraction during walking and reduce collision judgment performance, no significant change was observed between the performance with and without the HMD., However, it is unclear how applicable these results are in the real-world as the studies were conducted in a virtual environment that simulated walking through a supermarket corridor without requiring participants to walk physically.

In comparison with no enhancement, minification with contour images also improved visual search performance when the participants’ original VF was not too limited and auditory cues decreased search time for all the participants by 54% on average. Of 12 participants with PFL, those with a VF ≥ 10 degrees experienced a 22% reduction in search time when identifying low-contrast letters using minification. However, minification had a significant adverse effect on search time (177% increase) for those with VF < 10 degrees. Visual search task performance was better using minified images that were based on colored contours rather than black and white contours. These studies showed that minification techniques could also improve visual scanning, which is important for obstacle detection. The experiments in these studies used widely differing outcome measures, from VF size assessment to visual search and mobility course performance. Although these various measurements may be important predictors of mobility performance, more extensive experiments on actual mobility performance should be carried out before judging the usability of the minification technique for mobility.

If HMDs can provide VF expansion using minification in a way that does not compromise residual VA or CS, there is potential to improve mobility for people with PFL.

Other Alternative Visual Enhancements

This section will discuss the studies, summarized in Appendix Table A3, that used alternative visual enhancements, such as edge enhancement, digital zooming, and visual cues to direct the users’ attention to an area of interest.

Table A3.

Summary of Various Other Visual Enhancements

Author, Year	Technology	Intervention	Participants	Study design	Outcomes
Lin et al.812003	Custom device.• Processor: unspecified. • Camera: Teli CM3710 monochrome CCD, FoV:43°H x 34°V • HMD: Monocular, transparent, FOV: 40° x 30°, retinal scanning display (Laser beams scanned onto the retina of the right eye.)	Provided different view modes: • A (Augmented) – See through with superimposed images, no occlusion. • CO (Centre Occluded) – Periphery unobstructed. • CPO (Center and Periphery Occluded) – Camera view only. • PO (Periphery Occluded) – Augmented central view.	• 5 NV (correctable to normal vision) • Age: unspecified. • Gender: Unspecified. • Simulated low vision (20/200 or worse) by not using visual corrections.	• Observational. • Within-subject design. • Orientation study. • Training: 3 practice trials. • Method: Test scanning ability at 3 different contrast levels (81.4%, 61.8%, 38%) for different view modes. Used 32 different figurines. Target figurine is projected onto the screen until the participant recognises it. Then the target disappears, and participant closes their eyes with the gaze fixed at the same location. Then distractors along with the target is projected onto the screen and participant tries to identify the target figurine. • Outcome measures: scan times.	• Scan time increased in all view modes as the contrast decreased. • Best scanning performance when the peripheral is occulated. • Worse performance when the central vision is occulated.
Jang et al.50 2004	Custom device.• Processor: micro-controller AT89C2051 • Camera: Unspecified. • HMD: Binocular, occulated, 0.44” color TFT LCD, FOV: adjustable (using magnifying lens on the images on LCD).	Various image enhancements• Image magnification • visual axis control by refracting light onto the normal-functioning part of the retina • improves contrast by changing the brightness of red and blue colors • light intensity adjustment • focal length adjustment	• 21 LV participants • Age: mean = 33.6 • Gender: Unspecified. • Disease:▪ 29% optic atrophy ▪ 19% RP ▪ 14% congenital cataract ▪ 14% macular degeneration	• Clinical trial. • Longitudinal study (7 months). • Within-subject design. • Orientation study. • Method: Prescribe the devices to participants for 7 months. • Outcome measures: VA	• Distant VA improved to better than 20/25 (71%, 15 people) from best VA of (20/200 – 20/100 in 42%, 9 people) before prescription. • 57% commonly used this for reading. • Visually impaired person who was not able to walk independently was able to read TV caption after 1 month of usage, and 3 months later, driving was possible. Unclear if the improvement was due to the glasses or surgical operations. • Reported outcomes of the trial briefly.
Pela´ez-Coca et al.67 2009	• Processor: FPGA-based board (Field programmable gate array) • Camera: PC180XS, 1/3” CCD, with microlens for 78°H • HMD: Nomad 200D, monocular, transparent, FoV: 13° H	Provides different types of digital zooming (x2, x4, and x8) (bilinear interpolation, bicubic interpolation and pixel replication)	• 6 LV participants • Age: Unspecified. • Gender: Unspecified. • 6 NV participants with diffusors. • Age: Unspecified. • Gender: Unspecified.	• Observational. • Within-subject design. • Orientation study. • Method: Clinical measurement of VA with and without device. • Outcome measures: VA.	VA improved proportionately with the magnification level.
Luo et al.73 2011	Custom device.• Processor: Unspecified. • Camera: Unspecified. • HMD: Binocular, transparent (LCD shutters provided on-demand occlusion), FOV: unspecified.	Provides two image enhancements:• Electronic magnification – magnify (2x to 16x) and occlude the display for better contrast sensitivity. • Wideband enhancement – bright outlines of the scene superimposed onto the natural see-through view.	• 3 LV participants • Age: Unspecified. • Gender: Unspecified.	• Observational. • Within-subject design. • Orientation and mobility study. • Training: Unspecified. • Method: Clinical measurements and visual search and mobility test (Located and moved to targets on the walls in a large room (17’ x 27’)). • Outcome measures: VA, CS, and search time and no. of attempts of walking to targets.	• Number of attempts of walking to targets reduced (4.3 to 1.6 on average). • Search Time: reduced in only one patient. • VA and CS improved greatly.
Hwang et al.60 2014	Google Glass• Processor: Built-in processor. • Camera: Built-in camera, FoV: 75.7°H x 58.3°V • HMD: Monocular (right-eye), transparent, FOV: 13°H x 7.3°V	Edges of the objects are overlaid in real-time over the natural view of the world.	• 3 NV participants • Age: 25-45 • Gender: Unspecified • Simulated low vision with diffusor film.	• Observational/Explanatory study. • Within-subject design. • Orientation study. • Method: Measured CS of participants w/ and w/o edge enhancement and w/ and w/o light diffuse filter.	• CS improved to 1.5 log contrast, but no further. • With maximum edge enhancement setting in low contrast scene, strong effect of the noise observed.
Zhao et al.74 2015	Oculus Rift DK2• Processor: Unspecified laptop. • Camera: WideCam F100, FoV 120°H • HMD: Binocular, occulated display, FoV: ∼90°H	Different visual enhancements:• Customisable magnification (up to 35x) • contrast enhancement • edge enhancement • black/white reversal • text extraction via the video see-through display in full display mode and window display mode.	• 19 LV participants • Age: 21-68, mean = 46 • Gender: 6M, 13F • Diseases: Varied (including RP, Nystagmus, Usher's Syndrome, Stargaardt's Disease, Glaucoma, Myopia, etc.)	• Observational. • Orientation study. • Method: Reading text from handheld printed page and four printed signs hang 3 meters away to see the effects of enhancements and display modes and how they are customised. • Outcome measures: qualitative response from participants under different conditions.	• Magnification: effective for both near-distance and far-distance tasks and preferred full display mode. • Contrast enhancement: effective for people especially when they are not sensitive to light. Brightness hurt a few people in full display mode. Effective for all in window display mode. • Edge enhancement: not useful for majority as it makes the words bolder and makes them harder to distinguish. • Black-white reversal: negative effect in far-distance reading. Mixed opinions in near-distance reading. • Text extraction: improvement in far-distance task. Majority did not like it in near-distance task.
Zhao et al.59 2016	Oculus Rift DK2• Processor: Unspecified laptop. • Camera: Logitech C920 webcam • HMD: Binocular, occulated display, FoV: ∼90°H	Provided 5 types of visual cues to facilitate visual search tasks.• basic (magnification and contrast enhancement) • guideline: connect the target with red guideline connecting the centre of display and centre of object • spotlight: change the environment into grey scale • flash: flash effect added to target • movement: rotate the target • sunray: 8 red guidelines converging at the centre of the target.	• 12 LV participants • Age: 23-68, mean = 44.25 • Gender:6M, 6F • Diseases: Varied (including RP, Steven-Jonson Syndrome, Retinopathy of Prematurity, Stargardt's Disease, Nystagmus, Wet Macular Degeneration, Cataract, Retinopathy, etc.)	• Observational. • Orientation study. • Method: Provides visual cues to highlight objects on a shelf to facilitate visual search. • Outcome measures: Qualitative response from participants.	• Basic: no one preferred. • Guideline: improvement for 5 people, but tunnel vision people cannot see how to follow the line. • Spotlight: majority preferred it, but not useful for people with poor colour sensitivity. • Flash: useful for people with very poor vision. Too distracting for the rest. • Movement: not useful; distracting and can't read information on the box. • Sunray: half liked it, the rest thought it was overwhelming.
Zhao et al.77 2017	Epson Moverio BT-200• Processor: Onboard processor. • Camera: Built-in VGA camera. • HMD: binocular, transparent, FOV: ∼23°H	Shown different virtual elements in AR environment. Virtual elements included:• ETDRS R logMAR chart • Pelli-Robson chart • Shapes (circles, squares, and triangles) and texts with varying sizes, colours, thickness, and fonts.	• 20 LV participants (2 dropouts) • Age: 21-69, mean = 45 • Gender: 9M, 11F • Diseases: Varied (Glaucoma, RP, Detached retina, Cataracts, Nystagmus, etc.) • 18 NV participants • Age: 22-28, mean = 24 • Gender: 9M, 9F	• Observational. • Case-controlled study. • Orientation and mobility study. • Method:▪ Clinical vision tests using physical charts and virtual charts. ▪ Stationary setting: shown virtual shapes and texts to recognise in AR environment, ▪ Mobile setting: walked 20 feet in well-lit room while trying to recognise shapes and texts using AR glasses in conjunction with habitual mobility aids. • Outcome measures:▪ Clinical tests: VA, CS ▪ Stationary settings: collected participants’ preferences on sizes, colors, thickness and font types to be used for virtual elements. ▪ Mobile settings: walking times.	• Clinical tests:▪ VA reduced significantly in control group when looking at virtual eye chart (p< 0.001), no significant change in people with LV ▪ CS improved significantly in control group in virtual chart (p < 0.001), CS varied among LV participants. • Stationary setting:▪ prefer black background ▪ Yellow and white colours were best ▪ 7 LV liked thick borders; others didn't think thickness was important ▪ Text: mixed results from thickness, more readable in sans-serif when the text is small • Mobile setting: Both test and control groups walked significantly slower with AR glasses on. Recognising shapes and texts in AR environment had negative impact on people with LV's walking speed.
Bai et al.76 2017	Custom device.• Processor: unspecified microprogrammed control unit. • Camera: RGB-D depth camera. • Other sensor: ultrasonic depth sensor. • Display: Binocular, transparent, FOV: unspecified.	• Depth images were displayed in the HMD with a rectangle representing the destination. The possible path or direction to follow to reach the destination is represented by a circle, if the feasible path is straight forward, no circle is show, else, circle displayed to be followed. • Audio tones, instructions, and beeps were present in addition to the visual cues using earphone.	2 groups.• 10 participants (totally blind) • Age: Unspecified • Gender: Unspecified • Diseases: Unspecified. • 10 LV participants • Age: Unspecified • Gender: Unspecified • Diseases: 10 amblyopia	• Observational. • Within-subject design. • Mobility study. • Training: None • Mobility course: Traversed 3 courses;▪ Home: 40 m, 10 obstacles (5 cm to 1 m heights) ▪ Office: 150 m, 15 obstacles. ▪ Supermarket: 1 km, 15 obstacles. • Method: Visual cue + audio feedback was given to people who are completely blind and people with LV get only visual cue. • Outcome measures: Time taken and no. of collisions with obstacles.	• Blind group: walking time with tones and beeps were almost the same as using cane in home and office environments. The device is more efficient in supermarket environment. • Low vision group: walking efficiency improved in office and supermarket. No collision in any environment with the glasses, as opposed to high numbers of collisions in home, office, and supermarket (12, 28, 20 respectively).
Bai et al.75 2018	Custom device.• Processor: unspecified microprogrammed control unit. • Camera: RGB-D depth camera. • Other sensor: ultrasonic depth sensor. Display: Binocular, transparent, FOV: unspecified.	• Depth images were displayed in the HMD with a rectangle representing the destination. The possible path or direction to follow to reach the destination is represented by a circle, if the feasible path is straight forward, no circle is show, else, circle displayed to be followed. • Audio tones, instructions, and beeps were present in addition to the visual cues using earphone.	• 1 participant with total blindness • 1 participant with LV • Age: Unspecified • Gender: Unspecified • Diseases: Unspecified	• Observational • Mobility study. • Mobility course: unspecified size. • Method: Traverse the mobility course with and without obstacles by following the path generated by the glasses. • Outcome measures: deviation distances from the planned path.	• Without obstacles: max deviation distances are less than 1m, and average is less than 0.3m. • With obstacles: partially sighted participants have more deviation compared to the other group as they also relied on their own judgement based on visual information available.

The contrast of visual scenes can be improved by superimposing bright outlines or edges of the objects in the scene in real time., These contrast enhancements resulted in significant improvements in the CS of participants with CVL and in normal participants using diffusor films. Providing dynamic magnification also improved VA and CS measurements proportionate to the magnification level., These enhancements were useful in improving orientation and mobility performance (e.g. locating and moving to objects in a large room and reading signs from a distance).

Jang et al. also reported improved near and distant VA measurements and claimed that one participant went from being not able to walk independently to being able to drive after 3 months of using the device. However, the experimental outcomes were not clearly presented, and it was unclear if this improvement was because of using HMD alone or the result of cataract surgery.

HMDs can also be used to generate visual cues that can improve visual scanning performance. In comparison to using standard glasses with an unenhanced visual scene, an HMD using algorithms that can recognize objects and superimpose visual cues, such as flashing lights, to highlight or to direct the user's gaze, can reduce search times by approximately 46% while simultaneously improving search accuracy from approximately 93% to 100%. In two further studies,, an HMD that detects obstacles and identifies a safe path used visual cues as a directional indicator and/or various audio messages to aid navigation in indoor environments. Using this device, participants with amblyopia showed improved performance in terms of time taken and number of collisions, especially in unfamiliar settings. Although these visual cues showed potential, their success is contingent on the performance of the underlying computer vision algorithms.

Another study showed that having to recognize virtual elements in a scene, such as shapes and text in AR environments, significantly reduced walking speed for 18 participants with LV and 18 normal vision controls. Walking time for participants with LV increased by approximately 12% and approximately 10% when they were viewing text or shapes respectively. This study highlights the need to understand what information improves safe mobility without negatively impacting on efficiency.

The enhancements used in these studies show that visual function parameters, such as VA, CS, and scanning ability, can be usefully enhanced to improve mobility.

Off-The-Shelf Devices

This section discusses the studies, summarized in Appendix Table A4, that explored the effects of off-the-shelf devices on visual function. All these devices provided general vision enhancements, such as variable image magnification, contrast enhancement, and color reversal.

Table A4.

Summary of Studies Using OTS Devices

Author, Year	Technology	Intervention	Participants	Study design	Outcomes
Geruschat et al.79 1999	3 different HMDs.• V-max▪ FOV: ∼45°, zoom: 19x • LVES▪ FOV: ∼45°, zoom: 9-10x • Zoomer▪ FOV: ∼25°, zoom: 12.5x	Provided the devices with varying sizes of field of view and magnification	• 10 LV participants • Age: 12-21, mean = 17 • Gender: 5M, 5F • Disease: various including optic nerve hypoplasia, optic atrophy, glaucoma, cataracts, aphakia, cone dystrophy, corneal leukomas, retinal detachments, and retinopathy of prematurity	• Observational • Within-subject design. • Orientation and mobility study. • Method: introduced 3 HMDs and allowed participants to take one device to his/her room to use a few hours or a few hours each day for 3 days. Clinical vision tests under different conditions (with and without HMDs, non-magnified and magnified with HMDs) and qualitative feedback regarding the HMDs and uses. • Outcome measures: VA, CS. Informal and individualized functional assessments: reading, viewing chalkboards, viewing at distance, mobility in different light conditions, etc.	• VA: HMDs without magnification improved VA compared to no HMD. (F (3,33) = 2.92, P = 0.052). Compared to no HMD, VA improved significantly when HMDs with magnification, F (3, 37) = 240, P = 0.0000 • CS: Compared to no HMD, contrast sensitivity improved when using the devices without magnification F (3, 37) = 6.59, P = 0.002. However, V-max and Zoomer performed worse than unaided contrast sensitivity. Using the devices with magnification improved contrast sensitivity significantly F (3,37) = 11.19, P = 0.0000. • Functional assessments: zooming in and out quickly is useful seeing faces, and looking out windows, etc. mobility improved due to improved visual acuity.
Weckerle et al.78 2000	Low vision enhancement system (LVES)• Camera: 3 black-and white cameras • HMD: Binocular, opaque, FoV: 60H°x40V°	Introduced the HMD device with dynamic magnification ranging 1.5x to 12x	• 17 LV participants • Age: 18-95, mean = 49 ± 21 years • Gender: 16M, 1F • Disease:▪ 4 macular degeneration, ▪ 2 central areolar atrophy ▪ 4 Stargardt's disease ▪ 1 Juv. x-chrom. Retinoschisis ▪ 1 Achromatopsa ▪ 1 Myopic maculopathy ▪ 1 Leber's hereditary optic neuropathy ▪ 1 Anterior ischemic optic neuropathy ▪ 1 Grönblad-Strandberg syndrome ▪ 1 Congenital stationary night blindness	• Observational study. • Mobility study. • Mobility course: details unspecified (in clinic and on the street) • Method: assessment in semiquantitative manner for:▪ Reading – real newspaper ▪ Writing- three short sentences ▪ Mobility – walking with contrast enhancement and without magnification • Outcome measures: Semi-quantitative and qualitative measures:▪ Reading: number of words per minute ▪ Writing: evaluated by legibility ▪ Mobility: qualitative assessment.	• Reading: couldn't read if magnification required is more than 8x (three participants). The rest read fluently and without mistakes. • Writing: 6 legible, 7 shaky and 1 illegible handwriting. • Mobility: majority had minor to no problem while walking. Only 3 had problems with one of them very insecure while walking on flat surface.
Culham et al.51 2004	4 devices.• Flipperport▪ FOV: unspecified. 6-30x magnification • Jordy▪ FOV: Unspecified. 1-24x magnification. • Maxport▪ FOV: Unspecified, 16-28 magnification. • NuVision▪ FOV: Unspecified, 5-10 magnification.	Provided the commercially available devices with varying level of magnification to participants.	• 20 LV participants • Age: 21 – 82 • Gender: 9M, 11F • Disease:▪ 10 early onset macular degeneration (EOMD), ▪ 10 AMD	• Clinical trial. • Within-group design • Orientation study. • Method: Perform clinical vision tests, reading (passages of texts in different sizes) and writing (check) tests, and visual search tests (locating two random grocery items on a shelf), and compared the results between two groups (AMD, and EOMD) and between devices and spectacles • Outcome measures:▪ Clinical vision tests: VA, CS ▪ Reading: words/min ▪ Writing: time taken ▪ Search test: time taken	• Clinical vision tests:▪ VA – Using HMDs is significantly worse than optical devices for all distances. AMD group performed better in near VA test. ▪ CS - No significant change in contrast sensitivity. • Reading: EMOD patients tended to read N5 texts with HMDs better than with optical devices, AMD patients did not show this effect. HMDs caused significantly slower reading speed with N10, and N20 in both groups of patients. • Check writing: Significantly slower with NuVision, no change otherwise. • Visual search: No differences.
Zebehazy et al.52 2005	MultiVision• Camera: Unspecified. • HMD: Binocular, opaque, FOV: 32°H x24°V	Introduced MultiVision device	• 2 LV participants • Sex: Unspecified • Gender: Unspecified • Disease: 2 RP	• Observational. • Within-subject design. • Mobility study. • Training: 40 mins • Mobility course: two courses (median 15 lux and median 1.5 lux) of 8-9 short city blocks with natural occurring obstacles. • Method:▪ Mobility: traverse mobility courses with and without HMDs. ▪ Object recognition: search 5 objects of varying sizes and contrasts placed on the mobility course while walking and stop each time an object is identified. • Outcome measures:▪ Mobility performance scored by O&M specialist. Measured time to complete, obstacle contacts, cane contacts, and mobility errors (high stepping, walking off from path, etc.) ▪ Object recognition: distance from which the objects are cognised.	• Mobility:▪ In high-light course, no major difference in mobility performance w/ or w/o device. ▪ In low-light course, w/ device produced significant decrease in performance • Object recognition better with devices.
Wittich et al.49 2018	eSightHMD: Binocular, opaque, FOV: 32°H	Provided the HMD to participants. HMD provides magnification (1.3 – 12.3x), contrast enhancement, binarization,	• 74LV • Age: 13-75 • Gender: Unspecified. • Diseases: Unspecified.	• Clinical trial. • Longitudinal study (3-month follow-up) • Within-subject design. • Orientation and mobility study. • Method: Provided the glasses for daily use for 3 months. Compared the results without glasses, with glasses and with glasses 3 months later. • Outcome measures:▪ Clinical vision tests: CS, VA, reading speed, and critical print size. ▪ Veterans Affairs Low Vision Visual Functioning Questionnaire ▪ Face recognition test ▪ Subset of function items from the Melbourne Low Vision Activities of Daily Living Index	• Reading performance (near and far) improved once the glasses are introduced, no change after 3 months. • Contrast sensitivity improved when glasses are introduced, but no change after 3 months. • Face perception improved immediately, but no change after 3 months. • Melbourne LVADL improved immediately, no change after 3 months. • Veterans Affairs Low Vision Visual Functioning Questionnaire improved▪ Overall 0.84, P = 0.001 ▪ reading 2.75, P < 0.001 ▪ mobility, 0.04, no statistical significance ▪ visual information, 1.08, P < 0.001 ▪ visual motor, 0.48, P = 0.2

Two studies,, using some of the world's foremost head-mounted low vision devices, recorded improvements in VA and CS and near-distance task performance, such as reading and writing. Despite the limited functionality and bulkiness of the devices, self-reported mobility performance decreased only in a minority of participants with the rest having better mobility performance due to increased VA.

Compared to using conventional optical devices, Culham et al. (2004) used four commercially available devices (Flipperport, Jordy, Maxport, and NuVision) and found reduced performance in near-distant and far-distant tasks, including a visual search test. Flipperport, Maxport, and NuVision devices would not have been suitable for mobility tasks as Flipperport and Maxport are table-mounted and handheld camera systems, respectively, and NuVision lacked auto-focus functionality. Jordy, which produced a comparable performance to optical aids for visual search task, could be useful as an orientation aid. However, a study with a device called eSight recorded instant performance increases in clinical vision tests and the Melbourne ADL score, but this did not improve further after 3 months of home use. After 3 months, the mobility subscale from the Veterans Affairs Low Vision Visual Functioning Questionnaire (VA LV VFQ) did not improve. However, other subscales (i.e. reading, visual information, and visual motor) did improve significantly, suggesting the device may be useful for orientation.

A study assessing the usefulness of the MultiVision night vision device, in comparison with no device, showed reduced mobility performance as scored by an orientation and mobility expert based on the number of cane contacts, body contacts, and mobility errors while walking around poorly lit (14.5 lux and 2.5 lux) city blocks. Only two participants with night blindness took part in this study and, therefore, these results should be treated with caution.

Many of the studies that used off-the-shelf devices used devices which are now 10 to 15 years old. The relatively poor performance recorded in these studies suggests why these visual aids have not been adopted more widely.

Human Computer Interaction Aspects

This section will describe how the HMDs were perceived in terms of ergonomics and performance. The usability of devices will play a significant role in determining the success or otherwise of HMD-mobility aids, and yet most of the studies did not mention human computer interaction (HCI) aspects.

Color-based scene simplification may be problematic for people with poor color perception. Additionally, they remove potentially useful visual cues, for example, surface texture, original colors, and shadows. Minification techniques require users to have good central vision and, therefore, are unlikely to be useful for people with CVL. Additionally, this technique may also cause similar problems to those caused by prismatic field expansion (i.e. confusion due to different overlapping views and double vision). However, there is potential to improve usability of these minification techniques with long-term usage. For example, a study with 4 participants with RP who used a minification enhancement for 2 weeks at home showed improved orientation and mobility skills to a limited extent. Other techniques, such as magnification, edge enhancement, and visual cues could be useful for people with LV regardless of the type of field loss. Due to the various limitations discussed previously, HMDs remain impractical as independent mobility aids. However, they may be able to supplement current mobility aids by enabling people with LV to access information beyond that is available by long canes or guide dogs.

In the experiments with older and bulkier devices, some participants rejected them due to their fear of drawing unwanted attention to themselves and feeling different from others. One common positive that stood out across all the studies over two decades was the customization provided by HMDs ranging from older HMDs’ dynamic magnification to newer ones’ ability to combine different enhancements, highlighting the different needs of different individuals.

OSTs made it harder for some participants (both CVL and PFL) to recognize shapes and text when the visual augmentations and environment had similar background colours. Participants with night blindness and PFL also did not like having transparency in poorly lit situations; however, they did not feel the small screen size of HMD to be an issue. Participants with PFL reported the need for color displays at pedestrian crossings with signal lights, which were not distinguishable by shape.

In the study with the HoloLens that color-coded object distance, only one participant with RP and PFL found it to be helpful at night. The form factor of HoloLens along with display lag was also reported to be unhelpful.

Lack of familiarity with new technologies and visual enhancements could also affect performance. However, the majority of the studies described here did not specify how much time was given for training. When training information was included, training time appeared to be rather limited, ranging from 2 to 3 minutes to 30 minutes. The impact of extended continuous usage of HMDs could be a critical ergonomic issue, but it has not yet been evaluated. Some experiments took up to 90 minutes to complete, but it was not clear if breaks were factored in the experimental design and so fatigue may be another experimental variable.

Discussion

Only 10 of the studies reviewed involved a quantitative assessment of mobility performance and of these, 8 of them–,–, recorded a significant reduction in efficiency with vision enhancement. Of the other two studies, which observed similar or better efficiency, in both of them,, the mobility courses were relatively simple with large walkable areas and a small number of relatively large obstacles. Therefore, currently, there is no good evidence that vision enhancements using HMDs improve walking efficiency. However, vision enhancements, such as simplification and minification, did seem to help with obstacle detection and collision avoidance. This could result from the benefits of the enhancement, such as improved CS and VF but also could be due to other factors, such as increased participant concentration, the relatively large obstacles used in the experiments, or the participants merely taking their time to complete the mobility course safely. Although the effect on mobility has not been tested thoroughly, the minification technique proved to be useful in expanding VF, and improving visual search performance, in people with RP.

Typically, the experiments involved only a small number of people with LV (the mean sample size in all studies was just 16, and the mean sample size of participants with LV was 10). Some studies simulated LV in people with normal vision using occlusion foils, diffuser films,, or by avoiding using optical corrections to reduce VA and CS and simulate tunnel vision. Although these low vision simulations can be useful, they cannot simulate other visual problems, such as light sensitivity, color vision impairment, or patchy vision. Besides, many visual problems progress slowly over time, so people with LV may adapt to make use of less visual information and rely more heavily on other senses. In contrast, participants with simulated LV have not had the same chance to adapt to any visual deficiencies. Only two studies, involved mobility testing in the real world, and the experimental setups varied widely between studies. Often, the mobility courses did not reflect real-life environments. That is, they used large, high-contrast obstacles, large walkable areas, and did not include surface level changes, light level changes, or dynamically moving obstacles and stairs.–,, Therefore, it is difficult to understand how the results might apply in real-world situations. Another limitation of the existing literature is the failure to compare mobility performance with HMDs with that achieved using the person's usual mobility aids. Most of the mobility experiments used unaided mobility performance as the comparator. Only two studies compared the performance of the technology against that achieved with a cane., Hence, most studies have failed to assess the real-world benefit of HMDs. Measuring mobility performance using HMDs in conjunction with habitual aids, as three studies did,,, could also be useful to understand how HMD technology can augment existing aids. The familiarization period was not specified in most of the studies and was also limited when mentioned, ranging from 2 to 3 minutes to 30 minutes. The resulting lack of familiarity with the technology may also have had a significant impact on measured performance.

Another challenge in this area of research is how success is measured. Whereas time taken, number of contacts with obstacles, PPWS, and raw walking speed were mainly used as outcome measures,–,, other studies used some extrinsic measures, like confidence, hesitation, and perceived safe passage distance., Outcome measures related to visual search tasks included gaze efficiency, gaze directness, gaze path, and time taken to find the item. However, more evidence confirming the validity of these secondary measures is required. It would be instructive to find out what people with LV think are important mobility-related outcomes. Due to the use of different outcome measures and mobility courses, it is difficult to compare study results.

Although the observational, cross-sectional, within-subject study design is sufficient for preliminary studies, ultimately, future research should strive to evaluate new technologies using the randomized control trial study design. Future studies should assess the impact of HMDs on both broad classes of vision loss (i.e. PFL and CVL) using participants with low vision. Studies should also have sufficient power (sample size) to deliver meaningful results. Future studies should describe the biographical characteristics of the cohort in terms of VA, CS, VF, and underlying visual problems, and the adaptation time with the HMDs as these parameters have the potential to impact the outcomes of the studies significantly.

Future studies should consider measuring mobility performance using PPWS, a widely adopted outcome measure in mobility experiments, to enable comparison between different studies and different rehabilitation methods. Mobility performance with HMDs should be compared against the mobility performance with habitual aids and against the performance with the HMDs in conjunction with the habitual aids to better understand the benefits of HMDs and how they can supplement existing mobility aids. The mobility courses should also reflect the challenging aspects of the real-world environment experienced by people with LV, for example, variations in surface level, different sizes of obstacles at different heights, different light levels, and stairs. Currently, there is no well-established method to measure orientation performance. Nonetheless, studies should report how the HMDs affect VA, CS, VF, and the perception of real-world targets. For successful long-term adoption of HMDs, it is also essential to understand how acceptable the devices are to people with low vision. Therefore, it is of value to report subjective feedback regarding usability issues, for example, comfort, ease of use, confidence in using the HMD, and incorporate results from validated questionnaires (e.g. the mobility subscale from the Impact of Vision Impairment [IVI] questionnaire) where applicable, in addition to the quantitative results.

Future Directions and Conclusion

HMD technology offers an unprecedented opportunity to enhance vision in people with LV. However, to realize the full potential of this technology as a mobility aid, some fundamental questions need to be answered. For example, how does the user's underlying visual condition impact on the value of different image enhancements? Which enhancements are best suited to those with CVL, PVL, reduced CS, and reduced acuity? In which environmental conditions can HMDs offer an advantage? Are they more helpful in high or low luminance conditions, and is this dependent on the underlying eye condition? It may also be helpful to understand how the users’ age, gender, and familiarity with technology affects mobility performance when using HMDs. From this perspective, HMD-based enhancements can be tailored to the needs of specific users. Additionally, it would be helpful to know how long it takes for people with LV to adapt to new HMD technology.

With powerful HMDs becoming increasingly available, researchers in this area can focus on implementing and optimizing various image processing algorithms. Enhancements may adopt the traditional approach of making things bigger, bolder, and brighter. They can also offer different types of information that transcend what is possible with traditional low vision aids — for example, providing distance information, identifying objects, and providing a safe route to follow to reach the desired destination. Another challenge is designing efficient and intuitive augmented environments. As an extensive amount of customization will be required to adapt these enhancements to an individual's needs, it will be essential to identify a suitable means of achieving this. Possible input methods to explore range from speech commands to hand-tracking, gaze input, and gesture recognition. It is also possible to leverage the power of machine learning to understand the adjustments users would like to make based on their preferences and the environment. There is also room to improve accuracy, efficiency, and robustness of machine learning techniques in computer vision, such as SLAM techniques, to understand the environment and to generate a safe path to follow, segmentation techniques to highlight stairs or surface-level changes, and image style transformation techniques to maximize contrast and perceivability.

In addition, electronic vision aids could modify the enhancement dynamically on the basis of user commands (gestures or speech) or the users’ needs. For example, by zooming out to increase the FOV before a person moves, displaying visual cues to aid obstacle avoidance, and identifying a safe path. It would be useful to understand how enhancements can be used together and what are the best transition methods to switch functionality seamlessly to optimize mobility for people with LV.

With all these possible enhancements mentioned above, HMD-based AR technologies hold great potential as future mobility aids to provide accessible and essential information via different mediums (visual and spatial audio) as standalone aids or as complementary aids to existing mobility aids.

In conclusion, although current studies show improved visual scanning performance, object recognition, and obstacle avoidance with HMDs, there is currently no evidence that this translates to improved mobility performance. Nonetheless, there appears to be significant potential. HMD technology is advancing rapidly, and this combined with a better understanding of what enhancements work for people with low vision should lead to improved mobility.