Assistive technology solutions for aiding travel of pedestrians with visual impairment.

This work systematically reviews the assistive technology solutions for pedestrians with visual impairment and reveals that most of the existing solutions address a specific part of the travel problem. Technology-centered approach with limited focus on the user needs is one of the major concerns in the design of most of the systems. State-of-the-art sensor technology and processing techniques are being used to capture details of the surrounding environment. The real challenge is in conveying this information in a simplified and understandable form especially when the alternate senses of hearing, touch, and smell have much lesser perception bandwidth than that of vision. A lot of systems are at prototyping stages and need to be evaluated and validated by the real users. Conveying the required information promptly through the preferred interface to ensure safety, orientation, and independent mobility is still an unresolved problem. Based on observations and detailed review of available literature, the authors proposed that holistic solutions need to be developed with the close involvement of users from the initial to the final validation stages. Analysis reveals that several factors need serious consideration in the design of such assistive technology solutions.

Introduction

Traveling is an integral part of everyone's life. We travel for education, work, daily living, social integration, and various other reasons. In the absence of vision, a person with blindness has to rely on alternate senses of touch, smell, and hearing for accomplishing the travel task. The absence of visual cues and limited perception bandwidth of alternate senses pose serious challenges that severely affect independent travel and safety which in turn affects their quality of life.

Travel can be defined as a combination of mobility and environmental access. Mobility involves avoidance of obstacles, orientation in the environment, and navigation. Environmental access involves minimization of hazards and access to signs and related information. According to Brambring's model for locomotion of blind, the mobility for a blind person involves perception of obstacles, landmarks, and orientation.[1]

Persons with blindness primarily rely on sighted guidance and traditional travel and navigation aids: long (white) canes and guide dogs for overcoming the mobility challenges. Use of guide dogs is not universal and is limited mainly to developed countries. Long canes are the most popular and commonly used assistive device around the world. It increases the detection range to nearly a meter and provides rich information about the ground-level obstacles and the surfaces. However, physical touch by cane often results in embarrassing situations.[2] Effective cane usage involves learning of orientation and mobility techniques. There are many limitations of the long cane like uncertainty in discrimination and protection against drop-offs, limited preview range, danger of tripping pedestrians in crowded places, and no protection of the upper part of the body from the hanging or protruding obstacles which do not have significant footprint on the ground.[3] It is reported that walking without sight poses risk of head-level and fall accidents that often require medical attention. These accidents reduced the confidence and forced a substantial number of respondents to change their mobility habits.[4]

The limitations of traditional assistive solutions led to the development of electronic assistive travel aids, for the purpose of detection of objects as well as guiding the user by providing the required orientation and navigation information. Such solutions would improve the detection range of obstacles, objects, landmarks, etc. and would help in better orientation in the environment which in turn lead to less stressful, more safe, comfortable, and simplified navigation task.[5] According to Wiener et al.,[3] an assistive technology solution must try to address the following identified needs of pedestrians with blindness:

Detection of obstacles in the travel path from ground level to head height for the full body width

Travel surface information including textures and discontinuities

Detection of objects bordering the travel path for shore-lining and projection

Distant object and cardinal direction information for projection of a straight line

Landmark location and identification information

Information enabling self-familiarization and mental mapping of an environment

Most of the existing assistive solutions fall either in the category of obstacle avoidance or orientation and navigation devices, while trying to solve some of the individual problems related to the travel task. Integrated approach in developing assistive technology solutions has been proposed recently and few such systems have been implemented.[1] Based on this broad classification, the existing obstacle detection systems and navigation systems have been reviewed in “Obstacle detection systems” and “Navigation systems” sections, respectively. Subsequent section discusses the pros and cons in the existing solutions and various factors that need consideration in their design. The final section concludes the work.

Obstacle detection systems

Obstacle detection systems make use of laser, infrared, ultrasonic sensors, cameras, etc. for detection of objects in their field and convey this information to the user through the haptics, audio or tactile interfaces. Dozens of devices have been developed over the last several decades. Many of them are now discontinued. They include devices such as Sonic guide, Path sounder, Mowat sensor, Sonic Pathfinder, etc. Very few devices have continuously evolved and are commercially available.

Miniguide,[6] RAY,[7] and PalmSonar[8] are handheld ultrasonic sensor-based commercial obstacle detection systems. They convey the distance information about the nearest detected obstacle through vibration and/or audio beeps by varying its frequency with the change in distance of the nearest detected obstacle. PalmSonar uses ultrasonic sensors with asymmetric field of view that is narrower in horizontal direction and broader in vertical direction. It helps in avoiding detection of objects on sides while travelling through narrow corridors.

K Sonar[9] is a ultrasonic sensor-based handheld device that can also be mounted on a cane. It conveys information about all the detected objects using different audio tones with varying pitches rather than the distance information about the nearest obstacle. Multiple tones are used to represent different objects in the scene and pitch is used to convey the distance information; closer the object lower the pitch corresponding to that object. Tone color is used to convey the material characteristics of the detected object. To listen to these sounds, headphones are provided with the device but they interfere with the acquisition of the acoustic environmental cues. It conveys information about the size, shape, and material of the obstacles but it may require significant amount of learning and continuous practice for interpreting and using such information. Based on individual preference, person can use it with or without the cane. However, it does not provide upper body protection, i.e. it cannot detect chest or head level objects, when mounted on cane.

Tom Pouce 1 and 2[10,11] are cane-mountable obstacle detection systems that use multiple IR transmitters with 20° and 50° horizontal and vertical angular resolution, respectively. The resolutions are such that the width of the shoulders in horizontal direction and knee to head height in the vertical direction is covered for detection of objects that cannot be detected with the white cane. Handheld version of this device is MiniTact[11] primarily designed for indoor use. Teletact[10] uses laser beam for the detection of obstacles in the range of 10 cm to 6 m. However, user needs to continuously scan the environment as the laser beam is highly directional.

The handheld devices are highly portable. They qualify as secondary mobility aids and must always be used in conjugation with the white cane to ensure that drop-offs are detected with the cane. Such devices are primarily helpful in obstacle detection and avoidance in indoor environments.

Another category of obstacle detection device is cane-mounted systems. The standard cane allows detection of obstacles up to knee and if the sensor technology allows detection of objects from knee to head height and across the body width, then the system qualifies as primary mobility aid. The UltraCane[12] comes as a single cane assembly that uses a pair of ultrasonic transceivers. Lower sensor detects the objects on the ground or in front and the upper sensor detects the head-level objects. Distance information about the detected obstacles is conveyed through two vibrating buttons. The implication is that the device must be gripped in a manner such that fingers are exactly positioned on the buttons throughout the journey.

The SmartCane™[13] device comes fitted on the top fold of the standard white cane. It also uses ultrasonic ranging for obstacle detection and distinct tactile vibratory patterns for conveying distance information. The vibrations can be felt on the entire grip that allows the user to conveniently grip the device. This device allows adjustment of the sensors at one of three positions according to height and gripping style of the user to ensure reliable detection of obstacles at different heights. Unlike most of the other devices, it comes with internal rechargeable battery and can be charged just like a mobile phone. Another high point of this technology is that it is extremely affordable, roughly 1/10th of similar technologies. At present, there are around 15,000 users of this device in India. Table 1 summarizes the commercially available obstacle detection systems.

Table 1.

Comparison of commercially available obstacle detection systems.

Obstacle detection system	Type	Information conveyed	Sensor technology	User interface	Price	Special features
Miniguide[6]	H	Distance of nearest obstacle	U/S	Vibratory/audio	$399
RAY[7]	H	Distance of nearest obstacle	U/S	Vibratory and/or audio	$300	Trainer and learner mode
Palm Sonar[8]	H	Distance of nearest obstacle	U/S	Vibratory	$1000	Sensor's asymmetric field of view (discontinued)
K sonar[9]	H/C	About all the objects in the sensor's field of view	U/S	Audio tones of varying pitch	$1085	Informs about the size, shape, and material of the objects to some extent
Tom Pouce 1 and 2[10,11]	H/C	Distance of nearest obstacle	IR	Vibratory	$700–$1000	Additional IR sensor for detection of head high obstacles in version 2
Teletact[10]	C	Distance of nearest obstacle	Laser	Vibratory and audio	$2300	Two vibrators placed near the first and the second finger used to convey distance
UltraCane[12]	C	Distance of nearest obstacle	U/S	Vibratory	$825	Informs about the height of the obstacle to some extent
SmartCane™[13]	C	Distance of nearest obstacle from knee to head level	U/S	Vibratory, some info. through audio beeps	$60	Vibrations are produced over the entire grip, sensors can be adjusted according to the height and gripping style of the user, informs user about failure of sensors, vibrator, etc.
iGlasses[17]	W	Distance of nearest obstacle at head to shoulder level	U/S	Vibratory	$96	Hands free operation, adjustable arms, and vibration intensity

C: cane mountable; H: handheld; IR: infrared; U/S: ultrasonic; W: wearable.

A prototype system using ultrasonic ranging described in Niitsu et al.[14] conveys information about the direction as well as distance of the detected obstacle using compass. It informs the user about the direction and distance of the obstacle through audio interface. System also differentiates motionless obstacles from the moving ones. Moving objects are mostly vehicles, people, etc.; they can themselves alter their path to avoid collisions and therefore, the information about motionless obstacles is more crucial. It conveys information about moving objects only when they are at 1 m from the user. The system is not capable of protecting upper body from high obstacles.

An EYECane prototype[15] with web camera mounted on the white cane detects the presence of obstacles and then suggests alternative paths to the user through an audio interface. It uses portable computer for implementing image processing and neural networks algorithms. System extracts obstacles from the live video stream, generates occupancy grid map for the neural network, and uses machine learning techniques to suggest alternative path to the user.

Lopes et al.[16] proposed a set of three mobility devices with the first one incorporated in the cane for the detection of drop-offs and holes. Second, a handheld device, for the detection of far distance large obstacles and the third, sunglasses, for the detection of overhanging obstacles. Ultrasonic sensing is employed for detection of various obstacles. The authors proposed the idea of integrating signals from three devices using a smartphone and then conveying the necessary information through it.

Most of the reviewed systems support multiple detection range options varying from less than half meter to several meters. The short distance modes are useful in finding openings and pathways especially in crowded places, whereas long distance modes are more useful in unknown and outdoor areas.

The cane-mounted systems offer an advantage of single-handed operation, allowing the use of second hand for exploration of the environment.

Another category of obstacle detection systems are wearable. iGlasses[17] is worn over the eyes just like normal spectacles. It warns the person against the upper body and head high obstacles. Gentle vibrations are produced on the sides of forehead to inform about the nearest detected obstacle. It allows hands free operation but it may not be feasible to use the device for longer hours as the vibrations are produced on the head. Size and weight of the frame limits the usage for longer duration. User may take time to get accustomed to vibrations on forehead.

Kim and Song[18] designed a wearable walking guide system using multiple ultrasonic sensors. The wearable west integrated multiple ultrasonic transmitters and receivers for finding the distance, position, and height of the nearest obstacle. The sensor module transfers this information to the PDA through Bluetooth interface. Cardin et al.[19] proposed the use of multiple ultrasonic sensors for distance and position estimation. It conveys this information using vibrotactile feedback through multiple actuators. The system is capable of detecting only shoulder level objects. Another wearable system[20] with ultrasonic sensors integrated in carry sleeves of a back pack scans 3D space in front of the users is scanned using optimally placed ultrasonic sensors array. The accelerometer and tilt sensor capture the acceleration and orientation data. These captured data are compared with the user-specific training set data to estimate whether user is standstill, moving slowly, or moving fast. Once this decision is made then some of the sensors are turned ON and OFF appropriately to save a lot of power.

Several solutions have also been designed with the use of cameras. Intelligent glasses[21] prototype uses a stereovision for detection of obstacles and produces the simplified tactile representation of the 3D scene on a small shape memory alloy-based refreshable braille display. Inertial sensor is integrated with stereo vision to create a correspondence between the surrounding environment and the user. After processing of vision algorithms, the obstacle information in the captured scene is conveyed to the user in the form of tactile map on a small refreshable display array. Zeng et al.[22] used a 3-D Time of Flight (TOF) camera for detecting objects within a distance of 7 m beyond the end of white cane. The system can be used in both indoor and outdoor environments. Camera is mounted on to the waist for capturing precise distance, orientation, and nature of obstacles using a density-based spatial clustering algorithm. This information is presented to the user on a refreshable display using abstract tactile symbols. Another system using 3D TOF camera has been reported in Lee et al.[23] Fontana et al.[24] presented the design of a wearable eyeglasses fitted with two CMOS micro cameras. The system computes the angular position and depth of the light spot produced by the laser pointer. The spatialized sound provides information about the distance, azimuth, and the elevation of the scanned object. Real-time assistance prototype system using stereo cameras mounted on helmet for detecting objects and free path is presented in Dunai and Fajarnes.[25] Captured images are processed by sequence of image processing techniques to detect various objects in the scene. The detected objects are then classified as far, near, and dangerous moving obstacles. The captured information about the surrounding environment is presented to the user using stereophonic headphones with two frames presented in 1 s each consisting of 64 pixels. User needs to understand the auditory information describing the captured scene to avoid various obstacles without colliding with them. Another system[26] for the detection of human subjects uses a helmet-mounted web camera connected to an embedded computer. An application running on the device detects a human face from the captured image and if face is not captured, it tries to detect cloth and skin within a distance of 1.5 m.

Use of Kinect sensor for acquiring the depth map of the surrounding environment is proposed in Filipe et al.[27] The captured depth image is fed to the neural network for efficient classification of the extracted line profiles to detect patterns like free space, presence of a wall, upstairs and down strains with high accuracy. These results are obtained by mounting the system at the waist level. Sunlight exposed environments and surfaces covered with water are some of the examples where infrared-based solutions may produce unreliable results and hence use is limited to indoor environments. A prototype system for the detection of dynamic changes in the 3D space during navigation is presented in Bourbakis.[28] These changes are captured through a stereo camera to create a depth image of the scene and then this information is conveyed in real time by mapping it onto 2D vibration array placed on the chest of the user. The cells of the vibrating array change quickly between no vibration, slow vibration, and fast vibration states appropriately informing the user about the surrounding 3D space.

Camera-based solutions are capable of conveying lot of information about the nature and type of obstacles in addition to the distance information. However, most of systems described above are at prototype stages at the time of writing this review and are quite far from guiding a person with blindness from real-world situations.

Several others efforts described in the literature[29-37] also try to provide a solution to the obstacle avoidance problem in mobility. However, most of these systems are either at conceptual or initial prototypes.

Navigation systems

Navigation systems aim at providing directional information on a predetermined route. They also provide information of passing landmarks and points of interest (POI) that may help in better orientation on the correct route or exploring the surrounding environment. This section explores the systems that support navigation in outdoor and indoor environments.

Most of the systems supporting outdoor navigation make use of GPS along with other sensor technologies for assisting the visually impaired pedestrian to the destination. Pedestrian Guidance System[38] used DGPS with GIS and portable computer with orientation sensors. It directional cues through spatialized sound. Information about the surrounding area is retrieved from GIS database. A 24 keys keypad is used for interacting with the system and can select synthesized speech or virtual acoustic display for getting navigation assistance from the system. Drishti[39] provides contextual information for navigation in outdoor environments. A wearable computer, a DGPS receiver, an electronic compass, and a head-mounted display with integrated headphones are used for accomplishing the desired task. User interacts with the system using voice input. Geo-tact[10] uses a combination of GPS and inertial sensors for estimating user position. It uses synthesized speech for conveying directions and distance information when requested by the user. It conveys the distance to the next point in meters and direction using clock system like 9 o'clock and 3 o'clock for left and right turn, respectively. The user interface is quite intuitive. The effectiveness of the systems relies on the exploratory skills of users, as it does not convey whether there is a direct path between two points or not.

A location-aware navigation system[40] using PDA and a portable computer provides navigation information through voice cues at regular intervals through GIS database and GPS readings. System calculates the shortest path to the destination and then announces the route to the user through speech interface. It then uses GPS coordinates to estimate the exact location of the user and then reads out the landmark that the user is passing by. System also alerts the user on reaching road junctions. A similar system[41] considers factors like safety and comfort of the traveler with blindness for calculating route to destination. Highest weightage is given to wide roads with side walk, lesser weightage to narrow roads with no sidewalks even, and lowest to routes with overpass or underpass. This information can be very useful for pedestrians with blindness unlike sighted people. It uses layered maps customized as per user requirements. Speech is used as the output interface for the system. It also offers additional functions such as display of maps, route design, raising help request, etc. Another system[42] generates guidance information, obstacle warnings, information to facilitate environmental awareness acoustically as and when required. It also considers safety as a crucial parameter for calculating the path to destination. AudioGuide[43] uses camera mounted near the waist or the shoulder of the user to capture scenes and then extract the blind sidewalks and turning points. It also records the current position of the user using GPS and periodically updates the user about the remaining distance from the destination. It uses GIS database to provide directional information. It also considers safety and comfort level of the users while calculating the route to destination. The auditory display uses a combination of speech and nonspeech information. Auditory icons (familiar sounds representing objects) are used for conveying information about objects detected within a distance of 1–1.5 m in front of the user. Speech is used for conveying information about streets, buildings, etc. Information about turning direction, distance from destination, and direction discrepancy is conveyed by varying the amplitude, frequency of sound in one of the ears of the user. A navigation system proposed by Bousbia-Salah et al.[44] allows the user to record the route related information when traveling for the first time. On reaching a decision point, user can store information about this point using a coded keyboard for left turn, right turn, cross road, cross road junction, pedestrian crossing, steps, pause, stop, etc. When the user wants to travel to the same route again, system guides the user and informs about the actions to be performed on the decision points through synthesized speech. For the reverse direction, route information is processed in reverse order with left and right turns reversed. For distance measurements, a digital accelerometer is attached to the shoe or to a rigid part of the leg. Techniques for minimizing drift errors due to accelerometers and double integrators have been applied. Integration of GPS and more sophisticated position estimation technique has been proposed by the authors.

Talking Point(TP)3[45] is a positioning system implemented on a smartphone, capable of providing the need-based information depending on the requirement of the navigation task and user preference using GPS and WiFi. It basically provides information about the nearest POI using text-to-speech interface. This information is retrieved from a central database which is accessible for annotations by the users, community members, and even stakeholders. Point of interests are categorized as pathways, areas with recognizable characteristics, landmarks as reference points, decision points and locations that support navigation. Information about the immediate surroundings (within 10 ft) is pushed to the users in the form of sound alerts specifying the name and distance of the POI. Information about the nearby and distant surroundings can be retrieved by pointing the phone in a particular direction and giving a command. A trial with eight blind users highlighted the requirement of directional information also.

System for Wearable Audio Navigation (SWAN)[46] is a wearable outdoor navigation system that guides the user by providing the navigation information in nonspeech audio form. System provides the navigation information in the form of 3D spatialized sound cues using bone conduction earphones. It considers route as a set of nodes or waypoints joined together by path segments. SWAN determines the user's location and the waypoints to the desired destinations and then guides the user using spatialized nonspeech audio beacon indicating the desired direction. User moves from one waypoint to another, following the audio beacon to finally reach the destination. Environmental objects are indicated by the combination of auditory icons, earcons, and spearcons along with the spatialized sound. User can also make annotations to the GIS database, whenever required. A custom hardware input device with thumb wheel and two buttons is used for scrolling through audio menus. The system is controlled through speech commands. The use of bone conduction earphone enables the user to listen to surrounding sounds in addition to information from the system. One limitation of the system is that it assumes the path between consecutive waypoints has a line of sight, devoid of any type of obstacles.

Another variety of navigation solutions rely on remote assistance to the users. Baranski et al.[47] developed a system in which a user wears an eye module housing a digital camera, GPS receiver, a processing unit, and a headset. It is interfaced to the mobile phone through Bluetooth interface. User can initiate a GSM-based wireless internet link for seeking navigation assistance from a remote operator. Link transmits a video captured by a wide-angle digital camera along with GPS location. This information is displayed on the remote operator's terminal who in turn guides the blind traveler and warns about upcoming dangerous obstacles. Bandwidth requirements are not very high but the field trials have shown significant communication delays in areas of poor connectivity barring remote operator from providing assistance in time critical tasks.

In another variation of similar solution,[48] the user requests the remote operator to provide a route map to the destination. Operator provides an auditory map optimized as per the user preferences and in case the user deviates from the correct route then s/he may capture surrounding pictures on reaching the cross-section using a mobile application. It sends pictures to the image-mapping server along with the GPS coordinates of the user. Server matches the received images with the panoramic view images stored in the database corresponding to the GPS coordinates. Server sends the best matching panoramic image along with the position and direction information of the user to the remote operator, who in turn further guides the user to come back to the correct path. The system has an advantage of limited dependence on the remote operator.

Use of RF technology is also attempted to support navigation in both outdoor and indoor environments. SmartVision[49,50] uses the RF tags installed along the pathways as well as specific POI. GPS and WiFi are used for positioning in outdoor and indoor environments, respectively. Stereo vision system is also used for detection of obstacles, object recognition, alignment with the street walk, etc. The application runs on a laptop and GIS server provides the information about navigation and orientation. The audio interface provides the required information using text to speech software. The Samoset System[51] is a similar RFID-based solution using passive RF tags recovered from livestock identification. They integrated RF tag reader, antenna, battery, and other electronic circuitry in the cane shell of the customized foldable long cane. The electronic stick communicates to the PDA or smartphone, running a navigation software, using the bluetooth interface. In outdoors, the installation of such tags on all routes and effective detection of all tags is very difficult.

A system using ultra high frequency RF tags fitted on the ceiling to assist in indoor navigation is proposed in Giampaolo.[52] High gain directional antennas are used for identification from a long distance. Similar tags are also attached to fixed objects to help user avoid them without colliding. A remote server stores the tag positions and other details like room number, etc. On entering the building, server logs in the user and downloads navigation database onto the navigation device using WLAN or GSM network. It requires significant amount of customization as well as may incur high implementation costs.

Another system for indoor navigation named PERCEPT[53] uses passive tags fitted at a height of 4 ft near doors, entry, and exits of the building, elevators, and emergency exits. Tags are provided with room number, etc. in raised font with its braille equivalent for correct identification. Entrances and exits are provided with kiosks. System uses a glove with integrated embedded device, RF reader, antenna, rechargeable batteries, textured buttons, and a Bluetooth interface. After identifying the desired tag, user reads it with the help of the percept glove. This embedded module communicates with the android smartphone using Bluetooth, which in turn communicates with the percept server over WiFi to receive the required information mapped to the receptive RF tag. User can independently navigate inside a building with the help of appropriately placed kiosks and tags using speech output. Mobile Guide[54] supports navigation in the indoor environments like museums. The system uses electronic compass, PDA with integrated RFID reader to read the tags placed near different objects. Both orientation and artworks-related information are provided through audio interface. Improved system[55] uses integrated infrared-based obstacle detector module. The user can choose between all audio interface and mix of vibro-tactile and audio interface. User testing suggests that a combination of vibro-tactile and audio interface may improve interaction with the system. Researchers have also explored integration of small braille display with RFID-based navigation system.[56]

In addition, several navigation systems are commercially available. Most of them are on Personal Data Assistants and have been reviewed in detail by Manduchi et al.[57] Accessible GPS is available as an application on Braille note takers like Braille Note, Braille sense, PAC mate. Simple stand-alone solutions like Trekker Breeze are also available. They concluded that each solution has its own strength and weakness. The user needs also vary significantly and the factors like user interface, multifunctionality, portability, and price play a crucial role in deciding whether a system would fulfill the needs of an individual or not.

The results of user's evaluation study of four such commercially available navigation systems[58] suggest that the devices are still not fully tailored to fulfill the needs of pedestrians with visual impairment. Most of the users found them useful for exploring the unknown routes.

With the improved accessibility with screen readers on iPhone and Android devices, a recent trend is toward GPS applications. The key limitation for most of the applications is that they are designed for vehicle navigation and provide visual feedback.[57] The applications like BlindSquare,[59] Intersection Explorer[60] have accessible interfaces and provide useful information about various point of interest as well as nearby intersections.

Analysis and discussions

When a person with visual impairment is travelling, the relevant information must be provided in the right amount, in the right format, and in real time to ensure successful navigation to the destination. The type, amount, and format may vary according to the needs and preferences of the user.

Most of the systems reviewed above are developed for the purpose of either obstacle detection or navigation (primarily providing the directional information). Some of them have been developed really well and tested thoroughly by the visually impaired pedestrians. However, they solve only a part of the travel problem. Many are technology centered designs rather than user centric designs. The real need is to develop a user centric solution that would be capable of assisting visually impaired pedestrians in different environments and situations. The review of related systems, research efforts, and related studies suggests that effectiveness of such a solution is significantly influenced by the following factors:

Choice of sensor technology

System information processing capabilities

Type and amount of information provided by the system

Human–computer interface

User centric system design approach

Training

Matching technology to individual's needs

Choice of sensor technology

For the detection of obstacles in the travel path lasers, infrared, ultrasonic, or camera sensors are commonly used. Laser-based solutions can have very long sensing range but their field of view is narrow and focused which makes scanning a time-consuming task. For infrared-based solutions detection range is shorter. Infrared and lasers allow precise distance and shape estimation of the objects but their performance degrades in environments with bright sunlight.[22,61] Moreover, glass doors and windows cannot be detected using lasers.

Precise localization with ultrasonic-based obstacle detectors is difficult due to large divergence of ultrasonic waves. It may result in nondetection of smooth and small aperture surfaces.[22] However, ultrasonic sensors can reliably detect even small obstacles within a distance of 5–10 m. Availability of waterproof transceivers with varying beam angles makes them an ideal choice for use in obstacle detection systems. Most of the available obstacle detection devices use ultrasonic sensors.

Camera-based systems can capture high resolution and spatially dense images with no theoretical distance limitations and can even be used in intrusive environments where ultrasound or infrared emissions are not allowed.[62] Now cameras are integral part of smartphone; moreover, small size, wide angle, and low cost cameras are also easily available making them a potential candidate for obstacle detection as well as recognition systems. However, extracting useful details from the spatially dense images involves significant amount of processing. The challenge is also in conveying this information to the user in a simplified and understandable form. Lightening conditions, camera angle, and amount of clutter in the scene affects their performance.[63]

GPS-based systems are mainly suited for outdoor environments. Combination of GPS and GIS can result in rich and more meaningful navigation information. Recent trend is shifting toward Smartphone-based navigation applications because GPS modules with high accuracy are now a standard peripheral in Smartphones. However, most of the navigation applications have been designed for sighted travelers. They cater to the information needs of the travelers with blindness in a very limited way.

Inertial sensors are also used for position estimation. This allows their use in both indoor and outdoor environments. However, to improve accuracy and reliability, several systems have used the combined approach of integrating them with GPS for outdoors and WiFi for indoors.

RFID-based solutions can be effective in controlled environments like buildings, museums, etc. Simple RF tags must be in close vicinity of the RF reader for detection. Ensuring this can be a serious challenge for a person with blindness. Moreover, it requires equipping all the travel routes with the RF tags and considering decisions like distance between successive tags, choice of installation locations, effect of weather changes, maintenance and updating of databases, etc. RFID-based navigation system may have high power requirements to ensure detection of tags from a distance as it is nearly impossible for a blind traveler to know the exact location of the tag.[45] In case of emergency like fire or electricity outage, other systems may fail but the system using passive RF tags may continue to provide real-time navigation information.[52]

System information processing capabilities

Most of the obstacle detection systems that use single, multiple, or combination of sensors (other than camera) use embedded microprocessors or micro-controllers for the processing and computing task because of their small size and low power requirements. Availability of various onboard analog and digital interfaces for easy integration of sensors, communication modules, and output interfaces like vibrators and buzzers makes them an ideal choice for such systems. Camera-based solutions often require high end processors for meeting the computation requirements. Extracting useful information involves application of sequence of different image processing techniques which are computation intensive and often require use of sophisticated software tools. Hence, such systems often use portable computers. The primary advantage is the availability of large memory and other interfaces along with the powerful processors. The disadvantage is increased power requirements and limited battery life. They require specially designed ventilated backpacks and even external batteries for ensuring continuous use for longer periods, further adding weight to the system.

Recent advances in the processing capabilities of the smartphones or PDAs; availability of onboard sensors like magnetic compass, light sensor, tilt sensor, accelerometers, etc.; communication interfaces like GPS, GPRS, Bluetooth etc.; accessible user interfaces and long battery life make them a preferable choice for the processing platform in these systems. Many proposed systems which do not require application of computation-intensive image processing techniques on the camera-captured images have started using these devices.

Type and amount of information provided by the system

Independent travel to the destination can only be ensured if the information about obstacles, directions, landmarks, etc. can be conveyed at the right time and in the right amount so that the pedestrian can perceive and use it in real time. The information should be sufficient to allow creation of a rough mental picture of the surrounding environment. The way of presentation of relevant information is one of the key problems in existing devices. Huge amount of information sometimes overwhelms the user and further complicates the travel task. The amount of cognitive load created due to information through interfaces should also be considered keeping in mind the retention capabilities of the brain.[64] The type and amount of information conveyed by the system should always aim at simplifying and supporting the navigation task. One should not try to reproduce the information that a person with blindness is already getting from the surrounding environment through other senses.

Studies[65,66] investigating the use of auditory and olfactory cues used by persons with visual impairment suggest that environmental sounds like sounds of car passing, street traffic, people entering or leaving a shop, vehicle horn, etc. are used for collision avoidance, distance estimation, localization, orientation and crossing streets, etc. during navigation. Olfactory cues like smell from restaurant, bakery, garbage can, flower shop, sea, etc. are often used as point of reference, for location identification, collision avoidance, and as a landmark for self-orientation during the navigation task. However, many of the cues are culture and geography specific and are not universal. Navigation on familiar routes may pose difficulties when there is a change in the physical layout (like an appearance of a temporary barrier during construction work) or there are changes in environmental cues (like olfactory cues).[45] According to one of the work,[64] presenting small pieces of information which does not interfere with the information already being gathered, helps a blind person in improving the navigation performance.

Bavovic et al.[67] found that the amount, type, and details of information required by visually impaired persons to build and maintain cognitive maps depend on their personal interests, techniques used for navigation, level of familiarity with the environment, proficiency in orientation and mobility skills, as well as changes in the environment. Navigation information provided to the user should consider context, location, user interests, familiarity with the route, environmental situations. Bradley and Dunlop[68] found that visually impaired persons mainly rely on directional, structural, environmental, sensory, and social contact information for accomplishing the navigation tasks. Effective navigation can be ensured by providing personalized and task-specific contextual information. Inclusion of information about the immediate surroundings, landmarks relating to actions that are to be performed, distance to the next information point can enrich the effectiveness of verbal information provided by a navigation system.[69]

Convenience of an individual to navigate through a route is an important consideration. Shortest route to the destination is not always the safest path. Presence of sidewalks, smooth surfaces, less congestion on road, and limited number of obstacles are some of the important factors that need consideration. User convenience rating can be considered while recommending route to the destination.[70] Results of a recent study[71] show that persons with visual impairment often collaborate on navigation to identify and follow routes that are user friendly.

Information about specific landmarks instead of general distance or road information, information at key navigation decision points, as well as the information confirming a correct navigation decision is very important.[72] Vocabulary used in presenting the information should be unambiguous in nature and follow regular expression with focus on action, direction, reference to distance/time, side and finally some local object like “turn right after passing the pillar on your left”.[73] The information requirements may vary significantly from person to person. Type of information that system can provide must be available as a choice to meet individuals' needs. Objects can be divided into categories and users may be provided with the control over notifications related to each category of objects. Accurateness, consistency, correctness, completeness, and newness are the key attributes for the maps used in navigation systems for visually impaired.[42] According to a study,[74] users preferred a system which provides information as and when requested by the user. The proposed approach is that such systems should provide basic information, followed by prioritized context aware personalized and task-specific information. In addition, there should be a provision to extract more detailed information whenever required. Most of the systems reviewed in the previous section provide only fixed and limited information. The information needs highlighted in this section are a result of user studies and testing. Therefore, they must be considered and incorporated to practically address the travel problems of the blind pedestrians.

Choice of human–computer interface

Once the travel-related information is captured, processed, and prioritized by the system, then the next most important is the human–computer interface. The input interface should be such that it is accessible and users can communicate with the system in an interactive manner. The output interface should be such that it does not interfere with the user's alternate senses and conveys all the required information in simple and understandable form in real time with minimum cognitive load.

Input interface

Most of the systems have used buttons, keypads, speech recognition as the medium for interacting with the systems. The number and type of buttons varies based on design and features supported by the device. The buttons should be placed such that they are easy to locate, identify, and operate. Accidental change of state during use or storage should be prevented. Use of tactile or braille symbols around or near the buttons is required for easy identification.

The existing iOS, Android and Windows-based mobile devices are touch screen based. On such devices, a person with visual impairment can access a navigation or any other application with the help of a screen reader software. However, it is extremely important for the developers to follow accessibility guidelines and develop application interface that are completely accessible. At present, the accessibility of iPhone devices is considered best; however, due to higher cost, they are still not affordable for everyone. On the other hand, most of the android-based devices are affordable and accessibility is latest version 6.0 of the Android operating system is much better than the earlier versions.

Voice recognition is also attempted in several systems especially based on portable computers. Recognition inaccuracies especially in outdoor and noisy environments are a serious problem. According to a study,[74] given a choice between voice recognition and keypad interface for interacting with the system, most of the users opted for the latter one as it may allow system to be used without attracting attention of others in the vicinity.

Output interface

In the absence of vision, auditory or tactile or vibro-tactile interfaces are available choices for conveying the navigation information to the users. Basic information about the presence and absence of the obstacles can be easily conveyed through vibrations of different types. Use of vibrations results in an advantage of no interference with the acoustic information of the surrounding environment. Most of the reviewed systems use a similar output interface for conveying the distance information. The intensity and frequency of vibrations should be neither too high making the use of device for longer hours difficult nor too low resulting in missing of conveyed information. However, further investigations are required to understand the perception bandwidth of vibratory interface for person with blindness.

When combined with audio beeps of different types, a lot more information can be conveyed without overloading the user but the interface should be designed carefully by using vibrations for conveying the most commonly used information and beeps for conveying less frequent or priority information. This type of interface is used in SmartCane™ device as well as few other systems.

More detailed information can be conveyed through tactile interface like a small-customized electronic braille displays for conveying much more detailed information like simplified 2D representation of the 3D scene captured by camera and using abstract symbols for conveying the distance, size, position, and nature of objects in the captured scene. However, it may significantly add to cost. Moreover, limited information can be coded on an array of 8-dot braille cells.

Few systems have also used multiple vibrators mounted on the waist belt, fingers of wearable gloves, inside wearable vests, etc. for conveying information about the position of detected objects. Variation in vibration intensity is commonly used to convey the distance information. Solutions with limited vibrators have not been able to effectively indicate position of obstacles whereas multivibrator solutions may result in complicated designs and overwhelm the user with lot of information.[22] User interface is the most complex part of these systems. Use of vibrating wristbands is proposed for providing turning instructions and intensity of vibrations may be varied for notifying the distance of the obstacles. One hand keyboard with braille output for user interface is also proposed.[42] Information can be better conveyed using tactile feedback as it does not require continuous concentration. It offers less cognitive load.

Different parts of body have different sensitivity to touch; therefore, location of the tactile display on the body is dependent on nature of information. Precise information can be conveyed through fingers and tongue due to their high sensitivity to touch, whereas other parts of body can be used to convey simple information. Tactile interface is best suited for conveying simple directional information.

However, conveying descriptive information through a tactile or vibro-tactile interface can be a challenge. In such situations, audio interface is the only choice but it might interfere with useful environmental cues. Though interference can be minimized with the use of bone conduction earphones. Down conversion of received ultrasonic signals to auditory domain and then relying on the brain capabilities to decipher can be another way of conveying the spatial information.[75] Some of the existing systems make use of spatialized sound for describing the captured scenes to the users. Variations in amplitude, frequency, and pitch are commonly used for conveying the distance, size, and position information. Talbot and Cowan[76] highlighted that intensity, spatial filtering of sound, and the ratio of direct to reverberant sound are three naturally occurring changes in the character of sound (ecological cues) that can be used by both sighted and nonsighted to access distance of an object using sound. However, existing systems use nonecological cues such as pitch or temporal variation because substantial processing is involved in using ecological cues. Use of nonecological cues may be simple from the processing point of view but it increases the learning curve for the user.

Some systems also used natural sounds corresponding to different obstacles whereas others use synthesized voice for conveying various information. With synthesized speech, detailed information can be conveyed directly without the need of any further interpretations. Accelerated speech can also be used, as most of the blind users are comfortable in using it on screen readers. According to a study,[74] accelerated speech interface is the most preferred interface for the presentation medium. However, none of the reviewed systems has exploited this possibility. User can be given the control to change the amount of acceleration based on familiarity with the device. Additional cognitive load, need for continuous concentration, and interference with the surrounding auditory are some of the disadvantages of using synthesized speech.

Zeng et al.[22] evaluated the effectiveness of auditory displays by using synthesized voice for announcing the street names, buildings, facility, and crossroads. Results of evaluation by 100 sighted participants suggested that the change in volume for indicating closeness to the destination and road direction variation was preferred. Use of natural sounds corresponding to different obstacles such as sound of a metal for dustbin, sound of leaves for tree, etc. was highly preferred. Participants easily identified the objects corresponding to these sounds with very little training.

Most of the existing systems use single sensory modality like hearing or touch for conveying the surrounding information. This will quickly overload that modality resulting in limited perception. Learning vision to sound or vision to tactile mapping requires training and significant effort from the user.[77] Recent trend is to use combination of optimized sound and tapping interface.[78] These systems should offer user-selectable combination of the optimized interfaces for catering to the needs of individual users.[79]

User centric system design approach

Technology-centered approach with inappropriate user interfaces is one of the key problems in the current designs.[64] To ensure usability, devices should be designed with user centric design approach with special focus on the individual's travel needs. In a user evaluation study of two electronic mobility aids,[80] it was found that “individual users' characteristics and preferences appear to be critical for their appraisal of the devices.” The focus should be on the ergonomics, robustness, and simplicity of the user interface, comfort of the user, and adoption time.[81] Invasive design, bulkiness, and heavy weight often contributes to rejection of assistive devices by the users.[82] Designs that “advertise” a user's disability are often rejected.[36] Aesthetics play a significant role in the acceptance of these devices.[83] It makes sense in integrating assistive technologies in the common objects like mobile phone, portable computer, watch, keys, long cane, etc. that are common to everyday life of the users.[84]

Several prototype systems have used sensors or cameras, mounted on eyeglasses, forehead, or on helmets. This obtrusiveness attracts unwanted attention and in a way highlights the disability of a person with blindness. This is acceptable for testing purposes but the probability that would be used by people is very low even if they reach the product stage. For the design of wearable systems that are integrated in textiles, several factors like choice of fabrics, portability on different clothes, effect of sweat, protection during rain, wearability in different seasons, etc. need to be considered.

Users should be involved throughout the entire design cycle from the inception stage to the final design validation stage for incorporating their valuable feedback. According to a study[85] with 227 adults with various disabilities, “mobility aids were more frequently abandoned amongst other categories of devices due to lack of consideration of user opinion in selection, easy device procurement, poor device performance, and change in user needs or priorities.”

It is also equally important to consider the “cognitive ability” of the user during the development of such solutions. The fact is that even in the absence of such solutions, a large number of persons with blindness manage travel on their own by gathering surrounding information through sense of hearing, touch, and smell. Mental mapping of the travelled routes allows the person to find the way on its own with the help of landmarks. Therefore, the information that a person is capable of gathering on its own must not be reproduced by a travel aid.

Training

According to the national study on needs finding of Canadian visually impaired population,[86] 86% of the users have been able to use the technical aids properly after completion of appropriate training. For the effective usage of any travel aid, effective mobility and orientation skills are required.[19] In addition to training of orientation and mobility skills, effective use of assistive devices may involve additional training that may vary significantly based on complexity in device functionality and user interface.[87] Designers must collaborate with the orientation and mobility experts to come up with systematic training curriculum. Interactive learning with the help of teaching aids is required. Users must be provided with accessible learning resources at the time of device purchase.

Mapping of technology to users

The results of a user's evaluation study of Navigation systems stress that the careful selection of the device for the individual user is extremely important.[58] Similar findings were reported in another user's evaluation study of two obstacle detection systems.[80] It suggests that the effective use of technology is affected by user's needs, expectations from the technology, features and functionalities of the technology, as well as environmental factors.

Conclusions

A folding long cane (white cane) is the most widely used travel aid across the world for both indoor and outdoor environments. A person with good orientation and mobility skills can effectively detect obstacles on ground, surface changes, etc. and can manage travel especially in structured known environments by gathering surrounding information through sense of hearing, touch, and smell. It also helps in locating the landmarks but does not help much in gathering directional information for navigation at decision points.

Moreover, there is always possibility of collision with obstacles that cannot be detected with it like low hanging tree branches, signboards, open glass windows, etc. The probability of injuries due to collisions with such obstacles may be lower in structured environments but it is surely very high in unstructured and inaccessible environments prevalent in most of the developing countries around the world. The use of the cane-mountable electronic obstacle detection system using ultrasonic sensors seems to be one of the effective ways of addressing this problem, provided it effectively detects and informs about the presence of obstacles from ground to head height and across the body width of the person within a distance of few meters in front of the person. Multiple sensors and/or combination of sensors can be used to identify the exact position and nature of the detected obstacles. Simple vibratory interface with combination of audio beeps can be used to convey distance information about the obstacles on the path.

For outdoor navigation GPS-based systems can be a potential solution for providing directional information. Several commercially available GPS-based navigation devices provide this information for outdoor routes. Many of them are implemented on high-end note takers with electronic Braille and Speech output. Such solutions are multifunctional and can be extremely helpful for the users who are used to carrying PDAs as they don't need to carry an additional device for navigation information. However, such a solution can be of very limited use for a person who needs only navigation information. Moreover, they are highly unaffordable.

The recent trend is toward the use of map-based navigation applications running on Smartphone with integrated GPS. Some of the key examples are Google maps, Apple maps, etc. They are widely used by the sighted community for both vehicle and pedestrian navigation. The user interface for the applications is accessible to large extent for even the persons with blindness, when used with screen reading software. However, the information provided by the applications helps pedestrian with blindness to only a limited extent.

A GPS-based accessible navigation app designed to cater the information needs of the pedestrian with blindness is the way forward. It should have provision for annotating the information about the landmarks used by blind pedestrians such as bumps on the road, information about presence of footpaths, etc. It should also allow crowd sourcing of information specific to blind pedestrians.

However, inherent GPS inaccuracies may not allow the blind pedestrian to locate the final destination exactly especially within last 10–30 m. Low power Bluetooth beacons installed near the entrance of the buildings can help resolve this difficulty. Once the user is in the proximity of the beacon, the application can establish a connection with the beacon and guide the user to the destination.

The information can be conveyed to the user through synthesized speech with an option for accelerating it. Bone conduction earphones or a single ear Bluetooth headset can be used to avoid interference with the audio cues from the surrounding environment. However, further investigation is required to understand the navigation information needs as well as user interface preferences of pedestrians with blindness.

Another important consideration while developing a navigation solution is that the information needs can reduce significantly with the familiarization of the route. Therefore, the solution must allow customization of information in real time.

For indoor environments, RFID, Wi-Fi, Bluetooth-based solutions are the possible alternatives. Low cost and low power Bluetooth beacon-based implementations seem to be a promising solution for indoor environments. The primary advantage of such solutions is that the required directional information can be directly pushed to the mobile phone as the user passes the Bluetooth beacons installed at different places in the building.

The above discussion points that combination of cane-mountable obstacle detection system and Smartphone-based navigation solution can address the travel problem for blind pedestrian. This approach may provide complete flexibility to pedestrian with blindness to use one or both based on travel needs. On completely familiar routes, mental maps may be sufficient to guide the person to the destination without any need of the navigation aid. The obstacle detection system should be highly portable, less computation intensive, should have low power requirements, and simple interface. It may connect to navigation system through Bluetooth interface. The implementation of navigation assistance on Smartphone will eliminate any additional cost for the hardware and will reduce the learning curve. This in turn helps keep the complete solution highly affordable, noninvasive, user customizable with simple and intuitive user interface.