'Intel Acat' Assistive Platform for Arabic Speaking Disabled People: a Complete Integration.

Background: The Intel Assistive Context-Aware Toolkit (ACAT) is the highly configurable platform used by Dr. Stephen Hawking to communicate with his environment. After being released freely to the public, we, at the Embedded Systems Laboratory - UBMA, have been working on integrating the Arabic language on the different packages of the platform in order to make it accessible for disabled people from Arabic countries and decrease their communication limitations. Objective: This subproject concerns the Arabic Text-to-speech engine implementation and comes as a final step toward the full integration of the Arabic language into Intel ACAT assistive platform.
Methods: The text to speech conversions was integrated by implementing a mapping between the Arabic words and their phonetic spelling using Microsoft Text-To-Speech Synthesis on Intel ACAT modules and extensions. A full compilation was then executed and tested gathering all the modules and the features of the platform.
Results: Over this final integration step (which is freely accessible and open sourced for the public), people with severe disabilities from Arabic-speaking countries will have fully access to all the features of the ACAT platform and will be able communicate and interact easily with their computers.
Conclusion: The Arabic language Text-to-speech engine integration on 'Intel ACAT' Assistive Platform is the final milestone of our project toward making the platform fully accessible for Arabic-Speaking users and comes after our previous integrations of the Arabic language into the keyboard, the intelligent predictive text engine and all panels and interfaces of the platform.

BACKGROUND

According to the report on disability of the World Health Organisation, more than one billion persons live with some sort of impairment worldwide, with almost 200 million of them having significant functional challenges (1). Disability will become even more of a worry in the coming years as its prevalence rises. This is owing to the global growth in chronic health diseases such as diabetes, cardiovascular disease, cancer, and mental health disorders, as well as the aging population’s increased risk of incapacity (2). Those with disabilities have worse health outcomes, worse educational attainment, lower economic involvement, and higher poverty rates than people with-out disabilities all throughout the world. This is largely due to the fact that people with disabilities face challenges in gaining access to services that many of us take for granted, such as health care, education, employment, transportation, and information. These problems are compounded in underserved communities (1).

Among the different technologies supporting the inclusion of people with disabilities, the Assistive Context-Aware Toolkit (Acat) from Intel is the highly configurable platform used by Dr. Stephen Hawking to communicate with his environment (3). After being released freely to the public, we, at the Embedded Systems Laboratory - UBMA, have been working on integrating the Arabic language on the different pack-ages of the platform in order to make it accessible for disabled people from Arabic countries and decrease their communication limitations (4). This subproject comes after our earlier piece of work (4), as a final step toward the full integration of the Arabic language into Intel ACAT assistive platform.

The paper structure is as follows. The aim of the project is introduced in Introduction. Next section (Assistive Technology for the severely disabled) defines and explores the use of assistive platforms especially among the Arabic-speaking disabled. Section 3 introduces the Intel Assistive Context-Aware Toolkit (ACAT). The Arabic language integration is detailed in Section 4 while the Arabic Text-to-speech engine implementation is described in Section 5 in addition to the results of testing. The conclusion points out and recapitulates the main assumptions of this study.

Assistive Technology for the Severely Disabled

Nowadays Information and communication Technologies (ICT) are used in a different scale by both disabled and the non-disabled people, while used by normal people to make their lives even more sophisticated, the ICT are considered by the disabled as assistive platforms helping to make their lives looking more normal (5). Therefore, the efforts to make the ICT more inclusive are considerable and the development of new assistive platforms for people with disabilities has gained increased focus worldwide (6).

Assistive technology (AT) concerns persons of all ages and with all sorts of disabilities. Some assistive technologies are easy and affordable, others are sophisticated and expensive (7, 8). Also the provision of assistive technology is a complex issue, since a single AT solution is highly dependent on the user’s ambitions and his particular peculiarities, there is no solution that matches all users (9).

A Human Right

Since 2006, Assistive technology has designated as a human right over the Convention on the Rights of Persons with Disabilities (CRPD) of the United Nations (10). This agreement binds ratifying states (including Arabic countries) to enact suitable steps to make AT solutions more accessible to the disabled in order to increase their independence and make the society more inclusive. The CRPD has endorsed the establishment of AT systems, policies, and procedures. The CRPD establishes standards for ratifying countries to satisfy when it comes to disability inclusion. It is founded on values such as “respect for intrinsic dignity, individual autonomy, especially the right to make one’s own decisions” (Art. 3). Art. 4 requires ratifying governments to “ensure and promote the full fulfillment of the human rights and fundamental freedoms for all individuals with disabilities,” which includes expanding the availability, awareness, and deployment of AT (Art. 26).

Assistive platforms for Arabic speaking users

Due to the lack of study conducted on the subject, we investigated, within our project, the use of Assistive platforms among Arabic-Speaking users. Our study was aimed at determining the availability and the accessibility to assistive platforms in the Arab world throw the framework of Technology Acceptance Model (TAM) 3 (11).

According to our study, it is estimated that assistive technologies are accessible by only 2-10% of the disabled in the Arabic Countries. Two main reasons emerged from our findings concerning this digital divide on the use of assistive platforms:

A) The availability of Assistive platforms: despite the quantity of assistive plat-forms and their crucial importance, less has been developed for Arabic speaking users.

B) Users are digitally literate concerning the available platforms.

It was also observed that the available platforms supporting the Arabic language are not always affordable (expensive).

However, additionally to the digital divide between normal people and the disabled ones around the world (12), our study endorsed a new face of inequality for the Arabic persons with disabilities toward available assistive platforms.

Intel Assistive Context-Aware Toolkit (ACAT)

This section describes the ACAT platform, the history of its development, its main features and benefits.

An overview

Intel ACAT was originally developed for physicist Stephen Hawking and was instrumental in giving him a voice. The platform consists of three parts: motion recognition using a webcam or other sensors, an interface, and a contextual auto-selection system. Stephen Hawking interacted with the computer over ACAT using a single facial muscle and he was allowed to use various programs, including the Internet browser and Microsoft office applications.

Intel ACAT History

Stephen Hawking (one of the most renowned scientists) was diagnosed with Amyotrophic Lateral Sclerosis (ALS) at age 21. Degenerative, the disease progressively paralyzes motor functions, but does not affect the person’s brain activities (3).

Over the years, Hawking lost, among other activities, the ability to walk, move his arms, keep his head up, In addition to the fact that ALS also affects speech, Hawking faced severe pneumonia in 1985 that required him to undergo an emergency tracheotomy. The procedure ended up irreversibly damaging her vocal cords. Since then, the scientist is forced to use a voice synthesizer to communicate, through the voluntary contraction of one of his cheeks, which is translated into the composition of words and phrases in his voice synthesizer. “Medicine didn’t cure me, so I rely on technology to communicate and live” sais Hawking (13). Despite this achievement, Hawking could only speak at a rate of one word per minute, so he requested technical help from the Intel company to improve his word prediction system in order to increase his speech rate to five and even ten words per minute. Intel then began working on a new system called Assistive Context-Aware Toolkit (ACAT).

With a better word predictor that would interact with both the movement of the scientist’s cheek and his mouth and eyebrows, the software was created by Intel in partnership with Swiftkey, where the sensors installed in Hawking’s glasses continue to recognize commands from the scientist’s facial movements (Figure 1), ACAT high-lights the function to be used. The data was interpreted by a laptop running Windows whose screen is right in front of Hawking, as before (13).

Figure 1.

Hawking Using Intel ACAT

Swiftkey took care of the writing function while Intel took care of the interface, although, it was necessary to observe Hawking’s activities to identify language styles, thus, ACAT became able to suggest words according to the context (Figure 2): a technical document has a more elaborate vocabulary; an email with informal language leads to suggestions of words most used in everyday life. Using ACAT, Hawking only needed to enter 15% to 20% of the characters of the sentences to be written or spoken by the synthesizer. The rest is up to the software. Allowing him only to write was not enough, Stephane needed to perform other tasks, for this reason, Intel was primarily concerned with contextualizing the inter-face; the company’s engineers were observing how Hawking used the computer and adapted the interface to give him easy access to the most common activities, such as searching the web, reading PDFs (Figure 3), writing documents and emails and activating the voice synthesizer (Figure 4).

Figure 2.

ACAT Writing-sequence

Figure 3.

ACAT Apps Explorer

Figure 4.

ACAT Lecture Manager

Did it work? Stephen Hawking himself answered: “I am now able to give lectures, write articles or books and speak much faster” (13). With this ingenious communication system, he was able to dedicate all his time to communicating and expressing his knowledge to the rest of the world, becoming one of the most reputable scientists in the world.

Over time, Intel has expanded ACAT platform to make it more versatile so it can adapt to higher or lower disability levels. The North American giant has taken a new step by announcing that the software is available for free on GitHub. In addition, the ACAT code was published as open source for the developer’s community, Intel representatives expect that open access to the ACAT source code will allow software developers to adapt the system to the needs of many people who need it, “We hope this open source configurable platform will allow developers to keep work-ing by adding new interfaces, new sensors, improving the prediction system and add-ing many other features,” says Intel principal engineer Lama Nachman (14).

Assistive platforms for Arabic speaking users

The Project primarily intended to bridge the digital divide for the severely handicapped around the Arabic word, therefore, over an extensive research conducted by us, we found that integrating Arabic language into ACAT is the most suitable and cost effective way to provide assistive communication platform to Arabic-speaking persons and may be the key element to solve their digital divide problem. Thus, the potential of ACAT to improve their lives still remain unexplored because of the language barrier.

Our first attempt to partially translate the platform was successful (4), indeed, many packages were integrated and tested (the keyboard, the intelligent predictive text engine and all panels and interfaces of the platform) (Figure 5), since then, our focus was mostly on giving full access to Arabic-speaking users over the final mile-stone integrating of the Arabic voice-to-speech engine, which is the aim of this paper toward a full integration of the Arabic Language on ACAT assistive platform.

Figure 5.

Intel ACAT Partially working on Arabic Language

Arabic Text-to-Speech Engine implementation

As described in our previous paper (4), ACAT is organized as a modular system (Figure 6), our final integration concerns the text-to-speech module.

Figure 6.

ACAT Modular architecture

The Text-to-speech Management component has a helper class called Pronunciations which loads the pronunciations XML file and maintains the mapping between the actual spelling and the phonetic spelling. It looks up the mapping table and re-places words in an input string with their phonetic spellings. When the result string is converted to speech, the words will be pronounced properly (15).

To perform the text to speech conversions on ACAT (which is developed in C# and NET), an XML configuration file containing a mapping between the Arabic words and their phonetic spelling was implemented using Microsoft Text-To-Speech Synthesis:

…

using System.Globalization;

using System.Speech.Synthesis;

namespace ACAT.Extensions.Default.TTSEngines.SAPIEngine

{ /// Converts text to speech by sending the text string to the

/// Microsoft Speech Synthesizer

[DescriptorAttribute(“Arabic-Language”,

…

…

…

{ _speechSynthesizer.SelectVoiceByHints(SAPISettings.Gender, VoiceAge.NotSet, 0,

CultureInfo.DefaultThreadCurrentUICulture); }}

catch (Exception ex)

{ Log.Debug(“Error setting TTS settings “ + ex); }}

/// Event handler for bookmark reached. Completion of

/// event sender

/// event args

private void speechSynthesizer_BookmarkReached(object sender, BookmarkReachedEventArgs e)

{ try {int bookmark = Convert.ToInt32(e.Bookmark);

notifyBookmarkReached(bookmark); }

catch (Exception ex)

{ Log.Debug(“Invalid bookmark “ + e.Bookmark + “, exception: “ + ex);}}}}

The Text-to-Speech XML File was integrated following the Steps to create a TTS Extension of the Developer’s Guide [15].

The Overall Implementations done for this project was concerning the following elements (Table 1):

Table 1:

Implemented Intel ACAT elements

ACAT Element	Description
Panel Manager	Panel Manager is responsible for enumerating panels, handling requests to display them, maintaining the stack of active panels, activating and deactivating them.
Animation Manager	The Animation Manager in this component is responsible for loading animation files which contain the scanning sequence for scanners, executing the scanning process and handling transitions between scanning sequences.
Widget Manager	The Widget Manager in this component reads the attributes for the active scanner, instantiates the widget objects for the elements in the scanner and also controls the appearance of the scanner elements during scanning.
Word PredictorManager	Word Prediction Manager in this component enumerates installed Word prediction extensions, and activates and configures the preferred Word prediction extension
Theme Manager	The Theme Manager In this component enumerates and manages installed themes.
Talk WindowManager	The Talk Window Manager in this component activates and manages the Talk window
Text-to-Speech (TTS)Manager	The TTS Manager in this component enumerates installed TTS extensions, activates and configures the preferred TTS extension.
Agent Manager	The Agent Manager is responsible for functions such as enumerating Agents, monitoring the foreground application, and activating agents.
Actuator Manager	The Actuator Manager is responsible for functions such as enumerating the available switches and raising events to indicate switch triggers.
Spell Check Manager	The Spell Check Manager in this component enumerates installed spell check extensions, activates and configures the preferred extension to use.
User Manager	The User Manager manages ACAT users and profiles.
Command Manager	The Command Manager contains classes to map commands to their actions.
Widgets	The Widget library contains widgets for the various elements of the scanners, dialogs and menus.

OBJECTIVE

The aim of the study was to review subproject concerns the Arabic Text-to-speech engine implementation and comes as a final step toward the full integration of the Arabic language into Intel ACAT assistive platform.

METHODS

The text to speech conversions was integrated by implementing a mapping between the Arabic words and their phonetic spelling using Microsoft Text-To-Speech Synthesis on Intel ACAT modules and extensions. A full compilation was then executed and tested gathering all the modules and the features of the platform.

RESULTS AND DISCUSSION

Since our first sets of integrations of the Arabic language on the ACAT platform (extensions, libraries, Panels..), each new integration was generated, tested and compared with the original platform (in English) in order to catch any software bugs or missing.

A modular test of the added Text-to-Speech Engine has been carried out, many efforts have been undertaken to enhance the pronunciations of Arabic words by improving the phonetic mapping table and the voice synthesis of the used library (Microsoft Text-To-Speech Synthesis).

A full compilation was then executed gathering all the modules of the platform and was then entirely tested with all the features together. (Figure 7).

Figure 7.

ACAT Core Components from Input Switches to Text-to-speech

The results are very satisfying and the new platform has proven useful for the final users (Arabic-Speaking people with disabilities), it has a promising potential to greatly improve their lives. The upcoming challenge for us is raising the awareness about this platform and promote it adoption.

The English user guideline (16) is being translated to Arabic to help users under-stand the platform functionalities. As asked by the Intel Lab, once this work is published, our entire integration pro-ject will be sent to intel research team to be validated and indexed in the official page of the project (14).

The results of this project have been shared with the public over the following links:

A) The Software for Arabic-Speaking users can be downloaded from (17).

B) The Source code is available on this GitHub repository (18).

The developer’s community may refer to the Developer’s Guide (15) to understand the hierarchical modules of the ACAT platform

CONCLUSION

Intel ACAT is the affordable assistive platform that makes computers more accessible to people with disabilities, in this project, we aimed to make Intel ACAT even more accessible by providing an Arabic version for the Arabic-Speaking users. The different steps of this project was described in this paper, with a specific focus on the final integration, which concern the Text-to-Speech Engine implementation.

The developed new version of Intel ACAT including Arabic language was tested and improved before being generated and uploaded publicly on the internet.

We encourage the use of the software by the final users (people with disabilities) and also by developers for further contributions and enhancements.