An open-source and cross-platform framework for Brain Computer Interface-guided robotic arm control.

Brain Computer Interfaces (BCIs) have focused on several areas, of which motor substitution has received particular interest. Whereas open-source BCI software is available to facilitate cost-effective collaboration between research groups, it mainly focuses on communication and computer control. We developed an open-source and cross-platform framework, which works with cost-effective equipment that allows researchers to enter the field of BCI-based motor substitution without major investments upfront. It is based on the C++ programming language and the Qt framework, and offers a separate class for custom MATLAB/Simulink scripts. It has been tested using a 14-channel wireless electroencephalography (EEG) device and a low-cost robotic arm that offers 5° of freedom. The software contains four modules to control the robotic arm, one of which receives input from the EEG device. Strengths, current limitations, and future developments will be discussed.

INTRODUCTION

Brain Computer Interfaces (BCIs) have been described for various purposes: Communication and computer control, motor substitution, and entertainment.[12] Restoration of motor function, or motor substitution, has been an area of particular interest because of the associated loss of independence. Much work and research focused on this including Defense Advanced Research Projects Agency (DARPA) programs and the design of a revolutionary arm.[1011] BCI-based motor substitution has been described using robotic arms, exoskeletons, and functional electrical stimulation.[12457-913-1719-21] BCI development requires multiple elements, not the least important of which is a model for robotic arm control that can be used by bodily disabled people to regain some extent of self-care. This may also reduce the constant need for help by other people, which is time-consuming and costly. Open-source BCI software is available to facilitate cost-effective collaboration between research groups, but it mainly focuses on communication and computer control.[318] Motor substitution is an important and active field for BCI research, but there is no accessible model for researchers entering this field. To the best of our knowledge, an open-source software platform for cost-effective BCI-guided robotic arm control does not exist yet.

Ideally, such a platform should be open-source, cross- platform, extensible, and working with cost-effective equipment that allow researchers to enter the field of BCI-based motor substitution without major investments upfront. We developed such a platform and present our initial results.

MATERIALS AND METHODS

Robotic arm

A LynxMotion AL5D robotic arm was selected for this project (Lynxmotion.com). This is an all-aluminum arm offering 5° of freedom, 10.25” median reach, and 13 oz lift capacity. The robot features: Base rotation, single plane shoulder, elbow, wrist motion, a functional gripper, and optional wrist rotate. The robot arm kit consists of black anodized aluminum brackets, aluminum tubing and hubs, custom injection molded components, and precision laser-cut Lexan components. The robot uses an SSC-32 servo controller that connects to a personal computer or laptop using the serial port. USB-port access is possible with a USB-to-Serial adapter cable that uses a FT232R chip. This arm has been selected for our initial research for its low cost and customizable setup. It can be ordered online for less than $500 and similar technology has been proven effective in assembly lines.

Software platform

C++ was chosen as the programming language for development, in combination with the Qt framework to create a cross-platform graphical user interface. The software uses the external Qext Serial Port library version 1.2-beta1 to connect with the serial port. Development was done in Qt Creator 2.5.0 using the Qt SDK version 4.8.1. On Windows 7 SP1 (Microsoft, Redmond, WA) the program was compiled using the Microsoft Visual Studio compiler, as provided with Microsoft C++2010 Express. On Mac OS 10.7 (Apple Inc, Mountain View, CA) and Ubuntu Linux 12.04 (Canonical Group Ltd, London, UK) the program was compiled using the open-source GCC compiler.

Separately, a class for MATLAB (MathWorks, Natick, MA) was developed in version 7.14 (R2012a) on Windows 7. MATLAB is a high-level language and interactive environment for numerical computation, visualization, and programming, and is used in many BCI research groups.

User testing

For user testing, EEG signals were acquired using a commercially available, dry-electrode Emotiv EPOC headset (Emotiv; Australia), which comes preconfigured with 14 electrodes located over 10–20 international system positions AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4 using two reference electrodes. This arrangement has good coverage of the premotor and frontal regions of brain. The headset aligns, bandpass filters, and digitizes the signal at 128 Hz before transmitting it wirelessly to a personal computer or laptop.[4] The Emotiv Control Panel was used to process the signals and send them to the robotic arm application simulating key presses.

RESULTS

All source code has been released under the GNU GPL license v3.0. Source code and user instructions are available online (http://code.google.com/p/lynxmove-qt/)

Standalone software

LynxMove Qt is an open-source and cross-platform standalone software platform for robotic arm control. It has been tested successfully on Windows 7, Mac OS 10.7, and Ubuntu Linux 12.04. Its current version offers four operating modes: Buttons, sliders [Figure 1], BCI key commands [Figure 2], and ASCII commands [Figure 3]. The BCI key commands enable to move the arm with a customizable step size to a new position, depending on BCI input. The latter is fully configurable in any BCI software platform, and does not depend on a specific signal type (e.g. sensorimotor rhythms or steady-state visual evoked potentials).

Figure 1

LynxMove Qt for windows with slider control

Figure 2

LynxMove Qt for Linux with BCI key command control

Figure 3

LynxMove Qt for Mac with ASCII command control

MATLAB class

A separate MATLAB class (LynxArm.m) has been developed for implementation in custom MATLAB scripts that are used by many research groups. It does not offer a graphical user interface, but provides an object-oriented approach to connect to the robotic arm, and to move it using BCI key commands and ASCII commands with a user-configurable step size [Figure 4]. It has been tested successfully on Windows 7 using MATLAB R2012a.

Figure 4

LynxArm MATLAB class implementation on Windows

Brain computer interfaces (BCI) user testing

Three healthy volunteers succeeded in moving the robotic arm in 1° of freedom (base rotation) using LynxMove Qt on Windows. Brain signals were generated by actual hand movement or motor imagery. We did not quantify accuracy for this feasibility test, but both objective control and subjective feeling of control were better for actual hand movement. However, none of the volunteers felt being in control of movement using the standard Emotiv EPOC setup.

DISCUSSION

Several kind of robotic arms and exoskeletons have been applied in BCI research.[12457-91315-172021] Although exact cost cannot be derived based on the published articles, most of these arms are either expensive commercial devices (starting at a price around $25,000, like the iARM (Exact Dynamics, Didam, The Netherlands)) or custom-made devices that are not available to other research groups. Both options reduce accessibility of such devices to other research groups, thereby limiting further progress in this field. A low-cost device as the LynxMotion arm may not be suitable as an end-product for bodily disabled people, but as it is affordable to any group, it can help to expand research on BCI- guided robotic arm control in a cost-effective manner. LynxMotion offers a variety of robot arm construction kits that may be modified according to the needs within a specific research project.

Hochberg et al. described the use of a LynxMotion robotic arm for BCI purposes before,[6] but to date, no open-source and cross-platform software framework was available to control this device. This is in contrast with a variety of open-source platforms that are available for other BCI applications.[3] The platform that we present here fulfills this gap and can help to achieve the previously described goals.

Strengths and limitations

LynxMove Qt can be compiled for any major operating system (Windows, Linux, Mac OS X) and has been tested successfully on all of these. Due to its open- source nature, other research groups are free to use the software, and modify it to their own needs. BCI support is built- in using key commands, which allows for maximum compatibility with any other BCI software platform used for signal acquisition and signal processing. As it can run on any regular personal computer or laptop, a fully functional and customizable setup for robotic arm control for BCI-projects is available for less than $1000. For those groups working with MATLAB, a separate class is available to control the arm directly from MATLAB instead of using the LynxMove Qt platform.

In the current version of this software platform, the channels of the SSC-32 servo controller that the servos are connected to, are hardcoded for the default setup. A custom setup would require manipulation of the code, which is obviously possible. Nevertheless, a future release of the software should allow to manipulate this parameter directly. Further, the LynxMove window must have focus in order to receive key commands. This is another issue to be addressed for a future release. Optionally, integration into existing BCI platforms (such as BCI2000 or OpenViBE) could be tightened.

User performance

The Emotiv Control Panel offers very limited options to train the user, and no options for advanced signal processing or signal classification. This clearly limited user performance when attempting to control the robotic arm with EEG-based BCI directly. Current work is focused on improving user performance by connecting the Emotiv EPOC and LynxMove Qt using BCI2000, which offers processing and classification features besides signal acquisition. However, in contrast with a conventional 10-20 EEG setup, the Emotiv EPOC is more difficult to position in a standard manner, which reduces training effect.

CONCLUSION

We presented an open-source and cross-platform framework for BCI-guided robotic arm control that can help to expand this research branch in a cost-effective manner. The software runs on all major operating systems (Windows, Linux, Mac OS X) and from MATLAB, and is compatible with other BCI software platforms. The source code has been released under the GNU GPL license v3.0. Further instructions for installation and use are available online.