EEG Signature of Object Categorization from Event-related Potentials.

Human visual system recognizes objects in a fast manner and the neural activity of the human brain generates signals which provide information about objects categories seen by the subjects. The brain signals can be recorded using different systems like the electroencephalogram (EEG). The EEG signals carry significant information about the stimuli that stimulate the brain. In order to translate information derived from the EEG for the object recognition mechanism, in this study, twelve various categories were selected as visual stimuli and were presented to the subjects in a controlled task and the signals were recorded through 19-channel EEG recording system. Analysis of signals was performed using two different event-related potential (ERP) computations namely the "target/rest" and "target/non-target" tasks. Comparing ERP of target with rest time indicated that the most involved electrodes in our task were F3, F4, C3, C4, Fz, Cz, among others. ERP of "target/non-target" resulted that in target stimuli two positive peaks occurred about 400 ms and 520 ms after stimulus onset; however, in non-target stimuli only one positive peak appeared about 400 ms after stimulus onset. Moreover, reaction times of subjects were computed and the results showed that the category of flower had the lowest reaction time; however, the stationery category had the maximum reaction time among others. The results provide useful information about the channels and the part of the signals that are affected by different object categories in terms of ERP brain signals. This study can be considered as the first step in the context of human-computer interface applications.

INTRODUCTION

The neural activities of human brain generate one of the most significant biomedical signals in which they are the source of encoded information about the human surrounding and the human internal states. Understanding characteristic of neural data can be used to decode the human thoughts and actions and also can be applied for diagnosing the brain's neuropsychological diseases. For these applications, brain data collection, preprocessing and analysis methods has become one of the most challenging items in this field in recent decades. Among different non-invasive methods of brain signals acquisition such as magnetoencephalogram, functional magnetic resonance imaging and electroencephalogram (EEG), EEG is more appropriate for the brain data collection because of low cost of experiment design and data collection and its high temporal resolution than other procedures.[12] The inventor of the human EEG signals was Hans Berger, in 1929 in which he was able to measure, amplify and plot the human brain's electrical activities by placing an electrode on the scalp.

Human brain signals can be used for the diagnosis of the brain disorders by visual investigation or by using more sophisticated automatic analysis methods of the EEG signals. The EEG signals consist of five major frequency rhythms which are called delta (d), theta (θ), alpha (α), beta (β), gamma (g) bands indicating frequencies of (0.5-4 Hz), (4-7.5 Hz), (7.5-15Hz), (15-30Hz), and (30-50Hz) respectively. The EEG signal can easily be mixed with noises and artifacts from physical movements or environment. Preprocessing stages for the EEG signals consists of artifact detection and correction techniques. The discrete wavelet transform, independent component analysis and principal component analysis are the most popular and accurate strategies for this aim. To process and analyze the EEG signals, various methods have been proposed. Event-related potential (ERP) is one of the well-known way of signal analysis methods which was introduced in 1964 for the first time.[34] ERP has been useful tool in both psychiatry and neurology and widely applied in brain-computer interface (BCI) applications. ERP is the sum of electrical responses of EEGs in response to some cognitive events. The responses typically are generated during external stimulations such as visual or auditory that cause voltage fluctuations in EEG brain signals. The waveforms of ERPs are characterized across three key parameters: Amplitude, latency, and distribution over the scalp.[5] ERP has been used for many clinical applications as well. For example, Coben et al.,[6] in their work analyzed and compared the ERP components like N100, N400 or P300 in diversity of psychiatric diseases such as Alzheimer's diseases or other cases with healthy people. Visser et al.[7] and Cosi et al.[8] did a similar study in non-organic behavioural disorders in elderly people. Additionally ERP is a method to discuss about brain activities in a specific task in order to recognize how EEG signal potentials change when a stimulus is observed and how long it takes for the human to give a correct response to the stimuli. Evaluating the ERP of EEGs can produce an alphabet of brain language which can enable researchers to read the signals and understand the different states of the human brain. Various experimental design have been considered by researchers such as category-based brain activity for alive objects against man-made objects, and for different particular object classes such as furniture, faces, animals, fruits, vegetables, buildings, tools and parts.[9101112] Moreover, ERP of visual, verbal and auditory stimulus modalities, living versus non-living have been analyzed.[1314] Simanova et al.,[15] in their study investigated the possibility to identify conceptual representations from event-related EEG based on three different forms of object presentation: Object spoken name, object visual representation and written name of object. They achieved high classification performance with accuracy of 89% for drawing objects. Jeffrey and Bruno (2003)[16] in their study worked about the time of human visual system to recognize objects. They designed ERP experiment series in order to measure the time course of electrophysiological correlates of object recognition. They found two distinct components in the ERP recorded during categorization of natural images. One was an early presentation-locked signal arising around 135 ms after stimulus onset that was present when there were low-level feature differences between images. The other was a later, recognition-related component arising between 150 ms and 300 ms after stimulus onset. Jeffrey and Bruno (2005)[17] also designed three sets of ERP experiments to specify processes of the target minus non-target difference signals seen in visual cued-target paradigms. They proved that the same difference signals were obtained when the target match was made to word stimuli as well as to object stimuli. Additionally signal amplitude had reverse relation with task difficulty.

It has also been shown that objects and their sensory or functional attributes (such as tool-associated actions) activate the same neural regions, suggesting that these regions are implicitly involved in concept representation.[18192021] Martin, 2007[22] in his object recognition study proved that object's features coded very rapidly and played different functional roles while color and extra contours and edges delay it.

Martinovic et al.,[23] in their work studied about detection and identification of objects a visual perception. They compared the role of luminance and chromatic information between full-color and reduced-color object. In a subsequent electroencephalographic ERP experiment, advantages in accuracy and high-level discrimination were found for full-color stimuli over the reduced-color stimuli.

Rossion and Boremanse (2011),[24] showed that how the human brain discriminates complex visual patterns, such as individual faces testing sensitivity to individual faces using steady-state visual-evoked potentials (SSVEPs). They showed a large response at the fundamental stimulation frequency (3.5 Hz) over posterior electrode sites using fast fourier transform (FFT) of EEG. Nazari et al.[25] in their study proved that several differences in the P300 component are observed when responses must be executed in the “go/no-go task”. They examined the peak amplitude and latency of Go-P300 and Nogo-P300 component in healthy children. The P300 component was measured at frontal (F3, Fz, F4) and parietal (P3, Pz, P4) regions. The results displayed higher P300 amplitude in the Go relative to No-Go at parietal region. In addition, decrease in P300 latency was observed at the frontal in comparison to parietal region. Dering et al.,[26] in three experiments, measured the amplitude and latency of the mean P1 and N170 in response to faces, cars, and butterflies, cropped or morphed. The N170 was sensitive to cropping but did not differentiate frontal views of faces and cars. The P1 amplitude was larger for faces than objects. The authors concluded that P1, not N170, is a reliable face-sensitive event.

The aim of the current study is to clarify differences between ERP of target/rest and target/non-target signature of signals such as time courses and ERP signal forms. To this aim, much more categories are used in comparison with previous studies; consequently it is more complicated to study ERP signals. This makes the scenario similar to what the human faces in everyday life, so the results are more reliable.

Experimental setup of our study consists of visual stimuli presentation of twelve different categories (clothing, animals, foods, flowers, fruits, human body organs, transportation devices, dolls, electronic devices, jewelry, stationary, buildings, and scenes). We used the EEG recording system in order to measure ERP of the human brain. All of the participants in our study were in healthy condition and the patients are not addressed here.

MATERIALS AND METHODS

Participants

Ten human volunteers, 2 females and 8 males, mean age 25, participated in the study. All were right-handed except one, and reported that they did not have any psychological or neurological abnormalities. All participants were gifted for their participation and they were informed about the task paradigm before going to the experiment booth.

Stimuli

Twelve different categories of clothing, animals, foods, flowers, fruits, human body organs, transportation devices, dolls, electronic devices, jewelry, stationary, buildings, scenes were selected in which each category included five different images. All images were picked up from Internet; color images with same resolution of 600 × 800 pixel and same difficulty. Picture difficulty means that how easy it is to recognize the picture at the first glance and put it in a correct category. Monitor parameters such as luminance and contrast were normalized. Whole stimuli were presented in the exact center of the screen with black background. The monitor was 17-inch LCD with 1280*800 resolutions. Figure 1 shows some sample images are used for the experiment. Process was divided in two parts of visual tasks, six categories were presented in task Part 1 and the next six categories in Part 2 with a gap of three minutes in between. This was done in order to let subjects’ eyes to rest and reduce EEG eye-blinking artifacts. Figure 2 indicates the task paradigm.

Figure 1

Different three normalized sample images (not full-size images are shown here)

Figure 2

Task paradigm including the first and the second part of the experiment with timing of presentation

Task of Study

The experiment which was designed for this study was “cued-target” with “Go/No-go” ERP categorization task. Cued-target means the category name (target cue) was displayed on the screen before starting the trial to notify the participants about the target images. Target cue remained on the screen for 1 s, and after an additional 900 ms delay, the images were presented. Each image was presented for 700 ms, the rest time or blank screen interval between two images was 800 ms and 5 s rest between the two different categories. The subjects were asked to press left button of mouse with the right hand index finger to deliver their responses as quickly as possible and to delay their blinks to 800 ms in the resting time. All images were centrally shown on a CRT LCD monitor. Image presentation was controlled via a computer Pentium 4 with 512 MB RAM and 40 GB HDD PC running on the PsyTask software. PsyTask is a software for visual/auditory stimuli presentation and psychophysiology investigations. It works in conjunction with WinEEG software. Viewing distance was 75 cm from computer screen. In “go/no-go” task subjects had to left-click if they saw target images and did nothing on non-target images. They were given a maximum of 700 ms time to respond, and after this period any responses were considered as an incorrect response. Subjects observed 360 pictures in a two-section experiment [Figure 2].

EEG Recording System

The subjects wore a 19-channel electrode cap consist of Fp1, Fp2, F3, F4, C3, C4, P3, P4, F7, F8, T3, T4, T5, T6, Fz, Cz, Pz, O1, O2, additionally A1 and A2 connected to the left and right earlobes, respectively, as the reference electrodes [Figure 3]. Recording electrodes were selected from the 10-20 set of channel positions. Subjects performed the experiment in an electrically shielded and sound-damped booth. The EEG signals were sent to A/D converter after they amplified by MITSAR hardware. The electroencephalographic PC-controlled system “Mitsar-EEG” is purposed for acquisition, storage at personal computer's HDD, processing, displaying at PC's display and printing out of electroencephallographic signals. Recording was done at 500 Hz sampling frequency. Figure 4 shows the EEG recording system which it is comprised of two separate and synchronized PCs, one PC was for the psycho-task presenting stimuli and the other was for recording signals operating on WinEEG software, adding labels to the raw signals. Labeling is one of the significant methods of ERP computing where the WinEEG/Psycho-task has this capability. In this study label of ending one means non-target images and ending two indicates the target images that should be responded by the subjects. For example the integer 12 means an image of category 1 was presented that should be responded, and eleven means an image of category 1 was presented that should not be responded, so the integer number before the last integer indicated the category of the image.

Figure 3

Standard layout for electrode placement in a 19-channel electroencephalogram recording system with A1 and A2 references link to the earlobes. Red-marked electrodes are most involved electrodes in our task

Figure 4

Electroencephalogram recording system including two different PCs, an amplifier (MITSAR), mouse, 19 electrodes

EXPERIMENTAL RESULTS AND DISCUSSION

In current study, ERP of twelve categories through a 19-channel EEG system were computed and total results were averaged among participants using WinEEG software. Two different analogies was done: “Target/rest” and “target/non-target”. In order to localize active electrodes during task paradigm, ERP of all correctly responded stimuli of 12 category from 8 subjects was measured and averaged (because of large amount of artifacts appeared in the data from subject 4 and 6, they were not considered for the analysis) and compared with the rest time. This is called “target/rest” results. Moreover, the other ERP applied to the correctly responded target images in comparison to non-target images. This was done to recognize difference in brain signals images when they play roles as target images with the same images when they are in non-target order. Figure 5 shows the ERP of 19-channel “target/rest” with topographical maps and Figure 6 indicates ERP of only Cz (because of its highest voltage among other electrodes) electrode with peak latencies mapping. Figure 5 is related to ERP of “target/non-target” with full mapping from 100 ms to 800 ms. In target-rest case the “rest” ERP is after the “target” ERP, but in target/non-target both ERPs are in comparing mode. It can be clearly seen from Figure 5 that the most involved electrodes during visual task compared with the rest time are F3, F4, C3, C4, Fz, Cz. The Frontal electrodes F3, F4 and Fz are mostly related to judgment and problem solving wherein our subjects are in judgment state during categorization of objects. C3, C4, Cz are sensory-motor electrodes measuring muscles output, and because the exemplars use the left click in the task, these electrodes have high amplitude. The three components in the “target/rest” can be obviously seen from Figure 6: N248, P390 and P526. N248 have the most distribution on O1 and O2 channels which are related to the occipital visual state and have are negative. P390 is due to attention of subjects in which at 390 ms, stimuli were mentally recognized as target images by subjects and they responded to target images in about P526, therefore this component is related to the reaction or intention to respond. Figure 7 indicates “target/non-target” ERP unmatched features in target ERP. As it mentioned, it has two positive peaks compared with non-target ERP which has only one positive peak due to no reactions of subjects.

Figure 5

(a) Event-related potential of 19 scalp electrodes from 100 ms to 1400 ms including target and rest time averaged from 8 participants bar below the graphs indicates time points where waveforms differ significantly (P < 0.001), (b) topographical mapping of the brain activities from 100 ms to 1400 ms that indicates active electrodes during visual stimulation (F3, F4, Fz, C3, C4, Cz) versus inactive one (the remaining electrodes) with low amplitude amount in rest time

Figure 6

Occurring three peaks in target stimuli event-related potential: N248, P390, P526 with their related mapping

Figure 7

(a) “Target/non-target” event-related potential from 100 ms to 700 ms for F3, F4, Fz, C3, C4, Cz channels, (b) The activity of the Cz channel zoomed. Two peaks appeared in target but “non-target” only one peak can be seen due to its excluding reaction intention

Reaction Time

The reaction times of the 8 subjects were computed and listed in Table 1, from lowest to highest, in order to distinguish between all 12 categories. The results indicate that the category of flower has fastest reaction time with 392.2 ms and lowest standard deviation of ±81.2 ms (P < 0.001). In contrast, the stationery category has the highest time of reaction with 458.1 and maximum standard deviation of ±156.4 ms (P < 0.001). All statistical analysis was done by one way analysis of variance (ANOVA-1 test).

Table 1

Reaction times for 12 categories averaged over all subjects

DISCUSSION

In this study the EEG signals of eight subjects during visual stimuli, including various categories, was analyzed using the ERP analysis method. Our task is composed of two tasks discussed in previous study of Jeffrey and Johnson (2003), (cued-target and single category), however, our task comprises 12 categories; in contrast they had two kinds of pictures (animal images and nature images). In this study we indicated three major times of ERPs (N248, P390 and P526) which shows the observation, cognition and response, respectively. In comparison with Jeffrey and Johnson (2003) they proved signal arising around 135 ms and the other is a later, recognition-related component arising between 150 ms and 300 ms. Unlike the early component, the latency of the later component co-varies with the subsequent reaction time. Additionally Jeffrey and Johnson (2005) reported that signal amplitude decreases with increasing the pictures difficulty, while here we selected colored pictures with same difficulty level., Gruber, Muller (2008)[27] in their object recognition task have shown objects features coding and with extra features like color and edges to simulate delay of recognition, however, Simanova, Van Gerven, Oostenveld and Hagoort (2010)[15] investigated with the highest accuracy of 89% for labeled pictures and spoken stimuli. Based on these references, we have chosen colored images which are a more realistic scenario quite similar to real life images. Van Hoogmoed, et al.,[28] in their electrophysiological study compared five ERP scalp distribution mapping in different states (side change, depth change, disappearance, identity change and switch) for three N2, N3, and P3 latency windows. This study shows different activation area of brain scalp, in contrast we obtained same activation for all categories in different colors. The significant neural activity of brain was seen mostly in F3, F4, C3, C4, Fz, Cz electrodes. Comparing ERP of “target” with ERP of “non-target” images resulted in one positive peak versus two positive peaks for ERP that resulted from the “non-target” vs. “target” task. Additionally reaction time of 12 categories was measured and averaged through all subjects indicating that the flower category had the lowest reaction time while the stationery had the maximum reaction time. In terms of the nature of ERP, it is an averaged signal which does not give further information of exactly what happened when a picture was presented, so it is difficult to classify different categories using ERP information, however, it provides the changes that occurred due to different presentations, so it can provide useful information on selecting the proper channels and proper part of the signals for categorization.

Further work ERP of 12 different categories can be computed and compared with each other to understand differences in various images of grouping with much more details of processing to produce a brain language and application for BCI technologies.

BIOGRAPHIES

Mohammad Reza Daliri is an assistant professor with the Department of Biomedical Engineering, Faculty of Electrical Engineering, Iran University of Science and Technology (IUST), Tehran, Iran. Before joining the IUST, he was a postdoctoral researcher in Cognitive Neuroscience Lab., Germant Primate Center (DPZ), Goettingen, Germany. He has a PhD degree in Cognitive Neuroscience from the International School for Advanced Studies (ISAS/SISSA), Trieste, Italy. His main research interests are computational and cognitive neuroscience, vision in human and machine and pattern recognition

E-mail: daliri@iust.ac.ir

Mitra Taghizadeh received the B.S. degree in electronic engineering from Department of Electronic Engineering, Islamic Azad University, Tabriz, Iran in 2007. She has received two M.S. degrees in ICT and Mechatronics Engineering from Department of Computer Science, Virtual Center, Iran University of Science and Technology (IUST), Tehran, Iran and Department of Mechanical Engineering, Islamic Azad University, Tabriz, Iran in 2012, respectively. Her research interests include analysis of brain signals and EEG-based brain-computer interface development and pattern recognition.

E-mail: mitra.tagizadeh@gmail.com

Kavous Salehzadeh Niksirat received the B.S. degree in electronic engineering from Department of Electronic Engineering, Islamic Azad University, Tabriz, Iran in 2007. He received M.S. degree in Control Engineering from Department of Control Engineering, Science and Research Branch, Islamic Azad University Tehran, Iran in 2011. His research interests are Medical and Rehabilitation Robotics, Haptic and Teleoperation, Pattern Recognition and Brain Signals analysis

E-mail: kavus.salehzadeh@gmail.com