Neuromagnetic brain responses to other person's eye blinks seen on video.

Eye blinks, typically occurring 15-20 times per minute, rarely capture attention during face-to-face interaction. To determine the extent to which eye blinks affect the viewer's brain activity, we recorded magnetoencephalographic brain responses to natural blinks, and to the same blinks slowed down to 38% of the original speed. The stimuli were presented on video once every 2.3-6.2 s. As a control, we presented two horizontal black bars moving with the same time courses and the same extent as the eyelids in the blink video. Both types of blinks and bars elicited clear responses peaking at about 200 ms in the occipital areas, with no systematic differences between hemispheres. For the bars, these main responses were (as expected) weaker (by 24%) and later (by 33 ms) to slow-motion than normal-speed stimuli. For blinks, however, the responses to both normal-speed and slow-motion stimuli were of the same amplitude and latency. Our results demonstrate that the brain not only responds to other persons' eye blinks, but that the responses are as fast and of equal size even when the blinks are considerably slowed down. We interpret this finding to reflect the increased social salience of the slowed-down blinks that counteracted the general tendency of the brain to react more weakly and more slowly to slowly- vs. quickly-changing stimuli. This finding may relate to the social importance of facial gestures, including eye blinks.

Introduction

Spontaneous eye blinks usually occur 15–20 times per minute, interrupting visual input each time for 200–400 ms. Their well-established physiological role is to moisten and clean the cornea. But do eye blinks play other functional roles as well?

Blinking rate decreases during tasks that require visual attention (Oh et al., 2012), and video viewers tend to blink more during scenes that contain less relevant information (Nakano et al., 2009). Similarly, blinks cluster around pauses in speech seen on video, but only when the voice can be heard (Nakano & Kitazawa, 2010). Such inter-subject synchronization of eye blinks suggests that blinks play a role beyond corneal moistening, likely related to human social interaction. Eye blinks also affect the judgments that people make about others: those who blink very often are perceived as nervous or careless (Omori & Miyata, 2001).

Less is known about how the eye blinks of others activate the viewer's brain. In a recent scalp electroencephalography (EEG) study, a sequence of three images simulating eye blinks elicited several successive evoked potential deflections peaking at 100–600 ms (Brefczynski-Lewis et al., 2011). To investigate how the human brain reacts to natural blinks, we recorded magnetoencephalographic (MEG) brain responses to eye blinks presented via video. We also presented the same blinks in slow motion. The aim was two-fold: first, to determine whether these slow stimuli would evoke any brain responses; and, if so, to examine whether these responses would be weaker and more prolonged compared with responses to normal-speed stimuli, as would be expected on the basis of the general tendency of the brain to react weaker and later to stimuli changing at slower speed (for a review, see Heinrich, 2007). As control stimuli, we presented two horizontal black bars that had similar size, location and movement characteristics as the eyelids during the blinks.

Materials and methods

Subjects

Eleven healthy volunteers (five female, six male, age 21–55 years, mean age 26 years; 10 right- and one left-handed) participated in the experiment. The experiment conformed with the Code of Ethics of the World Medical Association Declaration of Helsinki [JAMA (2013), 310, 2191–2194], and all subjects signed an informed consent before participation. However, the study was not preregistered in a publicly accessible database before recruitment of the subjects because no such procedure exists in Finland. The MEG recordings had prior approval by the Coordinating Ethics Committee of Hospital District of Helsinki and Uusimaa (#28/13/03/00/11 and #95/13/03/00/08).

Stimuli and tasks

The stimulus was a video sequence comprising both normal and slowed-down blinks of a female staying immobile and looking at the camera. Only her face was visible (Fig.1). The video was constructed from these two video clips, both of them repeated 102 times in a random order and without breaks in-between, with the restriction that the same blink (either normal or slow) did not occur more than twice in a row. The normal-speed clip lasted for 2718 ms, with a single voluntary 452-ms blink starting at 1000 ms. The slow-motion clip was generated by slowing down the normal-speed video to 38% of the original speed, so that it lasted for 7135 ms, with a 1200-ms blink starting at 2634 ms.

Figure 1

Schematic presentation of the eye blink (top images) and bar (bottom images) stimuli. Durations of events in normal-speed (top scale) and slow-motion (bottom scale) videos are indicated: eyes open (OPEN); blink (BLINK); and eyes fully open again (OPEN). Dashed lines were added on top and below the bars and the eyes to indicate the similar movement amplitude for eye blinks and bars to improve the readability of this figure; the line was not present in the stimuli presented to the subjects.

To keep the subject's attention, five other normal-speed videos (2038–2686 ms in duration) of facial expressions (aimed to convey ‘thinking’, ‘agreeing’, ‘confused’, ‘smiling’ and ‘disgusted’, performed by the same person whose eye blinks were shown) were inserted between the blink video clips once after every 34 stimuli. The subjects were told to memorize the expressions and to recognize them after the session from a list of 10 possible expressions. A blank gray screen appeared for 1 s before and after each expression video. The complete stimulus sequence lasted for about 17 min.

The original eye-blink video was recorded with a high-speed camera (Fastec InLine 1000, Fastec Imaging, San Diego, CA, USA) at 500 frames/s in front of a white background. The videos were presented to the subject with Experiment Builder™ software v. 1.10.1 (SR Research, Ottawa, ON, Canada). Due to video-projector restrictions, the presentation frame rate was 60 frames/s, so that only every seventh frame of the normal-speed stimulus and every third frame of the slow-motion stimulus was shown. Both videos appeared smooth and natural. The background color and the luminance of the screen were matched to the background of the video.

The videos were presented on a screen (width 72 cm, height 54 cm) placed 1 m in front of the subject's eyes, so that the size of the face in the video was approximately 14 cm × 20 cm (visual angle 11.5 deg × 8.0 deg) in the center of the screen, and the size of an eye was approximately 2.6 cm × 1.2 cm. The eyelid was moving vertically approximately 1.2 cm during the blink. The Michelson contrasts between different eye features (cornea, eye white, eyelashes and eyelid) varied between 0.5 and 0.8, with an average contrast of 0.65.

We aimed to create control stimuli that would have the same movement characteristics as the eye blink, but would not resemble human eyes or facial gestures. Therefore, we showed videos of two horizontal black bars (constructed and presented at 30 frames/s) moving up and down on a gray background in the middle of the screen; the sizes, locations and movement time courses of the bars were matched to those of the blinks (Fig.1): two bars (2.6 cm × 0.035 cm) were moving 1.2 cm vertically on the screen with a time course similar to that of the blinks. During the ‘open’ and ‘closed’ periods, the bars remained at the highest or lowest position, correspondingly. The Michelson contrast between the bars and the background was 0.6.

Each experiment started with a control condition in which 102 normal-speed and 102 slow-motion bars were pseudo-randomized similar to the stimuli in the blink videos (but with no facial expressions in-between). The subjects were informed that a second visual task would follow, but its content (eye blinks) was not mentioned. After having seen the bar videos, the subjects were asked to freely describe what (if anything) the bar motion had resembled in their opinion.

Data acquisition

Magnetoencephalographic signals were recorded with a 306-channel whole-scalp neuromagnetometer (Elekta Neuromag Oy, Helsinki, Finland) in a magnetically shielded room (Euroshield, Eura, Finland) in the MEG Core of the Brain Research Unit of Aalto University. This MEG device comprises 102 sensor units, each containing two orthogonal planar gradiometers and one magnetometer. The signals were bandpass-filtered to 0.03–200 Hz, digitized at 600 Hz and averaged online, time-locked to the beginning of the blinks in the videos. The analysed period extended from −500 to 2000 ms (normal) or to 4000 ms (slow) from the blink onset.

Vertical electro-oculogram (EOG) was measured between electrodes located above and below the left eye, and horizontal EOG between electrodes on the left and right lateral eye canthi. Both EOGs were averaged, time-locked to stimulus onsets to later verify that artifacts from the subjects' own blinking did not contaminate the observed brain responses.

Four head-position-indicator coils were attached to the subject's scalp, and head coordinates were registered with a 3D digitizer by identifying the locations of the coils together with the locations of three anatomical landmarks (nasion and left and right preauricular points) and some additional points on the scalp. At the beginning of each measurement block, the position of the subject's head in the MEG helmet was measured by inducing weak currents into the indicator coils.

The subject was instructed to sit still and relax, and to observe the stimuli attentively. No instruction was given about eye blinking or where to direct the gaze. The whole experiment lasted for about 1 h, including the preparation, MEG measurement and answering of the questionnaire about the facial expressions observed during the measurement.

Analysis

MEG recordings

The MEG data were preprocessed with the temporal signal-space-separation method (Taulu & Simola, 2006; Taulu & Hari, 2009) using the MaxFilter™ software (version 2.2; Elekta Neuromag Oy, Helsinki, Finland). We computed vector sums over each pair of planar gradiometers to minimize the effect of source orientation in the analysed waveforms: the signals from the two orthogonal planar gradiometers in each MEG sensor were squared, summed, and finally the square root was computed of the sum. Thus, the resulting signals were always positive.

We next computed areal averages of the vector sums over the posterior parieto-temporo-occipital brain regions, separately for each subject's left hemisphere (nine sensor pairs), central area (eight pairs) and right hemisphere (nine pairs; for area selection, Fig.2). For each subject, the areal mean signals were normalized according to the individual maximum in any of the three areas (over all conditions); the maximum was allowed to occur at any time after the start of the blink in the video during the 2-s (normal-speed stimuli) or 4-s (slow-motion stimuli) analysis periods. We compared areal mean signals instead of carrying out source analysis because the field patterns of the long-lasting signals were not dipolar and did not allow reliable source modeling.

Figure 2

Mean ± SEM responses across all 11 subjects to normal-speed (blue lines and light blue bands) and slow-motion (red) blinks (top) and bars (bottom). The areal average responses are shown separately for the left-hemisphere, central and right-hemisphere posterior regions (see the schematic heads above). The duration of the normal-speed (light blue) and slow-motion (light red) stimulus is shown on the horizontal axis.

The peak amplitudes of the first prominent responses were measured with respect to a 500-ms baseline preceding the blink onset; during the baseline period, the immobile face was visible. Before the amplitude measurements, the signals were low-pass-filtered at 30 Hz.

To define the response onset, we searched for the best-fitting slope of the increasing areal mean response before the first prominent peak (polyfit function in Matlab version 8.0.0.783; MathWorks, Natick, MA, USA); before slope fitting, the peak amplitudes were normalized to 1. Response onset was defined as the crossing point of the slope and the mean + 2 standard deviations of the baseline amplitude.

Results

Behavioral results

After the bar videos, we asked the subjects to freely elaborate whether the bars had reminded them of anything. Ten out of 11 subjects said that the bars resembled human or cartoon character's eyes, and one subject mentioned old computer games; two additional notes referred to a computer game and a nodding head.

After the blink videos, all 11 participants remembered having seen a ‘smiling’ and a ‘thinking’ face, and 10 subjects recalled the ‘confused’ and nine the ‘disgusted’ expressions. The ‘agreeing’ expression was rather difficult to describe verbally and, accordingly, was recognized only by four subjects. Altogether, the participants recognized 4.3 ± 0.1 (mean ± SEM) out of the five facial expressions.

Brain responses

Figure2 shows the areal mean responses for normal-speed (blue) and slow-motion (red) blinks (top) and bars (bottom) separately in the left-hemisphere, midline and right-hemisphere posterior parieto-temporo-occipital regions, where the responses to both types of stimuli were most prominent. The responses did not differ between these areas in any of the measured parameters (onset time, rising slope, peak latency, peak amplitude), and we therefore carried out further analyses on signals averaged across these three areas.

Figure3 depicts the signals averaged over the three areas for blinks and bars. Responses to normal-speed eye blinks started at 151 ± 15 ms (mean ± SEM) and peaked at 196 ± 16 ms. The corresponding values for slow-motion blinks were 157 ± 17 and 210 ± 14 ms. Thus, the responses to normal and slow-motion blinks did not differ either in onset or peak latencies, nor did their rising slopes differ. The normalized peak amplitudes were 44.8 ± 7.1% (out of individual response maxima) for normal-speed blinks and 49.2 ± 7.2% for slow-motion blinks; the maximum signals (over all stimulus types) were obtained later than the first peak (on average, at 359 ± 39 ms).

Figure 3

Mean ± SEM responses across all 11 subjects to normal-speed (blue lines and light blue bands) vs. slow-motion (red) blinks (top) and bars (bottom), calculated over 52 occipital planar gradiometer channels. The duration of the normal-speed (blue) and slow-motion (red) stimulus is shown on the horizontal axis.

In contrast to responses to blinks (that did not differ between normal-speed and slow-motion stimuli), responses to bars started 24 ± 7 ms earlier to normal-speed than slow-motion stimuli (main effect for speed F1,10 = 11.2, P = 0.007), but the rise times (slopes) did not differ. The responses peaked on average 33 ms earlier to normal-speed than slow-motion bars (231 ± 3 vs. 264 ± 12 ms; F1,10 = 8.6; P = 0.015), and the normalized responses were on average a third stronger to normal-speed than slow-motion bars (normalized values 76.1 ± 6.6% vs. 58.2 ± 5.6%; main effect for speed F1,10 = 18.2; P = 0.002).

Compared with responses to eye blinks, the responses to bar stimuli (Fig.3, bottom panel) started significantly later. The delay was 34 ± 13 ms for normal-speed and 52 ± 19 ms for slow-motion stimuli; an anova for stimulus (bar vs. blink) × speed (normal speed vs. slow motion) showed a main effect for stimulus type (F1,10 = 17.4, P = 0.002). Responses peaked later to bars than blinks. The delay was 35 ± 14 ms for normal-speed and 54 ± 19 ms for slow-motion stimuli, respectively; an anova for stimulus × speed showed a main effect for stimulus type (F1,10 = 20.1, P = 0.001).

Discussion

Brain activity related to observed eye blinks vs. to socially irrelevant moving objects

By recording MEG brain responses to eye blinks shown on video, we found that other person's eye blinks elicit clear responses in the viewer's brain. Interestingly, neither the strengths nor the onsets or peak latencies differed between the blinks shown at normal speed vs. slow motion, with the speed decreased to 38% of the original. In contrast, responses to bars used as control stimuli were weaker and delayed when the stimulus speed was slowed down. This latter effect agrees with earlier findings that brain responses decrease when light-spot stimuli move at slower speed (Kawakami et al., 2002). Accordingly, transient cortical responses to simple auditory stimuli peak later to stimuli with slower rise times (for a review, see Hari, 1990). Cortical responses to blinks thus behaved clearly differently than expected on the basis of simple physical characteristics, as the strengths and peak latencies did not differ between considerably slowed-down and normal-speed eye blinks.

Social significance of other person's eye blinks

Blinks are socially relevant for the perceiver as they can give clues about the mental and physical state of the other person: blinking rate decreases (compared with silent rest) during cognitively demanding tasks (Fukuda, 1994; Bentivoglio et al., 1997; Oh et al., 2012), but rises during conversation (Bentivoglio et al., 1997) and during prolonged wakefulness (Barbato et al., 1995). Reduced blinking rate can signal subjective salience of an observed object, and the effect is present already in children (Shultz et al., 2011). Moreover, people tend to blink less while they tell a lie and, immediately after the lie is told, they blink more again (Leal & Vrij, 2008).

Blinking rate can be disturbed in several brain and mental diseases, such as schizophrenia (Chan & Chen, 2004), Parkinson's disease (Agostino et al., 2008) and attention-deficit-hyperactivity disorder (Caplan et al., 1996); medication can contribute to these changes. People with autism do not synchronize their blinking with the speaker seen on a video, in contrast to what healthy subjects do (Nakano et al., 2011), suggesting that reacting to other person's eye blinks is one sign of successful behavioral inter-subject synchronization.

The slow-motion eye blinks may have even higher salience for the perceiver than the normal blinks because of their unusual time course that makes the person appear drowsy or odd, thereby adding social significance to the expression. Attention is known to enhance cortical responses during visual discrimination (Spitzer et al., 1988) and spatial-attention tasks (Kanwisher & Wojciulik, 2000). Moreover, socially relevant stimuli, such as emotional vs. neutral faces, elicit stronger EEG responses in posterior temporal brain regions even when the perceivers concentrate on, for example, gender and not emotion (Sato et al., 2001). Therefore, it is possible that enhanced attention to slow-motion blinks contributed to the short latencies and large amplitudes of the responses.

Brain responses to eye blinks vs. to other observed facial gestures

So far, brain responses to observed eye blinks have been studied only rarely. In a previous EEG experiment, images of closed eyes were shown for 33 ms in-between open-eyes baseline images so that the sequence of pictures did not include any real eyelid movements (Brefczynski-Lewis et al., 2011). These ‘blinks’ elicited robust occipito-temporal P100 and right-hemisphere-lateralized N170 responses, which did not differ from responses to gaze movements and eye closures also presented within the same stimulus sequence. In our study, the natural blinks were considerably longer than 33 ms (lasting for about 450 ms), and the responses to them peaked about 200 ms after blink onset in the parieto-temporo-occipital cortex.

The human brain is known to react to many observed facial gestures, including gaze shifts and mouth movements that activate, in addition to the early visual cortex, for example, the superior temporal sulcus (STS) and the MT/V5 area (for a review, see Puce & Perrett, 2003). Social context or meaning of the observed facial gestures modifies the viewer's brain responses. For example, the 150–160-ms MEG responses arising from the MT/V5 region were stronger to gaze-change stimuli, composed of two images generating a percept of apparent movement, when the gaze shifted towards rather than away from the viewer (Watanabe et al., 2006). On the other hand, the 170-ms temporal-lobe EEG response (N170) was weaker to direct than averted gaze (Puce et al., 2000). In a more complex social setting, where the subjects were viewing images of three faces, N170 was unaffected, but the later EEG responses (P350 and P500) were modified by the scenario involving social attention (Carrick et al., 2007).

In addition to eye gaze, various mouth expressions can activate several brain regions. In an MEG study, static face images with both verbal and non-verbal mouth shapes triggered an activation sequence from the occipital visual areas to the STS, then to the inferior parietal cortex, and finally to the inferior frontal cortex and the primary motor cortex (Nishitani & Hari, 2002). The full activation sequence took about 220–250 ms.

Another example of how facial movements that are potential social cues can modulate the observer's brain responses comes from a functional magnetic resonance imaging study of yawning: videos of yawning faces elicited significantly stronger activity in the right posterior STS and in the anterior STS of both hemispheres than did non-nameable mouth movements (Schürmann et al., 2005).

In our study, both the socially relevant eye blinks and the moving bars used as control stimuli (but often interpreted as eyelid movements), elicited similar temporo-occipital responses peaking about 200 ms after the movement onset. A clear difference, however, was seen in the reactivity to stimulus speed: only responses to blinks, and not to bars, differed between the normal-speed and slowed-down stimuli.

Caveats

Our blink and control (bar) stimuli differed in many visual features. Still, 10 out of 11 subjects reported that the bars reminded them of human eye blinks, probably reflecting the predisposition to perceive faces in various visual patterns. Good examples are the famous paintings by Giuseppe Arcimboldo (1526–1593) persuading viewers to perceive faces in paintings containing only vegetables, fruits, fish, miniature humans or inanimate objects. Accordingly, it is very difficult to create control stimuli for human blinks that would be similar to the original stimuli in low-level physical features but would not elicit an impression of blinking or a human face.

Although we saw that the responses to eye blinks started and peaked earlier compared with similarly moving bars, this effect could be due to differences in physical stimulus features. Therefore, drawing any conclusions from those direct comparisons between responses to blinks vs. bars would not be well-founded. Nevertheless, the differences between blink and bar stimuli cannot explain the differences between responses to normal and slow-motion stimuli of the same type.

Conclusion

Taken together, our results show that normal eye blinks are clearly registered in the viewer's brain, and that the responses remain equally fast and strong even when the speed of the blinks is considerably slowed down. This behavior contrasts responses to other similarly moving stimuli as they become weaker and delayed when the speed of the stimulus decreases. These findings support the view that eye blinks of other persons are socially relevant behavioral events and they should be adequately considered when studying social interaction.