Altered spatial frequency content in paintings by artists with schizophrenia.

While it is difficult to imagine the way someone with mental illness perceives the world, paintings produced by mental illness sufferers with artistic talents offer a hint of this experience. Here we analyze these images in terms of statistics related to low-level visual processing. It is known that art in general possesses regular spatial frequency amplitude spectra, probably due to factors including luminance compression, approximation of natural scene spatial statistics, media, and aesthetics. Whatever the contributions of those factors may be, would the same ones apply for artists with schizophrenia? We find that spatial frequency content in paintings by five artists with schizophrenia or schizoaffective disorder is significantly different from that found in a large sample of art by painters without schizophrenia, while other basic spatial and intensity statistics are not different for the two groups. In particular, amplitude spectrum slopes are significantly steeper for paintings by artists with schizophrenia. A separate study of the works of one artist diagnosed with schizoaffective disorder confirmed these findings and showed no effect of medication type on amplitude spectrum slope. We suggest that these results support the notion that people with schizophrenia show decreased contrast sensitivity at low spatial frequencies. If people with schizophrenia cannot perceive low frequencies at the same level of contrast as that at which healthy individuals can, it follows that on average they will portray such components with higher contrast, resulting in steeper spectra.

Introduction

It is difficult to imagine how someone with a mental illness perceives the world. Paintings produced by mental illness sufferers with artistic inclinations offer a hint of this experience. In this paper we demonstrate that cautious analysis of such images can help advance current understanding of perceptual deficits in mental illness sufferers in ways that complement traditional approaches. Specifically, we consider basic image statistics in paintings by artists with schizophrenia in comparison with paintings by artists without schizophrenia.

Advances in visual neuroscience make clear that the design of visual systems is intimately tied to the regular statistical structure of the natural visual environment. Retinal and cortical neural coding strategies in particular appear to exploit statistical regularities in natural scenes [see Simoncelli and Olshausen (2001) for a review]. In turn, artworks are a reflection of the environment—one that is produced specifically for viewing by the human visual system—and recent work has shown that basic statistics of such images may be related to efficient coding of natural scenes [see Graham and Redies (2010) for a review]. For example, art (including abstract art) and natural scenes share roughly scale-invariant spatial frequency amplitude spectra: as a function of frequency, amplitude falls as 1/fp where p is approximately 1 (Graham and Field 2007, 2008a; Redies et al 2007). This finding suggests that artwork is shaped by fundamental neural processing strategies adapted to natural scene regularities. Moreover, because art is created for the human eye, it may also contain regularities that are related to efficient coding of higher-level perceptual information, such as similarity and preference (Graham et al 2010a, 2010b).

Given that regularities in basic image statistics of artwork provide new insights into visual processing in healthy adults, it is natural to examine statistical regularities in art by people with mental illness. This approach is especially germane to the study of deficits in visual perception associated with schizophrenia, which have recently been found to affect spatial vision. There is increasing support for the notion that people with schizophrenia show deficits in contrast sensitivity [see Butler et al (2008) for a review]. In particular, there appears to be a deficit in detecting low spatial frequencies—those spatial frequencies to which the magnocellular visual pathway (M-pathway) is most sensitive. Thus, evidence of dysfunction should in principle be present in measures of contrast sensitivity (see, eg, Skottun 2000), and indeed there is converging evidence of heightened contrast detection thresholds at low spatial frequencies associated with schizophrenia (Slaghuis 1998; Butler et al 2005; Butler and Javitt 2005; Keri 2008). Further evidence from psychophysical tests and electrophysiology and hemodynamic studies suggests that N-methyl-D-aspartic acid (NMDA) signaling dysfunction (which would affect other cortical and subcortical systems as well) could be the physiological cause of these effects (Butler et al 2005, 2007; Martinez et al 2008).

However, assessing contrast sensitivity is confounded by the fact that traditional tests using artificial stimuli consistently inflate sensitivity in healthy individuals, compared with tests with natural stimuli (Bex et al 2009). In healthy subjects spatial frequency contrast sensitivity is found to be lower across spatial frequencies for targets embedded in natural scene stimuli compared with targets in isolation, and sensitivity is even decreased when isolated targets are tested after subjects briefly view natural stimuli. Such methodologies for assessing contrast sensitivity in natural environments have, to our knowledge, been tested only in two human subjects and have yet to be employed for special populations.

Here we present a novel approach to investigate visual deficits in schizophrenia as they affect natural vision. In particular, we investigate perceptual dysfunction by analyzing willful, handmade visual representations produced by artists with schizophrenia. We compare spatial contrasts in such artworks (drawn from the NARSAD Artworks collection, http://www.narsadartworks.org) to those in a control group of paintings by artists without schizophrenia [drawn from the Cornell database of world artwork; see Graham and Field (2007, 2008a) for a description of the database].

To examine the role of medication (see, eg, Harris et al 1990), we also tested paintings from the oeuvre of Karen (Blair) Sorenson, a painter with schizoaffective disorder, who recently described her experience as an artist with mental illness in the Schizophrenia Bulletin (Blair 2007). Sorenson has taken a number of antipsychotic medications over the course of treatment, allowing us to test for possible influences of medication on image statistics in the paintings.

If contrast sensitivity in people with schizophrenia is lower at low spatial frequencies, one would expect artists with the disease to produce images with steeper Fourier spatial frequency amplitude spectra (ie, spectra with more negative slope, meaning greater relative amplitude in low spatial frequencies) than the control artists. This can be seen as a greater emphasis on large-scale structure rather than fine detail: if individuals with schizophrenia cannot perceive this large-scale structure at the same level of contrast as that at which healthy individuals can, we predicted that artists with schizophrenia would portray such structure with higher contrast, consequently generating steeper amplitude spectra in their paintings. However, a deficit in sensitivity to low-frequency contrast would not be expected to affect regularities in other basic spatial statistics, such as the spectral amplitude at different orientations; nor would it necessarily affect regularities in image intensity statistics. We therefore tested paintings by artists with schizophrenia and by the control group in terms of these statistics as well.

Methods

We tested twelve digitized artworks recently sold at NARSAD Artworks benefit auctions painted by five individuals who reported a diagnosis of schizophrenia (N = 4) or schizoaffective disorder (N = 1) against the control corpus. We also tested thirty-four digitized works by Karen Sorenson, a painter diagnosed with schizoaffective disorder. Sorenson's paintings were created while the artist was treated with various atypical antipsychotic medications-Seroquel (N = 6), Zyprexa (N = 9), Geodone (N = 6), and Risperdal (N = 13)—allowing us to examine possible effects of medication type. Works by artists with mental illness were produced because of interest and talent on the part of the artists, not as a form of therapy.

The control corpus included 140 paintings, from both the East and West, that covered nearly 900 years of art history and had been subject to extensive statistical characterization (Graham and Field 2007, 2008a). Paintings by artists with schizophrenia were cropped to a randomly chosen 300 × 300 pixel square patch (256 × 256 for the Sorenson paintings) and converted to greyscale before testing. In order to approximately normalize the spatial extent of a patch, we extracted 818 × 818 pixel patches from images in the control corpus and converted to greyscale. We note that other image and corpus sampling strategies did not change the outcome of the experiments—these include testing the largest square patch for each image, patches of the same pixel dimension in all sets (256 × 256), and patches scaled to be approximately the same physical dimension (23 cm) in each image as well as considering works of Eastern and Western provenance (which compose 49% and 51% of the control corpus, respectively) separately.

In all cases the amplitude spectrum slope was calculated by taking a rotational average of the Fourier spatial frequency amplitude spectrum and fitting it with a linear function on log-log axes (the 0th spatial frequency or “DC” component was ignored). Another method for finding slope—in particular, fitting the amplitude spectrum up to only half the maximum frequency (which ignores high-frequency components that could be affected by compression artifacts and noise)—produced no change in the outcome of the experiments.

In addition to rotationally averaged amplitude spectra, we also examined spectral amplitude as a function orientation. Both artworks (Koch et al 2010) and natural scenes (Oliva and Torralba 2001; Torralba and Oliva 2003; van der Schaaf and van Hateren 1996) have been found to contain disproportionate amounts of spectral energy in cardinal (horizontal and vertical) orientations. That is, such images typically have anisotropic spectra. We calculated the spectral anisotropy for paintings by artists with schizophrenia by performing circular Hanning windowing on all image patches (to compensate for square image frame artifacts in the amplitude spectra) and measuring the average spectral amplitude contained in 20 degree swaths around 0 and 90 degrees compared with the average amplitude in the other 140 degrees of orientation. This measure is termed the anisotropy index.

We also compared the first four statistical moments of pixel intensity distributions (mean, variance, skewness, and kurtosis) for the two image sets. In this case, statistics were calculated over the entire (uncropped) image, though we note that the same significant and insignificant results described below also held for calculations on the cropped portions of the images.

Results

We found that amplitude spectra showed significantly steeper slopes compared with those of a control group composed of a standard corpus of art images (mean of paintings by artists with schizophrenia = −1.42, SE = 0.05; mean of controls = −1.23, SE = 0.01; p value of two-sample t-test = 0.001). Test patches of paintings by artists with schizophrenia are shown in figure 1. Other spatial statistical properties that are thought to be regular in art and natural scenes show no statistical differences (p > 0.05) between the groups of paintings. As with natural scenes and other samples of artwork, the anisotropy index for paintings by artists with schizophrenia (mean = 0.307, SE = 0.006) indicates that there is proportionally more spectral amplitude in cardinal orientations. However, the anisotropy index was not significantly different compared with that for the control corpus (mean = 0.302, SE = 0.002), suggesting that only relative amplitude at different spatial frequencies—and not amplitude at different orientations—is affected by schizophrenia.

Figure 1.

Portions of the paintings by artists with schizophrenia analyzed in this study, shown with calculated amplitude spectrum slopes.

In addition, pixel intensity variance, skewness [a statistic that has shown consistent differences between samples of art from the East and West; see Graham and Field (2008a)], and kurtosis [a statistic related to sparseness; see Graham and Field (2006) for a review of sparse statistics] were not significantly different for the two sets, suggesting that spatial frequency content alone is affected by the illness. Mean pixel intensity was significantly different for the two groups, though such variation is to be expected, and it did not affect the amplitude spectrum slope calculation, since the 0th frequency component (ie, the image mean) was ignored.

The same results held for Sorenson's paintings—ie, amplitude spectrum slope was significantly steeper compared with the control group, while other statistics were not significantly different—except in the case of kurtosis, which was significantly lower for Sorenson's paintings compared with those of the control group (see table 1 for full data). Moreover, we found no significant differences in the mean amplitude spectrum slope for paintings produced while Sorenson was treated with different antipsychotic medications (amplitude spectrum mean: Seroquel = −1.44; Zyprexa = −1.44; Risperdal = −1.41; Geodone = −1.41). This result suggests that medication type does not affect spectral regularities in paintings.

Table 1.

Mean statistics by image class, with standard error in parentheses. ∗ indicates significant differences (p < 0.05) in these statistics compared with the control group, calculated according to a two-sample t-test.

Image set	Amplitude spectrum slope	Anisotropy index	Pixel intensity mean	Pixel intensity variance	Pixel intensity skewness	Pixel intensity kurtosis
Artists withSchizophrenia	−1.421∗(0.045)	0.307(0.006)	150.8∗(7.3)	2386.3(199.2)	−0.063(0.163)	−0.410(0.278)
Sorensonpaintings	−1.425∗(0.017)	0.293(0.001)	109.3∗(6.3)	2824.7(220.4)	0.466(0.137)	−0.071∗(0.194)
Control group	−1.229(0.014)	0.302(0.002)	119.6(3.5)	2464.5(150.5)	0.046(0.096)	1.054(0.25)

Discussion

Our analysis relates to typical statistics of artists with schizophrenia compared with healthy artists. Because healthy artists, on average, produce works that have a certain spectrum, it follows that without any perceptual deficit artists with schizophrenia would be driven by the same imperatives as those that conspire to produce this regularity. The fact that art has a certain typical spectrum is probably due to many factors, including luminance compression (Graham and Field 2008a, 2008b; Graham et al 2009), matching natural scene spatial statistics (Graham and Field 2007, 2008a), media, and perhaps aesthetics (Redies 2007; Koch et al 2010). Whatever the contributions of those factors may be, the same ones should apply for artists with schizophrenia—if those artists had no visual deficits. Therefore we would expect to find that a sample of spectral slopes for their works would be near the mean for artists without schizophrenia. However, we find that artists with schizophrenia produce images with a significantly greater proportion of amplitude in low spatial frequencies than do healthy artists, though other basic spatial and luminance statistics are not different. We note that this work is in no way aimed at diagnosing individual patients but rather at investigating perceptual deficits in schizophrenia. Nevertheless, our results are consistent with the notion that under broad constraints to match basic regularities in human artwork, artists with schizophrenia compensate for lowered contrast sensitivity at low spatial frequency by boosting spectral amplitude in low spatial frequencies.

Although further tests are necessary (particularly with natural stimuli), our results agree with a substantial body of evidence that contrast sensitivity deficits are more pronounced at low spatial frequencies (Slaghuis 1998; Butler et al 2005; Butler and Javitt 2005; Keri 2008). Our findings do not address the possible role of NMDA signaling dysfunction in low spatial frequency contrast sensitivity deficits associated with schizophrenia. Some have argued that decreased contrast sensitivity in schizophrenia may be more broadly distributed across spatial frequencies (Skottun and Skoyles 2007). Only one study that we are aware of, which measured contrast detection at a single spatial frequency, has questioned whether schizophrenia causes a significant drop in contrast sensitivity (Chen et al 2003).

We note that there is a danger in interpreting distortions in artwork as being the result of visual deficits, which is exemplified by the El Greco fallacy. This is the false notion that El Greco painted elongated figures due to optical distortion caused by astigmatism (eg, see Anstis 2002). However, this fallacy does not obtain in the present investigations for three reasons: (1) In the case of decreased contrast sensitivity, our results reflect loss of visual information, rather than an optical distortion. The same logic supports the suggestion that Monet's cataracts indeed affected his paintings (Marmor and Ravin 1997; see also Werner 1998); (2) We are considering effects on the mean measure across a number of artists, rather than on one individual; (3) The El Greco fallacy—which stipulates that an individual with visual dysfunction would paint the world as it appears to her—would in fact predict the opposite result from the one found. That is, the logic of the fallacy would imply that paintings by artists with schizophrenia would contain less, not more, relative amplitude at low spatial frequencies compared with control artists, in accordance with their putative visual deficit in low spatial frequency contrast sensitivity.

One can suppose that an artist with a low frequency contrast sensitivity deficit has access to low frequency contrast information simply by moving further away, and the artist may thus store this information—especially for familiar objects and scenes—in memory, which could be supplemented by visual information gathered during lucid intervals or when their drug regime is most effective. In attempting to make an image that looks “good” to them—in the sense of satisfying constraints imposed by natural scene statistics, media, and aesthetics—artists with this deficit would on average boost low frequency contrast based on prior knowledge, thus producing the observed results. In other words, they could capture scene contrasts that are normally invisible to them, but which they know exist. By necessity, the artist would do so at a level of contrast that is higher than what would be found in a comparable painting by an unaffected artist, so as to make such components visible to themselves. Thus, on average, there would be more low-frequency amplitude in the painting as a whole.

It remains possible that other explanations of our statistical results could exist: some researchers have suggested that the aesthetic experience of people with schizophrenia is fundamentally different from that of healthy individuals (Chen et al 2008), an effect which could conceivably manifest itself in paintings by artists with schizophrenia. At present, there is no evidence supporting this explanation; nor is it clear how such an effect would impact spatial frequency content. We speculate that aesthetics plays at most a small role in determining the typical spectrum of artwork, whereas matching natural scene spectra, luminance compression, and media play the more dominant roles.

While it would not be advisable to diagnose schizophrenia using art statistics, we note that it is quite common for clinicians to elicit hand-made images from patients in order to assess their mental state, using tasks such as clock drawing (eg, Wolf-Klein et al 1989), shape drawing (eg, Albert 1973), and the house-tree-person test (Buck and Warren 1992). The present work investigates evidence of contrast sensitivity dysfunction via evidence from willful visual productions by people with schizophrenia. Our primary goal is to shed light on perceptual deficits involved in the illness rather than on the patient's mental state. Nevertheless, the analysis of willful visual productions by patients is a natural laboratory for assessing the effects of a variety of mental illnesses on individuals' perception and cognition [eg, Alzheimer's disease (Utermohlen 2006) and cerebral achromatopsia (Sacks 1996); for a survey, see Chatterjee (2004)]. Future work in this vein could explore color, for example, and/or emotional cues and other forms of visual communication contained in artwork by healthy and mentally ill artists.