Fact, Fiction, and Fitness.

A theory of consciousness, whatever else it may do, must address the structure of experience. Our perceptual experiences are richly structured. Simply seeing a red apple, swaying between green leaves on a stout tree, involves symmetries, geometries, orders, topologies, and algebras of events. Are these structures also present in the world, fully independent of their observation? Perceptual theorists of many persuasions-from computational to radical embodied-say yes: perception veridically presents to observers structures that exist in an observer-independent world; and it does so because natural selection shapes perceptual systems to be increasingly veridical. Here we study four structures: total orders, permutation groups, cyclic groups, and measurable spaces. We ask whether the payoff functions that drive evolution by natural selection are homomorphisms of these structures. We prove, in each case, that generically the answer is no: as the number of world states and payoff values go to infinity, the probability that a payoff function is a homomorphism goes to zero. We conclude that natural selection almost surely shapes perceptions of these structures to be non-veridical. This is consistent with the interface theory of perception, which claims that natural selection shapes perceptual systems not to provide veridical perceptions, but to serve as species-specific interfaces that guide adaptive behavior. Our results present a constraint for any theory of consciousness which assumes that structure in perceptual experience is shaped by natural selection.

1. Introduction

If the experienced world of a neonate is unstructured, a “great blooming, buzzing confusion”, that of the adult is assuredly not. Consciously experienced visual space, for instance, has a non-Euclidean geometry [1,2,3,4]. Formal analyses of color experiences yields a variety of structures, including the RGB cube, the Schrödinger color solid, manifolds, fiber bundles, and the CIE xy-chromaticity space [5,6]. Visually experienced objects and surfaces admit description by differential geometry [7,8,9]. Experiences of sound intensity are ordered from soft to loud; pitch is ordered from low to high.

This is, of course, no surprise. The structure of experience has been the subject of experiments at least since the groundbreaking work, in the 1830s, of the physiologist Ernst Heinrich Weber. These investigations coalesced into a scientific field with the publication in 1860 of Elements of Psychophysics, by the physicist and philosopher Gustav Theodor Fechner. The goal of psychophysics is to investigate the structures of experience and their relationship to structures of the physical world. As Duncan Luce and Carol Krumhansl explain [10]:

[Psychophysics] anticipates the discovery of general laws relating the sensations to physical attributes. That is, the measured sensations are expected to correspond systematically to the physical quantities that give rise to them.

The systematic correspondences inferred from psychophysics experiments, which can be represented formally as maps from the observed/measured world to conscious experiences of the perceiver, are assumed to be homomorphisms. A homomorphism is a mapping that faithfully transports a structure, I, of a physical attribute of the world into a structure, , of the sensations. For instance, the physical attribute, I, of acoustic air pressure amplitude has the structure of a total order, from low amplitude to high amplitude. The corresponding sensation of auditory loudness, , is also a total order, from low to high. The relation between them, within the typical frequency range of the human auditory system, is a power law: (here for a 3000 Hertz tone, and k depends on the units used). This map transports, or respects, the total order: if acoustic amplitude is greater than , then perceived loudness is greater than , as long as one is within the human acoustic frequency range.

This form of power law, called Stevens’ power law in honor of the psychophysicist Stanley Stevens, holds for a variety of sensations, including vibrations, brightness, lightness, visual length, visual area, warmth, pain, tactile roughness and hardness, heaviness, and electric shock (for a critique, see [11,12]). The exponent in the power law depends, of course, on the sensation. But in each case the map from physical features of the world to sensations is also, within the typical dynamic range of the relevant human detection system, a homomorphism.

This is no minor point. Psychophysics assumes the existence of an observer-independent world, and moreover, one whose structure and function are those described by physics, even if no living creature perceives it. This assumption is tellingly illustrated by Einstein’s famous question to Abraham Pais about quantum theory. Pais recalls that he asked whether “I really believe(d) that the moon exists only when I look at it” [13]. If a sense is to inform us truly about the structure of such an observer-independent world (OIW), then the map from the OIW to experiences generated by that sense must not scramble or erase this structure. Only to the extent that the map approximates a homomorphism, can the sense inform us about the structure of the OIW. If a sense succeeds to inform us about the structure of the OIW, the resulting perceptions are called “veridical. ” Veridicality, in this usage, is “truth” in the traditional sense of a correspondence theory of truth in which a sentence is true if it reports the actual state of the OIW [14]. This assumption of veridicality is standard in the perceptual and cognitive sciences. This is made clear, among other things, by the fact that visual perception is standardly treated as implementing “inverse optics ”—namely, as a process that computes the most probable 3D world structure responsible for generating any given 2D retinal image(s) [15,16].

Psychophysics assumes that “physical variables,” including light intensity, acoustic amplitude, etc., are objective components of the OIW, and therefore, that descriptions of the OIW that are based on direct measurements of such physical variables provide a “ground truth” against which less-direct measurements can be validated. In this case, empirical evidence for psychophysical mappings (such as Stevens’ power law) from physical variables to experienced magnitudes or other outcomes is evidence that such experiences are homomorphic to structurers within the OIW, and hence are veridical. However, one can also argue that what we call physical variables are themselves the results of measurement procedures that observers conduct using their own perceptual and conceptual systems, and moreover, that the outcomes of such measurements are expressed either in terms of predicates that our perceptual representations employ, or simple generalizations of such predicates [17,18]. If this is the case, psychophysical mappings simply indicate systematic correspondences between two different forms of “measurements” that observers make—direct “physical” measurements (often implemented with specific measurement apparatus) and less-direct “sensory/perceptual” measurements. Such correspondences are non-trivial and interesting; however, they constitute homomorphic mappings between different perceptual experiences, not homomorphic mappings between perceptual experiences and the OIW. We return to this point when replying to objections in §5.

Even with the understanding that veridicality requires homomorphic mappings between the OIW and our experiences, most theories of perception (in both the sciences and philosophy of mind) conclude that some of our perceptions succeed in being veridical, and that we have natural selection to thank for this. They agree on this point, even though they disagree on other fundamental issues, such as whether perception involves representation or computation.

The neurophysiologist David Marr, for instance, argued that ”usually our perceptual processing does run correctly (it delivers a true description of what is there)” [19]. He ascribed this success to natural selection, claiming that we: Here “objective” means independent of any observer or observation: an “objective aspect of the visual world,” is a structure or state of the OIW. The philosopher Jerry Fodor was adamant that [20]: Fodor is here using “true” in the sense of correspondence referred to above. The cognitive scientist Zygmunt Pizlo concurs that [21]:

Definitely do compute explicit properties of the real visible surfaces out there, and one interesting aspect of the evolution of visual systems is the gradual movement toward the difficult task of representing progressively more objective aspects of the visual world.

There is nothing in the ’evolutionary,’ or the ’biological’ or the ’scientific’ worldview that shows, or even suggests, that the proper function of cognition is other than the fixation of true beliefs.

Veridicality is an essential characteristic of perception and cognition. It is absolutely essential. Perception and cognition without veridicality would be like physics without the conservation laws.

Each of these theorists proposes that perceptual systems process information, and that veridicality is achieved, in part, through sophisticated computations. Proponents of embodied cognition reject this proposal, and claim instead that natural selection achieves veridicality by shaping the joint dynamics of organism and environment. The philosopher and cognitive scientist, Anthony Chemero, for instance, says [22]:

OK, so (radical) embodied cognitive scientists can be realists. That is, they can believe that there is an animal-independent world, and that some of our perceptions and thoughts get it right.

Similarly, the philosopher Alva Noë and psychologist Kevin O’Regan conclude that “Perceivers are right to take themselves to have access to environmental detail” [23].

In what follows, we closely examine the claim that the structure of conscious experience is, at least some of the time, homomorphic to the structure of the presumed OIW, and hence can be regarded as, at least some of the time, veridical in the strong sense required by a correspondence theory of truth. We consider by far the most common argument for veridicality: that natural selection over evolutionary time will drive the perceptual systems of organisms to at least an approximation of veridicality. We formulate this argument in terms of evolutionary game theory and prove, under generic assumptions, that the probability that fitness payoff functions are homomorphisms of certain structures in the world approaches zero as the number of possible world states and potential payoff values become large. The structures we consider here are those of total orders, permutation groups, cyclic groups, and measurable spaces. These structures are critical for perceiving magnitudes (e.g., loudness, hardness, or heat as discussed above), rearrangements of objects, rotations and translations of objects, and probability distributions, respectively. As both Euclidean and well-behaved non-Euclidean geometries, including the experienced geometry of visual space, all respect rotational and translational invariants, structures implemented by cyclic groups are, in particular, crucial for the veridical perception of geometric space. The four theorems that we prove concern the information about these structures made available to perceivers by fitness payoff functions. These theorems are independent of any specific assumptions about perceivers or their structures, and are in particular independent of the organizational level or scale (e.g., as defined by [24,25]) at which selection acts.

These results show that unless novel, strongly restrictive (and non-circular) assumptions regarding the structures of fitness payoff functions are introduced, appeals to natural selection fail to support claims of veridical experiences. It is well understood that perception has limited range. The light that humans see, for instance, is only from a narrow band of the electromagnetic spectrum. What our theorems show is that our perceptions are not veridical even within the limited ranges where they do operate; i.e., they never faithfully report the structures in the observer-independent world even within those limits. The results are consistent with the interface theory of perception (ITP [17,18,26]), according to which natural selection shapes perceptual systems to evolve a species-specific interface to guide adaptive behavior, and not to provide veridical experiences of an objective reality. As such, the results present a constraint for any theory of consciousness which assumes that structure in perceptual experience is shaped by natural selection.

2. Natural Selection

The case for veridical perception is, as noted above, often based on natural selection. The core idea is that those of our predecessors who perceived the OIW more accurately had a competitive advantage over those who perceived it less accurately, and thus were more likely to become our ancestors, passing on their genes that coded for more accurate sensory systems. We are the offspring of such ancestors, so we have reason to be confident that our perceptions are, in the normal case, veridical.

The psychologist Stephen Palmer makes this case succinctly: “Evolutionarily speaking, visual perception is useful only if it is reasonably accurate” [27]. Evolutionary theorist Robert Trivers argues [28]: Similarly, the psychologist Roger Shepard proposes that evolution shaped our senses to internalize various regularities of the external world. In his article “Perceptual-cognitive universals as reflections of the world” he claims [29]: It is worth noting here that the assumption of an OIW underlies all of these statements.

Our sensory systems are organized to give us a detailed and accurate view of reality, exactly as we would expect if truth about the outside world helps us to navigate it more effectively.

Natural selection has ensured that (under favorable viewing conditions) we generally perceive the transformation that an external object is actually undergoing in the external world, however simple or complex, rigid or nonrigid.

However, some disagree, arguing that natural selection does not favor veridical perceptions. The philosopher Patricia Churchland claims instead that [30]:

Looked at from an evolutionary point of view, the principal function of nervous systems is [...] to get the body parts where they should be in order that the organism may survive [...] Truth, whatever that is, definitely takes the hindmost.

The cognitive scientist Steven Pinker agrees [31]: Later he concedes, however, that “we do have some reliable notions about the distribution of middle-sized objects around us” [32]. It is now widely understood that the primary selective forces in human evolution, at any rate, are social [33]. The “world” to which human perceptions are adapted is, therefore, not just the presumed OIW, but is also a world of other experiencing organisms. While the social character of the human world is often explicitly acknowledged (e.g., by Trivers [28]), the OIW is still regarded as the “ground truth” by theorists of veridical perception.

Our minds evolved by natural selection to solve problems that were life-and-death matters to our ancestors, not to commune with correctness.

We have here a standoff. Natural selection is used by theorists to argue both for and against veridical perceptions. So which argument is correct? Assuming that natural selection governs the evolution of perceptual systems, we need not speculate whether it favors veridical perceptions or not. We can prove theorems. In the next section we review basic ideas needed to understand these theorems and their remarkable implications.

3. Evolutionary Games

Darwinian theory can be cast into a precise formulation in the mathematics of evolutionary game theory [34]. To understand evolutionary games, it helps to think of a video game in which a player grabs points. The reward is reaching the next level of the game. A variety of strategies are available to the player, including a choice of tools and tactics.

Players in evolutionary games can compete by employing different strategies to grab fitness payoffs; indeed, the most interesting games are games in which distinct strategies are deployed with equivalent skill. A strategy which collects, on average, more payoffs than its competitors is said to be fitter. The reward is reproduction—a new generation in which more players wield that strategy. While “evolution” is viewed as an optimization method in genetic algorithm based search [35,36], biological evolution is only satisficing [37,38]. This is reflected in evolutionary game theory by the assumption of an arbitrary payoff function, as opposed to a goodness-of-fit function with an a priori target [34,39].

But fitness payoffs depend heavily on context. Consider the fitness payoffs offered by eucalyptus leaves. For a hungry koala wanting to eat, they offer nutrition. For a sated koala wishing to mate, they offer nothing. For a hungry person wanting to eat, they offer death by cyanide. For a sated person wishing to mate, they offer nothing. The same leaves offer wildly different payoffs, depending on the organism (koala versus person), its state (hungry versus sated), and the action (eating versus mating). The key insight is that fitness payoffs depend on the combined state of the OIW—in this example the leaves—and the perceiver(s) inhabiting it: in this case the animals, their states, and their actions.

The domain of a “global fitness function” would therefore not be just the observer-independent world W, but the Cartesian product , where O is the set of organisms, S their possible states, and A their possible action classes. Once we fix a particular organism , state , and action class , we then have a specific fitness function defined on W [17,18,40].

We can thus effectively represent the resulting (specific) fitness payoff function by a function that maps the states of the OIW, , into payoff values, . That is, for a fixed organism and action class, and suppressing the parameters and a, we have a function:

Such payoff functions drive evolution by natural selection. They shape perceptions and actions. They determine whether natural selection favors veridical perceptions.

We illustrate this with a simple example. The example also serves to highlight the important point that there need be no correlation between fitness payoffs and veridicality with respect to world (OIW) structure. Recall that the relevant notion of veridicality here (and the one standardly assumed in the perceptual and cognitive sciences) is indeed veridicality with respect to world (OIW) structure (see Section 1). Suppose the world has a resource, call it stuph, and a creature, call it kritre, that eats stuph. Kritres see just two colors, light gray and dark gray. As kritres forage for stuph, they choose where to eat by the colors they see. Suppose the payoff function assigns greater values to more stuph, as in Figure 1a. Consider a kritre that sees light gray if there is lots of stuph, and dark gray otherwise, as in Figure 1b. Its sensory map is a homomorphism of a total order – brighter color corresponds to more stuph. Thus its perceptions are veridical: they preserve the order structure of stuph in the world. This kritre has a simple way to reap greater payoffs: feed where it sees light gray. Consider a different kritre that sees light gray if there is a medium amount of stuph, and dark gray otherwise, as in Figure 1c. Its sensory map is not a morphism of a total order—darker color corresponds to more stuph and less stuph. Its perceptions are not veridical: they scramble the order structure of stuph in the world. This kritre has no way to consistently reap greater payoffs. If it feeds where it sees dark gray, it sometimes gets lots of stuph and other times gets little. It is less fit than the veridical kritre.

Figure 1

Assignments of fitness payoffs: (a) Fitness payoff is a linear function of the amount of stuph. (b) “Veridical” sensory map that is homomorphic to this function. (c) “Non-veridical” sensory map that is not homorphic to this function. It is less fit than the sensory map shown in (b).

Suppose instead that the payoff function assigns greater values to medium amounts of stuph, as in Figure 2. Now the veridical kritre is in trouble. It has no way to consistently reap greater payoffs. If it feeds where it sees light gray, it sometimes gets a big payoff and other times gets a poor payoff. It has the same problem if it feeds where it sees dark gray. However the non-veridical kritre has a simple way to reap big payoffs: feed where it sees light gray.

Figure 2

Payoff function that is a non-linear function of the amount of stuph. Now, the non-veridical sensory map of Figure 1c would be fitter than then sensory map of Figure 1b.

What made the difference? The key is whether the payoff function itself is a homomorphism of the structure in the world. If it is, as in Figure 1a, then veridical perceptions are fitter, and natural selection favors them. If it is not, as in Figure 2, then veridical perceptions are not fitter. Instead, non-veridical perceptions that are homomorphisms of the payoff function are fitter, and natural selection favors them [41,42]. Thus, whether or not a fitness function is a homomorphism determines whether it can support veridical experiences—those that preserve structure in the observer-independent world W.

What about payoff functions that are not homomorphisms of structure in the world? Can they really occur? Or are they just abstract and implausible possibilities? As it happens, they occur often. Consider oxygen. Too little or too much is fatal to us. Only a narrow range of partial pressures of oxygen, between and , can sustain life. Thus the payoff function here is not a homomorphism: both low levels and high levels of oxygen map to low fitness values, whereas intermediate levels map to high fitness values. The same is true of ultraviolet radiation, blood glucose levels, and a host of other examples. This is no surprise. Life is delicate, requiring strict maintenance of homeostasis. So the corresponding payoff functions will not be homomorphisms of total orders.

Payoff functions can fail to be homomorphisms. But is this likely? If so, then selection is likely to favor non-veridicality; if not, then selection is likely to favor veridicality. If we can determine the probability that payoff functions are homomorphisms, then we can determine the probability that our perceptions are veridical.

That is the focus of this paper. We compute the probability that payoff functions are homomorphisms for four kinds of structures: total orders, permutation groups, cyclic groups, and measurable spaces. This tells us the probability that we perceive these structures veridically. The answer in each case is the same: the probability of veridical perception is zero.

We compute these probabilities by counting. We count all possible payoff functions, and count all payoff functions that are homomorphisms. We divide the number of homomorphisms by the total number of possible payoff functions to get the probability. (To be more conservative, we actually count only the total number of “admissible” fitness functions—those that achieve maximal fitness for at least some —and thus can truly shape selection processes; see next section.) These counts depend, of course, on the number of states of the world and the number of possible payoff values. But we find, for each structure, that in the limit as the number of world states and payoff values goes to infinity, the probability of homomorphisms goes to zero.

4. Four Theorems

We now present four theorems, one for each of four structures: total orders, permutation groups, cyclic groups, and measurable spaces. These structures correspond to perceptions of magnitudes, such as sound intensity or heat, re-arrangements of objects, rotations or spatial translations of objects, and probability distributions, respectively. Each theorem says the probability is zero that payoff functions are homomorphisms of the structure. Thus the probability of veridical perception of each structure is zero. We emphasize that these theorems concern the mathematical properties of the fitness payoff functions alone. They make no assumptions about, and are completely independent of, the cognitive architecture of the perceiving organism, including whether this architecture implements representations of any kind.

In each case, we compute the probability assuming that there are n states of the world and m possible payoff values. We then let n and m go to infinity to obtain our result.

Counting payoff functions that are homomorphisms is a bit tricky; we leave it for the proofs in the Appendix (for their definitions see Appendix A.1). But counting the total number of payoff functions is straightforward and is used in all four theorems. So we address it here.

The total number of payoff functions from a set of world states into a set of payoff values is simply . The reason is that each world state can map to any one of m values, and there are n world states. So there are m possible values for the first world state, times m possible values for the second world state, …, times m possible values for the last world state. This is m multiplied by itself n times, i.e., .

One might object, however, that this count includes payoff functions that are implausible, such as payoff functions in which every world state is assigned the lowest possible payoff value. How could natural selection occur with such a defective payoff function? Every strategy would be punished no matter what it did.

We think this objection is well taken. So we restrict our count to those payoff functions that take the maximum possible payoff value for at least one state of the world. For such payoff functions there are strategies that can reap maximum payoffs for at least one state of the world. We call these the admissible payoff functions.

To count the admissible payoff functions, we take the total number of payoff functions, which we computed above, and subtract the number of payoff functions that are not admissible. A payoff function is not admissible if it does not take the maximum payoff value. That means it only takes at most possible payoff values. The number of functions from n states of the world into a set of possible values can be computed by the same logic we saw two paragraphs ago. The number is . Subtracting this from the total number of payoff functions, we find that the number of admissible payoff functions is .

We now state our four theorems. The first theorem concerns total orders. It says, roughly, that most payoff functions are not homomorphisms of total orders, and thus that natural selection does not generically support veridical perception of total orders. More precisely, it says that as the number of world states and the number of payoff values increases, the probability goes to zero that admissible payoff functions are homomorphisms of total orders.

Total Orders Theorem. The number of admissible payoff functions that are homomorphisms of total orders is . Thus for any fixed m, the ratio between admissible homomorphisms of total orders and admissible payoff functions goes to zero as n goes to infinity. Additionally, even if we let m increase at the same rate as n, e.g., , the ratio still goes to zero.

Proof. See Appendix A.2.

The second theorem concerns permutation groups, which preserve symmetry. Symmetry is ubiquitous in our perceptions, from the radial symmetry of an apple, to the bilateral symmetry of many animals, leaves, and human artifacts, to the roughly Euclidean symmetry of visual space. It seems natural to assume that these symmetries of perception faithfully present symmetries of the world, to assume [43]:

3D symmetrical shapes of objects allow us not only to perceive the shapes, themselves, veridically, but also to perceive the sizes, positions, orientations and distances among the objects veridically.

Our intuitive notion of symmetry is captured by the algebraic notion of a group [44]. A group is a set, G, together with a binary operation, , that is associative (), has an identity element (), and is such that each element has an inverse (, such that ). Some examples of groups are the real numbers under addition or, if 0 is excluded, under multiplication, the group of permutations of n objects, and the “general linear group” , the set of all matrices under the operation of matrix multiplication. Other examples are subgroups of , such as the orthogonal matrices or the orthogonal matrices with unit determinant . In physics, important examples of subgroups of , where we allow the matrices to have complex entries, are the unitary matrices , and the unitary matrices with unit determinant .

Here we investigate whether payoff functions are homomorphisms of symmetric groups. The symmetric group over any set is the group whose elements are all the bijections from the set to itself, and whose group operation is composition of functions. In the case of a finite set of n symbols, the symmetric group, , consists of all possible permutations of the symbols.

Our second theorem says, roughly, that most payoff functions are not homomorphisms of symmetric groups, and thus that we do not have veridical perception of symmetry. More precisely, it says that as the number of states of the world and the number of payoff values increases, the probability goes to zero that payoff functions are homomorphisms of a symmetric group.

In the statement of the theorem, the number of world states is identical to the number of payoff values; i.e., . If, as is usual, the number of world states exceeds the number of payoff values, we can think of the theorem as applying to a subset of, or a partition into, world states that enjoy some symmetry.

Permutation Groups Theorem. The number of payoff functions that are morphisms of the symmetric group, , is Thus the ratio of these to all admissible payoff functions is , which has limit 0 as .

Proof. See Appendix A.3.

Our third theorem continues the study of symmetry. We look at cyclic groups, which are groups that can be generated by a single element. One example is the set, , of integers under addition; in fact, every infinite cyclic group is homomorphic to . Another collection of examples are the additive groups , the integers modulo n; every finite cyclic group of order n is homomorphic to . Cyclic groups appear, for instance, in the rotational symmetries of a polygon and the n-th roots of unity (roots of the polynomial ). The group of rotations of the circle is not cyclic; there is no rotation whose integer powers generate all rotations.

Our third theorem says, roughly, that most payoff functions are not homomorphisms of cyclic groups, and thus that we do not have veridical perception of cyclic symmetry. More precisely, it says that as the number of states of the world and the number of payoff values increases, the probability goes to zero that payoff functions are homomorphisms of a cyclic group.

Cyclic Groups Theorem. The number of payoff functions that are homomorphisms of the cyclic group is , the greatest common divisor of m and n [45]. The ratio of the number of cyclically homomorphic functions to admissible functions goes to zero as n goes to infinity and .

Proof. See Appendix A.4.

The fourth theorem concerns measurable spaces, which provide a framework for describing probabilities. Consider, for instance, flipping two coins. There are four possible outcomes, which we can write . If the coins are fair, each outcome has probability . We might also be interested in complex events, which are subsets of X. For instance, the event “at least one head” is the subset . If the coins are fair, this event has probability . A measurable space simply specifies a set of possible outcomes, X, and a set, , of possible subsets of X called events, which includes all of X and is required to be closed under union and complement, i.e., to be an algebra; when X is countable, it is called a -algebra. Thus a measurable space is a pair . If X is finite, the largest algebra of events, , is the set of all subsets of X, which is called the power set of X and sometimes denoted by . It is called a discrete algebra. The smallest algebra of events consists of X and the empty set, and is called a trivial algebra.

In the case of measurable structures, the morphisms of interest are “reverse homomorphisms.” That is, if the world has a measurable structure and payoff values have a measurable structure , then we are interested in functions for which is a homomorphism, mapping elements of to elements of . Such functions are called measurable. Measurable functions are of interest because they allow probabilities of events in the range to be informative about probabilities of events in the domain.

If is discrete or trivial, or if is trivial, then all functions are measurable. However, in all other instances (i.e., those more relevant to perception), our fourth theorem says, roughly, that most payoff functions are not measurable, and thus that the probabilities of events in our experiences are not informative about probabilities of events in the world. More precisely, it says that as the number of world states and the number of payoff values increase, the probability goes to zero that payoff functions are measurable with respect to a large class of measurable structures. Each measurable structure in this class is characterized by the order k of its algebra, which is the minimal number of events which generate the entire algebra via disjoint union. For instance, if W has cardinality n and W is discrete, then . But if W is generated by events, each event containing two outcomes, then .

Measurable Structures Theorem. Suppose the measurable structure on W has order k and is neither trivial nor discrete. Additionally, suppose that the measurable structure on V is not trivial. Then the number of measurable functions is bounded by

For most values of k the ratio of measurable payoff functions to all admissible payoff functions has limit 0 as .

Proof. See Appendix A.5.

5. Discussion: Does Natural Selection Favor Veridical Perceptions?

This is a technical question that can be addressed precisely using the theory of evolutionary games. Here we analyzed the payoff functions of evolutionary games, and showed that generically they are not homomorphisms of total orders, symmetric groups, cyclic groups, and measurable structures in the world. We conclude that if payoff functions erase these structures, then perceptions and actions shaped by these payoff functions cannot veridically present or represent these structures. Our proofs make no assumptions about the role of representations or computations in perception and action, so their conclusions apply equally to any computational, embodied, radical embodied, or Bayesian theory of perception and action that simply assume that senses evolve by natural selection. We wish to discuss a list of potential objections to our approach:

We use the counting measure to prove that the probabilities of homomorphisms are zero. One might argue that this is the wrong measure. The main reason for using counting measure is that it is the canonical unbiased measure on finite sets of payoff functions. Proposing any specific biased measure would need careful explanation of why the logic of natural selection dictates this specific biased measure. We believe, however, that this burden cannot be met.

The conclusions of our proofs are immune to the objection, “You cannot say whether something is veridical or not without first knowing what it is saying.” This objection assumes a representational account of perception, which is not required by our proof. Moreover, this objection is false on its face: an error-correcting code detects that a message received is not a veridical copy of the message sent, without knowing what the message is saying.

One might wonder whether the theory of evolution can be an impartial arbiter in the debate over whether natural selection entails veridical perceptions. After all, does the theory itself not simply assume the veridicality of certain perceptions, such as organisms, species, physical resources, and (using some laboratory assay) DNA? How could the theory conclude against veridicality without refuting itself? This quandary has a simple solution, however. There is an algorithmic core to evolution by natural selection—variation, selection, and retention—which requires no commitment to DNA, organisms, and other such claims about the structure of the world. This algorithm, popularized as “Universal Darwinism,” applies to the evolution of organisms, but it has been speculated that it even applies to the evolution of art, music, memes, language, and social institutions [46,47].

Our argument is based on evolution by natural selection. One can object that evolution is affected by many other factors—including genetic drift, pleiotropy, linkage, and constraints from physics and biochemistry—and that natural selection plays a relatively minor role.

However, the standard evolutionary argument for veridical perceptions is that accurate perceptions are fitter, which is an argument from natural selection. To our knowledge, there are no arguments for veridical perception based on genetic drift, pleiotropy, linkage, or constraints from physics and biochemistry. Such arguments seem unlikely. It is hard to imagine how neutral drift, for instance, could favor veridical perceptions.

Our argument focuses on just four structures: total orders, symmetric groups, cyclic groups, and measurable structures. There are, of course, many other structures relevant to perception, such as topologies, metrics, and partial orders. These structures also need to be studied, to see whether they are preserved by payoff functions. Ideally, one can hope for a general theorem, perhaps using category theory, that specifies all structures that are not preserved and thus not veridically perceived.

One might object that many payoff functions are close to being homomorphisms of the structures of the world in, say, the sense of a norm, and thus that natural selection will shape perceptions to be close to veridical, if not precisely veridical. We reply that they will also be close to being homomorphisms of countless other structures that are not in the world, and thus that natural selection will equally shape perceptions to be close to countless non-veridical structures. There is no argument here for natural selection favoring perceptions that are close to veridical rather than close to countless non-veridical possibilities.

Our theorems show that perceptions are not veridical presentations of structures in the world, but do they show instead that perceptions are veridical presentations of fitness payoffs? Not at all. Natural selection finds satisficing solutions to adaptive problems. It does not need to be veridical. This gets particularly transparent if one considers that fitness is defined only relative to competition, modeled as an (evolutionary) game. If a cheap heuristic reaps more payoffs than the competition, then it is fit enough. Perception is veridical neither regarding the world, nor regarding fitness payoffs.

Instead, perception is more like a user interface [7,18,26,42,48,49]. A desktop interface hides the complex circuitry of a computer. It shows simple icons that let the user control the circuitry despite complete ignorance of the circuitry. That is what evolution has done for us. Space-time is our four-dimensional desktop, and physical objects are icons. They are not veridical presentations of the world. They are an interface that hides the world and guides adaptive interaction with that hidden world.

There are well-known cases of perceptions that code for fitness payoffs. The symmetry of a face, for instance, codes for reproductive potential [50]. The interface theory of perception says that such coding is ubiquitous: space-time and physical objects are data compressing and error correcting codes for fitness payoffs. They are satisficing solutions to the problem of compressing fitness payoffs into an actionable format. Physical objects are not veridical presentations of the world, but data structures that we create with a glance and garbage-collect with a blink.

6. Conclusions

Our intuitions rebel. Physical objects have a strong grip on the imagination. It is hard for us to imagine that the sight, smell, and texture of a red onion, which feel so real, which feel as though they present reality as it is, are instead just a data structure that we create as needed to guide adaptive action.

Fortunately, we have clear cases of this with some synesthetes. Carol Steen, for instance, sees a complex, three-dimensional object, with a clear color, motion, and surface texture, for each sound that she hears. She creates the object while she hears the sound, and then destroys it when the sound ceases. Each time she hears the same sound she creates the same object. This allows her to sculpt the object by replaying the sound until the sculpture is finished. She reports [51]:

These brilliantly colored and kinetic visions…are immediate and vivid…I work using just one ’sense trigger,’ such as sound…listening to only one selection of music at a time, played over and over again until the painting or sculpture is finished. A work need not be completed in one day provided I listen exactly to the same music when I return to work.

Michael Watson felt a complex, three-dimensional object with his hands each time he tasted something. Mint felt like tall, smooth, cool, columns of glass. Angostura bitters felt like a basket of ivy; Karo syrup like a tray full of ball bearings. He explained [52]: Evolution is likely not done with the perceptual interface of Homo sapiens. It is still tinkering. Here we see the data structures of physical objects given novel use in hearing and taste. This application is clearly not veridical. Ball bearings are not a veridical presentation of Karo syrup; ivy is not a veridical presentation of angostura bitters. The physical objects that we normally see when we open our eyes are, no less than these synesthetic objects, non-veridical data structures. They are just satisficing solutions to the problem of compressing and presenting fitness information for action, planning, and reasoning.

When I taste something with an intense flavor, the feeling sweeps down my arm into my fingertips. I feel it—its weight, its texture, whether it’s warm or cold, everything. I feel it like I’m actually grasping something. Of course, there’s nothing really there. But it’s not an illusion because I feel it.