No validity without a theory-a critical look at subjective measures of consciousness.

Michel (The mismeasure of consciousness: a problem of coordination for the perceptual awareness scale. Philos Sci 2019;86:1239-49) claims that the Perceptual Awareness Scale (PAS) faces the problem of coordination (also known as validity). We argue that his claim holds only under certain theoretical assumptions which need to be made explicit as these are likely not in line with the PAS proponents' standpoint. We also call for terminological clarity, an example being the usage of 'levels' of consciousness. Precise terminology combined with an explicit reference to the chosen theoretical perspective is necessary conditions for making progress in consciousness research and the development of consciousness theories.

The Perceptual Awareness Scale (PAS) is a 4-point scale developed to measure degrees of awareness of a stimulus (Ramsøy and Overgaard 2004; Fazekas and Overgaard 2016). It has enjoyed wide use in consciousness research over the past two decades. Yet the theoretical motivation behind PAS is often neglected. Rather than explicitly state the theoretical motivation behind choosing PAS as the measurement tool, in general, researchers (including the authors themselves) seem to employ it merely based on its face validity. It may thus come as a surprise to some consciousness researchers that PAS was developed under three theoretical assumptions: (1) content of consciousness is gradable (2) this graduality is represented by different degrees of clearness (3) estimating the clarity of content is necessary for evaluating states of consciousness (in the sense of ‘global states’, i.e. wakefulness or coma—Overgaard and Overgaard 2010; Fazekas and Overgaard 2016). According to Michel in %The Mismeasure of Consciousness: A problem of coordination for the Perceptual Awareness Scale (2019) (henceforth referred to as M), PAS does not measure what it is intended to measure, namely levels of consciousness of content (or degrees of awareness in Overgaard’s terminology). M argues that PAS faces the problem of coordination (also known as validity) which leads to systematic mistakes in the search for Neural Correlates of Consciousness. M’s paper pertains to an important debate about the criteria which an accurate measure of consciousness should fulfil. To advance consciousness research, it is essential to carefully consider the validity of common measurement tools such as PAS. We agree with M that PAS, or any subjective scale for that matter, should be subjected to a rigorous inquiry, as subjective scales still serve as the main source of evidence concerning the presence of consciousness in both behavioural and neuroimaging studies. We appreciate the novelty of M’s approach to not argue against the reliability of PAS, but to focus instead on its validity. However, some extensions and clarifications are needed.

We would like to discuss three major issues with the arguments raised by M. The first concerns the definition of validity. It has been argued that validity should only refer to whether a measure captures what it is intended to capture. We, as empirical psychologists, prefer to use it not as a singular construct but a multifaceted concept incorporating both empirical and theoretical aspects of a given measure (Cronbach and Meehl 1955). The different types include content validity (does a scale capture what it should, theory-wise), criterion validity (the extent to which scale ratings correlate with other variables that one would expect to correlate with), and discriminant validity (Does the measure correlate with measures of variables that are conceptually distinct?). M rightfully acknowledges that to go from measurement indication to a measurement outcome one needs a theory (Michel 2019, 1242).

However, M attempts to question the content validity of PAS outside of a theory. M mentioned further that: ‘A measurement procedure is valid if it measures the quantity that one intends to measure and not a different quantity’ (1242). It is a description of content validity combined with discriminant validity, but it is not mentioned that discriminant validity stems from the content validity, i.e. it is the theory that informs us what our measure should not capture. We suggest that part of the confusion regarding PAS validity is combining content and criterion validity with discriminant validity. By doing so, we obscure the distinction between what the measure is supposed to capture, according to the underlying theory, with what it is not. In the case of PAS, it is supposed to capture changes in perceptual quality (Fazekas and Overgaard 2016). However, according to M, perceptual quality is not supposed to be the construct of interest as it is not indicative of degrees of awareness. Therefore, M jumps from estimating PAS validity to undermining the theory underlying PAS, which, in our opinion, is a different endeavour entirely.

We are not against the arguments put forward by M, but we disagree with the approach. As the validity of a measure is strongly grounded in a theory behind the given measure, a critique should focus on that theory and from there, systematically tackle different aspects of the validity concept. Disagreement with the theory associated with a given measurement tool does not automatically call into question the tool itself.

Which brings us to the second issue—the role of theory. M seems to agree on the crucial role of theory while discussing measurement outcomes: ‘In order to get from an indication to a measurement outcome, a scientist has to interpret this indication as indicating something about what one intends to measure, usually by using a theory (Tal 2017)’ (Michel 2019, 1242). Yet M’s argument against PAS being a valid measure of degrees of awareness does not explicitly refer to any specific theory. M assumes, at least indirectly, that content of consciousness is not gradable and differences in perceptual clarity cannot be indicative of different levels of consciousness of content. This approach shares some similarities with: (1) the Higher-Order Thought theory, i.e. perceptual quality is not indicative of different degrees of awareness (HOT—Rosenthal 2019); (2) Global Neuronal Workspace Theory, that is awareness is not gradual (GNWT—Sergent and Dehaene 2004); and (3) partially with the approach represented by Bayne which was directly contrasted with Overgaard’s theory (Fazekas and Overgaard 2016). However, both HOT and GNWT use measures of visibility (or other introspective scales) as indicators of awareness (Sergent and Dehaene 2004; Seth ). On the other hand, Bayne , propose that reports of degraded perception can be accounted for without supposing that consciousness itself (in the sense of ‘local contents’) is graded. Depending on which theoretical stance one supports, one may find PAS either a valid or an invalid measure of degrees of awareness. In other words, M could be right that PAS is invalid, but only while undertaking certain theoretical assumptions that necessarily vary from the ones made by the authors of PAS. However, M does not state his theoretical view clearly thus preventing a proper validity assessment.

Last but not least, we call for terminological precision. M uses the term ‘levels’ of consciousness throughout the study, which is typically used in the consciousness literature to refer to the states of consciousness (Seth 2008; Jonkisz ). Bayne also use term ‘levels’ interchangeably, however, they introduce yet another classification which further clarifies the concepts (local and global states). It proved to be difficult for us to ascertain what M means specifically when writing about the ‘levels of consciousness’. Aside from the times when he explicitly refers to ‘levels of consciousness of a content’ which we understand to correspond to Overgaard’s degrees of awareness. This introduces confusion as to which phenomenon, we are discussing and solving the validity issue relies upon the answer. Clarity about the concept of ‘levels of consciousness’ is that more important as PAS is considered to directly measure the degrees of awareness (Ramsøy and Overgaard 2004) and at least partially inform about the states of consciousness. What Fazekas and Overgaard suggest (2016) is that degrees of awareness (understood as the quality of representation defined by intensity, precision, and temporal stability) play a meaningful role in characterising global states of consciousness (Fazekas and Overgaard 2016). PAS, being one of the methods to measure variables affecting the quality of experience may thus, if only partially, provide information on the state of consciousness. According to this approach, the clarity of the content of awareness, partially informs about the state of consciousness (Fazekas and Overgaard 2016). Thus, there are two separate discussions to be had with different arguments behind each of the topics (Overgaard and Overgaard 2010). We would like to see the discussion of PAS validity placed more firmly in the existing literature on the awareness measurement topic (Bayne ; Fazekas and Overgaard, 2016).

In conclusion, we cannot agree more with M to critically approach measures of consciousness, when it comes to validity. The criticism, however, needs to be theoretically grounded. First, to thoroughly test validity, we need a theory of validity and its components and to at least separate discriminant validity from content and criterion validity. Second, as we learn from that theory, measurement validity is necessarily assessed within the theory on which it was built on. Thus, any criticism concerning measurement validity needs to focus on the underlying theory, not a measurement tool taken out of context. Measurement does not mean much outside of its interpretative framework. Thus, we would recommend that future PAS users be explicit about their theoretical assumptions and explain how PAS or any other measure fits and is interpreted within the chosen framework. It would not only add conceptual clarity to the consciousness studies but also an easier comparison between different theoretical approaches. We believe that theory comparison can be the most important step for advancing the field. Consciousness research in order to move forward needs to take the views of all sides seriously when comparing arguments based on theories.