Where the eyes wander: The relationship between mind wandering and fixation allocation to visually salient and semantically informative static scene content.

Vision is crucial for many everyday activities, but the mind is not always focused on what the eyes see. Mind wandering occurs frequently and is associated with attenuated visual and cognitive processing of external information. Corresponding changes in gaze behavior-namely, fewer, longer, and more dispersed fixations-suggest a shift in how the visual system samples external information. Using three computational models of visual salience and two innovative approaches for measuring semantic informativeness, the current work assessed whether these changes reflect how the visual system prioritizes visually salient and semantically informative scene content, two major determinants in most theoretical frameworks and computational models of gaze control. Findings showed that, in a static scene viewing task, fixations were allocated to scene content that was more visually salient 10 seconds prior to probe-caught, self-reported mind wandering compared to self-reported attentive viewing. The relationship between mind wandering and semantic content was more equivocal, with weaker evidence that fixations are more likely to fall on locally informative scene regions. This indicates that the visual system is still able to discriminate visually salient and semantically informative scene content during mind wandering and may fixate on such information more frequently than during attentive viewing. Theoretical implications are discussed in light of these findings.

Introduction

Vision is crucial for many everyday activities, and an in-depth analysis of the visual world requires that the eyes move. This is because the visual system is subject to both physical constraints (i.e., the structure and organization of photoreceptors) and cognitive constraints (i.e., attention and memory). For example, visual input is acquired during fixations—periods when the eye remains relatively stable—but is perceptually (e.g., Matin, 1974; Zuber & Stark, 1966) and cognitively (e.g., Campbell & Wurtz, 1978; Irwin & Carlson-Radvansky, 1996; Irwin & Brockmole, 2004) suppressed during saccades—the ballistic eye movements that shift fixations from one location to another. Therefore, the timing and location of fixation allocation offer insight into the real-time information-processing priorities of the visual system (e.g., Just & Carpenter, 1976; Kowler, Anderson, Dosher, & Blaser, 1995).

There are a number of known factors that influence fixation allocation in static scene viewing, which is frequently used as a proxy for how the visual system samples information in the real world. These factors include the low-level, visually salient features of stimuli, such as color, contrast, and edge orientation (e.g., Mannan, Ruddock, & Wooding, 1996; Mannan, Ruddock, & Wooding,1997; Parkhurst & Neibur, 2003; Reinagel & Zador, 1999; Tatler, Baddeley, & Gilchrist, 2005), as well as higher order, observer-driven factors, such as semantic interest (e.g., Buswell, 1935; Loftus & Mackworth, 1978), momentary task goals (e.g., Land & Hayhoe, 2001; Land & Lee, 1994; Yarbus, 1967), and long-term schematic knowledge of scene structure (e.g., Mandler & Johnson, 1977; Shinoda, Hayhoe, & Shrivastava, 2001; Võ & Henderson, 2009). Although this is not an exhaustive list, it shows that converging influences from both stimulus- and observer-based factors impact fixation allocation, and a number of frameworks and computational models have attempted to characterize gaze control in light of these factors (e.g., Garcia-Diaz, Leboran, Fdez-Vidal, & Pardo, 2012; Harel, Koch, & Perona, 2007; Henderson & Hayes, 2017; Henderson & Hayes, 2018; Itti & Koch, 2000; Itti & Koch, 2001; Itti, Koch, & Niebur, 1998; Riche, Mancas, Duvinage, Mibulumukini, Gosselin, & Dutoit, 2013; Tatler, Brockmole, & Carpenter, 2017).

An implicit assumption of current frameworks and models of gaze control is that observers consistently and appropriately attend to their visual surroundings. In reality, however, people are not always avidly attentive, and instead they frequently mind wander, defined here as a shift in attentional priorities away from task-relevant goals to task-irrelevant internal thoughts (Smallwood & Schooler, 2006). In fact, laboratory and field-based research has shown that, when asked, people will report having been mind wandering 20% to 50% of the time (Killingsworth & Gilbert, 2010; Smallwood & Schooler, 2015; but see Seli, Beaty, Cheyne, Smilek, Oakman, & Schacter, 2018). Thus, current theories and models of gaze control likely fail to capture the full range of influences on the manner in which the mind and brain sample the visual world. Our goal in this report was to address this limitation by considering how visual and cognitive factors known to enter into gaze control decisions vary as a function of observers’ level of attentiveness to their tasks and goals.

Visual processing during mind wandering

Although the cognitive origin and progression of mind wandering is currently debated (Christoff, Irving, Fox, Spreng, & Andrews-Hanna, 2016; Christoff, Mills, Andrews-Hanna, Irving, Thompson, Fox, & Kam, 2018; Seli, Kane, Metzinger, et al., 2018; Seli, Kane, Smallwood, Schacter, Maillet, Schooler, & Smilek, 2018), a common theoretical view construes mind wandering as an attentional state that is, to some degree, decoupled from the external world (Murphy, Jefferies, Rueschemeyer, Sormaz, Wang, Margulies, & Smallwood, 2018; Schooler, Smallwood, Christoff, Handy, Reichle, & Sayette, 2011). A growing body of neurocognitive evidence supports this perceptual decoupling account (Kam & Handy, 2018). For example, mind wandering is associated with an attenuated P1 event-related potential (ERP) component (Baird et al., 2014; Kam, Dao, Farley, Fitzpatrick, Smallwood, Schooler, & Handy, 2011; Smallwood, Beach, Schooler, & Handy, 2008), the ERP component that reflects early low-level visual processing (Hillyard, Hink, Schwent, & Picton, 1973). Interestingly, Barron et al. (2011) showed that retrospective self-reported measures of mind wandering were also associated with reduced P3a, the component that reflects the capture of attention by rare distractor stimuli (Escera, Alho, Schröger, & Winkler, 2000; Knight, 1997), and the central-parietal P3b, a component that reflects the maintenance of a task-relevant stimulus in working memory (Polich, 2003). This collective evidence indicates that visual information processing during mind wandering is also attenuated at multiple levels of cognitive analysis: perception, attention, and working memory.

The perceptual decoupling observed during mind wandering suggests a deprioritization of visual information processing that might correspond to changes in fixation allocation during scene viewing. Indeed, Krasich, McManus, Hutt, Faber, D'Mello, and Brockmole (2018) found evidence in support of this idea. They asked participants to study pictures of urban scenes in preparation for a later memory test. Periodically, participants self-reported whether they were mind wandering or attentively viewing the scene at a given moment via pseudorandomly distributed thought probes that occurred 45 to 75 seconds into scene viewing. Findings showed that probe-caught mind wandering was associated with fewer, longer, and more dispersed fixations (compared to reports of attentive viewing), with the most robust observations found 10 seconds prior to the onset of the thought probe. Findings were conceptually replicated using a paradigm where scenes were presented contiguously, and thought probes occurred at pseudorandom intervals over the course of the viewing task. Accordingly, Krasich et al. (2018) inferred that, given the perceptual decoupling during mind wandering, the visual system becomes less efficient and slower to extract and evaluate visual information, thus prolonging fixations. The co-occurring increase in fixation dispersion, the authors suggested, may reflect a systematic, rather than random, shift in how information is sampled.

What remained unclear from Krasich et al. (2018), however, was whether changes in fixation duration and dispersion reflected a shift in how external information is prioritized during mind wandering. That is, does the visual system systematically change what visual information is sampled during mind wandering or does it simply alter how information is sampled (i.e., more slowly and broadly) throughout the scene? Answering this question will identify what visual information the visual system detects and prioritizes during conditions of attenuated visual processing such as during mind wandering.

The current study

The current study focuses on visual and cognitive factors that have been linked to gaze control and how these relationships vary over attentional states. Specifically, our goal was to assess how fixations are allocated to visually salient and semantically informative scene content prior to self-reported mind wandering. To do this, we re-analyzed the data reported by Krasich et al. (2018) with a new focus on fixation placement relative to scene content. Using this prior study as a basis for our investigation was advantageous because it has already been used to demonstrate that several parameters of gaze control vary as a function of attentiveness. Hence, it gave us a direct opportunity to compare the relationship between mind wandering and both content-independent (i.e., those considered by Krasich et al., 2018) and content-specific (i.e., those considered here) measures of visual sampling.

Visual salience

Some computational models of gaze control compute stimulus-based properties to predict and model fixation allocation (e.g., Garcia-Diaz et al., 2012; Harel et al., 2007; Itti & Koch, 2000; Itti & Koch, 2001; Itti et al., 1998; Riche et al., 2013). One popular approach to operationalizing stimulus-based properties assumes that visual input (such as from a static scene) can be represented with iconic topographic feature maps (e.g., color, contrast, edge orientation) that are first extracted and then computationally combined to create a single saliency map, which denotes the visual distinctiveness of any given location relative to surrounding locations or the entire image. These saliency maps, therefore, incorporate stimulus-based properties with little regard to higher order scene structure, and—in the absence of any goal-based, volitional control—salience-based models predict that fixations should be allocated to the most salient location first before moving to areas of lower saliency.

Given the shift of attentional priorities away from task goals during mind wandering (Smallwood & Schooler, 2006), mind wandering may provide conditions of reduced goal-based, volitional control. Fixations may therefore be allocated to highly salient scene content more frequently during bouts of mind wandering compared to attentive viewing. That said, following the perceptual decoupling accounts of mind wandering showing attenuated visual processing, it might be that the visual system becomes less sensitive to low-level, stimulus-based properties. This possibility predicts that fixations would not be more likely—and perhaps even less likely—to occur in visually salient scene content. No observable change in how fixations are allocated to visually salient scene content during mind wandering might also indicate that the changes in content-independent measures of gaze behavior observed in Krasich et al. (2018) are not reflective of a shift in how fixations are allocated to visually salient scene content.

To assess these competing hypotheses, we characterized visual salience for each of the images from Krasich et al. (2018) using three different salience-based computational models, which are among the most effective models of gaze control (Riche et al., 2013; Tatler et al., 2017): the Graph-Based Visual Salience model (GBVS) (Harel et al., 2007), the Adaptive Whitening Saliency Model (AWS) (Garcia-Diaz et al., 2012), and a rarity-based saliency model called RARE2012 (RARE) (Riche et al., 2013). The GBVS first computes multiscale feature maps (i.e., intensity, color, and orientation) via linear center-surround computations that mimic human visual receptive fields. Graph algorithms are then used to build activation maps by defining random-walk Markov chains from these feature maps. Activation maps are then merged into a final salience map such that the saliency of a given region reflects its contrasts to the local surrounding regions. The GBVS also incorporates a “center bias” that promotes higher saliency values in the center of the image, which accounts for past research showing a greater tendency for fixations to be allocated toward the center of a static images (e.g., Bindemann, 2010; Buswell, 1935; Parkhurst, Law, & Niebur, 2002; Parkhurst & Niebur, 2003; Tatler, 2007; Tatler et al., 2005).

The AWS model is biologically motivated by the idea that the nonlinear neural responses in the visual cortex should be considered as collective neuron populations rather than as single units (decorrelation of neural responses) (e.g., Olshausen & Field, 2005). It also assumes that low-level information is carried by high-order statistical structures and adopts a hierarchical approach to statistically whiten low-level features and remove second-order information (i.e., decorrelation and contrast normalization). The AWS model uses L*a*b* color space, which reduces the correlation between color components. Next, log-Gabor filters are used to transform luminance into multiscale and multi-oriented representations, which are then decorrelated using a principal component analysis (PCA). The final saliency map is then computed by taking the sum of the squared norm vectors in the final representation and normalizing it to the sum across all pixels of the image. Thus, visual salience in the AWS represents a global decorrelation of the entire image.

The RARE model first extracts several feature maps. Low-level feature maps are computed through a hierarchical color transformation (PCA decomposition), and medium-level feature maps (e.g., texture) are extracted using Gabor filters that are modeled after simple cell neuronal activity in the visual cortex. A multiscale rarity mechanism—the unique feature of the RARE model—is then applied to each feature map to compute rarity maps that denote locally distinctive contrasts, as well as regions that are rare throughout the entire image. Finally, an intra-channel and inter-channel fusion combines rarity maps into a final saliency map. Visual salience in the RARE model thus represents local rarity relative to the entire image.

The GBVS, AWS, and RARE models therefore compute visual salience with unique approaches. For example, visual salience computed by the GBVS model reflects contrasts of local regions that favor more central regions, whereas visual salience in the AWS and RARE models reflect either adaptive whitening or rarity contrasts relative to the entire image without incorporating a center-bias. Computing visual salience using these different models allowed for the relationship between mind wandering and fixation allocation to visually salient scene content to be explored in multiple ways according to the procedures by which salience was computed.

Semantic informativeness

The success of salience maps in characterizing gaze control has fostered a robust empirical endeavor. That said, visual salience may not fully account for fixation allocation (e.g., Henderson, Brockmole, Castelhano, & Mack, 2007), and the convenience of quantifying visual salience may disregard critical influences from semantic information (e.g., Henderson, 2017). For example, observers have a greater tendency to fixate locations that are rated as (e.g., Hayes & Henderson, 2019; Henderson & Hayes, 2017; Henderson & Hayes, 2018; Loftus & Mackworth, 1978; Mackworth & Morandi, 1967) or predicted to be (e.g., Bar, 2009; Clark, 2013; Friston, 2010; Lupyan & Clark, 2015) more semantically informative than surrounding regions, even when those locations are less visually salient than surrounding regions (e.g., Henderson, Malcolm, & Schandl, 2009). Moreover, semantic information might be mischaracterized if it is also visually salient (e.g., Einhäuser, Spain, & Perona, 2008; Elazary & Itti, 2008; Henderson, 2017). In fact, when compared directly, visual salience sometimes failed to explain fixation allocation above and beyond semantic information (e.g., Einhäuser et al., 2008; Hayes & Henderson, 2019; Henderson, 2017; Henderson & Hayes, 2017; Henderson & Hayes, 2018; Henderson et al., 2007), although this is not universally true (Tatler et al., 2017). Therefore, the extent to which visual salience and semantic information contribute to gaze control is still debated; however, converging evidence indicates the importance of considering semantic information when investigating factors of gaze control.

Measuring semantic information poses somewhat of a challenge (in comparison to visual salience), given the subjective nature of observer evaluation. For example, it is not always clear how objects should be defined, evaluated, and prioritized (e.g., Borji, Sihite, & Itti, 2013a; Borji, Sihite, & Itti, 2013b; Einhäuser et al., 2008; Nuthmann & Henderson, 2010), and objects can be valued as important even before they are completely identified (Spain & Perona, 2011). Recent efforts, however, have attempted to map the variation in semantic information across an entire image in a conceptually similar way as a visual saliency map; semantic values are spatially distributed nonuniformly across an image with certain regions measured as more semantically informative than others.

Currently, there are two approaches that characterize semantic information in different ways. One approach, which we refer to as the semantic interest map, identifies regions within a scene that are judged to be the most semantically informative locations relative to the entire global scene context (Tatler et al., 2017). Specifically, third-party observers subjectively select the five most semantically informative regions of a scene while viewing the entire image, which allows observers to consider scene context but results in only a few areas that are indicated as being highly informative. The other method, referred to as the meaning map approach, gauges how locally informative or recognizable information is within small region of a scene (vignette) that is rated independently of scene context (Hayes & Henderson, 2019; Henderson & Hayes, 2017; Henderson & Hayes, 2018). For these maps, third-party observers rate how informative or recognizable information is within each vignette, and then vignettes are interpolated to produce a cohesive map so that each location within a scene contains a semantic value. The critical differences between these two approaches are (1) whether semantic information is evaluated in relation to or independently from the entire scene, and (2) whether values are based on the most informative (semantic interest map) or on locally informative (meaning map) semantic information. Therefore, these two different approaches allow for the relationship between mind wandering and the prioritization of semantic information in fixation allocation to be explored in different ways.

In terms of hypotheses, the visual system may be less able to discriminate what scene content is semantically informative during mind wandering given perceptual decoupling; thus, gaze would less frequently fixate on highly informative scene content regardless of how semantic information is characterized. Alternatively, because visual and cognitive processing is only attenuated, not entirely eliminated, during mind wandering, it is possible that the visual system can still manage to detect and prioritize information that is the most semantically informative while neglecting less informative content. This idea predicts that fixations would be more likely, or at least equally likely, to occur within semantically informative regions during mind wandering, especially when characterized by semantic interest maps. That said, if the meaning maps better predict fixation allocation during mind wandering, findings would suggest that locally informative scene content remains detectable and perhaps becomes prioritized during mind wandering. No observable change in how gaze is allocated to semantically informative scene content during mind wandering would also suggest that the changes in the content-independent measures of gaze observed in Krasich et al. (2018) cannot be characterized as a systematic shift in how fixations are allocated to semantically informative scene content.

Empirical approach

Using the previously described models, visual salience and semantic informativeness scores were computed for each of the images used in Krasich et al. (2018). Then, scores for locations where fixations occurred in Krasich et al. (2018) were measured and compared across self-reports of mind wandering and attentive viewing, which were obtained via pseudorandomly distributed thought probes.

Methods

Participants

Eye movement and mind wandering data were obtained from the study by Krasich et al. (2018), which included 51 volunteers from the University of Notre Dame. All participants were compensated with course credit.

Semantic interest maps were generated with data collected from 31 college-aged students from the University of Notre Dame who did not participate in the Krasich et al. (2018) study. This sample size was established following a similar sample size (n = 27) used in Tatler et al. (2017). Participants volunteered through the university psychology subject pool following procedures approved by the university institutional review board (IRB) and received course credit for participation.

Meaning maps were generated with data collected from 150 volunteers from Amazon Mechanical Turk (MTurk) who had a hit approval rate of 95%, had at least 100 hits approved, and were from the United States. This sample size was similar to that used in Henderson & Hayes (2017, 2018) (n = 79 and n = 165, respectively). Participants volunteered through MTurk following procedures approved by the university IRB and were monetarily compensated for participation. Fifteen MTurk participants were removed for not properly completing the task (i.e., pressing the same response for all 300 patches); therefore, ratings from 135 participants were used to generate the meaning maps.

Stimuli and apparatus

The stimuli consisted of the 12 digitized color photographs of real-world urban scenes (800 pixels × 600 pixels) that were used in Krasich et al. (2018). Images were presented in 32-bit color on a 20-inch cathode-ray tube monitor with a screen refresh rate of 85 Hz and a resolution of 1024 × 768 pixels. Examples images are shown in Figure 1.

Figure 1.

Example images and corresponding visual salience (A) and semantic maps (B). Note the difference in scale across the salience and semantic maps for visualization purposes.

Eye movements were sampled using an EyeLink 2K tower-mounted eye tracking system (SR Research, Ltd., Kanata, Canada) at a rate of 1000 Hz. A viewing distance of 80 cm was maintained by a chin and forehead rest. Saccades were operationally defined as changes in recorded fixation position that exceeded 0.2° with either a velocity that exceeded 30°/s or an acceleration that exceeded 9500°/s2. The eye tracker was calibrated using a nine-point calibration at the beginning of the study and a one-point calibration before the presentation of each image to correct for any subtle drift in the eye tracker signal over time.

Experimental procedures

Participants from Krasich et al. (2018) were instructed to study 12 images for a later recognition test. They also received instructions for responding to the thought probes, and mind wandering (which was colloquially termed as “zoning out”) was defined as the act of “looking at the picture but thinking of something else entirely” unrelated to the task or the scene content. After receiving all instructions, participants viewed each image, presented in a different random order, sequentially for 45 to 75 seconds (M = 60.0 seconds, SD = 8.49 seconds). Eight thought probes were randomly presented at the end of eight trials. These probes asked, “In the moments right before this message, were you paying attention to the picture or were you zoning out?” (Schooler, Reichle, & Halpern, 2004). Each participant received eight image-probe pairings, but the pairings differed across participants. Across participants, the resulting number of probes per image ranged from 28 to 41 (Mdn = 34.5, IQR = 29–36.5), with the number of reports of mind wandering per image ranging from 5 to 13 (Mdn = 10, IQR = 6–10.5); therefore, mind wandering occurred with each of the images.

Computing visually salience and semantically informative scene content

Saliency maps

The GBVS, AWS, and RARE models were used to generate saliency maps for each of the images used in Krasich et al. (2018). Example images and maps are illustrated in Figure 1. Each model computes an arbitrary “salience” value for each pixel, with greater values indicating greater salience. Following typical practice, values for each of these maps were then normalized so that the sum salience score of all pixels within an image was equal to 1.

As reported in Table 1, salience scores computed by the GBVS were moderately correlated with scores from the AWS and RARE models, and scores from the AWS and the RARE models were strongly correlated. These findings indicate that, although each model adopts different approaches for computing salience, a certain degree of similarity exists in how each map characterizes visually salient scene content. Measured coefficients, however, do indicate some variability across maps, which suggests an advantage to using multiple models to characterize visual salience.

Table 1.

Pearson's correlation coefficient matrix for salience and semantic scores for all images. Notes: Correlation coefficients represent a pixel-by-pixel comparison for all images.

	AWS	RARE	Semantic map	Meaning map
GBVS	0.55	0.52	0.47	0.54
AWS	—	0.75	0.53	0.61
RARE		—	0.51	0.48
Semantic interest map			—	0.49

Semantic interest map

The procedures for computing the semantic interest maps mirrored those used in Tatler et al. (2017). Participants viewed each full scene used in Krasich et al. (2018) and selected (with a mouse click) the five most semantically informative locations within the scene ignoring visual characteristics such as color or brightness (see Appendix A for full task instructions). “Semantically informative” locations were defined prior to the experiment as locations that were the most “informative about the meaning of the scene.” Participants were able to reselect locations prior to moving on to the next trial but were not able to reselect locations after moving on to the next trial. The selections were used to create semantic interest maps in the same manner as Tatler et al. (2017) by centering Gaussians with full width at half maximum of 2° around each selected location. This approach computed an arbitrary value for each pixel of the image, with greater values indicating greater semantic informativeness. Values were normalized so that the sum score of all pixels was equal to 1 within each image.

Meaning maps

The procedures for computing the meaning maps were drawn from those used in Hayes & Henderson (2019). Each of the scenes was decomposed into partially overlapping circular patches with 3° (“fine” patches) and 7° (“coarse” patches) diameters. The full patch stimulus set consisted of 3600 unique fine patches and 960 course patches for a total of 4560 patches. Each subject rated 300 random patches (for a total of 40,500 ratings) without scene context with the instructions to assess the meaningfulness of each patch in terms of how informative or recognizable it was considered (see Appendix B for full task instructions). Specifically, participants rated the meaningfulness of the patch using a six-point Likert scale (very low, low, somewhat low, somewhat high, high, very high). The ratings for each pixel at each scale (fine and coarse) were averaged to produce fine and coarse rating maps for each scene, which were then averaged together into a single map. This map was then smoothed with a Gaussian filter using the imgaussfilt function in MATLAB (Mathworks, Natick, MA). Finally, values were normalized so that the sum meaning score of all pixels within each image was equal to 1.

Example semantic interest and meaning maps are illustrated in Figure 1. As reported in Table 1, scores across the two maps were moderately correlated, but the measured coefficients do indicate variability across the two maps, which suggests an advantage to investigating the spatial allocation of gaze using multiple models of semantic informativeness.

Computing salience and semantic scores at fixated locations

The (x,y) coordinates were extracted for each fixation made by participants in Krasich et al. (2018). Then an area subtending 2° visual angle around each coordinate was established, and the mean and maximum salience and semantic informativeness scores among the pixels within each of these areas were calculated. This was done for several reasons. First, because of the physical characteristics of the human eye, visual acuity is best at central viewing (an area subtending roughly 2° visual angle), and a greater proportion of neurons within the primary visual cortex are devoted to processing central vision as opposed to the periphery. Accordingly, central vision is particularly apt for high-resolution visual analysis; therefore, measuring visual salience within a 2° area around a fixation provided insight into how the visual system prioritizes a high-resolution analysis of visual salience. Second, this approach also accommodated subtle instrument error (typically 0.15°) in saccade recording. Finally, computing the mean and maximum scores among the pixels surrounding fixated locations characterized the data in two different ways. Specifically, mean scores indicated the mass visual salience of a particular area and maximum scores indicated the highest pixel of salience within the fixated area. By computing both of these variables it could also be determined which measure might best predict gaze behavior during mind wandering. Average means and maximum values were then centered and scaled (z-scored) using the scale function in R (R Foundation for Statistical Computing, Vienna, Austria) (Becker, Chambers, & Wilks, 1988).

Fixations that occurred outside of the scene boarders (3% of fixations), were shorter than 50 ms (2% of fixations), or were longer than 10,000 ms (<0.01% of fixations) were excluded. In total, 95% of all fixations were analyzed.

Results

First, the frequency of reported mind wandering from Krasich et al. (2018) is provided for ease of interpreting the results. Then, the main research question regarding the relationship between mind wandering and fixation allocation to visually salient and semantically informative scene content is reported.

Frequency and validation of mind wandering

Participants from Krasich et al. (2018) reported mind wandering on 27% of probes (SD = 22%). Eleven participants reported no instance of mind wandering, and one participant reported mind wandering for all eight probes. This rate of mind wandering is within the range of rates typically reported in laboratory and field settings (Killingsworth & Gilbert, 2010; Seli, Beatty, et al., 2018; Smallwood & Schooler, 2015). Rates of mind wandering were correlated with worse performance on the memory tests, which further validated the self-reports. A more detailed description of performance on the memory test and its relationship to mind wandering is found in Krasich et al. (2018).

Fixation allocation prior to reported mind wandering

Of the 408 trials (out of 612 total trials) that included a thought probe, only two trials were excluded due to tracking errors; therefore, a total of 406 trials were analyzed. Fixation allocation for trials with reported mind wandering (107 trials) were compared to trials with reported attentive viewing (299 trials).

Analyses focused on those fixations that occurred 10 seconds prior to the thought probes. Krasich et al. (2018) observed the most robust changes in the spatial aspects of fixation allocation (i.e., increased fixation dispersion) associated with mind wandering 10 seconds prior to the self-report, and past research has suggested that, depending on the task, some gaze behaviors show stronger associations with mind wandering (Faber, Krasich, Bixler, Brockmole, & D'Mello, in press) and attentive viewing (Marsman, Renken, Haak, & Cornelissen, 2013; Unema, Pannasch, Joos, & Velichkovsky, 2005) within smaller spans of time. Accordingly, mean and maximum salience and semantic scores of fixated locations were averaged across fixations that occurred 10 seconds prior to the thought probes.

Mixed-effects linear regression analyses were conducted using the lme4 package in R (Bates, Mächler, Bolker, & Walker, 2015) to model each dependent variable (average mean and maximum salience and semantic score as measured by the respective models), with probe response (two levels: paying attention [reference group] and mind wandering) and image viewing time (z-scored) (Becker et al., 1988) as fixed-effects variables and with participant and image as random effects. Standardized coefficients (β) were computed, which indicated the predicted change per unit (SD) in the dependent variable net of the other predictor variables. Wald chi-square ratios and p values were also computed using the Anova function from the car package in R (Fox & Weisberg, 2011) with a type II sum of squares to investigate the main effects of mind wandering controlling for covariates. Treatment contrasts were used for all comparisons across mind wandering and attentive viewing.

Because the three models of visual salience (GBVS, AWS, and RARE) were meant to measure the same construct, we elected to be conservative and correct for familywise error when analyzing each of the dependent variables (mean and maximum salience). Significance testing was, therefore, conducted using two-tailed tests with α set to 0.05, with Bonferroni correction. Thus, we rejected the null hypothesis when p < 0.017 (i.e., 0.05/3). We were similarly conservative in our analyses of mean and maximum semantic interest scores because we used two approaches to operationalize semantic content (semantic interest maps and meaning maps). We therefore rejected the null hypothesis in cases where p < 0.025 (i.e., 0.05/2).

Coefficients and test statistics for each predictor are reported in Appendix C. Test statistics most relevant to the effect mind wandering on fixations to visually salient and semantically informative scene content are reported in Table 2.

Table 2.

Standardized coefficients and test statistics assessing the main effect of mind wandering on the average mean and maximum salience and semantic scores of fixated locations. Notes: β = standardized coefficients; SE = standard errors; degrees of freedom for all chi-square ratios = 1; asterisk (*) indicates statistical significance after Bonferroni adjustments (salience scores p < 0.017; semantic scores p < 0.025).

	Main study				Joint analyses
	β	SE	χ2	p	β	SE	χ2	p
GBVS
Avg. mean	0.076	0.105	0.529	0.467	0.100	0.886	1.263	0.261
Avg. maximum	0.064	0.106	0.357	0.550	0.093	0.089	1.086	0.297
AWS
Avg. mean	0.328	0.104	9.951	0.002*	0.296	0.090	10.828	0.001*
Avg. maximum	0.343	0.102	11.377	0.001*	0.299	0.087	11.789	0.001*
RARE
Avg. mean	0.281	0.108	6.763	0.009*	0.292	0.093	9.890	0.002*
Avg. maximum	0.296	0.100	8.706	0.003*	0.299	0.088	11.435	0.001*
Semantic map
Avg. mean	0.043	0.106	0.164	0.685	−0.024	0.089	0.073	0.787
Avg. maximum	0.038	0.103	0.134	0.715	−0.036	0.087	0.173	0.678
Meaning map
Avg. mean	0.174	0.091	3.69	0.055	0.167	0.081	4.208	0.040
Avg. maximum	0.164	0.087	3.58	0.058	0.143	0.077	3.419	0.064

Minimal detectable effect sizes

We first assessed the sensitivity of our main study given its sample size by estimating the minimal detectable effect size (MDES) of mind wandering on each dependent variable net of the aforementioned covariates included in the mixed-effects linear regression analyses. The coefficient of the mind wandering fixed effect served as the effect size measure, and the magnitude of these coefficients reflected the estimated change in the dependent variables in SD units. Using the simr package in R (Green & MacLeod, 2016), a power analysis was conducted for each dependent variable to estimate the power associated with effect sizes ranging from 0.05 to 0.55 in increments of 0.05, α set to 0.05, and the number of simulations set to 1000. The lowest effect size that would on average yield a power of at least 0.80 was retained as the MDES. The average MDES was 0.305 and ranged from 0.250 to 0.350. These findings indicate that 0.305 was on average the smallest effect detectable at power of 0.80. It further suggests that, at a power of 0.80, effect sizes smaller than 0.305 would on average require a larger sample size to achieve statistical significance. The specific MDES and associated power for each dependent variable are reported in Appendix D.

The effect of mind wandering 10 seconds prior to thought probes

When measuring visual salience with the GBVS, there was no effect of mind wandering on the average mean and maximum salience score of fixated locations. When measuring visual salience with the AWS, however, mind wandering was associated with greater average mean and maximum salience scores for fixated locations. The same observations held when measuring visual salience with the RARE. Together, these findings showed that, as measured by two (admittedly highly correlated) salience models, fixations made 10 seconds prior to self-reported mind wandering occurred in regions of higher visual salience than fixations made 10 seconds prior to reports of paying attention.

There was no effect of mind wandering on the average mean and maximum semantic informativeness score of fixated locations when measured by the semantic interest maps. This finding indicates a similar propensity during mind wandering and paying attention to look at scene content rated as the most semantically informative. When measuring semantic informativeness with the meaning maps, mind wandering tended to be associated with greater average mean and maximum scores, but this effect was not statistically significant. In light of our MDES analyses, though, a larger sample size could potentially yield statistically significant differences.

To increase confidence that the AWS- and RARE-based results were not spurious, we conducted an analysis in which we shuffled participants’ fixation points (x and y coordinates) that were recorded while viewing each scene onto different randomly selected scenes. This process thereby broke the natural association between image content and fixated locations while retaining the given fixation pattern with its corresponding probe response (as well as continuing to incorporate natural biases in oculomotor behavior). Hence, in this analysis we would predict that no relationship between salience and mind wandering should be observed. We repeated the analyses conducted above after computing the average mean and maximum salience and semantic informativeness scores for each fixated location with respect to the overlaid shuffled images. We modeled each of these dependent variables using mixed-effects linear regression analyses with probe response (two levels: paying attention [reference group] and mind wandering) and original image viewing time (z-scored) as fixed-effects variables and with participant and shuffled image as random effects. The relevant coefficients and test statistics are reported in Table 3. Findings showed that, across all models, the average mean and maximum salience and semantic scores did not differ across trials with reported mind wandering and attentive viewing. These analyses provide additional support for the conclusion that observers view scene content—as measured by two models of visual salience—differently when they are mind wandering versus paying attention.

Table 3.

Standardized coefficients and test statistics assessing the main effect of mind wandering on the average mean and maximum salience and semantic scores of fixated locations computed using randomly shuffled overlaid images. Notes: β = standardized coefficients; SE = standard error; degrees of freedom for all chi-square ratios = 1. The analysis assessing average mean salience score as measured by the RARE failed to converge; the original image view time variable was then removed, and the results from this revised analysis are reported here.

	Main study: shuffled images
	β	SE	χ2	p
GBVS
Avg. mean	0.119	0.111	1.147	0.284
Avg. maximum	0.117	0.109	1.153	0.283
AWS
Avg. mean	0.097	0.111	0.763	0.382
Avg. maximum	0.055	0.109	0.251	0.616
RARE
Avg. mean	0.129	0.108	10.423	0.233
Avg. maximum	0.068	0.101	0.450	0.502
Semantic map
Avg. mean	0.095	0.112	0.719	0.397
Avg. maximum	0.052	0.110	0.221	0.638
Meaning map
Avg. mean	0.139	0.111	1.565	0.211
Avg. maximum	0.151	0.108	1.939	0.164

The effect of mind wandering across time

The findings thus far have demonstrated that scene content fixated 10 seconds before reported mind wandering was more visually salient than the content that was fixated before reported attentive viewing. These findings corresponded to previously identified mind wandering-related changes in content-independent measures of gaze behavior within the same dataset (i.e., fewer, longer, and more dispersed fixations) (Krasich et al., 2018), thus suggesting a shift in both what and how visual information was sampled. These analyses focused on the fixations 10 seconds prior to a thought probe because Krasich et al. (2018) had previously shown the most robust mind wandering-related changes in content-independent measures of gaze behaviors within this time frame. Here, we report a secondary post hoc analysis of the association between mind wandering and visual salience across viewing time, predicting that the relationship would dissipate further back in time before the mind wandering report.

We first created four 10-second time windows with respect to the onset of the thought probe (40–30 seconds before probe, 30–20 seconds before probe, etc.). Then, within each window, we averaged the mean and maximum salience and semantic scores of fixated locations (z-scored). We then modeled each dependent variable as a probe response (two levels: paying attention [reference group] and mind wandering) by time window (four levels with 10–0 seconds before probe as the reference) interaction, with image viewing time (z-scored) (Becker et al., 1988) as the fixed-effect variable and with participant and image as random effects. Significance testing was conducted using two-tailed tests with α set to 0.05, with Bonferroni corrections (i.e., visual salience p < 0.017; semantic informativeness p < 0.025).

We did not observe any significant mind wandering by time window interactions in scores from the GBVS model, semantic maps, or meaning maps (all p > 0.150). This indicates that the previously observed null effects of mind wandering with respect to these dependent variables in the 10 seconds prior to reports of mind wandering were consistent across viewing time. There were, however, trending or significant interactions when visual salience was measured by the AWS (mean, χ2 = 10.048, p = 0.018; maximum, χ2 = 10.498, p = 0.015) and RARE models (mean, χ2 = 7.392, p = 0.060; maximum, χ2 = 8.979, p = 0.031). We followed up these interactions with pairwise comparisons within each time window using the emmeans package in R (Lenth, 2018) and again controlled for multiple comparisons with Bonferroni corrections (p < 0.013; 0.05/4). The effect of mind wandering could only be observed in the 10 seconds prior to the mind wandering report (see Table 4). Any time frame more than 10 seconds prior showed no effect of mind wandering.

Table 4.

Standardized coefficients and test statistics investigating the effect of mind wandering on visual salience within 10-second time windows with respect to probe onset in the main study. Notes: β = standardized coefficients; SE = standard error; asterisk (*) indicates statistical significance after Bonferroni adjustments for multiple comparisons (p < 0.013).

	AWS
	Average mean			Average maximum
	β	SE	p	β	SE	p
Time before probe
0–10 s	0.292	0.104	0.005*	0.316	0.103	0.002*
10–20 s	0.080	0.104	0.443	–0.002	0.102	0.987
20–30 s	0.051	0.104	0.662	0.049	0.102	0.633
30–40 s	–0.160	0.104	0.123	–0.132	0.102	0.196
	RARE
	Average mean			Average maximum
	β	SE	p	β	SE	P
Time before probe
0–10 s	0.261	0.108	0.017	0.266	0.100	0.008*
10–20 s	0.129	0.108	0.232	0.112	0.099	0.257
20–30 s	–0.092	0.108	0.397	–0.104	0.099	0.296
30–40 s	–0.060	0.108	0.583	–0.050	0.099	0.617

Conceptual replication and joint-experiment analyses

As we noted in the Introduction, Krasich et al. (2018) reported a separate successful conceptual replication of their main study that showed clear and robust mind wandering-related changes in content-independent measures of gaze behavior (i.e., fewer, longer, and more dispersed fixations). In this study, a different group of 41 participants completed a scene memorization task that was embedded within a larger task battery (Faber et al., in press). Participants studied six contiguously presented images of urban scenes for 60 seconds each in preparation for a later memory test. Thought probes were presented in pseudorandom time intervals of 90 to 120 seconds (M = 33.03 seconds, SD = 15.58 seconds) such that they occurred mid-viewing, and participants received a total of three thought probes each. Participants reported mind wandering on an average of 47% of thought probes (SD = 50%).

We endeavored to use this replication experiment to verify our findings related to visual salience and semantic informativeness described above. Visual salience maps for each image were generated using the GBVS, AWS, and RARE models. Semantic interest maps were generated using ratings from a new sample of 28 laboratory participants, and meaning maps were generated using a new sample of 150 MTurk participants following the same procedures used in the main study. The average mean and maximum salience and semantic scores of fixated locations were computed across the fixations that occurred 10 seconds prior to probe onset. Each of these dependent variables was modeled using linear mixed-effects regression models with probe response (two levels: paying attention [reference group] and mind wandering) and image viewing time (z-scored) (Becker et al., 1988) as fixed-effects variables and participant and image as random effects. Because this task was randomly embedded within a larger task battery, we also included its task order as a categorical fixed-effect covariate.

As with the main study, we first assessed the sensitivity of this replication study with respect to the analyses of interest by estimating the MDESs of mind wandering on each dependent variable following similar procedures as in the main study. The specific MDES and associated power for each dependent variable are reported in Appendix D. The average MDES was 0.50 and ranged from 0.45 to 0.50, which indicates that this study was on average only powerful enough to detect an average effect sizes of 0.50 and greater. The largest effect size observed in the analysis of the main study, however, was 0.343, indicating that the replication was not suitably powered to assess the effect of mind wandering on content-dependent behaviors (which were substantially smaller than the effects observed using content-independent measurements by Krasich et al., 2018). As a result, it would not be surprising to fail to replicate our previously observed findings related to mind wandering and visual salience within this dataset (indeed, analysis of the replication data returned universally null results; coefficients and test statistics for each predictor are reported in Appendix F).

We can, however, make use of the replication from Krasich et al. (2018) by combining it with the main study in a set of joint-experiment analyses. In doing so, we can ensure that the relationship between mind wandering and visual salience holds when additional data, collected from a different group of participants in a different experimental context, is also considered. We can also determine if the effects of semantic informativeness in the main study emerge in a more powerful statistical analysis. For these analyses, we modeled each dependent variable (z-scored by experiment) using mixed-effects linear regressions with probe response (two levels: paying attention [reference group] and mind wandering), image viewing time (z-scored by experiment), and experiment (two levels: main study [reference group] and replication) as fixed-effect variables and participant and image as random effects. Bonferroni corrections were again incorporated to account for familywise error.

MDESs

The MDESs for these joint-experiment analyses were estimated following similar procedures as the main study. The specific MDES and associated power for each dependent variable are reported in Appendix D. The average MDES was 0.27 and ranged from 0.25 to 0.30, which shows improved sensitivity over the main study.

The effect of mind wandering on fixations 10 seconds prior to thought probes

Coefficients and test statistics for each predictor are reported in Appendix G, and the most relevant test statistics are reported in Table 2. The findings showed that, when measuring visual salience with the GBVS, there was still no effect of mind wandering on the average mean and maximum salience score of fixated locations. Mind wandering was, however, associated with greater average mean and maximum AWS and RARE salience scores. These effects survived Bonferroni corrections and were stronger than those observed in the main study. There was no effect of mind wandering on fixations to semantically informative scene content as measured by the semantic interest maps, but, when measuring semantic informativeness with the meaning maps, mind wandering tended to be associated with greater scores. These effects were not statistically significant, however, especially after correcting for multiple comparisons.

We further quantified the observed effects of mind wandering by comparing the aforementioned regression analyses with baseline models that predicted salience and semantic scores without probe response as a predictor variable. That is, these baseline models included only image viewing time (z-scored by experiment) and experiment (two levels: main study [reference group] and replication) as fixed-effect variables and participant and image as random effects. The findings for these model comparisons are reported in Table 5. They showed that including probe response as a predictor variable significantly improved the ability of the baseline models to predict average mean and maximum AWS and RARE scores, as indicated by lower Akaike information criterion (AIC)/Bayesian information criterion (BIC) values and significantly different deviances. Including probe response as a predictor variable did not significantly improve the ability of the model to predict average mean and maximum GBVS, semantic map, and meaning map scores. These findings further indicate a link between mind wandering and the propensity to fixate on visually salient scene content.

Table 5.

Test statistics exploring unique variance in mind wandering explained by visual salience and semantic informativeness. Notes: LLV = log-likelihood value; df = degrees of freedom; asterisk (*) indicates statistical significance after Bonferroni adjustments (salience scores p < 0.017; semantic scores p < 0.025).

	df	AIC	BIC	LLV	Deviance	χ2(1)	p
GBVS
Average mean
Baseline	6	1402	1427	–695	1390
With probe response	7	1402	1432	–694	1388	1.258	0.262
Average maximum
Baseline	6	1405	1430	–696	1393
With probe response	7	1406	1435	–696	1392	1.079	0.299
AWS
Average mean
Baseline	6	1432	1458	–710	1420
With probe response	7	1423	1453	–705	1409	10.783	0.001*
Average maximum
Baseline	6	1402	1428	–695	1390
With probe response	7	1392	1422	–689	1378	11.695	0.001*
RARE
Average mean
Baseline	6	1465	1491	–726	1453
With probe response	7	1458	1487	–722	1444	9.810	0.002*
Average maximum
Baseline	6	1424	1450	–706	1412
With probe response	7	1415	1445	–700	1401	11.259	0.001*
Semantic map
Average mean
Baseline	6	1410	1436	–699	1398
With probe response	7	1412	1442	–699	1398	.066	0.797
Average maximum
Baseline	6	1393	1419	–691	1381
With probe response	7	1395	1425	–691	1381	.160	0.689
Meaning map
Average mean
Baseline	6	1325	1350	–656	1313
With probe response	7	1323	1352	–654	1309	4.217	0.040
Average maximum
Baseline	6	1275	1300	–631	1263
With probe response	7	1273	1303	–630	1259	3.433	0.064

Considered collectively, findings from the joint-experiment analysis were consistent with those observed in the main study; that is, mind wandering was most associated with fixations to scene content that was more visually salient than fixated content before reports of attentive viewing.

General discussion

The current work focused on visual and cognitive factors that have been linked to gaze control and how these relationships vary across attentional states. Previous work has shown that mind wandering is associated with fewer, longer, and more dispersed fixations in a scene memorization task, with the most robust effects occurring 10 seconds prior to reported mind wandering (Krasich et al., 2018). Our goal in this report was to determine if such shifts in gaze behavior were characterized by content-dependent adjustments to gaze control mechanisms. To do so, we assessed how the visual system samples visually salient and semantically informative scene content during mind wandering.

We operationalized mind wandering as moments directly prior to when participants reported not being focused on the scene memorization task and, thus, to some degree were perceptually decoupled from the processing of the external world (e.g., Murphy et al., 2018; Schooler et al., 2011). Mind wandering was self-reported following occasional thought probes that queried the focus of participants’ attention. We then compared the associations among fixation location, visual salience, and semantically informative content within the scene prior to probes where participants indicated that they were paying attention to their scene memorization task and where participants admitted to mind wandering.

Our main study revealed an increased propensity for fixated scene content to be more visually salient (as measured by two of the three models used to operationalize salience) in the 10 seconds prior to reported mind wandering compared to reported attentive viewing. This time window corresponds to that in which changes in content-independent measures of gaze behavior were previously observed (Krasich et al., 2018). This fixated scene content also tended to be more semantically informative when operationalized in terms of local identifiability, but the effect of mind wandering was not statistically significant. No differences were observed across attentional states when semantic content took into account the importance of local scene regions to the overall scene content. As such, findings from the main study indicate that gaze was directed to scene content that was more visually salient during mind wandering compared to attentive viewing. This suggests that changes in the spatial aspects of gaze during mind wandering reflect a content-dependent shift in what visual information is sampled.

Unfortunately, a conceptual replication provided by Krasich et al. (2018) was underpowered with respect to the effect sizes observed for visual salience, even though this same replication study was able to reveal strong mind wandering-related changes in content-independent measures of gaze behaviors (i.e., fewer, longer, and more dispersed fixations). That said, combining the data from the main study and the replication study yielded a more powerful statistical analysis in which all effects from the main study alone were maintained. Furthermore, we were able to show with this analysis that the ability to predict what content will be fixated by an observer is improved by knowing the observer's attentional state while viewing the scene.

Both the main study and the joint-experiment analyses also highlight a contrast in magnitude between the smaller association between mind wandering and what information is viewed (i.e., more salient regions) versus the much larger association between mind wandering and how information is viewed (e.g., more slowly). From a theoretical point of view, this suggests that the link between mind wandering and changes in local gaze behaviors are weaker, more fragile, and/or more sensitive to task-specific idiosyncrasies than content-independent measures of gaze behaviors. For example, the effects of mind wandering were only observed in two of the three models of visual salience, indicating that at least the idiosyncratic procedures for computing salience characterizes the mind wandering–salience link. That is, the AWS and RARE models reflect contrasts relative to the entire image and do not incorporate a center bias. The GBVS model characterizes salience in terms of difference across local regions and does favor regions centrally located. Although it is unclear which computational difference across these models best characterizes the mind wandering–visual salience link, our findings do indicate some nuance in this relationship. This nuance requires further exploration, but it does suggest that content-independent gaze measures may provide a more efficacious set of parameters for identifying mind wandering across a range of contexts and tasks.

Future work is certainly needed to establish the links among gaze, scene content, and mind wandering, as well as whether, and to what extent, stimulus-specific or task-specific idiosyncrasies might influence these effects. The stimuli used in this study were admittedly few in number (12) and restricted in range (urban scenes). Also, beyond memorization tasks like the one used here, observers have many different goals when viewing or interacting with visual information. Thus, the extent to which the relationship between gaze and mind wandering may be modulated by exposure to different scenes, tasks, or intensions remains an interesting question. Despite these limitations, however, a clear message that emerges from our data is that contemporary frameworks of gaze control are incomplete, and explanatory models of gaze must account for both shifts in sampling rate (i.e., longer fixations) and shifts in the kind of information that is sampled (i.e., higher salience, higher local semantics) during mind wandering.

Although the exact nature of these mechanistic changes will require a great deal of additional work, our results give us an important first look into new ways of thinking about gaze and attention during mind wandering. Our data, for example, suggest that gaze control mechanisms may “rebalance” salient and semantically informative information during mind wandering. As attention shifts away from in-depth visual processing, gaze is more likely to be directed toward scene content that is visually distinct and stands out, and less time is spent interrogating visually indistinct or difficult to interpret scene regions. Our findings are also consistent with the levels of inattention hypothesis derived from studies of mindless reading (Schad, Nuthmann, & Engbert, 2012). The levels of inattention hypothesis conceptualizes mind wandering as being a matter of degree, where “weak” and “deep” mind wandering have different effects on gaze. During deep mind wandering both low- and high-level processes are decoupled, whereas during weak mind wandering high-level processing is decoupled but low-level processing is intact. The shift in fixations toward salient information in our study may reflect weak mind wandering, where low-level properties become more important in the absence of higher-level cognition. Thus, the shift from weak to deep mind wandering may constitute the basis for our proposed rebalance of information that influences gaze control as mind wandering occurs.

An alternative account posits that the visual system may operate following similar principles across bouts of attentive viewing and mind wandering, but with an inefficiency that decreases sampling rate (i.e., fewer and longer fixations) and elicits a sort of exploration–exploitation tradeoff (e.g., Jepma & Nieuwenhuis, 2011) reflected by an increase in fixation dispersion (Faber et al., in press; Krasich et al., 2018). Moreover, increased noise or variability in gaze control may inconsistently give rise to content-dependent changes or unspecified changes not directly captured by visual salience or semantic informativeness. Future work should further discern the relationship between mind wandering and content-dependent factors aside from visual salience and semantic informativeness to further assess this possibility.

In conclusion, everyday thoughts frequently consist of mind wandering, during which visual and cognitive processing is attenuated. Corresponding changes in gaze behaviors suggest a shift in how the visual system samples information in light of perceptual decoupling thought to occur during mind wandering. This reflects a prioritization of visually salient scene content (and perhaps local semantics), although this effect may be sensitive to task-specific as well as mind wandering-specific idiosyncrasies. Theoretical frameworks and computational models of gaze control that consider fixation allocation should account for these changes in gaze associated with mind wandering for a comprehensive account of the visual processing priorities of the visual system across various attentive states. Doing so would inform applied efforts to predict and detect mind wandering in real time (e.g., Hutt, Krasich, Mills, Bosch, White, Brockmole, & D'Mello, 2019), disentangling how the specific content of a scene should be considered or whether focusing on content-independent measures of gaze behavior is optimal.

Table C1.

Coefficients for all variables in the regression models assessing the mean visual salience of fixated scene content prior to self-reported mind wandering for the main study. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient; asterisk (*) indicates statistical significance after Bonferroni adjustments (salience scores p < 0.017; semantic scores p < 0.025).

	GBVS			AWS			RARE
Predictors	β	95% CI	p	β	95% CI	p	β	95% CI	p
Intercept	–0.028	–0.301 to 0.246	0.843	0.096	–0.354 to 0.163	0.469	–0.072	–0.290 to 0.145	0.515
Probe response, MW	0.076	–0.129 to 0.282	0.467	0.328	.124 to 0.532	0.002	0.281	0.069 to 0.493	0.009*
View time	0.046	–0.042 to 0.134	0.304	0.016	–0.074 to 0.105	0.731	–0.035	–0.128 to 0.058	0.465
Random effects
σ2	0.73			.81			.89
τ00	0.11ID; 0.18Image			0.01ID; 0.17 Image			0.00ID; 0.11 Image
ICC	0.28			.19			.11
N	12Image; 51ID			12Image; 51ID			12Image; 51ID
Observations	406			406			406
Marginal R2/conditional R2	0.003/0.284			0.021/0.206			0.016/0.127

Table C2.

Coefficients for all variables in the regression models assessing the mean semantic informativeness of fixated scene content prior to self-reported mind wandering for the main study. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	Semantic map			Meaning map
Predictors	β	95% CI	p	β	95% CI	p
Intercept	0.008	–0.266 to 0.283	0.952	–0.058	–0.419 to 0.304	0.754
Probe response, MW	0.043	–0.164 to 0.250	0.685	0.174	–0.004 to 0.352	0.055
View time	–0.025	–0.114 to 0.064	0.584	0.017	–0.059 to 0.092	0.666
Random effects
σ2	0.76			0.53
τ00	0.09ID; 0.18Image			0.15ID; 0.35Image
ICC	0.26			0.49
N	12Image; 51ID			12Image; 51ID
Observations	406			406
Marginal R2/conditional R2	0.001/0.264			0.006/0.490

Table C3.

Coefficients for all variables in the regression models assessing the maximum visual salience of fixated scene content prior to self-reported mind wandering for the main study. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient; asterisk (*) indicates statistical significance after Bonferroni adjustments (salience scores p < 0.017; semantic scores p < 0.025).

	GBVS			AWS			RARE
Predictors	β	95% CI	p	β	95% CI	p	β	95% CI	p
Intercept	–0.027	–0.292 to 0.239	0.843	–0.096	–0.376 to 0.190	0.469	–0.069	–0.376 to 0.237	0.657
Probe response, MW	0.064	–0.145 to 0.272	0.550	0.343	0.144 to 0.543	0.001	0.296	0.100 to 0.493	0.003*
View time	0.045	–0.044 to 0.135	0.320	0.008	–0.079 to 0.096	0.854	–0.032	–0.118 to 0.055	0.472
Random effects
σ2	0.77			0.77			0.77
τ00	0.09ID; 0.17Image			0.01ID; 0.22Image			0.00ID; 0.26Image
ICC	0.25			0.23			0.11
N	12Image; 51ID			12 Image; 51ID			12Image; 51ID
Observations	406			406			406
Marginal R2/conditional R2	0.003/0.250			0.023/0.245			0.023/NA

Table C4.

Coefficients for all variables in the regression models assessing the maximum semantic informativeness of fixated scene content prior to self-reported mind wandering for the main study. Note: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	Semantic map			Meaning map
Predictors	β	95% CI	p	β	95% CI	p
Intercept	0.018	–0.266 to 0.283	0.952	–0.062	–0.454 to 0.329	0.755
Probe response, MW	0.038	–0.164 to 0.250	0.685	0.164	–0.006 to 0.335	0.059
View time	–0.023	–0.114 to 0.064	0.584	–0.004	–0.076 to 0.069	0.922
Random effects
σ2	0.75			0.48
τ00	0.05ID; 0.22 Image			0.12ID; 0.43 Image
ICC	0.27			0.53
N	12Image; 51ID			12Image; 51ID
Observations	406			406
Marginal R2/conditional R2	0.001/0.270			0.005/0.536

Table D1.

The minimum detectable effect sizes and associated power for the main study, replication, and joint-experiment analyses. Notes: MDES = estimated minimum detectable effect size that retained a power of at least 0.80; (1 – β) = average power associated with the MDES; 95% CI = 95% confidence interval for the estimated average power.

	Main study			Replication			Joint experiment
	MDES	(1 – β)	95% CI	MDES	(1 – β)	95% CI	MDES	(1 – β)	95% CI
GBVS
Avg. mean	0.30	0.80	0.78–0.83	0.50	0.87	0.85–0.89	0.25	0.82	0.79–0.84
Avg. maximum	0.30	0.80	0.77–0.82	0.45	0.85	0.83–0.87	0.25	0.81	0.79–0.84
AWS
Avg. mean	0.30	0.82	0.79–0.84	0.50	0.82	0.79–0.84	0.30	0.92	0.91–0.94
Avg. maximum	0.30	0.84	0.82–0.86	0.45	0.84	0.81–0.86	0.25	0.81	0.79–0.84
RARE
Avg. mean	0.30	0.85	0.83–0.87	0.55	0.82	0.79–0.84	0.30	0.90	0.88–0.92
Avg. maximum	0.35	0.89	0.87–0.91	0.50	0.80	0.78–0.83	0.30	0.92	0.90–0.93
Semantic map
Avg. mean	0.30	0.84	0.82–0.86	0.50	0.82	0.79–0.84	0.30	0.92	0.90–0.94
Avg. maximum	0.30	0.81	0.79–0.84	0.50	0.87	0.85–0.89	0.25	0.82	0.80–0.84
Meaning map
Avg. mean	0.30	0.90	0.88–0.92	0.55	0.85	0.82–0.87	0.25	0.87	0.85–0.89
Avg. maximum	0.25	0.80	0.78–0.83	0.50	0.87	0.85–0.89	0.25	0.90	0.88–0.92

Table F1.

Coefficients for all variables in the regression models assessing the mean visual salience of fixated scene content prior to self-reported mind wandering for the replication. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	GBVS			AWS			RARE
Predictors	β	95% CI	p	β	95% CI	p	β	95% CI	p
Intercept	–0.078	–1.049 to 0.892	0.875	–0.046	–0.844 to 0.752	0.910	–0.247	–0.833 to 0.339	0.409
Probe response, MW	0.106	–0.216 to 0.428	0.518	0.233	–0.108 to 0.575	0.180	0.345	–0.022 to 0.711	0.066
View time	0.258	0.022 to 0.494	0.032	0.176	–0.075 to 0.427	0.168	–0.094	–0.338 to 0.149	0.448
Trial position 2	–0.410	–1.105 to 0.285	0.248	–0.622	–1.407 to 0.163	0.121	–0.291	–1.075 to 0.494	0.468
Trial position 3	0.467	–0.174 to 1.108	0.153	0.671	–0.054 to 1.396	0.070	0.573	–0.150 to 1.296	0.120
Trial position 4	0.182	–0.380 to 0.743	0.526	0.072	–0.562 to 0.705	0.824	0.231	–0.402 to 0.864	0.475
Trial position 5	–0.235	–0.826 to 0.356	0.435	–0.395	–1.062 to 0.272	0.246	–0.075	–0.740 to 0.589	0.824
Trial position 6	0.207	–0.425 to 0.839	0.521	–0.064	–0.780 to 0.653	0.862	–0.083	–0.795 to 0.630	0.820
Trial position 7	0.369	–0.405 to 1.143	0.350	0.060	–0.814 to 0.934	0.892	0.352	–0.517 to 1.222	0.427
Random effects
σ2	0.57			0.62			0.77
τ00	0.12ID; 0.60Image			0.20ID; 0.32Image			0.14ID; 0.10Image
ICC	0.56			0.46			0.24
N	3Image; 41ID			3Image; 41ID			3Image; 41ID
Observations	116			116			116
Marginal R2/conditional R2	0.096/0.600			0.118/0.523			0.090/0.305

Table F2.

Coefficients for all variables in the regression models assessing the mean semantic informativeness of fixated scene content prior to self-reported mind wandering for the replication. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	Semantic map			Meaning map
Predictors	β	95% CI	p	β	95% CI	p
Intercept	–0.117	–0.950 to 0.717	0.784	0.071	–0.577 to 0.718	0.831
Probe response, MW	–0.208	–0.549 to 0.134	0.233	0.126	–0.238 to 0.491	0.497
View time	–0.030	–0.270 to 0.211	0.808	0.256	0.006 to 0.506	0.045
Trial position 2	0.070	–0.610 to 0.751	0.839	–0.461	–1.220 to 0.297	0.233
Trial position 3	0.263	–0.364 to 0.889	0.411	0.165	–0.534 to 0.864	0.644
Trial position 4	0.181	–0.369 to 0.731	0.519	–0.100	–0.713 to 0.513	0.749
Trial position 5	0.467	–0.110 to 1.045	0.113	–0.373	–1.016 to 0.271	0.256
Trial position 6	0.406	–0.209 to 1.022	0.196	–0.035	–0.723 to 0.653	0.921
Trial position 7	0.304	–0.451 to 1.059	0.430	–0.002	–0.843 to 0.840	0.997
Random effects
σ2	0.70			0.77
τ00	0.06 ID; 0.41Image			0.11 ID; 0.16Image
ICC	0.40			0.26
N	3Image; 41ID			3Image; 41ID
Observations	116			116
Marginal R2/conditional R2	0.032/0.423			0.084/0.326

Table F3.

Coefficients for all variables in the regression models assessing the maximum visual salience of fixated scene content prior to self-reported mind wandering for the replication. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	GBVS			AWS			RARE
Predictors	β	95% CI	p	β	95% CI	p	β	95% CI	p
Intercept	–0.056	–1.106 to 0.994	0.917	0.037	–0.933 to 10.007	0.941	–0.164	–0.730 to 0.402	0.570
Probe response, MW	0.103	–0.202 to 0.408	0.507	0.207	–0.103 to 0.517	0.191	0.355	–0.016 to 0.726	0.060
View time	0.269	0.045 to 0.492	0.018	0.192	–0.040 to 0.424	0.104	–0.048	–0.286 to 0.190	0.691
Trial position 2	–0.406	–1.055 to 0.243	0.220	–0.674	–1.383 to 0.035	0.062	–0.398	–1.224 to 0.428	0.344
Trial position 3	0.439	–0.160 to 1.037	0.151	0.562	–0.092 to 10.216	0.092	0.471	–0.291 to 1.233	0.226
Trial position 4	0.160	–0.364 to 0.684	0.549	–0.051	–0.623 to 0.521	0.861	0.090	–0.576 to 0.756	0.791
Trial position 5	–0.286	–0.838 to 0.266	0.310	–0.506	–1.108 to 0.097	0.100	–0.281	–0.980 to 0.418	0.431
Trial position 6	0.202	–0.387 to 0.792	0.502	–0.028	–0.674 to 0.618	0.933	–0.045	–0.797 to 0.707	0.907
Trial position 7	0.331	–0.391 to 1.053	0.369	–0.029	–0.819 to 0.760	0.942	0.276	–0.638 to 1.191	0.554
Random effects
σ2	0.52			0.51			0.77
τ00	0.10ID; 0.74Image			0.16ID; 0.59Image			0.19ID; 0.06Image
ICC	0.62			0.60			0.25
N	3Image; 41ID			3Image; 41ID			3Image; 41ID
Observations	116			116			116
Marginal R2/conditional R2	0.094/0.653			0.106/0.639			0.088/0.313

Table F4.

Coefficients for all variables in the regression models assessing the maximum semantic informativeness of fixated scene content prior to self-reported mind wandering for the replication. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	Semantic map			Meaning map
Predictors	β	95% CI	p	β	95% CI	p
Intercept	–0.141	–0.809 to 0.527	0.678	0.104	–0.702 to 0.911	0.800
Probe response, MW	–0.163	–0.490 to 0.164	0.329	0.056	–0.279 to 0.391	0.742
View time	–0.245	–0.473 to 0.018	0.035	0.285	0.050 to 0.520	0.017
Trial position 2	–0.036	–0.693 to 0.622	0.915	–0.417	–1.077 to 0.243	0.216
Trial position 3	0.651	0.046 to 1.257	0.035	0.210	–0.398 to 0.817	0.499
Trial position 4	0.122	–0.409 to 0.654	0.651	–0.153	–0.687 to 0.381	0.574
Trial position 5	0.417	–0.141 to 0.975	0.143	–0.397	–0.957 to 0.164	0.165
Trial position 6	0.202	–0.393 to 0.797	0.506	0.006	–0.591 to 0.603	0.984
Trial position 7	0.401	–0.329 to 1.130	0.281	0.042	–0.690 to 0.775	0.910
Random effects
σ2	0.64			0.68
τ00	0.07ID; 0.22Image			0.05ID; 0.38Image
ICC	0.31			0.39
N	3Image; 41ID			3Image; 41ID
Observations	116			116
Marginal R2/conditional R2	0.105/0.384			0.091/0.446

Table G1.

Coefficients for all variables in the regression models assessing the mean visual salience of fixated scene content prior to self-reported mind wandering for the joint-experiment analyses. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient; asterisk (*) indicates statistical significance after Bonferroni adjustments (salience scores p < 0.017; semantic scores p < 0.025).

	GBVS			AWS			RARE
Predictors	β	95% CI	p	β	95% CI	p	β	95% CI	p
Intercept	–0.034	–0.329 to 0.260	0.820	–0.087	–0.348 to 0.174	0.513	–0.075	–0.291 to 0.140	0.493
Probe response, MW	0.100	–0.074 to 0.273	0.261	0.296	0.120 to 0.473	0.001	0.292	0.110 to 0.474	0.002*
View time	0.066	–0.015 to 0.147	0.112	0.030	–0.053 to 0.113	0.481	–0.039	–0.125 to 0.047	0.373
Experiment [Replication]	0.018	–0.613 to 0.649	0.956	–0.032	–0.597 to 0.533	0.911	–0.059	–0.523 to 0.405	0.803
Random effects
σ2	0.71			0.79			0.88
τ00	0.11ID; 0.22Image			0.04ID; 0.17Image			0.01ID; 0.11Image
ICC	0.32			0.21			0.12
N	15Image; 92ID			15Image; 92ID			15Image; 92ID
Observations	0.71			0.79			0.88
Marginal R2/conditional R2	0.11 ID			0.04 ID			0.01 ID

Table G2.

Coefficients for all variables in the regression models assessing the mean semantic informativeness of fixated scene content prior to self-reported mind wandering for the joint-experiment analysis. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	Semantic map			Meaning map
Predictors	β	95% CI	p	β	95% CI	p
Intercept	0.027	–0.263 to 0.316	0.857	–0.056	–0.404 to 0.292	0.753
Probe response, MW	–0.024	–0.199 to 0.151	0.787	0.167	0.007 to 0.327	0.040
View time	0.004	–0.622 to 0.629	0.991	–0.010	–0.762 to 0.741	0.978
Experiment (replication)	–0.028	–0.110 to 0.055	0.510	0.040	–0.035 to 0.114	0.295
Random effects
σ2	0.74			0.57
τ00	0.08ID; 0.22Image			0.15ID; 0.32ImageF
ICC	0.28			0.45
N	15Image; 92ID			15ImageF; 92ID
Observations	522			522
Marginal R2/conditional R2	0.001/0.282			0.007/0.455

Table G3.

Coefficients for all variables in the regression models assessing the maximum visual salience of fixated scene content prior to self-reported mind wandering for the joint-experiment analyses. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient; asterisk (*) indicates statistical significance after Bonferroni adjustments (salience scores p < 0.017; semantic scores p < 0.025)

	GBVS			AWS			RARE
Predictors	β	95% CI	p	β	95% CI	p	β	95% CI	p
Intercept	–0.035	–0.332 to 0.262	0.818	–0.082	–0.380 to 0.217	0.592	–0.070	–0.359 to 0.219	0.634
Probe response, MW	0.093	–0.082 to 0.267	0.297	0.299	0.128 to 0.469	0.001	0.299	0.126 to 0.472	0.001*
View time	0.066	–0.016 to 0.147	0.114	0.024	–0.056 to 0.105	0.552	–0.028	−0.111 to 0.054	0.503
Experiment (replication)	0.023	–0.618 to 0.664	0.944	–0.032	–0.683 to 0.620	0.924	–0.068	−0.701 to 0.566	0.835
Random effects
σ2	0.73			0.74			0.80
τ00	0.08ID; 0.23Image			0.03ID; 0.24Image			0.01ID; 0.23Image
ICC	0.30			0.27			0.23
N	15Image; 92ID			15Image; 92ID			15Image; 92ID
Observations	522			522			522
Marginal R2/conditional R2	0.006/0.304			0.019/0.283			0.018/0.243

Table G4.

Coefficients for all variables in the regression models assessing the maximum semantic informativeness of fixated scene content prior to self-reported mind wandering for the joint-experiment analyses. Notes: β = standardized coefficients; CI = confidence interval; σ2 = within-group variance; τ00 = between-group variance; ICC = intraclass correlation coefficient.

	Semantic map			Meaning map
Predictors	β	95% CI	p	β	95% CI	p
Intercept	0.038	–0.268 to 0.344	0.808	–0.057	–0.444 to 0.330	0.774
Probe response, MW	–0.036	–0.208 to 0.135	0.678	0.143	–0.009 to 0.295	0.064
View time	–0.048	–0.129 to 0.033	0.245	0.024	–0.047 to 0.094	0.511
Experiment (replication)	–0.002	–0.670 to 0.666	0.995	0.005	–0.840 to 0.850	0.991
Random effects
σ2	0.73			0.52
τ00	0.05ID; 0.25Image			0.12ID; 0.42Image
ICC	0.29			0.51
N	15Image; 92ID			15Image; 92ID
Observations	522			522
Marginal R2/conditional R2	0.003/0.293			0.005/0.510