Brain activity forecasts video engagement in an internet attention market.

The growth of the internet has spawned new "attention markets," in which people devote increasing amounts of time to consuming online content, but the neurobehavioral mechanisms that drive engagement in these markets have yet to be elucidated. We used functional MRI (FMRI) to examine whether individuals' neural responses to videos could predict their choices to start and stop watching videos as well as whether group brain activity could forecast aggregate video view frequency and duration out of sample on the internet (i.e., on youtube.com). Brain activity during video onset predicted individual choice in several regions (i.e., increased activity in the nucleus accumbens [NAcc] and medial prefrontal cortex [MPFC] as well as decreased activity in the anterior insula [AIns]). Group activity during video onset in only a subset of these regions, however, forecasted both aggregate view frequency and duration (i.e., increased NAcc and decreased AIns)-and did so above and beyond conventional measures. These findings extend neuroforecasting theory and tools by revealing that activity in brain regions implicated in anticipatory affect at the onset of video viewing (but not initial choice) can forecast time allocation out of sample in an internet attention market.

In a world awash in information, individuals must constantly choose how to allocate their time (1). Furthermore, “a wealth of information creates a poverty of attention and a need to allocate that attention efficiently” (2), which might be measured in terms of time use. Thus, as the availability of information increases, time grows more scarce and valuable. While most decision-making research focuses on choices involving the allocation of money, this work focuses on internet “attention markets”—in which people exchange their time (rather than money) to access online content (3).

Since the turn of the 21st century, online markets have offered an expanding range of products (e.g., television, movies, music, books, and games). As individuals peruse these products, search engine optimization tools (e.g., on google, youtube, and netflix) and social media applications (e.g., facebook, instagram, and twitter) collect data in efforts to increase consumer engagement (4). As a result, people currently spend over 1 billion h/d in attention markets watching video content (https://www.statista.com), and the world’s second-most popular search engine is a video site (i.e., youtube.com) (5). Neurobehavioral mechanisms underlying this extensive online engagement with video content, however, have yet to be elucidated.

Affective neuroscience advances potentially offer new ideas and tools for understanding online time allocation. Researchers have noted that positively aroused affect (e.g., excitement) precedes approach behavior, whereas negatively aroused affect (e.g., anxiety) precedes avoidance behavior (6). In humans, functional MRI (FMRI) has been leveraged to investigate second-to-second changes in neural correlates of this “anticipatory affect” and subsequent choice (7). An affect–integration–motivation (AIM) framework arising from this research specifically associates nucleus accumbens (NAcc) activity with positive arousal, anterior insula (AIns) activity with negative or general arousal, and medial prefrontal cortex (MPFC) activity with value integration of affect with other considerations (e.g., probability or time; “integration”). Activity in these regions then promotes behavioral approach toward or avoidance of stimuli under consideration (“motivation”) (8). Other accounts additionally implicate posterior cingulate cortex (PCC) activity in value-based integration and attention (9, 10). Based on the AIM framework, neural predictions of individual choice have extended across a broad range of scenarios. For instance, anticipatory NAcc and MPFC activity predicts individuals’ choices to approach purchases (11, 12), investments (13, 14), and charitable appeals (15, 16). Conversely, anticipatory AIns activity predicts individuals’ choices to avoid purchases (17, 18), investments (13, 14), and charitable appeals (19). Research has not yet examined, however, whether this activity can also predict time allocation.

If brain activity predicts time allocation in individuals, might it also forecast time allocation at the aggregate level in attention markets [here, “prediction” refers to predicting behavior within individuals, whereas “forecast” refers to predicting behavior across individuals but not necessarily across time (20)]? In laboratory samples, group NAcc activity can forecast music sales (21), microloan appeal success (22), advertisement efficacy (23), food purchases (24), news article popularity (25), and crowdfunding support (26). Group MPFC activity can also forecast aggregate responses to antismoking advertisements (27) and news articles (25). In some cases, sampled neural measures can even forecast market-level behavior above and beyond traditional behavioral and self-report measures (20). While counterintuitive, the notion that brain activity can forecast aggregate behavior even when behavior cannot is consistent with a “partial scaling” account, in which some components of the choice process generalize more broadly across people than others. For instance, in the context of the AIM framework, partial scaling might occur if early affective components generalize more broadly than later value integrative components, motivational components, or even subsequent choice behavior (20). Thus, partial scaling implies that neural measures might reveal “hidden” information about market demand (28).

This work aimed to test whether brain activity in regions implicated in anticipatory affect and value integration might predict individual choices to allocate time and furthermore, whether group activity in a subset of those regions (i.e., those implicated in anticipatory affect) would scale to forecast aggregate time allocation. The current research, therefore, critically aimed to forecast video engagement in an internet attention market (youtube.com) based on video thumbnail attributes rated in a pilot study (). During a “video choice” task, participants first chose whether they would like to watch videos based on thumbnails selected from youtube.com. Next, in a “video viewing” task, participants watched 32 videos from the video choice task and were given the option to stop watching each video after viewing the first 4 to 8 s. Then, participants rated each video based on how engaging they and others might find it and how positive (vs. negative) and aroused (vs. not aroused) they felt while watching it (Fig. 1). Throughout, FMRI scans and eye blink data were acquired.

Fig. 1.

Viewing task procedure and trial structure. Participants received instructions and underwent auditory and eye-tracking calibration. During scanning, they saw thumbnails of each of the videos and indicated whether they wanted to view the video (video choice task); then, they were forced to watch at least 4 s of each video before responding to an option to stop watching the video, after which they rated the video on four scales (video viewing task). Finally, participants completed a debriefing survey (Upper). In video viewing task trials, participants watched each video (4 to 8 s), saw a prompt that allowed them to stop watching (stop), and rated four aspects of the video sequentially (engaging for self, engaging for others, valence, and arousal; 4 s each; Lower).

By extending the AIM framework (8) from the allocation of money to time, we predicted that brain activity in the NAcc (positively), AIns (negatively), and MPFC (positively) would predict individuals’ choices to allocate time to videos. Consistent with partial scaling of anticipatory affect (20), we further predicted that sampled anticipatory affective responses (positive NAcc and negative AIns activity) but not value integrative responses (MPFC or PCC activity) or behavior would forecast aggregate video viewing frequency and duration on the internet. Since aggregate viewing frequency metrics minimally require people to watch at least a few seconds of online videos, these hypotheses primarily focused on brain activity during the initial viewing period in the video viewing task (although similar analyses of brain activity during the video choice task are also described for comparison).

Results

Individual Prediction.

Linear mixed effects models with random intercepts for participants and videos tested whether choice behavior, affect ratings, and brain activity predicted individual behavior in the video viewing task. We first tested whether behavioral choices in the video choice task predicted view percentage (i.e., duration as a percentage of total video length) during the subsequent video viewing task. Videos that were chosen vs. not chosen in the video choice task were viewed for 12% longer during the subsequent video viewing task (t = 12.30, P < 0.001), accounting for 40% of the variance in individual view duration. Next, we tested whether affect ratings (i.e., positive arousal and negative arousal) were associated with view duration (engagement ratings correlated strongly with affect ratings and therefore, were not included in these models). As predicted, positive arousal was positively associated with view percentage (β = 0.186, t = 22.49, P < 0.001), but negative arousal was not. Overall, self-report ratings accounted for 53% of the variance in individual view percentage.

We then tested whether brain activity in regions implicated in anticipatory affect (i.e., NAcc, AIns) and value integration (i.e., MPFC, PCC) predicted individual view percentage (). Average percentage signal change was extracted from each volume of interest during the video viewing task at video onset (first 4 s or onset), over the entire video (average), and prior to video offset (final 4 s prior to stopping or video ending or offset), and lagged by 6 s to account for the hemodynamic peak.

For onset activity and consistent with an early phasic response (29, 30), NAcc onset activity positively predicted view percentage (β = 0.039, t = 3.39, P < 0.001), whereas AIns onset activity negatively predicted view percentage (β = −0.047, t = −3.91, P < 0.001). PCC onset activity also negatively predicted view percentage (β = −0.025, t = −2.53, P < 0.05), but MPFC onset activity did not (β = 0.013, t = 1.27, P = 0.20). For average activity, both NAcc (β = 0.053, t = 4.39, P < 0.001) and MPFC average activities positively predicted view percentage (β = 0.041, t = 4.33, P < 0.001), while AIns average activity again negatively predicted view percentage (β = −0.065, t = −5.20, P < 0.001), but PCC average activity did not (β = −0.012, t = −1.23, P = 0.22). For offset activity, only PCC offset activity predicted view percentage (β = 0.023, t = 2.13, P < 0.05).

Together, these findings suggest that brain activity can augment behavioral and self-report predictions of individual choices to allocate attention to videos, consistent with the AIM framework implication that both early activity in affective regions and later activity in value integrative regions contribute to individual choice (8). In the subset of participants with analyzable physiological data (n = 30), eye blinks also accounted for significant additional variance in predicting video duration but not video choice, consistent with attentional engagement ().

Aggregate Forecasting.

Internet metadata were extracted for each experimental video from youtube.com () and translated into two aggregate outcome metrics. The first was calculated as aggregate view frequency , which normalized the log normal distribution of view frequency (31) and indexed aggregate video engagement. The second was calculated as aggregate view percentage , which controlled for total video duration and indexed aggregate (inverse) video disengagement. Since each video received one value for each of the aggregate metrics, critical analyses included group activity averaged by video from brain regions in which activity at video onset had predicted individual behavior. To verify the forecasting specificity of brain activity sampled at video onset, additional exploratory analyses also examined average and offset activity from these regions ().

Preliminary analyses tested whether sampled behavior and ratings could forecast the aggregate time allocation metrics (Fig. 2 and Table 1). For behavior, to closely align laboratory measures with aggregate metrics, sampled choice in the video choice task was used to forecast aggregate view frequency, while sampled view percentage in the video viewing task was used to forecast aggregate view percentage. For ratings, since affect and engagement ratings were strongly correlated, only affect ratings were included in the critical models (similar results were obtained after substituting engagement for affect ratings) ().

Fig. 2.

Sample and aggregate view choice and duration. Large points represent means. Error bars represent 95% CIs. Points represent data for each participant. Sample choices to watch videos were not significantly associated with aggregate view frequency (Left). Sample view percentage was positively associated with aggregate view duration (Right).

Table 1.

Forecasting aggregate behavior

	Aggregate view frequency				Aggregate view percentage
	Behavior	Ratings	Brain	Combined	Behavior	Ratings	Brain	Combined
Constant	2.770 (0.15)***	2.770 (0.15)***	2.770 (0.13)***	2.770 (0.13)***	0.684 (0.02)***	0.684 (0.02)***	0.684 (0.02)***	0.684 (0.01)***
Choice (yes/no)	0.141 (0.15)			0.180 (0.18)
View percentage					0.071 (0.02)***			0.098 (0.02)***
Positive arousal		−0.020 (0.16)		−0.118 (0.22)		0.025 (0.02)		−0.057 (0.02)*
Negative arousal		0.223 (0.16)		0.121 (0.16)		−0.006 (0.02)		−0.007 (0.02)
NAcc (onset)			0.653 (0.18)***	0.604 (0.19)**			0.055 (0.03)*	0.044 (0.02)†
AIns (onset)			−0.540 (0.22)*	−0.545 (0.25)*			−0.081 (0.03)*	−0.070 (0.03)*
MPFC (onset)			−0.026 (0.16)	0.010 (0.18)			−0.017 (0.02)	−0.012 (0.02)
PCC (onset)			0.295 (0.18)	0.257 (0.20)			0.017 (0.03)	0.027 (0.02)
Adjusted R2	−0.004	0.005	0.279	0.231	0.354	−0.024	0.158	0.509
AIC	83	84	75	79	−57	−42	−46	−61
Classification accuracy	0.531	0.438	0.688	0.625	0.656	0.344	0.594	0.688
CV RMSE	0.870	0.854	0.790	0.875	0.097	0.122	0.120	0.097

Statistics are standardized coefficients and SEs. Significance is as indicated.

P < 0.05 (two tailed); **P < 0.01 (two tailed); ***P < 0.001 (two tailed); †P < 0.10 (nonsignificant trend).

Behaviorally, neither sampled choice nor affect ratings forecast aggregate view frequency (all P values > 0.05). Sampled view percentage did, however, forecast aggregate view percentage (β = 0.071, t = 4.24, P < 0.001; accounting for 35.4% of its variance), but affect ratings did not (Fig. 2 and Table 1). Bivariate correlations first probed whether brain activity was associated with aggregate time allocation metrics (Fig. 3). NAcc onset activity alone was positively correlated with aggregate view frequency (r = 0.46, P < 0.01; ρ = 0.54, P < 0.01), while AIns onset activity alone was negatively correlated with aggregate view percentage (r = −0.37, P < 0.05; but this nonparametric association was not significant: ρ = −0.09, P = 0.61). These focused pairwise associations matched activation patterns in exploratory whole-brain analyses ().

Fig. 3.

Associations between group brain activity and aggregate view frequency and duration. Zero-order bivariate Pearson’s and Spearman’s correlations. (Top) NAcc, (Middle) AIns, and (Bottom) MPFC. n = 36 videos. Significance is as indicated. *P < 0.05 (two tailed); **P < 0.01 (two tailed).

Next, the critical tests examined contributions of brain activity in two sets of multivariate models forecasting the aggregate time allocation metrics. For view frequency, while choice and ratings models did not significantly forecast aggregate view frequency, the neural model did (Table 1). An SD increase in NAcc onset activity increased view frequency by 0.65 (t = 3.73, P < 0.001), but an SD increase in AIns onset activity decreased view frequency by 0.54 (t = −2.44, P < 0.05). This neural model accounted for 28% of the variance in aggregate view frequency, which decreased to 23% after including choice and ratings measures and adjusting for added predictors (with contributions from both NAcc and AIns onset activity). Furthermore, both the Akaike information criterion (AIC) and cross-validated root mean squared error (CV RMSE) increased for the combined model, consistent with superiority of the neural model. Activity from these brain regions did not significantly contribute to the combined model when average or offset activity was substituted for onset activity ().

For view duration, while sampled view duration did significantly forecast aggregate view duration (β = 0.706, t = 4.24, P < 0.001), ratings did not. As with view frequency, the neural model forecasts view duration, with significant contributions from both NAcc (β = 0.055, t = 2.11, P < 0.05) and AIns (β = −0.081, t = −2.44, P < 0.05) onset activity. This neural model, however, accounted for less variance (16%) than did the behavioral model (35%) (Table 1). Still, the combined model indicated that neural measures continued to significantly forecast view duration above and beyond behavior and self-report (51%) while reducing AIC, consistent with superiority of the combined model. Activity from these brain regions did not significantly contribute to the combined model when average or offset activity was substituted for onset activity, with the exception of NAcc offset activity (). Together, these findings suggest that NAcc and AIns video onset activity may forecast both aggregate engagement and disengagement with videos in opposite directions (Table 1).

Subsequent exploratory analyses indicated that brain activity during the video choice task did not forecast aggregate video engagement metrics (). Supplementary analyses yielded findings that were similar to the critical analyses after controlling for participants’ previous choices in the video choice task (), after omitting activity from value integrative brain regions from the models (), and after excluding participants familiar with any of the videos (). Similar results were also obtained for NAcc activity when forecasting rank-ordered aggregate metrics, although the contribution of AIns activity was diminished in these models (). Finally, average eye blink measures did not forecast either of the aggregate metrics ().

Discussion

Combining neuroimaging with a behavioral task allowed us to test whether activity in affective brain regions might foreshadow peoples’ allocation of time to watching videos—both in individuals undergoing scanning and out of sample in an internet attention market (youtube.com). In individuals, brain activity in regions previously shown to predict allocation of money also predicted choices to allocate time to watching videos. In an internet attention market, sampled activity in a subset of these regions implicated in anticipatory affect at video onset generalized to forecast the frequency of choices to allocate time as well as the duration of time allocated to videos.

At the level of individual choice, these findings extend applications of the neuroeconomic toolkit from predicting choices regarding money allocation (11) to choices about time allocation. The findings are broadly consistent with an AIM framework account in which affective, integrative, and motivational neural responses predict and promote individual choice (8). Specifically, early NAcc activity and later MPFC activity were associated with video engagement, while early AIns activity was associated with disengagement. The temporal variation in predictive activity further suggested that these different regions supported distinct functions since NAcc and AIns onset activity predicted view percentage, consistent with anticipatory affective responses, whereas MPFC average activity predicted view percentage, consistent with subsequent value integration.

At the level of aggregate choice, these findings provide evidence for partial scaling in time allocation (20) since activity in a subset of regions that predicted individual choice also forecasts video engagement metrics on the internet. Specifically, activity during video onset in regions associated with anticipatory affect (i.e., increased NAcc and decreased AIns activity) forecasts aggregate view frequency, but activity in more cortical regions associated with value integration (i.e., MPFC and PCC) as well as behavioral choice did not. While both were predicted, NAcc activity scaled more robustly than AIns activity, suggesting that positive arousal may generalize more broadly to these video attention markets than negative arousal—although such an inference requires further verification. Activity in response to video onset in regions associated with anticipatory affect also forecasts aggregate view duration along with behavioral choices to stop viewing videos, while activity in regions associated with value integration did not. Although MPFC activity did not forecast time allocation in this research, it has been associated with aggregate behavior in the few other neuroforecasting studies that have examined time-relevant outcomes (i.e., responding to advertisements and forwarding newspaper articles) (25, 27, 32). Although anticipatory affect might represent particularly salient features for video attention markets, other markets might elicit considerations more relevant to value integration (e.g., identity and future plans)—a possibility ripe for future exploration. Notably, brain activity during a previous video choice task did not forecast online engagement, suggesting that brain activity in response to stimuli that directly generate aggregate engagement metrics (i.e., video viewing) may support more robust forecasts. These findings highlight contextual factors that might help sharpen neural forecasts of time allocation related to regions (implicated in anticipatory affect), timing (in response to video onset), and task (matched across levels of analysis).

From a practical standpoint, these findings potentially inform choice applications by demonstrating that “hidden information” from neural data can improve market forecasts. Forecasting the viral spread of videos on the internet with behavioral and content measures has proven challenging (33, 34), although some evidence suggests that affective content may promote the transmission of news and messages (35–37). While the current forecasts targeted aggregate metrics that were collected prior to scanning, they were unlikely to be influenced by prior exposure since few reported familiarity with any of the stimuli, and omission of their data did not alter the findings (). Nonetheless, implicit familiarity is more difficult to account for than explicit familiarity and therefore, should be addressed in further work (38). Furthermore, as in other neuroforecasting studies (24), future research might ideally forecast aggregate time allocation metrics collected after acquisition of brain data (although changes in the youtube.com interface have rendered these metrics less accessible). The cost-effectiveness of neuroforecasting applications may vary since tracking subcortical activity with FMRI is currently expensive and requires technical expertise. In large markets (such as youtube.com), however, even a small increment in forecasting might translate into millions of views and substantial revenue (31, 39). Optimization of techniques for harvesting relevant brain signals may improve forecasts. While this initial foray used easily computable and interpretable time course activity summaries from predefined brain regions of interest, more complex analyses, such as intersubject correlation (40), might yield additional useful signals but are difficult to implement with the current data since video durations necessarily varied across participants. Future research might also systematically deconstruct and label dynamic video content (41) to determine whether specific video features influence aggregate engagement. Although eye-tracking measures of blink rate did not forecast aggregate metrics in the current study (), other peripheral measures might eventually augment forecasts.

Overall, this research extends a growing literature on neuroforecasting by demonstrating the possibility of forecasting time allocation online. Few studies have directly compared neural predictors of individual vs. aggregate choice, and existing comparisons have focused on the allocation of money rather than time (20). Design innovations allowing the assessment of engagement as well as disengagement within the same video stimuli at both individual and aggregate levels of analysis made such direct comparisons possible. These results, therefore, catalyze future work that may more precisely specify which features of stimuli, individuals, and markets best support neuroforecasting.

Materials and Methods

Participants and Procedure.

All procedures were approved by the Stanford Institutional Review Board and conducted at the Stanford Center for Neurobiological Imaging.

Participants.

Forty participants (25 female; age 25.28 ± 7.35 y) were recruited online to participate in an FMRI study about watching videos. Participants completed a prescreening survey prior to being invited to take part in the study. Exclusion criteria included FMRI eligibility (metal implants, pregnancy, psychotropic medication, claustrophobia, neurologic disorders, or prior brain trauma) and corrected vision (due to eye tracking). Four participants were excluded from analysis due to frequent volume displacements (more than four instances greater than 6 mm or two voxel sides) during scan acquisition (otherwise, only trials with motion exceeding this threshold were omitted from analyses).

Procedure.

Participants completed informed consent and screening forms, were briefed about the experiment, and entered the FMRI scanner. Participants inserted earbuds with volume that was calibrated, and a near-infrared eye tracker was set up and calibrated before scanning ().

Video choice task.

Participants viewed 64 video thumbnails with associated titles and captions presented in pseudorandom order ( discusses stimulus selection procedures) split into two runs of 32 trials each. In each trial, a video thumbnail was displayed (2 s) followed by the title and caption (6 s). Participants indicated whether they wanted to watch the video with a button box using either the index or little finger of their left hand to choose the option on the right or the left, respectively (4 s). Accept (vs. reject) response buttons were randomized laterally across trials. Between trials, participants visually fixated on a central cross-hair (intertrial interval; 2 to 6 s).

Video viewing task.

After the video choice task, participants read instructions for the video viewing task as an anatomical scan was acquired. Participants then completed 32 trials of the video viewing task in four runs of 8 trials (Fig. 1). To control stimulus content, all participants watched the same 32 videos. Thus, selection of presented videos did not depend on participants’ responses in the previous video choice task. Trials were presented in pseudorandom order in either a forward- or reverse-ordered sequence. The videos included 54- to 172-s clips selected from the popular science channels “Discovery” and “Animal Planet” on youtube.com and were sampled from a larger database of 2,950 videos with thumbnails that had previously been normed using a larger online (Amazon Mechanical Turk) sample in a pilot study (). Video stimulus sampling aimed to maximize variance in aggregate view duration (calculated as a percentage of the total video length; view percentage) as well as affective ratings (i.e., high vs. low arousal and high vs. low valence) (). Participants were informed that they would see videos that they had previously encountered in the video choice task but were not required to respond consistently with their previous choices. Regardless of their responses in the video choice task, each trial began with video playback followed by a gray square that randomly appeared after 4 to 8 s on the left or right side of the centrally displayed video. Participants then had the option of skipping the rest of the video at any subsequent point by pressing a button with their left or right index finger, corresponding to the gray square’s position. After watching (or skipping) each video, participants rated the video on four four-point Likert scales: 1) how engaging they found the video (not at all engaged to strongly engaged), 2) how engaging others would find the video (not at all engaged to strongly engaged), 3) how positively vs. negatively they felt about the video (strongly negative to strongly positive), and 4) how unaroused vs. aroused they felt about the video (not at all aroused to strongly aroused). Participants had 4 s to respond to each rating prompt, and the directions of rating anchors (i.e., ascending or descending) were laterally counterbalanced for each trial in a pseudorandom order. Prior to statistical analysis, ratings were mean deviated within each participant, and arousal and valence ratings were projected onto independent affective axes of positive arousal and negative arousal (as described in ref. 7). To encourage viewing at least part of each video and to discourage time- and order-dependent responses (including skipping), progress indicators were omitted, and participants were not informed about the number or duration of videos in the task.

Behavioral summary.

Of 40 scanned participants, 39 completed all 32 trials, while 1 completed 31 trials (due to a technical interruption); 38 participants watched at least one video to completion, while 2 skipped every video at some point. On average, 74% of the videos were eventually skipped, and the average video view duration was 48%. Only 17 video choice trials (1.32%) had missing values due to participants not responding within 4 s. Only 13 video viewing trials (1.02%) lasted less than 8 s, justifying distinct analyses of neural responses to video onset, average, and offset. Linking behavior in the video choice task with the video viewing task revealed that participants had previously chosen to watch an average of 44% of the 32 videos in the video viewing task. Rated positive arousal and negative arousal values of the videos were relatively independent and uncorrelated (r = 0.254, t = 1.40, P = 0.20).

Neuroimaging acquisition and analysis.

Brain images were acquired and processed as described in previous neuroforecasting studies (26) ().

Data Availability.

Data and code that support reported analyses are available online in the Open Science Framework (42) and Neurovault (43).