Digital assessment of working memory and processing speed in everyday life: Feasibility, validation, and lessons-learned.

OBJECTIVES: Cognitive functioning is often impaired in mental and neurological conditions and might fluctuate throughout the day. An existing experience-sampling tool was upgraded to assess individual's cognition in everyday life. The objectives were to test the feasibility and validity of two momentary cognition tasks.
METHODS: The momentary Visuospatial Working Memory Task (mVSWMT) and momentary Digit Symbol Substitution Task (mDSST) were add-ons to an experience sampling method (ESM) smartphone app. Healthy adults (n = 49) between 19 and 73 years of age performed the tasks within an ESM questionnaire 8 times a day, over 6 consecutive days. Feasibility was determined through completion rate and participant experience. Validity was assessed through contextualization of cognitive performance within intrapersonal and situational factors in everyday life.
FINDINGS: Participants experienced the tasks as pleasant, felt motivated, and the completion rate was high (71%). Social context, age, and distraction influenced cognitive performance in everyday life. The mVSWMT was too difficult as only 37% of recalls were correct and thus requires adjustments (i.e. fixed time between encoding and recall; more trials per moment). The mDSST speed outcome seems the most sensitive outcome measure to capture between- and within-person variance.
CONCLUSIONS: Short momentary cognition tasks for repeated assessment are feasible and hold promise, but more research is needed to improve validity and applicability in different samples. Recommendations for teams engaging in the field include matching task design with traditional neuropsychological tests and involving a multidisciplinary team as well as users. Special attention for individual needs can improve motivation and prevent frustration. Finally, tests should be attractive and competitive to stimulate engagement, but still reflect actual cognitive functioning.

Introduction

Cognition is a key determinant when it comes to the question how well an individual manages daily tasks and performs everyday activities. Only if a person can remember, concentrate, communicate, plan, and reason, is he/she able to cope with the requirements of life. The link between cognition and functioning has been demonstrated not only in people with cognitive impairments (Aretouli and Brandt, 2010) or mental health issues (Mansueto et al., 2018), but also healthy individuals (Shimada et al., 2016). Therefore, it is necessary to take cognition into account when aiming to understand daily patterns or support functioning.

Memory functions, processing speed, and other cognitive abilities are usually assessed in clinical or laboratory settings rather than natural environments. Brief cognitive screenings are used in routine primary care to identify individuals at risk for cognitive dysfunction, while comprehensive, multidimensional neuropsychological batteries have the purpose to establish a diagnose or functional profile (Roebuck-Spencer et al., 2017). Casaletto and Heaton (2017) highlight the ‘common complaint’ regarding neuropsychological assessments to be ‘their apparent lack of relevance to the real-life problems’ (p.11) that individuals experience in their everyday life. Furthermore, as doctor visits occur periodically (Palladino et al., 2016), the assessments provide a rather temporary picture of one's cognitive ability. This traditional approach may thus affect the ecological validity of neuropsychological test results (Chaytor and Schmitter-Edgecombe, 2003).

An empirically validated daily diary method, known as the Experience Sampling Method (ESM) or Ecological Momentary Assessment (EMA), allows to collect real-world information (Csikszentmihalyi and Larson, 1987; Delespaul, 1995; Shiffman et al., 2008). Digital ESM technologies using smartphone apps prompt participants repeatedly over the day to reflect on their own behaviour, affect, and contextual factors (Hébert et al., 2018; van Os et al., 2017). Over the last decade, interest has increased to not only depict affect and activities with experience sampling, but also to include the area of cognitive functioning. It is relevant to bridge the lab-life gap and observe cognition closely in everyday life and in a more dynamic way (Bielak et al., 2017). Learning that cognitive performance can fluctuate over time and grasping which daily circumstances influence cognitive performance can help patients to optimize activities in daily life. Momentary cognition tasks provide a more dynamic understanding of cognition throughout the day that can be clinically relevant when recovering from somatic or psychological complaints. For ecologically valid cognitive assessments, digital diary methods delivered via smartphone apps offer unique opportunities.

The status of cognitive assessments in everyday life

Cognitive assessments in everyday life through technology are relatively new. A recent review by Moore et al. (2017) has identified 12 studies that use self-administered digital cognitive assessments. A brief literature search on PubMed identified 13 additional studies (see Fig. 1 for a visualized summary).

Fig. 1

Visualized summary of studies focussing on cognitive assessments in everyday life by cognitive domains, populations, and the intrapersonal factors set in relation with outcome(s) (see Publication); this summary does not aim to be exhaustive, but provides a first overview of the field). NP = Neuropsychological.

Various cognitive domains are considered for the assessments in everyday life. Often, more than one task is used and multiple domains are evaluated. Working memory and attention/reaction time are most prevalent, which may be explained by the fact that these domains are generally relevant for various patient populations as they are affected in neurological conditions (Huntley and Howard, 2010; Silveri et al., 2007) as well as mental illnesses (Luck and Gold, 2008; Roitman et al., 2000). Participants' age ranged from adolescents to older adults. While some studies focused on healthy individuals (e.g. Hyun et al., 2018; Riediger et al., 2014; Thomson et al., 2009; Tiplady et al., 2009), others included patient populations (e.g. Bouvard et al., 2018; Marhe et al., 2013; Waters et al., 2013). Normative data remains unavailable and, therefore, the validation of mobile cognitive assessments in healthy individuals is still relevant.

To describe the internal validity of the momentary tasks, the contextualization of the momentary performance is a key element. Therefore, previous studies took intrapersonal factors such as age, mood, and drug use into account. Furthermore, psychometric properties of the mobile cognition tasks, including between-person reliability, within-person reliability, and construct validity should be elaborated (Moore et al., 2017).

The present study

The present study aims to evaluate the feasibility and validity through contextualization of two newly developed, short momentary cognition tasks implemented in an existing platform with a high sampling frequency. A high sampling frequency of eight times a day allows to describe a detailed picture of daily mood and cognition fluctuations. Furthermore, high-frequency sampling is the standard in this particular ESM platform (Delespaul, 1995), which is broadly used in research as well as in clinical settings (Jongeneel et al., 2018; Verhagen et al., 2017). This ESM tool has the advantage of being available for both Android and iOS users. It is crucial to build on existing technologies rather than reinventing new devices to support sustainable use. Therefore, this ESM tool is an important target for future cognition task development. A healthy sample was recruited to perform a visuospatial working memory task and a processing speed task within a momentary assessment. Additionally, the ESM items assess a wide range of momentary intrapersonal and situational factors such as mood and social context.

To determine the feasibility, completion rate and participant satisfaction were used. Validity was assessed through an exploration of the contextual variation of cognitive performance. Cognitive performance measured with the momentary cognition tasks was evaluated by relating cognition outcomes to each other as well as to relevant ESM-measures such as mood, fatigue, and current company. The results are discussed with regard to lessons-learned during the development process and future implications. Prospectively, researchers and clinicians who are already familiar with the smartphone app can include cognitive measures in everyday life alongside affect and context to understand and support various patient populations.

Methods

Participants

Recruitment from the general population was performed via snowball sampling, using media advertisements and the personal network as seeds. Sample size was based on previous feasibility studies using the same experience sampling app and aimed for a minimum of 30 participants (Edwards et al., 2016; van Knippenberg et al., 2017). Fifty-one participants provided written informed consent. Participants were at least 18 years old, had a full understanding of the Dutch language, and were able to handle a smartphone device (Android) with a beta version of the PsyMate™ app (version 213–253) (see Section 2.2.1 for details). Participants who could not use their own smartphone device were provided with a 5th generation iPod on which the same version of the PsyMate™ app was installed. Individuals were excluded based on medication use that could influence cognitive performance and current treatment for cognitive or mental health complaints. The standing ethical committee of the Faculty of Psychology and Neuroscience, Maastricht University (ref.no.183_02_09_2017) granted ethical approval and the study was carried out in accordance with the Declaration of Helsinki.

Measurements

No traditional neuropsychological tasks were included in this study as the focus lay on the contextualization of the momentary cognition scores and their within-day fluctuation. To validate ESM items, the correlation between similar and dissimilar ESM items is suggested (Delespaul, 1995). As suggested by Chen et al. (2015), patterns of associations between items of quality of experiences (i.e., cognition) and other momentary items such as emotions related to the experiences should be logical, thereby supporting the internal validity of the data.

Experience sampling method (ESM)

The ESM was administered using the PsyMate™ Suite; a smartphone app and a cloud-based platform developed by Maastricht University and Maastricht UMC+ (www.psymate.eu). PsyMate™ is a parametrized and flexible tool for repeated assessments in everyday life. The application was programmed to emit an auditory and visual prompt (beep signal) eight times a day for six consecutive days, signaling the availability of a self-report questionnaire. These beep questionnaires were provided at semi-random time blocks of 112.5 min, between 7.30 AM and 10.30 PM and remained available for response during 15 min. Beep questionnaires included mood (i.e., positive and negative affect), physical status (i.e., hunger, fatigue, and pain), and context (i.e., location, activity, and social company) items as well as the two cognition tasks. Positive affect (PA) included the items ‘cheerful’, ‘energetic’, ‘relaxed’, ‘enthusiastic’, and ‘satisfied’, while negative affect (NA) was composed of the items ‘insecure’, ‘down’, ‘irritated’, ‘lonely’, ‘anxious’, and ‘guilty’. In line with ESM guidelines (Delespaul, 1995), a minimum of 16 valid beep questionnaires (1/3 of total) per participant had to be completed to be included in the analyses. One ESM assessment including the two momentary cognition tasks would not take longer than 2 min to complete.

In addition to the beep questionnaires, participants completed a morning assessment and an evening assessment every day, each consisting of seven self-report items. The morning questionnaire focused on self-reported sleep quality. The evening questionnaire focused on a global appraisal of the day. The majority of the items were answered on a seven-point Likert scale (1 = not at all, 4 = moderate, 7 = very). Some items contained categories (e.g. 0, 1, 2, 3, 4, 5, or >5 times awake during the night). The full list of ESM items is included in Appendix A.

PsyMate™ momentary Visualspatial Working Memory Task (mVSWMT)

The concept of the PsyMate™ mVSWMT is based on the popular card game ‘Memory’, also known as ‘Concentration’ or ‘Match Match’, where players turn cards to find matching pairs. ‘Memory’ has been used to study concentration and memory functions in various age groups (Lavenex et al., 2011; Schumann-Hengsteler, 1996). The mVSWMT aims to measure concentration and visuospatial working memory (i.e., encoding, maintaining, and retrieving visual information). The development team included psychiatric and neuropsychological healthcare professionals as well as ESM researchers. This team defined the following requirements for the mDSST: participants should be able to perform the mobile cognition task several times a day, the task needs to be short, sensitive to cognitive variation, and demonstrates no or a small learning effect.

The participants were instructed that they would see nine icons to remember. After the participant pressed the start button, icons were presented in a three-by-three grid for eight seconds (encoding phase; see left part of Fig. 2 a). Next, participants answered two interference questions on a seven-point Likert scale: ‘I think I remembered it all’, and ‘Generally, I feel well at the moment’. During the recall phase (see right part of Fig. 2 a), participants were presented with a three-by-three grid of blanc squares with one icon from the original nine above. An instruction stated to select the square of the original location of the presented icon. The selected square revealed the icons underneath to provide feedback. In this first conceptualization, only one trial per beep is provided to keep the ESM assessments as short as possible. Every beep moment, a unique set of symbols was presented. The grids were filled at random from a selection of 122 unique icons (see Fig. 2 b) and a random icon cue was selected from the grid. The outcome measure was correct/incorrect (correct = 1) during recall. The icons presented in the mVSWMT were chosen as they represent well-known objects of everyday life and are easily recognizable.

Fig. 2

a) PsyMate™ momentary Visuospatial Working Memory Task (mVSWMT) encoding and recall phase. b) Summary of icons presented in the PsyMate™ mVSWMT.

PsyMate™ momentary Digital Symbol Substitution Task (mDSST)

The PsyMate™ mDSST was inspired by the paper-pencil version of the Digit Symbol Substitution Task (DSST) of the Wechsler Adult Intelligence Scale (WAIS). The original WAIS task measures information processing speed and short-term working memory (Wechsler, 2008). The PsyMate™ version aims to measure information processing speed. The mDSST fulfils the momentary task requirements (performable several times a day; short; sensitive to cognitive variation; no/small learning effect) and the same team was consulted during task development.

At the end of the regular ESM beep questionnaire, an instruction screen appeared with a start button to be pressed when ready. At the top of the task screen, the numbers 1 to 9 with a corresponding symbol (similar to the WAIS DSST) were displayed for encoding. In the middle of the screen, different numbers were displayed one-by-one for each trial. At the bottom of the screen, participants had to select the symbol that corresponds to the number presented in the middle of the screen (see Fig. 3). Within a 30-seconds timeframe, participants had to accurately complete as many trials as possible. The 30-second timeframe was chosen to keep the ESM assessments as brief as possible, thereby minimizing interference during daily routines. While the number-symbol combinations stayed the same during a beep questionnaire, different sets of combinations were used across beeps. In total, ten unique encoding combinations with corresponding answer keys were used at random. Two mDSST outcome measures were computed: the number of trials (speed) and the percentage of correct trials (accuracy).

Fig. 3

PsyMate™ momentary Digit Symbol Sustitution Task (mDSST)

Debriefing questionnaire

Participants received a debriefing questionnaire that focused on their general experiences during the ESM week (e.g., ‘Was this a normal week?’, ‘Did the PsyMate™ use influence your daily activities?’), the usability of the PsyMate™ (e.g., ‘Were the PsyMate™ instructions clear?’, ‘Was using the PsyMate™ stressful?’), and their experiences with the PsyMate™ mDSST and mVSWMT (e.g., ‘To what extent was the task pleasant to perform?’, ‘Did you experience any technical difficulties?’). Both seven-point Likert scale questions and open-ended questions were used.

Procedure

Following informed consent, a one-hour briefing session took place at Maastricht University or at the participant's home. Sociodemographic information was gathered (e.g., gender, age, living situation, education level, current occupation, and ethnicity) and additional information was asked on current medication use, earlier treatment for mental illnesses, and cognitive complaints. After these general assessments, either the PsyMate™ was installed on the participant's smartphone device or the participant received an iPod with the latest PsyMate™ version installed. After checking the device settings for battery saving options and allowing push notifications, participants were instructed on how to use the PsyMate™ and the cognition tasks. Test trials were completed to become acquainted with the ESM procedure. Participants started with their six-day ESM period on the following day. During the second day, participants were called to check whether the application was working properly and to clarify potential questions. After the ESM period, a one-hour debriefing session took place during which participants completed the debriefing questionnaire and provided specific feedback with regard to the two cognition tasks.

Statistical analyses

Participant characteristics, feasibility, and acceptability of both cognition tasks were assessed by means of descriptive statistics (frequencies). The completion rate was calculated by comparing the mean percentage of valid beep moments to the total number of beep moments. The data collected with the PsyMate™ have a multilevel structure; beeps (level 1) were nested within participants (level 2). Multilevel regression analyses were used to assess cognitive variation over time and to check for learning effects. The session counter score was used as a proxy measure of time and consists of a sequence of beeps within subjects, ranging from 1 (first beep) up to 48 (last beep). Learning effects were examined by using the session counter to assess the effect over time, hours to assess a within-day time effect and study day (day 1 to 6) to assess a between-day time effect. It is expected that learning will not be linear; therefore, all time variables will be transformed to a logarithmic or quadratic function. Correct/Incorrect (mVSWMT), the number of trials within the 30-seconds time interval (speed), and the percentage of correct trials (accuracy) (mDSST) were used as dependent variables and a log transformation of the session counter (time), hour and its quadratic function, and a log transformation of study day as independent variables. To assess contextualization, dummy variables were created for location (at home vs. somewhere else) and company (alone vs. with others). Activity-related stress was conceptualized as an average of the items ‘I would rather be doing something else’, ‘This is difficult for me’, and ‘I can do this well’ (reverse coded). To assess the association between PA, NA, fatigue, activity-related stress, distraction, worrying, focusing, location, company, and sleep quality as independent variables and the cognition outcomes of the mVSWMT and mDSST as dependent variables, multilevel regression analyses were computed. Covariates in these multilevel models were quadratic age, gender, and possible learning effects as measured with the time variables. In order to investigate age effects, subgroup multilevel regression analyses were performed, splitting participants into a young group (<45 years) and an old group (45 years or older). Furthermore, Fischer-z transformations of by-subject Pearson's pairwise correlations were calculated between the cognitive outcome measures. Analyses were carried out using Stata version 13.0 (StataCorp, 2013). A two-sided significance level of 0.05 was used throughout.

Results

Seventy-one individuals expressed their interest in the study, of which 66 met the eligibility criteria. From these individuals, seven were not allowed to use mobile phones during work, four individuals did not have enough time, one could not participate because no device was available, and three individuals did not reply after receiving the information. In total, 51 individuals consented to be included, resulting in a 66% recruitment rate.

From the 51 participants who provided informed consent, two participants were excluded because of current treatment for mental health problems. This left data from 49 participants and 1499 beep records. Data of five participants could not be used due to technical problems (beta release of the software: loss of 70 records (4.67%), n = 3 transmission problems, n = 2 broken devices), leaving a final dataset with 44 participants and 1429 valid beep records. On average, participants completed 34 out of 48 beeps (SD = 7.03, range 17–47), resulting in a completion rate of 71%. The age of the n = 44 participants ranged from 19 to 73 years with a median of 36 years (M = 40, SD = 14.82). Sixty-six percent were women. Highest education level was skewed, with 6% having finished low education, 18% middle education, and 76% high education. Most participants had a fulltime job (61%), others worked part time (23%), studied (7%), took care of their own household (5%), or were retired (4%).

Feasibility

Evaluation of the PsyMate™ procedure

The items represented the participants' experiences well (M = 5.70, SD = 1.53), the PsyMate™ was easy to use (reverse coded, M = 1.25, SD = 0.78), and the verbal and written instructions were clear (respectively M = 6.86, SD = 0.41; M = 6.84, SD = 0.48). The PsyMate™ did not influence the participants' mood (M = 2.11, SD = 1.53), activities (M = 1.91, SD = 1.01), or social contact (M = 1.91, SD = 1.18). The number of beeps, duration of a beep, and sound had a low impact on the burden (respectively M = 3.2, SD = 1.79; M = 2.61, SD = 1.86; M = 2.32, SD = 1.62). Three people found the length of the questionnaire too long.

Evaluation of the mVSWMT

Participants reported that the mVSWMT was pleasant to use (M = 5.05, SD = 1.57), but rather difficult (M = 4.61, SD = 1.79). Six participants indicated that the interference questions between encoding and recall made the task difficult. Participants did not get distracted during the task (M = 2.93, SD = 1.53) and were highly motivated to perform well (M = 5.84, SD = 1.16). They indicated that they made few inaccuracies (M = 1.75, SD = 1.28) and would recommend the task to others (M = 5.25, SD = 1.62). They provided some suggestions for further improvement, namely a longer encoding phase and a timer. Participants reported strategies to recall the icons: reading aloud (7 times), creating a story or a mnemonic (6 times), remembering the icons and the location of the icons (5 times), and remembering the first, the middle, or the last row (6 times).

Evaluation of the mDSST

Participants reported that the mDSST was pleasant (M = 5.66, SD = 1.22) and easy to use (reverse coded, M = 1.86, SD = 0.90). They were not distracted during the task (M = 3.00, SD = 1.43) and highly motivated to perform well (M = 5.93, SD = 1.13). Participants reported that they made few inaccuracies (e.g., tapping symbol X instead of symbol Y) (M = 3.00, SD = 1.54). Fourteen people commented that the size of the response buttons was too small, potentially leading to inaccuracies. Participants would recommend the task to others (M = 5.82, SD = 1.11). They provided some suggestions for further improvement: to increase the symbol and number size or rotate the screen horizontally. This was especially an issue for iPod users since the screen was smaller.

Contextual factors

Participants experienced high PA (M = 5.08, SD = 0.69, range 3.35–6.66) and low NA (M = 1.49, SD = 0.55, range 1.01–3.20). Furthermore, they felt moderately fatigued (M = 3.01, SD = 1.19, range 1–5.64), were a little worried (M = 2.23, SD = 1.15, range 1–5.43), and experienced low activity-related stress (M = 2.44, SD = 0.56, range 1.61–3.69). Overall, participants reported a high level of focus during an activity (M = 4.78, SD = 0.77, range 3.11–6.45) and experienced low to moderate distraction during the mDSST (M = 2.79, SD = 0.88, range 1.05–4.53). Participants were alone in 29% of the time and in company 71% of the time. Furthermore, they spend 56% of the time at home and 44% somewhere else. According to the morning questionnaire, participants fell asleep after 5 to 15 min (40%) and woke up once during the night (34%). Participants slept well (M = 5.25, SD = 0.80, range 3.46–7) and felt well rested at the start of the day (M = 4.62, SD = 1.03, range 2.67–6.65).

Cognition in relation to contextual factors

mVSWMT

Overall, participants were correct in 37% of the mVSWMT assessments (SD = 0.16, range 0.07–0.74). There was no association between time (session counter score) and the mVSWMT outcome, (B = 0.01, SE = 0.01, p = .32, 95% CI = −0.01, 0.04), showing no within-day time effect (B = −0.004, SE = 0.003, p = .17, 95% CI = −0.01, 0.002), nor between-day time effect (B = 0.04, SE = 0.03, p = .17, 95% CI = −0.02, 0.09), indicating no learning-effect.

Participants made more mistakes when experiencing high PA (B = −0.03, p = .04), when in company (vs. being alone; B = −0.10, p < .001), and when being distracted (B = −0.03, p = .001). Being able to focus during an activity resulted in more correct answers (B = 0.04, p < .001). NA, fatigue, activity-related stress, location, worrying, and sleep quality (morning questionnaire) were not associated with the mVSWMT outcome. More mistakes were made with higher age (B = −0.00005, p = .002), whereas gender was not associated. For all results, see Appendix B.

To build the final multilevel regression model, a basis of PA, NA, and its interaction effect was extended with variables that were associated with the cognition outcome measure. No interaction effect was found between PA, NA, and the mVSWMT outcome. The effect of PA disappeared and participants again made more mistakes when in company (B = −0.09, p = .002), when distracted (B = −0.02, p = .03), and with older age (B = -0.00006, p < .001). Being able to focus during an activity was associated with more correct answers (B = 0.05, p < .001). The results of this analysis indicated that the seven predictors explained 6% of the overall variance (3% within-subject variance and 28% between-subject variance). Results of the final model for correct/incorrect are presented in Table 1.

Table 1

Final model of the mVSWMT outcome correct/incorrect.

	Correct/incorrect
	B	SE	p	95% CI
Model			<0.000⁎⁎⁎
PA	−0.06	0.03	0.08	−0.13,	0.01
NA	−0.06	0.09	0.48	−0.25,	0.12
PA × NA	0.01	0.02	0.33	−0.03,	0.05
Focus	0.05	0.01	<0.001⁎⁎⁎	0.03,	0.07
Company$	−0.09	0.03	0.002⁎⁎	−0.14,	−0.03
Distraction	−0.02	0.01	0.03⁎	−0.03,	−0.001
Age2	−0.00006	0.00002	<0.001⁎⁎⁎	−0.00009,	−0.00002

Note. PA = positive affect. NA = negative affect. PA × NA = interaction between positive and negative affect. Company = dummy variable of being alone versus with others. Age2 = quadratic function of age. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

mDSST

Due to technical problems, 82 times (5.7%) the mDSST did not follow the ESM questionnaire. These records were removed, leaving a sample of 42 participants with 1347 beep records. Participants completed on average 12 trials (SD = 2.57, min = 7.21, max = 16.83) within the 30-second timeframe, with on average 97% accuracy (SD = 1.81, min = 92.29, max = 100). Participants completed more trials over time (beep 1 vs. beep 48; B = 0.32, SE = 0.04, p < .001, 95% CI = 0.24, 0.40), showed no within-day time effect (B = 0.01, SE = 0.01, p = .12, 95% CI = −0.003, 0.03), but more completed trials at later study days (B = 0.61, SE = 0.08, p < .001, 95% CI = 0.45, 0.76). Accuracy was not associated with time (B = 0.08, SE = 0.20, p = .71, 95% CI = −0.32, 0.47), showing no within-day time effect (B = 0.05, SE = 0.04, p = .25, 95% CI = −0.03, 0.13) nor between-day time effect (B = −0.02, SE = 0.36, p = .96, 95% CI = −0.72, 0.68).

NA, activity-related stress, location, sleep quality, and gender were not associated with either cognitive outcome measure. Looking at speed, participants completed less trials when in company (B = −0.29, p = .001), when being distracted (B = −0.15, p < .001), and with older age (B = −0.001, p < .001). Furthermore, participants completed more trials when worrying (B = 0.09, p = .02). PA and being able to focus were not associated with speed. Looking at accuracy, participants made more mistakes when being tired (B = −0.34, p = .007) and with more distraction (B = −0.54, p < .001). Participants made less mistakes when they experienced more PA (B = 0.45, p = .03) and when they could focus better (B = 0.36, p = .01). Company was not associated with accuracy. For all results, see Appendix C.

Again, the basic model was extended with variables that were associated with the mDSST outcome measures. In the final model of speed (see Table 2), mood showed no effect, and time and worry effects disappeared. Participants completed fewer trials when in company (B = −0.18, p = .04), when being distracted (B = −0.15, p < .001), and with older age (B = −0.002 p < .001). The results of this analysis indicated that the nine predictors explained 48% of the overall variance (8% within-subject variance and 56% between-subject variance).

Table 2

Final model of the mDSST speed outcome.

	Speed
	B	SE	p	95% CI
Model			<0.001⁎⁎⁎
PA	0.04	0.11	0.74	−0.18,	0.26
NA	0.01	0.30	0.97	−0.57,	0.59
PA × NA	−0.02	0.06	0.81	−0.14,	0.11
Worry	0.06	0.04	0.14	−0.02,	0.14
Company$	−0.18	0.09	0.04⁎	−0.34,	−0.01
Distraction	−0.15	0.02	<0.001⁎⁎⁎	−0.20,	−0.10
Age2	−0.002	0.0002	<0.001⁎⁎⁎	−0.002,	−0.001
Time$	0.17	0.10	0.08⁎⁎	−0.02,	0.37
Study day$	0.34	0.18	0.06	−0.02,	0.70

Note. PA = positive affect. NA = negative affect. PA × NA = interaction between positive and negative affect. Company = dummy variable of being alone versus with others. Age2 = quadratic function of age. Time = log-transformed session counter score. Study day = log-transformed day of study score. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

In the final model of accuracy (see Table 3), the effect of fatigue and PA disappeared and no other mood effects were found. Participants made more mistakes when being distracted (B = −0.51, p < .001). The results of this analysis indicated that the six predictors explained 3% of the overall variance (2% within-subject variance and 20% between-subject variance).

Table 3

Final model of the mDSST accuracy outcome.

	Accuracy
	B	SE	p	95% CI
Model			<0.001⁎⁎⁎
PA	−0.22	0.50	0.67	−1.19,	0.76
NA	−1.15	1.28	0.37	−3.66,	1.36
PA × NA	0.25	0.28	0.37⁎	−0.30,	0.81
Fatigue	−0.22	0.14	0.13⁎⁎	−0.50,	0.06
Focus	0.14	0.15	0.34	−0.15,	0.44
Distraction	−0.51	0.11	<0.001⁎⁎⁎	−0.72,	−0.30

Note. PA = positive affect. NA = negative affect. PA × NA = interaction between positive and negative affect. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

Young vs. older age and cognition

In order to gain more insight into age effects, exploratory subgroup analyses were performed for both cognition tasks. Splitting age groups for the mVSWMT data resulted in n = 26 (59%) in the young group (<45 years) and n = 18 (41%) in the older age group (45 years or older). In both groups, being able to focus resulted in more correct answers (young age: B = 0.05, p < .001; older age: B = 0.04, p = .01). However, in the older group, participants made more mistakes when being in company (B = −0.13, p = .001) and with increasing age (B = −0.00007, p = .047).

With the mDSST data, data from two participants were excluded due to technical problems (see Section 3.4.2), leaving n = 26 in the young group (62%) and n = 16 in the older age group (38%). Results for speed remained largely the same in both groups, showing a main effect of distraction with higher distraction resulting in more mistakes. A similar result was found in both groups for accuracy, where higher distraction resulted in fewer trials. The difference was that the younger group completed more trials over time (B = 0.30, p = .047). The older group did not show a time effect on number of beeps (log-transformed session counter score), but completed more trials over study days (B = 0.47, p = .02) and when PA was higher (B = 0.33, p = .04). In addition, fewer trials were completed when being in company (B = −0.22, p = .02), and with increasing age (B = −0.001, p < .001) within the older group but not the younger group. Results of the subgroup analyses are presented in Appendix D.

Correlations between mVSWMT and mDSST

Fisher-z transformations of by subject pairwise correlations were used between the mVSWMT outcome correct/incorrect, and the mDSST outcomes speed and accuracy. Over subject averages, there were no significant correlations between correct/incorrect and speed (z = 1.36, p = .09), nor accuracy (z = −0.35, p = .64).

Discussion

Feasibility of the momentary cognition tasks

This study confirms the feasibility of two newly developed momentary cognition tasks within the PsyMate™ app in healthy individuals. The completion rate was high (71%) and is in line with other ESM studies with and without cognition tasks (Lenaert et al., 2019; Swendsen et al., 2019). Furthermore, participants overall experienced the cognition tasks as pleasant and were motivated to perform well.

Although entertaining, the mVSWMT was experienced as difficult. In only 37% of the mVSWMT assessments the icon location was remembered correctly, with a range from 7% to 74% between participants. Differences may reflect the use of strategies, such as thinking of an ‘icon-story’ or trying to group the icons per row. In other momentary visuospatial working memory studies, participants identified the correct location(s) in 90% of their responses (Schuster et al., 2015, Schuster et al., 2016; Sliwinski et al., 2018). In these studies, neutral circles in a grid were presented while meaningful icons were used in the mVSWMT (see Fig. 2 b). Remembering the location combined with the meaning of the icon requires a higher cognitive demand. The choice for the here-used icons was made to test a greater working memory capacity (Brady et al., 2016; Chen et al., 2017). Furthermore, the meaningful icons were expected to motivate the participant, which was confirmed by the positive feedback (see Section 3.2.2). Participants responded randomly on the interference items to perform better on the mVSWMT. Prospectively a different interference (Sliwinski et al., 2018) should be considered. For example, a fixed timer could be applied to standardize the interference between encoding and recall.

In contrast, participants experienced the mDSST as easy, which resulted in a ceiling effect for accuracy (97% correct). This high accuracy is in line with another digital processing speed task named the Colour-Shape Test, where participants answered correctly in 97% of the attempts (Brouillette et al., 2013). In the paper-pencil DSST version, participants also make little mistakes reflected by a high accuracy, while the speed outcome proves more sensitive to cognitive variations (Wechsler, 2008). The mDSST speed performance varied between and within subjects, which indicates this outcome to be suitable in detecting momentary cognitive fluctuations.

These feasibility results confirm that both tasks are appropriate, but need fine-tuning. For instance, the font size could be increased or the screen rotated to further improve the mDSST. Nevertheless, some limitations need to be acknowledged. In the mVSWMT analyses, the position of the icons could not be taken into account due to technical limitations. Descriptive background analysis revealed that a slight primacy and recency effect appeared, as participants remembered the first or last icons slightly more often. The discrepancy between identified location and actual location may be an interesting aspect of a momentary working memory task in the future. Additionally, the sample was healthy and highly educated resulting in limited generalizability of the feasibility results. Next steps may include testing the adjusted mDSST and mVSWMT in more diverse healthy and clinical populations.

Validity of momentary cognition tasks through contextualization

Initial evidence for the validity of the momentary cognition tasks was provided by relating cognitive performance with intrapersonal factors (e.g., mood, fatigue, stress, sleep-related outcomes) and contextual factors (e.g., being in company of others) to evaluate and understand momentary cognitive performance. Surprisingly, mood and fatigue had no effect on cognitive performance. One explanation could be that the participants were healthy, well rested, and overall in a positive mood (see Section 3.3). Timmers et al. (2014) also found no indication for an effect of fatigue on cognition (i.e., short-term memory) in healthy young adults. Previous findings on the relation between mood and cognition in daily life are inconclusive. While one study found no association between changes in mood and cognitive functioning (von Stumm, 2018), another study reported that higher positive mood resulted in less interference during an emotional Stroop task (Waters and Li, 2008). Stronger associations may appear in clinical populations (Hung et al., 2016; Schulze et al., 2018).

Individual performance on the mVSWMT and mDSST was diminished when distracted and in social company. Logically and confirmed by experiments, distraction has a negative influence on cognitive performance (Lustig et al., 2006). The negative influence of being in company was also previously reported: Von Stumm (2018) argues that being alone may help to focus one's attention and thus improve cognitive performance. In contrast, it cannot be assumed that social context truly lowers a person's cognitive ability. This result may rather be related to variations in situational demands (von Stumm, 2018). In the present study, participants were in company in 71% of the time. Hence, future studies may take the potentially mediating factors of company and distraction into account when analyzing momentary cognitive fluctuations.

Age sensitivity was found for both tasks with associations between age and visuospatial working memory and processing speed in everyday life. As expected (Zimprich and Kurtz, 2013), younger adults performed better. It is important to disentangle the cognitive decline in performance from an assessment bias. Compared to younger adults, older adults tend to experience technologies as less easy to use (Hauk et al., 2018). Additionally, older adults may have impaired hearing or vision, potentially affecting the usability and thus outcomes of technology-based assessments. Previous studies, however, confirmed the feasibility, reliability, and validity of digital assessments in the elderly (Allard et al., 2014; Brouillette et al., 2013; Schweitzer et al., 2017). All participants became better over time, but the learning curve was steeper for younger adults than for older adults. Learning effects were reported in previous mobile cognition tasks and may not affect sensitivity negatively (Schweitzer et al., 2017). Descriptive background analysis revealed that learning stabilized after 10 to 15 beeps and a steady state is reached within the first days of the study.

The non-significant correlation between the two momentary tasks suggests that different cognitive domains are measured, namely processing speed (mDSST) and visuospatial working memory (mVSWMT). This finding is in line with previous research that also found no correlation between momentary working memory and processing speed, possibly due to the unreliability of one task (von Stumm, 2018). Strategy use was different between the two momentary tasks in this study. The lack of correlation could be due to a different approach in both tasks, hindering a comparison of cognitive performance per se. It is expected that the current tasks will correlate after adjustment are made and when tested in a clinical sample with cognitive complaints. If no correlation shows, it might be that strategy use is moderating cognitive performance.

The present validation study did not focus on correlations with traditional neuropsychological tests. However, an in-house (unpublished) trial with 50 healthy participants showed that outcomes of a two-minute mDSST and paper-pencil DSST correlated (partial r = 0.50, p < .001). Participants reported that the digital version was slightly more difficult, as learning the digit-symbol combinations was challenging and the next number could not be anticipated. This preliminary finding provides initial evidence that the 30-second mDSST measures processing speed, but needs extension to confirm construct validity.

Overall, these validity findings can be seen as a proof of concept for the contextualization of momentary cognitive performance. Future research is needed to disentangle the complex interaction between mood, context, and cognition further. In addition, limitations include not taking the education level and the relation with traditional neuropsychological tests into account as part of the validation. Furthermore, the developed tasks are still artificial (Bielak et al., 2017), in the sense that individuals normally do not perform these tasks, but actually search for their keys or process information to plan their days.

Next steps are the final adjustments of the app and testing both tasks in populations with different cognitive profiles. Clinical populations may include people with schizophrenia, major depression, or brain damage. Furthermore, situations that influence the cognitive performance such as tiredness, alcohol use, or medication intake may provide relevant insight into the validity of the tasks. Cognitive assessments in everyday life, in combination with momentary sampling of mood and context, may prospectively give individuals more insight into their functioning and thus support self-management and planning. This study is an important step in the anticipated personalization of holistic mobile health (van Os et al., 2017).

Suggestions for future development and use of momentary cognition tasks

Reflecting on the overall development and use of the mDSST and mVSWMT, a number of lessons were learned and can be implemented in future studies (see Table 4). Most momentary cognition tasks are inspired by traditional neuropsychological tests. For example, in addition to the DSST used as reference in the present study or by Suffoletto et al. (2017), other studies used momentary processing speed tasks based on the Trail B test or the Stroop task (Hung et al., 2016; Lee et al., 2018). Furthermore, the laboratory n-back task assessing working memory capacity has been adapted to fit into a momentary approach (Frings et al., 2008; Keenan et al., 2014; Kennedy et al., 2011; Veasey et al., 2015). A direct translation can be problematic due to the variability of everyday life that needs repeated assessments in complex environments. Smartphone assessments can offer new possibilities for task development and use. Furthermore, input from a multi-disciplinary team involving neuropsychological healthcare professionals and ESM experts should guide the development. Other stakeholders, including clinicians and patients, should be consulted during development and evaluation. The participant's self-report and observations of the technology use (Bartels et al., 2019) can provide insights into their perspective and experience when measuring momentary cognition within an ESM paradigm.

Table 4

Suggestions for future task development and use.

Suggestions for momentary cognition tasks
Task development	Task use
• Involve a multi-disciplinary team • Orientate concepts on traditional neuropsychological tests • Balance enjoyment/gamification with context information (experiences and physical context) • Ideal outcomes need to show clinically relevant within- and between-subject variance, be age-sensitive, and show no ceiling-effect • Use comparison data to determine between-subject variance; within-subject data serves as its own control	• Tailor beep frequency and sampling duration to the research/clinical question • Balance length and number of tasks, and additional momentary items • Limit assessment time (e.g., 2 min) • Adjust difficulty levels to individual abilities to prevent frustration and maintain motivation • Consider momentary context during interpretation (e.g. distraction) • Consider learning-effects (particularly in early trials)

Gamification is a strategy to increase motivation. An example of gamification to measure cognition is the Sea Hero Quest smartphone app (www.seaheroquest.com). Participants orient themselves in a virtual sea world and get rewards when performing well. This quest may be a valid method to assess navigation skills in a fun way (Coutrot et al., 2018), however, it may be less suited for clinical practice were repeated assessments of cognitive performance are interwoven with assessments of mood and context. It is important to strike a balance between a thoughtful completion of the ESM items and the competitiveness and enjoyment of a cognitive performance task.

Testing the cognition task in a healthy sample is a useful way to test feasibility and validity (Schuster et al., 2015; Timmers et al., 2014). Adjustments can be made before introducing the task to a more vulnerable clinical population. A benefit of ESM is that individuals can be their own controls and performance can be compared within one dataset (Moore et al., 2017; Verhagen et al., 2016).

Another aspect to consider is the beep frequency. While a high intensity may reveal more fluctuations, beep length and time investment need to be considered. The strength of a good ESM questionnaire lies in the intuitiveness of assessments and it is very important that users are able to complete the questionnaires without over thinking the answers and with minimal interference to their usual routine (Verhagen et al., 2016)). Adding a cognition task should not change the adherence to good ESM practice. When assessments are made repeatedly and in the flow of daily life, the beep length should not exceed a couple of minutes to prevent interference. Potentially, tasks can alternate at random across assessments to minimize fatigue effects. In general, participant's experience should be explored and if necessary, guide task adjustments when feelings of over- or under stimulation appear. Task difficulty may be tailored to the individual's ability. Working memory tasks with varying levels of difficulty have been tested in other ESM tools (Dirk and Schmiedek, 2016; Suffoletto et al., 2017). Ideally, momentary assessment promotes a flow in everyday life. Tasks should remain challenging when users reach a level of experienced achievement that relates to the individual's overall cognitive ability (e.g., being correct in 70% of the cases). The right difficulty level can prevent a loss of motivation and simultaneously preclude frustration or even resentment (Csikszentmihalyi, 1997; Engeser and Rheinberg, 2008). Prospectively, studies may consider automatically adjusting the level based on an individual's performance.

When implementing momentary cognition tasks, the sampling duration needs to be tailored to the research or clinical question (Verhagen et al., 2016). Examples include single case assessments running over months or years, where continued experience sampling can illustrate useful insight into the course of a disease (Lenssen and Verhagen, 2019; Wichers et al., 2016). In the assessment process at memory clinics, weeklong momentary cognition tasks can supplement traditional neuropsychological test batteries and provide information on cognitive fluctuations in everyday life (Allard et al., 2014). Side effects of new medications may furthermore be detected when using momentary cognition tasks parallel to the dose adjustments of drugs (Frings et al., 2008). Finally, in rehabilitation centres, e.g., after brain damage, momentary cognition tasks may determine the effectiveness of the treatment when applied before and after a program. Insight into momentary cognitive fluctuations in context can be used to provide individuals with feedback and guidance to deal with cognitive complaints in daily life.

Conclusions

Momentary cognition tasks aim to depict fluctuation of cognitive performance in everyday life and hold promise for future research and clinical use. Prospectively, the task application needs to be extended, for example into different cognitive domains or patient populations. Furthermore, the interaction with other intrapersonal factors requires further disentanglement. Next steps can be guided by the suggestions resulting from this study such as involving a multi-disciplinary team, tailoring the set-up to the individual, and balancing the level of enjoyment and seriousness.

Table A.1

Experience sampling protocol: Beep Questionnaire.

	Item	7-point Likert scale or categorical options
1	I feel cheerful	1 = not at all 4 = moderate 7 = very much
2	I feel energetic	1 = not at all 4 = moderate 7 = very much
3	I feel insecure	1 = not at all 4 = moderate 7 = very much
4	I feel relaxed	1 = not at all 4 = moderate 7 = very much
5	I feel down	1 = not at all 4 = moderate 7 = very much
6	I feel irritated	1 = not at all 4 = moderate 7 = very much
7	I feel satisfied	1 = not at all 4 = moderate 7 = very much
8	I feel lonely	1 = not at all 4 = moderate 7 = very much
9	I feel enthusiastic	1 = not at all 4 = moderate 7 = very much
10	I feel anxious	1 = not at all 4 = moderate 7 = very much
11	I feel guilty	1 = not at all 4 = moderate 7 = very much
12	I'm worrying about things	1 = not at all 4 = moderate 7 = very much
13	mVSWMT instruction screen
	mVSWMT part 1: encoding
14	I think I remembered it all	1 = not at all 4 = moderate 7 = very much
15	I generally feel well at the moment	1 = not at all 4 = moderate 7 = very much
16	mVSWMT instruction screen
	mVSWMT part 2: recall
17	What am I doing (right before the beep)	work, school/housekeeping/self-care/relaxing/sport, movement/eating, drinking/traveling, on the road/having a conversation/something else/nothing
18	I can do this well	1 = not at all 4 = moderate 7 = very much
19	This is difficult for me	1 = not at all 4 = moderate 7 = very much
20	I would rather be doing something else	1 = not at all 4 = moderate 7 = very much
21	I am focused	1 = not at all 4 = moderate 7 = very much
22	Where am I (just before the beep)	at home/at someone else's home/work, school/public space/on the road/somewhere else
23	Who am I with (just before the beep)	partner/family/housemates/friends/colleagues/acquaintances/strangers, others/nobody
24a	Company: I like this company	1 = not at all 4 = moderate 7 = very much
25a	Company: I would rather be alone	1 = not at all 4 = moderate 7 = very much
24b	Alone: I like being alone	1 = not at all 4 = moderate 7 = very much
25b	Alone: I would rather be in company	1 = not at all 4 = moderate 7 = very much
26	I don't feel well	1 = not at all 4 = moderate 7 = very much
27	I am tired	1 = not at all 4 = moderate 7 = very much
28	Since the last beep I have used	alcohol/medication/coffee, caffeine/smoking, nicotine/cannabis/other drugs/nothing
29	mDSST instruction screen
	mDSST	30-s timeframe
30	I got distracted during the task	1 = not at all 4 = moderate 7 = very much
31	This beep disturbed me	1 = not at all 4 = moderate 7 = very much
31	Thanks!

Table A.2

Experience sampling protocol: Morning questionnaire.

	Item	7-point Likert scale or categorical options
1	I generally felt well today	1 = not at all 4 = moderate 7 = very much
2	I generally felt tired today	1 = not at all 4 = moderate 7 = very much
3	I generally felt tense today	1 = not at all 4 = moderate 7 = very much
4	I generally worried a lot today	1 = not at all 4 = moderate 7 = very much
5	I generally could concentrate well today	1 = not at all 4 = moderate 7 = very much
6	I generally felt forgetful today	1 = not at all 4 = moderate 7 = very much
7	Goodnight!	1 = not at all 4 = moderate 7 = very much

Table A.3

Experience sampling protocol: Evening questionnaire.

	Item	7-point Likert scale or categorical options
1	How long did it take before I fell asleep last night?	0–5 min/5–15 min/15–30 min/30–45 min/45 min–1 h/1–2 h/2–4 h/>4 h
2	How often did I wake up last night?	0/1/2/3/4/5/>5
3	How long did I lie awake this morning before getting up?	0–5 min/5–15 min/15–30 min/30–45 min/45 min–1 h/1–2 h/2–4 h/>4 h
4	I slept well	1 = not at all 4 = moderate 7 = very much
5	I feel well rested	1 = not at all 4 = moderate 7 = very much
6	I am looking forward to this day	1 = not at all 4 = moderate 7 = very much
7	Thanks!	1 = not at all 4 = moderate 7 = very much

Table B

Individual multilevel regression analyses of mood, context, and sleep quality on the mVSWMT outcome correct/incorrect.

	Correct/incorrect
	B	SE	p	95% CI
Positive affect	−0.03	0.02	0.04⁎	−0.06,	−0.001
Negative affect	−0.0003	0.03	0.99	−0.05,	0.05
Worry	0.002	0.01	0.89	−0.02,	0.02
Fatigue	−0.01	0.01	0.54	−0.02,	0.01
Focus	0.04	0.01	<0.001⁎⁎⁎	0.02,	0.06
Activity-related stress	−0.01	0.01	0.49	−0.03,	0.02
Company$	−0.10	0.03	<0.001⁎⁎⁎	−0.16,	−0.05
Location$	−0.02	0.03	0.38	−0.07,	0.03
Distraction	−0.03	0.01	0.001⁎⁎	−0.04,	−0.01
Time until sleep	0.01	0.01	0.23	−0.01,	0.04
Number of wake-ups	−0.005	0.01	0.67	−0.03,	0.02
Slept well	0.01	0.01	0.47	−0.01,	0.03
Well rested	0.005	0.01	0.65	−0.02,	0.03
Age2	−0.00005	0.00002	0.002⁎	−0.00009,	−0.00002
Gender	−0.04	0.05	0.47	−0.14,	0.06

Note. Company = dummy variable of being alone versus with others. Location = dummy variable of being at home versus somewhere else. Age2 = quadratic function of age. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

Table C

Individual multilevel regression analyses of mood, context and sleep quality on the mDSST outcomes speed and accuracy.

	Speed					Accuracy
	B	SE	p	95% CI		B	SE	p	95% CI
PA	−0.01	0.05	0.90	−0.11,	0.10	0.45	0.21	0.03⁎	0.03,	0.87
NA	−0.05	0.10	0.61	−0.24,	0.14	−0.39	0.34	0.25	−1.04,	0.27
Worry	0.09	0.04	0.02⁎	0.01,	0.17	−0.20	0.15	0.17	−0.50,	0.09
Fatigue	−0.02	0.03	0.46	−0.09,	0.04	−0.34	0.12	0.007⁎⁎	−0.58,	−0.09
Focus	0.02	0.03	0.50	−0.04,	0.08	0.36	0.14	0.008⁎⁎	0.09,	0.63
Act-stress$	0.05	0.04	0.18	−0.03,	0.13	−0.14	0.18	0.44	−0.48,	0.21
Company$	−0.29	0.09	0.001⁎⁎	−0.46,	−0.12	−0.003	0.39	0.99	−0.77,	0.76
Location$	−0.09	0.08	0.26	−0.25,	0.07	0.34	0.36	0.34	−0.36,	1.05
Distraction	−0.15	0.02	<0.001⁎⁎⁎	−0.20,	−0.10	−0.54	0.41	<0.001⁎⁎⁎	−0.74,	−0.33
Time-sleep$	−0.04	0.04	0.32	−0.11,	0.04	−0.07	0.17	0.67	−0.40,	0.25
Wake-ups$	−0.01	0.04	0.82	−0.08,	0.07	−0.08	0.16	0.63	−0.40,	0.24
Slept well	−0.01	0.04	0.84	−0.08,	0.07	−0.16	0.17	0.35	−0.49,	0.17
Well rested	0.01	0.04	0.86	−0.07,	0.08	0.08	0.16	0.62	−0.24,	0.40
Age2	−0.001	0.0002	<0.001⁎⁎⁎	−0.002,	−0.001	0.0002	0.0002	0.47	−0.0003,	0.0006
Gender	0.47	0.85	0.58	−1.19,	2.14	0.71	0.58	0.23	−0.44,	1.85

Note. PA = positive affect. NA = negative affect. Act-stress = activity-related stress. Company = dummy variable of being alone versus with others. Location = dummy variable of being at home versus somewhere else. Time-sleep = time until sleep. Wake-ups = number of wake-ups at night. Age2 = quadratic function of age. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

Table D.1

Subgroup analyses on age for the mVSWMT outcome correct/incorrect.

	<45 years of age					45 years or older
	B	SE	p	95% CI		B	SE	p	95% CI
Model			<0.001⁎⁎⁎					<0.001⁎⁎⁎
PA	−0.08	0.05	0.08	−0.18,	0.009	−0.01	0.06	0.88	−0.14,	0.12
NA	−0.10	0.11	0.36	−0.32,	0.12	0.11	0.22	0.62	−0.32,	0.54
PA × NA	0.02	0.03	0.56	−0.04,	0.07	−0.03	0.04	0.55	−0.11,	0.06
Focus	0.05	0.01	<0.001⁎⁎⁎	0.02,	0.08	0.04	0.02	0.01⁎⁎	0.008,	0.07
Company$	−0.05	0.04	0.21	−0.13,	0.03	−0.13	0.04	0.001⁎⁎	−0.21,	−0.05
Distraction	−0.02	0.01	0.11	−0.04,	0.004	−0.01	0.01	0.22	−0.04,	0.009
Age2	−0.00005	0.00009	0.55	−0.0002,	0.0001	−0.00007	0.00004	0.047⁎	−0.0001,	−1.13e-06

Note. PA = positive affect. NA = negative affect. PA × NA = interaction between positive and negative affect. Company = dummy variable of being alone versus with others. Age2 = quadratic function of age. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

Table D.2

Subgroup analyses on age for the mDSST speed outcome.

	<45 years of age					45 years or older
	B	SE	p	95% CI		B	SE	p	95% CI
Model			<0.001⁎⁎⁎					<0.001⁎⁎⁎
PA	−0.05	0.17	0.77	−0.37,	0.28	0.33	0.16	0.04⁎	0.02,	0.63
NA	−0.27	0.39	0.49	−1.04,	0.49	1.01	0.53	0.06	−0.02,	2.05
PA × NA	0.03	0.09	0.74	−0.14,	0.20	−0.17	0.11	0.10	−0.38,	0.04
Worry	0.10	0.06	0.10	−0.02,	0.21	−0.01	0.05	0.78	−0.12,	0.09
Company$	−0.17	0.13	0.20	−0.43,	0.09	−0.22	0.09	0.02⁎	−0.40,	−0.03
Distraction	−0.19	0.04	<0.001⁎⁎⁎	−0.26,	−0.13	−0.08	0.03	0.004⁎⁎	−0.13,	−0.02
Age2	−0.0007	0.001	0.57	−0.003,	0.002	−0.001	0.0003	<0.001⁎⁎⁎	−0.002,	−0.0008
Time$	0.30	0.15	0.047⁎	0.004,	0.59	0.009	0.11	0.94	−0.21,	0.23
Study day$	0.24	0.28	0.40	−0.31,	0.78	0.47	0.21	0.02	0.07,	0.88

Note. PA = positive affect, NA = negative affect. PA × NA = interaction between positive and negative affect. Location = dummy variable of being at home versus somewhere else. Company = dummy variable of being alone versus with others. Age2 = quadratic function of age, Time = log-transformed session counter score. Study day = log-transformed day of study score. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.

Table D.3

Subgroup analyses on age for the mDSST accuracy outcome.

	<45 years of age					45 years or older
	B	SE	p	95% CI		B	SE	p	95% CI
Model			0.004⁎⁎					0.003⁎⁎
PA	−0.86	0.68	0.21	−0.2.18,	0.47	0.88	0.91	0.34	−0.91,	2.67
NA	−1.92	1.57	0.22	−5.004,	1.15	1.11	3.11	0.72	−4.98,	7.20
PA × NA	0.37	0.36	0.30	−0.34,	1.08	−0.15	0.61	0.80	−1.36,	1.05
Fatigue	−0.34	0.20	0.10⁎	−0.74,	0.06	−0.08	0.20	0.70	−0.47,	0.31
Focus	0.25	0.21	0.23	−0.15,	0.65	−0.02	0.22	0.93	−0.45,	0.41
Distraction	−0.44	0.15	0.003⁎⁎	−0.73,	−0.15	−0.57	0.16	<0.001⁎⁎⁎	−0.88,	−0.27

Note. PA = positive affect. NA = negative affect. PA × NA = interaction between positive and negative affect. CI = Confidence Interval.

p < .05.

p < .01.

p < .001.