Serious Games as Potential Therapies: A Validation Study of a Neurofeedback Game.

Serious (biofeedback) games offer promising ways to supplement or replace more expensive face-to-face interventions in health care. However, studies on the validity and effectiveness of EEG-based serious games remain scarce. In the current study, we investigated whether the conditions of the neurofeedback game "Daydream" indeed trained the brain activity as mentioned in the game manual. EEG activity was assessed in 14 healthy male volunteers while playing the 2 conditions of the game. The participants completed a training of 5 sessions. EEG frequency analyses were performed to verify the claims of the manual. We found significant differences in α- to β-ratio between the 2 conditions although only in the amplitude data, not in the power data. Within the conditions, mean α-amplitude only differed significantly from the β-amplitude in the concentration condition. Our analyses showed that neither α nor β brain activity differed significantly between game levels (higher level requiring increased brain activity) in either of the two conditions. In conclusion, we found only marginal evidence for the proposed claims stated in the manual of the game. Our research emphasizes that it is crucial to validate the claims that serious games make, especially before implementing them in the clinic or as therapeutic devices.

Introduction

Patients with psychotic disorders, besides being challenged by the primary symptoms of their specific disorder, often suffer from comorbid problems such as anxiety and attention deficits. These problems usually persist or even worsen after psychopharmacological treatment. Cognitive behavioral therapy is sometimes effective but rather time-consuming and difficult, especially for patients with limited verbal or cognitive skills.[1] As a result, patients with psychotic disorders frequently are found to use unhealthy ways to achieve relaxation, for example, by means of self-medication or substance abuse.[2,3] Therefore, there is an urgent need for alternative and/or additional treatments. Therapeutic tools designed with the use of innovative technology offer promising ways to supplement or perhaps even replace more expensive face-to-face interventions. These computerized interventions are easy to use, accessible whenever the individual has time and save health care costs.[4,5] Given the popularity of computer games in younger generations, games are increasingly applied in health care. These so-called applied or serious games do not have entertainment as primary purpose but are designed for education, training, or health improvement in the “real” world.[6]

Serious games are found to be appealing to psychiatric patients, especially the younger ones,[7] which may increase the patient’s motivation for treatment that is so frequently lacking in other forms of therapies. This is particularly relevant for patients who lack insight into their mental condition. In addition, serious games may result in more sustainable effects by the intrinsic motivation to play longer and repeatedly.[8,9] Furthermore, these games may enlarge self-management as patients can use them at home or in other settings (e.g., in public transport, at school, or at work) using mobile technology.

Serious games have been found effective training tools in mental health care, for example, for improving cognitive abilities in older adults,[10] improving cognitive functioning in patients with alcohol abuse,[11] enhancing emotional regulation in individuals with eating disorders,[12] and improving executive functioning in children with attention deficit hyperactivity disorder (ADHD).[13] These findings exemplify the potential effectiveness of serious games for mental health–related symptoms.

One line of serious games uses electroencephalography (EEG) technology. EEG is a noninvasive methodology to record the electrical activity of an individual’s brain. This neural activity can be modulated by neurofeedback, a technique based on learning theory principles.[14] With neurofeedback on brain activity, individuals learn to self-regulate their neural activity following operant conditioning. Real-time feedback (mostly visual or auditory) is given based on an individual’s current brain activity and the desired activity is rewarded while other activity is punished. This way the individual learns with training how to elicit the targeted brain activity. They are informed on which cognitive state is targeted, but they do not get instructions on how to reach the desired state.

Neurofeedback is clinically applied as an alternative or supplementary treatment for several disorders, for example, epilepsy[15] and also for some psychiatric disorders such as ADHD.[16] There is also some evidence of efficacy of personalized neurofeedback for individuals with psychosis,[17] but literature on this topic is scarce.[18] In contrast to clinically applied neurofeedback, neurofeedback games are designed for entertainment and therefore the real-time feedback is given in the form of a game. Since wireless headsets with improved comfort and portability became more affordable and accessible for consumers, the use of EEG-based technology in serious games is booming.[19] However, studies on the validity and effectiveness of these EEG-based serious games remain inadequate.[20,21] In addition, there is often a lack of understanding the design principles among the individuals and institutions that develop or apply these games.[22] These issues are worsened by the fact that (detailed) information on individual games is often difficult to find or not publicly available.[23] Also, entertainment game developers do not always have the necessary expert knowledge about brain waves and functions that is required for developing EEG-based serious games.[24] Thus, before implementing serious games as therapeutic devices, it is crucial to investigate the validity and effectiveness of these games with independent well-controlled empirical studies.

The current study therefore investigated whether the training conditions of the neurofeedback game “Daydream” were indeed training the brain activity as specified in the manual of the game, that is, in the relaxation condition α (alpha) waves should be trained (high alpha to beta ratio) while the concentration condition should train β (beta) waves (low alpha to beta ratio). As such, a controlled EEG study was performed with healthy volunteers, in which EEG activity was assessed while playing the 2 different conditions of the Daydream game. The study consisted of a 5-day training series based on the assumption that practice would result in better performance in the game. Our hypothesis was that the game would indeed train the brain activities as specified in the manual.

Methods

Participants

A total of 14 healthy male participants, aged 20 to 44 years, were included in the study. The participants were recruited through websites such as proefbunny.nl and by flyers spread in the environment. Exclusion criteria of this study were the following: age >45 years; current use of any medication or investigational medication within 30 days prior to the start of this study; history of neurologic or psychiatric illness in the participant or in first-degree relatives, evaluated with the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria; history of alcohol and drug abuse; participants being unable to understand the study outline; and/or provide written informed consent. The Mini International Neuropsychiatric Interview 5.0.0 (M.I.N.I. 5.0.0)[25] was administered to confirm the absence of any neuropsychiatric disorder. In addition, to avoid unnecessary acute or withdrawal effects of caffeine or smoking, participants were asked to refrain from smoking and drinking caffeine containing beverages 1 hour, respectively 2 hours before testing. Finally, the participants received a small financial incentive for their efforts and time spent in this study.

Neurofeedback Game and Task Conditions

This study examined the neurofeedback game “Daydream”, which is designed to stimulate players to relax or concentrate by producing respectively more alpha or beta brain activity. The game consists of 2 components: the software installed on a computer, developed by GainPlay studio (Utrecht, the Netherlands) and the MindWave EEG headset from NeuroSky (San Jose, CA, USA). The lightweight, wireless headset has one dry sensor (electrode) that is placed on the forehead (approximate 0.5-2 cm below FPz depending on a player’s head size) and an ear clip that is attached to the player’s left side earlobe. The headset sensor reads brainwaves and sends this information as feedback to the computer. The software of the game uses this feedback to adjust visual and auditory output, based on the level of a previously chosen specific brain frequency that is extracted out of the feedback signals of the headset; The version of the Daydream game that we tested offered 2 training conditions: (1) relaxation, corresponding to the level of alpha activity (8-12 Hz) produced in the brain and (2) concentration, corresponding to the level of beta activity (13-20 Hz). The player was instructed to monitor the computer screen and, depending on the condition, to either concentrate or relax. The player was further instructed to induce a state of mind that would result in a change in the landscape presented on the screen. At start, a winter landscape is presented and if the player is doing well, that is, the player is getting more relaxed or concentrated, the player moves to the next level in the game (winter turns into spring). When the player grows deeper in this particular state of mind, the spring landscape turns into a summer landscape (highest level). However, when the desired mental state of the player regresses, the player moves back to a lower level (summer or spring turns into autumn and finally winter again). The goal of the game is to stay in the summer landscape as long as possible, which reflects the player’s desired state of mind.

Experimental Design

The study had a longitudinal within-subject design, consisting of 5 assessments within a maximum period of 2 weeks. The underlying assumption of the training series was that participants gain more control over their brain activity (α- to β-ratio), that is, mental state (concentration/relaxation) with practice and subsequently reach the highest level (summer) quicker and for a longer period of time. Each assessment consisted of playing the above mentioned 2 conditions of the game (relaxation and concentration) for 20 minutes (10 minutes per condition). The order of the conditions was randomized yet balanced, that is, the order was randomized across the participants yet was identical for each of the 5 separate test sessions of an individual.

While playing the game, the participant was not only wearing the headset to control the neurofeedback game but also a traditional EEG cap. EEG was assessed continuously. Recording started from the moment that the participant started a condition and lasted for the 10 minutes corresponding to the time that the participant played that condition. This procedure was followed for both conditions. The assessments took place in a quiet psychophysiological laboratory at the psychiatry department of the University Medical Center Utrecht (the Netherlands).

EEG Signal Recording and Processing

A BioSemi system with 64 active-2 electrodes, placed according to an extended 10-20 system, was used to record the EEG signal. The EEG signal was digitized with a frequency of 2048 Hz and analyzed and processed using Brain Electrical Source Analysis (BESA) software (version 5.2.4, MEGIS Software GmbH, Gräfelfing, Germany). Only data from electrode FPz were processed, since this electrode was nearest to the headset sensor. First, the data was downsampled to 250 Hz to allow for easier file handling. Then, at least 1 but if possible 2 epochs of 2-second artifact-free signals each were selected per season every time the player reached that season (per condition an average of 11 epochs was analyzed). This was done in such a way that these epochs were not too close (>3 seconds) to a switch in season. Subsequently, the level (maximum amplitude as well power) of 5 brain frequencies (delta [0.1-3 Hz], theta [4-7 Hz], alpha [8-12 Hz], beta [13-25 Hz], and gamma [25-50 Hz]) were extracted out of these epochs by BESA’s fast Fourier transform (FFT) module. Last, average amplitude and power were calculated per player, per season.

Statistical Analyses

All statistical analyses were performed using Statistical Package for the Social Sciences (SPSS; IBM Corp, Armonk, NY) for Windows, version 20. P values <.05 were considered statistically significant. Only EEG data from the last training session was used in the analyses because in this session the participants would be most experienced in playing the game, thus providing the most optimal circumstances for the purpose of this project, which was validation of the game. Given that only 5 participants reached the highest level (summer), even after all (except one) subjects completed the training series, mean amplitude/power of the desired brain frequency of the specific training condition (relaxation/concentration) in the first level (winter) was compared with that of the next highest level, that is, the spring season. This comparison was explored with a repeated-measures analysis of variance, with within factors “condition” (relax vs concentration), “season” (winter vs spring) and “frequency” (alpha vs beta). Age and educational level were tested as covariates, but since they did not reach significance they were removed from the analyses.

Results

A total of 14 healthy male participants, aged 20 to 44 years (M = 25.4 years; SD = 5.9) with a preuniversity educational level, were included in the study. All participants met the inclusion criteria and completed a minimum of 4 out of 5 training sessions. There was only 1 participant who completed 4 instead of 5 training sessions due to technical difficulties, all others completed all training sessions.

Amplitudes

Repeated-measures analyses revealed no significant main effects of condition [F(1, 12) = 0.20, P = .66, ηp2 = .016], season [F(1, 12) = 0.39, P = .55, ηp2 = .031] and frequency [F(1, 12) = 0.45, P = .51, ηp2 = .036] on mean (α or β) amplitude. However, a first order interaction was observed between condition and frequency [F(1, 12) = 9.33, P = .01, ηp2 = .438], indicating that the mean α-amplitude was higher in the relaxation (M = 16.42, SD = 2.97) than in the concentration condition (M = 13.25, SD = 1.83), whereas the mean β-amplitude was lower in the relaxation (M = 14.94, SD = 1.32) than in the concentration condition (M = 16.65, SD = 1.49), regardless of season (see Figure 1). Further analyses revealed significantly higher β-amplitude (M = 16.39, SD = 5.27) than α-amplitudes (M = 13.07, SD = 6.36) in the concentration condition [t(13) = −3.87, P < .01, d = 0.57], but no significant differences between α- and β-amplitudes in the relaxation condition [t(13) = 0.64, P = .54, d = 0.038]. Neither an interaction effect between condition and season, season and frequency nor a second-order interaction between condition, season, and frequency was found, all F ≤ 0.50, P ≥ .49, ηp2 ≤ .04 (see also Figure 2 and Table 1).

Figure 1.

Mean α- and β-amplitude per condition at Fpz electrode.

Figure 2.

Mean amplitude per season on an individual level.

Table 1.

Mean Amplitude and Power per Condition, Season, and Frequency.

Condition	Season	Frequency	Mean Amplitude	Mean Power
Relaxation	Winter	Alpha	16.39 ± 3.3	57.73 ± 28.2
		Beta	14.59 ± 1.4	12.54 ± 2.6
	Spring	Alpha	16.44 ± 2.8	54.75 ± 24.1
		Beta	15.29 ± 1.3	13.42 ± 2.4
Concentration	Winter	Alpha	13.22 ± 1.6	27.31 ± 8.1
		Beta	16.36 ± 1.4	18.32 ± 4.9
	Spring	Alpha	13.27 ± 2.2	32.29 ± 14.6
		Beta	16.95 ± 1.8	17.44 ± 3.8

Power

Similar as for amplitude, neither significant main effects of condition [F(1, 12) = 0.90, P = .36, ηp2 = .070] nor season [F(1, 12) = 0.03, P = .86, ηp2 = .003] on mean α- or β-power were found. However, a main effect of frequency was observed, although it only reached trend level of significance [F(1, 12) = 3.49, P = .086, ηp2 = .225], indicating that mean α-power (M = 43.02, SD = 16.51) was higher than the mean β-power (M = 15.43, SD = 2.57) regardless of conditions or seasons. No significant interaction effects were found, all F ≤ 2.75, P ≥ .12, ηp2 ≤ .186 (see also Table 1 and Figure 3).

Figure 3.

Mean power per season on an individual level.

Sample Size Calculation

Based on the results of the frequency band power analyses above we would need 108 participants to reach a statistically significant second order interaction effect between condition, season, and frequency with a statistical power of 0.8; for the amplitude analysis this sample size would need to be much larger still, given that its F value is less than 0.01. On top of this, the average values of α and β for the seasons within the conditions is going into the wrong direction, that is, higher β in the spring than in the winter for the relaxation condition and higher α in the spring than in the winter for the concentration condition.

Discussion

Serious games are increasingly applied in health care because they offer promising ways to supplement or replace more expensive and laborious face-to-face interventions. However, studies on the validity and effectiveness of these games remain scarce. The current study investigated the validity of the neurofeedback game “Daydream”, which claims to increase a player’s α- or β-activity in the brain, depending on the specific condition played. Our hypothesis was that the game would indeed train the brain activities as specified in the game’s manual, that is, increased α-activity in the game’s relaxation condition (high α- to β-ratio), and increased β-activity (low α- to β-ratio) in the concentration condition.

Conforming to the manual, our analyses showed that the α- to β-ratio indeed differed between conditions and even in the right (above mentioned) directions. However, in contrast with the claims stated in the manual this effect was regardless of game level (depicted by seasons). Furthermore, we found this α- to β-ratio only in the amplitude data and not in the power data, while additionally α- and β-amplitudes only differed significantly from each other in the concentration condition, and not in the relaxation condition. Analyses of the power data only showed a trend level of significance toward higher α- than β-power regardless of playing condition or season.

From a broad perspective, we found only marginal evidence that playing the game induced the promised changes in α- and β-brain activities. The interaction that was found was regardless of the levels (seasons) the players were in; meaning that within the playing condition there was no difference between winter and spring in α- to β-ratio, whereas the manual states that the game switches only to higher levels (warmer season) when the anticipated ratio is approached. In addition, within the playing conditions we found only significantly higher β-than α-amplitudes (low α- to β-ratio) in the concentration condition, whereas we found no differences between α- and β-amplitudes in the relaxation condition. A likely explanation for this interaction effect is that the subjects were of course aware which condition they were playing. Therefore, they were trying to relax while playing the relaxation condition and trying to concentrate during the concentration condition. It is likely that only this instruction, and not the game itself, was enough to activate the desired brain waves. In other words, given the instructions of the game: try to relax (or: focus on relaxation) in the relaxation condition and try to concentrate (or: focus on concentration) in the concentration condition, likely has caused a stronger shift toward concentration (β-waves) than toward relaxation (α-waves). This makes sense, since logically, focusing is strongly associated with concentrating.

As stated, the above discussed interaction between condition and frequency was regardless of season. According to the manual the seasons are only supposed to change when there is a shift of α- or β-frequency into the desired direction, that is, an increase of α-frequency from winter to spring in the relaxation condition and an increase in β-frequency from winter to spring in the concentration condition. The absence of this association in our data does not support the manual’s claim that subjects are more relaxed or concentrated in higher levels of the conditions. A possible explanation for this finding might have been that we had to compare winter with spring instead of summer, due to a lack of players reaching summer, in spite of the fact that all except one had 4 previous training-sessions; maybe the difference between winter and spring was simply too small to detect. However, on an individual level the pattern appeared rather random: some subjects had higher amplitudes in winter compared to spring, and vice versa, resulting in near negligible differences in average amplitudes or power between the seasons.

Last, our analyses showed that the mean α-power was overall higher than the mean β-power, regardless of playing conditions and seasons, although this only reached trend level of significance. Given that α-activity is associated with relaxation,[26] this finding indicates that the subjects were overall feeling more relaxed than concentrated (or nervous) when playing the game, independent of condition or level (season). This is not very surprising, given that players of the game sit still in front of a computer monitor, and do not actively engage in more exciting gaming activities such as responding to or keeping track of exhilarating fast changing sceneries as is common in, for example, war or adventure games.

From a subjective perspective, some of the participants reported feeling more relaxed after playing the game, however others felt no difference or even felt a little tensed because they were not able to reach the highest level of the game. It is possible that the calm atmosphere created by the game was enough to generate a more relaxed mental state.

To our knowledge, this is one of the few studies that investigated the validity and effectiveness of EEG-based serious games. The strength of our study is that we investigated the validity by means of a controlled study with a validated EEG system. This is the best method to assess the validity of these games because it directly measures brain activity, as opposed to questionnaires or other subjective measures. However, we tested only a small sample of male subjects with a limited age range. It is possible that this sample is not representative for the population that would use this game, although it is a well-known fact that adolescents and young adults are known to spend a significant amount of time playing video or computer games.[27] Yet this would still not explain why the seasons changed on an apparent random basis. Another possible limitation is that our results cannot be automatically generalized to other versions of the Daydream game. This is a well-known difficulty in empirical research in the field of serious gaming: research cannot keep up with the pace of new versions continuously popping up on the market.

In conclusion, we were unable to replicate the effects as stated in the manual of the neurofeedback game “Daydream”, meaning that neither α- nor β-activity in the brain was enhanced by playing this game. From a broader perspective, our findings emphasize the importance of testing the validity and effectiveness of serious games, especially before implementing them in clinical or therapeutic programs. We encourage game developers and empirical researchers to collaborate more closely in order to establish validated serious games on the market.