Evaluation of a scalable online videogame-based biofeedback program to improve emotion regulation: A descriptive study assessing parent perspectives.

Individuals and families increasingly turn to e-mental health apps for education, diagnosis, and treatment of mental health disorders and to promote mental wellness. These apps provide significant increases in convenience from existing services, since they can augment or replace services with on-demand access within the home. This raises important questions about self-selection of interventions. Who uses these applications? How do individuals perceive their own progress within applications? This study is a retrospective data analysis-based evaluation of a commercially available e-mental health program that includes biofeedback video games that help children build emotion regulation skills by demonstrating and prompting children to practice bodily focused emotion regulation techniques. The e-mental health program also provided parent psychoeducation-focused coaching at the time of the evaluation. Data collection instruments used to inform the retrospective study included parent intake surveys, gameplay engagement data, and notes from parent coaching calls. The evaluation revealed families presenting for common symptoms associated with emotion regulation deficits, as opposed to a wellness cohort looking for additional support. Families near-universally activated and engaged with the intervention, willing to carry out an extended "dose" of the e-mental health program in their home. Parents self-reported their perceptions of their children's emotion regulation progress, primarily in terms of children's increased use of emotion regulation skills, improved emotion awareness and communication, calmer demeanor, greater confidence, and improved relationships. More work is needed to understand the corresponding clinical progress from this in-home training, as well as its implications for how emotion regulation skills grow.

Introduction

Why scale an evidence-based digital mental health intervention?

Access to evidence-based mental health care remains a fundamental problem. Estimates based on national surveys in the UK, the US, and Australia have estimated that 25–56% of children and adolescents with mental health disorders access specialist mental health treatments (Green et al., 2005; Merikangas et al., 2011; Lawrence et al., 2015). Digital mental health has shown promise as an evidence-based intervention for mental health disorders (Firth et al., 2017a; Firth et al., 2017b) and as having the potential to scale (Fleming et al., 2018). However, many widely available interventions lack robust evidence of effectiveness (Bry et al., 2018; Wang et al., 2018), and many evidence-based interventions struggle to find scale (Gordon et al., 2020). The aim of scaling an evidence-based intervention is to increase access to mental health care, in a format that can be rapidly deployed while maintaining the gains demonstrated in experimental settings.

E-mental health apps are an important part of the growing health app field, targeting general mental health and specific disorders (Truschel, 2019). A subset of mobile health tools makes use of gamification to support health and well-being through activities focused on symptom management and reduction, and well-being promotion (Griffiths et al., 2017). Research studies have demonstrated the effectiveness of game-based e-mental health apps in reducing symptoms of depression (Merry et al., 2012; Roepke et al., 2015), anxiety (Wols et al., 2018), and post-traumatic stress disorder (James et al., 2015; Butler et al., 2020). But challenges persist. Recent systematic reviews and meta-analyses have found that only a minority of e-mental health apps have shown evidence of effectiveness (e.g., Bry et al., 2018; Donker et al., 2013; Wang et al., 2018). And data indicate that users often do not engage with digital health apps long enough to see improvement (Fleming et al., 2018; Rodde, 2019). Despite these challenges, and criticisms of e-mental health apps relating to the lack of universal industry standards and shortcomings in usability, effectiveness, and data security (Hendrikoff et al., 2019; Torous et al., 2019), health providers and consumers have found using e-mental health apps acceptable, feasible and beneficial (Dragovic et al., 2018; Gindidis et al., 2019).

Why focus on emotion regulation?

A large body of research identifies emotion regulation – the ability to control one's affects, along with related impulses, cognitions, and behaviors – as a precursor to achievement (Diamond et al., 2007; Duckworth and Seligman, 2005). Chronic emotion dysregulation raises children's risks for negative outcomes across relational, social, academic, and other domains (Miller et al., 2004; Oram et al., 2017; Campbell et al., 2000; Brotman et al., 2006). Research studies have shown benefits of a variety of therapeutic interventions aimed to help children and adolescents improve their emotion regulation, including CBT, mindfulness, and self-regulation skill-building (Herman et al., 2011; Schlesier et al., 2019; Reyes et al., 2019). Given the high prevalence of child mental health disorders (Perou et al., 2013) and high rates of overlap between mental illness and emotion dysregulation (Compas et al., 2017), scaling an emotion regulation intervention makes it possible to provide transdiagnostic care to large numbers of patients.

An open-label, large-scale mixed-methods retrospective study

This open-label, mixed-methods, retrospective evaluation study provides descriptive analyses of demographics of users, patterns of engagement, and self-reports of improvement for a scalable online videogame-based biofeedback program that embeds emotion regulation skill-building opportunities into video game play.

The authors conducted this study for the purposes of (1) developing a deeper understanding of practical, scaled, commercial use of this emotion regulation skill-building intervention, and (2) contributing to the scientific literature, given that little data is available describing the need and impact of evidence-based digital interventions at scale. The authors utilized a mixed-methods approach that allows for a richer understanding of motivations of individuals within a dataset larger than those typically seen in digital health research.

Our research questions are: (1) What are the characteristics of children whose families choose to acquire this child-focused e-mental health app that is commercially available for in-home use?, (2) How do families use the app in their homes?, (3) What behaviors or situations do parents interpret as signs that the app is working for their child?, and (4) do parent perceptions of their children's progress change over time?

Methods

Participants

In 2017 and 2018 the app was marketed commercially in the United States through internet advertisements and the app internet site. At the time, 91.8% of Americans had the use of some type of computer, and 85.3% had internet access (Martin, 2021). The target population of app marketing was families seeking help for their children's symptoms of emotion dysregulation. We retrospectively reviewed de-identified data for 3636 players from 1426 families that commercially purchased the app between March 29, 2017 and September 30, 2018. Of these, 1045 players met the following criteria for having their data included in this retrospective data study: (1) a parent completed self-report measures on the app intake questionnaire, (2) a player was created on the app system, (3) play could be identified as belonging to a specific individual player whose parent had completed the app intake questionnaire, and (4) the family was living in North America.

All procedures contributing to this work comply with international ethical standards on human experimentation including the Helsinki Declaration of 1975, as revised in 2008, and were approved by the HML institutional review board (IRB approval #565NLI18).

Study design

This is a retrospective data review of de-identified data from app customers in 2017 and 2018. Due to the retrospective nature of the data review, we did not obtain informed consent from individuals whose data were included in the review; however we did ensure anonymity in the retrospective data set, to uphold ethical research standards. All app users whose data is included in the review purchased the app commercially. At the time of purchase, parents completed an intake questionnaire. After receiving the intervention, individuals' use of the intervention was assessed for a period of up to 16 weeks. Use was ad-lib, meaning that individuals used all components of the intervention at their discretion in a naturalistic manner. Characteristics of children using the app were collected by family self reports, and app usage data was collected passively via automatic upload to the app data network. Parent perceptions of child progress were collected through consultation call notes. Logs of activity on Mightier games and family self reports collected in data forms and logs of consultation calls were subjected to quantitative and qualitative analysis.

Intervention

Mightier biofeedback video games

Mightier is a biofeedback video game platform where players are asked to regulate heart rate while completing video-game tasks. During app use, Mightier players wear an armband-affixed heart rate monitor that communicates with the app via BlueTooth® technology. Between March 2017 and October 2018, the review window for this retrospective data review, while they were using the app, players wore a Scosche Rhythm+™ heart rate monitor with embedded Valencell PerformTek® optical biometric sensor continuous heart rate monitoring technology (Scosche Industries, 2022). In March 2017 there were 6 different video games in the Mightier app, and games were added to the app regularly during the data collection window for this retrospective data review, increasing to a count of 18 games in the app in October 2018. As of this writing, there are 27 games in the app. Each Mightier game consists of a game task and an inhibitor that responds to the biofeedback when the player's heart rate exceeds a rolling average threshold. In an example game, Hibachi Hero, players are asked to slice falling foods while avoiding slicing trash, creating a go/no-go inhibitory control task as the core game task. Players simultaneously control heart rate. If their heart rate goes above a rolling average, players get visual feedback that their heart rate is elevated. If their heart rate remains elevated, smoke begins to rise from the bottom of the screen and obscure the falling objects, creating the game's inhibitor (see Fig. 1. Game screenshot). The smoke dissipates when the player returns their heart rate to the baseline. When play is inhibited, players can either play through the challenge and calm through an individualized strategy, or optionally begin a guided deep breathing relaxation activity. Deep breathing is used as a guided calming strategy in the app due to its known effectiveness as a physiological down-regulator (Jerath et al., 2006).

Fig. 1

Game screenshot.

Coaching calls

Families had the option to engage with a master's level clinical coach in a half hour call once per month during the measured 16-week intervention period. The primary goal of coaching was to provide psychoeducational content related to emotion regulation and proceeded as semi-scripted interactions. Scripted coaching call content focused on (1) child and parent engagement with the app, (2) parent psychoeducation about how engaging with the biofeedback-based game app could contribute to improvements in players' emotion regulation, (3) child well-being, (4) problem-solving on the usage of Mightier in individual home routines, and (5) assistance with technological barriers to play. Studies have shown that providing psychoeducation to parents relating to child behavioral interventions increases the likelihood of parent participation in children's treatment (Martinez et al., 2017), and that parent participation in child treatment improves child treatment outcomes (Dowell and Ogles, 2010).

Ad-lib usage

Both Mightier play and coaching calls were ad-lib, meaning that dosage of the intervention was at the discretion of the family.

Measures and data collection

Intake survey

Parents completed an intake onboarding survey (Supplement A) where they identified child age, gender, identified symptoms and problems, and child stressors. Additionally, at intake parents rated family stress levels in the previous month on a 0–10 Likert scale, with 10 being extreme stress. Family onboarding survey questions were designed to help the app developers understand their customers, and were not based on scientifically validated scales. Onboarding questions asking parents about the degree to which a number of symptoms, including anxiety, frustration, and outbursts, interfered with their children's daily success, were informed by clinical and scientific literature linking symptoms to emotion dysregulation (for example, see Hannesdottir and Ollendick, 2007; Mitchison et al., 2020; Vacher et al., 2020).

Game usage

Game play data from the first 16 weeks after account activation was uploaded automatically to the app's secure network during app user gameplay, and was included in the retrospective study if it was collected during the first 16 weeks after account activation between March 29, 2017 and September 30, 2018. Length of time in program was examined as a predictor of outcomes. A dichotomous intervention completer versus non-completer analysis was undertaken. An individual app user is considered to have completed the intervention if they played during a minimum of 5 different weeks, and at least 75 total minutes of play in the first 16 weeks.

Coaching call notes

The retrospective data set includes qualitative data from 1716 coaching calls that occurred during the data collection window. During coaching calls in the data collection window, coaches followed a semi-structured interview script and recorded notes in a secure online database. Coaches recorded parent responses to open-ended questions as close to verbatim as they were able.

Mixed-methods analysis

Statistical methods

We conducted quantitative data analyses of passively collected gameplay data and closed-ended survey responses using Excel spreadsheet software and Stata Statistical Software (StataCorp, 2019). Descriptive statistics were used to describe the characteristics of children participating, gameplay data, and coaching call logs. We used t-tests to compare means between population segments, and z-tests to compare proportions.

Qualitative methods

We conducted qualitative thematic coding of parent responses to open-ended questions during coaching calls, following Stuckey's (2015) 3-step model. The first author, SRW, read through parent open-ended responses to coaching call questions about child emotional and behavioral health and signs of progress in emotion regulation since the start of e-mental health app use, and generated initial codes representing parent interpretation of child emotion regulation progress. SRW and SEW discussed initial theme codes, then SEW separately read through the same set of parent responses, generating additional codes. SRW and SEW used short-form memoing to finalize the code list, reviewed and updated their separate initial codings, then cooperatively reviewed and revised coding, generating a final set of themes. SRW and SEW coded using the final themes (interrater reliability 0.93), then met to resolve coding discrepancies. SRW conducted frequency counts of theme codes in coaching calls overall, and over time in program.

Results

Characteristics of the population

Age and gender

The mean age of players was 8.5 years old (SD 2.6 years, median 8 years). Player age ranged from 3 to 19 years old, with 60% of the player population between 7 and 11 years old. Over two-thirds of players during that time period were boys (71.5%), and 27.9% were girls.

Presenting problem

Table 1 shows frequent parent-reported player emotion regulation challenges. The most frequent player emotion regulation challenge reported by parents before their families started using the app was frustration, with nearly nine out of ten (89.8%) parents reporting their children's frustration as one of the reasons they purchased the app. Other top emotion regulation challenges that players were facing before starting app use include outbursts (82.6%), anger (76.5%), anxiety (71.1%), and aggressive behaviors (68.1%). By parent report, boys in the data set were as likely to experience frustration and anxiety as girls, and more likely to experience outbursts (85.4% of boys, 76.4% of girls, z = 3.48, p = .0003), anger (80% of boys, 68.8% of girls, z = 3.75, p = .0001), and aggressive behaviors (71.6% of boys, 60.3% of girls, z = 3.54, p = .0002). Parent-reported child emotion regulation challenges did not vary significantly by age.

Table 1

Player characteristics and emotion regulation challenges.

	#	%
Player gender
Boy	747	71.5
Girl	292	27.9
Non-binary or prefer not to answer	6	0.57
	1045
Player age
Younger than 5 years old	37	3.54
5	86	8.23
6	92	8.80
7	158	15.1
8	167	16.0
9	148	14.2
10	120	11.5
11	98	9.38
12	57	5.45
13	40	3.83
14 years old or older	31	2.97
	1034
Emotion regulation challenges
Frustration	938	89.8
Outbursts	863	82.6
Anger	799	76.5
Anxiety	743	71.1
Aggressive behavior	712	68.1

Self-report of family stress

Family stress in the month previous to intake averaged across all player families was 6.33 (SD 2.82, median 7.5) on a scale of 0 to 10. More than 1 in 6 parents (17.2%) who purchased the e-mental health app in the study time frame rated past month family stress 10 out of 10. Parents of boys reported slightly more stress (M = 6.48, SD = 2.80) than parents of girls (M = 5.99, SD = 2.81), (t(1037) = 2.50, p = .006). No statistical difference in family stress existed across child age groups.

Engagement

Of 1487 families that obtained a subscription during the data collection window, 840 satisfied criteria for being included in the study data. Within these families, 1045 players had individual play data we reviewed, of which 1013 (96.9%) had at least one minute of playtime. Median playtime in the first 16 weeks of e-mental health app use was 189 min, with a mean play time of 315 min (SD = 365 min). Children whose playtime was in the lowest quartile for the data set played 53 min or less over 16 weeks, while children in the highest quartile played between 448 and 2718 min over 16 weeks. The median number of weeks played in the first 16 weeks was 10.4 (M = 9.5 weeks, SD = 6.3 weeks).

By our definition of intervention completion, 59.6% of individual app users were completers, with 70.3% of players engaging for at least 75 min of app use, and 67.7% of players engaging for 5 weeks or more.

Parent-reported improvements

Parent perceptions of child progress were collected through thematic coding of clinical consultation call notes. During the data review window for this retrospective study, the developer of the biofeedback-based video game application was not yet sending parents standardized customer satisfaction data collection instruments or follow-up questionnaires corresponding to onboarding questions about child emotions and behaviors and stress. Customer satisfaction and symptom follow-up questionnaires were added to the customer experience later, beyond the data review window for the current study.

Thematic coding of parent-described improvements

Parents described the emotion regulation improvements they perceived in their children who used the app in terms of children's use of emotion regulation skills, awareness, communication, calm demeanor, confidence, and relationships. Table 2 describes themes that emerged from 1716 Mightier coaching calls that occurred during the data collection window with players' parents, with example quotes.

Table 2

Emotion regulation improvement themes and relevant quotes.

Theme	Example quote(s)
Use of emotion regulation skills	“Mom says that outside of the games she is able to prompt her now by saying ‘you need to take a deep breath’ and she does it. Mom says this was never done prior to Mightier and she really is making great progress towards staying calm.”“[The player] uses deep breathing in real life situations. He says he used it during a soccer game when the other team was cheating and he was angry. He says he's getting faster at calming down in game.”“[The player] says he used deep breathing at recess when a boy pushed him down while playing tag. He breathed instead of getting really angry.”
Increased awareness	[Player is] “understanding games well and really seeming to understand the connection between mind and body.”“She's noticing her body more now when her body is moving fast or slow.”
Improved communication	“Using deep breathing in game and talking about the game with his mom and sister. Able to talk about emotions more.”“The other day she told her mom when she was super excited and hyper and she blurted out - ‘My heart rate is going all crazy.’ Mom is really happy with her progress.”“When angry he will say to his mom ‘You're not giving me time to calm down’ and then will take deep breaths. Mom can also suggest he use his breathing when he is upset and he will.”
Calmer demeanor/calmer home	M“om credits the program with helping [Player] stay a little bit calmer- [Player] agrees to some extent. ‘I think I am ‘yes and no’ better. I am better at taking deep breaths.’”“[Player] has had a calmer couple of weeks and had a very calm conversation with his therapist who said that was the calmest conversation she's had with him!”Teachers feel he's been a lot calmer in the last month or so. He hasn't been breaking as many pencils lately.”
Increased confidence	“He said, ‘I feel like a Jedi because I can calm myself…’”
Improved relationships	“On top of [Player] being more overall positive behavior, seeming less emotional, and seeming to have a calmer temperament overall, parents also report that she seems more calm and reasonable when solving problems with them and with her brother.”

Change in parent-described improvement with exposure to the intervention

Parents were statistically significantly more likely to notice many manifestations of emotion regulation improvements during months 3 and 4 of app use, as compared to the first two months of app use. Table 3 reports the changes in parent-described improvement with greater exposure to the intervention.

Table 3

Parent-perceived child emotion regulation improvements over time in program.

Theme	Weeks 1–8	Weeks 9–16	Stats
Use of emotion regulation skills	79.1%	86.9%	z = 3.20, p = .001
Increased awareness	26.1%	43.3%	z = 6.11, p < .0001
Improved communication	15.7%	31.1%	z = 6.39, p < .0001
Calmer demeanor	12.7%	36.0%	z = 9.97, p < .0001
Increased confidence	2.0%	2.7%	z = 0.784, p = .435
Improved relationships	1.6%	2.7%	z = 1.33, p = .184

Discussion

This study describes the characteristics of children and families who used a commercial online videogame-based biofeedback program for emotion regulation skills, demonstrates that children will engage with a videogame-based biofeedback application with an appropriate dose to see change, and reports both parent perceptions of progress and higher frequencies in parents' perceptions of progress over time. In total, the data provides a window into the use of a digital e-mental health intervention in a community population in a way they are not often described: outside of therapeutic or clinical relationship.

Caretakers see their children in term of symptoms

The descriptive summary of these data can be taken to answer the question: “Who seeks care for emotion regulation?” Parents who purchased this e-mental health app tended to focus on behavioral concerns primarily, with a heavy weighting towards “frustration” and “outbursts” before more traditional symptomatic concerns of “anger” or “anxiety.” When aiming for scale of digital applications, developers would be well-served by targeting behavioral manifestations of challenges, and adopting the language that parents use to understand their children's difficulties.

Families engage with in-home solutions on levels comparable to in-person therapy

With a median engagement length of 10.4 weeks and a 59.6% completion rate, app use by families in the study was comparable or better than rates reported in the literature for family engagement with in-person therapy. Typically, just over half of children with behavior disorders engage with treatment (Ghandour et al., 2018), and just under 40% of children who begin psychotherapy receive at least minimally adequate treatment (Cummings et al., 2019).

App usage by individuals in the data set compares favorably to usage of children's apps: Nearly all children who opened the app engaged at least once, and over 70% played for more than 75 min. In comparison, a 2018 systematic review showed that 21% to 88% of e-mental health app users engaged with the apps minimally, with completion rates ranging from 0.5% to 28.6% (Fleming et al., 2018).

Families see and value outward expression of an emotional toolkit

A majority of parents in the data set considered this e-mental health app effective in helping children improve their emotion regulation when used as an in-home intervention. After engaging with the app, parents reported that their children demonstrated more positive behaviors and improved ability to manage emotions, citing use of self-calming skills, increased awareness, and other behaviorally manifested signs of emotion regulation improvements. Parent perceptions of child improvement changed over time, increasing proportionally as children had more exposure to the application. This evidence of the application's effectiveness as an in-home, self-managed tool for emotion regulation improvement adds to earlier evidence of its effectiveness in psychiatric settings (Ducharme et al., 2012).

Limitations, strengths, and next steps

This study is limited in its lack of a control group and random assignment to a control versus intervention condition, which is a barrier to isolating the e-mental health application as the exclusive cause of player emotion regulation improvements. Data collection related to parent perceptions of child emotion regulation progress was limited by the fact that parents were not asked to respond to follow-up questions about child emotions and behaviors and family stress corresponding to the questions they completed at onboarding. The lack of data representing child perspectives on the app, beyond children's willingness to engage with the app as evidenced by their play data, is a further data collection limitation. Data analyzed was limited to that from families whose parents completed the intake questionnaires and ratings and engaged with the app at least enough to create a player whose play could subsequently be tracked, introducing the risk of selection bias, with families who are liking the intervention more likely to complete these questionnaires. Furthermore, only families that purchased the app were eligible for inclusion and families with this level of motivation may have more benefit than those in a randomly ascertained sample of families. The results may also be affected by the design and specific characteristics of this particular e-mental health app.

Despite these limitations, there are strengths in this study, including the large number of app players included in study data, the exploration of the application as it is used in real-life settings, and the opportunities parents were given to respond to open-ended prompts. Future studies of pediatric e-mental health applications should test for potential associations between measures of player improvement and app engagement metrics such as playtime.

In pediatric digital interventions, both caretakers and their children play an integral role in engagement. Activation, the act of getting started, requires a caretaker to present the intervention and encourage child engagement. Furthermore, when working in naturalistic environments, app developers may not have the same kinds of opportunities that are present in clinical settings to bring psychoeducation to parents. Future studies should explore the interplay between parent psychoeducation and parent and child engagement in pediatric digital interventions.

Conclusion

The aims of this retrospective data study were to identify characteristics of children whose families adopted an e-mental health app for their in-home use, and to explore app engagement patterns and parent interpretations of child progress. The data tells an optimistic story for in-home usage of online pediatric, game-based applications, demonstrating that families are able to engage with this type of child e-mental health app for several hours of play over several months, and that parents and caregivers have seen signs that the e-mental health app is effective in helping children strengthen their emotion regulation skills. Study findings suggest that there is interest in online game-based pediatric e-mental health interventions among families whose children are experiencing mental health symptoms, and that families are able to self-manage at-home use of these types of interventions.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Neuromotion Labs Inc., makers of the commercially available Mightier e-mental health app, sponsored this research study. JK was involved in the original development of the Mightier e-mental health app, is employed by Neuromotion Labs, is involved with ongoing development of the Mightier app, and may own stock options in Neuromotion Labs. SRW and AP are employed by Neuromotion Labs and involved in ongoing development of the Mightier e-mental health app, and they may own stock options in Neuromotion Labs. SEW was employed by Neuromotion Labs when this research was conducted. JGH was involved in the original development of the Mightier e-mental health app and may own stock options in Neuromotion Labs.