A weak scientific basis for gaming disorder: Let us err on the side of caution.

We greatly appreciate the care and thought that is evident in the 10 commentaries that discuss our debate paper, the majority of which argued in favor of a formalized ICD-11 gaming disorder. We agree that there are some people whose play of video games is related to life problems. We believe that understanding this population and the nature and severity of the problems they experience should be a focus area for future research. However, moving from research construct to formal disorder requires a much stronger evidence base than we currently have. The burden of evidence and the clinical utility should be extremely high, because there is a genuine risk of abuse of diagnoses. We provide suggestions about the level of evidence that might be required: transparent and preregistered studies, a better demarcation of the subject area that includes a rationale for focusing on gaming particularly versus a more general behavioral addictions concept, the exploration of non-addiction approaches, and the unbiased exploration of clinical approaches that treat potentially underlying issues, such as depressive mood or social anxiety first. We acknowledge there could be benefits to formalizing gaming disorder, many of which were highlighted by colleagues in their commentaries, but we think they do not yet outweigh the wider societal and public health risks involved. Given the gravity of diagnostic classification and its wider societal impact, we urge our colleagues at the WHO to err on the side of caution for now and postpone the formalization.

This reply reflects the personal opinion of the authors involved. The content of this paper does not necessarily reflect the official opinion of their respective institutions.

Introduction

In our debate paper, we argued that formalization of a gaming disorder in International Classification of Diseases-11 (ICD-11) has potentially problematic medical, scientific, public health, societal, and rights-based repercussions that should be considered (Aarseth et al., 2017). The difficulty of identifying the divide between “normal” behavior and actual illness has long been a problem in psychiatry and psychiatric epidemiology, leading to false-positive diagnoses with significant economic and societal consequences (Frances, 2013; Wakefield, 2015). Thus, caution is warranted. Keeping this in mind, we maintain that the proposed new disorder lacks the necessary scientific support and sufficient clinical utility to justify making the jump from research construct to recognized diagnostic category. Given that ICD-11 has no category that would allow it to propose a tentative diagnosis for further study (Van den Brink, 2017), it seems premature to advance to full classification.

We acknowledge there could be benefits to formalizing gaming disorder, many of which were highlighted by colleagues in their commentaries, but we think they do not yet outweigh the wider societal and public health risks involved. Ultimately, given the gravity of diagnostic classification and its wider societal impact (Frances, 2013) and the low quality of the existing evidence base, we urge our colleagues at the World Health Organization (WHO) to err on the side of caution for now and postpone the formalization.

Commentaries and Debate Paper

We greatly appreciate the care and thought that is evident in the commentaries that discuss our debate paper (Billieux, King, et al., 2017; Griffiths, Kuss, Lopez-Fernandez, & Pontes, 2017; Higuchi et al., 2017; James & Tunney, 2017; Király & Demetrovics, 2017; Lee, Choo, & Lee, 2017; Müller & Wölfling, 2017; Saunders et al., 2017; Shadloo et al., 2017; Van den Brink, 2017). The authors of this response paper agree that there are some people whose play of video games is related to life problems. We believe that understanding this population and the nature and severity of the problems they experience should be one focus area for future research. But even if patients are simultaneously (A) intensively playing video games and (B) functionally impaired to a clinically significant level, the question remains whether A causes B and whether there is benefit from formalizing a disorder based on this assumption.

The authors of the current response are not opposed to the creation of new research constructs that cover both excessive forms of behavior and functional impairment (Billieux, Blaszczynski, et al., 2017; Billieux, Van Rooij, et al., 2017; Kardefelt-Winther et al., 2017). However, moving from research construct to formal disorder requires a much stronger evidence base than we currently have. As has been extensively debated in the context of psychiatric classification systems, both the scientific object of study and the practical utility of the disorder should be clearly and unambiguously established before formalizing new disorders in disease classification systems (Frances, 2013; Pingani et al., 2014; Wakefield, 2015). The burden of evidence and the clinical utility should be extremely high, because there is a risk of abuse of diagnoses both within and beyond the clinical context, as we argued in our original paper (Aarseth et al., 2017). From our perspective, this utility has not been sufficiently demonstrated at this point and the scientific evidence base is not yet sufficiently clear to address fundamental ambiguities that are still in play.

To illustrate the latter point about a lack of clarity in the evidence base, we point to the fact that authors are still found to be discussing the inherent properties of the proposed disorder within the 10 commentaries. Much confusion remains – even among authors supporting the diagnosis – regarding what, exactly, the gaming disorder is. Would a gaming disorder relate only to gambling oriented games or to video games more generally (James & Tunney, 2017)? Is the problem behavior caused by other underlying mental disorders (Billieux, King, et al., 2017), or is it a consequence of alluring game mechanics (James & Tunney, 2017)? Are we diagnosing people who play online games or offline games, or both (Király & Demetrovics, 2017)? And is gaming disorder just a subcategory of a broader Internet addiction disorder or perhaps just one of many behavioral addictions (Higuchi et al., 2017)? What, exactly, are the symptoms of gaming disorder? Or are we to presume that clinicians will know it when they see it? As of this writing, the WHO appears to have proposed four separate categories for gaming disorders, all of which appear to differ from the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5)’s Internet gaming disorder (IGD) (WHO, 2017a, 2017b, 2017c, 2017d). This suggests us considerable confusion in the field regarding what gaming disorder is. In our view, too many critical questions remain unanswered to support formalizing the disorder.

With regard to the clinical utility, it remains unclear what the clinical advantages are of a “gaming disorder” label. Clinicians need to be aware of intensive leisure time behaviors such as gaming and their (positive or negative) interplay with disorders (Müller, Beutel, & Wölfling, 2014; Van Rooij, Schoenmakers, & van de Mheen, 2017), but the question is whether a new diagnostic category will lead to improved treatment for patients. Arguments are made that it is financially practical to have a diagnosis, which may very well be true, but if this is all that is required to formalize a disorder, then we could make the same argument about almost every societal issue. The associated risks of stigmatization and diagnostic inflation should be considered as well.

Some of the commentaries addressed the issue of whether gaming disorder is truly a unique stand-alone disorder or merely symptomatic of other primary causes. Many existing diagnoses occur together and, in fact, this has sparked extensive debate about the generally overlapping nature of mental health diagnoses (Fried et al., 2017). However, the issue is not whether gaming disorder overlaps with other diagnoses, but whether it reflects a patient’s response to those disorders. Specifically, is what we call “gaming disorder” merely a coping strategy for those with depression, attention-deficit/hyperactivity disorder, or other disorders?

If gaming is a coping behavior in some cases (Kardefelt-Winther, 2016), it would make more sense to explore the underlying causes for this behavior first and be sensitive to the extent to which treating these first-order challenges might resolve gaming problems. This is a concrete research question that could be operationalized by clinicians who are actively working in this field. If existing approaches are clinically sufficient, we must ask ourselves: Do we need a new gaming-specific disorder category? This should be properly established before a disorder is formalized. Recently, some high-quality, preregistered studies have isolated gaming disorder symptoms and failed to find that they, in and of themselves, directly link to health over time (e.g., Przybylski, Weinstein, & Murayama, 2017), which further questions the value and accuracy of formalizing gaming disorder as a stand-alone disorder.

Multiple clinicians and researchers also argue, through the commentaries, that the proposed disorder will be useful to stimulate research and treatment (e.g., Higuchi et al., 2017). While ambiguities are indeed confirmed and problems with evidence are acknowledged (Shadloo et al., 2017), it is argued by multiple authors that the disorder will be the start of a process that results in a “better” disorder classification. It may indeed be possible that the formalization of a disorder will generate the momentum needed to study patients instead of healthy high-school/college students or non-representative online samples recruited from Internet gaming forums, as is currently the case in much of the literature (Van den Brink, 2017).

However, similar expectations were voiced when the DSM-5 included IGD as a proposed category for further study (Petry & O’Brien, 2013), but we have not seen any improvements with respect to more patient-centric research or a stronger evidence base. As some have argued, including Internet gaming disorder in the DSM-5 might have been more harmful than helpful (Billieux, Schimmenti, Khazaal, Maurage, & Heeren, 2015). It seems to have locked the research field into a confirmatory approach aiming to prove the existence, utility, and psychometric properties of the IGD criteria in various populations, rather than dealing with some of the more fundamental questions about the nature of problematic gaming raised in the commentaries, as well as this response paper. Findings from confirmatory studies are irrelevant if we have not established the scientific object of study and have some confidence in the accuracy and utility of the criteria that cover essential components. Such exploratory work was unfortunately marginalized in the wake of the DSM-5’s IGD proposal and this may continue if gaming disorder is included in ICD-11.

With respect to the claim that a formal diagnosis is necessary because it justifies treatment administered to those gamers who seek it: this line of reasoning assumes that there has to be a diagnosis for every therapeutic modality, which is clearly not the case. For example, people who consult family therapists do not do so because they suffer from a “family disorder,” but because they need professional help to deal with a perceived problem. Individuals who seek professional help for their gaming could receive such help without their gaming habits being converted into a psychiatric diagnosis. If there is a sufficient number of such individuals, clinics and specialized services would be established and would grow. This is analogous to services for other mental health problems, which are not tied to a particular diagnosis, such as services for sexual assault victims or bereavement.

Furthermore, formalizing a disorder with the intention to improve research quality neglects the wider non-clinical societal context. Formalization serves the immediate goals of some of the stakeholders, but not all. For example, clinicians obtain diagnostic clarity and opportunities for insurance coverage for their patients and researchers obtain opportunities to secure research funding. Unfortunately, the possibly negative societal consequences fall to the general population of individuals who play video games. Hundreds of millions of people spend billions of hours each week playing games (Brown, 2017; Lanxon, 2017; McGonigal, 2011). A move to pathologize gaming could have important ramifications for the potentially stigmatized or misdiagnosed healthy “highly engaged” gamers, a group that has been identified (in representative samples that postulated its existence) as comprising between 1.1% and 10.9% of the gaming population (Colder Carras, Van Rooij, et al., 2017; Colder Carras & Kardefelt-Winther, 2018; Van Rooij, Schoenmakers, Vermulst, Van den Eijnden, & van de Mheen, 2011; Wittek et al., 2016). This is a group that may strongly resemble problematic cases in the current diagnostic approaches, such as the WHO and DSM-5 frameworks, but that does not seem to experience significant life impairment as a consequence of their gaming (Deleuze et al., 2017; Snodgrass et al., 2014, 2018). While some commentaries argued that a diagnosis would only apply to clinical cases and not be applicable to regular healthy gamers (Müller & Wölfling, 2017; Shadloo et al., 2017), we find this view to be rather optimistic and not in tune with today’s media climate. The continuous flow of flawed and exaggerated media reporting around the assumed harms of gaming should serve as a reminder that whatever we may propose in a clinical setting tends to reach far beyond the setting for which it was originally meant (e.g., Kardaras, 2016). The influence of a gaming disorder diagnosis on wider society and its impact on parents and children everywhere is not something we can afford to ignore in our work.

Further Arguments in Favor of Caution

We will now provide some further arguments as to why it is necessary to err on the side of caution and limit the conceptualization of “gaming disorder” – or preferably wider behavioral addiction disorders – to the research context. We remain opposed to enshrining gaming disorder in diagnostic classification manuals that are widely used and consulted in policy settings, school systems, and healthcare. They are used by individuals who might not be knowledgeable about the nuances of media use, moral panic, and normative game-related behavior (including parents of children).

Robust scientific standards are not (yet) employed

In recent years, major concerns have been raised about the empirical foundations of psychological science in general (Schimmack, 2012) and the study of technology in particular (Elson & Przybylski, 2017), in addition to concerns about psychiatric epidemiology discussed above. We firmly believe the reproducibility crisis (Cybulski, Mayo-Wilson, & Grant, 2016) in the clinical (Nosek, Spies, & Motyl, 2012) and psychological (Open Science Collaboration, 2015) sciences have direct implications for the study of and debates surrounding gaming disorder. Standards exist for reporting and evaluating evidence for both clinical and observational studies (McLeroy, Garney, Mayo-Wilson, & Grant, 2016), and the use of these standards is rare in gaming disorder literature (as elsewhere). For example, even the most detailed and comprehensive recent review of IGD (Mihara & Higuchi, 2017) does not include a discussion or formal assessment of bias or limitations of the reviewed studies or the systematic review itself, which is not consistent with guidelines for systematic reviews of epidemiological literature (Stroup et al., 2000). In contrast to our stance, these important meta-scientific concerns were not apparent in the commentaries, many of which referenced the exact studies whose low-quality evidence motivated us to write the original debate paper. In our view, those studies suffer from a lack of scientific transparency and/or poor methodological choices that undermine our confidence in the findings. Much of the literature that seems to reflect a confirmation bias toward proving gaming disorder exists rather than treating the concept with appropriate scientific skepticism and curiosity. The inconsistencies in the literature have been commented elsewhere (e.g., Griffiths et al., 2016; Mihara & Higuchi, 2017; Quandt, 2017). To reiterate three key points:

First, basic reporting standards need to be improved, because the provision of data under analysis is not always clear. For example, several commentaries referred to separate publications that arose from a single data set of schoolchildren in Singapore (Gentile et al., 2011). Subsamples from this single data set have been used in upward of 19 publications, without proper cross-attribution (Przybylski & Wang, 2016). This practice, colloquially known as “salami slicing” (Chambers, 2017), presents a challenge from a publication ethics perspective (Šupak Smolčić, 2013), inflates the number of independent investigations one might conclude have been done on a topic, and thwarts efforts to systematically review outcomes from what is ultimately a single study (Mayo-Wilson et al., 2017). In some cases, it appears that identical constructs from the Singapore data set are conceptualized and assessed differently across publications (Ferguson, 2015).

Second, data transparency is generally low (Elson & Przybylski, 2017; Wicherts, Borsboom, Kats, & Molenaar, 2006). As far as we are aware, only three of the dozens of publications in the gaming disorder literature openly share their data and materials (Przybylski, 2016; Przybylski et al., 2017; Weinstein, Przybylski, & Murayama, 2017). The fact that so few data sets are publicly available is problematic, because it raises the concern that errors in these data sets will go undetected (King, Haagsma, Delfabbro, Gradisar, & Griffiths, 2013). The inaccessibility of raw data prevents researchers and reviewers from being able to directly interrogate work and precludes the scientific community at large from building upon the investments, researchers and taxpayers have already made to study gaming disorder (Morey et al., 2016). Third, although most of the studies in the literature are presented as providing confirmatory tests of research questions, we cannot rule out the possibility that these are in fact post hoc analyses and interpretations, because the sampling and analysis plans were not registered prior to data collection (Wagenmakers, Wetzels, Borsboom, van der Maas, & Kievit, 2012). This practice, known as Hypothesizing After the Results are Known, is thought to be widespread in psychology (Gelman & Loken, 2013) and increases the likelihood of false-positive findings (i.e., type 1 error). Until researchers responsibly distinguish between exploratory (i.e., theory-building) and preregistered confirmatory (i.e., theory-testing) analysis, much of the present research findings must be understood as tentative and thus inappropriate for creating diagnoses that will influence lives, health systems, and policies (Chambers, 2017; Munafò et al., 2017). Furthermore, researchers carefully need to select appropriate data sources for preregistered confirmatory analysis, including both the choice of cross-sectional or longitudinal data collection and the choice of population. We cannot rely on findings from cross-sectional research conducted with largely healthy populations – which arguably constitutes most research on this topic – when undertaking such a critical activity as formalizing a new disorder (Van Rooij & Kardefelt-Winther, 2017).

As mentioned previously, the onus to demonstrate that the evidence base is of sufficiently high quality to underpin a formal diagnosis is on those who advocate for formal classification. Given the problems listed in this section, we maintain that the quality of the existing evidence base is low. We ask those who wish to formalize gaming disorder in ICD-11 to demonstrate otherwise, as a high-quality evidence base needs to precede a formal disorder classification.

The argument for singling out video games is not convincing

A behavioral addiction definition focused purely on video games is on its face arbitrary. A convincing rationale for focusing on gaming, rather than the myriad of other activities one might overdo, is lacking. We acknowledge that some individuals may overdo gaming, just as they may overdo social media, work, or sex, or tan to excess or, indeed, dance. A brief subject search on PsychINFO using the terms “video game addiction” or “videogame addiction” or “gaming addiction” and similar terms for other behavioral issues, such as food (e.g., “food addiction” or “eating addiction”), exercise, work (“work addiction” or “occupational addiction”), plastic surgery, etc., reveal that other behaviors have also received considerable scientific attention. The results are presented in Table 1.

.

Literature citations in PsychINFO for various “addictions”

Proposed addiction	Number of found articles
Food addiction	229
Video game addiction	149
Sex addiction	117
Gambling addiction	71
Work addiction	65
Exercise addiction	55
Shopping addiction	19
Tanning addiction	6
Dance addiction	2

Yet, only gaming disorder has been proposed for ICD-11 inclusion, with no formal or transparent review of the evidence quality for any of the various addictions. We suggest that a general behavioral addiction category might be more initially defensible, both theoretically and in terms of clinical utility, than a myriad of specific behavioral addictions. Of course, such a broader disorder suffers from the same risks of abuse as a narrower one: the discussed robust scientific standards should be employed.

Moral panic might be influencing formalization and might increase due to it

Many of the commentaries sought to ease our concerns about moral panic by assuring us that a gaming disorder diagnosis would reduce moral panic. However, a historical analysis of moral panics finds that they usually work in the inverse direction – official reification promotes the panic, not eases it (Bowman, 2016; Ferguson, 2013). These can result in poorly thought out and ineffectual public policy efforts to restrict gaming time, such as South Korea’s “shutdown” law (which blocked online playing for children between 12 and 6 a.m.). While such “solutions” may lead parents, clinicians, and society to feel that something is being done to address the perceived problem of excessive gaming, in fact, this intervention has had a negligible positive effect and even some negative outcomes (Lee, Kim, & Hong, 2017). Moreover, as the United Nations Children’s Fund stated in their recent report “Children in a Digital World”: “applying clinical concepts to children’s everyday behaviour does not help support them in developing healthy screen time habits.” (UNICEF, 2017, p. 115)

Given that we have already seen a proliferation of both dubious treatment centers and authoritarian regulations of speech and content in the name of gaming addiction, we believe the course of events is predictable. The commentaries on our original paper have given us no reason for optimism, given how readily some authors equated gaming disorder with substance abuse (Müller & Wölfling, 2017), a comparison we consider misleading to the public and lacking in sound evidence (Kardefelt-Winther et al., 2017). We are also concerned that the formalization of gaming disorder might be a product of moral panic (Bean, Nielsen, Van Rooij, & Ferguson, 2017). We understand that certain countries face pressure to formalize gaming disorder as a response to perceived excessive gaming, but this does not reflect the situation in all participating countries that use the ICD. For countries where excessive gaming is perceived to be a problem, it may be more appropriate to seek solutions on a domestic level. This would enable cultural and context-specific considerations to be made that the ICD, although somewhat flexible, cannot easily accommodate.

Conclusions

To conclude, we would like to address a question raised by Griffiths et al. (2017) as a point of criticism to our call for more clinical samples in research: “How can there be clinical samples in relation to a mental disorder that should not exist in the first place?” The answer is, of course, that prior to enshrining gaming disorder as a diagnosis, its clinical utility must be demonstrated in high-quality, transparent research with patients. If no patients are found, this would suggest that we do not need the formal disorder category. However, even if we find patients to study, patients themselves (or their parents) can also be influenced by moral panic and may come to believe they suffer from a disorder due to news coverage. A highly publicized “disorder” may offer a simplified explanation for problems that, in fact, have a deeper meaning. There is no easy solution to this challenge, but it is clear that clinicians need to be critical evaluators of underlying disease processes. Unfortunately, clinicians too can be influenced by moral panic and, as the saying goes, “If the only tool you have is a hammer, everything begins to look like a nail.” Those who eventually conduct research with patients need to find ways to distinguish gaming patterns with significant negative long-term health effects from coping behaviors or temporary excessive gaming behavior without any serious long-term harm.

Griffiths et al. (2017) also ask, “How can such playing of video games be problematic, yet not be disordered?” This is a logical fallacy, in that the premise of the answer is implied in the question. Many issues are problems without being disorders. Experiencing stress due to work demands is a problem, yet not a disorder. Experiencing body dissatisfaction due to perceived competition with peers over mates can be regarded as a problem, yet not a disorder. This is because many problem behaviors are normative reactions to difficult circumstances. Relegating an excessive, by societal norms, behavior to the realm of the pathological has come under heavy criticism recently, e.g., in the removal of the bereavement exclusion for major depressive disorder in DSM-5 (Frances, 2013; Regier, Kuhl, & Kupfer, 2013; Wakefield, 2015).

As indicated above, some people may excessively play as a method of coping with other mental health issues. For others, gaming could be a way to avoid unpleasant activities such as work or school as part of an existential crisis about the direction of one’s life. Much like individuals who have lost a loved one may experience extreme mental states similar to major depressive disorder for an extended period; people who play video games may exhibit extreme behaviors in reaction to a stressor. The issues can be quite complicated. If we equate coping or responding to problems with a mental disorder, this will further expand the elastic boundaries of psychiatric diagnosis (Frances, 2013). They might stretch to the point of meaninglessness, potentially resulting in a dismissive view of behavioral addiction research (Billieux et al., 2015; Kardefelt-Winther et al., 2017).

We join other researchers in psychiatry, psychiatric epidemiology, and the social sciences who propose that a clinical disorder should rest on the foundation of rigorous, transparent, and standardized methods. This requires acknowledging the reality that all studies are not created equal. Empirical evidence derived from gold standard research featuring standards, such as formal assessment of multiple sources of bias, preregistration, open data, open materials, open analytic code, and comprehensive reporting of conflicts of interest, is fundamentally more valuable than the closed source and non-transparent work, which characterizes most of the current literature on gaming disorder. Confirmatory (i.e., hypothesis testing) data analysis plans must be publicly preregistered in advance of data collection and these must be distinguished from exploratory (i.e., hypothesis generating) findings to facilitate the generation of high-quality evidence. Such evidence must then be synthesized using appropriate standards to determine consistency and strengths of effects as well as sources of bias in a way that can inform policy-making (Liberati et al., 2009; Stroup et al., 2000). This level of evidence is not yet reached in gaming disorder research.

Risk of abuse of a formalized new disorder that solely involves the behavior of playing video games – a stigmatized entertainment activity – can only expand the false-positive issues in psychiatry. This expansion will likely have a psychological and societal cost, potential harming the well-being of our children. We understand the arguments for wanting a clinical disorder, but maintain that the clinical utility of the proposed diagnosis is still unclear and the evidence base is not yet good enough. In short, we believe this debate is worthwhile and that a case might be made for diagnostic formalization in the future, but currently it is premature.

How to move forward: Beyond the echo chamber

We agree with Kuss, Griffiths, and Pontes (2017) on the importance of including multiple stakeholders in the process of formalizing a potential disorder, including not just academics, but gamers, industry executives, therapists, and others. We suggest that the WHO solicit input and feedback from a wider variety of stakeholders, including individuals who currently seek help or who have sought help for gaming-related problems and their family members/carers, as recommended for the development of mental disorder classifications in general (Pingani et al., 2014; Stein & Phillips, 2013). Children in particular should be included in this process as one of the primary stakeholder groups playing games regularly. According to the United Nations Convention on the Rights of the Child, children have a fundamental right to have their voices heard in matters that concern them: formalizing a disorder classification that involves one of children’s most popular everyday behaviors certainly concerns them.

Involving the aforementioned groups in the decision-making process will lead to a more holistic and accurate view of the diversity of video gaming. Stakeholder engagement has proven to be a fruitful endeavor in participatory mixed-methods research conducted with gamers (Colder Carras, Porter, et al., 2017). Indeed, we would take this a step further and encourage such stakeholders to participate in the peer review of studies before they are conducted by following the Registered Reports methodology (www.cos.io/rr/). In this approach, multiple stakeholders must agree on the value of the study hypotheses, methods, and sampling plans, so that the findings of any given study are not subjected to control by any interested party. Finally, we suggest that the WHO working group conducts or commissions a critical review of the existing literature following the highest standards for evidence evaluation with an eye toward the Open Science principles mentioned in this paper.

We remain optimistic for the future. Some preregistered and methodologically rigorous papers have recently been published in this field (e.g., Weinstein et al., 2017) and valuable qualitative work is starting to appear more widely, challenging the disorder model in some cases (Snodgrass et al., 2017, 2018). In the meantime, we would strongly encourage the WHO to err on the side of caution, halt further formalization of new gaming disorders, and stimulate better research into the role that screen time plays in our lives.