Lost in the chaos: Flawed literature should not generate new disorders.

The paper by Kuss, Griffiths, and Pontes (2016) titled "Chaos and confusion in DSM-5 diagnosis of Internet Gaming Disorder: Issues, concerns, and recommendations for clarity in the field" examines issues relating to the concept of Internet Gaming Disorder. We agree that there are serious issues and extend their arguments by suggesting that the field lacks basic theory, definitions, patient research, and properly validated and standardized assessment tools. As most studies derive data from survey research in functional populations, they exclude people with severe functional impairment and provide only limited information on the hypothesized disorder. Yet findings from such studies are widely used and often exaggerated, leading many to believe that we know more about the problem behavior than we do. We further argue that video game play is associated with several benefits and that formalizing this popular hobby as a psychiatric disorder is not without risks. It might undermine children's right to play or encourage repressive treatment programs, which ultimately threaten children's right to protection against violence. While Kuss et al. (2016) express support for the formal implementation of a disorder, we argue that before we have a proper evidence base, a sound theory, and validated assessment tools, it is irresponsible to support a formal category of disorder and doing so would solidify a confirmatory approach to research in this area.

This paper provides a commentary on a recent paper by Kuss, Griffiths, and Pontes (2016), in which the authors set out to review conceptual issues relating to the study of Internet Gaming Disorder and provide clarity for future research. We agree with many of the points raised in the original paper. In particular, we appreciate the review of existing criticisms directed toward the poor criteria for Internet Gaming Disorder proposed in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), a review which draws on a larger collaborative effort published in the journal Addiction last year and of which we were a part (Griffiths et al., 2016). However, we also find ourselves disagreeing with some aspects of the manuscript, such as the approach taken to summarize the state of knowledge on Internet Gaming Disorder and the leveraging of this flawed evidence base to support a new category of disorder.

Kuss et al.’s (2016) paper examines issues relating to the concept of Internet Gaming Disorder, with a specific focus on the overlap and differences between problematic Internet use or “Internet addiction” on the one hand, and gaming problems or “Internet Gaming Disorder” on the other hand. Although this is a relevant distinction that deserves consideration (Király et al., 2014; Starcevic & Billieux, 2017; Van Rooij, Ferguson, Van de Mheen, & Schoenmakers, 2017), we believe it is an issue of relatively minor importance when aiming to solve the “chaos and confusion” that currently surrounds Internet Gaming Disorder. At present, the field lacks basic theory, definitions, and properly validated and standardized assessment tools. This is quite common in behavioral addiction research (Kardefelt-Winther et al., 2017), where researchers have been moving along far more quickly than our understanding permits.

While support for a new clinical disorder is expressed in the conclusions of Kuss et al.’s (2016) paper, we note that the vast majority of studies on this subject – including the ones proposing new assessment tools for the disorder – have not included patients (Pontes, 2016; Van Rooij, Schoenmakers, & Van de Mheen, 2016; Van Rooij, Van Looy, & Billieux, 2016). Combined with an overreliance on survey results and a lack of sound theoretical footing, the past two decades of research must unfortunately be said to have contributed mostly to a proverbial air castle: many claims of knowledge and consistency in findings do not hold up under closer scrutiny.

This is evident throughout the existing literature, and inconsistencies have been well documented in a series of critical studies (Billieux, Schimmenti, Khazaal, Maurage, & Heeren, 2015; Clark, 2015; Griffiths et al., 2016; Kardefelt-Winther, 2015, 2016; Kardefelt-Winther et al., 2017; Starcevic, 2013; Van Rooij & Prause, 2014; Wood, 2008). Borrowing from the paper by Kuss et al. (2016), to provide a more specific example, a typical statement common in many research papers in this field may look like this:

“Twenty years of research on technology use related problems have indicated that technology overuse may result in problems that are traditionally associated with substance-related addictions, including addiction symptoms, such as salience, mood modification, withdrawal, tolerance, conflict, and relapse (Kuss, Shorter, Van Rooij, Griffiths, & Schoenmakers, 2014).”

From this statement, a reader might infer that there is ample evidence that technology overuse is associated with problems of the same nature and severity as substance-related addictions. This is not true: the evidence base is much weaker than this statement suggests and little is truly known about the outcomes of technology overuse. In fact, we do not even have a clear idea of how to properly define overuse or problematic use of technology (Kardefelt-Winther et al., 2017); a complicated task in a world where constant access is a reality for many. This confusion is clearly visible in the continuing trend to ambiguously discuss and compare overuse of entire media channels (Internet), certain interactive entertainment products (e.g., specific games), and digital devices (e.g., smartphone “addiction”), as highlighted by Kuss et al. (2016) in their paper. However, the problems in this field go far beyond determining what precisely we are overusing, whether it is Internet more broadly or only Internet-based games. For instance, the cited paper and the accompanying research base referenced in the paper by Kuss et al. (2016) are non-clinical in nature and represent mostly survey studies or theoretical papers.

While we respect survey work and theoretical papers in general, having contributed to many such papers ourselves, one must appropriately recognize the value of such work, and when and how to use it. Most survey-based assessments of “outcomes” of disorder are non-clinical in nature. This means that we are not able to conclude that Internet Gaming Disorder co-occurs with a clinically significant depressive disorder by finding a significant correlation between Internet Gaming Disorder and a depression measure, but rather that elevations in (averaged) Internet Gaming Disorder scores are associated with an elevation in depressive mood as self-reported by the surveyed respondents. This makes certain statements in Kuss et al.’s (2016) paper, such as “Problematic Internet and gaming use furthermore appears highly comorbid with various other mental and physical disorders, including depression, anxiety disorders, obesity, and attention-deficit hyperactivity disorder” somewhat misleading to the uninformed reader.

Self-reported survey scores are not a valid way to establish presence of a disorder in the same way as a clinical interview (Maraz, Király, & Demetrovics, 2015), which means that most claims of clinical co-morbidity are unsupported by evidence. In fact, although researchers in this field make many inferences and claims relevant only to a clinical context, a majority of the empirical work is still conducted with non-clinical populations. Thus, most populations used in survey research on technology overuse are ill-suited for the purpose of validating a new disorder. To illustrate this point, Figure 1 presents a theoretical schematic overview of a typical population triangle, assuming a low prevalent issue, adopted from approaches taken to assess gambling harm (Blaszczynski, Ladouceur, & Shaffer, 2004). In survey research on technology overuse, we are likely to find at least three groups of people: a large group of healthy users who have no problems at all with their technology use (group A: non-problematic users); a small group of people who might experience some light problems with their use (group B: problematic users); and a very small group of severely problematic, potentially functionally impaired, people who are still available in schools or universities to answer survey questions (group C: functional impairment).

.

Schematic overview of the population reachable by survey research

Survey studies of technology overuse, such as Internet Gaming Disorder, lean on assessment instruments that involve questions such as “Have you been feeling miserable when you were unable to play a game?” (Lemmens, Valkenburg, & Gentile, 2015), “I feel sad if I am not able to play games” (Pontes, Király, Demetrovics, & Griffiths, 2014), or “When you were not playing, how often have you fantasized about gaming, thought of previous gaming sessions, and/or anticipated the next game?” (Király et al., 2017). Summarizing individual scores on these types of scales fairly consistently presents us with a small but sizable group of respondents that self-report experiencing some problems with their use (group B) (Ferguson, Coulson, & Barnett, 2011).

Problematically, even if these instruments would be able to assess true disorder with the same accuracy as a clinical interview (group C), which seems unlikely (Maraz et al., 2015), there is another fundamental issue at play: truly clinical cases will rarely be available to fill out survey questions. So, a problem inherent in the approach taken is the fact that a (sizable) part of the hypothesized disordered group C will not be reachable by surveys (Figure 1, top), because they are non-functional and/or likely to be at home in front of the computer. At best, a very small number of potential clinical cases are reached and included in analysis. If the researcher accepts this and is willing to study a mostly healthy population (group A) together with some potentially problematic users (group B) and a few still functional clinical cases (group C), then the researcher must recognize that the sample is highly skewed, as most people do not self-report many problems resulting from technology overuse. This makes statistical inferences about problematic users difficult based on this largely healthy population. Partly for these reasons, we argue here that we actually know very little about both (causal) risk factors and outcomes of technology overuse, and certainly not enough to make statements about comorbidity or to support a formal category of disorder. Most studies up until this point have dealt with largely healthy populations.

We believe it to be crucial for future empirical studies that this lack of knowledge is accurately reflected in literature review sections. For methodology sections, we suggest that researchers provide clear indications about the nature of the sample under study (e.g., groups A, B, or C, as explained), by carefully stating what kind of population it can actually be said to represent (e.g., students, patients, self-reported problematic users, and clinical cases).

Tying the discussion back to the original paper by Kuss et al. (2016), we argue that the current evidence base supporting the Internet Gaming Disorder construct is deeply flawed. Not only because of conceptual confusion between Internet addiction and Internet Gaming Disorder, but because the uncritical citing of survey-based findings has indeed thrown the evidence base into chaos. We cannot trust summaries of the existing body of evidence, because too much of it is based on misinterpreted survey data and exaggerated interpretations of second-hand references.

Thus, we are confused by the authors’ apparent support for inclusion of Internet Gaming Disorder in official nomenclatures, even more so given their critique of the criteria proposed. However, we are not suggesting that having a clinical diagnosis would be useless. It might stimulate research, provide a common reference point for treatment plans, and provide an incentive for insurance providers to cover patients, as Kuss et al. (2016) suggest. But political reasons, practical aspects, and an increase in funding opportunities for our research endeavors should not guide this decision process. We should call for official nomenclatures to include new disorders only when we can recognize the disorder, diagnose it and treat patients in need – currently, we lack the knowledge required to do any of this reliably.

Moreover, there are genuine risks involved in creating a new disorder. We believe that Kuss et al. (2016) do not fully consider the impact that recognizing a formal disorder would have on gamers everywhere. Gaming is different from substance abuse behaviors in that it is one of the most popular hobbies for children and adolescents worldwide, with many healthy and positive outcomes resulting from it (Granic, Lobel, & Engels, 2014). Therefore, whether we formalize extensive gaming as a disorder or a normal past-time activity is likely to impact the general population of gamers and the attitudes of their parents. Creating a formal disorder in the end might cause more harm than good: we have already witnessed the rise of “treatment centers” for Internet addiction in certain countries where military regimens are employed, followed by anecdotal reports of physical and psychological abuse of children and adolescents (Russon, 2016; The Paper, 2016). The risk of further legitimizing such violations of children’s rights should be given great consideration before a category of disorder is formally supported by researchers active in this field. This illustrates why it is vital that our thinking around potential behavioral disorders that encompass popular leisure activities or hobbies demand a different mind-set and a range of new considerations compared to when we study substance abuse behaviors (Kardefelt-Winther et al., 2017).

Conclusion and Suggestions for the Future

In conclusion, supporting the formal implementation of a disorder before we have a proper evidence base, a sound theory, or validated assessment tools, is irresponsible and solidifies a confirmatory approach to research that seems to serve the research community rather than patients, ignoring genuine risks to gamers everywhere, as a large group of authors have recently stated (Aarseth et al., 2016).

At this point, a good way forward would be for researchers to re-conceptualize what Internet Gaming Disorder truly represents, without taking practical shortcuts by grounding it in addiction theory. Researchers will need to work hard to reduce the quantity of papers produced in this field and instead improve their depth and quality. The focus on whether to call the current problem Internet addiction or Internet Gaming Disorder is not the most pertinent question at this time. While it is certainly a part of the necessary theoretical discussion, the problems we need to address are much more fundamental:

A basic theory needs to be developed for both Internet Gaming Disorder and behavioral addiction more broadly. A recent paper (Kardefelt-Winther et al., 2017) advances a first proposal for a definition of behavioral addiction. This definition can and should be developed further in a transparent fashion. We invite researchers to join in this development through an open science foundation framework (Billieux et al., 2016).

We also need to work with populations that are suitable for the results we wish to achieve, or at the very least be upfront about eventual shortcomings. Conducting more clinically oriented studies on patients that present with a combination of life impairing problems and technology overuse should help us develop proper theory and assessment tools that reflect clinical reality (Kardefelt-Winther et al., 2017).

Finally, we need to re-think the ways in which we produce, cite, and use “evidence” from existing literature, to avoid causing the same chaos and confusion that we now find ourselves in. Until we have more clinical studies in place, it is crucial to pay attention to what existing studies show and to not exaggerate findings.

Accounting for the points reflected above should be a cornerstone of a researcher’s formal training, but we fear that they are often overlooked in practice.