Computational Psychiatry of ADHD: Neural Gain Impairments across Marrian Levels of Analysis.

Attention-deficit hyperactivity disorder (ADHD), one of the most common psychiatric disorders, is characterised by unstable response patterns across multiple cognitive domains. However, the neural mechanisms that explain these characteristic features remain unclear. Using a computational multilevel approach, we propose that ADHD is caused by impaired gain modulation in systems that generate this phenotypic increased behavioural variability. Using Marr's three levels of analysis as a heuristic framework, we focus on this variable behaviour, detail how it can be explained algorithmically, and how it might be implemented at a neural level through catecholamine influences on corticostriatal loops. This computational, multilevel, approach to ADHD provides a framework for bridging gaps between descriptions of neuronal activity and behaviour, and provides testable predictions about impaired mechanisms.

The Need for a Better Neurocomputational Understanding of ADHD

Maintaining one's mental focus is hard, especially when reading a dry and complicated paper. Suddenly you would rather clean the kitchen or surf the Internet. Nevertheless, most people maintain focus and persist with the task at hand. Neurobiologically, we propose that the catecholaminergic brain systems (see Glossary) modulate attention [1] by increasing the neural gain and, thus, suppressing cognitive switching [2] (Box 1).

For 5% of the population, the ability to focus is disturbed to an extent that strongly affects their daily functioning. Many are diagnosed with ADHD [3], a developmental psychiatric disorder thought to arise, in part, out of a genetic vulnerability [4]. ADHD is characterised by inattention, hyperactivity, and/or impulsivity [5] and its negative effects on a person's occupational success, wellbeing, and health risks (e.g., for substance abuse [6]) make it important to understand this disorder.

Research on ADHD has intensified since the early 1990s [6] without clear candidate genes or brain response patterns predicting the disorder having been identified. There is no unifying theory explaining the pathophysiology of ADHD. Indeed, current classification criteria are likely to subsume multiple brain disorders with a similar behavioural expression within the label ‘ADHD’.

Here, we use a multilevel approach to propose that ADHD crucially involves an impairment of neural gain modulation leading to inappropriately variable behaviour. By using Marr's three levels of analysis [7] (Box 2), we show how it is possible to translate behavioural findings into mathematical algorithms and neural circuit impairments (and vice versa). This approach also provides fruitful hypotheses about potential neurobiological subgroups, which could be the object of future investigation.

Neurocognitive Impairments in ADHD

To understand a psychiatric disorder, it is important to unite several levels of impairments spanning symptoms, behaviour, neural, and neurochemical markers. Here, we selectively review the most consistent neurocognitive impairments and go on to argue that these can all be explained by impaired neural gain.

Behavioural Markers: The Consistency of Inconsistencies

Behavioural findings in ADHD are numerous, and here we confine ourselves to a general pattern of ADHD-related impairments consistently present across domains and tasks.

Reaction Time Variability

One of the most consistent findings in subjects with ADHD is an increase in reaction time (RT) variability (such as RT standard deviations). This is reliably found across many tasks, laboratories, and countries [8] and is one of the best behavioural classifiers for ADHD [9].

Response Inconsistencies

Simple response tasks, such as the continuous performance task (CPT, Box 3), require a participant to respond to prelearned target stimuli while withholding an action for nontarget stimuli. This simple response-to-target, nonresponse-to-nontarget pattern is used in a variety of task settings that investigate different cognitive domains, such as attention (alertness, vigilance, and sustained attention tasks), response inhibition (Go/NoGo and Flanker tasks), or working memory (n-back tasks). Across all these tasks, patients with ADHD generally make less target-related responses (errors of omission) and more nontarget responses (errors of commission) [10]. Subsequently, we use the CPT as an example of these response biases and to illustrate how these impairments can be caused by decreased neural gain (Box 3, Box 4).

Decision Making and Reward Learning

In the context of neuroeconomic approaches to behaviour, decision-making has received considerable attention from the ADHD community 11, 12, 13, 14, 15. However, relatively few studies have used neuroeconomic tasks and models that address actual mechanisms and their putative impairment in ADHD. In one of the first such studies, Hauser et al. [16] investigated decision-making in adolescent patients with ADHD using learning models and found that an increased decision temperature parameter (Box 3) accounted for the more stochastic behaviour seen in ADHD. This is in line with previous computational and animal work relating ADHD-like behaviours to decision temperature [17]. Other studies investigated delay gratification and temporal discounting to study impulsivity in ADHD. While such initial reports suggested increased discounting in ADHD, more recent studies reveal a more complex picture [18]. However, we note evidence that increased discounting is strongly associated with increased choice variability [19].

Neural Markers: The Catecholaminergic Systems

In contrast to other psychiatric disorders, ADHD has relatively few candidate neurotransmitter systems. Studies from different fields have converged on the catecholamine neurotransmitter systems (Box 1) dopamine (DA) and noradrenaline (NA) as contributing to the impairments seen in ADHD 13, 20, 21, 22, 23.

Methylphenidate is a highly effective treatment in ADHD whose mode of action is a targeting of dopaminergic reuptake from synaptic cleft [24]. By preferentially blocking the re-uptake of DA, methylphenidate increases synaptic DA and, hence, dopaminergic transmission. Nonstimulant medications, such as atomoxetine, more specifically target the noradrenergic system in prefrontal areas and may be more effective in patients with a putative deficit in NA regulation [25]. While atomoxetine prevents NA from being removed from the synaptic cleft, other drugs specifically stimulate α2-adrenoceptors rather than acting on all NA receptor types 22, 26.

A source of more direct evidence comes from human positron emission tomography (PET) and animal studies that suggest a hypofunction in a DA system in striatal and prefrontal areas in ADHD 13, 14, 21, 23, 27. Less evidence is available for NA involvement due to methodological reasons [21]. In addition, genetic studies implicate DA- and NA-related genes in ADHD 6, 22, 28.

In line with the relatively widespread effects of neural gain in the brain [29], functional neuroimaging in ADHD has revealed multiple brain networks as affected 6, 30, including the striatum [15] and medial prefrontal cortex 16, 30. It is of interest that both are densely innervated and modulated by catecholamines 31, 32, 33 and show deficient functioning during task performance and at rest 30, 34.

Deficient Neural Gain Modulation in ADHD

Here, we illustrate how lowering neural gain at the neurophysiological (implementation) and algorithmic levels can induce ADHD-like neurocognitive impairments. To understand why the brain uses neural gain modulation to guide behaviour in the first place, we first discuss the importance of balancing between choice stability and choice variability from a theoretical standpoint.

Computational Level: Why Arbitrate between Stable and Unstable Behaviours?

So far, we have concluded that a consistent feature of ADHD is an increased variability in behaviour. According to Marr, the first level of analysis should describe the problem a system (i.e., the brain) faces and how it tries to solve it [7]. So why should the healthy brain allow for substantial behavioural variability? Why does the brain not always select the option with the highest returns according to the information available? Why do we sometimes go for options that are not the best and explore? We note that this is not about simple imperfection, because there are numerous biological functions that are executed with engineering precision.

The dilemma that the brain has to solve arises from acting in environments where different options may change their value for the subject. Agents not only have to exploit the option it estimates as the best, but must also explore the value of alternative options so as to gather more information 35, 36. One example is foraging, where different trees may change the amount of fruits they carry. Thus, it is more adaptive to occasionally try alternative trees. This might be particularly important in a developmental context, where a child has a limited prior knowledge about an environment and, thus, can profit from exploring unknown environments.

From both a reinforcement learning and information theoretic perspective, the arbitration between different options is construed as balancing ‘exploitation’ and ‘exploration, information gathering’. This is a hard problem to solve, but there are simple, well-established, methods, such as randomly sampling from one's beliefs, or Thomson sampling [37]. Recent neuroscientific work suggests that both immediate utility and information gathering drive our behaviour [38]. We note that controlled addition of noise to a system to optimise its behaviour is by no means confined to decision-making and applies to many problem-solving systems (e.g., stochastic resonance or simulated annealing).

The increased variability in ADHD can be seen as altered exploitation–exploration trade-off. In paradigms with no uncertainty, increased exploration makes no sense; by contrast, in a natural environment, the optimal amount of attentional stability, in view of uncertainty, is a matter of degree. Moving to a societal level, increased exploratory behaviour in a proportion of the population may be advantageous. Simulations by Williams and Taylor [39] demonstrate that groups with 5% of ADHD-like agents show optimal foraging behaviours and increased survival, and may explain why ADHD remains prevalent in the population despite its negative effects on the individual.

In summary, the brain has to arbitrate between either exploiting currently preferred options or sampling alternatives and learn from experience. While low exploration in most members of a group ensures stability, a low proportion of people with ADHD allows learning from exploration and, thus, can be evolutionarily beneficial for a group.

Algorithmic Level: How to Arbitrate between Exploitation and Exploration?

The second level of Marr asks how a problem is solved. Specifically, it asks for mathematical descriptions of how the system solves its task. In recent years, these approaches have gained increased interest. Bayesian reasoning and reinforcement learning theories in particular have provided biologically useful algorithms that the brain appears to exploit 40, 41, 42, 43, 44.

Can reinforcement learning account for behavioural variability across different tasks and cognitive domains? In Box 3, we propose that increased variability can be explained by an altered action selection process. At the core of this action, selection process is the decision temperature parameter τ, a measure of choice stochasticity. It describes to what extent the agent sticks to what it effectively believes to be the best choice. Higher decision temperatures make the agent more likely to choose from options currently estimated to have less-than-maximum values. By contrast, lower temperatures make the agent choose the highest value option more often, thereby avoiding alternatives even if they have almost the same value (Box 3). Thus, increasing τ elicits more variable behaviours, even in simple stimulus–response tasks. A similar effect has been shown in the context of delay gratification [17]. It is important to note that, in temporal discounting, subjects with high temperatures also tend to have high discounting preferences [19]. Lower temperatures are good for exploiting current beliefs, while higher ones help exploration of uncertain options, as well as evening out resource utilisation.

In the context of learning and decision-making, previous theories 12, 14 proposed that impaired learning would elicit ADHD-like behaviour, driven by impoverished reward prediction error (RPE) signals. However, recent empirical data that addressed learning and decision-making in ADHD demonstrated that ADHD participants are not well characterized by impaired learning, but instead by an increased decision temperature [16].

We can tentatively conclude that increased variability at the algorithmic level is explained by an increased decision temperature in relation to an action selection process. We suggest that this is likely to be underpinned by lowered neural gain, potentially caused by malfunctioning catecholamine systems (Box 3) and altered connectivity 45, 46. Neural underpinnings apart, an understanding of the key deficits of ADHD at the algorithmic (information-processing) level may inform learning-based treatments for this disorder, for which there is currently great demand but limited evidence as to their efficacy 47, 48.

Implementation Level: How Does Gain Affect Computations in Neural Loops?

The implementational level asks how the algorithm functions of the second level are realised in neural hardware, that is, in this instance, how the brain circuits instantiate and dynamically select between different options, what structural change is associated with an hypothesised neural gain impairment, and how this is translated into behavioural dysfunctions.

Neural models of corticostriatal circuits provide tools to study a catecholaminergic modulation of behavioural selection processes, such as the effects of reduced DA in striatal areas. Since refined maps of DA receptor distributions in the striatum are established, the majority of these models investigate the role of DA 49, 50. Such models describe how information is propagated from the striatum to the cortex (and back) through multiple pathways, and how these loops process and represent complex information. Striatal DA has a crucial role in this information processing (Box 4) and these models have been successful in describing neural processes underlying motor impairments in disorders such as Parkinson's disease (PD) 51, 52, 53. Previous corticostriatal models have also been successful in describing ADHD-like response inhibition and working memory deficits, but do not explain increased response variability through DA impairments [23]. Recent refinements in understanding the specific functions of the basal ganglia pathways 54, 55 have led to a substantial change in how we think of a D2-driven indirect pathway that allows us to account for ADHD-related variability by means of DA impairments (Box 4) 56, 57. This has also facilitated an understanding of why the same pharmacological increase in DA can improve disorders that are at the opposite side of a motor activity spectrum, namely ADHD and PD. Few frontostriatal loop models have considered the contribution of other catecholamines, such as NA. Notably, Frank et al. [23] showed that impaired NA function increased behavioural variability as seen in ADHD by changing neural gain in prefrontal areas.

The aforementioned models of corticostriatal loops demonstrate that multiple impairments in neural gain (such as decreased frontal NA [23] or lowered striatal DA efficacy [56]) can cause increased behavioural variability. This raises interesting new questions that can be addressed in future behavioural, modelling, and (pharmaco-) neuroimaging work. Key here is to understand how different catecholamines can be dissociated, not only in terms of their impact on behaviour, but also with respect to the neural correlates of these impairments. Moreover, it is important to determine which receptor types are involved in ADHD. We consider it likely that different ADHD subgroups can be characterised by specific receptor impairments and, thus, a specific neurocognitive pattern. For example, our corticostriatal loop models 2, 56 suggest that neural gain impairments can be caused by either reduced DA release in the striatum or impairment at the level of D1 or D2 receptors. Current PET studies support an impaired striatal DA release as well as changes in D2 receptor density [21]. For NA, ADHD has mainly been associated with impairments in α2-adrenoceptors 26, 58, known to boost prefrontal representations 58, 59. More recent evidence also highlights the importance of β-adrenoceptors for modulating neural gain [60]. Only by finding specific neurocognitive markers of catecholaminergic impairment, we will be able to obtain neurobiologically valid ADHD subtypes and, thus, refine the targeting of pharmacological therapy (see Outstanding Questions). Moreover, such refinements of ADHD subtypes could facilitate nonpharmacological interventions, such as neurofeedback 47, 61, 62, 63, 64 and transcranial brain stimulation, allowing a focus on more specific neural substrates (Box 2).

Concluding Remarks

To understand psychiatric disorders such as ADHD it is important to determine which neurocognitive processes go awry, and how. Psychiatry has traditionally suffered an explanatory gap between neurobiological mechanisms and symptom-level behaviours. Mathematical attempts to bridge different levels of description are few, but only by working across levels that span computational theory to neural implementation and back, can we better understand the neurocognitive impairments causing psychiatric disorders.

Here, we illustrate that ADHD can be described in terms of impaired neural gain across different levels of analysis. Based on the premise that the brain needs to arbitrate between exploration and exploitation, we show that an increased behavioural variability in ADHD can be expressed as neural gain impairments by an increased decision temperature parameter at an algorithmic level, as well as by catecholaminergic impairments at a neural implementation level.

Similarly, we can conceptualise key symptoms of ADHD as stemming from neural gain impairments. For example, inattention can be seen as a frequent shifting between different goals and an inability to stay with, and focus on, the currently most valuable option (as illustrated in Box 4). Likewise, decreased neural gain and, hence, behavioural switching may contribute to hyperactivity. By contrast, it can be conceptualised as akin to inattention, where frequent switches between cognitive goals propagate through the motor system and lead to frequent changes in motor programs, possibly characterising a combined ADHD subtype. A characteristic of such an impairment might be sudden standing up during class or the abrupt stopping of an ongoing behaviour. Alternatively, the neural gain impairments could only arise at a motor level, where one would expect markedly increased, undifferentiated motor actions and an inability to suppress evanescent, but inappropriate, motor response tendencies without marked inattentive symptoms (i.e., hyperactive-impulsive subtype).

Despite a likely heterogeneity in ADHD, we propose that neural gain modulation is a consistent impairment across many clinical subgroups. We can now hypothesise that ADHD subgroups may be better delineated by the specific profile of their neural gain impairments. One subgroup might primarily suffer from striatal DA impairment, expressing itself by more reward-related stochasticity and possibly striatal RPE impairments. Another subgroup might lack in frontal NA functioning, which might be expressed by impaired prefrontal signals and altered multiattribute processing. However, to be able to dissociate such subgroups, we need to develop better behavioural tasks and models, further advance computational neuroimaging, and develop neural models that are capable of dissociating different aspects of neural gain (see Outstanding Questions).

Can we dissociate different forms of neural gain impairment (e.g., DA versus NA; prefrontal versus striatal; α- versus β-adrenoceptor subtypes) behaviourally?

What are the unique features of NA and DA gain impairments behaviourally, algorithmically, and in neural loop models?

How can (computational) non-invasive neuroimaging contribute to dissociating different forms of neural gain impairment?

Can a neural gain-based classification of subgroups be predictive of pharmacological treatment efficiency? Can the understanding of the associated information processing inform psychotherapy?