Toward operationalizing deficient emotional self-regulation in newly referred adults with ADHD: A receiver operator characteristic curve analysis.

BACKGROUND: A growing body of research suggests that deficient emotional self-regulation (DESR) is common and morbid among attention-deficit/hyperactivity disorder (ADHD) patients. The main aim of the present study was to assess whether high and low levels of DESR in adult ADHD patients can be operationalized and whether they are clinically useful.
METHODS: A total of 441 newly referred 18- to 55-year-old adults of both sexes with Diagnostic and Statistical Manual of Mental Disorders: Fifth Edition (DSM-5) ADHD completed self-reported rating scales. We operationalized DESR using items from the Barkley Current Behavior Scale. We used receiver operator characteristic (ROC) curves to identify the optimal cut-off on the Barkley Emotional Dysregulation (ED) Scale to categorize patients as having high- versus low-level DESR and compared demographic and clinical characteristics between the groups.
RESULTS: We averaged the optimal Barkley ED Scale cut-points from the ROC curve analyses across all subscales and categorized ADHD patients as having high- (N = 191) or low-level (N = 250) DESR (total Barkley ED Scale score ≥8 or <8, respectively). Those with high-level DESR had significantly more severe symptoms of ADHD, executive dysfunction, autistic traits, levels of psychopathology, and worse quality of life compared with those with low-level DESR. There were no major differences in outcomes among medicated and unmedicated patients.
CONCLUSIONS: High levels of DESR are common in adults with ADHD and when present represent a burdensome source of added morbidity and disability worthy of further clinical and scientific attention.

Introduction

Symptoms of low frustration tolerance, impatience, and quickness to anger have long been associated with ADHD [1-3], and emotional symptoms deficits in emotional regulation have been included as associated features of ADHD in the DSM [4]. Yet, there has been limited research on the subject.

Barkley has argued that the emotional symptoms associated with ADHD are the result of a weak self-regulatory process in ADHD that leads to emotionally reactive behavior [2,5] and termed it deficient emotional self-regulation (DESR) to distinguish it from mood disorders. Using selected items from the Barkley Current Behavior Scale (CBS), we previously reported that 61% of adults with ADHD had DESR of greater severity than 95% of controls [6] and when present, it was associated with significant functional impairments. Although these data suggest that DESR is common and morbid at the group level, uncertainties remain on how to best operationalize DESR at the individual level.

The main aim of the present study was to assess whether high and low levels of DESR in adult ADHD patients can be operationalized. To this end, we analyzed data from a large sample of consecutively newly referred adults with ADHD assessed in multiple domains of functioning. We hypothesized that high levels of DESR would be common in adults with ADHD and that their presence would be associated with morbidity and dysfunction.

Methods

Sample

The sample consisted of 441 newly referred adults 18–55 years of age of both sexes with DSM-5 ADHD. There was no selection bias based on social class or insurance restrictions. We received institutional review board approval to review, analyze, and report anonymously on these subjects.

Assessment procedures

Patients completed a battery of rating scales before their initial evaluation. The demographic interview collected information on age, race, sex, socioeconomic status (SES), and history of head injury or trauma. Medication history collected information on current and past treatments for ADHD and other disorders. The Adult ADHD Self-Report Scale (ASRS) is an 18-item patient-rated questionnaire to determine how often ADHD symptoms occur [7,8]. The Behavior Rating Inventory of Executive Function—Adult version (BRIEF-A) is a 75-item patient-rated questionnaire to assess an adult’s cognitive, emotional, and behavioral functions within the past month [9]. Raw scores are calculated and used to generate t-scores for nine scales, two summary index scales, and one scale reflecting overall functioning. The Social Responsiveness Scale—Second edition (SRS-2) Adult form is a 65-item self-rated assessment used to measure the severity of autism spectrum symptoms [10]. Raw scores are calculated and used to generate t-scores for five subscales and one total scale. The adult self-report (ASR) is a 126-item self-rated assessment of adult behavior, social competence, and substance use [11]. Raw scores are calculated and used to generate t-scores for eight scales, two composite scales, and one total scale. The Barkley Emotional Dysregulation (ED) Scale is a subset of eight questions from the CBS designated by Barkley as measuring DESR [12,13] that asks subjects to describe their behavior in the past 6 months. The Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q) is a self-rated 16-item rating scale to assess enjoyment and satisfaction levels in various areas of daily life [14].

Statistical analysis

We computed inter-item correlations for all eight items of the Barkley ED Scale and Cronbach’s alpha for the entire Barkley ED Scale to determine the scale’s internal consistency.

We used receiver operator characteristic (ROC) curves to examine the ability of the Barkley ED Scale to identify those with and without clinical impairment on the ASRS, BRIEF, SRS, and ASR. Based on the information from the ROC curve analysis, we used the Liu approach [15] to calculate the optimal cut-point on the Barkley ED Scale to identify those with and without impairment on each rating scale and used conditional probabilities to examine the diagnostic utility of those optimal cut-points. We then averaged the optimal cut-points across all the rating scales and used it to categorize patients in our sample as having high- versus low-level DESR.

We compared demographic characteristics of those with high- versus low-level DESR using t-tests, Kruskal–Wallis rank sum tests, and Pearson’s chi-square tests. We analyzed clinical characteristics using linear, logistic, ordered logistic, or truncated Poisson regression models depending on the outcome. We included an interaction term in the model between DESR level (high versus low) and medication status (medication versus no medication) to examine the effect medication had on the association between DESR level and the outcome of interest. If the interaction term was not significant, we collapsed across medication status; if it was significant, we examined the outcome within the strata of medication status. Tests were two-tailed and performed at the 0.05 alpha level using Stata (version 15.1) [16].

Results

Psychometric analyses of the Barkley ED Scale

We calculated Cronbach’s alpha as 0.89 for the Barkley ED Scale, which indicates a high level of internal consistency. The alphas with one item deleted at a time ranged from 0.87 to 0.91.

ROC curve analysis

The ROC curve and conditional probability analyses for each rating scale are presented in Table 1. Of all the clinical scales used, the Barkley ED Scale best identified clinical impairment on the ASR aggressive behavior subscale (area under the curve (AUC) = 0.94). Of the 12 subscales examined, 4 had an optimal Barkley ED Scale cut-point of 6, 4 had an optimal cut-point of 8, 3 had an optimal cut-point of 9, and 1 had an optimal cut-point of 14 (Table 1). Sensitivity ranged from 61% (ASRS hyperactivity) to 88% (ASR Externalizing Problems) and specificity ranged from 56% (ASR Attention Problems) to 91% (ASR Aggressive Behavior). Based on the ROC curve analyses, we averaged the optimal Barkley ED Scale cut-points across all subscales and categorized patients as having high-level DESR (N = 191) and low-level DESR (N = 250), as defined by having a Barkley ED Scale score of ≥8 or <8, respectively. Subsequent comparisons were made between subjects with low versus high Barkley ED scores.

Table 1.

ROC curve (AUC) and conditional probability analyses to identify the optimal cut-off point of the Barkley Emotional Dysregulation (ED) Scale using clinical scores on scales measuring ADHD, executive function deficits (EFDs), autism spectrum disorder (ASD), and psychopathology (ASR)

Rating scale	RS clinical scores	AUC statistic	Barkley ED optimal cut-point	Sensitivity	Specificity	Positive predictive value	Negative predictive value	Correctly classified
ADHD symptomatology
ASRS total	≥24 in either subdomain	0.74	6	73%	64%	70%	68%	69%
ASRS inattention	≥24	0.72	6	72%	62%	66%	68%	67%
ASRS hyperactivity	≥24	0.75	9	61%	76%	42%	87%	72%
Executive function deficits
BRIEF-GEC	≥65	0.81	6	80%	72%	77%	75%	76%
ASD symptomatology
SRS total	≥60	0.78	9	63%	79%	54%	85%	74%
SRS total	≥66	0.75	9	66%	74%	33%	92%	73%
Psychopathology (ASR)
ASR total	≥64	0.84	8	75%	77%	68%	83%	76%
ASR externalizing	≥64	0.87	8	88%	70%	48%	95%	75%
ASR internalizing	≥64	0.79	8	71%	74%	64%	80%	73%
ASR attention problems	≥70	0.69	6	77%	56%	49%	81%	63%
ASR aggressive behavior	≥70	0.94	14	85%	91%	36%	99%	90%
ASR anxious/depressed	≥70	0.79	8	80%	67%	41%	92%	70%
Average optimal Barkley ED Scale cut-point = 8

Demographic characteristics

As shown in Table 2, there were no significant differences in age, SES, sex, or race between those with high- and low-level DESR. Fifty percentage (N = 214) of patients reported currently taking psychiatric medications. The most commonly reported medication types were stimulants (64%), followed by antidepressants (42%), antianxiety (18%), antipsychotic (7%), nonstimulants for ADHD (6%), and mood stabilizers (4%). Sixty-nine percentage of patients reported currently taking only one medication type while 23% reported currently being on 2 different medication types and 8% reported currently being on ≥3 different medication types. When we compared the two DESR groups, there was no significant difference in the rate of those currently taking stimulant medications (Table 2). However, there was a significant difference in the rate of those taking other psychiatric medications, with more patients currently taking other psychiatric medications in the high-level DESR group.

Table 2.

Demographic and medication characteristics of subjects with high-level deficient emotional self-regulation (DESR; total Barkley ED score ≥ 8) and low-level (total Barkley ED score < 8)

	Low-level DESR (N = 250)	High-level DESR (N = 191)
	Mean ± SD	Mean ± SD	Test statistic	p-Value
Age	34.5 ± 13.5	32.5 ± 11.4	t 439 = 1.66	0.10
Socioeconomic status a	1.8 ± 0.9	1.8 ± 0.9	z = 0.21	0.84
	N (%)	N (%)
Gender (% male)	105 (42)	90 (47)	χ 2 = 1.15	0.28
Race (% Caucasian) a	202 (82)	139 (76)	χ 2 = 2.73	0.10
Current psychiatric medications a
Stimulants	80 (33)	57 (31)	χ 2 = 0.27	0.69
Other psychiatric medications	53 (22)	68 (37)	χ 2 = 11.03	<0.001

Smaller sample sizes. Socioeconomic status: low-level: N = 181, high-level: N = 128; race: low-level: N = 245, high-level: N = 183; current psychiatric medications: low-level: N = 241, high-level: N = 185.

DESR and ADHD symptoms

The interaction between DESR level and medication status was not significant for the ASRS subdomain and total scores (all p > 0.05), and was removed from the models. As shown in Figure 1A, patients with high-level DESR had significantly more impaired scores in both the ASRS inattention and hyperactivity domains and on the total scale score (all p < 0.001). Upon examining the individual ASRS symptoms, the interaction between DESR level and medication status was not significant for all hyperactive symptoms (all p > 0.05) and all but one inattentive symptom (all p > 0.05 except “delay starting tasks that require a lot of thought,” p = 0.04). We removed the interaction from all models except the significant one, in which case, we performed the analysis stratified by medication status. For “delaying starting tasks that require a lot of thought,” medicated patients with high-level DESR experienced the symptom significantly more often than medicated patients with low-level DESR (p < 0.001). Conversely, there was no significant difference between the two DESR groups among unmedicated patients on this item (p = 0.12; Table S1). As shown in Figure 1B,C, for the rest of the inattentive items and all of the hyperactive items, patients with high-level DESR experienced the symptoms significantly more often than patients with low-level DESR (all p < 0.001).

Figure 1.

Adult ADHD Self-Report Scale scores of subjects with high (total Barkley ED score ≥ 8) and low (total Barkley ED score < 8) DESR scores. (A) Subdomain and total scores; (B) inattentive symptom scores; and (C) hyperactive symptom scores. Patients with high-level DESR were significantly more impaired than those with low-level DESR. *Significant interaction between DESR level and medication status ( p = 0.04). Stratified analyses revealed significantly higher scores in those with high-level DESR among medicated patients but not unmedicated patients.

DESR and executive functioning

The interaction between DESR level and medication status was significant for only the BRIEF initiate subscale as both a continuous (p = 0.01) and dichotomized (p = 0.04) outcome. Thus, we stratified by medication status for analysis of this subscale and removed the interaction for all other subscales. Patients with high-level DESR demonstrated worse executive functioning as measured by the BRIEF-A. Those with high-level DESR had significantly more impaired mean scores (all p < 0.001; Figure 2A) and higher rates of patients with scores in the clinical range (t-score ≥ 65) versus nonclinical range (t-score < 65; all p < 0.001; Figure 2B). This remained true when we stratified by medication status for the initiate subscale. Both medicated and unmedicated patients with high-level DESR were significantly more impaired than those with low-level DESR; however, the difference between those with high- and low-level DESR was greater among the medicated (p < 0.001) than unmedicated (p = 0.01; Table S1).

Figure 2.

Behavior Rating Inventory of Executive Function—Adult version (BRIEF-A) and Social Responsiveness Scale—Second edition (SRS-2) adult self-report scores of subjects with high (total Barkley ED score ≥ 8) and low (total Barkley ED score < 8) DESR scores. (A) BRIEF-A subscales; (B) subjects with T-scores in the clinical range on the BRIEF-A subscales; (C) SRS-2 subscales; and (D) subjects with T-scores in the clinical range on the SRS-2. Patients with high-level DESR were significantly more impaired than those with low-level DESR. *Significant interaction between DESR level and medication status (both p-values <0.05). Stratified analyses revealed significantly higher T-scores and a greater percentage of scores in the clinical range in those with high-level DESR among both medicated and unmedicated patients.

In examining the SRS, there were no significant interactions between DESR level and medication status (all p > 0.05) and the interaction and medication status variables were removed from the models. Patients with high-level DESR were more socially impaired, with significantly higher scores on all SRS subdomains and the total score (all p < 0.001; Figure 2C). When the SRSs were dichotomized into clinical range (t-score ≥ 60) versus nonclinical range (t-score < 60), those with high-level DESR had significantly higher rates of patients with scores in the clinical range (all p < 0.001; Figure 2D).

DESR and psychopathology

The interaction between DESR level and medication status was significant for the ASR attention problems and intrusive subscales as continuous outcomes and the ASR somatic complaints subscale as a dichotomous outcome (all p < 0.05). Thus, we stratified by medication status for analysis of those three subscales and removed the interaction for all other subscales and composite scales. Those with high-level DESR had significantly more impaired scores on six clinical scales, all composite scales, and all adaptive functioning scales (all p < 0.001; Figure 3A,C). Among medicated and unmedicated patients, those with high-level DESR had more impaired scores on the attention problems and intrusive subscales, with the difference between high- and low-level DESR greater for medicated (both p < 0.001) compared with unmedicated (both p < 0.001; Table S1). When we dichotomized the ASR scale t-scores into clinical range (clinical scales: t-scores ≥70, composite scales: t-scores ≥64, adaptive functioning scales: t-scores ≤30) versus nonclinical range (clinical scales: t-scores <70, composite scales: t-scores <64, adaptive functioning scales: t-scores >30), a significantly greater proportion of patients with high-level DESR had scores in the clinical range on the seven clinical scales analyzed unstratified (all p < 0.001), all composite scales (all p < 0.001) and on five of the six adaptive functioning scales (all p < 0.05 except spouse/partner scale, p = 0.05; Figure 3B,D). For somatic complaints, there was a significant difference in the proportion of patients with scores in the clinical range between the two DESR groups among the medicated patients (p = 0.003) but not among the unmedicated patients (p = 0.37; Table S1).

Figure 3.

Adult self-report (ASR) scores in subjects with high (total Barkley ED score ≥ 8) and low (total Barkley ED score < 8) DESR scores. (A) ASR Clinical and Composite Scales; (B) subjects with T-scores in the Clinical Range on the ASR Clinical and Composite Subscales; (C) ASR Adaptive Functioning Scales; and (D) subjects with T-scores in the clinical range. †Sample sizes vary. Spouse/partner: low-level: N = 115, high-level: N = 79; family: low-level: N = 245, high-level: N = 188; job: low-level: N = 200, high-level: N = 150; education: low-level: N = 76, high-level: N = 68. Patients with high-level DESR were significantly more impaired than those with low-level DESR. *Significant interaction between DESR level and medication status (all three p-values <0.05). Stratified analyses revealed significantly higher T-scores on the attention problems and intrusive subscales in those with high-level DESR among both medicated and unmedicated patients. They also revealed a significantly greater percentage of scores in the clinical range on the somatic complaints subscale in those with high-level DESR among the medicated patients but not the unmedicated patients.

DESR and quality of life

In examining the Q-LES-Q total score and individual items, there were no significant interactions between DESR level and medication status (p > 0.05) and they were removed from the models. Overall quality of life was significantly lower in patients with high-level DESR (p < 0.001; Figure 4A). When we examined the individual items of the Q-LES-Q, we found that patients with high-level DESR rated their degree satisfaction significantly lower than those with low-level DESR on 13 of the 14 items (all p < 0.05 except “vision in terms of ability to do work or hobbies,” p = 0.08; Figure 4B).

Figure 4.

Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q) scores of subjects with high (total Barkley ED score ≥ 8) and low (total Barkley ED score < 8) DESR scores. (A) Q-LES-Q mean total score and (B) Q-LES-Q individual items. Patients with high-level DESR were significantly more impaired than those with low-level DESR.

There were no significant interactions between DESR level and medication status when examining rates of employment and completion of 4 years of college (both p > 0.05), but there was a significant interaction when examining rates of learning disabilities (p < 0.05). We found no significant differences between the two DESR groups when we examined the rates of patients who were currently employed (high-level: 73%, N = 135/186 versus low-level: 80%, N = 195/246; p = 0.05) or who completed at least 4 years of college (high-level: 73%, N = 130/178 versus low-level: 73%, N = 175/241; p = 0.92). Among the medicated patients, there was no significant difference in the rates of those with learning disabilities between two DESR groups (p = 0.44). However, there was a significant difference among the unmedicated, with higher rates of learning disabilities in those with high-level DESR (p = 0.03; Table S1).

Impact of medication on DESR

Given that half of the sample was taking psychiatric medications at the time of the referral, we examined whether medication treatment impacted DESR level. To this end, we compared rates of high-level DESR among patients taking different types of psychiatric medications and patients who were unmedicated. At the broadest level, there were neither statistically nor clinically meaningful differences in the rate of high-level DESR among patients taking any psychiatric medication (N = 214) compared with those who were unmedicated (N = 212; any medication: 46% versus unmedicated: 41%, p = 0.32). Next, we compared patients taking stimulant medication (regardless of if they were taking other medication types; N = 137) to unmedicated patients and again failed to find statistically or clinically meaningful differences in the rates of high-level DESR between the groups (stimulant medication: 42% versus unmedicated: 41%, p = 0.92). Lastly, we compared patients who were taking stimulant medication only (N = 93), antidepressant/antianxiety medication only (N = 71), a combination of stimulant and antidepressant/antianxiety medications (N = 41), and those who were unmedicated. We did not include the other medication types because the groups were too small. The omnibus test revealed that there were significant differences among the four groups (p = 0.004). As shown in Figure 5, those taking stimulant medication only had a significantly lower rate of high-level DESR compared with those taking antidepressant/antianxiety medications only (p = 0.01) and those taking a combination of stimulant and antidepressant/antianxiety medications (p = 0.001). Additionally, those taking a combination of stimulant and antidepressant/antianxiety medications also had a significantly higher rates of high-level DESR compared with those who were unmedicated (p = 0.01). No other pairwise comparisons were statistically significant (all p > 0.05).

Figure 5.

Rates of high DESR scores (total Barkley ED score ≥ 8) by treatment. Abbreviation: NS, not significant.

Discussion

Relying on ROC curves and conditional probability analyses, we operationalized high and low levels of DESR based on scores on selected items from the Barkley Scale in a sample of consecutively referred adults with DSM-5 ADHD. These analyses showed that 43% of adult ADHD patients had high levels of DESR and those affected had significantly more severe symptoms of ADHD, executive dysfunction, autistic traits, levels of psychopathology, and worse quality of life when compared with ADHD patients with low levels of DESR. These findings indicate that high levels of DESR are common in adults with ADHD and represent a source of added morbidity and dysfunction.

Our findings are consistent with previously reported results in a separate community sample that showed that 61% of adults with ADHD had DESR symptoms of greater severity than 95% of control subjects [6]. They are also consistent with data from a family study [17] and two clinical trial studies [18,19] showing that DESR is highly prevalent among ADHD adults and predict persistence of symptoms. As we argued previously [17], these findings are consistent with the hypothesis that DESR is an important comorbidity within ADHD.

Our finding showing that ADHD adults with high levels of DESR have significantly more impaired ADHD symptoms and higher level of psychopathology than those with low levels of DESR indicates that the clinical picture is more severe in ADHD adults with DESR.

The finding that high levels of DESR is associated with higher levels of comorbid psychopathology and executive dysfunction are consistent with results from previous studies documenting similar findings [6,17,18,19]. The finding that high levels of DESR were associated with higher levels of autism spectrum symptoms are consistent with previously reported findings in a pediatric sample documenting high prevalence and morbidity of autistic traits in youth with ADHD [20].

Our finding that high levels of DESR were significantly associated with lower quality of life expand our previous finding showing that DESR was associated with significantly lower quality of life, significantly worse social adjustment, reduced marital status, and higher risk for traffic accidents and arrests in adults with ADHD [6].

The high Cronbach’s alpha of 0.89 for the DESR items derived from the Barkley Scale are consistent with those reported previously [6], indicating a high level of internal consistency for DESR items used to define DESR in this study.

Our results failing to identify meaningful differences in the rates of high levels of DESR among patients taking stimulant medication are consistent with results from recent meta-analyses [21] and Moukhtarian et al. [22] showing that stimulants were less effective in the treatment of ED than on core symptoms of ADHD. Although Asherson et al. [23] suggested that atomoxetine was associated with improvements in emotional control in adults with ADHD, our findings showed that antidepressant/antianxiety medications had a worsening effect on DESR. More work is needed to further examine what treatments can best target DESR in ADHD.

Our findings need to be viewed in light of some methodological limitations. Our sample was primarily Caucasian and, thus, may not generalize to other ethnic groups. We assessed DESR using selected items from the Barkley CBS [12,13] and do not know whether similar results would be obtained using different instrumentation to assess DESR. Although we cannot identify from our data whether DESR should be considered as co-occurring with ADHD or a result of ADHD, the fact that it is not universally associated with ADHD supports the hypothesis that DESR should be best conceptualized as a comorbidity of ADHD. Future research comparing comorbid and noncomorbid ADHD subjects could further clarify this issue. Although our analyses demonstrate internal consistency of the items that we chose as a measure of DESR and suggest that DESR as identified by these items has external validity because they are associated with greater impairment on measures of functioning, further study could clarify the validity of the eight-item scale we utilized to identify DESR.

Despite these considerations, we identified a robust association between DESR and ADHD in a large sample of clinically referred adults with ADHD that correlated with impaired quality of life and a wide range of functional impairments indicating that the clinical picture is more severe in ADHD adults with DESR. More work is needed to help identify appropriate interventions for DESR in ADHD including psychosocial treatments.