Concurrent Validity of the Behavior Rating Inventory of Executive Function in Children With Attention Deficit Hyperactivity Disorder.

BACKGROUND: Attention deficit hyperactivity disorder (ADHD) is a common psychiatric disorder in which impairment of executive functions plays an important role.
OBJECTIVES: The main objective of this study was to assess the validity of the Behavior Rating Inventory of Executive Function (BRIEF) in children with ADHD. PATIENTS AND METHODS: Thirty children, aged 7-12 years, attending the child and adolescent clinic of Roozbeh hospital and diagnosed with ADHD according to interview with a child and adolescent psychiatrist, formed our ADHD group. In contrast, thirty participants of the control group were selected from 7 to 12 year-old students according to Conners' Teacher/Parent Rating Scale and did not have ADHD. The kiddie schedule for affective disorders and schizophrenia-present and lifetime version-Persian version was also completed for all children to rule out other psychiatric disorders. After oral consent, parents of 60 children (ADHD = 30, control = 30), completed three questionnaires of ADHD-Rating Scale-IV, Conners' Parent Rating Scale-Revised: Short Version and BRIEF.
RESULTS: Children in ADHD group got higher scores than those in the control group in all subscales and indices of BRIEF (P < 0.001). There were also good correlations between subscales and indices of BRIEF and the two other rating scales (P < 0.001).
CONCLUSION: BRIEF could be used as a valid tool to assess behavioral aspects of executive functions, especially to discriminate children with ADHD and normal ones.

1. Background

Executive functions (EFs) are referred to a group of cognitive abilities responsible for goal-directed activities. They inhibit inappropriate behaviors and help us to respond to new situations by shifting our mind quickly and easily from one subject to another. These functions are considered as important factors in cognitive development of children. In fact, they are the main part of all social and personal activities (1). Observing executive dysfunction in patients with brain damage has suggested the prefrontal lobe as the central part of the brain in management of executive functioning (1). Besides, presence of executive dysfunction in many psychiatric disorders such as attention deficit hyperactivity disorder (ADHD) has become an important field of research in child and adolescent psychiatry (2).

ADHD is a common psychiatric disorder characterized by hyperactivity, impulsivity and inattentiveness, inappropriate to child’s age. Children with ADHD can be easily distracted by any stimulus and cannot sustain their attention for a long period on a subject (3). ADHD is not a simple deficit in attention, but a developmental impairment in a broad spectrum of EFs, which needs to be assessed carefully (4). For many years, assessment of EF in children was based on psychiatric interview and behavioral scales and questionnaires, which were not designed specifically for children and could not capture all aspects of executive abilities (2, 5).

Although many neuropsychological tests were developed to assess EF using direct measures of attention, mental flexibility and inhibitory control that do not rely on parent report, they are usually expensive, time-consuming and need experienced professional work. Therefore Gioia et al. (6) designed the Behavior Rating Inventory of Executive Function (BRIEF) to evaluate EF in 5-18 year-old children. Through its behavioral aspects, BRIEF makes it easy to assess existence and quality of EFs by accomplishment of a pen and paper questionnaire. From the early appearance of BRIEF, many researchers tried to compare its reliability, validity and correlation with other standardized instruments in ADHD and other psychiatric impairments such as Tourette syndrome and autism spectrum disorder (4, 7, 8).

2. Objectives

According to their results, diagnostic groups got significant higher scores in all subscales of BRIEF than the normal group. This is the first study in Iran to assess the validity of BRIEF in a group of children with ADHD compared with normal children.

3. Patients and Methods

3.1. Participants

Thirty children aged 7 to 12 years referred to the child and adolescent psychiatry clinic at Roozbeh hospital in Tehran were recruited using the simple convenient sampling method. All the participants were diagnosed as having ADHD-combined type was diagnosed in all children by a board certified child and adolescent psychiatrist based on DSM-IV-TR criteria. They did not have any history of major psychiatric disorders such as oppositional defiant or conduct disorder. All of them were drug naïve as well. The control group included 30 volunteers selected randomly from students of two elementary schools in Tehran, a female school and a male school. The controls matched for their ages, genders and IQs (based on the scores of the Raven IQ test) with the ADHD group. They did not have any history of psychiatric disorders or referral to mental health services. According to the teacher and parents’ Conners’ rating scale, they did not have the diagnosis of ADHD.

3.2. Procedure

The study was explained for parents and written consents were obtained. It was also conducted with full approval of the Institutional Review Board (IRB). The Kiddie Schedule for Affective Disorders and Schizophrenia, Present and Lifetime version, Persian version (KSADS-PL-P), was performed for all participants by a fellow of child and adolescent psychiatry to evaluate other psychiatric disorders. The Raven Progressive Matrices test was performed to assess IQ in the two groups. To eliminate any bias, IQ tests were performed between 2-4 pm. Then parents filled three questionnaires for their children; the ADHD rating scale (ADHD-RS), the Conners’ parent rating scale revised-short version (CPRS) and the BRIEF Parent form.

3.3. Rating Scales

3.3.1. BRIEF

The BRIEF (6) is an 86-item questionnaire to assess the EFs in girls and boys of 5-18 years old. There are two rating forms; teacher and parent questionnaire. Parents with at least a fifth grade education can answer these questions easily. BRIEF assesses eight non-overlapping subscales, inhibition, shifting and emotional control that result in an index; Behavioral Regulation Index (BRI), and initiation, working memory, monitoring, planning/organizing and organization of materials which are combined to make Meta Cognition Index (MCI), and a total score; “Global Executive Composite” made by these two indices. The higher the score is, the more dysfunction exists. BRIEF is developed for a wide spectrum of children with developmental and acquired neurological disorders such as learning disabilities, ADHD, traumatic brain injury, low birth weight, Tourette syndrome and pervasive developmental disorder/autism. It also contains two validity subscales of negativity and inconsistency to detect bias associated with rating scales. BRIEF has gotten both convergent and divergent validity, as well as high internal consistency (α = 0.8-0.98) and the test-retest reliability which was 0.82 and 0.88 for parent and teacher forms, respectively. There were also moderate correlations between teacher and parent ratings (r = 0.32-0.34) (6).

3.3.2. ADHD Rating Scale IV (ADHD-RS-IV)-Home Version

The ADHD-RS-IV (9) is an 18-item questionnaire, developed specifically to screen ADHD children and adolescents between 5 and 18 years old. There are two questionnaires; home and school versions, which reveal child’s behavior over the past six months at home and school. It consists of two subscales; inattention (9 items) and hyperactivity/impulsivity (9 items). Thus, we would have three raw scores at the end; 1) inattention score, 2) hyperactivity/impulsivity score and a total score by adding them together.

The scale has acceptable levels of inter-rater reliability, test-retest reliability, internal consistency, convergent validity, discriminant validity and responsiveness (10). The psychometric properties of the Farsi version of ADHD-RS-IV has been reported by Ghanizadeh previously (11).

3.3.3. Conners’ Parent Rating Scale-Revised: Short Version (CPRS-RS)

The Revised Conners’ Rating Scale (12) has three types; parent, teacher and self-report. Each has a short and a long version. CPRS-RS contains 27 items, introducing four subscales; 1) oppositional subscale, 2) inattention/cognitive Problem, 3) hyperactive/impulsive and 4) ADHD Index. The items are based on DSM-IV criteria of ADHD, so it can be used to confirm the diagnosis of ADHD.

Psychometric properties of the revised scale appear adequate as demonstrated by good internal reliability coefficients, high test-retest reliability and effective discriminative power (13). Tehrani Doost et al. found that psychometric properties of the CPRS were good in an Iranian clinical population. They suggested that CPRS was a useful instrument to discriminate children with ADHD from typically developing individuals (14).

3.3.4. K-SADS-PL

The K-SADS-PL (15) is a semi-structured interview to assess psychiatric diagnosis in children and adolescents aged 6-18 years. It assesses present and lifetime status of psychiatric disorders as well as the severity of symptoms. The K-SADS-PL is introduced from K-SADS-P according to the fourth edition of DSM and is capable of generating 32 DSM-III-R and DSM-IV Axis I child and adolescent psychiatric disorders. Diagnoses are made as definite, probable (greater than or equal to 75% of symptoms criteria met) or not present. The K-SADS-PL has excellent inter-rater reliability and results comparable to semi-structured and fully structured child diagnostic interviews. The test-retest reliability kappa coefficients are in the excellent range for present and lifetime diagnosis of major psychiatric disorders (15). The Persian version of the K-SADS-PL (K-SADS-PL-P) has been validated by Shahrivar et al. (16) in Iran. Based on their study, specificities were more than 81% for all disorders and sensitivities for most major diagnosis were between 75% and 100%. The Kappa agreements for most diagnoses were higher than 0.4 and the test-retest reliabilities were between 0.38 and 0.87.

3.3.5. Raven’s Standard Progressive Matrices

The Raven’s Standard Progressive Matrices (SPM) is a standard measure for non-verbal intelligence and perceptual reasoning. It was used as the intelligence test to match the IQs of the two groups (17). SPM has been validated and standardized for 5-18-year old children and adolescents in Iran (18).

3.4. Statistical Analysis

Statistical analysis was performed using SPSS 15 (Softonic International, S.A.). The prevalence of subscales and indices were calculated by descriptive statistics. T-independent test was performed to compare the differences between two groups of variables. The Pearson correlation test was performed to find the correlation between the variables of the questionnaires.

4. Results

As can be seen in Table 1, thirty children in each group were enrolled (6 girls and 24 boys). The mean ages were 8.96 years (SD = 1.5) in ADHD group and 9.03 (SD = 1.4) in control group. The IQ means were 110.86 (SD = 10.77) and 115.33 (SD = 7.07) in ADHD and control groups, respectively. There were no significant differences between the two groups regarding their age, gender and IQ.

Table 1.

Demographic Characteristics of the Attention Deficit Hyperactivity Disorder and the Control Group [a,b]

Demographics	ADHD (n = 30)	Control (n = 30)	Significancy
Gender			1.000
Female	6	6
Male	24	24
Age, y	8.96 ± 1.55	9.03 ± 1.41	0.863
IQ	110.86 ± 10.77	115.33 ± 7.07	0.064

a Data are presented as mean or mean ± SD.

b Abbreviation: ADHD, attention deficit hyperactivity disorder.

4.1. Group Comparison on BRIEF Subscales and Indices

The means of all eight subscales and two indices of BRIEF are summarized in Table 2. It shows significant differences between ADHD and control groups for all BRIEF subscales (P < 0.001).

Table 2.

Comparing Attention Deficit Hyperactivity Disorder and Control Groups Based on the Behavior Rating Inventory of Executive Function Scales [a,b]

BRIEF Subscales and Indexes	ADHD Group	Control Group	P Value
Inhibit	22.90 ± 4.85	12.20 ± 2.52	0.001
Shift	15.17 ± 3.72	10.47 ± 2.11	0.001
Emotional control	22.27 ± 4.51	14.43 ± 3.60	0.001
Initiate	17.07 ± 3.21	11.57 ± 2.77	0.001
Working memory	22.37 ± 4.08	13.60 ± 2.90	0.001
Plan/organize	26.70 ± 4.36	16.80 ± 3.19	0.001
Organization of materials	14.00 ± 3.22	9.27 ± 2.87	0.001
Monitor	19.23 ± 3.05	12.43 ± 2.50	0.001
BRI	59.50 ± 11.20	37.10 ± 6.62	0.001
MCI	98.40 ± 14.37	63.67 ± 11.68	0.001
GEC	157.90 ± 23.14	100.77 ± 17.02	0.001

a Data are presented as or mean ± SD.

b Abbreviations: ADHD, attention deficit hyperactivity disorder; BRI, behavioral regulation index; BRIEF, behavior rating inventory of executive function; GEC, global executive composite; MCI, meta cognition index.

4.2. Group Comparison on ADHD-RS

As shown in Table 3, in both inattention and hyperactivity subscales, the differences between the two groups were significant.

Table 3.

Comparing Attention Deficit Hyperactivity Disorder and Control Groups Based on the Attention Deficit Hyperactivity Disorder-Rating Scale Results [a,b]

ADHD-RS	ADHD	Control	P Value
Inattention	16.17 ± 5.33	3.13 ± 3.45	0.001
Hyperactive	17.90 ± 5.76	3.30 ± 3.31	0.001
Total	34.07 ± 10.44	6.43 ± 5.80	0.001

a Data are presented as or mean ± SD.

b Abbreviations: ADHD, attention deficit hyperactivity disorder; ADHD-RS, attention deficit hyperactivity disorder-rating scale.

4.3. Group Comparison on CPRS-RS

Scores of all four subscales of CPRS-RS are listed in Table 4. Just like the two other questionnaires, the differences were significant between the two groups.

Table 4.

Comparing Attention Deficit Hyperactivity Disorder and Control Groups Based on the Conners’s Parent Rating Scale-Revised: Short (CPRS-RS) [a,b]

CPRS-RS Subscales	ADHD		Control		P Value
	Value	T Score	Value	T Score
Oppositional	10.50 ± 4.42	69.06	2.87 ± 2.22	48.93	0.001
Inattention	11.83 ± 4.12	69.06	1.93 ± 2.39	47.48	0.001
Hyperactive-Impulsive	11.00 ± 4.50	74.66	1.87 ± 1.94	49.06	0.001
ADHD Index	23.87 ± 6.87	69.86	5.13 ± 4.08	46.31	0.001

a Data are presented as or mean ± SD.

b Abbreviations: ADHD, attention deficit hyperactivity disorder; CPRS-RS, Conners’ parent rating scale revised-short version.

4.4. Correlations Between BRIEF and ADHD-RS-IV

Correlations between BRIEF and ADHD-RS are presented in Table 5. In ADHD group, most BRIEF variables were significantly correlated with ADHD-RS variables (P < 0.05). In both groups MCI was much more correlated with ADHD-RS Inattentive symptoms (r = 0.723 in ADHD group, r = 0.739 in control group) than with hyperactivity symptoms (r = 0.560 in ADHD group, r = 0.259 in control group). On the contrary, BRI was strongly correlated with ADHD-RS hyperactivity symptoms (r = 0.591 in ADHD group, r = 0.549 in control group) than with Inattentive symptoms (r = 0.437 in ADHD group, r = 0.548 in control group).

Table 5.

Correlations Between Behavior Rating Inventory of Executive Function scales and the Attention Deficit Hyperactivity Disorder-Rating Scale Subscales [a]

BRIEF Subscales and Indexes	ADHD Rating Scale IV
	Inattention		Hyperactivity		Total
	ADHD	Control	ADHD	Control	ADHD	Control
Inhibit	0.589 [b]	0.503 [c]	0.714 [b]	0.468 [c]	0.695 [b]	0.566 [c]
Shift	0.398 [c]	0.374 [c]	0.473 [c]	0.151	0.464 [c]	0.308
Emotional control	0.511 [c]	0.457 [c]	0.608 [b]	0.621 [b]	0.596 [b]	0.626 [b]
Initiate	0.462 [c]	0.547 [c]	0.252	0.283	0.375 [c]	0.487 [c]
Working memory	0.674 [b]	0.624 [b]	0.447 [c]	0.163	0.591 [b]	0.464 [c]
Plan/organize	0.644 [b]	0.586 [b]	0.492 [c]	0.119	0.600 [b]	0.416 [c]
Organization of materials	0.587 [b]	0.681 [b]	0.530 [c]	0.062	0.592 [b]	0.440 [c]
Monitor	0.397 [c]	0.636 [b]	0.244	0.486 [c]	0.337	0.655 [b]
BRI	0.437 [c]	0.548 [c]	0.591 [b]	0.549 [c]	0.549 [c]	0.639 [b]
MCI	0.723 [b]	0.739 [b]	0.560 [c]	0.259	0.678 [b]	0.587 [c]
GEC	0.660 [b]	0.717 [b]	0.633 [b]	0.387 [c]	0.687 [b]	0.647 [b]

a Abbreviations: ADHD, attention deficit hyperactivity disorder; BRI, behavioral regulation index; BRIEF, behavior rating inventory of executive function; GEC: global executive composite; MCI: meta cognition index.

b P < 0.001.

c P < 0.05.

4.5. Correlations Between BRIEF and CPRS-RS

Correlations between BRIEF and CPRS-RS are shown in Table 6. In both groups, most BRIEF variables were correlated with CPRS variables significantly. As can be seen, similar to the pattern of Table 3, the BRIEF MCI was more correlated with CPRS-RS inattentive symptoms (r = 0.621 in ADHD group, r = 0.730 in control group) than hyperactivity symptoms (r = 0.560 in ADHD group, r = -0.021 in control group). Considering BRI, although it was strongly correlated with CPRS-R: S Oppositional symptoms (r = 0.778 in ADHD group, r = 0.523 in control group), there was a significant correlation between BRI and hyperactivity symptoms (r = 0.585 in ADHD group, r = 0.387 in control group).

Table 6.

Correlations Between Behavior Rating Inventory of Executive Function Scales and the Conners’ Parent Rating Scale Revised-Short Version Subscales and Index [a]

BRIEF Subscales and Indexes	Conners’ Parent Rating Scale
	Oppositional		Inattention		Hyperactivity		ADHD Index
	ADHD	Control	ADHD	Control	ADHD	Control	ADHD	Control
Inhibit	0.514 [b]	0.441 [b]	0.340	0.344	0.569 [b]	0.289	0.520 [b]	0.291
Shift	0.721 [c]	0.121	0.381 [b]	0.423 [b]	0.551 [b]	0.031	0.464 [b]	0.509 [b]
Emotional control	0.889 [c]	0.608 [c]	0.382 [b]	0.196	0.597 [c]	0.512 [b]	0.554 [b]	0.435 [b]
Initiate	0.351	0.386 [b]	0.570 [b]	0.473 [b]	0.210	0.081	0.585 [c]	0.399 [b]
Working memory	0.427 [b]	0.020	0.730 [c]	0.745 [c]	0.396 [b]	-0.066	0.716 [c]	0.723 [c]
Plan/organize	0.551 [b]	0.190	0.740 [c]	0.758 [c]	0.502 [b]	-0.162	0.652 [c]	0.536 [b]
Organization of materials	0.510 [b]	0.160	0.140	0.464 [b]	0.534 [b]	-0.128	0.507 [b]	0.471 [b]
Monitor	0.279	0.260	0.388 [b]	0.546 [b]	0.260	0.236	0.470 [b]	0.645 [c]
BRI	0.778 [c]	0.523 [b]	0.331	0.367 [b]	0.585 [c]	0.387 [b]	0.490 [b]	0.503 [b]
MCI	0.576 [c]	0.241	0.621 [c]	0.730 [c]	0.560 [b]	-0.021	0.701 [c]	0.669 [c]
GEC	0.717 [c]	0.364 [b]	0.546 [b]	0.643 [c]	0.631 [c]	0.132	0.673 [c]	0.652 [c]

a Abbreviations: ADHD, attention deficit hyperactivity disorder; BRI, behavioral regulation index; BRIEF, behavior rating inventory of executive function; GEC: global executive composite; MCI: meta cognition index.

b P < 0.05.

c P < 0.001.

4.6. Age Effects

According to linear regression, BRIEF scores were not influenced by children’s age in any group (P > 0.05).

5. Discussion

This study was designed to evaluate the validity of parent rating of BRIEF in children with ADHD. Therefore, as the first step, we intentionally eliminated referees who had another disorder in addition to ADHD, since EDF can be observed in many psychiatric disorders. Our findings strongly indicated that BRIEF could differentiate children with ADHD from normal group; executive dysfunction of these children could be reflected on BRIEF questionnaire.

The parent ratings of BRIEF were significantly correlated with two other parents’ ratings, which are used to evaluate ADHD symptoms in children and adolescents. This finding supports the concurrent validity of the parent form of BRIEF, which has been reported by others (4, 19, 20). The patterns of correlation between BRIEF subscales and indices with CPRS and ADHD-RS suggest discriminative validity of the BRIEF rating to compare different types of ADHD. Working memory is theoretically based on attentional functioning, so the BRIEF working memory subscale is expected to be correlated with attention scales of other rating scales. Likewise, the BRIEF inhibit subscale should correlate with measures of externalizing behaviors. The results of our study showed that the BRIEF MCI, especially working memory has a high degree of correlation with ADHD-RS and CPRS inattention scales. Conversely, the BRIEF BRI particularly inhibition is strongly correlated with hyperactivity indices of the two other parent ratings. These results are consistent with the findings of multiple studies (4, 8, 21), although Sullivan and Riccio (22) suggested that the ability of BRIEF to distinguish ADHD and different clinical groups deserves further investigations.

The BRIEF considers behavioral patterns, which are compatible with ADHD symptoms, so it can be used to find the symptoms of basic behavioral dysregulation such as sustained attention and inhibition of competing responses. The items of BRIEF can help parents and teachers to better understand the nature of behavioral criteria of ADHD in their kids. Another finding of our study was lower mean scores of all subscales and indices of BRIEF in our control group than the normal children reported in the manual of BRIEF. It would be due to some methodological or probable cross-cultural differences and suggests further investigations.

Our study had some limitations. First, the small sample size and the narrow age range of the participants limit generalizability of the results. Secondly, we did not consider teachers’ ratings on ADHD symptoms, an informative pool of data, which could be quite helpful. Third, as all participants with ADHD had its combined type, we could not evaluate the validity of BRIEF to differentiate ADHD subtypes. In conclusion, the BRIEF can be considered as a valid tool to discriminate children with ADHD from typically developing children referred to psychiatric clinics.