Evaluating ADHD Assessment for Dogs: A Replication Study.

The family dog, in its natural environment, exhibits neuropsychological deficits redolent of human psychiatric disorders, including behaviours similar to human Attention-Deficit/Hyperactivity Disorder (ADHD) symptoms. For dogs, Vas and colleagues developed a 13-item questionnaire to measure inattention and hyperactivity/impulsivity (Dog ARS; 2007). We re-assessed, in a large sample of dogs (N = 319), psychometric properties of the Dog ARS, to identify possible limitations as a basis for further development. We examined the cross-study stability of factor structure and 40-day temporal stability of item and subscale scores and compared owner-report with expert (dog trainer)-report (n = 86), paralleling human parent/teacher assessments. To identify ambiguous items, we administered a modified version (including "I don't know" options, N = 520) to a different sample. We could replicate the factor structure with evidence of good internal consistency and test-retest reliability of both subscales. Agreement between owner and trainer ratings was fair (inattention) and moderate (hyperactivity/impulsivity). Three ambiguous items were identified. Overall, we claim that the Dog ARS is a reliable tool to assess ADHD-like behaviour in dogs, but in its current form, it is not suitable to detect diagnosable individuals, as it does not comprise items assessing functional impairment, and also, the inclusion of owner-expert ratings in the evaluation process would be necessary.

1. Introduction

Attention-deficit/hyperactivity disorder (ADHD) is one the most common neurodevelopmental disorder in humans, with worldwide prevalence estimates around 7.5% in children [1] and approximately 3.5% in adults [2,3]. ADHD primarily manifests as inattention, hyperactivity and impulsivity symptoms, though it is a complex and heterogeneous syndrome [4,5,6]. Individuals with ADHD show impairments in a variety of functional domains, including the academic and social domains [7,8].

Beyond rodent models, due to its unique evolutionary past, the family dog is a promising animal model of human socio-cognitive behaviours [9,10,11], and it has been gaining empirical attention as a model for human neurodevelopmental disorders, including autism [12] and ADHD [13]. In the case of ADHD, contrary to rodents, dogs naturally exhibit large ADHD-related phenotypic variability [14,15]. Dogs also show associations between dopaminergic receptor D4 (DRD4) gene and the tyrosine hydroxylase (TH) gene polymorphisms and hyperactivity/impulsivity traits, comparably to human polymorphisms related to ADHD [16]. Finally, dogs exhibit functional impairment in relation to ADHD-like behaviours [17,18], similarly to human children (e.g., [8]).

In humans, evidence-based assessment of ADHD involves a multimethod battery comprising clinical interviews and testing, behavioural observation, and behavioural rating scales completed by relevant informants, such as the child’s parent and/or teacher [19]. Children with ADHD exhibit functional impairments, including in school (e.g., poor achievement) and socially (e.g., family, sibling, and peer relationship problems) [5,20], though behaviours of interest may be cross-situationally different [19,21]. Accordingly, information about the behaviour of the child in different settings is ideally obtained from both parents and teachers is necessary for diagnosis [5,19,21]. Regarding behavioural rating scales for measurement of symptoms and impairment, various questionnaires are available. One such rating scale is the ADHD Rating Scale-IV [22], which measures child behaviour over the past 6 months and consists of 18 items comprising two subscales: inattention (9 items) and hyperactivity/impulsivity (9 items) [22].

Questionnaires originally designed to assess human behaviour have been adapted to measure dog behaviour and also of ADHD-related behaviours and symptoms, including the Dog ADHD Rating Scale (Dog ARS; [14]) and the Dog Impulsivity Assessment Scale (DIAS; [15]). The Dog ARS is an adaptation of the ADHD Rating Scale-IV [22], consisting of 13 items that make up two subscales measuring inattention and hyperactivity/impulsivity. The Dog ARS has been used to assess: breed specificity [23,24,25], comorbidity and prevalence [24,25,26], and genetic correlates [16,27,28,29,30] of ADHD-like behaviours; relations between ADHD-like behaviours and behavioural inhibition [13,31], cognitive bias [31], selective attention [32], and problem solving and temperament [33]; and to compare behavioural observation with owner ratings of inattention and hyperactivity/impulsivity traits [34] and for cross cultural comparisons [35]. In an earlier examination of psychometric properties, Vas and colleagues [14] obtained evidence of acceptable internal consistency for both Dog ARS subscales, but they did not assess test–retest reliability, and to assess evidence of validity, they only examined the relation between age (categories: juvenile, adult and old) and subscale scores. This approach to validity, however, is overly simplistic given the complex and dynamic relations between age and ADHD [36]. In childhood, hyperactive-impulsive symptoms are more dominant than inattention symptoms, but inattention symptoms are more often observed in adolescence and early adulthood [36,37]. Although the manifestation of ADHD symptoms changes with age, the behavioural problems, and functional impairment associated with the disorder (albeit also manifesting differently) persist during development [36,37]. Others evaluated evidence of convergent validity between the Dog ARS and a behavioural test battery probing activity and impulsivity, with results indicating that hyperactivity/impulsivity scores correlated with behavioural performance [16,29].

Lit et al. [38] aimed to replicate the Vas et al. [14] study on a North American dog population, addressing similar research questions and employing a similar design [38]. Lit and colleagues used a slightly modified version of the Dog ARS. One question was removed (“It is excessive, difficult to control, if it lunges it is hard to hold back.”), as the skills assessed could depend on equipment and/or training, and instead of a 4-point, a 5-point Likert response format scale was used (of note, human rating scales also have the 4-point response format to force raters to make a decision on the frequency of symptoms). Consistent with earlier findings [14], age was associated with subscale scores, with older dogs having lower subscale scores, and physical size and sex (when controlling for effects of age) were not associated with subscale scores, suggesting that the Dog ARS has good external validity [38]. The initial factor structure [14] did not replicate, however, as a 3-factor solution (inattention, hyperactivity/impulsivity-1, and hyperactivity/impulsivity-2) fit the data better [38], suggesting a potentially unstable factor structure. Although hyperactivity/impulsivity items loaded onto two factors, these were not hyperactivity and impulsivity—as in humans—but two mixed factors. A design limitation was that a cross-loading item was retained, despite best-practice guidelines recommending elimination of such items [39]. In addition, as previously [14,16,29], test–retest reliability was not examined.

Another measure, the Dog Impulsivity Assessment Scale (DIAS; [15]), albeit not designed to measure ADHD-related behaviours per se, is also relevant as it was developed to measure impulsivity in dogs. The DIAS consists of 18-items that comprise three subscales (see Table 1) [15]. Regarding reliability, internal consistency was unacceptable for responsiveness (α = 0.44). Although they are not the most appropriate measures of test–retest reliability [40,41], Pearson’s correlation coefficients between test and retest scores were calculated and indicated a correlation between scores. In a follow-up study, in which Wright and colleagues’ DIAS results were replicated [42], the responsiveness subscale did not exhibit acceptable agreement across years. Regarding validity, two DIAS items correlated with DIAS total score and subscale scores [15], and this was interpreted by the authors as evincing convergent validity, although item-total correlation is more of an index of internal consistency [43,44]. Further, DIAS total scores correlated with delayed gratification task performance [45], though elsewhere, convergent validity was not supported [46].

Table 1

Summary of obtained factor structure and assessed psychometric properties of the Dog ARS across studies.

Study	Measure	Factors and Number of Items	Psychometric Property
			Internal Consistency	Test–Retest Reliability	Convergent Validity	Expert Rating
Vas et al., 2007 [14]	ADHD	IA (6)	Acceptable internal consistency	–	Age is not a key correlate of ADHD	–
		H/I (7)
Lit et al., 2010 [38]	ADHD	IA (6)	Only on a priori subscales	–	No key correlates	–
		H/I-1 (4)
		H/I-2 (2)
Wright et al., 2011 [15]	I	Behavioural regulation (8)	Unacceptable Cronbach’s alpha on the Responsiveness factor (=0.44)	Pearson’s correlation between test and retest scores	Two items were correlated with the total questionnaire scores, contradictory results with behaviour tests	–
		Aggression and response to novelty (5)
		Responsiveness (5)

Note. IA = Inattention; H = Hyperactivity; I = Impulsivity.

For a summary of data on the available questionnaires assessing dog ADHD-related traits, see Table 1 with information on the obtained factor structure and the evaluated aspects of psychometrics across studies.

To contribute to human ADHD research as a model in a reliable and valid manner, finding methods to accurately measure the ADHD characteristics in dogs is crucial. In humans, diagnostic guidelines recommend assessment of symptoms and impairments in different settings (i.e., home and school), obtaining data from multiple informants, usually parents and teachers [21,47,48]. Accordingly, comparability will necessitate canine ADHD research to focus not only on owner but also on expert (e.g., dog trainer) view of dog behaviours, to parallel teacher report.

Current Study

Our general aim in the current study was to conduct a comprehensive evaluation of the Dog ARS [14] to gather and present evidence on the psychometric properties of—and identify limitations to—the measure, and accordingly suggest potential modifications and refinements that may be implemented. We organized our overarching aim into four specific aims and corresponding research questions as follows.

Aim 1—Compare data obtained in the current and in a prior study on the Dog ARS, to assess the extent to which owners’ evaluation of dog inattention and hyperactivity/impulsivity changed across a 14-year period.

Q1: Is there a difference between the item ratings of the current (hereafter: C Dog ARS) and the prior sample obtained by Vas and colleagues (hereafter: V Dog ARS)?

Aim 2—Examine the factor structure, test–retest reliability, and external validity of the Dog ARS.

Q1: Is there a difference between the factor structure of the current (C Dog ARS) and the prior sample (V Dog ARS)?

Q2: How reliable is the C Dog ARS across time (40-day test–retest reliability) at the level of the total score, subscale scores, and individual item scores?

Q3: To what extent are C Dog ARS scores associated with dog age, sex, and training status (external validity)?

Of note, in examining the factor structure and external validity of the C Dog ARS we simultaneously aimed to determine the extent to which relevant findings obtained earlier [14] are replicable.

Aim 3—Identification of ambiguous items of the Dog ARS. For this aim, we collected data from owners of a second, separate sample on a modified Dog ARS, where an “I don’t know” (hereafter: IDK) response option was added to all items (hereafter: Dog ARS IDK). Further, for the first time, we also collected data from each dog’s trainer (i.e., experts).

Q1: Are there any ambiguous items on the Dog ARS IDK for owners (Dog ARS IDK-O), as indicated by a high proportion of owner IDK responses?

Q2: Are there any ambiguous items on the Dog ARS IDK for trainers (Dog ARS IDK-T), as indicated by a high proportion of owner IDK responses?

Aim 4—Compare owner and trainer ratings on the Dog ARS IDK. Especially relative to owners, trainers typically have a large reference group to compare each dog to.

Q1: How reliable is the Dog ARS IDK across raters (interrater reliability), at the level of the total score, subscale scores, and individual item scores?

In Table 2, we summarize our aims and corresponding methods, sample characteristics and sample sizes.

Table 2

Summary of the aims and corresponding methods and samples used in the present study.

Aim/Question	Method	Sample
A1. Replicability of the Dog ARS at the item levelQ1. Stability of item ratings	Comparing current and previously obtained data: Q1. Average item ratings and SDs	Q1.: V Dog ARS (N = 220) vs. C Dog ARS (N = 319)
A2. General replicability of the Dog ARSQ1. Stability of factor structureQ2. Test-retest reliability of the Dog ARSQ3. Effect of age, sex, and training status on subscales	Comparing current and previously obtained data:Q1. Exploratory factor analysis, Cronbach’s alphaQ2. Intraclass correlation coefficient and item level, agreement rate (%) between test–retest ratingsQ3. GLM	Q1. and Q3.: V Dog ARS (N = 220) vs. C Dog ARS (N = 319)Q2.: C Dog ARS vs. its retest (n = 140/319)
A3. Identification of ambiguous items in the Dog ARSQ1. Identify ambiguous items for ownersQ2. Identify ambiguous items for trainers and owners	Q1. & Q2.: Inclusion of an “I don’t know” response option for each item	Q1.: Dog ARS IDK-O (N = 520)Q2.: Dog ARS IDK-T subsample: (n = 86/520)
A4. Owner vs. Trainer ratings on the Dog ARSQ1. Interrater reliability at the level of the total score, subscale scores, and individual item scores	Comparing owner and trainer ratings:Q1.: Intraclass correlation coefficient and item level agreement rate (%) between trainer-owner ratings	Q1.: Dog ARS IDK-T subsample: (n = 86/520)

Note. Abbreviations: A: aim, Q: question, V Dog ARS: previously obtained data by Vas et al. [14], C Dog ARS: current dataset on Dog ARS, GLM: Generalized Linear Models, Dog ARS IDK-O: current dataset on Dog ARS with “I don’t know” option for owners, Dog ARS IDK-T: current dataset on Dog ARS with “I don’t know” option for trainers.

2. Materials and Methods

2.1. Ethics Statement

The present study was conducted at Eötvös Loránd University in Budapest, Hungary, where animal experiments are overseen by the University Institutional Animal Care and Use Committee (UIACUC). According to the definition of “animal experiments” by the currently operating Hungarian law—the Animal Protection Act—our questionnaire study on dog behaviour was not considered as an animal experiment under the law and was therefore allowed to be conducted without any special permission from the UIACUC. Informed consent was obtained from all dog owners who completed our online questionnaire. Participants were informed about the goals and circumstances of the present study a priori, and they were informed that they may discontinue participating in this research at any time. Participation in the present study was voluntary and anonymous, and the data obtained were used for scientific purposes only. In compliance with relevant data protection laws, personally identifying data were treated confidentially and stored separately from the rest of research data.

2.2. Subjects

Participants were recruited through the Department of Ethology participant pool and website, popular social networking sites, and via snowball sampling. Dog trainers were recruited via the dogs’ owners (owners were asked to involve the trainer working with their dog). Sample sizes differed across research questions and were accordingly indicated separately below for each question. Information on sample demographics for the different samples are summarized in Section S3 in the Supplementary Material.

2.3. Measures

Questionnaires were completed online (see Sections S1 and S2 in the Supplementary Material) between January 2017 and December 2019 in Hungary. The questionnaires were submitted to the participants in their native language, in Hungarian.

Respondents were asked to provide their own name and e-mail address as well as the dog’s name, breed, date of birth, sex, neuter status, and training status (none-basic-advanced). Dog inattention and hyperactivity/impulsivity were measured using two versions of the Dog-ADHD Rating Scale (see below).

2.3.1. Dog ARS

Inattention and hyperactivity/impulsivity in dogs were measured using the Dog-ADHD Rating Scale [14], a 13-item (6 inattention items and 7 hyperactivity/impulsivity items) owner-report measure of inattention and hyperactivity/impulsivity in dogs. Owners indicate the frequency with which their dog behaves as described in each item (4-point Likert-type response format, ranging from ‘never’ to ‘very often’). Inattention and hyperactivity/impulsivity subscale scores were calculated by summing item scores for each subject. Greater scores indicate greater difficulties with inattention and hyperactivity/impulsivity. For the questionnaire, see Section S1 in the Supplementary Material.

2.3.2. Dog ARS IDK

The Dog ARS was modified via inclusion of an “I don’t know” (IDK) response option for each question. The questionnaire was used to assess owner (Dog ARS IDK-O) and trainer (Dog ARS IDK-T) ratings of dogs. For the questionnaire, see Section S2 in the Supplementary Material.

2.3.3. Covariates of Non-Interest

Relevant covariates that have been previously hypothesized or shown to be associated with differences in canine inattention and hyperactivity/impulsivity, were dogs’ owner-reported age, sex, and training status [13,14,31].

2.4. Methods, Statistical Analyses, and Samples Used for the Different Aims in the Present Study

2.4.1. Aim 1/Question 1: Is There a Difference between the Item Ratings of the Current (C Dog ARS) and the Prior Sample (V Dog ARS)?

Previously obtained data by Vas et al. [14] on N = 220 dogs (V Dog ARS), 106 males and 114 females, with Mage = 35.56 months, SD = 7.79 were used for the present question. They reported no data on neutering status. In the current study we used the Dog ARS [14] questionnaire without modification (C Dog ARS). C Dog ARS data were available for N = 319 dogs: 162 male (83 intact, 79 neutered) and 157 female (61 intact, 96 spayed) (Mage = 48.44 months, SD = 36.23).

2.4.2. Aim 2/Question 1: Is There a Difference between the Factor Structure of the Current (C Dog ARS) and the Prior Sample (V Dog ARS)?

To address this research question, we used the Dog ARS [14]. The analysis sample was the same as for Aim 1/Question 1, the C Dog ARS and the V Dog ARS.

2.4.3. Aim 2/Question 2: How Reliable Is the C Dog ARS across Time (40-Day Test–Retest Reliability) at the Level of the Total Score, Subscale Scores, and Individual Item Scores?

For assessing test–retest reliability, the C Dog ARS was completed, on average, 40 days apart, with a range of 24 to 49 days. Data for these analyses were available for n = 140 dogs: 74 male (37 intact, 37 neutered) and 66 female (27 intact, 39 spayed) dogs (Mage = 48.65 months, SD = 36.58).

2.4.4. Aim 2/Question 3: To What Extent Are C Dog ARS Scores Associated with Dog Age, Sex and Training Status (External Validity)?

For the present research question, we collected data with the Dog ARS [14]. The analysis sample was the same as for Aim 1/Question 1, the C Dog ARS, and the V Dog ARS.

To examine external validity (relations with age, sex, and training status on inattention and hyperactivity/impulsivity) and make valid comparisons across prior [14] and the current findings, training status was indexed as “none” (no training), “basic”, or “advanced” (IPO Schutzhund, rescue, service, or gun dog exam) as in Vas et al. [14]. In that study, age was also treated as a categorical variable (juveniles, adults, and old dogs), and interactions between age, sex, and training status were not examined. In the current study, age was treated as a continuous variable, sex and training status as categorical variables, and interactions were considered.

2.4.5. Aim 3/Question 1: Are There Any Ambiguous Items on the Dog ARS IDK-O, as Indicated by a High Proportion of Owner “I Don’t Know” Responses?

To identify ambiguous questions, the Dog ARS was modified via inclusion of an IDK response option for each question (Dog ARS IDK). Assuming that respondents select the IDK option when a question is difficult to respond to or they are uncertain in their rating, higher IDK response counts may indicate items that need to be changed in the future.

The analysis sample consisted of N = 520 dogs for the Dog ARS IDK-O (independent sample from the N = 319 sample, see Table 2). This sample consisted of 227 male (120 intact, 107 neutered) and 293 female (98 intact females, 195 spayed females) dogs (Mage = 55.01 months, SD = 39.59).

2.4.6. Aim 3/Question 2: Are There Any Ambiguous Items on the Dog ARS IDK-T, as Indicated by a High Proportion of Trainer “I Don’t Know” Responses?

The Dog ARS IDK-T (with IDK response option) was used to reveal ambiguous items for trainers. We also included in the analysis the owner ratings (using the Dog ARS IDK-O) which were available for the same dogs that were rated by the trainer.

The analysis sample was a subsample of the sample used for Aim 3/Question 1 (see Table 2). Linked to the previous research question, trainer ratings were available for n = 86 dogs: 40 male (19 intact, 21 neutered) and 46 female (13 intact, 33 spayed) dogs (Mage = 44.31 months, SD = 28.96). Fifteen trainers completed the questionnaire, with a range of minimum 1 to maximum 20 responses (dogs) per trainer.

2.4.7. Aim 4/Question 1: How Reliable Is the Dog Ars across Raters (Interrater Reliability), at the Level of the Total Score, Subscale Scores, and Individual Item Scores?

To compare owner and trainer ratings, the Dog ARS IDK-O and the Dog ARS IDK-T were used.

The same data were used for assessing owner–trainer inter-rater agreement as for Aim 3/Question 1 (see Table 2). On average, owners and trainers completed the rating scale 24 days apart.

2.5. Statistical Analyses

Statistical analyses were conducted in IBM SPSS Statistics version 22.0.0.0. for Aim 1, Aim 2, and Aim 4. For Aim 3 and Aim 4 IDK response and agreement rate were calculated and presented using Microsoft Excel 2010 version 14.0.7268.5000 (32 bit). Analyses used for the different aims and research questions are described in detail below.

2.5.1. Aim 1/Question 1: Is There a Difference between the Item Ratings of the Current (C Dog ARS) and the Prior Sample (V Dog ARS)?

Independent samples t-tests (adjusted for Levene’s test in the case of unequal variances) were conducted to evaluate differences between the C Dog ARS and the V Dog ARS sample means of each item, with p < 0.05 considered significant.

2.5.2. Aim 2/Question 1: Is There a Difference between the Factor Structure of the Current (C Dog ARS) and the Prior Sample (V Dog ARS)?

Exploratory factor analysis was used with varimax rotation to explore the factorial structure, and item loadings and findings were compared with those obtained previously [14]. Internal consistency for the inattention and hyperactivity/impulsivity subscales was estimated using Cronbach’s alpha. The independence of the subscales was evaluated with Pearson correlation.

2.5.3. Aim 2/Question 2: How Reliable Is the C Dog ARS across Time (40-Day Test–Retest Reliability) at the Level of the Total Score, Subscale Scores, and Individual Item Scores?

To estimate test–retest reliability, intraclass correlation coefficients (ICC) with corresponding 95% CIs were computed, as they measure agreement and account for both consistency of performances from test to retest, as well as for the systematic change in the mean [41]. ICCs represent the ratio of between-subjects variance to total variance for assessing test–retest reliability when observations are not independent [49]. ICCs can range from −1 to 1 and, in accordance with convention, were interpreted as follows: 0–0.2 as poor, 0.3–0.4 as fair, 0.5–0.6 as moderate, 0.7–0.8 as strong, and >0.8 as almost perfect [50]. Of note, it is possible for ICCs to be negative when the within-group variance exceeds the between-groups variance, suggesting a measure is not reliable. We also reviewed the consistency of owner responses at the item level by calculating an agreement rate (%).

2.5.4. Aim 2/Question 3: To What Extent Are C Dog ARS Scores Associated with Dog Age, Sex, and Training Status (External Validity)?

The association between age, sex, training status, and their interactions and the dependent variables, i.e., inattention and hyperactivity/impulsivity, were examined in generalized linear mixed models with backward elimination. Age was entered as a covariate, sex, and training status as fixed factors and subject as random factor. Following backward elimination, variables were removed in order of decreasing significance, starting with the interactions, until only significant variables were in the model.

Tweedie with log link option was used as model type, given that the dependent variables had zero scores and skewed distributions, resulting in non-normal distribution of residuals. Sidak correction was applied to account for multiple comparisons. Assumptions were considered prior to all analyses, these were met.

2.5.5. Aim 3/Question 1: Are There Any Ambiguous Items on the Dog ARS IDK-O, as Indicated by a High Proportion of Owner “I Don’t Know” Responses?

The proportion of IDK responses for each item are calculated (i.e., summed) and presented. For identifying a cut-off point, when an IDK response proportion was considered high, we calculated the average IDK response rate, and then, if the absolute difference between the average and the particular “IDK” response proportion was higher than 150%, we considered those questions as potentially problematic.

2.5.6. Aim 3/Question 2: Are There Any Ambiguous Items on the Dog ARS IDK-T, as Indicated by a High Proportion of Trainer “I Don’t Know” Responses?

The analysis was the same as for Aim 3/Question 1.

2.5.7. Aim 4/Question 1: How Reliable Is the Dog ARS across Raters (Interrater Reliability), at the Level of the Total Score, Subscale Scores, and Individual Item Scores?

Intraclass correlation coefficients (ICC) with corresponding 95% CIs were computed for measuring owner–trainer inter-rater agreement at the level of the total score and subscale scores. To review the consistency of owner–trainer ratings at the item level, an agreement rate (%) was calculated.

3. Results

3.1. Aim 1/Question 1: Is There a Difference between the Item Ratings of the Current (C Dog ARS) and the Prior Sample Obtained by Vas and Colleagues (V Dog ARS)?

Stability of Item Scores

Descriptive statistics for each item are presented in Table 3. Comparing V Dog ARS and C Dog ARS data, inattention items 1, 2, and 12 as well as hyperactivity/impulsivity item 6 differed across the datasets (see Table 3). The mean scores on inattention items 1, 2, and 12 mean were higher in the current sample whereas the mean score on hyperactivity/impulsivity item 6 was higher in V Dog ARS sample (see Table 3).

Table 3

Descriptive statistics, cross-study differences, and indices of reliability across individual items of the V Dog ARS for the prior Vas et al. (V) (N = 220) [14] and the current C Dog ARS (C) sample (N = 319) for Aim 1/Question 1 and Aim 2/Question 1.

Items	M (SD)		Difference between Variance and Means (Levene’s Test and Independent Samples t-Tests)	Corrected Item—Total Correlation		Cronbach’s Alpha If Item Deleted		Cronbach’s Alpha
	V	C		V	C	V	C	V	C
	Inattention subscale							0.78	0.81
1. My/this dog has learning difficulties, because it is careless or because other things easily attract its attention.	0.71 (0.73)	0.84 (0.69)	F(454) = 6.26, p = 0.013 *t(454) = −2.04, p = 0.042 *	0.58	0.72	0.74	0.75
2. It is easy to attract my/this dog’s attention, but it also quickly loses interest.	0.82 (0.84)	0.98 (0.76)	F(441) = 13.44, p < 0.001 *t(441) = −2.23, p = 0.026 *	0.64	0.60	0.72	0.78
3. It is difficult for my/this dog to concentrate on a task or play.	0.50 (0.69)	0.59 (0.63)	F(537) = 1.37, p = 0.242t(537) = −1.69, p = 0.091	0.58	0.62	0.74	0.78
7. It seems that my/this dog does not listen even if it knows that someone is directly speaking to him/her.	0.50 (0.70)	0.56 (0.67)	F(537) = 0.20, p = 0.654t(537) = −0.94, p = 0.346	0.41	0.45	0.78	0.81
10. My/this dog solves simple tasks easily, but he/she often has difficulties with complicated tasks, even if those are known or have been often practiced.	0.66 (0.77)	0.62 (0.67)	F(425) = 5.24, p = 0.022 *t(425) = 0.55, p = 0.584	0.48	0.47	0.76	0.81
12. My/this dog’s attention can be easily distracted.	0.96 (0.79)	1.15 (0.72)	F(537) = 0.37, p = 0.545t(537) = −2.84, p = 0.005 *	0.50	0.62	0.76	0.77
	Hyperactivity/impulsivity subscale							0.73	0.78
4. My/this dog leaves from its place when it should stay.	1.20 (0.96)	1.08 (0.74)	F(391 = 34.60, p < 0.001 *t(391) = 1.58, p = 0.114	0.35	0.43	0.72	0.77
5. My/this dog cannot be quiet or easily calmed.	0.81 (0.98)	0.83 (0.91)	F(537) = 2.34, p = 0.127t(537) = −0.25, p = 0.806	0.40	0.42	0.71	0.77
6. My/this dog fidgets all the time.	1.39 (1.10)	1.18 (0.93)	F(415) = 22.99, p < 0.001 *t(415) = 2.38, p = 0.018 *	0.58	0.58	0.67	0.74
8. My/this dog is excessive or difficult to control, or if it lunges, it is difficult to hold back.	0.71 (0.81)	0.76 (0.80)	F(537) = 0.001, p = 0.972t(537) = −0.59, p = 0.553	0.55	0.61	0.68	0.73
9. My/this dog would always play and run.	1.40 (1.00)	1.30 (0.92)	F(537) = 3.54, p = 0.060t(537) = 1.23, p = 0.220	0.45	0.50	0.70	0.76
11. My/this dog is likely to react hastily, and that is why it is failing tasks.	0.93 (0.84)	1.02 (0.74)	F(537) = 7.94, p = 0.005 *t(537) = −1.27, p = 0.206	0.37	0.49	0.72	0.76
13. My/this dog cannot wait as it has no self-control.	0.98 (0.98)	1.10 (0.90)	F(537) = 1.48, p = 0.224t(537) = −1.51, p = 0.131	0.45	0.52	0.70	0.75

Note. V = previously obtained data by Vas et al. [14]; (V Dog ARS); C = current data (C Dog ARS). t-tests indicated differences in the case of items 1, 2, 6, and 12, denoted by *.

3.2. Aim 2/Question 1: Is There a Difference between the Factor Structure of the C Dog ARS and V Dog ARS?

3.2.1. Reliability of Subscales

The inattention subscale with six items had good internal consistency, α = 0.81. All items appeared retainable, as Cronbach’s alpha would not measurably improve with exclusion of any (Table 3). All items correlated with the total subscale (all rs ≥ 0.45) (Table 3).

The hyperactivity/impulsivity subscale with seven questions had lower but acceptable internal consistency, α = 0.78. As with the inattention subscale, there would be very little improvement with exclusion of any item (Table 3) and all individual items correlated with the total subscale (all rs ≥ 42).

3.2.2. Factor Analysis

The scree plot indicated two factors (Factor 1 and Factor 2, see Table 4). Item 4 (“My/this dog leaves from its place when it should stay.”) was deleted after the first run of the factor analysis because it loaded >0.4 on factor 1 (0.432) and on factor 2 (0.411) (for the other aims, we retained item 4, to have comparable results with the previous study). After removal of the cross-loading item, the exploratory factor analysis (EFA) was repeated and yielded a final factor structure. The two factors accounted for 51.3% of the total variance (eigenvalues >1.5), with the first explaining 36.9% and the second explaining 14.4% of the total variance.

Table 4

Factor loadings of Dog ARS items for the V Dog ARS [14] and the current C Dog ARS samples for Aim 2/Question 1.

Aimed to Measure	Item	Factor 1		Factor 2
		V	C	V	C
Inattention	Item 3	0.765	0.800	−0.120	0.016
	Item 1	0.729	0.781	0.134	0.270
	Item 2	0.795	0.772	−0.043	−0.012
	Item 12	0.584	0.700	0.366	0.305
	Item 10	0.664	0.578	0.039	0.209
	Item 7	0.512	0.485	0.218	0.337
Hyperactivity/impulsivity	Item 6	−0.119	−0.026	0.794	0.822
	Item 9	−0.157	−0.010	0.715	0.759
	Item 8	0.308	0.325	0.682	0.699
	Item 11	0.207	0.299	0.517	0.558
	Item 13	0.096	0.334	0.581	0.538
	Item 5	0.091	0.167	0.515	0.503
	* Item 4	0.377	0.432	0.392	0.411

Note. V = previously obtained data by Vas et al. [14]; C = current data. Item 4 was removed after EFA due to cross-loading, denoted by *. Bold numbers: indicate which item belongs to which factor.

EFA results were generally comparable to those obtained previously [14]. In the current sample, Factor 1 contained six inattention items, and Factor 2 contained six hyperactivity/impulsivity items. Items of the first factor (Factor 1) all had high factor loadings (all loadings >0.578) except for item 7, which had a lower though still acceptable factor loading (0.485). All items of the second factor (Factor 2) had high factor loadings (all loadings >0.503). Items on Factor 1 had low loadings on Factor 2 (all loadings <0.337) and the same was the case regarding Factor 2 item loadings on Factor 1 (all loadings <0.334).

As a final step, we examined the association between scores on the previously established subscales [14] with the currently generated factors. Subscale scores and factor scores were strongly correlated (Subscale/Factor 1: r = 0.95, Subscale/Factor 2: r = 0.95, all ps < 0.0001). The robust relations between previously established subscale scores and current factor scores suggest that Factor 1 is associated with inattention and Factor 2 with hyperactivity/impulsivity.

3.3. Aim 2/Question 2: How Reliable Is the C Dog ARS across Time (40-Day Test-Retest Reliability) at the Level of the Total Score, Subscale Scores, and Individual Item Scores?

There was strong agreement across questionnaire completions of both the inattention factor (ICC = 0.860; 95% CIs = [0.805; 0.900], p < 0.001) and the hyperactivity/impulsivity factor (ICC = 0.881; 95% CIs = [0.834; 0.914], p < 0.001), indicating excellent test–retest reliability. The ADHD total score, including all items indicated excellent test–retest reliability (ICC = 0.896; 95% CIs = [0.855; 0.925], p < 0.001).

To examine reliability in more depth, we examined the item-level agreement within owner ratings (see Aim 4/Question 1., Table 5). The item with the lowest temporal consistency was item 13 (53.9%) (“My/this dog can not wait as it has no self-control.”).

Table 5

Agreement between owner–trainer (interrater) (n = 86) and within-owner (test–retest) (n = 140) ratings (Aim 3/Question 2) at the item level.

Item	Interrater Agreement (Owner–Trainer)	Test–Retest Agreement (Owner)
Inattention subscale
1. My/this dog has learning difficulties because it is careless or because other things easily attract its attention.	55.3%	67.4%
2. It is easy to attract my/this dog’s attention, but it also quickly loses interest.	47.1%	63.1%
3. It is difficult for my/this dog to concentrate on a task or play.	55.3%	68.1%
7. It seems that my/this dog does not listen even if it knows that someone is directly speaking to him/her.	52.9%	66.7%
10. My/this dog solves simple tasks easily, but he/she often has difficulties with complicated tasks, even if those are known or have been often practiced.	35.5%	63.8%
12. My/this dog’s attention can be easily distracted.	50.6%	61.7%
Hyperactivity/impulsivity subscale
4. My/this dog leaves from its place when it should stay.	57.8%	67.4%
5. My/this dog cannot be quiet or easily calmed.	47.1%	61.0%
6. My/this dog fidgets all the time.	47.1%	58.9%
8. My/this dog is excessive or difficult to control, or if it lunges, it is difficult to hold back.	42.9%	64.5%
9. My/this dog would always play and run.	44.2%	65.2%
11. My/this dog is likely to react hastily and that is why it is failing tasks.	46.9%	62.4%
13. My/this dog cannot wait as it has no self-control.	47.6%	53.9%

3.4. Aim 2/Question 3: To what Extent Are C Dog ARS Scores Associated with Dog Age, Sex, and Training Status (External Validity)?

3.4.1. Inattention

Sex and the interaction between age and training status were unrelated to inattention scores. Age was associated with inattention (χ2(1) = 5.138, p = 0.023); younger dogs had higher scores (Figure 1a). Following elimination of the non-significant interaction term, training status was also associated with inattention (χ2(1) = 12.696, p = 0.002), with post hoc tests indicating a difference in inattention scores between dogs with no training and advanced training (p < 0.001, [0.80; 3.12]) and between dogs with basic and advanced training (p = 0.006, [0.36; 2.83]), but no difference between dogs with no training and basic training (p = 0.796, [−0.70; 1.43]); see Figure 2b.

Figure 1

Greater age is associated with lower inattention (a) and with lower hyperactivity/impulsivity (b). The regression lines are default SPSS fitted lines representing trends in the data, with the slope corresponding to the unstandardized regression coefficient b of a linear regression equation, where Y = a (i.e., intercept) + bX (in this case, for inattention-age: y = 5.6 + −0.02 * x and y = 8.56 + −0.03 * x for hyperactivity/impulsivity-age). Note. The coloured circles represent subjects.

Figure 2

The association between training status and inattention (a) and hyperactivity/impulsivity (b); * indicates significant differences (p < 0.05).

3.4.2. Hyperactivity/Impulsivity

Sex and the interaction between age and training status were unrelated to hyperactivity/impulsivity scores. Age was associated with hyperactivity/impulsivity (χ2(1) = 9.878, p = 0.002); younger dogs had higher scores (Figure 1b). Following elimination of the non-significant interaction term, training status was also associated with hyperactivity/impulsivity (χ2(1) = 12.763, p = 0.002), with post hoc tests indicating a difference in hyperactivity/impulsivity scores between dogs with no training and advanced training (p = 0.001, [0.72; 3.68]) and between dogs with no training and basic training (p = 0.050, [0.00; 2.40]), but no difference between dogs with basic training and advanced training (p = 0.305, [−0.51; 2.51]); see Figure 2b.

3.5. Aim 3/Question 1: Are There Any Ambiguous Items on the Dog ARS IDK for Owners (Dog ARS IDK-O), as Indicated by a High Proportion of Owner “I Don’t Know” Responses?

The proportion of IDK responses by owners was ≤1.2% on all items, except for item 10 (“My/this dog solves simple tasks easily, but he/she often has difficulties with complicated tasks, even if those are known or have been often practiced.”) where the proportion of IDK responses were the highest (5.2%), and item 11 (“My/this dog is likely to react hastily and that is why it is failing tasks.”) with an IDK proportion of 4.2% (Figure 3a).

Figure 3

Proportion of “I don’t know” responses of (a) owners (N = 520) and (b) owners and trainers (n = 86) on the Dog ARS IDK.

The average IDK response proportion for the items was 1.17%. The absolute difference between the average IDK response proportion (1.17%) and the individual item IDK response proportion was higher than 150% in the case of item 10 (344.3%) and item 11 (262.0%).

3.6. Aim 3/Question 2: Are There Any Ambiguous Items on the Dog ARS IDK for Trainers (Dog ARS IDK-T), as Indicated by a High Proportion of Owner “I Don’t Know” Responses?

The proportion of IDK responses by owners was ≤1.2% on all items, except for item 11 (“My/this dog is likely to react hastily and that is why it is failing tasks.”), where data indicated a higher degree of uncertainty (3.5%), similarly to the results on the larger sample of owners on 520 dogs (Figure 3a).

In the subsample of dogs where trainer ratings were available, the proportion of IDK responses by trainers were the highest in the case of item 10 (9.3%) (“My/this dog solves simple tasks easily, but he/she often has difficulties with complicated tasks, even if those are known or have been often practiced.”) and item 13 (4.7%) (“My/this dog can not wait as it has no self-control.”) (Figure 3b).

In the case of trainer responses, the average IDK response proportion for the items was 1.79%. The absolute difference between the average IDK response proportion (1.79%) and the individual item IDK response proportion was higher than 150% in the case of item 10 (420.0%) and item 13 (160.0%).

Regarding owner responses, the average IDK response proportion for the items was 1.07%. The absolute difference between the average IDK response proportion (1.07%) and the individual item IDK response proportion was higher than 150% in the case of item 11 (225.0%) (“My/this dog is likely to react hastily and that is why it is failing tasks.”).

3.7. Aim 4/Question 1: How Reliable Is the Dog ARS IDK across Raters (Interrater Reliability), at the Level of the Total Score, Subscale Scores, and Individual Item Scores?

ICCs indicated fair agreement between owner and trainer ratings on the inattention subscale (ICC = 0.415; 95% CIs = [0.103; 0.618], p < 0.001) and moderate agreement on the hyperactivity/impulsivity subscale (ICC = 0.636; 95% CIs = [0.445; 0.762], p < 0.001). Owner and trainer ratings exhibited moderate agreement on the ADHD total score (ICC = 0.569; 95% CIs = [0.337; 0.719], p < 0.001).

Interrater (owner–trainer) agreement results at the item level are summarized in Table 5. We also included here our earlier results on the item-level test–retest reliability of owner ratings for easier comparability (Aim 2/Question 2). The lowest interrater agreement was for item 10 (35.5%) (“My/this dog solves simple tasks easily, but he/she often has difficulties with complicated tasks, even if those are known or have been often practiced.”).

4. Discussion

In this study, we repeated the analyses conducted by Vas and colleagues using the Dog ARS [14], addressing similar research questions and employing a similar design on new samples to examine evidence of the reliability and validity of the rating scale and to identify possible limitations. Our first aim was to examine the extent to which previous results on the Dog ARS replicate, as an index of how stable dog behaviour and/or owner rating styles are across a 14-year period (it is possible that either dog behaviour or owner rating style or both have changed since data on the Dog ARS were first collected by Vas et al. in 2007)/how stable the questionnaire is. In a similar way to human ADHD assessments, for the first time, we aimed to include expert ratings in the evaluation process, and compare expert and owner ratings. To identify possible problematic items, we included an IDK response option and examined the item level consistency in the case of within-owner and between owner–trainer agreement. We could replicate the factor structure of the Dog ARS, demonstrating evidence of good internal consistency and test–retest reliability on the inattention and hyperactivity/impulsivity subscales. Expert and owner rating comparisons revealed fair (inattention) and moderate (hyperactivity/impulsivity) agreement between raters. Based on examination of raters’ uncertainty (IDK response rates) and item level consistency, we identified three ambiguous items.

Examining whether findings replicate is important not only from the perspective of evaluating the psychometric properties of the scale but also to test its contemporary utility. Vas and colleagues conducted their study nearly fifteen years ago and, as dogs and the dog–owner relationship have likely changed a lot since then, owners may also rate their dogs’ inattention and hyperactivity/impulsivity differently. Indeed, findings of a US poll assessing changes in pet keeping habits between 2007 and 2015 indicated that pets are being increasingly viewed by owners as family members and more dogs are kept inside and allowed to sleep in their owners’ bed [51]. Moreover, in Hungary, 93.3% of dog owners consider their dogs to be family members [52].

We replicated the main findings of Vas et al. [14] in new samples, and our results confirmed the previously obtained factor structure of the questionnaire (inattention and hyperactivity/impulsivity) with a minor modification (deletion of item 4 due to cross-loading). Perhaps item 4 (“It leaves from its place when it should stay.”) reflects obedience, and thus, rather than relating only to hyperactivity/impulsivity, it is also related to attention to absolving a task and hence the cross-loading of this item. Our findings indicated good internal consistency of both subscales (inattention and hyperactivity/impulsivity). These results are inconsistent with those obtained by Lit et al., [38], who found the Dog ARS comprised three factors (inattention, hyperactivity/impulsivity-1, and hyperactivity/impulsivity-2). These differences across studies are probably due to slight differences in methodology (here, one item was removed, and a 5-point Likert-type response format scale was used instead of a 4-point scale) and differences in sample composition (a Hungarian vs. a North American dog population and/or owner bias).

Relative to item means obtained previously [14], three inattention and one hyperactivity/impulsivity item means differed in the current study. Interestingly, dogs in the present sample had higher mean scores on these items, suggesting that in the case of some traits, dogs are more inattentive or, alternatively, owners rate their dogs as more inattentive nowadays compared to 14 years ago. As cited earlier, this might be the result of the changing dog–human bond, which might shape how owners perceive their dogs’ ADHD related traits [35,51].

Our findings on external validity, specifically, age-related decrease in owner-rated inattention and hyperactivity/impulsivity, effect of training experience, and sex on these factors, complement earlier research. Vas and colleagues found associations between the inattention scale and age and training status, but no associations between hyperactivity/impulsivity and these variables. Some studies report a quadratic relationship between attention skills and age [53] and between activity and age [54]. In our sample, we did not observe such fine patterns.

Data on the test–retest reliability of the Dog ARS have not been published before. Our findings show both factors of the measure have good temporal consistency across 40 days. In a more in-depth examination of test–retest reliability, we also evaluated which items are ambiguous (IDK response rate) for owners and dog trainers and which items are problematic regarding consistency (percentage of agreement between two completions). For all items, test–retest findings were stronger than interrater agreement results. Regarding items 10 (“My/this dog solves simple tasks easily, but often has difficulties with complicated tasks, even those are known and have been often practiced.”), 11 (“My/this dog is likely to react hastily and that is why it is failing tasks.”), and 13 (“My/this dog cannot wait as it has no self-control.”), evaluators were uncertain and did not respond consistently, as indicated by a high IDK ratio and a low agreement between evaluations (across time in owners) and evaluators (across raters) at the item level. The uncertainty in the case of these items may stem from the fact that a question contains multiple statements, one of which may occur frequently while the other almost never. As an example, in the case of item 11, the dog may tend to react hastily, but this is not related to his performance in obedience tasks. Similarly, this problem arises in item 13: the dog may not be able to wait, but this is not related to its self-control, or waiting may mean a different context for respondents. Regarding item 10, it may be relative for each dog, which task counts as easy or difficult, and what counts as frequent exercise for each task. In a recent study where the Dog ARS’s factor structure was analysed, item 11 was excluded as it was equally loaded on the two factors, which may also indicate that the item is problematic [25]. These items could be potential targets for modification in the future, to improve the confidence and consistency of ratings and, moreover, to better reflect ADHD-like behaviours in dogs.

Owner–trainer rating comparisons revealed fair agreement between raters for the inattention scale and moderate agreement for hyperactivity/impulsivity factor. Compared to test–retest reliability results, lower owner–trainer agreement on dogs is in line with parent–teacher agreement on ratings of children [55,56,57,58]. The lower agreement on inattention and higher on the hyperactivity/impulsivity factor are also comparable to human results. Human findings report that raters find it more difficult to recognize the symptoms of inattention than detect hyperactive or impulsive symptoms of ADHD [59]. The literature on interrater reliability of assessments of internalizing and externalizing symptoms has also shown that parent–teacher agreement is higher for externalizing compared to internalizing behaviours [60]. Maybe a similar underlying mechanism could explain the difference that we found between inattention and hyperactivity/impulsivity rating comparisons.

Owners who had a trainer rating for their dog responded much more confidently (lower IDK response rates) than owners who did not have a trainer rating for the dog. It is likely that those owners who attend dog schools are generally more familiar with dog behaviours, more experienced, and thus more confident in their assessment. Owner recognition of fear-related behaviours in videos can be improved via education [61], supporting the assumption that experience on dog behaviour has an effect on ratings in general.

Compared to the earlier researcher-assisted, paper based questionnaire [14], our online data collection produced similar results, consistent with earlier findings [38]. However, we cannot determine whether the presence of the researcher can help respondents’ uncertainties regarding questionnaire items, as we do not know how much they helped the respondents in completing the questionnaire.

In sum, the Dog ARS proved to be a reliable tool to evaluate inattention and hyperactivity/impulsivity in dogs. The rating scale has a stable factor structure and good test–retest reliability, and in general, both owners and trainers can evaluate dogs with confidence. Although recent behaviour studies indicate that dogs may also show behaviours similar to human ADHD [23,62], none of the two questionnaires, which have been used so far to measure ADHD-like behaviours in dogs, provide distinction between normal and pathological levels of inattention, hyperactivity, and impulsivity. Therefore, it is still unknown whether there are dogs that can be diagnosed with ADHD applying a human-analogue methodology. Besides including questions of inattention, hyperactivity, and impulsivity characteristics, human questionnaires have a set of items which focus on functional deficits or symptom checklists used in combination with functionality assessments. This key component of ADHD has been completely ignored so far in dog ADHD-related assessments.

Another aspect to review is the factor structure of ADHD assessment in dogs. Although many human studies measure hyperactivity/impulsivity as one dimension of ADHD, more and more evidence is being published suggesting that the factor structure of symptoms of ADHD may change during adolescence and, specifically, that hyperactivity and impulsivity symptoms may diverge at some point in adolescence [63]. Studies indicate that impulsivity diminished at a slower rate than hyperactivity during the transition to adolescence [37], and this trend in the development of these traits may explain that during adulthood a three-factor structure of ADHD (inattention, hyperactivity, and impulsivity) fits better than a two-factor structure (inattention and hyperactivity/impulsivity) applied to children [7,37,64,65,66]. Thus, it would be worth inspecting closer whether impulsivity in dogs is a distinct factor from hyperactivity [67]. In humans, the trait of impulsivity and ADHD symptom severity are important correlates of reactive/impulsive aggression [68,69], which can also be a relevant factor to investigate in dogs.

Moreover, given that the present scales were created for dog owners, there is a need to include expert ratings, such as veterinarians or dog trainers to provide a comprehensive evaluation of ADHD-like behaviours in dogs. Inclusion of expert ratings and functional impairment assessments are key components of ADHD evaluation in humans. Thus, we suggest that the present rating scale can be a basis for further development, including expert-owner ratings and functional impairment in the evaluation process, to establish whether there are diagnosable individuals with ADHD in the case of dogs.

5. Conclusions

Summarizing key aims and results, the current study replicated the factor structure of the Dog ARS, with evidence of good internal consistency and test–retest reliability of inattention and hyperactivity/impulsivity subscales, revealed three ambiguous items that may need modification in the future, and for the first time included expert ratings of the Dog ARS, where fair (inattention subscale) to moderate (hyperactivity/impulsivity subscale) agreement was detected between owners and trainers. Our findings add to a growing body of research by extending the dog as an animal model of human ADHD and designates potential directions for future research to increase the reliability of the ADHD measurement in dogs and its comparability with humans.