Conceptual review of measuring functional impairment: findings from the Weiss Functional Impairment Rating Scale.

OBJECTIVE: This is a narrative review of validation and outcome studies using the Weiss Functional Impairment Rating Scale (WFIRS). The objective of the review is to establish a framework for understanding functional impairment and create a definition for functional response and remission.
METHODS: We conducted a literature search via MEDLINE, EBSCO and Google Scholar with no date restrictions and reviewed bibliographies of selected publications. Publications found in languages other than English were translated and clarification obtained from the author(s) if needed. Inclusion criteria were any manuscript that was either a WFIRS psychometric validation study or a clinical trial using the WFIRS as an outcome. There were no exclusion criteria.
RESULTS: The WFIRS has been validated in multiple cultures, and in clinical, research and control populations. The WFIRS has robust psychometric properties across ages, psychiatric status and informants. Outcome studies show variable improvement, with different response patterns between domains and among different interventions.
CONCLUSION: Symptom improvement and remission needs to be complemented with evaluation of functional improvement and remission to obtain a full picture of clinical status over the course of treatment. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) has been found to be associated with many deleterious functional outcomes in both cross-sectional and prospective outcomes.1 More recently it has become apparent from epidemiological work using the Danish registry that ADHD is also associated with a significant increase in the odds for mortality, criminality, substance use, accidents and later adult psychiatric comorbidity (eg, schizophrenia).2 It is remarkable that despite multiple, specific and serious functional impairments in ADHD, screening and evaluation of functional impairment has yet to become routine in clinic settings or in research. A comprehensive review of child psychiatric treatment studies between 1996 and 2011 found that although 95% of studies focused on symptom and diagnoses as treatment outcomes, less than half (47.5%) of the studies included functioning/impairment as a treatment outcome.3 Ultimately, it is the magnitude of improved functioning that is predictive of long-term beneficial outcomes.4 Given the necessity for measures of functional impairment, the Weiss Functional Impairment Rating Scale (WFIRS) was developed to measure ADHD specific impairment via a self-report (WFIRS-S) and parent report (WFIRS-P).

This study is a conceptual, narrative review of measurement of functional impairment in ADHD based on review of WFIRS psychometric validations and ADHD treatment trials using the WFIRS as an outcome measure. We present the rationale for interest in functional outcomes and the methodological criteria for reliable measurement of functioning as ways to address these clinical and research needs. Cross-comparison of multiple international studies that have used the WFIRS are used to elucidate common characteristics of functional impairment in ADHD and patterns of specific domain and overall response to targeted interventions. From this history and current research, we propose a model for definition of functional response and functional remission to further future clinical evaluation and research.

Understanding functional impairment

The chief complaint that brings a patient to treatment is often related to a problem with functioning (eg, getting into trouble at school, not having friends, losing a job). While this is likely to be the patient’s focus in treatment, the physician’s focus has historically been more on treating symptoms and decreasing overall symptom severity. A patient comes to treatment hoping the doctor will treat their problem, while the physician hopes that the patient’s problem will respond to medical treatment of the presenting disorder’s symptoms. Multidimensional assessment looking at both symptoms and functioning facilitates the dialogue between the clinician’s perspective and the patient’s perspective. Just as a symptom rating scale complements the information obtained on mental status in the clinical interview, the use of a functioning and/or impairment rating scale complements information obtained in the clinical history. For example, adolescents may be embarrassed to report risky activities, which they may nonetheless endorse on a ‘confidential’ rating scale when specifically asked. Parents who complain that the teacher is reporting behaviour problems of ADHD symptoms that are not present at home may nonetheless endorse concrete aspects of functional impairment on a scale.

The Diagnostic and Statistical Manual of Mental Disorders Fifth Edition (DSM-5)5 recognises the salience of including a functional perspective in diagnosis, while at the same time offers limited guidance about how this can be operationalised.6 The DSM-5 notes that the task force and WHO expressed the need ‘to separate the concepts of mental disorder and disability (impairment in social, occupational or other important areas of functioning).’5 The International Classification of Functioning, Disability and Health (ICF) provides a comprehensive framework for description of functioning, and has recently begun a process to develop an ICF Core Set for ADHD across the lifespan.7 This work is essential to defining both generic domains of functional impairment and any areas of impairment that may be specific to ADHD. Measures of functional impairment that allow for scoring in distinct domains allow us to evaluate how different interventions compare in effectiveness on specific targets, thus allowing for personalised treatment planning. For example, parent training may have differential impact on family functioning or life skills, while educational interventions may have a differential impact on learning. In the same way, there may be some areas of functioning that may show sluggish response to a particular intervention.

Symptoms, functioning, adaptive life skills and quality of life (QoL) describe distinct outcomes that have been understood loosely, and therefore sometimes are confused or intermingled. Symptoms can be defined as a set of characteristics, which together describe the manifestation of a clinical disorder. Functional impairment can be defined as the real-life consequences of the disorder. Adaptive life skills are abilities acquired through development that are necessary for the daily tasks of independent living. QoL is an umbrella concept inclusive of symptoms and impairment, as well as well-being and life satisfaction, and evaluated with measures such as the Child Health Illness Profile: Child Edition8; defining symptoms, functioning and QoL has facilitated research looking at their inter-relationships and how these domains are often intermingled.6 Coghill et al 9 showed small to moderate correlations between ADHD symptoms, QoL and ADHD functional impairment which varied by the type of drug treatment (ie, methylphenidate, amphetamine and non-stimulant) and by differential response between measurement domains. The authors noted that these three domains ‘assess partially intersecting but distinct aspects of the response to pharmacological treatment.’9

The relationship between change in symptoms and change in functioning is of considerable clinical relevance. If the patient has symptom improvement or even succeeds in reaching symptom remission, but remains functionally disabled in a particular domain, this would indicate the need for additional treatment. Alternatively, some children may be symptomatic but are functioning well in the demands of their environment. Previously, it was assumed that the symptoms were a reliable proxy for overall functioning, but this assumption is only partially and sometimes true.

It is necessary to look at functional improvement as an independent construct against what is defined as symptom improvement and symptom remission if we are to determine how well symptom change serves as a proxy for real-life impact of a treatment. If symptom remission is defined arbitrarily as a 50% drop in symptoms, but in reality, most of these children remain significantly impaired, this suggests that the ‘remission’ construct appertains more to the criteria for the disorder as measured by a scale, rather than the patient’s well-being. One investigation specifically attempted to validate the criteria for ADHD remission against actual functional outcomes, and found that patients who achieved remission via two differing definitions based on an ADHD symptom severity had significantly greater functional improvement at 8 and 24 weeks of follow-up than non-remitters.10 While these findings are reassuring in demonstrating that treatment of symptoms has downstream impact on functioning and well-being, they also illustrate that optimisation of functional improvement can only be achieved if we measure and identify those areas of impairment that are problematic. Examples might include life skills that require organisation skills training, deficits in self-esteem that indicate the need for psychotherapy, or family conflict that would benefit from parent training or family therapy.

A recent review describes the psychometrics of various multidimensional measures of functional impairment in ADHD and their utility in recent pharmacological and behavioural clinical trials for ADHD.11 Generic measures such as the Children’s Global Assessment Scale12 and the Columbia Impairment Scale13 show overlap with symptoms and QoL. These may be better understood as measures of overall clinical severity than specific to functional impairment. Similar difficulties are found with ADHD specific measures. The Impairment Rating Scale includes questions pertaining to peers, family, academics and self-esteem, but each domain is represented by a single item and the scale loads as a single factor.14 15 The ADHD-FX scale examines school, home and peer functioning, but is not specific to functional impairment and does not include a designated time frame.16 The Barkley Functional Impairment Scale (BFIS) for children and adults assesses impairment over the past 6 months.17 This is the only measure of functional impairment that includes population norms. The BFIS is designed to look at absolute impairment as a trait, rather than relative impairment as a state that fluctuates as symptoms change. The BFIS has a 6-month time frame, limiting its ability to be sensitive to change in brief intervention periods.

In our opinion, to better understand the impact of ADHD on functional impairment, a measure needs to be specific to functioning and exclusive of symptoms. It has to be informative of specific functional domains (so as to allow for personalised treatment planning), and also overall functioning as a cross-cutting concept. The measure needs to be time and treatment sensitive. To allow standardised comparison across domains and informants, it is necessary that each domain be rated as a standardised mean score. Clinicians in practice need a measure that is readily available, free of charge and easy to interpret. A measure requires appropriate psychometric validation in both the research and the clinical populations in which it will be used. Psychometric validation must include empirical definitions of improvement as well as cut-offs from normal populations. A measure should include comparable versions appropriate to self, parent and collateral informants. To assure appropriate usage in varying cultures, a measure of functional impairment needs to include a breadth of potential items, while being scored in such a way as to only rate those items that are relevant to that subject. This feature increases the utility of the scale across different levels of development, where particular items that are developmentally inappropriate are not calculated in the total score. Allowing for items to not be included in the total calculation avoids having items that are irrelevant being scored as normal, thus inflating the patients’ scores to show less impairment than actually exists. Moreover, a measure of functional impairment assumes that patients can reliably understand, perceive and report on the extent to which emotional difficulties impact their ability to function. Lastly, measurement of functional impairment might require both self-report and collateral report, particularly in adolescents. For example, parents may be naïve to high-risk activities their children have not revealed to them, where an adolescent self-report may under-report life skills or other deficits which they deny or externalise.

WFIRS overview

The methodological requirements described above were taken into consideration when the WFIRS was developed.18 The WFIRS was authored by the first author of this paper, and first published by the Canadian Attention Deficit Disorder Association as a measure of functional impairment in ADHD in their first edition guidelines, where it came into widespread public use.19 The measure is considered to be specific to ADHD only because it was developed from an ADHD population.

The WFIRS-P consists of 50 items and the WFIRS-S consists of 69 items. The WFIRS-P and WFIRS-S are not parallel forms, but there are many parallel items. The WFIRS-P collects the parent’s perspective of their child’s overall functioning across six domains: Family (10 items), School (10 items), Life Skills (10 items), Child’s Self-Concept (3 items), Social Activities (7 items) and Risky Activities (10 items). The WFIRS-S items collect the reporter’s perspective of their own functioning across seven domains: Family (8 items), Work (11 items), School (10 items), Life Skills (12 items), Self-Concept (5 items), Social (9 items) and Risk (14 items). Across both measures, the instructions are to rate each item according to the extent to which emotional or behavioural problems have impacted functioning in the last month on a 0–3 Likert scale ranging from ‘not at all or never’ to ‘very much or very often’ as well as a ‘not applicable’ (NA) option. The anchor points include both descriptors of severity and frequency since some items may be rare but have serious consequences (eg, physical aggression), while other items might occur frequently with modest impact (eg, does not get along with siblings).

Two methods of scoring were created to address distinct clinical and research needs. Clinicians who are using the WFIRS as a clinical tool to quickly identify patient impairment can easily do a visual scan of the measure for those domains with more items rated 2 or 3. Any domain that has one item rated 3 ‘very often or very much’ or two items rated 2 ‘often or pretty much’ would be considered to be impaired. As per the DSM-5 diagnostic criterion for ADHD, the clinician can then quickly determine whether the patient meets the required functional impairment in at least two domains or settings. The mean score allows for immediate comparison between domains, as well as comparison between informants. Using the mean, rather than a sum of the items, ensures items marked NA are not included in the computation of the overall score.

The WFIRS has grown to become an internationally used measure of ADHD-related functional impairment. The WFIRS has been translated using rigorous translation methodology that involves both forward and back translation.20 Collectively across the WFIRS-S and WFIRS-P, the measure has been translated into 18 languages (Swedish, Chinese, Danish, English, French, European Spanish, American Spanish, Japanese, Norwegian, Thai, Urdu, German, Dutch, Italian, Turkish, Polish, Russian and Persian). Additionally, psychometric investigations have been conducted on the WFIRS using samples from Japan,21 Thailand,22 Iran,23 China,24 Turkey,25 Germany,26 the USA,27 and in a sample from clinical trials conducted in North America, Australia and Europe.28

This study is a narrative review of validation studies of the WFIRS and clinical trials using the WFIRS as an outcome measure. There are two objectives for this review. The first is to elucidate the methodological criteria for measurement of function. Having established how to measure functioning, we then turn to the results of clinical trials of treatment of ADHD that have attempted to do so. These trials are used to explore the characteristics of functional impairment in ADHD and examine possible patterns of domain-specific functional impairment associated with the disorder. We also look at treatment response across different interventions and cultures. Lastly, the authors propose a standardised definition of reporting functional response, functional remission and symptom improvement and remission in clinical trials.

Psychometric validation studies of the WFIRS

Psychometric validation refers to demonstrating that a measure actually measures what it is designed to measure and does so in a reliable way. Inter-rater reliability determines the degree to which reports by different informants are consistent. Parallel forms reliability refers to the consistency between multiple versions of a questionnaire. Test–retest reliability refers to the consistency of observations over time, assuming no other changes are occurring. Internal consistency looks at the extent to which different items on the measure refer to the same construct. Convergent validity looks at how well the measure correlates with other measures that are designed to measure the same construct. Discriminant validity looks at how the measure discriminates from constructs that are unrelated. This section reviews the different psychometric validation studies that have been done on the WFIRS with the specific objective of obtaining a better conceptual understanding of functional impairment in ADHD.

Table 1 provides a summary of the findings from published WFIRS psychometric investigations, three for the WFIRS-S and five for the WFIRS-P, and one that used both reports.22 Studies were extracted from searches performed in MEDLINE, EBSCO, and the first 100 articles in Google Scholar. Additionally, studies were collected from review of the reference sections of included studies, and personal communication from the authors. Suitable articles found that were not in the English language were translated and the original authors were contacted if further clarification was required. Inclusion criteria for the validation search were studies examining the psychometric properties of the WFIRS based on the following search terms: [Weiss Functional Impairment Rating Scale, WFIRS] AND [psychometric, validation] and all validation studies of the WFIRS we identified were included. No studies from this search were excluded. All of the validation studies were done on ADHD and comparator normal control populations. Parameters for describing the magnitude of relationships were based on the following: very strong = ≥±0.9, ±0.7 ≤ strong <±0.9, ±0.5 ≤ moderate <±0.7, ±0.3 ≤ weak <±0.5, very weak= ≤±0.3.29

Table 1

Review of Weiss Functional Impairment Rating Scale psychometric investigations

Study	Weiss et al 30†	Qian et al 24	Gajria et al 28‡	Punyapas et al 22	Tarakçıoğlu et al 25	Dose et al 26	Canu et al 27	Takeda et al 21	Hadianfard et al 23
Measure	WFIRS-P	WFIRS-P	WFIRS-P	WFIRS-P, WFIRS-S	WFIRS-P	WFIRS-P (excluded Risky Activities)	WFIRS-S	WFIRS-S	WFIRS-S
Location	Canada	China	North America, Australia, Europe	Thailand	Turkey	Germany	USA	Japan	Iran
Sample (n)	Parents of youth with ADHD (209)	Parents of youth with ADHD (123) and parents of normal youth (240)	Parents of youth with ADHD from pharmacological clinical trials treating ADHD (2357)	Parents (143) and their youth with ADHD (137)	Parents of youth with ADHD (250) and healthy youth (250)	Parents of youth with ADHD and/or ODD from randomised controlled trials examining a telephone-assisted self-help programme (264)	Undergraduate adult college students (2098–16.5% met research cut-off for ADHD) and collateral reporters (CR) (262)	Outpatient adults with ADHD (46), adult university students (889), adults without psychiatric disorders (104)	Public secondary school students (386)
Age range (years) (% male)	6–11	ADHD 6–15 (84), non-ADHD group 6–16 (76)	6–17 (75)	4–18 (78)	(M±SD) ADHD 9.61±2.18 (77), non-ADHD 9.85±2.12 (77)	4–12 (80); parent’s M age 38.83 years (4)	18–25 (33)	ADHD 19–53 (52), students 18–45 (53), non-psychiatric 22–65 (38)	12–18 (50)
Average total score (SD)	1.1 (0.4)	ADHD 0.77 (0.35), non-ADHD 0.29 (0.26)	1.03 (0.47), 1.01 (0.45)	–	–	1.00 (0.44)	ADHD 0.88 (0.49), non-ADHD 0.35 (0.28)	–	0.31 (0.29)
Internal consistency (α)	Very strong: Family, Total Strong: School, Life Skills, Self-Concept, Social Activities, Risky Activities	Strong: Total and all domains	Very strong: Family, Total Strong: School, Self-Concept, Social, Life Skills, Risky Activities	Very strong: WFIRS-P School, WFIRS-P Life Skills, WFIRS-P/S Self-Concept, WIFRS-S Social, WFIRS-P Risky Activities, WFIRS-P/S Total Strong: WFIRS-P/S Family, WFIRS-S Work, WFIRS-S School, WFIRS-S Life Skills, WFIRS-S Social, WFIRS-S Risky Activities	Very strong: Total Strong: Family, School, Self-Concept, Social, Life Skills, Moderate: Risky Activities	Very strong: Total Strong: Family, School, Life Skills, Self-Concept, Social	Very strong: Total, Work, School, Self-Concept, Social Strong: Family, Life Skills, Risky Activities	Very strong: Self-Concept Strong: Family, Work, School, Life Skills, Social, Risky Activities	Very strong: Self-Concept, Total Strong: Family, Work, School, Life Skills, Social, Risky Activities
Interdomain relationships	–	–	–	–	All domain-to-domain relationships were <r=0.67	–	–	–	All domain-to-domain relationships were <r=0.66
Test–retest reliability (1) Time between ratings (2) Strength (r or ICC)	–	(1) 1–2 weeks (2) Strong: Family, School, Life Skills, Social Activities Moderate: Self-Concept, Risky Activities	(1) 2–3 weeks (2) Strong: Family, School, Life Skills, Self-Concept, Social Activities, Total Moderate to strong: Risky Activities	(1) 2–3 hours (2) Strong: WFIRS-P Total Moderate: WFIRS-S Total	(1) 4 weeks (2) Very strong to strong: Total and all domains	–	–	(1) 2 weeks (2) Strong: Family, Work, School, Self-Concept Moderate: Life Skills, Social Activities, Risky Activities	(1) 2 weeks (2) Strong: School, Self-Concept, Social Activities, Risky Activities, Total Moderate: Family, Work, Life Skills
Concurrent/convergent validity (r)	Moderate: Total—CHIP:CE Risk Avoidance Weak: Total—ADHD-RS-IV Total, Total—CHIP:CE Satisfaction, Comfort, Resilience, Achievement Very weak: Total—GAF, Total—CGI-S	Weak: Total—ADHD-RS-IV Total, all domains—ADHD-RS-IV Total	Moderate to weak: Total—ADHD-RS-IV Total Moderate to very weak: Total—CGI-S	–	Strong: Total—CGI-S Moderate: Total—PedsQL Total, Total—PedsQL Psychosocial Health, Total—CGAS, Total—T-DSM-IV-S ADHD Total Weak: Total—PedsQL Physical Health	–	Strong: Total—CSS Impairment Total Weak: School—self-reported GPA	Strong: Self-Concept—RSES-J, School—GAF, CAARS Moderate: Family, Work, Life Skills, Self-Concept Social—GAF, CAARS Very weak: Risky Activities—GAF, CAARS	Strong: Total—PedsQL Total, Total—PedsQL Psychosocial Health Weak: Total—PedsQL Physical Health
Discriminant/divergent validity (r)	–	Weak to very weak: School, Life Skills—ADHD-RS-IV Hyperactivity impulsivity ADHD group had higher scores in all domains and Total score compared with non-ADHD group***	–	–	Total and domain scores discriminated between groups categorised by overall ADHD severity: control group<mild<moderate<severe***	Moderate: Total—FBB-ADHS Total, FBB-SSV ODD	Only School and Work were more strongly associated with inattention when compared with relationships with depression, anxiety and stress, supporting the WFIRS as an ADHD-specific impairment measure. ADHD group had higher impairment across more domains compared with non-ADHD group.	Moderate: Self-Concept—BDI Weak: Family, Work, School, Life Skills, Social Activities, Risky Activities—BDI Total and all domain scores were significantly higher in the ADHD group compared with the other groups***	–
Confirmatory factor analysis		(school domain was excluded) Five factors; ADHD group CFI=0.89, RMSEA<0.08. Non-ADHD group CFI=0.97, RMSEA<0.08.	Six factors; CFI=0.79–0.86, RMSEA=0.084–0.094	–	Seven factors; CFI=0.95, RMSEA=0.061	Five factors; CFI=0.93, RMSEA=0.05	–	Seven factors; CFI=0.91, RMSEA=0.043	Seven factors; CFI=0.60, RMSEA=0.08. When 12 items were removed, CFI=0.70, RMSEA=0.07

This does not represent a systematic review of all studies.

Magnitude definitions29: very strong= ≥±0.9, ±0.7 ≤ strong <±0.9, ±0.5 ≤ moderate <±0.7, ±0.3 ≤ weak <±0.5, very weak= ≤±0.3.

*P<0.05; **P<0.01; ***P<0.001.

†Only baseline data for Weiss et al [30] are reported.

‡Gajria et al [28] sample was randomly split into two groups for replication and analysed at baseline and a follow-up visit.

ADHD, attention-deficit/hyperactivity disorder; ADHD-RS-IV, ADHD Rating Scale Version IV; BDI, Beck Depression Inventory; BRIEF, Behavior Rating Inventory of Executive Function; CAARS, Conners’ Adult ADHD Rating Scale; CFI, comparative fit index; CGAS, Children’s Global Assessment Scale; CGI-S, Clinical Global Impressions-Severity; CHIP-CE, Child Health Illness Profile: Child Edition; CSS, Current Symptom Scale; DASS, Depression Anxiety Stress Scale 21; FBB-ADHS, Symptom Checklist for Attention-Deficit/Hyperactivity Disorder; FBB-SSV, Symptom Checklist for Oppositional Defiant and Conduct Disorder; GAF, Global Assessment Functioning; ICC, intraclass correlation coefficient; M, mean; ODD, oppositional defiant disorder; PedsQL, Pediatric Quality of Life Inventory; RMSEA, root mean square error of approximation; RSES-J, Rosenberg Self-Esteem Scale Japanese Version; SDS, Sheehan Disability Scale; T-DSM-IV-S, Turgay’s DSM-IV-based ADHD and Disruptive Behavior Disorders Screening Scale-Parent; WFIRS-P, Weiss Functional Impairment Rating Scale-Parent; WFIRS-S, Weiss Functional Impairment Rating Scale-Self.

Weiss Functional Impairment Rating Scale-Self

The samples used in the WFIRS-S investigations included adult university students,21 27 adults with ADHD, adults without any psychiatric disorders,21 an adolescent public school sample23 and youth with ADHD.22 The average WFIRS-S Total score for a subset of adult students with ADHD was 0.88 and adult students without ADHD was 0.35.27 The non-ADHD average score found in Canu et al’s27 study is similar with the average score of 0.31 found in the non-psychiatric public school sample of adolescents.23 No research has been done to date, however, to determine the receiver operating characteristics (ROC) that would best distinguish clinical from non-clinical samples on the WFIRS-S.

Despite the differences of settings and sample populations used, the internal consistency of the WFIRS-S Total was strong across all investigations and moderate for particular domains. Test–retest reliability found moderate to strong correlations between the ratings of WFIRS-S domains and Total score across varying time points. Canu et al 27 created a collateral report version of the WFIRS and found strong internal consistency and small to moderate cross-informant reliability between their student sample and collateral reporters. The WFIRS demonstrated higher levels of impairment in those with ADHD across two investigations. When compared with another established measure of impairment (ie, Current Symptom Scale), the WFIRS-S demonstrated a strong relationship. The WFIRS-S showed a moderate relationship with the Global Assessment Functioning and Conners’ Adult ADHD Rating Scale21 on most domains, and a strong correlation with the Pediatric Quality of Life Inventory (PedsQL) Total and Psychosocial Health subscale.23 In relation to divergent validity, the WFIRS demonstrated a relatively weak relationship with a measure of depression.21 Confirmatory factor analysis (CFA) confirmed a seven-factor solution as specified in the WFIRS-S.21 23

Weiss Functional Impairment Rating Scale-Parent

Similar to the diversity of investigations for the WFIRS-S, the WFIRS-P has been psychometrically investigated using varying populations in Canada,30 China,24 Turkey,25 Germany,26 Thailand,22 and in a large sample drawn from multiple research studies conducted in Europe, North America and Australia.28 The WFIRS-P Total yielded strong internal consistency and was moderate to high for all domains, with the lowest reports for the Risky Activities domain in the Turkish sample (α=0.56). Additionally, four of the studies conducted test–retest reliability and found strong correlations between the two ratings across varying time point assessments. The WFIRS-P demonstrated moderate convergent validity with the PedsQL Total and Psychosocial Health subscale and weak with the Physical Health subscale.25 In relation to an ADHD symptom severity measure (ie, ADHD Rating Scale Version IV) and overall illness severity (ie, Clinical Global Impressions-Severity (CGI-S)), the WFIRS-P was moderate to weakly related.24 28 However, in Tarakçıoğlu et al’s25 study the CGI-S was strongly related with the Total score (r=0.71). The WFIRS-P demonstrated higher scores in those with ADHD when compared with those without ADHD24 and scores were distinguishable across categorised overall illness severity.25 A CFA was conducted in four WFIRS-P investigations although using different approaches. The root mean square error of approximation confirmed a five-factor model,26 a six-factor model28 and a seven-factor model which distinguished school learning and behaviour separately.25

Implication of WFIRS validation studies for understanding functional impairment

In summary, robust psychometric properties were found from the validation studies of both WFIRS-S and WFIRS-P despite differences in countries, populations (ie, clinical vs normal), settings (ie, clinical vs research), age and informants. The studies consistently showed robust and consistent psychometric properties including internal consistency, convergent and discriminant validity, test–retest reliability and CFA.

The consistency of the psychometric properties of the WFIRS measure noted above is only possible if various characteristics of functional impairment are also consistent across these samples and settings. While some findings were similar despite differing cultures and settings, it should also be noted that there may be other outstanding cultural differences. Altogether, these psychometric findings provide insight into the reliability and validity of the measure, and insight into some of the characteristics of functional impairment in ADHD per se.

It is a common assumption that individuals function differently across settings. This contradicts the findings of statistically significant interdomain correlations and the robust correlation between each domain and the measure as a whole (eg, school and home functioning being related).23 25 This suggests that patients tend to have an ‘impairment factor’ which is reflected across different areas. This is similar to the ‘g factor’ commonly mentioned with regard to IQ tests. This being said, what captures both the interest of the patient and the clinician are those specific areas of impairment that could potentially be remediated by treatment intervention.

The relatively high internal consistency within most domains and the Total score suggests that there is a clear construct of perceived impairment that cuts across very different specific iterations used to operationalise impairment in a particular domain. The Risky Activities domain had strong to very strong internal consistency, except in one investigation (ie, <0.7)25; this domain is also characterised by a lower mean score and more items marked as ‘not applicable’. Despite floor effects of some items in this domain,21 23 25 the domain of Risky Activities was included in the scale because it provides essential clinically salient information. What is notable and of considerable clinical relevance is that this domain, intended to capture serious but infrequent difficulties, has stable characteristics across age groups and is sensitive to change and to treatment even in short duration trials.28 This finding would suggest that the often made assumption (as in Dose et al 31) that Risky Activities is predominantly a problem of adolescents is not correct.

It should be noted that the absolute value of lower mean scores is not necessarily indicative of life impact. The low frequency of risky activities may contribute to lower mean scores, while even the infrequent occurrence of such activities may still have serious consequences. By the same token, if despite low frequency, floor effects and lower mean scores, the domain of Risky Activities is sensitive to treatment effects, this may have considerable clinical relevance for improving long-term outcomes.

The WFIRS shows high test–retest reliability, which implies that when patients or other informants are asked to report on how emotional or behavioural problems have affected functioning, they can provide a reliable response. Furthermore, patients and parents are also able to report change in functioning over time. Prior to the data analyses done for the clinical trials described below, it was a common and mistaken assumption that while symptoms change with stimulant treatment over the course of a few weeks, it would take months for a change in symptoms to translate into a change in functional impairment.

The studies described below generally show that the timing of change in ADHD symptoms and functional impairment is moderately correlated and change is evident within a relatively short time frame such as in short randomised placebo-controlled trials. Studies using repeated measures designs which find moderate to large effect sizes demonstrate the capacity of informants to reliably report functional improvement over time. Furthermore, this supports the notion that patients and collateral informants can reliably report the perceived impact of symptoms on functional impairment.

Functional outcomes in treatment studies

The WFIRS has been used among pharmacological, psychological and multimodal outcome studies as summarised in table 2. A review of the outcomes from these studies allows us to identify patterns of change in functional impairment and impairment characteristics across treatment modalities. Variables of interest include: the time course of treatment, comparison of baseline scores and change in domain/total scores. Using the model of how we examine symptom outcomes, we also report the effect sizes of functional improvement and examine whether functional outcomes showed response or remission.

Table 2

A review of selected attention-deficit/hyperactivity treatment trials using the Weiss Functional Impairment Rating Scale as an outcome measure

Study	Location	Intervention	Design	Treatment	Endpoint*	Age range (n)†	Measure	Domains with statistically significant improvement
Maziade et al 35	Canada	PHARM	OL	ATX	6 months	6–11 (16)	WFIRS-P	T, F, S, LS, SC
Stein et al 48	USA	PHARM	RCT	ER d-MPH, ER MAS	8 weeks	9–17 (65)	WFIRS-P	T, F, S, Soc, RA
Hantson et al 36	Canada	PSY	OL	Summer treatment programme	3 weeks	6–12 (48)	WFIRS-P	All, large ES
Banaschewski et al 39	Europe	PHARM	RCT	LDX, OROS-MPH, placebo	7 weeks	6–17 (336)	WFIRS-P	All, largest ES in S for LDX
Fuentes et al 49	Europe, Mexico	PHARM	OL, RCT	ATX, OEST	6–12 months	6–16 (399)	WFIRS-P	All
Meisel et al 41	Spain	PSY+PHARM	RCT	Neurofeedback, MPH	5 months or 40 sessions	7–14 (23)	WFIRS-P	T in both groups with large ES
Zavadenko and Suvorinova 50	Russia	PHARM	OL	Hopantenic acid	6–8 months	6–12 (32)	WFIRS-P	All
Banaschewski et al 51	Europe, USA	PHARM	OL, RWP	LDX, placebo	26 weeks OL, 6 weeks RWP	6–17 (153)	WFIRS-P	OL: All, greatest S
Hervas et al 38	Europe, USA, Canada	PHARM	RCT	GXR, ATX, placebo	10–13 weeks	6–17 (338)	WFIRS-P	T, S, F, Soc
Montoya et al 40	Spain	PSY+PHARM	RCT	MPH/ATX+psychoeducation, MPH/ATX	12 months	6–12 (208)	WFIRS-P	No significant differences between groups
Gandía-Benetó et al 52	Spain	PHARM	OL	LDX	9 months	6–18 (41)	WFIRS-S	None
LaCount et al 37	USA	PSY	OL	CBT	20 hours in 10 weeks	18–38 (17)	WFIRS-S	Among completers, S, W (F, Soc, RA were excluded)
Su et al 10	China	PHARM	OL	OROS-MPH	16 weeks	6–16 (205)	WFIRS-P	Symptom remitters greater functional improvement except RA
Stein et al 34	Canada, USA	PHARM	RCT	GXR, placebo	8 weeks	6–12 (333)	WFIRS-P	T, F, S, Soc, RA; change in scores was congruent with symptom improvement
Vidal et al 53	Spain	PSY	RCT	Group CBT, waitlist control	12 sessions	15–21 (119)	WFIRS-S, WFIRS-P	CBT had greater change via the WFIRS-P; at baseline, impairment was greater on WFIRS-S than WFIRS-P
Wilens et al 33	USA	PHARM	RCT	GXR, placebo	13 weeks	13–17 (401)	WFIRS-P	None
Nagy et al 54	Europe, USA, Canada	PHARM	RCT	LDX, ATX	9 weeks	6–17 (267)	WFIRS-P3	LDX>ATX T, S and Soc but all domains improved in both groups
Newcorn et al 55	Europe, USA, Canada	PHARM	OL, RWP	GXR, placebo	13 weeks OL, 26 weeks RWP	6–17 (316)	WFIRS-P	RWP: the placebo group had significant increases S
Dose et al 31	Germany	PSY+PHARM	RCT	TASH+MPH, MPH	12 months	6–12 (103)	WFIRS-S	Among TASH completers, all except S (RA was excluded)
Ni et al 56	Taiwan	PHARM	RCT	MPH, ATX	8–10 weeks	18–50 (63)	WFIRS-S	All
Zavadenko et al 57	Russia	PHARM	RCT	Hopantenic, placebo	4 months	6–12 (89)	WFIRS-P	S, SC

This does not represent a systematic review of all studies. All samples were participants with attention-deficit/hyperactivity disorder (ADHD).

*Endpoint is defined as the last visit with valid data.

†Either randomised n or completer n.

ATX, atomoxetine; CBT, cognitive–behavioural therapy; ER d-MPH, extended release dexmethylphenidate; ER MAS, mixed amphetamine salts; ES, effect size; F, Family domain; GXR, guanfacine extended release; LDX, lisdexamfetamine dimesylate; LS, Life Skills; MPH, methylphenidate; OEST, other early standard therapy; OL, open label; OROS-MPH, osmotic release oral-system methylphenidate; PHARM, pharmacological; PSY, psychological; RA, Risky Activities; RCT, randomised controlled trial; RWP, randomised withdrawal period; S, School; SC, Self- Concept; Soc, Social; T, Total score; TASH, telephone-assisted self-help; W, Work; WFIRS-S/P, Weiss Functional Impairment Rating Scale-Self/Parent.

Studies using the WFIRS as an outcome measure in a treatment trial of ADHD were targeted. The same databases and methodology described above for the psychometric studies were used, but with the following search terms: [Weiss Functional Impairment Rating Scale, WFIRS] AND [ADHD, attention-deficit/hyperactivity disorder] AND [treatment, medication, stimulant, pharmacotherapy, behavioral, trial, outcome]. All studies that were identified from this search and were treatment trials of ADHD were included. Effect sizes were interpreted with the following parameters: small ≤0.20, moderate 0.3–0.5 and large ≥0.60.32

Across the pharmacological studies selected, symptom improvement was associated with clinically and statistically significant change over the course of treatment in overall Total score in all the studies, except for a study of parent report on adolescent response to guanfacine extended release (GXR) which found clinical but not statistically significant improvement.33 The most robust improvement overall for stimulant studies was found in the School (learning/behaviour) domain. Life Skills and Self-Concept, and in some studies Risky Activities, were the three domains with the most sluggish response to pharmacological intervention, suggesting the possibility that these domains may have a decreased or slower response to stimulant treatment, or that they require additional intervention in their own right.

It is possible that the time course to obtain a full response for particular functional outcomes is longer than symptom response, particularly with non-stimulants. A trial of GXR that failed to show response of these two domains in the randomised trial did demonstrate response during open-label follow-up.34 In a 6-month open-label pilot study of atomoxetine (ATX), significant improvement was found in Family, School and Life Skills after 2 months and at study endpoint (ie, 6 months), whereas improvement in Self-Concept was only significant at study endpoint.35 It is also possible that these domains require a longer period of time for caregivers to be able to accurately observe and thus report change.

Outcome studies suggest differences in domain response and time course of functional response between stimulant and non-stimulant medication.9 Coghill et al 9 found the effect size for change in functional impairment showed drug-to-drug differences: lisdexamfetamine (0.92), osmotic release oral-system methylphenidate (0.77), GXR (0.44) and ATX (0.28). The relatively slower timeline to respond and lower effect sizes for functional outcomes of non-stimulant medications may in part be driven by a subtler symptom response and relative absence of clear on and off comparison over the course of the day. Patients on non-stimulant medications are less likely to have rebound or weekend drug holidays and so parents may require more time to be aware of and able to report improvements in functioning.

Despite widespread use of psychosocial interventions to target specific areas of functional impairment, only a limited number of psychosocial trials using the WFIRS were found. In a randomised controlled trial of adolescents assigned to cognitive–behavioural therapy (CBT) or a waitlist control using both the WFIRS-P and WFIRS-S, significant improvement was only found on the WFIRS-P Total score. This raises the important possibility that parents and adolescents may perceive and report impairment differently, and that different interventions may capture different types of improvement from parent versus adolescent report. In an open-label comparison of functional outcome on the WFIRS-P in a 2-week summer treatment programme targeting self-regulation, social skills and parent psychoeducation and training, there was improvement across all functional domains.36 An open-label study of CBT for college students showed improvement in both attention symptoms and the Work and School domains.37

The only study via a post hoc analysis of Hervas et al 38 and Banaschewski et al 39 of the relationship between symptom, QoL and functional outcomes with different treatments found ‘not only that control of ADHD symptoms was associated with improved functioning and HRQoL (health related quality of life) in children and adolescents with ADHD in these studies, but also that the symptom-based scale may not have captured a complete picture of the treatment response.’9 Differences in domain response between stimulants and non-stimulants may reflect the time course of effect over the course of the day, as well as the profile of ADHD and non-ADHD symptoms that they target. A child who has good symptom control in the evening or early morning, as opposed to being either premedication or in rebound, may show selectively greater improvement in the family domain since family interactions typically occur in the early morning or evening.

Coghill et al 9 interpret domains that are highly sensitive to medication (School, Family, Total) as more proximally related to ADHD symptoms, and domains that are less sensitive (Life Skills, Self-Concept and Risky Activities) as ‘more distally related to ADHD symptoms’. It is just as likely that the issue is not the relationship between ADHD symptoms and the functional outcome, but rather that medication treatment of symptoms may not address these impairments. A study of functional outcome of organisational skills training might find a close proximal relationship with change in attention items. Children with ADHD might have a positive illusory bias (eg, an overly positive view of themselves), and may not report impairment in self-concept until they have a more accurate awareness of self. An intervention designed to address self-awareness in ADHD might demonstrate a close proximal relationship with self-concept and ADHD symptoms.

Future research might then want to look at how direct intervention for symptoms, functioning or the combination leads to differential outcomes in either. There are a few such studies. The addition of a telephone-assisted self-help parenting programme to methylphenidate significantly improved functional impairment, oppositional defiant disorder symptoms and negative parenting beyond medication alone.31 Some studies directed at functional targets may still impact symptom outcomes. For example, a study of psychoeducational intervention improved symptoms but not functional impairment.40 A comparative study looking at symptoms and functioning as outcomes in neurofeedback versus methylphenidate found that both improved.41 It would be of considerable interest to know whether sequential treatment to target residual functional impairment after administration of medication might successfully target those domains which are less sensitive to medication alone.

A paradigm for defining functional improvement and remission

This review of outcome studies suggests the need for a standardised paradigm for analysis and interpretation of functional response to treatment. This would allow comparison across treatment interventions, assessment of domain differences with different types of intervention and time course of response. Most of all, we need to know how our standardised definitions of response and remission of symptoms are related to response and remission of functioning.

Symptom improvement or ‘responders’ is most often understood as a 30% reduction in symptoms as measured on 18-item symptom scales of ADHD42 or a score of ‘much improved’ or ‘very much improved’ on the Clinical Global Impression Improvement scale.43 Swanson  et al introduced the concept that over and above looking at whether or not an intervention led to improvement, it would also be of considerable interest to know whether a child had achieved ‘normalization’ of symptoms, which he defined as a mean score ≤1. The rationale for selecting this cut-off for ‘remission’ was based on the finding that this captured 87% of a control population.44

It should be noted when considering the rationale for response/remission criteria, improvement as measured by per cent change in symptoms is highly sensitive to baseline severity. A severe patient may have a 50% change in symptoms, but still be symptomatic. A mild patient may have a 30% change in symptoms and achieve normalisation. Therefore, there is value in looking at outcomes both from the point of view of per cent change and also from the point of view of treatment endpoint. This is especially true in that results are reported as scores on all 18 items, even for patients where a more purely inattentive or hyperactive presentation means that looking at total ADHD symptom score may fail to capture the full extent of improvement in the particular dimension that is clinically relevant. Despite consensus agreement on a paradigm for measurement of symptom response or remission, there are no studies looking at how these criteria relate to improvement in functioning, and no definitions of what can be considered meaningful improvement and remission in functioning.

We propose a standard for consensual definition of functional improvement and functional remission in outcome of ADHD that is developed out of empirical criteria. Hodgkins et al 45 provide a full discussion of the use of minimal important difference (MID) to describe whether or not a clinically meaningful change has occurred, over and above description of statistical significance or even effect size. We propose that the MID is the best empirical standard for definition of improvement on any measure, since it is anchored on both statistical response and patient perception of meaningful change. This study identified that the MID for the WFIRS using multiple methods was a change in the total mean score of 0.25.45 This definition of the MID remained consistent across specific domains of the measure with the exception of Risky Activities, where the degree of change rated as an MID was found to be somewhat lower. It makes clinical sense that patients perceive even a small degree of improvement in Risky Activities as clinically meaningful, probably because these items have been selected to be markers of high risk.

The concept of ‘remission’ or ‘normalization’ has been variously defined as a 50% decrease in mean symptom score (or a great deal of improvement), or as a mean score ≤1.0 (the score that approaches what is seen in control populations). While the idea of normalisation may be complex in neurodevelopmental disorders that wax and wane, a standard for identifying treatment success has been of considerable heuristic value. There is no literature to define functional remission, nor has functional outcome been used as an anchor against which to identify the real-life value of our standard for symptom remission.

An ROC study on WFIRS scores in ADHD versus non-ADHD children found that a score of 0.65 accurately classifies functional impairment in ADHD versus non-ADHD children.46 A score less than 0.65 therefore means that the child no longer carries functional impairment secondary to ADHD that would accurately classify them as distinct from the normal population.46 We propose that this is a reasonable cut-off for remission on the WFIRS. A clinical limitation of this definition is that even if a patient is doing well enough in most domains to achieve an overall low score on the WFIRS, severe functional impairment even in one or more particular items or domains may still be clinically significant. Further ROC research studies are required in different populations and using different designs to see if this cut-off remains appropriate.

It is recommended that data analysis of functional impairment in clinical trials target functional improvement (a change score of >0.25) and functional remission (a final score <0.65). This would then allow us to see what fraction of the patients who are symptom responders or symptom remitters were also either functional responders or in functional remission. It would then become clear whether or not symptom remission actually translates into clinically meaningful functional change and in what way.

Further research is needed for an empirical or consensual method of defining functional improvement and functional remission. There are no studies of the MID or ROC for the WFIRS-S as used in adolescents or adults. This could be explored against other potential definitions of functional improvement and functional remission, such as a 30% vs 50% drop in total score. Nonetheless, it will take thoughtful and rigourous research to examine the heuristic concepts of functional improvement and remission. This research would have considerable clinical impact in changing how we understand patient response to treatment.

Discussion

This study is limited to a selection of studies using a particular measurement tool (ie, the WFIRS) and to translation of foreign language studies, at a time when there are many descriptive, exploratory and treatment studies in process (personal communication). It is a limitation of the current study that the first author is also the developer of the measure. The strength of this narrative review is to establish both the conceptual foundation for the WFIRS in particular, and measurement of functional impairment in general. The WFIRS is now currently in use in non-ADHD studies (i.e., other psychiatric and medical disorders). Previous validation studies have shown the WFIRS has robust psychometric properties in non-ADHD and normal populations, so the principles elucidated in this review may be extended to more extensive measurement and evaluation of functional impairment in other disorders. For example, one might anticipate that patients with depression might report more difficulty with self-concept, while patients with autism might report more difficulty with social functioning. Measures of functional impairment that are non-specific, that is not limited to the functional deficits of only one disorder, allow for research on the differences in functional  impairment that differentiate different disorders from each other.

This review clearly points to the need for further research. Evaluation of the relationship between symptom improvement, remission and residual functional impairment will allow the ability to determine if symptom outcomes have as much real-life impact as hoped for. As more studies become available, a meta-analysis would be of potential value. The availability of multiple translations and validation of the WFIRS makes it possible to examine ADHD functional impairment in different cultures. The findings on cultural differences reported here are difficult to interpret since they may reflect differences in study design rather than culture per se. Data on the WFIRS in treated and untreated ADHD populations, as well as normative populations, could extend the empirical base for developing meaningful scoring procedures, as well as generating needed MID and ROC data for the WFIRS-S. The lack of evidence-based measures of functional impairment relative to illness has led to use and testing of the WFIRS in populations other than ADHD, which is going to allow for comparison of the impact of illness in various domains across a variety of clinical conditions. The availability of parent, self and collateral forms allows for more research on the differing perspectives of different informants, especially within adolescents. Furthermore, a teacher form of the WFIRS would aid in collecting an even more comprehensive view of overall functioning for youth. More studies are needed looking at domain-to-domain differences among different interventions. Assessment of the differential impact of different conditions on patient’s overall functional impairment may provide us with new insights of the interplay between commonly comorbid conditions.47 This new research will continue to provide us with a better lens to understand the functional impact of illness across cultures, age groups and informants.