Reliability and validity of the child and adolescent functioning impairment scale in children with attention-deficit/hyperactivity disorder.

OBJECTIVE: The purpose of the present study was to develop reliable and valid parent and teacher scales for measurement of functional impairment in children and adolescents in order to assist the diagnosis of attention-deficit/hyperactivity disorder (ADHD).
METHODS: Seventy-two children with ADHD fulfilling the Diagnostic and Statistical Manual of Mental Disorder, 4th Edition criteria and forty-two normal controls were enrolled in this study. Parents and teachers of the subjects completed the parent and teacher form of the preliminary items of Child and Adolescent Functioning Impairment Scale (CAFIS) made up by the authors. Based on the reliability and factor analysis, the final parent (CAFIS-parent form) and teacher version (CAFIS-teacher form) were constructed. Scales were analyzed for reliability and validity. Relative operating characteristics curve was drawn to calculate the cutoff scores of these scales for children with ADHD.
RESULTS: The CAFIS-parent and CAFIS-teacher forms consist of four and three factors, respectively. Internal consistency and test-retest correlation of the scales were satisfactory. The CAFIS and Children's Global Assessment Scale were significantly correlated. All scores of subscales of CAFIS in ADHD group were significantly higher than those of control group. The sensitivity and specificity of the subscales were mostly at an appropriate level.
CONCLUSION: The CAFIS is a brief layperson-administered scale to assess functional impairment of children and adolescents. It can be a useful tool for parents and teachers to objectively measure the functions of children at home and in school. This scale was found to be reliable and valid, and it appears to be a valuable instrument in Korean language.

INTRODUCTION

Diagnostic criteria for attention-deficit/hyperactivity disorder (ADHD) in the Diagnostic and Statistical Manual of Mental Disorder, 4th Edition (DSM-IV)1 is consisted of two dimensions: symptoms and impairment. The 'symptom' criteria include three areas (i.e., inattention, hyperactivity, and impulsivity) and specifies symptoms related to each area (e.g., often has difficulty organizing tasks and activities). However, the 'impairment' criteria do not provide a specific guideline, but describes simply that "there must be clear evidence of clinically significant impairment in social, academic, or occupational functioning)". The nebulous description of the definition of impairment in DSM-IV creates difficulties in assessing functional impairment of children, and therefore in the diagnostic procedures of ADHD.

The relationships between symptoms and impairment of a disorder need to be delineated. Some clinicians or researchers make diagnostic decisions predominantly based on the number and intensity of the symptoms reported, but not on the functional impairment. The assumption is that symptoms and impairments are closely associated to each other. Alternatively researchers and clinicians build the notion of impairment into their symptom rating. That is, one does not rate someone as "unable to concentrate" unless the patient is impaired by the symptom or behavior. However, there is a growing body of evidence suggesting that symptoms and impairment are certainly not identical, but are at best moderately related.2,3 In the early field trial, the number of children meeting criteria for psychiatric diagnosis fell dramatically when impairment measures were considered.4,5 Thus, overlooking impairment in the diagnosis of ADHD would lead to false positive diagnoses and increase of the prevalence. On the other hand, overemphasizing impairment can lead to false negative diagnoses and decrease of the prevalence. Some children have significant functional impairment yet do not meet the symptom criteria for psychiatric diagnoses.6,7 Specifically, Angold et al.6 reported that 40% of children who used specialty mental health services did not meet symptom criteria, but did have functional impairment. Clinical service use was concentrated in families who had a youth evidencing impairment whether or not the child met symptom criteria. More specifically, parents or teachers rarely complain about impairment associated with the problems related to ADHD symptoms such as 'often does not seem to listen when spoken to directly' or 'often talks excessively'. Rather they reported that the child was rejected by peers, failed academic classes in school, or disrupted family and classroom functioning. In other words, use of clinical services is usually precipitated by impairment, not by symptoms.8 Furthermore, impairment in functioning during childhood is one of the best predictors of negative long-term outcomes.8 For example, poor peer relationships and academic underachievement in childhood predict poor outcomes in adulthood.9,10 As it is well known that ADHD with it's profound impairment in many functional areas can persist throughout the adulthood,11 improvement in impairment domains must be achieved to avoid continued problems throughout the development. Therefore, it is critical to include impairment not only in diagnostic decision making, but also in treatment planning and in prediction of outcome in adulthood. Given all that, we concluded that symptoms and impairment are related, but likely have distinct correlates and importance in diagnosis and assessment. Therefore, impairment should be measured independently in addition to the symptoms to determine the presence of ADHD.

For assessing functional impairment in children and adolescents, there exist three types of rating scales: global impairment scales, domain-specific scales, and symptom-specific scales. The global impairment scale enable the interviewer to rate a child on a single scale of functioning. The Children's Global Assessment Scale (C-GAS),12 the representative scale for a global impairment provides a single global rating of functioning. The advantage of this scales is that it is very brief and can easily be administered to children and adolescents allowing comparison of functional impairment between groups of patients with different diagnoses.13,14 However, C-GAS is not an independent measure of impairment and is strongly associated with psychiatric symptoms.15 In addition, global scales fail to provide information about the most impaired domains of functioning thereby compromising the most suitable targets for treatment planning.16

By contrast, domain-specific impairment scales provide separate scores for each domain of functioning. The Child and Adolescent Functional Assessment Scale (CAFAS),17 The Brief Impairment Scale (BIS),18 and Ontario Child Health Study (OCHS) scale,19 are representative of domain-specific scales. The CAFAS17 is the most widely used domain-specific impairment scale. It can be administered during a clinician-administered interview and contains 5 subscale domains pertaining to the child's functioning and 2 to the child's caregivers to assess the degree of impairment associated with emotional, behavioral, or substance abuse problems of children. The youth subscales include Role Performance (how effectively the youth fulfills societal roles in school, home and community), Behavior Toward Self and Others (appropriateness of the youth's daily behavior), Thinking (ability to use rational thought processes), Moods/Emotions (modulation of the youth's emotional life), and Substance Abuse (extent of use and degree to which it is disruptive). The caregiver subscales include Basic Needs (caregiver ability to provide for the child's basic needs) and Family/Social Support (the degree to which the relationship and nurturance provided by the caregiver meets the child's needs). The BIS18 to be administered by an adult informant, covers three domains: Interpersonal Relations, School/Work; and Self-Fulfillment. The OCHS19 scale provides measures of function that assess social, behavioural, and academic impairment of children. It includes child's social participation, quality of the child's social relationships, school participation and achievement, global child/youth functioning, family activities, family conflict and anxiety, and global family situation. These scales provide information about impairment in several specific areas of functioning and permit precise assessment of areas necessary to planning intervention or prevention. However, the items in these domain-specific scales are not tied specifically to psychiatric symptoms. In other words, the measure of functional impairment indicated in domain-specific scales is not necessarily caused by the symptoms reported. Domain-specific impairment scales are inappropriate for use as diagnostic measures because they are not linked to specific diagnostic symptoms of psychiatric disorders.

By contrast, symptom-specific impairment scales measure the degree of disability associated with a specific symptom or diagnosis. Diagnostic Interview Schedule for Children, Version IV (DISC-IV)20 Impairment measures and Child and Adolescent Psychiatric Assessment (CAPA)21 incapacity ratings are the representatives of symptom-specific scales. Like the domain-specific impairment scales, these scales assess various domains of functioning. For example, the DISC-IV contains 6 questions probing how often the caregiver and teacher becomes upset with the child and how often the child has problems doing things with the family, doing things with other children, doing homework or grades, or feeling badly or upset. Since these scales measure functional impairment due to one's psychiatric symptoms reported and include all areas associated with functional impairment of in DSM-IV, they are useful for the diagnostic purpose of child's psychiatric disorders. Each measurement have their own weakness and strength in assessment of functional impairment in children.

Taken altogether, there exist following reasons for conducting our study. First, so far, there is no common operationalization of measuring impairment in children. The lack of the gold standard in measuring impairment applies to ADHD as well, though it is one of the most commonly diagnosed childhood disorders nowadays. Among the above mentioned types of rating scales, symptom-specific impairment scale would be the most appropriate instrument for diagnostic purposes. However, for most symptom-specific scales, the construct validity of the functional domains is not clearly verified yet, because factor analyses of the items in the scales were not performed. According to our survey, there is a scarcity of studies on the diagnostic purpose of the individual impairment rating scales. Thus, we first wanted to develop a reliable and valid scale measuring the social and academic functional impairment of children with ADHD. Second, impairment in functioning at home and in school settings needs to be assessed objectively based on the respondent-based format. Thus, we wanted to develop a separate rating scale for parents and teachers to assess impairment of functioning of children in two different situations. And at last, we wanted to draw an optimal cutoff score of the developing scales for distinguishing functional impairment between children with ADHD and normal children. This may serve as an impairment-based criterion for ADHD.

METHODS

Participants and procedures

Preliminary items for Child and Adolescent Functioning Impairment Scale (CAFIS), a self-administered rating scale for measurement of functional impairment in children and adolescents in Korean language, were based on the review of the previously well-known scales described below. An expert committee reviewed the items collected, and obtained 25 preliminary items for parent form and 20 preliminary items for teacher form of CAFIS.

Subjects enrolled in this study were children or adolescents of aged 6 to 14 who visited the department of psychiatry at Soonchunhyang University Bucheon Hospital for assessment of inattention, hyperactivity and impulsivity. Exclusion criteria for the study were as follows: pervasive developmental disorder, psychosis, bipolar disorder, past history with special education or brain injury, a sensory deficit, or full scale IQ below 80. The mean IQ, measured by the K-WISC-III22 of ADHD and normal group were 98.1 (SD=11.9) and 102.9 (SD=15.1), respectively. There was no significant difference in the mean scores of IQ between the two groups.

For the psychiatric diagnosis of the subjects, certified clinical psychologists interviewed the children and their parents using the Korean schedule for affective disorders and schizophrenia for school-age children present and lifetime version (K-SADS-PL).23 Children in ADHD group met the full DSM-IV criteria for ADHD, except the criteria B. ADHD participants were drug-naïve at the time of evaluation. We did not include the age of onset criterion because the validity of age of onset criterion has been questioned.24 At the diagnostic interview, the interviewer also rated the children's global functioning using the C-GAS in order to investigate the concurrent validity of CAFIS.

Children for the control group are recruited by advertisement. They did not met DSM-IV criteria for any psychiatric disorders on the interview using the K-SADS-PL. Particularly, the children in the control group had no more than two inattentive or hyperactivity/impulsivity symptoms of DSM-IV criteria for ADHD. Based on Gorsuch25 and Hatcher26's recommendation, we determined that the subject to item ratio for exploratory factor analysis should be at least 5 : 1.

Parent and teachers of children with ADHD and normal groups completed the preliminary items for CAFIS-parent form and CAFIS-teacher form, respectively. Based on the results of reliability and factor analysis, items that have corrected item-total correlation coefficients lower than 0.2 and factor loadings lower than 0.4 were removed from preliminary items for CAFIS-parent and CAFIS-teacher forms. As a result, we obtained 14 items for the parent version of CAFIS and 10 items for teacher version of CAFIS. Thereafter, we performed the reliability, validity, and relative operating characteristics (ROC) analysis of the CAFIS-parent and CAFIS-teacher forms.

Internal consistency was analyzed for results of CAFIS-parent and CAFIS-teacher forms. Parents and teachers were asked to complete the rating scales a second time at 4 weeks later for analysis of test-retest reliability. Four weeks was considered to be a reasonable time for regarding stability in response and minimizing individuals' memory effect. Construct, concurrent and discriminant validity were analyzed for results of CAFIS-parent and CAFIS-teacher forms. In addition, ROC analyses were employed to calculate the sensitivity and specificity and determine the optimal cut-off point of the total and subscale scores of the CAFIS-parent and CAFIS-teacher forms.

This study protocol was approved by the Institutional Review Board of Soonchunhyang University Bucheon Hospital and the subjects' parents and teachers were thoroughly briefed in the design of the study, allowed to read and sign a written consent form before participating.

Measures

Korean schedule for affective disorders and schizophrenia for school-age children present and lifetime version

The K-SADS-PL was developed to assess the severity of symptoms as well as the present and lifetime status of 32 DSM-IV child and adolescent psychiatric disorders. It was designed for interviewing both the parents and the children and provides a standardized method of obtaining and recording symptoms necessary for the assessment of most Axis I categories. In the Korean version of K-SADS-PL (K-SADS-PL-K),27 the consensual validity of threshold diagnoses were good to excellent (Kappa=0.70) and inter-rater (Kappa=0.42) and test-retest reliabilities (r=0.76) were fair to excellent for ADHD. In this study, board certified clinical psychologist with experience in child and adolescent psychiatry interviewed the subjects and the parents. Interviewers rated patients' symptoms without recourse to impairment.

Child and Adolescent Functioning Impairment Scale

At first, we collected the items of widely used, validated scales measuring impairment of children and adolescents. Included scales were Impairment measures of DISC-IV, CAFAS, BIS, and OCHS. In addition to the aforementioned scales such as DISC-IV, CAFAS, BIS, and OCHS, examples of functional impairment in children and adolescents visiting our outpatient clinic were collected using a questionnaire with open questions, for example "Do your child have any problems at home?" All items assembled were reviewed by an Expert Committee consisting of Korean child and adolescent psychiatrists, bilingual child psychiatrist, and child and adolescent clinical psychologists. Items overlapping in meaning were deleted, and the preliminary items were selected and modified appropriately for our language and culture. The preliminary items for parent version of CAFIS was composed of 25 questions asking parents for functioning of children at home, and the preliminary items for teacher version of CAFIS of was composed of 20 questions asking teachers for functioning of children in school. The questions ask how much the problems of child affect various functioning of daily activities due to current symptoms. In this study, the current symptoms were defined specifically as inattentive, hyperactive or impulsive symptoms of the individual subjects. The responder is asked to select one from a five-point scale ranging from 0 (no impairment) to 4 (severe impairment) for each question. Ratings reflect the most severe level of dysfunction within the one month.

Among the preliminary items of CAFIS-parent and CAFIS-teacher forms, 11 items of CAFIS-parent form and 10 items of CAFIS-teacher forms that had lower than 0.2 corrected item-total correlation coefficients or lower than 0.4 factor loadings were eliminated. As a result, CAFIS-parent form consisting of 14 items had four following factors: 1) family relationship; 2) teacher relationship; 3) peer relationship; 4) academic achievement. The CAFIS-teacher form consisting of 10 items had three following factors: 1) peer relationship; 2) teacher relationship; and, 3) academic achievement. To summarize, the CAFIS-parent and CAFIS-teacher forms included inquiries about social and school functioning of children and adolescents at home and in school.

More specifically, the social functioning items enquire about the quality of the relationship of the child with family, teacher, and peers. For example, the items asking about family relationships consist of questions addressing problems with parents, siblings or other important members of the family. Whether treatment is needed due to the problems in relationship with the family members is also addressed. Problems with the teachers are asked in the same way. Items addressing the peer relationships consist of questions asking problems in getting along with friends, making new friends, rejected by friends and withdrawn or isolated by peers. Items relating to school functioning included problems due to low grade in school and the necessity of treatment due to low academic achievement. The CAFIS-teacher form includes items similar to the parent version of CAFIS except the items asking the relationship with the family.

Children's Global Assessment Scale

The C-GAS provides a global measure of level of functioning in children and adolescents at home, in community and school, and with peers. Scores range from 1 (most impaired) to 100 (highest level of functioning) to reflect the degree of functional impairment. This scale was known to be reliable between raters (r=0.83-92) and across time (r=0.85).12 In this study, the C-GAS was rated to investigate the concurrent validity of the CAFIS.

Data analysis

The CAFIS-parent and CAFIS-teacher forms were analyzed for their reliability, validity and diagnostic validity (ROC analysis). To test the internal consistency, Cronbach's alpha was calculated on the total and subscales of the CAFIS-parent and teacher forms. Correlation coefficients between the first and second CAFIS-parent and CAFIS-teacher forms were calculated to analyze the test-retest reliability of the scales.

To establish the construct validity, an exploratory factor analysis was conducted (extraction method: principal axis factoring; rotation method: varimax with Kaiser normalization). Concurrent validity was assessed by examining the relationship between the scores on the subscales of CAFIS and C-GAS. To demonstrate discriminant validity, T-test was used to estimate the differences in the scores on the subscales of CAFIS between ADHD and normal control groups.

ROC curves were drawn to analyze the sensitivity and specificity of CAFIS-parent and CAFIS-teacher forms, and to determine the optimal cut-off point of the CAFIS. All analyses were conducted in SPSS version 14.0 (SPSS Inc., Chicago, IL, USA).

RESULTS

Participants

One hundred fourteen children and adolescents, aged 6 to 14 years, were involved in this research. A total of 72 children with ADHD (65 male, 7 female) and 42 normal children (30 male, 12 female) were enrolled. There was significant difference in the ratio of sex between two groups (χ2=6.79, p<0.01). The mean ages of ADHD and normal groups were 8.8 (SD=1.9) and 9.5 years (SD=2.1), respectively, and the difference in two groups was not significant. The comorbidities of the children with ADHD were shown as follows: 8 (11.1%) with tic disorders, 7 (9.7%) with oppositional defiant disorder, 2 (2.8%) with elimination disorders, 1 (1.4%) with conduct disorder, 1 (1.4%) with obsessive-compulsive disorder and 1 (1.4%) with social phobia.

Psychometric properties of CAFIS

The CAFIS-parent form consisting of 14 items and four factors and the CAFIS-teacher form consisting 10 items and three factors are shown on Table 1 and Table 2. For the four factors of CAFIS-parent form, the Kaiser's Measure of Sampling Adequacy (MSA)28 was 0.82 as meritorious level29 and the variance explained by 4 factors were 63.72%. For the three factors of CAFIS-teacher form, the Kaiser's MSA was 0.82 as meritorious level and the variance explained by 3 factors were 66.15%.

Table 1

Results of factor analyses on CAFIS-parent form

CAFIS: child and adolescent functioning impairment scale

Table 2

Results of factor analyses on CAFIS-teacher form

CAFIS: child and adolescent functioning impairment scale

Reliability

The internal consistency and the test-retest reliability of CAFIS-parent and CAFIS-teacher forms are presented on Table 3. The Internal consistencies for the subscales of CAFIS-parent and CAFIS-teacher forms were satisfactory, Cronbach's alpha ranging from 0.71 to 0.90 and from 0.74 to 0.90 for the parent and teacher form, respectively. The test-retest reliability coefficient of the CAFIS was found to be r=0.71 to 0.86. for the parent form and r=0.81 to 0.89. for the teacher form. Among the subscales, the correlation of the academic achievement subscales of CAFIS-parent and CAFIS-teacher forms were highest.

Table 3

The Internal consistency and test-retest reliability for the subscales of the CAFIS-parent and CAFIS-teacher forms

*p<0.01, **p<0.001. (1): family relationship, (2): teacher relationship, (3): Peer relationship, (4): academic achievement, CAFIS: child and adolescent functioning impairment scale

Validity

Concurrent validity

To examine the concurrent validity of the CAFIS, correlations between the subscale scores of CAFIS-parent and CAFIS-teacher forms and the C-GAS scores were calculated (Table 4). All subscales of CAFIS-parent and CAFIS-teacher forms were significantly correlated to the C-GAS score. The correlation coefficients (Pearson r) between the scores of CAFIS-parent form and the C-GAS ranged from -0.27 to -0.46 and the scores of CAFIS-teacher form and the C-GAS from -0.30 to -0.47. Correlations between the scores of the academic achievement subscales of CAFIS-parent and CAFIS-teacher forms and C-GAS were highest.

Table 4

Correlations between the CAFIS and C-GAS

*p<0.01, **p<0.001. (1): family relationship, (2): teacher relationship, (3): Peer relationship, (4): academic achievement, CAFIS: child and adolescent functioning impairment scale, C-GAS: children's global assessment scale

Discriminant validity

To examine the discriminant validity of the CAFIS, differences in the mean scores of the subscales of CAFIS-parent and CAFIS-teacher forms between the ADHD and normal groups were analyzed (Table 5). All subscales of CAFIS-parent and CAFIS-teacher forms of the ADHD group were significantly higher than the control group.

Table 5

Differences in mean scores of the subscales of CAFIS between ADHD and normal control groups

*p<0.01, **p<0.001. (1): family relationship, (2): teacher relationship, (3): Peer relationship, (4): academic achievement, ADHD: attention-deficit/hyperactivity disorder, CAFIS: child and adolescent functioning impairment scale, SD: suicidal depression

ROC analyses

We calculated the ROC curve and the sensitivity and specificity for the subscales of CAFIS, with the presence of ADHD as the gold standard. For all subscales, areas under the curve (AUCs), which is a measure of overall accuracy,29 were significant. In CAFIS-parent form, AUCs were found as follows: 0.81 for total score (p=0.000), 0.71 for the family relationship scale (p=0.000), 0.71 for the teacher relationship scale (p=0.000), 0.69 for the peer relationship (p=0.001), and 0.77 for the academic achievement (p=0.000). In CAFIS-teacher form, AUCs were found as follows: 0.79 for total score (p=0.000), 0.65 for the teacher relationship scale (p=0.009), 0.73 for the peer relationship (p=0.000), and 0.78 for the academic achievement (p=0.000). The ROC curve for total scores of CAFIS-parent and CAFIS-teacher form is presented in Figure 1.

Figure 1

ROC curves for total score of CAFIS-parent and CAFIS-teacher forms. ROC: relative operating characteristics, CAFIS: child and adolescent functioning impairment scale.

To determine the optimal cutoff scores, the sensitivity and specificity of different cutoff scores are calculated and presented on Table 6.

Table 6

Results of ROC analyses on CAFIS

*p<0.01, **p<0.001. AUC: areas under the curve, CAFIS: child and adolescent functioning impairment scale, ROC: relative operating characteristics, (1): family relationship, (2): teacher relationship, (3): Peer relationship, (4): academic achievement, Sen: sensitivity, Spe: specificity, 0.9-1: excellent, 0.8-0.9: very good, 0.7-0.8: good, 0.6-0.7: average, 0.5-0.6: poor

DISCUSSION

Clinicians and researchers tend to focus on syndrome features per se instead of coding patient's overall functioning or adjustment. Though researchers have developed several impairment-specific instruments, so far, the scales have not found their way into routine clinical practice. This is mainly because the DSM-IV, the most widely used diagnostic criteria, does not specify the how impairment should be measured or taken into account in clinical practice. Some previous instruments such as the CAFAS are time-consuming and need training to administer reliably. Other measures cover a narrow or too global range of functioning. Existing impairment measures lack of construct validity.

The CAFIS is a new symptom-specific impairment scale that measure functional impairment independently of psychiatric symptomatology in children and adolescents. This scale is the first one developed in this area in Korean language so far. It is very brief and easy to administer for layperson. Both parent-rated and teacher-rated versions are available, enabling measuring impairment in two different settings as required in the DSM-IV.

The results of this study show that the CAFIS has sound psychometric properties. The CAFIS is reliable and valid for the diagnosis of children and adolescents with ADHD. The internal consistency of total and subscale scores of the parent and teacher forms were satisfactory comparable to other impairment scales such as CAFAS (0.63 to 0.68) or BIS (0.56 to 0.88). The test-retest reliability at a 4 week interval was also satisfactory comparable to those of CAFAS (0.77) and CAPA Incapacity rating (0.76).

The CAFIS has shown to have satisfactory construct validity as well. The parent form comprised of four factors that are family, teacher, and peer relationship and academic achievement. The teacher form comprises identical factors to the parent form except for the inclusion of the family relationship factor. Factor analysis was not performed on the CAPA Incapacity ratings and the DISC-IV Impairment measures. We can say that the CAFIS is more valid for construct validity than the two scales. In addition, all subscales included the item related to the requirement to treatment (or treatment-seeking), except for the subscale for peer relationship indicating that this item is necessary when measuring the impairment of social or academic functioning. Using the C-GAS, concurrent validity was verified. Compared to the result of BIS study, the correlations between CAFIS and C-GAS were smaller than those between BIS and C-GAS (-0.45 to -0.60). However, since both the BIS and C-GAS measure children's functioning independently of symptomatology and whereas the CAFIS is a symptom-specific scale, the results do not threaten the concurrent validity of the CAFIS. In our study, both parent (-0.46) and teacher (-0.47) rated impairment on academic achievement was highly correlated with the C-GAS. This indicates that academic functioning of children is an important factor affecting global functioning perceived by the parents and teachers. In addition, the CAFIS shows good discriminant validity. The ADHD group had higher scores on all subscales of CAFIS-parent and CAFIS-teacher form in comparison to those of normal controls. In particular, the mean score of the academic achievement subscale was highest for the ADHD group and the difference in the subscale scores of academic achievement between the two groups was greatest.

In the ROC analyses, we found that the CAFIS could discriminate ADHD from normal children. In CAFIS-parent form, AUC of total score was at an excellent level (0.8-0.9)29 and most of AUCs fell into acceptable level (0.7-0.8), except the peer relationships subscale. In CAFIS-teacher form, most of AUCs were at an acceptable level (0.7-0.8), except the teacher relationships subscale. In particular, for both the parent and teacher forms of CAFIS, the AUCs for the academic achievement subscale were higher than those of other subscales.

For a scale with a diagnostic purpose, increased specificity is desirable at the price of decreased sensitivity in order to decrease the number of false positive cases. Since the CAFIS was intended to get utilized in the diagnostic process of ADHD, the optimal cutoff score was determined first with high specificity, and second, balanced sensitivity with specificity. On the basis of these criteria, the optimal cut-off score of the CAFIS-parent form was 7 for total score (sensitivity: 0.75; specificity: 0.83), 3 for the family relationship subscale (sensitivity: 0.60; specificity: 0.74), 1 for the teacher relationship subscale (sensitivity: 0.61; specificity: 0.76), 1 for the peer relationship subscale (sensitivity: 0.64; specificity: 0.67), and 3 for the academic achievement subscale (sensitivity: 0.63; specificity: 0.79). In the CAFIS-teacher form, optimal cut-off scores were 6 for total score (sensitivity: 0.68; specificity: 0.69), 1 for the teacher relationship subscale (sensitivity: 0.54; specificity: 0.72), 3 for the peer relationship subscale (sensitivity: 0.63; specificity: 0.67), and 3 for the academic achievement subscale (sensitivity: 0.58; specificity: 0.79). For both, the parent and teacher forms of CAFIS scale, AUC and the sensitivity and specificity of all subscales were relatively low in comparison to those of the total scores of each scale. Possible explanation for these results would be that individual subscale of the CAFIS does not serve for the functioning impairment per se to determine the caseness of ADHD. Instead, impairment in several functional areas together as a whole would fit for the impairment criteria described in the DSM system. Based on the results of ROC analysis, the value of AUC and the sensitivity and specificity of academic functioning again was highest among other subscales, indicating that academic achievement is an important factor to be considered in the diagnosis of ADHD in our sample.

There are some limitations of this study. Though the CAFIS instruct parents and teachers to rate children's impairment due to inattentiveness, hyperactivity or impulsivity, confounding of functioning by symptomatology cannot be excluded completely. Further, impairment due to psychiatric symptoms other than the ADHD symptoms cannot be easily distinguished by the raters. Especially, functional impairment in relevance to children's developmental disabilities and comorbidities is an issue to be addressed further. In this study, we excluded children with previous and current development disorders, and further excluded children with lower intelligence by administering standardized intelligence tests. Impairment in ADHD children having comorbid psychiatric disorders were not considered separately in this study, though the comorbidities obviously would impact the function in children and may drive ratings of impairment more than does ADHD itself. The reason why we did not compare functional impairment in ADHD children with comorbidities to compare that of children without comorbidities, because the percentage of children with comorbidities in our study were relatively low, and thus, the sample size was too small in comparison to previous studies.30 Because of the restriction of enrollment into the study and comorbid features of our sample, generalization of the results to ADHD samples in general practice could be decreased. There is another issue of cultural variation. Although the questions in the scales were formulated generally as such, "is there problems in..." or "need the child intervention or treatment because of...", these scales still need to be evaluated in more culturally diverse samples. In addition, a nationally representative normative data should be provided in order to get broadly used in clinical, research, and administrative applications.

In conclusion, the CAFIS is a newly developed, brief, respondent-based, and easily administered scale for measuring functional impairment in children and adolescents. The parent and teacher versions which are nearly identical in format and content will allow comparison of parents' and teachers' perceptions as to the children's functional impairment. The scale has satisfactory levels of reliability and validity, and appears to be valuable in differentiation of children with and without ADHD. Although the psychometric properties of this scale have shown to be satisfactory, the CAFIS needs to be verified further with other scales in which the interviewer makes a clinical judgment about the rating in order to be used for the diagnostic purpose. When age and gender-specific normative data is provided in the future, these scales will not only assist in ADHD identification but also promote planning and monitoring effective management of children with ADHD.