Reliability and Validity of the Supports Intensity Scale (SIS) Measured in Adults with Physical Disabilities.

The objective of this study was to investigate the internal consistency and the construct validity of the Dutch version of the Supports Intensity Scale (SIS-NL1.0; Buntinx 2006) in individuals with physical disabilities (N = 65). To investigate the construct validity, the relationship between SIS subscales and practical skills (Barthel Index; BI) was calculated. Support was provided for the internal consistency. The SIS subscales (except Behavior) had moderate to high intercorrelations and the SIS was able to discriminate between groups with different number of disabilities. However, weak relationships were found between the BI and four out of eight SIS subscales. For people with physical disabilities, future revisions of the SIS should also take into consideration limitations in practical skills in other support domains.

Since the mid-1980s, a paradigm shift has been taking place in the manner people with disabilities have been viewed (Luckasson et al. 2002). Whereas people with disabilities were previously described on the basis of their underlying deficits, the focus has been moved to the fundamental idea that people with disabilities are people with personal needs (Thompson et al. 2002), in which both “inclusion and self-determination are central components” (Guscia et al. 2006, p. 148). This new paradigm, referred to as the support model, takes into consideration which resources of support are needed to enhance people’s participation in the community, help them to establish valued social roles and, in general, enhance their quality of life (Thompson et al. 2002).

The adoption of thinking in terms of support needs has important implications for the way these needs are met by service providers. That is, the level of needed support will be used as a determinant of funding resources. It has been proposed that such a system of funding may not only enhance the quality of life of people with disabilities, but may also ensure more efficiency and equity in the provision of services (Guscia et al. 2005). Therefore, objective measurement of support needs is required for appropriate planning and equal provision of resource allocations for health services (Soo et al. 2007).

The American Association on Intellectual and Developmental Disabilities (AAIDD), formerly known as the American Association on Mental Retardation (AAMR), has developed the Supports Intensity Scale (SIS). The SIS is a multidimensional scale used to measure the intensity of support needs of people with intellectual and other developmental disabilities. The scale is composed of three sections. “Section 1” measures support needs in six life domains, namely Home Living, Community Living, Life-Long Learning, Employment, Health and Safety and Social Activities. “Section 2” measures individual support requirements in the area of Protection and Advocacy. Finally, in “Section 3” both exceptional Medical and Behavioral support needs are measured (AAIDD 2004; Thompson et al. 2004). In contrast to adaptive behavior scales, the SIS is not used as a measure to assess typical skills in daily activities, but is used as a tool to estimate the amount of support a person needs to successfully perform these aforementioned activities. In typical daily activities, for example preparing food, adaptive behavior scales are concerned with the specific skills a person has (e.g., cutting foods with knife, preparing a meal for 4 people), whereas the SIS focuses on the amount of support the person needs to successfully accomplish such an activity as food preparation (AAIDD 2004; Luckasson et al. 2002; Thompson et al. 2004).

The SIS has primarily been developed for use with people with intellectual and developmental disabilities (Luckasson et al. 2002). Evidence has been found for both the reliability and validity of the SIS in samples of people with intellectual disabilities using the English version of the SIS (see e.g., Harries, Guscia et al. 2005; Thompson et al. 2002). As the authors of the SIS have pointed out, further work is needed to’to trial their assessments with different disability groups’ (Guscia, et al. 2006, p. 149). In a study by Thompson et al. (2002), in which the psychometric properties of the SIS was investigated in people with ID, a large amount of the sample had additional physical disabilities. However, only one study by Guscia and colleagues (2006) used a subgroup of people with a primary physical disability in their sample. In their study, the validity of the SIS was examined in a mixed sample (N = 114) of people with disabilities, including people with an intellectual disability and people with a physical disability. However, if the two groups were analyzed separately, support for the criterion validity was found for both the groups of individuals with an intellectual disability (N = 83) and a small heterogeneous group of individuals with a physical disability (N = 31). To our best knowledge no other studies have specifically investigated the reliability and validity of the SIS in people with physical disabilities. The aim of the current study is to fill this gap.

First, the internal consistency of the SIS subdomains was determined for a representative group of people with physical disabilities. Second, with regard to the validity, Thompson and colleagues (2002) stated that a close relationship between the SIS and measures for adaptive skills is an important indicator for the validity of SIS, because these scales measure a related but different construct (Thompson et al. 2002). For example, in studies with people with intellectual disabilities, high correlations were found between support needs as measured by the SIS and adaptive behavior skills (Harries et al. 2005; Thompson et al. 2002). The AAIDD (Schalock et al. 2010) distinguishes three dimensions of adaptive behavior, namely conceptual, social and practical skills. However, there is still a lack of agreement of what specific skills are included in these dimensions. For example, according to the definition of AAIDD, practical skills include skills of activities in daily living (ADL), vocational activities and skills to maintain safe environments. However, in a study by Harries et al. (2005), factor-analyses showed that the dimension of Practical Skills was mainly associated with physical and motor competence and self-care skills such as eating, toileting, and personal care. Therefore, in the current study, in order to test the construct validity of the SIS, the relationship between the SIS and the skills identified in the study of Harries et al. (2005) of the practical skills dimension is investigated. Previous research has shown that people with physical disabilities often experience problems with such skills. For example, many people with cerebral palsy have difficulties with incontinence and mobility (Balandin and Morgan 1997). A study of Andrén and Grimby (2000) revealed, for instance, that more than 75% of their sample of adults with cerebral palsy and spina bifida perceived difficulty with ADL-activities, such as bathing, toileting and mobility and needed assistance in performing these activities. Therefore, this study focuses on the most relevant practical skills of people with physical disabilities: mobility and self-care skills.

Moreover, limitations in practical skills may not only affect activities of daily living, but also other domains of support. For example, in the study of Andrén and Grimby (2000) the group with spina bifida reported more problems with bowel and bladder control than the group with cerebral palsy. Persons with Spina Bifida reported also more difficulty and dependency using public transport and shopping than persons with cerebral palsy. The authors suggest that problems with incontinence may also lead to problems in other domains of support. In another study of people with cerebral palsy it was shown that problems with walking, which were experienced by many of the respondents, resulted in reduced social activity and increased anxiety and depression (Balandin and Morgan 1997). Thus, it can be suggested that limitations in practical skills also might be associated with higher levels of support in other domains. In order to analyze the construct validity of the SIS in people with physical disabilities, the association between practical skills and the SIS subscales was calculated.

Method

Participants

Participants were recruited in the Dominiek Savio Institute, a service center for people with physical disabilities in Belgium. The Dominiek Savio Institute offers ambulatory, semi-residential, and residential services to children and adults who need support in the domains of living, education, and work. Participants of this study were included if they were diagnosed with a neurological condition resulting in a major physical disability (e.g., cerebral palsy, spina bifida, and traumatic brain injury). In order to achieve a homogenous sample, individuals with a medical condition that was progressive by nature (e.g., Huntington’s disease, neuromuscular diseases) were excluded from participation. This resulted in a sample of 65 adults (31 male; 34 female), representing the most common physical disabilities. Medical and demographic information of our sample is provided in Table 1. Most participants were diagnosed with cerebral palsy (80%) followed by Traumatic Brain Injury (15%) and Spina Bifida (5%). A total of 48% of the sample was diagnosed with an intellectual disability (IQ-score lower than 70). All participants were served in a residential setting.

Table 1

Demographic and background variables of the participants (N = 65)

	Percentage (%)
Age
21-30	12
31-40	29
41-50	40
51-60	19
Primary disability
Cerebral Palsy	80
Spina Bifida	5
Traumatic Brain Injury	15
Secondary disability
Intellectual disability	48
Deafness/hearing impairment	6
Blindness/visual impairment	8
Speech/language impairment	46
Psychiatric impairment	19
Cognitive impairments (e.g., apraxia, ataxia, amnesia)	12
Intelligence level
> 71	52
56–70	25
46–55	11
36–45	9
21–35	3

Procedure

The Supports Intensity Scale (SIS) and the Barthel Index (BI) were administered according to the guidelines. Both scales were scored by caregivers who were familiar with the person for at least 3 months. For the SIS, individuals were rated by staff workers who held an academic degree and had sufficient knowledge of the concept of ‘support’ which is also recommended by the SIS manual (AAIDD 2004). The BI was rated by the personal assistant (PA) of each participant. For the BI, raters were assisted by the first author (e.g., instruction of’unclear’ items, verifying that all items were scored).

Materials

Support Intensity Scale-NL

(SIS-NL1.0: Buntinx 2006). The SIS-NL is the Dutch translated version of the SIS. Evidence for the good psychometric properties of the instrument has been provided in both Dutch and Belgium samples (Buntinx 2006; Claes et al. 2009). The SIS is composed of three separate areas in which support may be needed: Life Activities (57 items), Medical conditions (16 items) and Behavioral conditions (13 items). The domain of Life activities is composed by 7 subscales: Home Living, Community Living, Life-long Learning, Employment, Health and Safety, Social Activities, and Protection and Advocacy. For these subscales, ratings are made on a 5-points scale with regard to the frequency of support (ranging from none to hourly), the daily support time needed (ranging from none to 4 h or more) and the type of support (ranging from none to full physical assistance). Additionally, ratings in the areas of Medical and Behavioral conditions are assessed on a 3-points scale, ranging from no support, some support and extensive support needed. By summing up all scores on the subscales, a total raw score is derived.

Barthel Index

(BI; Mahoney and Barthel 1965). A Dutch validated version of BI was used in order to measure the practical skills of the individuals included in our sample. The BI is a widely used and accepted instrument in clinical and scientific research to assess a range of functional abilities in activities of daily living (ADL). The BI has been considered as the gold standard to measure ADL-abilities (Dijkstra et al. 2005). A total of ten domains of ADL are rated by a primary caregiver on a scale ranging from (0) fully dependent to (3) fully independent. These domains of ADL include: Bowel Control, Bladder Control, Personal Hygiene, Toilet Use, Feeding, Transfer, Mobility, Dressing, Stair Climbing, and Bathing. A total score is derived by cumulating the sum scores of the domains. The total score (ranging from 0 to 20) gives an indication of the care dependency of the person. The lower the scores, the more dependent the person is on care, and vice versa. The BI has been considered as a reliable and valid instrument to measure activities of daily living (ADL) in people with neuromuscular and muscoskelatal disorders. (Mahoney and Barthel 1965; Plantinga et al. 2006)

Data-Analyses

Internal Consistency

Cronbach’s alphas were calculated for both the SIS subscales and the total SIS scale to determine the internal consistency of the scales. A Cronbach’s alpha higher than .70 is considered as an acceptable criterion for internal consistency (Kline 2005).

Construct Validity

The measurement of construct validity “is involved whenever a test is to be interpreted as a measure of some attribute or quality which is not operationally defined” (Cronbach and Meehl 1955, p. 281).

There are multiple validation procedures to determine the construct validity of a scale (Arvey 1992). The first manner is to correlate the subscales with one another. If correlations between subscales are in the moderate to high range (.4 – .9), it is evidenced that the subscales measure the same overall construct (Thompson et al. 2002).

The second manner to examine construct validity is to correlate the SIS subscale scores with the total score of the Barthel Index (BI). Construct validity could be considered adequate if correlations are .35 or higher. Moreover, one would expect a significant correlation between the SIS and BI because they measure two related but different constructs, namely support needs and adaptive skills respectively (Thompson et al. 2002). Therefore, we hypothesized that the correlation between the SIS subscales and the BI would be in the moderate range (.4 to .6).

A third procedure to determine the construct validity of the SIS is to test whether there are expected differences among groups with different number of disabilities on total SIS-scores. One would expect that persons with more disabilities have higher SIS scores (indicating higher support needs) than persons with less disabilities. Therefore, three groups were composed according to the number of disabilities: a group with one disability (n = 20), a group with two disabilities (n = 30) and a group with three or more disabilities (n = 14). The group with one disability exclusively had a motor disability, whereas the other two groups had additional disabilities, such as a hearing, visual, speech/language, psychiatric or cognitive disabilities. A one-way analyses of variance (ANOVA) was conducted to test our hypothesis that the SIS total scores increased as the number of disabilities was higher.

Results

Internal Consistency

Internal consistency coefficients were high and exceeded the criterion of .70 (see Table 2): Cronbach’s alphas ranged from .71 for Medical to .96 for Life-Long Learning, and the Cronbach alpha for the total SIS was .98. Thus, internal consistency was evidenced for both the subscales and the total SIS scale.

Table 2

Intercorrelations between SIS Subscales (N = 65)

SIS subscale	HL	CL	LLL	EMP	H&S	SOC	MED	BEH
HL	.95
CL	.62**	.95
LLL	.47**	.78**	.96
EMP	.46**	.65**	.71**	.94
H&S	.59**	.76**	.82**	.69**	.94
SOC	.44**	.77**	.72**	.75**	.76**	.93
MED	.74**	.63**	.52**	.55**	.58**	.46**	.71
BEH	-.00	-.02	-.01	-.01	.16	.05	-.05	.77

**significant at p<.01;

HL Home Living, CL Community Living, LLL Life-Long Learning, EMP Employment, H&S Health & Safety, SOC Social Activities, MED Medical, BEH Behavior

Coefficients on the diagonal are the Cronbach’s alphas

Construct Validity

First, to test the construct validity, the intercorrelations between the SIS subscales were calculated. As can be seen in Table 2, construct validity was present in 7 out of 8 subscales, as shown by the statistically significant intercorrelations. However, the subscale Behavior did not significantly correlate with the other subscales (range .00–.16). The significant intercorrelations ranged from moderate to high. That is, the highest significant correlation was found between Life-Long Learning and Health and Safety (.82), whereas, Home Living and Social Activities revealed the lowest significant correlation (.44). Thus, all SIS subscales seemed to measure the same construct, namely support needs, with the exception of the Behavior scale.

Second, SIS subscale scores were correlated with the BI sum score (Table 3). Four of the eight correlations between the SIS subscales and the BI exceeded the criterion of .35, namely Home Living (-.78), Community Living (-.41), Health and Safety (-.41) and Medical (-.70). The other four SIS subscales (Life-Long Learning, Employment, Social Activities and Behavior) did not correlate higher than .35 with the BI.

Table 3

Correlations between SIS Subscales and the Sum Score of the BI (N = 65)

	BI
HL	-.78**
CL	-.41**
LLL	-.26*
EMP	-.29*
H&S	-.41**
SOC	-.22
MED	-.70**
BEH	-.03

* significant at p < .05; ** significant at p < .01

HL Home Living, CL Community Living, LLL Life-Long Learning, EMP Employment, H&S Health & Safety, SOC Social Activities, MED Medical, BEH Behavior

BI Barthel Index

Finally, we tested whether subgroups with different number of disabilities had different total scores on the SIS. Our analysis revealed significant differences between groups in total SIS scores, F(2,62) = 4.98, p < 01, η 2 = .14. Post-hoc comparisons with LSD indicated that the group with 1 disability (M = 242.0; SD = 72.8) required less support compared with the group with 2 (M = 291.1; SD = 86.3) and 3 or more disabilities (M = 329.6; SD = 87.5). Although the mean SIS score was higher for participants in the group of 3 or more disabilities compared to the participants with 2 disabilities, the difference was not statistically significant. These results indicate that the SIS discriminates between subgroups with different number of disabilities, which could be regarded as an indicator for construct validity.

Discussion

The present study investigated the internal consistency and the construct validity of the SIS in a sample of people with physical disabilities. Internal consistency was demonstrated for both the total SIS scale and the SIS subscales. The Cronbach’s alphas were comparable to the coefficients reported in previous research of Thompson and colleagues (2002) and Buntinx et al. (2008) including people with intellectual disabilities, albeit that the Cronbach’s alphas of SIS Medical and SIS Behavior were somewhat lower in the present study. A lower internal coefficient of the Behavior scale (.77) was found in the present study compared to research by Thompson et al. (2002) and Buntinx et al. (2008) (.98 and .86, respectively), both using samples of people with intellectual disabilities. In addition, the SIS Behavior subscale did not correlate significantly with other SIS subscales. An explanation may be found in the content of the items. That is, most items of the Behavior subscale refer to support needs of behaviors that are externalizing by nature (e.g., aggression and automutulation). In contrast, only two items (Maintenance of mental health treatments and Prevention of other serious behavior problems) are included that may capture more internalizing problems, such as depression and anxiety. It is possible that these externalizing behaviors are less relevant for people with physical disabilities. It is known from research that, in comparison to people without disabilities, individuals who are physically disabled have a higher risk to develop mood and anxiety disorders (McColl and Friedland 1993). Therefore, if support needs in people with physical disabilities are assessed, instruments should encompass a broader range of behavioral issues, including behaviors that are more relevant for individuals with physical disabilities.

In order to investigate the construct validity of the SIS, the relationship between support needs and practical skills was examined. Past research reveals that people with physical disabilities often have limitations in practical skills, such as toileting, bathing, and mobility (e.g., Andrén and Grimby 2000; Balandin and Morgan 1997). As Balandin and Morgan (1997) have suggested, these limitations may be accompanied by higher support needs in, for example, people with cerebral palsy. In accordance with this expectation, a moderate correlation was found between total SIS scores and the BI. Overall, these results indicate that participants had more support needs as they had more limitations in practical skills.

However, further analyses of the data revealed that the construct validity was evident in only four subscales of the SIS. As was expected, a strong correlation was found between SIS Home Living and BI, as the former typically requires such practical skills, as dressing and toileting. It was, however, hypothesized that limitations in practical skills would be also linked to support requirements in a range of life domains, as was suggested by some previous studies of support needs of people with physical disabilities (Andrén and Grimby 2000; Balandin and Morgan 1997). For example, stronger correlations were expected between BI-scores and Life-Long Learning, Employment, and Social Activities. For instance, it is reasonable that people with physical disabilities may also need assistance (in for example toileting or walking), while performing activities such as participating in educational activities (Life-Long Learning) or participating in recreational activities (Social Activities). In addition, the correlations between the BI and Community Living and Health and Safety exceeded the minimum criterion .35. However, these correlations were still lower than the correlations reported in a study with people intellectual disabilities of Harries and colleagues (2005), who found higher correlations between these SIS subscales and several subscales of the Adaptive Behavior Scale (ABS) and the Inventory Client and Agency Planning (ICAP) measuring practical skills (ABS Self-Sufficiency, ICAP Motor Skills and ICAP Personal Living Skills). Therefore, compared to previous research, this study reveals a weak relationship between limitations in practical skills and support needs in specific SIS domains, and additionally, construct validity was evidenced in only four out of eight SIS subscales.

An explanation for the weak relationship between the BI and the SIS subscales could be that the SIS subscales might have underestimated the limitations of practical skills associated with physical disabilities. In the study of Harries et al. (2005), in which scores of the ABS, ICAP and SIS section subscales were combined in an explorative factor-analysis, it was found that the SIS Support Need Scale and these adaptive behavior scales had the same underlying construct. Although initial analyses showed a strong association between SIS subscales and the subscales of the ABS and the ICAP, a three-factor solution showed that no SIS subscales were present in the third factor, representing the practical dimension. The outcomes revealed that, “considered in terms of the three adaptive behavior skill areas, the underlying construct related predominantly to the conceptual skills dimension” (Harries et al. 2005, p. 402). The authors suggest that the raters used “an individual’s conceptual skills as a framework of reference when assessing support needs” (p. 402). Although Harries and colleagues (2005) do not further explain this suggestion, it may be that the frame of reference is a bias in the SIS itself. It might be that support needs, as measured by the SIS, are based more on the limitations in conceptual skills (e.g., cognition and language) rather than limitations in practical skills (e.g., mobility, bladder and bowel control).

All subscales (except the Behavior subscale) were intercorrelated in the moderate to high range, comparable to the outcome of Buntinx et al. (2008). Moreover, the SIS was able to discriminate between groups who varied in the number of disabilities. Specifically, the group of people with a physical disability and 2 or 3 or more additional impairments were rated as having higher support needs than people with only a physical disability. Thus, the analysis of the construct validity reveals that the SIS measured the overall construct of support needs in the current sample. However, in contrast to the expectation that the SIS subscales would be related to the BI, this construct is only slightly based on limitations in practical skills.

Limitations and Future Research

Some limitations of this study should be mentioned. First most of the participants in the sample were diagnosed with cerebral palsy and only a small proportion consisted of people with Spina Bifida and TBI. Subgroups consisting of people with spina bifida and TBI were too small to conduct separate analyses. Future research is needed to investigate whether these groups differ in the nature of support needs. For example, many people with spina bifida may need support related to bladder and bowel control and shunt-related problems (Oi et al. 1996). In addition, people with TBI often need support in the behavioral domain (Soo et al. 2007), as they often exhibit externalizing behaviors (Ducharme 1999). Moreover, this study was conducted in a residential setting. It could be argued that people differ in their support needs across settings. Therefore, future research of the SIS should take into account other settings, such as ambulatory and semi-residential services. Furthermore, support needs of the participants were measured at one point in time. However, people with cerebral palsy often experience a decline in functional status and health status at a young age (Hemsley et al. 2007). As Thompson et al. (2002) argue, support needs are proposed to be dynamic, which indicates that “they change across settings, across situations, and over time” (p. 392). For this reason, future investigations should also focus on the test-retest reliability of the SIS. Equally important, in order to consider the changing nature of support needs in people with a physical disability, research should also focus on longitudinal assessments on individuals.

In summary, the present research was the first attempt to measure the psychometric properties of the SIS in people with physical disabilities. Although there are some indications that the SIS may be useful assessing support needs in people with physical disabilities, the SIS might not take sufficiently into account the limitations in practical skills of these people. For people with physical disabilities, future revisions of the SIS should take into consideration limitations in practical skills in a variety of support domains as measured by the SIS.