Factors that predict good Active Support in services for people with intellectual disabilities: A multilevel model.

BACKGROUND: Active Support, now widely adopted by disability support organizations, is difficult to implement. The study aim was to identify the factors associated with good Active Support.
METHODS: Data on service user and staff characteristics, quality of Active Support and practice leadership were collected from a sample of services from 14 organizations annually for between 2 and 7 years, using questionnaires, structured observations and interviews. Data were analysed using multilevel modelling (MLM).
RESULTS: Predictors of good Active Support were adaptive behaviour, practice leadership, Active Support training, and time since its implementation. Heterogeneity, having more than six people in a service and larger organizations were associated with lower quality of Active Support.
CONCLUSIONS: In order to ensure that Active Support is consistently implemented, and thus, quality of life outcomes improved, organizations need to pay attention to both service design and support for staff through training and practice leadership.

INTRODUCTION

Active Support is a practice designed to facilitate the quality of life of people with intellectual disabilities through engagement in meaningful activity and social relationships (Mansell & Beadle‐Brown, 2012). Developed during the late 1970s, the theoretical foundations of Active Support are in behavioural psychology. It has been widely adopted in the United Kingdom (UK) and Australia, and to a lesser extent in Scandinavia, the United States (US), Taiwan and New Zealand, and most commonly been used by staff in supported accommodation services (services) (Mansell, Beadle‐Brown, & Bigby, 2013).

A growing evidence base points to the positive impact of Active Support on the quality of life of people with intellectual disabilities (Flynn et al., 2018). A systematic review of 20 papers and meta‐analysis of the 14 studies reported in these indicated that Active Support was effective in changing the way staff interacted, moment to moment, with service users. Studies demonstrated significant improvement in the quality of staff support and assistance to residents to be engaged, leading to “significant increases in the amount of time residents spent engaged in all types of activities at home” (Flynn et al., 2018, p. 994). Although an association between Active Support and changes in residents’ depressive symptoms, challenging behaviour, adaptive skills, choice and community participation were reported across studies, Flynn et al.’s (2018) meta‐analysis did not demonstrate convergence on the direction or significance of change for any of these factors. The evidence, albeit limited, of an association between Active Support and reduction in challenging behaviour suggests its complementarity to behavioural support strategies. For example, Ockendon, Ashman, and Beadle‐Brown (2017) argued that Active Support is a foundational element of Positive Behaviour Support (PBS), setting the context for its successful implementation, and McGill, Ashman, and Beadle‐Brown (2014) demonstrated Active Support as an integral component of PBS, which was associated with reductions in challenging behaviour. From a staff perspective, Active Support has been found to be associated with increased staff job satisfaction and a lower propensity for staff to leave their employment (Beadle‐Brown, Hutchinson, & Whelton, 2012; Rhodes & Toogood, 2016).

Although the benefits of Active Support in terms of increased resident engagement appear unequivocal, experience of its implementation has not been straightforward. The quality of Active Support may decline over time, and staff training or organizational adoption of Active Support has not always led to practice changes or increased resident engagement. For example, in an Australian study of 33 services managed by six organizations that had adopted Active Support more than five years previously, only one organization was found to be delivering good Active Support (Mansell et al., 2013). Mansell, Beadle‐Brown, Whelton, Beckett, and Hutchinson (2001), in a UK‐matched sample study of services in general supporting those with less severe disabilities in which staff in 36 of the 72 houses staff were trained in Active Support, found that only 53% of residents were receiving good Active Support. Studies of more severely disabled populations have generally found that only between one fifth and one third of people are receiving good Active Support (Beadle‐Brown et al., 2016). Such findings have led to the question “what factors influence the extent to which staff provide Active Support?” (Mansell, Beadle‐Brown, Whelton, Beckett, & Hutchinson, 2008, p. 399). Many possible explanations have been proposed, but the evidence has been limited (see Bigby & Beadle‐Brown, 2018).

Factors thought to influence quality of Active Support fall into three groups: (a) staff training in terms of type, take up and coverage—for example, Qian, Tichá, and Stancliffe (2017); (b) staff motivation, in terms of qualifications, competing demands and quality of leadership—for example, Mansell et al. (2008) and Mansell and Elliott (2005); and (c) management commitment, demonstrated through support from managers and organizational processes—for example, Fyffe, McCubbery, and Reid (2008) and Mansell et al. (2008). Mansell et al. (2008) argued there was as yet no clear understanding of organizational factors that facilitated Active Support, but they were likely to operate in combination and could be situation‐specific. Flynn et al. (2018) found tentative evidence in their synthesis of 10 studies about the positive influence on Active Support of training comprised of classroom and interactive elements, settings with relatively low staff‐to‐resident ratios, services with relatively more residents (up to a maximum of 6), organizational leadership, and management support and processes, such as team meetings. Another strand of work has shown a weak but positive correlation between good Active Support and strong front‐line practice leadership (Beadle‐Brown, Bigby, & Bould, 2015), or at least the presence of a practice leader in a service (Bould, Beadle‐Brown, Bigby, & Iacono, 2018a).

The strength of studies into the factors associated with good Active Support has been limited by use of staff self‐report data about the strength of the five elements of practice leadership (Beadle‐Brown et al., 2014; Mansell et al., 2008), which has since been shown to differ considerably from use of an observational measure (Bould, Beadle‐Brown, Bigby, & Iacono, 2018b). Further, in the largest studies to date, no account has been made of multilevel data, such that individuals living in the same house may be assigned different scores relating to an individual trait, but the same score as others in the house on a measure relating to a trait of the house. As a result, data at the level of the individual with intellectual disability are treated the same as data entered at the level of the service into linear regressions. For example, Mansell et al. (2008) accounted for 44% of the variance in Active Support scores across 72 services, but data dependency within clusters in their multiple regression analyses was evident; as a result, groups of individual service users from the same service would have shared the same scores for certain variables, such as staff training or ratios. Not accounting for this aggregation of group‐level data increases the likelihood of type 1 error: that is, finding an effect that may not be there (Raudenbush & Bryk, 2002; Snijders & Bosker, 1999). Since this and other large‐scale studies of Active Support, researchers have applied statistical analyses that accurately accommodate data from multiple levels in studies of services (Qian, Tichá, Larson, Stancliffe, & Wuorio, 2015).

The aim of the present study was to identify factors associated with individuals, services and organizational variables that predict the quality of Active Support using multilevel modelling (MLM). The data were drawn from a longitudinal study of services in Australia involving repeated data collection at 12‐18 month intervals. Since 2009, when the study commenced, additional organizations have joined, bringing the total to 14 by 2017. The data reported in the present study are from a cross‐sectional sample taken from the longitudinal study.

METHODS

Design

The study was a repeated cross‐sectional design in which data were collected from 2009 to 2017, at 7 time points. Consent was obtained from staff and service users, or, for those without consent capacity, from a person who usually made decisions for them, typically a parent or senior staff member of the service. The study received approval from the University Human Research Ethics Committee.

Data collection points were anchored to each organization and did not necessarily coincide with the same calendar year or number of years in the study.

Participants and settings

A total of 461 service users from 134 services managed by 14 not‐for‐profit organizations participated in the study. Services provided 24‐hr support for 1–12 people (M = 4.84) in ordinary community houses. As Table 1 shows, the number of services managed by each organization varied from 5 to 34, and the time since they first adopted Active Support varied from 1 to 14 years.

Table 1

Number of supported accommodation services managed by each organization and years implementing Active Support

Organization	Total number of services managed	Total number of service users	Number of years implementing Active Support
1	5	21	8
2	15	28	14
3	5	18	13
4	34	155	12
5	25	100	6
6	7	29	5
7	10	62	5.5
8	33	138	11
9	27	140	2
10	38	131	9
11	23	66	2
12	7	42	1
13	16	78	1
14	31	142	1

The seven time points at which data were collected reflected differences in when organizations joined the study. In order to increase the sample size, data collected from different services from the same organization in different years were also included. Table 2 shows the number of services, service users and staff included in the analysis from each organization at each time point.

Table 2

Number of services, consenting service users (SUs) and staff surveys from each organization included in the analysis at each time point

Organization	1	2	3	4	5	6	7	8	9	10	11	12	13	14	ALL
Time Point 1
Services	0	0	0	0	4	0	0	7	0	2	0	0	0	0	13
Service users	0	0	0	0	9	0	0	25	0	7	0	0	0	0	41
Staff	0	0	0	0	24	0	0	25	0	6	0	0	0	0	55
Time Point 2
Services	0	0	0	0	5	1	1	4	3	1	3	5	4	5	32
Service users	0	0	0	0	26	4	4	16	11	4	6	28	17	13	129
Staff	0	0	0	0	15	3	3	12	9	4	14	20	12	15	107
Time Point 3
Services	0	0	2	4	4	2	1	4	4	3	5	0	0	0	29
Service users	0	0	5	11	16	8	5	15	14	13	11	0	0	0	98
Staff	0	0	6	16	16	10	7	15	21	11	15	0	0	0	117
Time Point 4
Services	1	1	0	4	5	0	0	6	0	0	0	0	0	0	17
Service users	5	1	0	15	13	0	0	23	0	0	0	0	0	0	57
Staff	7	3	0	13	15	0	0	23	0	0	0	0	0	0	61
Time Point 5
Services	2	2	1	6	5	4	5	6	0	0	0	0	0	0	31
Service users	5	2	3	18	16	13	18	21	0	0	0	0	0	0	96
Staff	6	6	5	18	15	12	15	36	0	0	0	0	0	0	113
Time Point 6
Services	0	0	0	4	0	0	0	0	0	0	0	0	0	0	4
Service users	0	0	0	11	0	0	0	0	0	0	0	0	0	0	11
Staff	0	0	0	20	0	0	0	0	0	0	0	0	0	0	20
Time Point 7
Services	0	3	1	4	0	0	0	0	0	0	0	0	0	0	8
Service users	0	7	5	17	0	0	0	0	0	0	0	0	0	0	29
Staff	0	11	3	13	0	0	0	0	0	0	0	0	0	0	27

As part of the larger longitudinal study, data were collected from the same services at multiple data points and, where this was the case, the data point which showed the mean highest level of Active Support across service users in a service was included in the current data set.

Measures

Quality of active support

The Active Support Measure (ASM) (Mansell et al., 2018) indicated the quality of Active Support, and, hence, was the predicted variable. It has been used in multiple studies of Active Support and its use ensured comparability of results (see Mansell & Beadle‐Brown, 2012). The ASM is completed for each service user at the end of a 2‐hr observation period. It has 15 items, with each rated on a scale of 0 (poor, inconsistent support) to 3 (good, consistent support) to yield a maximum score of 45, which is converted to a percentage. A percentage over 66.66 is considered a good level of Active Support (Mansell & Beadle‐Brown, 2012). Two items relate to support for people showing challenging behaviour—if none is observed, these two variables are missing, giving a maximum score of 39.

Across the seven time points, data for the ASM were collected by 13 observers, with each trained by one of the authors using video material and having completed at least two observations with an experienced observer before collecting data alone. Percentage agreement across the 15 items of the ASM for the seven observers involved in Time Point 1 and the two observers at Time Point 2 was 60% on average (range 29% – 98%, n = 24). Average Kappa was 0.32: the low score reflected frequent non‐occurrence of Active Support. ASM inter‐rater reliability was not conducted for Time Point 3 because all observations were by one observer. At Time Point 4, percentage observer agreement across three observers averaged 84% (range 73%–100%, n = 15), and average Kappa was 0.61 (range 0.21–0.80) (for more detail, see [removed for review]). At Time Points 5, 6 and 7, percentage agreement across four observers averaged 66% (range 55%–100%, n = 10); 58% (range 30%–100%, n = 10); and 87% (range 69%–100%, n = 26), respectively. Average Kappa 0.55 (range 0.20–0.100); 0.51 (range 0.29–0.100); and 0.73 (range 0.53–0.100), respectively. Despite low agreement for some ASM items, paired t tests showed agreement for the overall score obtained (i.e. on the basis of the maximum score) was not significantly different at each time point (range p = .271 to p = .385).

At each time point, organizational, service and service user‐level data were collected as predictor variables. Organizational‐level data were the number of services managed, total service users supported, and time since adoption of Active Support. Service‐level data were staff‐to‐resident ratios during the 2‐hr observation, and measures of staff experiences and satisfaction, and front practice leadership. Service user‐level data were measures of adaptive behaviour and other characteristics.

Staff‐to‐resident ratio

A proforma completed by the observer was used to record the numbers of residents present and staff on duty during the 2‐hr observation. The staff‐to‐resident ratio was obtained by dividing the number of staff by the number of residents.

Staff experiences and satisfaction survey

An adapted version of the Staff Experiences and Satisfaction Questionnaire (SESQ) (Beadle‐Brown, Gifford, & Mansell, 2005) was completed by staff in services. It includes three sections: (a) demographics and training; (b) experiences at work—satisfaction, role clarity and conflict, and perception of practice leadership and quality of senior management; and (c) attitudes towards people with intellectual disabilities.

The observed measure of practice leadership

Developed by Beadle‐Brown et al. (2015), this measure provided data on the five elements of practice leadership: (a) overall focus on the quality of life of the people supported by the service; (b) allocation and organization of staff; (c) coaching, observing, modelling and giving feedback to staff about the quality of their support; (d) reviewing performance with individual staff in supervision; and (e) reviewing team performance in team meetings. It was completed using (a) unstructured observations of the front‐line manager during the service visit; (b) semi‐structured interviews with the front‐line manager and, where possible, direct support staff; and (c) review of paperwork associated with practice leadership, such as team meeting minutes and staff allocation. Based on this information, observers rate the five elements of practice leadership on a five‐point rating scale (1 indicating no/ almost no evidence of the element being in place to 5 indicating excellent—this element could not be improved). The scores for each element are equally weighted and tallied to give a mean score of the overall strength of practice leadership provided by the service's front‐line manager. Data for this measure were collected by five researchers who had been trained by one of the authors and conducted at least two visits with a trained observer before collecting data alone. The measure was developed during the early stages of this study and was described in detail in Beadle‐Brown et al. (2015). The measure has been shown to be a valid and reliable measure, with good internal consistency across several studies (Cronbach alpha over 0.9), acceptable inter‐rater reliability (average Kappa value over 0.6 across the five domains) and good construct validity in terms of good discriminatory power for the main outcome measure (the active support measure)—better practice leadership was consistently associated with higher levels of active support (e.g. t (171) = 3.88, p < .001 in Beadle‐Brown et al., 2015).

Service user characteristics questionnaire

An audit questionnaire included the short form of the Adaptive Behavior Scale (SABS) Part 1 (Hatton, 2001 and, the Aberrant Behavior Checklist (ABC) (Aman, Burrow, & Wolford, 1995). The reliability and validity of the Adaptive Behaviour Scale (ABS, from which the SABS was drawn), and the ABC have been studied and reported as acceptable by their authors. The full‐scale score for Part 1 of the ABS was estimated from the SABS using the formula provided in Hatton et al. (2001). Additional socio‐demographic information was obtained for each service user.

Procedures

Services were selected at each time point from a deidentified audit database of service and service user characteristics. Each database entry had a unique code for each service and service user. The unique code had been generated by a contact person in the organization, who followed instructions on how to replace names with codes in the database, and to replace names of service users on questionnaires completed by a staff member who knew the individual well with this code. The deidentified code database and questionnaires were returned to the research team in pre‐paid. The deidentified coded databases were returned electronically to the research team, and the deidentified questionnaires were returned in pre‐paid envelopes. The audit database was updated every 12 months, and audit questionnaires were completed for any new service users.

The audit database was used to select a sample for each time point and ascertain the total number of services managed and service users supported by each organization. Information and consent forms were sent to each organization to be distributed to selected services, staff and service users. For the study to proceed in any service, consent of at least one service user was required; then, staff questionnaires distributed to consenting staff via supervisory and managerial staff associated with each service. Completed staff questionnaires were returned directly to the researchers by mail in a pre‐paid envelope.

A researcher then visited each service to conduct the 2‐hr observation and complete the ASM for each consenting service user. On another day, a researcher visited the service to complete the Observed Measure of Practice Leadership. Hence, two visits were made to each service, within 2–4 months unless services shared a front‐line manager, in which case only one visit was made across these services to complete the practice leadership measure.

Analyses

Data were entered into IBM SPSS 24, and descriptive statistics and correlational analyses conducted to examine relationships among predictors. Cohen’s (1988) guidelines were used to report effect sizes where appropriate. For each service user, an ASM percentage was calculated, and an ABS score derived (Hatton et al., 2001). The ABS score was used to categorize each service user as <80, 81–150 or 151 and above These data were aggregated to the service level to ascertain the number of service users in each ABS category, and the number of ABS groups. For example, if two of five service users in a service was in the 81–150 ABS group and three in the 151 + group, the number of ABS groups for that service was two. Also at the service level, a mean score was calculated across the five elements of the Observed Measure of Practice Leadership for each service or services in which the front‐line manager worked. The unique codes from each organization derived from the audit database were used to ascertain the total number of service users in each service, which were grouped into two categories based on earlier studies by Tøssebro (1995) and Flynn et al., (2018): 1–6 and 7+. These aggregated data were assigned to all the individual service users within the same service(s).

The criterion for inclusion of data in the analysis was a minimum of three staff questionnaires returned for a service. Individual staff scores on job satisfaction, role clarity and conflict, perception of practice leadership, quality of senior management and attitudes towards people with intellectual disabilities were calculated for each service. These data, along with data on training in Active Support, were aggregated to the service level using a mean score for each service, which was subsequently assigned to all the individual service users within the same service. Some services were excluded from the analyses because of missing data, resulting in final totals of 461 service users from 134 services, managed by 14 organizations.

Finally, for each time point, the unique codes from the audit database were used to ascertain the total number of services and service users supported by the organization. These data, along with the number of years implementing Active Support, were included at the organizational level and subsequently assigned to all the individual service users within the same organization.

Table 2 presents data on the number of services, consenting service users and surveys from staff that were included in the final analysis at each time point. Taking into account the data clustering, there were four levels: Level 1, individual service users (461); Level 2, services (134); Level 3, the data collection time point (cross‐sectionally, there were 39 groups, formed by the combination of services included across the 7 time points from the 14 organizations); and Level 4, organizations (14).

MLM regression was implemented using the MLwiN program (version 3.02; Charlton, Rasbash, Browne, Healy, & Cameron, 2017). The size and structure of the data set required the variance to be partitioned at four levels. Because of the small number of organizations the Markov chain Monte Carlo (MCMC) (Browne, 2017), estimation was used to calculate the deviance information criterion (DIC) (Spiegelhalter, Best, Garlin, & van der Linde, 2002) statistics for model comparison (Rodriguez, 2007). Using this approach, any decrease in the DIC (goodness‐of‐fit diagnostic) indicates a better model fit. All models were estimated using non‐informative priors (Browne, 2004) with a burn‐on of 1,000 and 20,000 iterations to allow each model to converge on the correct posterior distribution, and collect sufficient independent samples from the posterior distribution to permit a good estimate. An initial null model (i.e. includes no predictor variables) was estimated which also computes an intraclass correlation coefficient (ICC): that is, the proportion of the total residual variance attributable to differences between groups, referred to as the variance partition coefficient (VPC) (Goldstein, 2003). The formula for calculating the VPC is the ratio of the variance at each level to the total variance. Subsequently, a series of multilevel models were built using a bottom‐up approach (Hox, 2010; Raudenbush & Bryk, 2002). The fully adjusted model was:

Here, i refers to the service user, j the service, k the time point and l the organization. β refers to the grand mean (i.e. average Active Support score for 461 service users from 134 services across seven time points from 14 organizations), and e refers to a random effect.

Results were considered significant if the estimates were more than twice their estimated empirical standard error. All predictors were grand‐mean‐centred (the intercept was centred around the mean of the sample) to facilitate the interpretation of the intercepts and slopes, and because the influence at the higher levels (service and organization) was of primary interest (Enders & Tofighi, 2007).

RESULTS

As can be seen from the descriptive statistics for service users at each time point in Table 3, service users had a varied profile of needs and characteristics, and on average, the sample was relatively able compared to those in previous studies (e.g. Mansell et al., 2013). Table 4 provides the results of the correlational analyses (at the service user level), used to examine relationships among predictors included in the final model. The largest correlation with the quality of Active Support (ASM Score) was the level of adaptive behaviour (ρ = .400, n = 461, p < .001), with a medium effect (Cohen, 1988).

Table 3

Characteristics of service users at each time point and the quality of Active Support

Variable	Descriptive Statistic	Time point
		All	1	2	3	4	5	6	7
	N/n	461	41	129	98	57	96	11	29
Age (years)	M	47	40	48	46	50	47	43	45
	Range	20–87	22–63	21–87	21–81	27–70	20–81	28–65	21–70
Males		53%	51%	54%	49%	53%	56%	27%	59%
ABS score	M	150	154	149	140	163	156	122	147
	Range	22–272	36–253	31–272	22–263	67–249	31–272	36–201	72–251
ABC total score	M	27	26	29	26	26	24	46	21
	Range	0–119	0–107	0–119	0–97	0–104	0–81	3–87	0–68
Non‐verbal		25%	24%	26%	27%	28%	21%	36%	10%
ASM score	M	65	63	60	60	60	80	63	67
	Range	13–100	13–97	13–97	15–97	31–85	33–100	43–87	18–92

Table 4

Spearman correlations between predictor variables

	ABS Score	Mean Practice Leadership	Service size ‐ Total number of SUs	Number of ABS Groups	Staff with training in Active Support	Organization Size: Total number of services	Years implementing Active Support
ASM	0.400b	0.274b	0.030	−0.282b	0.161b	−0.234b	0.052
ABS Score		0.009	0.160b	−0.191b	0.078	0.002	0.030
Mean practice leadership			0.108a	0.038	−0.019	−0.215b	−0.114a
Service size—total number of SUs				0.326b	0.003	−0.107a	−0.171b
Number of ABS groups					−0.111a	0.160b	0.008
Staff with training in Active Support						−0.083	−0.006
Organization size: total number of services							0.519b

Correlation is significant at the 0.05 level (2‐tailed).

Correlation is significant at the 0.01 level (2‐tailed).

Table 5 presents the modelling results as parameter (beta) coefficients and their standard errors, along with the model‐fitted diagnostic DIC. Model 1 is the null, which includes no predictor variables, and the VPC indicated 6% of the variance in the ASM scores were accounted for by differences between organizations, 11% by differences between time points, 54% by differences between services and 29% by differences between individual service users. In Model 2, individual predictors were included. Only one predictor (ABS Score) was significant, indicating that individuals with greater adaptive behaviour received higher scores on the ASM. Model 3 included variables associated with the service; higher practice leadership scores and a higher percentage of staff who had received training in Active Support had higher ASM scores. Conversely, services with 7 + service users and services with a great deal of heterogeneity among service users (i.e. ABS scores falling within each of the three ABS groups) were associated with lower ASM scores. No other service‐level variables contributed to the model. Model 4 allowed for examination of variables associated with the organization. Inclusion of two organization variables resulted in a further improvement in the model‐fitted diagnostic DIC. Greater time implementing Active Support was associated with higher ASM scores. However, organizations larger in size (as indicated by the total number of services managed) were associated with lower ASM scores. No other organization‐level variable contributed to the model.

Table 5

Parameter (beta) estimates of the multilevel models and deviance information criterion (MCMC)

	Model 1 (SE)	Model 2 (SE)	Model 3 (SE)	Model 4 (SE)
Fixed parameters
Constant	64.711 (2.432)	65.106 (2.391)	68.815 (2.023)	67.152 (1.594)
Individual (level 1) predictors
ABS Score		0.14 (0.012)	0.127 (0.013)	0.129 (0.013)
Service‐level (level 2) predictors
Mean practice leadership			7.889 (1.634)	7.178 (1.542)
Service size—total number of SUs (6 or less base)			−10.254 (5.158)	−10.871 (4.889)
Number of ABS groups in the service (1 or 2 ABS groups base)			−8.626 (3.367)	−6.9 (3.245)
Staff with training in Active Support			7.797 (5.191)	8.023 (4.882)
Organization‐level (Level 4) predictors
Organization size: total number of services				−0.524 (0.142)
Years implementing Active Support				0.776 (0.391)
Random parameters
Level 4: Between organizations	31.532 (48.908)	36.593 (46.606)	19.091 (27.364)	3.651 (8.95)
Level 3: Within organizations, between‐ data collection time point	55.323 (42.104)	53.808 (37.995)	11.681 (18.348)	9.283 (12.799)
Level 2: Between services	272.43 (41.706)	197.74 (30.172)	153.097 (28.951)	141.78 (25.711)
Level 1: Within individuals	146.659 (10.093)	122.007 (8.401)	123.361 (9.77)	123.494 (9.751)
Deviance information criterion (DIC)	4,897.284	4,768.525	4,608.863	3,634.669
Change in DIC		128.759	159.662	974.194

All estimates are significant at 0.05 probability level or smaller.

Although there remained significant variance at each of the four levels, as indicated in Figure 1, the predictors included in Model 4 accounted for 88% of the between organization variance, 83% of the between data collection time point variance, 48% of the between service variance and 16% of the within‐individual service user variance.

Figure 1

Factors that predict good Active Support

DISCUSSION

This study extended previous research into factors predictive of the quality of Active Support using linear regression analysis, in particular, using MLM to test for variables at the person as well as group levels (service and organization). MLM allowed examination of multiple factors with potential to influence the quality of Active Support, while accounting for variability in scores across levels that occur in disability service provision.

At the service user level, the finding that only one factor, a higher level of adaptive behaviour, was predictive of better quality of Active Support is consistent with previous research (Mansell & Beadle‐Brown, 2012). As the ASM provides a measure of the quality of Active Support relative to the context and characteristics of each service user, these findings suggest two things: first, staff are not skilled in tailoring Active Support to the needs of each individual, and second, staff are less skilled in supporting people with lower levels of adaptive behaviour. This explanation aligns with the finding that, at the service level, the percentage of staff trained in Active Support was also predictive of the quality of support, again as found previously (Mansell & Beadle‐Brown, 2012). Originally developed in services for people with severe and profound disabilities, staff use of Active Support aims to compensate for the difficulties the people they support have in initiating engagement and completing tasks. Of relevance to people who are already able to engage relatively independently in a range of activities and interactions, is the potential for Active Support to create more opportunities for engagement or support engagement in more complex activities. A key principle of Active Support is the adaption of support to the level of ability as well as each individual's other needs and preferences. This requires skills in, for example, giving intensive hand‐over‐hand assistance to individuals with profound intellectual disability, as well as knowing when and how to stand back to give a more able person time to complete a task independently. While potential levels of engagement of some people with severe disabilities may be lower than those with less severe disabilities due to their capacity to sustain physical involvement, energy and attention, the quality of Active Support should be similar.

The present study provided less clarity than previous studies about the nature of staff training, although findings did reflect previous evidence about the significance of training per se to the quality of Active Support (Flynn et al., 2018). The relevance of the type of training could not be explored because few staff reported this information, thereby precluding evaluation of previously identified advantages of classroom combined with in situ training (Flynn et al., 2018). Nonetheless, it could be argued that the effectiveness of training staff in situ as found previously (Flynn et al., 2018) is reflected in the finding that strong practice leadership is predictive of good Active Support. At least two features of practice leadership, coaching and individual performance review, are also elements of in situ Active Support training. Importantly too, the present study provides stronger evidence than previously obtained by Beadle‐Brown et al. (2014) and Beadle‐Brown et al. (2015) about the positive influence of front‐line practice leadership on quality of Active Support.

The use in this study of the Observed Measure of Practice Leadership avoided the overestimation and social desirability effects identified when self‐report measures of practice leadership have been used (Bould et al., 2018b). Further, this measure brings previously identified factors, such as frequency of supervision and staff meetings, staff awareness of task allocation, and supportive leadership (Mansell et al., 2008) into the unifying concept of front‐line practice leadership.

Service structure characteristics have received some attention in previous research. Although Flynn et al. (2018) suggested the evidence was only tentative, they identified that larger settings (within a maximum of 6 service users), and lower staff‐to‐user ratios facilitated the implementation of Active Support. In the present study, having seven or more service users in a service was negatively associated with the quality of Active Support and no effect related to staff‐to‐user ratios was found. This first finding supports Tøssebro’s (1995) evidence about the importance of small‐sized services and is also consistent with current policies in Australia, the UK and Ireland.

A novel finding of the present study was that the quality of Active Support was negatively predicted by very heterogeneous groupings of residents, defined in this study, as a service including individuals who fell within each of three ABS groups—ABS scores of less than 80, 81–150 and 151 and above. This finding may reflect the difficulties in tailoring Active Support to service users’ individual support needs, which may be compounded in services with very heterogeneous residents. However, these three groupings span a very wide range of abilities and service user groupings comprising any two rather than three ABS groups were associated with higher Active Support scores. Hence, a complex pattern that links service user with service‐level characteristics is apparent, adding to limited knowledge about groupings of service users. To date, this information has been confined to the effects of grouping people with challenging behaviour and those with severe mobility difficulties, which have shown an advantage of heterogeneity among service users (Mansell, Beadle‐Brown, Macdonald, & Ashman, 2003).

This study is the first to be able to explore the impact of organizational‐level variables, because almost all previous large‐scale studies of Active Support were conducted in only one organization. Although data were available for only a few organizational‐level variables, a large proportion of the variance between organizations (88%) was explained by the predictors in the model. The length of time since Active Support had been adopted was associated with higher quality support. Managing more services was associated with poorer quality support. This pattern may speak to Mansell et al.’s (2008) argument about the destabilizing effect of organizational change, with the converse of a stabilizing effect of having Active Support bedded down over a number of years implied. The notion of stability may extend to problems in maintaining it across a large number of services, which, in turn, could create challenges for ensuring consistent and high‐quality practice leadership and maintaining training in Active Support across all staff.

Limitations and Directions for Future Research

Some limitations with the study point to potential directions for research to further understanding of contributors to good quality Active Support. The reliance on staff self‐report yielded limited data about the type of training received. In future, training items in the staff questionnaire could be complemented with data from organizational training records. In addition, staff culture, frequently proposed to influence the quality of support (Flynn et al., 2018), was not included due to the lack of a measure relevant to intellectual disability services. Notably, such a measure has recently been developed through the doctoral work of Humphreys (2018).

The service‐level measures yielded data about staff characteristics and factors associated with staff motivation, sometimes referred to as “organisational hygiene” (Mansell et al., 2008, p. 399), such as staff qualifications and attitudes, but these dropped out early in analyses as they failed to predict Active Support. It is possible that some variables were accounted for by others, such as staff training. Further research that allows for exploration of how certain variables subsume others, such as through analyses of latent variables, could better explain interactions among them. Such analyses do require large sample sizes at all levels, which can prove logistically difficult, as demonstrated in the present study that relied on combining data collected over several years.

Finally, the repeated cross‐sectional design did not permit the study of individual change, or factors that sustain its quality over time. Findings from the longitudinal study reported in further articles are anticipated to address these limitations.

Further research about effective ways of delivering practice leadership is warranted given its significant role in the quality of Active Support. This is particularly pertinent at a time when changes to funding formulae and recognition of the administrative burden on front‐line managers (Clement & Bigby, 2012) are generating new structures for delivering practice leadership that move away from the model of one supervisor per service.

Practice Implications

These results highlight the need for attention to the quality of Active Support for people with severe and profound intellectual disabilities and thus their engagement‐related support needs. They suggest a case for a greater focus on skilling staff in tailoring Active Support to individuals with differing levels of ability, especially people with high support needs. This group often have low visibility in services, cannot complain using standard procedures, often do not have family or advocates as they are “known well by no one” (Bigby, 2008, p.148) and are poorly represented by self‐advocacy organizations (Bigby & Henderson, 2018; Petri et al., 2017) . The study demonstrates that the potential of Active Support to improve the quality of life of people with severe and profound intellectual disabilities is not being realized and reinforces the need for independent service audits that include observation of quality of support for service users with more severe intellectual disabilities who cannot self‐report satisfaction or service quality.

These findings further demonstrate the significance of adequate funding of front‐line practice leadership, which has been under threat in Australia from pricing models in the National Disability Insurance Scheme and, in the UK, from austerity measures. Practice leadership may be particularly important for motivating staff and providing coaching to develop nuanced skills in supporting service users with varied ability levels. Strong practice leadership could also maximize the potential of Active Support, as an integral part of PBS, in improving the quality of life of service users with challenging behaviours.

CONCLUSIONS

The contributions of this study emanate from being the largest investigation into Active Support in services in Australia and in evaluating the multilevel nature of factors at individual service user, service and organizational levels that predict its quality. The MLM model addressed shortcomings arising from analysis based on erroneous treatment of nested variables, such as individuals nested within services. The results highlight how characteristics of disability services interact to reveal features warranting service provider, funder or regulator attention in ensuring the delivery and monitoring of high‐quality Active Support. Specific attention is needed to (a) training and practice leadership to improve staff skills in tailoring Active Support to each service user; (b) improving the quality of Active Support for people with lower levels of adaptive behaviour; (c) full coverage of Active Support training among staff teams; (d) strength of front‐line practice leadership; and (e) resident groupings and size of services.

As well as providing indicators of high‐quality Active Support for those involved in service production, the findings are also relevant to families and advocates of people with intellectual disabilities to assist them in selecting quality services and exercising their rights in the market place as consumers. Also evident from this study is that organizations must do more than simply claim to have adopted Active Support and that the exercise of meaningful choice about the quality of services requires some form of independent evidence about its continued implementation and quality of delivery for all service users.