The Problem Behaviour Checklist: short scale to assess challenging behaviours.

BACKGROUND: Challenging behaviour, especially in intellectual disability, covers a wide range that is in need of further evaluation. AIMS: To develop a short but comprehensive instrument for all aspects of challenging behaviour.
METHOD: In the first part of a two-stage enquiry, a 28-item scale was constructed to examine the components of challenging behaviour. Following a simple factor analysis this was developed further to create a new short scale, the Problem Behaviour Checklist (PBCL). The scale was subsequently used in a randomised controlled trial and tested for interrater reliability. Scores were also compared with a standard scale, the Modified Overt Aggression Scale (MOAS).
RESULTS: Seven identified factors - personal violence, violence against property, self-harm, sexually inappropriate, contrary, demanding and disappearing behaviour - were scored on a 5-point scale. A subsequent factor analysis with the second population showed demanding, violent and contrary behaviour to account for most of the variance. Interrater reliability using weighted kappa showed good agreement (0.91; 95% CI 0.83-0.99). Good agreement was also shown with scores on the MOAS and a score of 1 on the PBCL showed high sensitivity (97%) and specificity (85%) for a threshold MOASscore of 4.
CONCLUSIONS: The PBCL appears to be a suitable and practical scale for assessing all aspects of challenging behaviour. DECLARATION OF INTEREST: None. COPYRIGHT AND USAGE:

There is increasing concern about problem behaviours in many forms of psychiatric care, and research has been handicapped by the absence of formal incorporation of these behaviours into diagnostic systems. Although there are several instruments that record these features,[1] many have poor internal consistency and reliability or only assess one component of challenging behaviour, and others with much better psychometric properties such as the Aberrant Behaviour Checklist[2] are a little long and not ideal for repeat assessments. We describe the development of a seven-item 5-point scale, the Problem Behaviour Checklist (PBCL), and tested its reliability and utility in practice.

Problem behaviours, mainly in people with intellectual disability, cover a wide range of disturbance, are a source of considerable distress to hospital staff and carers[3,4] and are often expensive to manage in practice, especially at more severe levels.[5] Assessment is handicapped by the absence of a satisfactory diagnostic system for recording challenging behaviour and the overlap with existing diagnoses such as personality and mood disorders.[6] Thus, although the prevalence of challenging behaviours is higher than any formal diagnostic group,[7] the fact that it is not recognised diagnostically makes it even more important for it to be reliably and consistently assessed. In the course of research into interventions for these problems, we recognised the need to examine the full range of behaviours reported as challenging and felt that these could be condensed into a much shorter instrument.

Method

Participants

During a randomised trial on the management of aggressive challenging behaviour[8] it was observed that some forms of challenging behaviour apparently independent of aggression were not identified and these were noted. We subsequently attempted to encompass the range of all potential behaviours suitable for inclusion by close examination of two international comprehensive descriptions.[9,10] Two studies were then involved in testing the scale. In the first, a field study was carried out for the World Health Organization on personality status and aggressive challenging behaviour in patients with intellectual disability in Jamaica.[11] The participant population was selected from a specialist intellectual disability high school (School of Hope), a supported care home and an adult day centre for people with intellectual disabilities, all operated by the Jamaican Association on Intellectual Disabilities (JAID) in Kingston, Jamaica.

In the second study, linked to a trial of nidotherapy (details available from the author on request) in the treatment of challenging behaviour in 200 residents in care homes, the same list of behaviours was recorded for all residents at monthly intervals over the course of at least 1 year. The projects were ethically approved by the JAID and North West Wales Research Ethics Committee (10/WNo01/1).

Statistical analysis

Each of the possible challenging behaviours were scored using a 4-point scale. A key aim of the analysis was to understand the associations between the different behaviours, and factor analysis was the main tool of investigation, using a standard Varimax rotation. An individual's behaviour was considered to be associated with each factor if the factor loadings for that variable were greater than 0.5. Separate factor analyses were performed for each of the two data-sets. The scale, called the PBCL, was created after the first analysis.

A secondary objective was to compare the levels of agreement in the total scores of the scale in a subset of the data where the scores were determined by two different observers. During the course of the randomised trial, each of the two independent observers (A.T. and R.E.) visited at different times. A large proportion of the scores on the PBCL were zero and to avoid spurious agreement the scores were divided into five categorical groups (0, 1–3, 4–6, 7–11 and ≥12). Weighted kappa was used to examine the level of agreement between observers.

Concurrent validity was also assessed by examining scores on a well-established scale for aggression, the Modified Overt Aggression Scale (MOAS).[12] Receiver operating characteristic curves were used to identify the optimum cut-off point for the total score in the prediction of aggressive challenging behaviour, defined as an MOAS score of 4. The sensitivity and specificity at this cut-off point were calculated.

Results

Thirty-seven potential types of challenging behaviour were identified from study of the literature (Table 1) but because several of these appeared to be very similar the number was reduced to 28 in the final analysis (Table 1). Factor analysis revealed seven discrete factors, personal and property violence, self-harm, sexually inappropriate behaviour, contrary behaviour, demanding and difficult behaviour, and wandering. Several of these made only a small contribution to the total variance, but at this stage it was felt they were sufficiently distinct to be included. Together these factors accounted for 91% of the variance in the Jamaican study with a Cronbach's alpha of 0.7. The first factor explained around a third of the variation in the data, with components 2 and 3 explaining more than 10% of the variation in the data. Components 4–10 explained a minor proportion of the variation. Each element was scored in terms of severity, with degree of disturbance and risk being the main driving force leading to higher scores (Table 2). The final PBCL (Appendix) therefore comprised seven problem behaviour groups with five levels of severity. Aggressive behaviour was by far the most common of these. In the second factor analysis, threatening, violent, demanding and contrary (oppositional) behaviour clustered with the aggressive factor, with sexually inappropriate and self-harming behaviour accounting for much less variance (Table 3).

Table 1

Data recorded in the Jamaican study of 37 challenging behaviour variables[11]

	Score 0	Score 1	Score 2	Score 3
Group	n (%)
Verbal behaviour	19 (50)	8 (21)	6 (16)	5 (13)
Physical
Pushing	28 (74)	4 (11)	3 (8)	3 (8)
Slapping	29 (76)	6 (16)	2 (5)	1 (3)
Punching	37 (97)	1 (3)	0 (0)	0 (0)
Kicking	33 (87)	4 (11)	1 (3)	0 (0)
Biting	36 (95)	2 (5)	0 (0)	0 (0)
Pulling hair	36 (95)	0 (0)	1 (3)	1 (3)
Physical assault	38 (100)	0 (0)	0 (0)	0 (0)
Threatening	37 (97)	0 (0)	1 (3)	0 (0)
Destructive
Tearing paper	30 (79)	1 (3)	4 (11)	3 (8)
Smashing furniture	38 (100)	0 (0)	0 (0)	0 (0)
Damaging doors	37 (97)	1 (3)	0 (0)	0 (0)
Serious damage	38 (100)	0 (0)	0 (0)	0 (0)
Self-harm
Bruising	38 (100)	0 (0)	0 (0)	0 (0)
Scarring	37 (97)	1 (3)	0 (0)	0 (0)
Skin picking	31 (82)	3 (8)	3 (8)	1 (3)
Scratching	33 (87)	2 (5)	2 (5)	1 (3)
Hair pulling	36 (95)	1 (3)	0 (0)	1 (3)
Face slapping	37 (97)	0 (0)	1 (3)	0 (0)
Biting hands	21 (64)	0 (0)	3 (9)	9 (27)
Biting lips	37 (97)	0 (0)	0 (0)	1 (3)
Poking	38 (100)	0 (0)	0 (0)	0 (0)
Head banging	38 (100)	0 (0)	0 (0)	0 (0)
Cutting	38 (100)	0 (0)	0 (0)	0 (0)
Sexual
Touching	33 (87)	4 (11)	0 (0)	1 (3)
Unwelcome kissing	35 (92)	2 (5)	1 (3)	0 (0)
Obscene communication	38 (100)	0 (0)	0 (0)	0 (0)
Any exposure	38 (100)	0 (0)	0 (0)	0 (0)
Public masturbation	38 (100)	0 (0)	0 (0)	0 (0)
Sexual assault	34 (89)	4 (11)	0 (0)	0 (0)
Oppositional
Defying rules	20 (53)	5 (13)	7 (18)	6 (16)
Refusing engage	25 (66)	3 (8)	6 (16)	4 (11)
Demanding
Repeated requests	26 (68)	0 (0)	4 (11)	8 (21)
Impatient	28 (74)	1 (2)	2 (8)	3 (14)
Wandering
Wandering	32 (84)	2 (5)	4 (11)	0 (0)
Darting	36 (95)	0 (0)	1 (1)	1 (1)
Running away	38 (100)	0 (0)	0 (0)	0 (0)

Table 2

Results of first factor analysis of 38 patients in the Jamaican study

	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7	Factor 8	Factor 9	Factor 10
% variation explained	32	14	11	8	7	6	5	<5	<5	<5
Verbal behaviour							0.95
Physical
Pushing	0.85
Slapping	0.87
Kicking	0.81
Destructive, tearing paper								0.88
Self-harm
Skin picking					0.96
Scratching			0.59
Biting hands				0.90
Sexual
Touching		0.69
Unwelcome kissing	0.75
Sexual assault		0.88
Oppositional
Defying rules			0.54
Refusing engage			0.90
Demanding
Repeated requests									0.93
Impatient						0.92
Wandering										0.87

Comparison with scores on the MOAS

In the randomised trial, scores for aggressive challenging behaviour were assessed using the MOAS at the same assessment using the PBCL. The MOAS is a well-established and reliable instrument for assessing aggressive behaviour in this population[13] and a common threshold for aggression is a MOAS score of 4 or greater.[8] The PBCL was strongly associated with this outcome, giving an area under the receiver operating characteristic curve of 0.95 (Fig. 1). A PBCL score of 1 was found to give the best prediction of this outcome, which yielded high levels of sensitivity (97%) and specificity (85%).

Fig. 1

Relationship between the scores of 2300 assessments for a threshold of 4 on the Modified Overt Aggression Scale and 1 on the Problem Behaviour Checklist. ROC, receiver operating characteristic.

Reliability

In the randomised trial, two raters (A.T. and R.E.) assessed data from 38 participants in 7 care homes over an extended period up to 1 year, providing a total of 407 monthly repeat assessments. In this study, a large proportion of scores on the PBCL (62%) was zero and to avoid spurious agreement the scores were divided into five categorical groups (0, 1–3, 4–6, 7–11 and ≥12). Using weighted kappa, the level of agreement was 0.91, with the 95% confidence interval ranging from 0.83 to 0.99. This high value indicates very good agreement between the two observers.[14]

Discussion

The results suggest that the PBCL is a useful measure of challenging behaviour in people with intellectual disability. It has the advantages of simplicity, shortness and repeatability, and may be of particular use in longitudinal studies. It also appears to be a comprehensive measure even though its main use in these studies has been to assess aggressive challenging behaviour and so many of the factors have correlates with aggression. The high agreement between the MOAS and PBCL scores also adds construct validity to the scale as the MOAS is a frequently used measure in the assessment of challenging behaviour.[15,16]

Although the current work has been confined to people with intellectual disability it might well be extended to other populations with challenging behaviour (e.g. dementia, head injury), where direct questioning of participants may yield limited information. Its weaknesses are the relative absence of personal input by people with intellectual disability in scoring the scale. Although the high correlation between the PBCL and MOAS scales suggests that both scales are equivalent in recording challenging behaviour, the PBCL covers a broader range of items than the MOAS and so is more comprehensive. It needs further testing before the preferred populations for assessment can be chosen.

Table 3

Results of second factor analysis of 2300 observations in 200 care home residents and 30 variables of challenging behaviour[a]

Component	Eigenvalue	% total variation
1 Threatening, oppositional, demanding and aggressive behaviour	6.4	21.4
2 Aggressive sexual behaviour	1.9	6.3
3 Self-harming behaviour	1.6	5.4
4 Hair pulling, scratching and head-banging	1.5	4.9

Only factor loadings >0.5 are reported.

	Behaviour absent	Minor and often frequent behaviour but little disruption to others	Moderate problem behaviour creating distress and disruption	Serious problem behaviour leading to major concerns and risk to others	Extreme behaviour leading to threat of loss of life or permanent injury and damage
	0	1	2	3	4
Personal violence Score (0–4):	No verbal abuse and no form of violent behaviour	Verbal abuse	Threatened violence or minor assault with no lasting injury or breaking of skin (e.g. slapping, pushing)	Physical assault with likelihood of, or consequent, injury with temporary handicap or psychological damage (e.g. bruising, fear avoidance)	Physical assault with permanent or life-threatening injury (e.g. poking through eyes, stabbing, loss of consciousness)
Property violence Score (0–4):	No damage	Minor damage with no serious consequences (e.g. tearing paper)	Moderate damage with need for minor repairs (e.g. breaking front window)	Serious damage requiring major property repairs or creating some risk to others	Very serious damage with threat to life or limb (e.g. arson, floor collapse)
Self-harm Score (0–4):	No self-harm	Minor harm with no breaking of skin (e.g. minor head banging)	Moderate self-harm with breaking of skin, scarring or small overdose but no long term	Serious self-harm with potential of risk of death (e.g. swallowing bleach, poking own eyes)	Suicidal act or violent self-harm leading to death or permanent handicap
Sexually inappropriate behaviour Score (0–4):	No inappropriate behaviour	Obscene gestures or sexually abusive comments	Touching, fondling and kissing (non-violent but bodily contact)	More serious sexual assault with bodily contact or indecent exposure	Violent sexual assault including rape and coercive sexual contact
Contrary behaviour Score (0–4):	No contrary behaviour	Verbal negativity and initial refusal to obey instructions	Oppositional behaviour, single or recurrent, creating problems for others but not serious disruption	Severe contrary behaviour leading to potential danger to health and welfare (e.g. refusal to take prescribed medicine when essential; deliberate flooding of bathroom)	Dangerous oppositional behaviour causing problems for health and welfare (e.g. refusal to leave burning building, running into path of car when asked to walk on pavement)
Demanding behaviour Score (0–4):	No demanding behaviour	Frequent need for attention but little disruption	Threatening and disturbing demanding behaviour that disrupts	Violent demanding behaviour that distresses others, not only at the time, but subsequently	Violent demands on others that are a serious threat to psychological health and function (e.g. stalking)
Disappearing behaviour Score (0–4):	Does not disappear and never goes away without warning	Absent minded, gets lost easily, or tends to drift away from group and has to be recalled	Needs constant supervision to avoid getting lost or running off	Darting and other deliberate movements that may put person in danger (e.g. runs across main road)	Complete disappearance over long distance with need to search for person with help of other agencies (e.g. police)
Total score Score (0–28):