Autism Spectrum Social Stories In Schools Trial (ASSSIST): study protocol for a feasibility randomised controlled trial analysing clinical and cost-effectiveness of Social Stories in mainstream schools.

INTRODUCTION: Current evidence suggests that Social Stories can be effective in tackling problem behaviours exhibited by children with autism spectrum disorder. Exploring the meaning of behaviour from a child's perspective allows stories to provide social information that is tailored to their needs. Case reports in children with autism have suggested that these stories can lead to a number of benefits including improvements in social interactions and choice making in educational settings. METHODS AND ANALYSIS: The feasibility of clinical and cost-effectiveness of a Social Stories toolkit will be assessed using a randomised control framework. Participants (n=50) will be randomised to either the Social Stories intervention or a comparator group where they will be read standard stories for an equivalent amount of time. Statistics will be calculated for recruitment rates, follow-up rates and attrition. Economic analysis will determine appropriate measures of generic health and resource use categories for cost-effectiveness analysis. Qualitative analysis will ascertain information on perceptions about the feasibility and acceptability of the intervention. ETHICS AND DISSEMINATION: National Health Service Ethics Approval (NHS; ref 11/YH/0340) for the trial protocol has been obtained along with NHS Research and Development permission from Leeds and York Partnership NHS Foundation Trust. All adverse events will be closely monitored, documented and reported to the study Data Monitoring Ethics Committee. At least one article in a peer reviewed journal will be published and research findings presented at relevant conferences. TRIAL REGISTRATION NUMBER: ISRCTN96286707. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Schools in the study will be cluster randomised to avoid treatment contamination.

The study will address an under-researched area and will produce important research evidence to inform the education of young people in the UK National Health Service (NHS).

An economic analysis will be used to make the trial comparative to large numbers of other studies.

The sample of participants will be obtained from only one NHS Trust resulting in potential for minorities to be under-represented.

Blinding of participants will not be feasible due to the nature of the intervention.

Introduction

Children with autistic spectrum disorder (ASD) are less able to intuitively understand how they are expected to behave or to learn social rules by observing others as their typically developing peers might do. Consequently, their behaviour can be interpreted as disruptive.1 Parents and teachers report more behaviour problems in children with ASD than in typically developing children.2 Many children with ASD therefore need specific training to learn these skills.

Social Stories are short stories which describe a social situation or skill to help children with ASD. They are commonly used to enable children to understand socially expected behaviours. Gray3, the original designer, identified 10 criteria which guide story development. These criteria ensure that the story structure and content is descriptive, meaningful and safe for its audience.

Social Stories can be effective in tackling problem behaviours when they set out to explicitly teach social skills.4 Until recently, research exploring efficacy and outcome has been confined to single case studies. These case reports in children with ASD have suggested improvements in social interactions,5 6 choice making in an educational setting,6 voice volume in class7 and mealtime skills.8 Success has also been reported in addressing disruptive behaviours, including reducing tantrums9 10 and behaviours associated with frustration.11

Non-comparative research has examined Social Story use in special education12 and mainstream education settings,13 as well as their application within the home.14 This research suggests that it is possible to train tier one professionals, for example teachers, in the use of Social Stories and for these stories to benefit children with ASD.15

There is little consensus on the effects of Social Stories, highlighted in two systematic reviews of their effectiveness.4 16 The reviews identified largely single case designs, displayed a paucity of good quality, comparative evidence on Social Stories and showed the necessity of further research regarding whether positive outcome (as seen in case series) could be replicated within a randomised controlled trial (RCT).

Evidence suggests that Social Stories delivered in a general education setting produced significantly larger effect sizes on behaviour than those implemented at home or in self-contained settings, such as separate schools or self-contained classrooms4; therefore, this study specifically focuses on the feasibility of assessing Social Stories delivered within a mainstream schooling system.

Although Social Stories have much potential to improve behaviour, the comparative evidence base is limited and attributed levels of effect are based mostly on single case research. A well-designed RCT is required to address this important gap in our knowledge, and a feasibility RCT is required to plan the appropriate experimental design. In particular, the intervention has largely been used in special education and its use in mainstream education, where it has large potential, needs exploring.

Aims and objectives

At present, there is insufficient information to indicate how to conduct a full-scale RCT. Therefore, we plan to design and conduct a feasibility study which will inform the design of a full RCT of clinical and cost-effectiveness. Our feasibility study has the following aims:

To conduct a feasibility RCT comparing a manualised Social Stories intervention with an attention control (demonstrating recruitment, delivery of the intervention and successful follow-up).

To establish the acceptability and utility of the manualised Social Stories intervention to teachers, parents and children.

To identify parameters, outcomes and cost-effectiveness from the feasibility RCT in order to inform a future full-scale RCT.

Methods

Study design

We aim to examine the feasibility of a cluster RCT to be conducted to compare the clinical and cost-effectiveness of Social Stories with an attention control. Qualitative interviews will be conducted within the feasibility design to explore the acceptability of the intervention and its delivery. The study will be conducted between November 2011 and October 2014.

Inclusion criteria

Child participants will be recruited between the ages of 5 and 15 years, will be required to have a previous diagnosis of ASD (including autism, Asperger syndrome and atypical autism) using International Classification of Diseases 10th revision, research diagnostic criteria,17 supported by Autism Diagnostic Interview-Revised (ADI-R),18 and/or autism diagnostic observation schedule (ADOS).19 Participants will be required to have behavioural or social problems in school (as reported by parents and teachers). We have deliberately kept the age range broad in order to explore the acceptability of this intervention across age ranges and different settings (eg, primary and secondary schools) to inform a fully powered trial.

This is a pragmatic trial which recognises the frequency and complexity of comorbidities in children with ASD. We will therefore not exclude participants on the basis of comorbidity since the intervention would eventually be offered to this group if adopted more widely in the National Health Service (NHS). This will ensure that the research remains relevant to everyday clinical practice.

Exclusion criteria

Children will not be included in the study if they have used a Social Story within the past 6 months. We will exclude children if they are likely to be moving school during the trial period, or if they develop an illness or behaviour that warrants admission to a psychiatric unit (eg, psychosis and serious self-harm). These exclusion criteria were informed by discussions with parents who had previously used Social Stories in a focus group prior to submitting the funding application.

Recruitment procedure

The research team will give presentations about the research to local autism support groups,20 and provide presentations and leaflets through clinicians to parents of children with ASD attending mainstream schools. Letters will also be sent to potentially eligible schools inviting them to participate. Potential participants within these schools will either be sent information leaflets by post or schools will introduce the research. The study will also recruit through the York Autism Spectrum Disorders Forum, a multiagency, multidisciplinary forum for diagnosing and discussing provision of supporting interventions for all local children on the autism spectrum. Letters will also be sent from this Forum to parents whose children are enrolled in a potentially eligible school.

Interested families will be invited to meet a researcher for further information if their child has a diagnosis of ASD and has behavioural problems in a mainstream school. Schools will be approached to take part in the study through an existing network of clinical and educational practitioners and researchers. There are 131 primary and secondary schools in the York Local Authority. Two of these are secondary schools designated as ASD supportive schools. We anticipate that we will need to recruit from 40 schools, although we will test the feasibility of this and be adaptable to recruit fewer or more by staggering school involvement and leaving enough time in the study for flexibility.

Randomisation and allocation concealment

We will adopt a cluster randomisation approach. This is to minimise the likelihood of participants in the intervention and comparator groups being affected by changes in teacher or school behaviour. It also recognises that Social Story expertise within a school is likely to be enhanced by the sharing of skills and networking of ideas. This design therefore more accurately reflects what would happen if these stories were used within a school in practice. We will stratify school randomisation using minimisation to take into account numbers of children with ASD, levels of support, socioeconomic indices and value-added measures. To limit detection bias, the schools will be randomised only once consent has been gained for all participating families in each school.

Allocation to groups will be conducted by the York Trials Unit through the use of a custom-made computer program which will account for the stratification variables. The program will not contain any personal details but will allocate participants by anonymous participant numbers.

Sample size

As this is a feasibility study, a formal sample size calculation was not undertaken. We have considered that collecting outcome data on approximately 50 children (25 in each arm) would be sufficient to address our objectives. As this is a cluster randomised trial, we estimate that 1–2 children will be recruited from each school, hence aim to recruit at least 30 schools.

Treatments

Schools will be randomised to one of two arms; the intervention group and the comparator group. In both groups, children will receive all other treatment or support as usual and we will monitor this.

Intervention

The children in the intervention group will receive the Social Story intervention. Teachers and parents of children in the intervention group will attend a Social Stories training session which will involve providing information on the design and implementation of Social Stories. During the training session the teachers and parents will construct a Social Story with input from the research team. Particular attention will be paid to the construction of each Social Story to ensure they adhere to the guidelines. Teachers and parents will be instructed to read the stories to the child approximately three times a week for 2 weeks. Teachers and parents will receive support as part of the study from the research team.

Comparator

The children in the comparator group will receive an equivalent amount of time reading a story of similar length chosen by teachers to be appropriate to the child's interests and abilities. Teachers in the comparator group will be asked to choose a story for the child and read it for an equivalent length of time to a typical Social Story (approximately 5 min). Participants will be instructed to read the stories to the child approximately three times a week for 2 weeks. At the end of the study, a free workshop on the design and implementation of Social Stories will be offered to the parents and teachers of children participating in the comparator group.

Blinding

Blinding of the participants (inclusive of teachers, parents and children) will not be feasible due to the nature of the intervention, nor will blinding of all members of the study team who are actively involved in the administration of the study. However, members of the study team responsible for the statistical and economic analysis will be kept blind to group allocation.

Outcome measurement

Feasibility outcome measurement

The feasibility of participant recruitment will be determined by examining the number of children assessed for eligibility; the number eligible; reasons for ineligibility; reasons for non-participation and the number randomised. Additionally, comparisons will be made between recruitment techniques.

The feasibility and acceptability of data collection processes will be investigated through the number of missing items and follow-up rates relating to the clinical outcome measurements likely to be used in a phase III trial. Additionally, we will examine levels of attrition through treatment and follow-up rates. The acceptability of the treatment will be examined through reasons reported for withdrawal from treatment, qualitative interviews and follow-up questionnaires.

Clinical outcome measurement

Teachers, parents and children will each complete a questionnaire at baseline, 6 and 16 weeks postintervention delivery start date (actual date if allocated to intervention and hypothetical date if allocated to comparator group, respectively).

As part of baseline data collection, parents and teachers (and children where appropriate) will be guided by clinicians to identify an appropriate individualised goal. Measurements related to this goal will be recorded through the questionnaires at baseline and follow-up points.

Parents and teachers will also be asked to complete diaries at baseline and all follow-up points to record details such as the frequency of challenging behaviours, as well as details about the intervention use such as setting and times delivered.

One of the purposes of this feasibility RCT will be to elect an appropriate primary outcome measure for a full trial. We will examine the potential of the following measurements for this purpose:

The Strengths and Difficulties Questionnaire (SDQ),21 a brief behavioural screening questionnaire. It will be completed by the parents, teachers and older children (11–15 years).

The Social Responsiveness Scale-2 (SRS-2),22 identifies social impairment associated with ASD and quantifies its severity. It will be completed by the teachers.

A diary, designed by the research team to collect data on the frequency of use of the intervention and the frequency of the behaviours across the school day. It will be completed by the parents and teachers.

A goal-based outcome measure designed by the research team, enables individualised goals and is measured on an 11-point Likert scale. It will be completed by the parents, teachers and children.

The secondary outcomes of the study will be:

Bespoke resource use questionnaires, developed by the health economist to capture the resource implications of child's behavioural problems at school and home. These will be completed by the parents and teachers.

The EQ-5D and the EQ-5DY,23 standardised instruments for use as measures of generic health outcomes recommended by National Institute of Health and Care Excellence (NICE). They will be completed by the parents and children, respectively.

Health Utilities Index 2 (HUI2),24 alternative preference-based generic health outcome measure to establish health states in children, report their health-related quality of life and produce utility scores. It will be completed by the parents.

The Parental Stress Index, Fourth Edition Short Form (PSI-4),25 designed to evaluate the magnitude of stress in the parent–child system. It will be completed by the parents.

Spence Childhood Anxiety Scale (SCAS),26 a 44-item questionnaire developed to assess the severity of anxiety symptoms broadly in line with the dimensions of anxiety disorder. It will be completed by the children.

Statistical methods

Feasibility and clinical outcomes analysis

In line with recommendations about good practice in the analysis of feasibility studies,27 analysis will be descriptive and no comparisons of the outcomes between the two arms of the trial will be conducted. Descriptive statistics will be calculated for recruitment rates, follow-up rates, and attrition and baseline characteristics. These will be presented as means and SDs or 95% CIs, medians and IQR, or percentages. Descriptive statistics and 95% CIs will also be calculated for the outcome measures. We will examine the effects of diagnosis and age of the participants when carrying out the analysis and use the data to develop estimates for a fully powered RCT that will include estimates of change in outcome measures over time taking into account attrition and follow-up rates. All analyses will be undertaken on Stata.

Economics analysis

Economic analysis that will be conducted alongside this feasibility RCT aims to inform the choice of appropriate measures of generic health. Additionally, it will enable us to identify the relevant resource use categories for the cost-effectiveness analysis, and evaluate the feasibility and challenges of measuring costs and outcomes in the target population.

To assess whether economic analysis conducted alongside a fully powered RCT would be feasible, a within-trial cost-effectiveness analysis will be conducted. This analysis will aim to provide provisional estimates of the incremental cost per unit of effectiveness of the Social Stories toolkit compared with an attention control in children with ASD.

This economic analysis will first indicate the expected implications of implementing various generic health questionnaires on the research outcomes. EQ-5DY and HUI2 will primarily be evaluated to determine which measures of generic health are most appropriate within the study context and cohort.

The perspective will evaluate appropriate cost consequence that should be considered in a full evaluation. Primarily the perspective of the NHS will be adopted in line with explicit guidelines. Furthermore, as the implications of behavioural problems extend beyond healthcare alone, costs falling outside of the healthcare system will also be evaluated. The cost of the Social Stories intervention will be calculated by estimating the time spent by individuals delivering the intervention, the cost of training and other resources used.

Qualitative analysis

Qualitative semistructured interviews will be conducted using a topic guide developed to ascertain information on perceptions about the feasibility and acceptability of the intervention. In addition, we will gather a range of opinions and themes about the helpful and unhelpful parts of the Social Story intervention, the characteristics of those who would be best placed to deliver the intervention, which professional groups are best equipped to deliver and/or support this package; and the mode of delivery including the role of parents or carers and the style of the manual. Where appropriate this may include information on construction of the Social Stories, the teaching of the Social Story theory and dissemination.

Trial status

Recruitment of participants is completed. The first participant was enrolled in February 2013. The last participant will complete follow-up in July 2014.

Ethics and dissemination

Ethics

We do not anticipate that trial participants will be subject to any risks during this study. Social Stories are widely used and focus on positive social and situational coping, and are therefore unlikely to cause any harm to participants. A possible ethical concern with the study is that the control group will not receive the Social Story intervention during the trial. We have minimised this by holding a free workshop, at the end of the study, about the use of Social Stories for those parents and teachers within the control group.

It is possible that parents may become distressed when talking about their child's ASD. The clinicians involved in the Social Stories training are very experienced in dealing with parents experiencing distress and are well placed to either provide support or refer on to other support as necessary. Similarly, should a child become distressed, the teachers, clinicians and researchers will have agreed prior to the study initiation, any necessary actions, distractions, activities or support that will be employed in this event.

All participant information will be stored in accordance with the Data Protection Act 1998. Participants’ personal identifiable information will be stored in a locked filing cabinet and all participant data will be anonymised by allocating each participant with an ID number. Anonymised participant data will be saved on a password-protected secure computer drive which only members of the research team will have access to. Participants’ personal identifiable data will be stored in a separate location to anonymise participant data. All data will be maintained by the research coordinator.

Informing potential trial participants

Information sheets will be provided to teachers and parents. Separate information sheets for children aged 5–10 and 11–15 years will be used. Interested parents will choose either to contact a researcher by telephone or let their school know they would like to find out more. Therefore, potential participants will always have a minimum of 24 h to decide whether or not to take part. We will contact the participants by letter or telephone if any relevant information related to ASD or Social Stories becomes available during the study. All information leaflets and consent forms will be codeveloped by the research team and the Patient and Public Involvement group to ensure acceptability among participants.

Obtaining informed consent

If, after reading the information sheets, children and parents are still interested in taking part, an appointment will be made with a researcher. At this appointment there will be the opportunity to ask any questions relating to the research and to consent to the study if they are happy to do so. Written consent will be obtained from parents and consent or assent from children and young people will be obtained where possible.

Dissemination plan

We will publish the results of our study in high-profile mainstream and specialist science journals and publications with high readership among clinical staff. Presentations of study findings will be taken to relevant research conferences, local research symposiums and seminars for Child and Adolescent Mental Health Service (CAMHS) professionals. In addition, the National Autistic Society and members of service user groups such as Autism Spectrum Conditions—Enhancing Nurture and Development (ASCEND) will be consulted in the development of methods for dissemination which will be effective in reaching families of children with ASD. To specifically address empirical issues uncovered in the economic analysis, publication will be sought in a peer-reviewed journal. Additionally, we will produce a short summary of results that can be distributed to all trial participants as well as relevant patient and other interest groups. Finally, we will aim to ensure coverage of our findings in the wider media by issuing a press release.