Parent skills training for parents of children or adults with developmental disorders: systematic review and meta-analysis protocol.

INTRODUCTION: Developmental disorders, including intellectual disability and autism spectrum disorders, may limit an individual's capacity to conduct daily activities. The emotional and economic burden on families caring for an individual with a developmental disorder is substantial, and quality of life may be limited by a lack of services. Therefore, finding effective treatments to help this population should be a priority. Recent work has shown parent skills training interventions improve developmental, behavioural and family outcomes. The purpose of this review protocol is to extend previous findings by systematically analysing randomised controlled trials of parent skills training programmes for parents of children with developmental disorders including intellectual disabilities and autism spectrum disorders and use meta-analytic techniques to identify programme components reliably associated with successful outcomes of parent skills training programmes. METHODS AND ANALYSIS: We will include all studies conducted using randomised control trials designs that compare a group of parents receiving a parent skills training programme to a group of parents in a no-treatment control, waitlist control or treatment as usual comparison group. To locate studies, we will conduct an extensive electronic database search and then use snowball methods, with no limits to publication year or language. We will present a narrative synthesis including visual displays of study effects on child and parental outcomes and conduct a quantitative synthesis of the effects of parent skills training programmes using meta-analytic techniques. ETHICS AND DISSEMINATION: No ethical issues are foreseen and ethical approval is not required given this is a protocol for a systematic review. The findings of this study will be disseminated through peer-reviewed publications and international conference presentations. Updates of the review will be conducted, as necessary, to inform and guide practice. TRIAL REGISTRATION NUMBER: PROSPERO (CRD42014006993). Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Developmental disorder is an umbrella term that includes conditions such as disorders of intellectual development (referred to as intellectual disability in Diagnostic and Statistical Manual of Mental Disorders (DSM)-5, mental retardation in International Classification of Diseases (ICD)-10, DSM-IV) and autism spectrum disorders (e.g., autistic disorder, Asperger's disorder, atypical autism).1–3 Developmental disorders typically have onset in childhood, affect multiple functional domains and often persist into and through adulthood. Intellectual disability is defined as “a condition of arrested or incomplete development of the mind, which is especially characterized by impairment of skills manifested during the developmental period, which contribute to the overall level of intelligence, i.e., cognitive, language, motor, and social abilities.”4 Individuals with autism spectrum disorders can have a variable level of intellectual and adaptive behaviour skills, but share a characterisation of impaired capacity for reciprocal socio-communicative interaction and the presence of restricted, stereotyped repetitive repertoire of interests and activities.2 4

Developmental disorders, including intellectual disability and autism spectrum disorders, affect individuals worldwide, and account for more than 0.4% of all disability-adjusted life years.5 Although effective comprehensive treatment programmes have been identified,6–8 provision of these interventions requires significant resource output.9 10 Given that a majority of individuals with neurodevelopmental disorders reside in low-resource settings, the scale up of such resource-intensive programmes is challenging. In fact, it has been estimated that up to 85% of individuals with developmental disorders living in low-income and middle-income countries do not receive adequate interventions.11 Therefore, finding effective treatments to help the individuals with developmental disorders and their families should be a high priority. Previous systematic reviews on psychosocial interventions for intellectual disabilities and autism and other pervasive developmental disorders conducted in low-income and middle-income countries identified few relevant papers and many of the studies had significant methodological shortcomings.12–14 A recent systematic review of psychosocial interventions for children with developmental disorders where the intervention or parent training was delivered by non-specialist providers15 showed parent skills training to be an effective intervention for improving developmental, behavioural and familial outcomes. However, because the Reichow et al review was focused on non-specialist delivered interventions, it did not include all studies on parent skills training programmes; thus, broader knowledge of the effectiveness of parent skills training, including programmes delivered by specialists, is unknown. Therefore, we sought to extend the findings of Reichow et al by examining all parent skills training programmes for parents who have a child with a developmental disorder.

Objectives

The primary objective of our review is to systematically review and meta-analyse evidence to determine if parent skills training programmes for parents who have a child with a developmental disorder produce greater benefits than no treatment or standard care on child functioning and on parental or family functioning, as measured across multiple domains, and to use meta-analytic techniques to determine which programme components are most reliably associated with the most successful outcomes of parent skills training programmes. The secondary objective of our review is to examine the results of studies conducted in low-income and middle-income countries to determine if systematic differences exist on outcomes between low-income and middle-income countries and high-income countries.

Methods

Study selection

Types of studies

We will limit our inclusion to studies conducted using randomised control trial designs. We have chosen to limit the inclusion to studies using randomised control trial designs to decrease the likelihood of including studies with high risk of bias, which are more likely in studies conducted using quasi-experimental designs. We will only include studies that have at least 10 participants per pairwise comparison in order to have our inclusion criteria consistent with unit of analysis rules described below.

Types of participants

We will include studies that recruited parents who have a child, irrespective of the child's age, who has one of the following developmental disabilities: disorders of intellectual development (intellectual disability, mental retardation), developmental delay, Down syndrome and autism spectrum disorders. We will consider additional developmental disorders or groups of mixed disabilities if the average IQ for the child participants is lower than two SDs below the mean (i.e., IQ <70).

Types of interventions

We will consider all published and unpublished studies, irrespective of language, comparing a group of parents receiving parent skills training programme to a group of parents in a no-treatment control group, including waitlist control, or a treatment as usual comparison group.

Types of settings

We will include studies in which the parent skills training programmes were delivered in community clinics, homes, university clinics and schools; no study will be excluded based on where the programme was delivered.

Types of outcome measures

We will examine the effects of parent skills training programmes on child outcomes and parent outcomes. The primary outcome measure for children will be adaptive behaviour (e.g., functional skills, daily skills). We will also include two secondary child outcomes: child development and problem behaviour. The primary outcome measure for parents will be quality of life. We will also include three parental secondary outcomes: psychological health, parent skills and family quality. Finally, we will examine consumer (parental) satisfaction and attrition.

Search methods for identification of studies

Electronic searches

We will search African Index Medicus, AFRO Library, the Cochrane Central Register of Controlled Trials, Cumulative Index to Nursing and Allied Health, Dissertation Abstracts International, EMBASE, Education Resources Information Center, Western Pacific Region Index Medicus, Literatura Latinoamericana y del Caribe en Ciencias de la Salud, MEDLINE and PsycINFO for relevant studies. We will not use any filters, such as language, publication in peer-reviewed sources or randomisation, to prevent missing any relevant studies. The electronic search strategy is shown in online supplementary appendix A.

Searching other resources

We will use ‘snowballing method’16 by examining citations of included studies, searching for studies citing included studies, and examination of reference lists from key reviews to identify additional studies not identified in the electronic search. We will also contact experts, including experts residing or conducting research in low-income and middle-income countries, to ask if they know of additional studies, including unpublished data.

Identification and selection of studies

Two authors will initially screen all titles and abstracts in order to exclude clearly irrelevant articles. These authors will then independently screen the remaining titles and abstracts in order to identify articles that appear to meet inclusion criteria. Two authors will then confirm which studies meet all inclusion criteria by independently reviewing the full text of potentially relevant articles. Data from multiple reports of the same study will be linked together and used to supplement information obtained from the primary report.

Data extraction

Two researchers will independently perform data extraction, with discrepancies resolved through consensus. If consensus cannot be reached, a third researcher will make a final judgement on the data. If critical information is missing from a report, we will contact the authors to inquire of the data. Data will be extracted on the following variables:

Study characteristics: including, but not limited to: publication year; study location, including determination if study was conducted in a low-income and middle-income country according to World Bank method.

Population characteristics: including, but not limited to: sample size; diagnostic characteristics; child age; parental age; child skill level; gender; parent education level; marital status.

Intervention characteristics: including, but not limited to: content of parent skills training, adopted from Kaminski et al17; intervention delivery methods; training format and location; trainer qualifications; trainer training and supervision; intervention density, including number of group sessions, number of individual sessions, number of home sessions, duration of each session and duration, in weeks, of complete programme.

Study results and effects: including, but not limited to: outcome measures used; number of completers; effect size estimates, including an assessment of statistical significance.

Assessment of risk of bias

Two authors will independently assess the study level risk of bias using the methodology and risk of bias tool detailed by the Cochrane Collaboration.18 We will resolve discrepancies through consensus, and if consensus cannot be reached, a third researcher will make a final judgement on the risk of bias. We will use the tool to assess the following domains: sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting and other potential sources of bias.

Data synthesis and statistical analysis

We will present a narrative synthesis including visual display of study effects and, if appropriate, conduct a quantitative synthesis of the effects of parent skills training programmes using meta-analytic techniques.

Extraction of individual study estimates

For studies reporting continuous data, we will extract individual study estimates using the standardised mean difference with small sample correction; i.e., Hedge's g.19 We will use the standardised mean difference effect size instead of the weighted mean difference because we expect to locate studies in which multiple measures with different scales being used to assess outcomes. We will calculate effect size estimates for each study outcome from the post-treatment scores and SDs provided in each study report. If means and/or SDs are not provided in the report, we will contact authors to request the data. If authors do not provide data, we will estimate effect sizes if possible from p values, t-values results, or F-values if they are provided using the effect size calculator of the Campbell Collaboration, which uses the formulae provided in the text of Lipsey and Wilson.20 We will extract up to one point estimate per outcome for each study. If a study reports multiple measures for one outcome, we will select one measure on which to calculate the effect size by selecting a measure using the following hierarchy: (1) measure reporting a total domain score using a validated rating scale, (2) measure that is most frequently used in research and (3) measure that is most closely related to other frequently used measures of the specific construct. We will address unit of analyses issues according to the methods consistent with the recommendations of the Cochrane Collaboration.21 If trials reporting multiple pairwise comparisons between distinct intervention group and control group are found, we will analyse and report the results of each comparison separately. If trials with multiple intervention groups receiving identical treatments are found, the intervention groups will be combined to create a single pairwise comparison. If trials with multiple intervention groups receiving different variations of parent skills training are found, we will analyse each intervention group separately by dividing the sample size for the common comparator groups proportionately across each comparison. When dividing samples, we will only divide a sample if the resulting group sample size is greater than 10 participants per group; if a division would create a sample size less than 10, the study will be considered not to meet our inclusion criteria and will be excluded from our review. If trials comparing a single intervention group to multiple control groups are found, we will combine the control groups and conduct a single pairwise comparison. Because the control groups will not receive parent skills training, we expect treatment acceptability to be only reported for the intervention group. For this outcome, we will extract descriptive statistics summarising either the percentage of participants who found the treatment to be acceptable, or the average level of satisfaction with the treatment programme if ordinal data are reported. For dichotomous, or event-like, data, relative risks will be calculated with a 95% CI. For statistically significant results, we will calculate the number needed to treat to provide benefit (NNTb) or harm (NNTh).

Calculation of pooled estimates

If appropriate, we will perform separate meta-analyses for each primary and secondary outcome by combining results of studies using a random effects meta-analysis with a 99% CI. We will use a random effects model because we expect to find heterogeneity between types of parent skills training programmes and because a random effects model has highest generalisability in our empirical examination of summary effect measures for meta-analyses.22 We are choosing to use a 99% CI to limit the risk of multiplicity23 arising from the high number of planned subgroup analyses. In our analysis of the standardised mean difference effect sizes, we will consider an effect to be clinically relevant, irrespective of statistical significance, by transforming the standardised mean difference effect sizes into common language effect sizes.24 25 An example of transforming standardised mean difference effect size into common language effect size is shown in a meta-analysis of social skills group interventions for children with autism spectrum disorders by Reichow et al,26 who showed the weighted mean effect size of d=0.47 equated to a gain of 24 additional social skills for the treatment group compared with control. Possible transformations that might be possible from data anticipated to be analysed in this review would also include extrapolation of the number of fewer behaviour problems per week for children whose parents receive a parent skills training programme or the likelihood of a parent moving below a clinical threshold for depression.

Assessment of heterogeneity and sensitivity analysis

If a meta-analysis is conducted, we will conduct two statistical estimates of heterogeneity. First, we will examine heterogeneity using the Q-statistic,19 which provides a test of statistical significance indicating whether the differences in effect sizes are due to participant-level sampling error alone or other sources. Because recent criticism has been raised about the validity of the Q-statistic as a test of homogeneity in meta-analyses,27 we will also estimate heterogeneity using I2, which estimates the proportion of between-studies variance. Given the limitations of the Q-statistic, we will emphasise the I2 values in our analyses and reporting of results, as suggested by the Cochrane Collaboration.28 In order to explore the impact of studies with high risk of bias on the robustness of the results of the review, we will conduct sensitivity analyses by removing studies with a high risk of bias on sequence generation and blinding of outcome assessment and reanalysing the remaining studies to determine whether these factors effect the results. We will also conduct sensitivity analyses to examine the robustness of the random effects summary measure by checking the results under a fixed effect model, and report where material differences between the models occur.

Assessment of risk publication bias

Publication bias occurs when there are unpublished studies with negative results (e.g., file-drawer problem), and is often a problem when conducting research syntheses. We will assess the possibility of publication bias by constructing a funnel plot and visually analysing the plot if an adequate number of studies (n>10) are included in the meta-analysis.29

Analysis of subgroups

We will conduct further investigation of the causes of heterogeneity using subgroup analyses. Possible subgroups that we may examine include: type of parent skills training programme; inclusion of specific parent skills training programme components; use of specific training methods; parent skills training leader characteristics and training; type of comparison groups (e.g., waitlist control, treatment as usual, no-treatment control) and if there are systematic differences between studies conducted in low-income and middle-income countries and high-income countries. We will compare subgroups separately for each outcome using analysis of variance where at least three studies are represented for each category. We will also examine continuous moderators using meta-regression techniques. Possible continuous moderators include intervention density (intensity) and duration; and pretreatment participant characteristics (e.g., chronological age, symptom severity, IQ, communicative ability and level of adaptive behaviour).