Comparative efficacy and acceptability of psychosocial treatments for disruptive behaviour disorders in children and adolescents: study protocol for a systematic review and network meta-analysis.

INTRODUCTION: Disruptive behaviour disorders are common among children and adolescents, with negative impacts on the youths, their families and society. Although multiple psychosocial treatments are effective in decreasing the symptoms of disruptive behaviour disorders, comprehensive evidence regarding the comparative efficacy and acceptability between these treatments is still lacking. Therefore, we propose a systematic review and network meta-analysis, integrating both direct and indirect comparisons to obtain a hierarchy of treatment efficacy and acceptability. METHODS AND ANALYSIS: The present protocol will be reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols. Ten databases, including Web of Science, PubMed, PsycINFO, MEDLINE, APA PsycArticles, Psychology and Behavioral Sciences Collection, OpenDissertations, The Cochrane Library, Embase and CINAHL, will be searched from inception for randomised controlled trials of psychosocial treatments for children and adolescents with disruptive behaviour disorders, without restrictions on language, publication year and status. The primary outcomes will be efficacy at post-treatment (severity of disruptive behaviour disorders at post-treatment) and acceptability (dropout rate for any reason) of psychosocial treatments. The secondary outcomes will involve efficacy at follow-up, severity of internalising problems and improvement of social functioning. Two authors will independently conduct the study selection and data extraction, assess the risk of bias using the revised Cochrane Collaboration's Risk of Bias tool and evaluate the quality of the evidence using the Grading of Recommendations Assessment, Development and Evaluation framework to network meta-analysis. We will perform Bayesian network meta-analyses with a random effects model. Subgroup and sensitivity analyses will be performed to evaluate the robustness of the findings. ETHICS AND DISSEMINATION: The research does not require ethical approval. Results are planned to be published in journals or presented at conferences. The network meta-analysis will provide information on a hierarchy of treatment efficacy and acceptability and help make a clinical treatment choice. PROSPERO REGISTRATION NUMBER: CRD42020197448. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Bayesian network meta-analysis can synthesise all direct and indirect evidence and allows the comparison of multiple treatments simultaneously within a single analysis.

We investigate psychosocial treatment types rather than delivery conditions for treating disruptive behaviour disorders, which contributes to complement the guidelines of National Institute for Health and Care Excellence.

Bayesian network meta-analysis will provide a hierarchy of effectiveness and acceptability of all psychosocial treatment types for disruptive behaviour disorders, which can help clinical practitioners make optimal and evidence-based decisions.

We will not exclude trials with participants suffering disruptive behaviour disorders comorbid with attention deficit and hyperactivity disorder, which may raise the risk of bias for outcomes.

The generalisability may be impacted by the differences between treatments within categories/nodes.

Background

Disruptive behaviour disorders (DBDs), which include conduct disorder (CD) and oppositional defiant disorder (ODD),1 are common mental and behavioural problems among children and adolescents (more common in boys than in girls), affecting more than 50 million children and adolescents worldwide according to the survey by Global Burden of Disease Study 2013 Collaborators.2 According to the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5), CD is characterised by repetitive and persistent patterns of antisocial, aggressive or rule-breaking behaviour. ODD is characterised by irritable mood, argumentative behaviour or vindictiveness.3 Patients with DBDs have a high rate of comorbidity with other mental health problems such as attention deficit hyperactivity disorder (ADHD).4 DBDs are costly and can persist over time and bring about many serious consequences to the individual, school, family and society, including poor physical health, poor school performance, social disadvantage, family conflicts and criminal behaviours.5 Besides, untreated CD can influence the functioning and quality of life of children and adolescents during their childhood and adolescence and may develop into antisocial personality in their adulthood.6 On the contrary, an effective treatment could increase adaptive behaviours, improve social functioning and further save 128 disability-adjusted life years (DALYs) per 100 000 boys and 90 DALY per 100 000 girls in the USA.7 Given the adverse influence of DBDs and the benefits of effective treatment, researchers and practitioners have devoted their efforts to providing professional treatments effectively and timely to children and adolescents with DBDs.

Several disciplines, such as developmental psychopathology, child psychiatry and social psychology, have contributed to understanding the course, causes and development of DBDs and have yielded different perspectives for treating DBDs. Among the evidence-based interventions, psychosocial treatment is an important approach for treating DBDs, taking priority over pharmacological treatment. Although some medications (eg, risperidone) may have beneficial effects for DBDs, they have side effects and are not suggested for routine management.4 Evidence from previous meta-analyses indicates that psychosocial treatment can significantly improve DBDs1 8 9; hence, this study will focus on psychosocial treatment for youth with DBDs. To date, there are many types of psychosocial treatments available for treating DBDs, such as behaviour therapy,10 cognitive–behavioural therapy (CBT),11 psychodynamic therapy12 and play therapy.13 These psychosocial treatments can be delivered with diverse conditions, including parent training programmes, foster carer/guardian training programmes, child-focused programmes and multimodal interventions.4 National Institute for Health and Care Excellence (NICE) has suggested suitable delivery conditions for different age groups. NICE has recommended: (1) parent and foster carer or guardian training programmes or (2) child-focused programmes or (3) parent and child training programmes, for children and adolescents with DBDs aged 3–14 years. Meanwhile, for children and adolescents aged 11–17 years, NICE has recommended multimodal interventions. Previous research has not checked the comparative effectiveness of all the psychosocial treatment types yet, therefore, the current research will focus mainly on treatment types rather than delivery conditions unless there are enough numbers of included trials to distinguish between them. Another concern for psychotherapists, psychological counsellors and social workers is the acceptability of these psychosocial treatments. Some treatments are not easily accepted by participants due to high demands, long durations, stigmatisation, etc, even though they can significantly reduce symptoms of DBDs.14 Besides, due to methodological restrictions of conventional meta-analyses, it is still unclear which are the most efficacious and the most acceptable psychosocial treatments for DBDs. Fully investigating the comparative effectiveness and acceptability of all psychosocial treatments is beneficial for clinical practitioners to make an optimal and evidence-based decision on the treatment of DBDs.

Network meta-analysis (NMA), which can consider both direct (head-to-head comparison) and indirect (comparison of treatments via a common comparator) evidence, is an appropriate method to answer the questions above.15 The standard pairwise meta-analysis cannot be used to assess relative effects across treatments if the comparisons have not been evaluated in head-to-head trials. Instead, NMA allows the comparison of multiple treatments simultaneously within a single analysis as long as every treatment is connected to at least one of the other treatments under evaluation through direct comparisons.16 Some researchers have compared the efficacy of psychotherapies or psychosocial therapies for the treatment of mental illnesses in children and adolescents, including depression,17 acute anxiety disorders18 and post-traumatic stress disorders.19 One NMA has investigated the comparative effects of psychosocial and pharmacological interventions for disruptive behaviour in children and adolescents.20 However, the previous NMA grouped the same delivery conditions of psychosocial interventions (ie, child component, parent component and multicomponent) into a node. With an increasing body of research investigating the effectiveness of treatment for DBDs, it is possible to group the same types of psychosocial treatments into a node and conduct NMA. Besides, the previous research has not explicitly made a distinction between prevention and treatment, though their backgrounds, places and intervention methods are different.21 Therefore, the present protocol aims to address the above limitations and synthesise all direct and indirect evidence for identifying the optimal psychosocial treatment for children and adolescents with DBDs.

Objectives

The purpose of this study is to evaluate and compare the effectiveness and acceptability of psychosocial treatments for DBDs. Specifically, we aim to:

Assess the relative psychosocial treatment effects at post-treatment for DBDs, in comparison with one another.

Determine the acceptability of these psychosocial treatments.

Assess the relative psychosocial treatment effects at follow-up for DBDs, in comparison with one another.

Compare the effectiveness of psychosocial treatments on the improvement of internalising problem outcomes at post-treatment in children and adolescents with DBDs.

Compare the effectiveness of psychosocial treatments on the improvement of social functioning outcomes at post-treatment in children and adolescents with DBDs.

Methods and analysis

The systematic review and NMA is registered in the PROSPERO database (registration number: CRD42020197448) and has been developed according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) Protocols checklist, the extension statement for NMA and proposed additional considerations for protocols of systematic reviews including NMA.22–24 The planned start and end dates are 7 June 2020 and 31 August 2021, respectively.

Eligibility criteria for study design, participants, treatments and outcomes

Designs of studies

All randomised controlled trials (RCTs), including cluster RCTs and first phase of cross-over studies, will be included. Quasirandomised trials (eg, randomisation by the last number of the date of birth or day of the week) will be excluded. Moreover, according to many previous NMA,18 25 if the sample size is less than 10 per arm, the trials will be excluded for ensuring the power. Because the psychosocial treatments are difficult to be conducted in a double-blind design, we will include single-blind RCTs (ie, raters were blinded) or trials in which participants were assessed by self-rating scales. Considering the validity of young children’s reports, we will exclude trials in which outcomes were only reported by children younger than 11 years old. Language, year of publication and publication status will not be restricted.

Types of participants

Children and adolescents with DBDs, who were no more than 18 years old when they enrolled in trials, will be included. We will identify DBDs by either a formal diagnosis of DBDs on the basis of the DSM or the International Classification of Diseases or a standardised rating scale on DBDs. The common scales measuring DBDs and their clinical cut-off values are shown in table 1. Trials in which participants have a secondary diagnosis of comorbid psychiatric disorders (autism spectrum disorder, depressive disorder, anxiety disorder, etc), or intellectual, or neurological, or physical disability will be excluded. We will also exclude trials in which participants are at risk of other mental disorders (learning disorder, substance-related disorders, etc). It is noteworthy that trials in which participants have comorbidity with ADHD or emotional problems (ie, do not meet diagnostic criteria for mood disorders, anxiety disorder, etc) will be included. All treatment settings (ie, outpatient, inpatient services, community clinics and schools) will be included.

Table 1

Hierarchy of DBDs severity scales and their clinical cut-off values

Hierarchy	Scales	Abbreviation	Clinical range
1	Eyberg Child Behavior Inventory	ECBI	Norway version: >90 th percentile.US version: >15 for the problem score scale and/or >132 for the intensity score scale (>93 th percentile), problem score >15.
2	Achenbach System of Empirically Based Assessment	ASEBA	>98th percentile (T score >70) for oppositional defiant problems or conduct problems in DSM-oriented scales.
3	Strengths and Difficulties Questionnaire	SDQ	Conduct problems subscale score >5.
4	Revised Behavior Problem Checklist	RBPC	T score >70 for conduct disorder subscale.

DBDs, disruptive behaviour disorders; DSM, Diagnostic and Statistical Manual of Mental Disorders.

Types of treatments

According to NICE guidelines, psychosocial interventions for DBDs can be delivered through parent training programmes, parent and child training programmes for children with complex needs, foster carer/guardian training programmes, child-focused programmes and multimodal interventions.4 Psychosocial interventions are categorised according to the delivery conditions but not treatment types. Online supplemental table 1 shows descriptions and examples of common psychosocial treatment types and control conditions on the topic. The effects of many types of psychosocial treatments on DBDs have been explored in several reviews, traditional meta-analyses or RCTs. Because CBT is the most common treatment type for DBDs, CBT with different treatment focuses (social skills training, anger coping/management training or problem-solving skills training) different delivery conditions (child focused, parent focused or both parent and child involved), delivery mediums (internet based or face to face) and delivery formats (group, individual or group plus individual) will be separated as independent nodes if data are available. For the other treatment types, trials comparing the same treatment types will be grouped into the same node no matter which delivery conditions, delivery mediums and delivery formats they used. Control groups include no treatment, waitlist and treatment as usual.

Efficacy at post-treatment, measured using the change score between baseline and end-point (at post-treatment) from scales assessing the severity of DBDs.26 Where multiple scales are reported, we will extract data from the DBDs severity scales in a hierarchical fashion (table 1). Besides, when multiple raters report a DBDs severity scale, we will calculate the composite score according to NICE guidelines.4

Acceptability of psychosocial treatment, defined as the dropout rate for any reason during psychological treatments.

Types of outcome measures

Secondary outcomes

Besides the primary outcomes, we will also assess relative psychosocial treatment effects at follow-up for DBDs because we want to know whether the short-term effects of the psychosocial types are different from the long-term effects. Moreover, we will also compare the effectiveness of psychosocial treatments on the improvement of internalising problems and social functioning. We focus on these questions because DBDs are always complied with internalising problems and impairment of social functioning. It would be valuable to examine whether treatments that are effective in decreasing DBDs symptoms are equally effective in improving internalising problems and social functioning.

Efficacy at follow-up, measured by the change score of DBDs severity scales between baseline and 6-month follow-up/nearest 6-month follow-up.

Internalising problems, measured by the change score between baseline and end-point (at post-treatment) from internalising problem scales, such as Achenbach System of Empirically Based Assessment (ie, Child Behavior Checklist, Teacher’s Report Form and Youth Self-Report), Revised Behavior Problem Checklist, Child and Adolescent Functional Assessment Scale, Revised Children’s Manifest Anxiety Scale, State Trait Anxiety Inventory for Children, Beck Depression Inventory, Children’s Depression Rating Scale Revised and Hamilton Depression Rating Scale. If the mentioned scales are not available, other valid scales on internalising problems will be used.

Social functioning, measured by the change score between baseline and end-point (at post-treatment) from social functioning scales, such as Children’s Global Assessment Scale, Social Competence Inventory, Matson Evaluation of Social Skills with Youngsters, School Social Behavior Scales and Social Skills Improvement System Rating Scales. If the above scales are not available, other valid scales on social functioning will be used.

Search strategy

Ten electronic databases, including Web of Science, PubMed, PsycINFO, MEDLINE, APA PsycArticles, Psychology and Behavioral Sciences Collection, Open Dissertations, The Cochrane Library, Embase and CINAHL will be searched without restriction on language, publication status or publication period. We take Web of Science as an example; the following search terms are applied:

TS=(“conduct problem*” OR “conduct disorder*” OR “oppositional behavior*” OR “oppositional behaviour*” OR “oppositional defiant disorder*” OR “externalizing behavior*” OR “externalizing behaviour*” OR “externalizing disorder*” OR “disruptive behavior disorder*” OR “disruptive behaviour disorder*” OR “disruptive behavior*” OR “disruptive behaviour*” OR agressi* OR antisocial* OR callous-unemotion* OR delinquen* OR devian* OR hostile OR hostility OR bully* OR bullie*)

AND TS=(youth* OR child* OR adolescent* OR juvenile* OR boy* OR girl* OR parent* OR teenage*)

AND TS=(intervention OR therapy OR treatment OR evaluation OR “randomized controlled trial” OR “controlled clinical trial” OR effectiveness OR efficacy OR “controlled trial” OR randomized OR trial)

NOT TS=(animal*)

In order not to omit any relevant research, we will search eligible studies of reviews and meta-analyses on related topics and further search reference lists of all eligible studies manually. Moreover, we will contact the corresponding author to complement incomplete data.

Selection of studies and data extraction

All results generated from systematic searches will be imported in Noteexpress, and duplicates will be excluded. Two independent authors will identify initially 10% of studies from the titles and abstracts according to the predefined eligibility criteria for ensuring consistency. If a high level of inconsistency occurs, the eligibility criteria of the studies will be clarified through a discussion with a senior researcher. If there is a high degree of consistency, all potentially eligible articles will be identified from titles and abstracts by the two authors independently, and disagreements will be resolved by a senior researcher. Then, all full-text potentially eligible articles will be obtained and screened by the two independent authors according to the same criteria, and disagreements will be resolved as aforementioned. Additional information will be obtained from study authors if required. Reasons for exclusion for each trial will be reported at the stage of full-text screening. Finally, the process of study selection will be shown by using a PRISMA flow chart.

Data extraction

The following data will be extracted by two authors independently from all selected trials, and disagreements will be resolved by a senior researcher if required.

Study characteristics include study title, study authors, publication year, publication type, publication journal, country and source of funding, study design, randomisation and blinding.

Participant characteristics include age, gender, race/ethnicity (if it was reported in the study), socioeconomic status (if it was reported in the study), sample size, diagnostic criteria for DBDs, age of onset of DBDs symptoms, comorbidities, baseline severity, maternal mental health, parent–child relationship, parenting skills, parent readiness for treatment and the total number of participants.

Intervention characteristics include the type of psychosocial therapy, delivery condition (child focused, parent focused or both parent and child involved), delivery medium (internet based or face to face), delivery format (group, individual or group plus individual), treatment setting, duration of a session, number of sessions, frequency of treatment, length of treatment, fidelity (ie, the average implementing sessions divided by the total sessions of the programmes), people who delivered the treatment, follow-up duration and cointerventions.

Outcome measures include scores of mean and SD, number of participants, and people who rated the outcome (ie, children, parents, teachers, clinicians or researchers) for each predefined outcome.

Adherence measures include the total number of subjects at pretreatment and post-treatment and at follow-up measurements, and reasons for attrition to treatment.

Data at the closest time point to 6-month follow-up will be extracted if data for multiple follow-up time points were provided in the studies. We will contact the corresponding authors by sending emails if any information that we want to extract was not provided in their studies.

Risk of bias assessment

The risk of bias assessment will be assessed by two authors independently according to the revised Cochrane Collaboration’s risk of bias tool (RoB V.2.0) for RCTs.27 Any disagreement will be resolved by a senior researcher if required. The overall risk of bias will be rated as ‘low risk’ (ie, low risk of bias in all domains), ‘high risk’ (ie, high risk of bias in at least one domain or having some concerns in multiple domains) or ‘some concerns’ (ie, having some concerns in at least one domain and no high risk of bias in any domain). Specifically, we will answer the signalling questions following available algorithms and judge the risk of bias as low, high or some concerns for each domain: (1) bias deriving from the randomisation process (eg, sequence generation and allocation concealment), (2) bias arising from the blinding (eg, blinding of participants and blinding of outcome assessors), (3) bias caused by incomplete outcome data, (4) bias due to the measurement of outcome and (5) bias due to the selective reporting. The result of the assessment of the risk of bias will be presented in a risk of bias summary graph.

Data analysis

Bayesian NMA with a random effects model will be performed by using WinBUGS V.1.4.3 to synthesise all evidence for each outcome. Compared with the frequentist approach, the Bayesian framework can benefit decision making, which can help create stable estimates and their credible intervals, which is further advantageous for making probabilistic statements and predictions on the treatment effects more straightforward.28 Besides the Bayesian NMA, conventional pairwise meta-analyses with a random effects model will be employed for the comparison between active treatments and control arms by using Stata V.16. as a reference for the results of NMAs.

For continuous outcomes, standardised mean difference will be used as a measurement of effect size. We will use published mean values and SDs; if not available, we will estimate values by conversion from SEs, p values, CIs or t-values. We will contact the authors of the study to obtain information if none of the above values is provided in the published paper. If we cannot obtain the information using this approach, missing SDs will be derived from those of the other studies using a validated imputation technique.29 For the dichotomous outcome, the risk ratio, and its 95% CIs will be calculated as effect sizes. Missing data will be managed with respect to the intention-to-treat principle. Participants who drop out after randomisation are regarded as non-responders.

In order to assess the transitivity assumption of NMA, we will assess the distribution of clinical and methodological variables.30 Concerning clinical variables, we have assured the transitivity by limiting our samples to participants with DBDs and excluding participants with comorbid psychiatric disorders (autism spectrum disorder, depressive disorder, anxiety disorder, etc), or intellectual, or neurological, or physical disability. Other clinical or methodological variables that may influence the efficacy of psychosocial treatments include age, number of sessions and length of treatment.

Heterogeneity will be assessed using the I statistic and its 95% CI. For visualisation, the overlap of the CIs will be shown with forest plots. For the NMA, we assume a common heterogeneity variance across the various treatment comparisons and assess it with τ.2 Possible reasons for heterogeneity will be examined by subgroup analysis.

Furthermore, we will assess the global inconsistency as well as local inconsistency. Global consistency will be evaluated by calculating the design-by-treatment interaction test. The local inconsistency will be evaluated by comparing the disagreement between direct and indirect evidence in evidence loops. The results of the inconsistency test will be interpreted with caution because the test is known to have a low power.31

Finally, the surface under the cumulative ranking curve and mean ranks will be used to summarise the probabilities of treatments and provide a hierarchy of competing treatments.

Subgroup analyses, meta-regression and sensitivity analyses

If there are sufficient data in each subgroup, we will conduct a subgroup analysis to examine how treatment efficacy varies across different subgroups: (1) study setting (clinic, school or community), (2) age group (3–10 years, 11–14 years or 15–17 years). The age group is divided according to NICE guidelines, in which parenting training programmes were recommended for 3–11 years, cognitive behavioural approaches were recommended for 9–14 years and multimodal programmes were recommended for 11–17 years, (3) socioeconomic status, (4) outcome rater (composite, mother, father, teacher, child or observer), (5) age of onset of DBDs symptoms (3–10 years or 11–17 years), (6) diagnosis (formal diagnosis of ODD, formal diagnosis of CD or scale-assessed DBDs) and (7) country. Besides, we will conduct network meta-regression meta-analyses of data on the outcome of efficacy at post-treatment to evaluate the influence of the following potential moderators: (1) number of sessions, (2) length of treatment, (3) fidelity (ie, the average implementing sessions divided by the total sessions of the programme), (4) baseline severity (Strengths and Difficulties Questionnaire, Achenbach System of Empirically Based Assessment, Eyberg Child Behaviour Inventory or Revised Behavior Problem Checklist score at baseline), (5) maternal mental health, (6) parent–child relationship, (7) parenting skills and (8) parent readiness for treatment. Moreover, we will explore the sensitivity analyses by excluding: (1) studies in which missing data have been imputed, (2) studies in which high risk of bias rating have been assessed and (3) studies in which participants comorbidity with ADHD have been included.

Publication bias

First, if 10 or more studies are included in the NMA, funnel plots of pairwise NMAs will be examined. Moreover, comparison-adjusted funnel plots will be used to examine the association between study size and effect size. Furthermore, Egger’s test will be used to examine the significance of publication bias.

Quality of the evidence

Grading of Recommendations Assessment, Development, and Evaluation framework specifically developed for NMA will be applied to evaluate the quality of evidence.32 Specifically, we will characterise the credibility of a body of evidence on the basis of within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence by using the CINeMA software.33 The starting point for confidence in each network estimate is high but will be downgraded according to the assessments of the above six domains.

Ethics and dissemination

The NMA does not need ethical approval as no primary data are collected, and none human nor animal participants will be involved directly. Findings of the present research are planned to be published at national or international psychological conferences, or in a reputable scientific journal.

Patient and public involvement

No patient involved.

Discussion and summary

The systematic review and NMA will provide an overview and information on the comparative efficacy and acceptability of psychosocial treatments for children and adolescents with DBDs. The results will show a hierarchy of comparative efficacy with regard to symptoms of DBDs at post-treatment and follow-up, as well as in terms of acceptability, improvement of internalising problems and improvement of social functioning. Moreover, the results of subgroup analysis and meta-regression can help personalise the information to the youth, setting or other factors. To the best of our knowledge, this study will be the first NMA focusing on the treatment types rather than the delivery conditions in investigating the hierarchy of effectiveness and acceptability of psychosocial treatments for DBDs. The findings are expected to assist psychological counsellors, psychotherapists and social workers to make a better and evidence-based treatment choice.

It is worth noting that the findings need to be understood in light of study limitations. First, because of the fact that DBDs have high comorbidity with ADHD, we will not exclude trials with participants suffering DBDs comorbid with ADHD. Although results will reflect the real situation and contribute to more generalisable inferences, the risk of bias for outcomes will be raised. Second, when interpreting the results of NMA, we have to consider the heterogeneity of some variables (eg, duration of the treatment), which are not always the same across psychosocial treatments. Last, it is an excessive challenge to determine how to separate psychosocial treatments as independent nodes because some psychosocial treatments are multicomponent and vary in module, content, etc. In further research, we could conduct component NMA for a specific psychosocial treatment (eg, CBT) to further investigate whether some components are superior to others in the DBDs treatment.