Network meta-analysis on the comparative efficacy of family interventions for psychotic disorders: a protocol.

INTRODUCTION: Family interventions are effective and are strongly recommended for psychotic disorders. However, there is a variety of intervention types, and their differential efficacy is widely unclear. The aim of the planned network meta-analysis (NMA) is to compare the efficacy of family interventions that differ in content (eg, psychoeducation, mutual support, skills training) and format (eg, number of sessions, inclusion of patients, form of delivery). METHODS AND ANALYSIS: We will include randomised controlled trials comparing psychosocial interventions directed at the adult relatives, friends or non-professional carers of people with a diagnosis of a psychotic disorder (schizophrenia spectrum) to any kind of control condition. The main outcomes will be global clinical state for the patients and coping with psychosis as well as attitudes towards psychosis for the relatives. Additional outcomes will be severity of symptoms, functioning, burden and compliance/drop-out. We conducted a comprehensive search of Cochrane Central Register of Controlled Trials, MEDLINE(R), PsycINFO, Cumulative Index to Nursing & Allied Health Literature (8 August 2019) and reference lists of review articles. Full-text assessment of eligibility, data extraction and risk-of-bias assessment will be done by two independent reviewers. An NMA will be conducted for any of the planned outcomes and intervention characteristics for which sufficient and appropriate data are available. The analyses will make use of a random effects model within a frequentist framework. Estimates for all pairwise treatment effects will be obtained using standardised mean differences for continuous outcomes and risk ratios for dichotomous outcomes. Interventions will be ranked according to their relative efficacy. We will address the assumption of transitivity, heterogeneity and inconsistency using theoretical and statistical approaches. The possibility of publication bias and the strength of evidence will also be examined. ETHICS AND DISSEMINATION: There are no ethical concerns. Results will be published in peer-reviewed journals and presented at practitioners' conferences. PROSPERO REGISTRATION NUMBER: CRD42020148728. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Network meta-analysis will provide new information on the efficacy of different types of family interventions for psychotic disorders.

It will enable comparisons and increase the precision of effect estimates.

It will possibly result in a ranking of intervention types that can inform guidelines for clinical practice.

Diversity of studies may result in heterogeneity or inconsistency, which will be addressed with theoretical and statistical approaches.

Scarcity of studies for specific intervention types or for direct comparisons may reduce the connectivity of networks and limit the interpretation of results.

Introduction

Schizophrenia and other psychotic disorders are among the most severe mental disorders and cause immense suffering for millions of people and their families worldwide. The most widely available treatment is antipsychotic drug therapy. However, because of unsatisfactory response, problems with adherence and disabling side-effects,1 2 the focus has begun to switch more towards psychosocial interventions. Family interventions are one of two psychosocial therapies strongly recommended in recent clinical practice guidelines.3–5

The relatives of people with psychotic disorders play an important role in the course of the disorder. High levels of expressed emotions (EE), that is, criticism, hostility and emotional overinvolvement expressed by the family,6 were shown to be a reliable predictor of relapse in numerous studies.7 Additionally, relatives are often the primary caregivers of patients with psychosis and are thus generally considered ‘important in the process of assessment and engagement in treatment and also in the successful delivery of effective interventions and therapies for people with psychotic disorders’ (British National Institute for Health and Care Excellence guidelines, p28).4 As a result, attempts were made to positively influence the course of psychotic disorders by improving its management within the patients’ families. However, caring for a person with a psychotic disorder is often a heavy burden and relatives face considerable emotional, social and economical challenges.8–10 These stressful conditions are likely not only to affect the well-being of the relatives, but also to limit their long-term ability to support the patient. Therefore, reducing the burden of care and enhancing the well-being of carers has become an additional focus of family interventions.

Extensive research has been done on family interventions’ efficacy. Previous meta-analyses have found family interventions to substantially and consistently reduce relapse and rehospitalisation rates at follow-up assessments (risk ratios (RR) ≈ 0.5–0.8).4 11–13 Symptom severity and social functioning also show slight improvements, although the evidence base is less solid (standardised mean differences (SMD) ≈ 0.3–0.4).4 13 Regarding caregivers’ outcomes, the most profound finding seems to be a significantly reduced amount of high-EE families and a reduced burden of care.13 14

One well-known limiting factor of the evidence base and the delivering of efficacious family interventions is their great variability regarding content, aims and format.3 12 15 Pharoah et al13 list seven different strategies of family interventions and Lobban et al15 identified 11 content components. For example, interventions vary from psychoeducation in group settings (eg, the Survival Skills Workshop16) to individual systemic therapies (eg, the Milan systemic intervention17), from training of communication and problem-solving skills (eg, the Behavioural Family Therapy18) to mutual support groups,19 and from a few sessions to years of support.

Despite the variety in the content of interventions, we still know little about whether some approaches are more efficacious than others. Separate meta-analyses have been conducted on specific types of interventions, such as psychoeducation, psychoeducation plus skills training and systemic therapy.20–23 All of them found evidence for the specific interventions’ efficacy, although in regard to different outcomes and follow-up periods. One meta-analysis differentiated between cognitive–behavioural, purely behavioural and ‘pragmatic’ family interventions and found no difference in regard to relapse.24 For recent-onset psychosis, a meta-analysis found mutual support to be more effective than psychoeducation in improving family functioning one to two years after the intervention.14 However, this analysis was only based on two studies that had directly compared these approaches. In a systematic review of randomised controlled trials (RCT) on outcomes for relatives of people with psychotic disorders, none of the content components or types assigned to the studies (eg, psychoeducation only, psychoeducation plus mutual support, psychoeducation plus skills training) reliably distinguished effective from ineffective interventions.15

Similarly, we are still in the dark about the ideal format of a family intervention. Some meta-analyses have calculated effects for subgroups of studies with specific formats. These indicate that more extensive interventions are more successful in reducing relapse4 12 and demonstrate that the positive evidence for family interventions is mainly based on interventions that include the patient.4 Meta-analyses on studies that have directly compared interventions with a focus on single families versus multiple family groups did not find them to differ in terms of relapse, but suggest that working with single families is better accepted than group settings involving multiple families.4 13

The existing meta-analytic approaches have several limitations when it comes to evaluating the comparative efficacy of different family intervention types:

The differentiation of intervention types is unclear. In addition to unclear descriptions in primary studies,15 many interventions comprise a variety of different components. The definitions used in existing meta-analyses and reviews are diverse and often imprecise.

There are only few studies that directly compare different types of interventions and these have rarely been meta-analysed. Most of the meta-analytic evidence is based on comparisons of a single intervention type to a non-intervention control condition.

There is a wide variety of outcome measures and time points. The specificity of previous meta-analyses and the diversity of findings make it difficult to compare the results and to draw reliable conclusions.

For most of the intervention types the number of studies is small. For example, many subgroup meta-analyses for specific formats are based on less than five studies.4 13 This results in imprecise effect size estimates and limits the interpretation of significance.

The differential effects of some important characteristics of family interventions (eg, media-based vs face-to-face formats) have not yet been investigated meta-analytically.

On this background, it is still widely unclear whether there are significant differences in efficacy between different types of family interventions. This study aims to compare the types of family interventions in a systematic and consistent manner, using the method of network meta-analysis (NMA). NMA extends on classical meta-analysis because it allows to simultaneously compare multiple intervention types and to combine the evidence of direct comparisons with indirect comparisons to common control conditions.25 26 This enables us to evaluate the comparative efficacy of intervention types that have not yet been directly compared. Due to the inclusion of additional information from indirect comparisons, the efficacy can also be estimated more precisely.27 Finally, all intervention types can be ranked according to their relative efficacy. This information could be of high value for patients, their families as well as healthcare providers and may serve as a guideline for delivering the most efficacious interventions.

Objectives

Using NMA, this study aims to compare the efficacy of different types of family interventions for people with psychotic disorders and their relatives. Interventions will be differentiated by content (eg, psychoeducation, mutual support, skills training) and format (eg, number of sessions, inclusion of patients, form of delivery).

Methods and analysis

Methods for this NMA are based on the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P),28 the PRISMA extension statement for reporting of systematic reviews incorporating NMA of healthcare interventions29 and the chapter of the Cochrane Handbook on undertaking NMA.25 The NMA has been registered in the PROSPERO database (CRD42020148728); the record will be updated with any changes made to the protocol.

Eligibility criteria

To include a sufficient number of studies for the specific direct and indirect comparisons and to increase the connectivity of the networks, some of the eligibility criteria are more lenient than in traditional meta-analysis.30

Population

Eligible studies will have to deal with two populations:

People with a diagnosis of a psychotic disorder, defined as schizophrenia, schizoaffective disorder, schizophreniform disorder, brief psychotic disorder or delusional disorder. In order to meet the diversity of clinical practice,31 all kinds of definitions and diagnostic procedures as well as all kinds of comorbid disorders will be accepted. Populations with subthreshold psychotic symptoms (eg, high-risk populations) will be excluded. There will be no restrictions regarding age as family interventions are also common in younger people/adolescents (eg, first episode of psychosis). Samples also including people without psychotic but with other diagnoses are eligible if most of the patients are diagnosed with a psychotic disorder. ‘Most’ is defined as a minimum of 75% to ensure that the included studies allow drawing conclusions about the population of interest and in accordance with a Cochrane meta-analysis on family interventions for schizophrenia.13 If only broad categories of psychotic disorders in major classification systems, such as ‘Schizophrenia and other psychotic disorders’ in the Diagnostic and Statistical Manual of Mental Disorders IV32 or ‘Schizophrenia, schizotypal and delusional disorders’ in the International Statistical Classification of Diseases and Related Health Problems: 10th revision,33 are used to describe the diagnoses, we assume that at least 75% are psychotic disorders as defined above.

The relatives, spouses/partners, friends or non-professional carers of the people with psychotic disorders, as defined by the study. They will have to be of an adult age, minor children of people with psychotic disorders will be excluded.

Intervention

Family interventions, defined as any psychosocial intervention directed at the relatives, spouses/partners, friends or non-professional carers of people with a psychotic disorder, will be included. If the intervention also comprises treatment elements for patients only, a substantial part of the intervention as a whole (ie, at least 50%, excluding treatment as usual or comparator elements) has to be directed at a relative, spouse/partner, friend or non-professional carer.

The exact types and definitions of interventions to be compared in the analyses will depend on the distinct interventions and comparisons realised in the included studies and may be adjusted, amended or lumped. The following categories were created in consideration of the classifications resulting from previous review articles3 12 15 34 and after pilot screening of about 25 primary study articles. These articles were identified as key publications in reviews and on the basis of the current state of research. AL read the full texts of the publications; the study designs were then discussed in the research team.

Content of the intervention:

Counselling and management: general support in organisational, social or emotional issues.

Mutual support: bringing relatives together to talk about their experiences and support each other.

Psychoeducation: providing manualised information on the psychotic disorder, its treatment or ways of coping.

Skills training: learning and application of predefined skills to enhance coping with the patient or own distress (eg, regarding communication, problem solving, stress management).

Individual psychological therapy: in-depth work on individual problems using psychological (eg, systemic, cognitive, psychodynamic) approaches.

Number of sessions: 1–10 sessions versus 11–30 sessions versus >30 sessions.

Inclusion of patients: in no session versus in some sessions versus in all sessions.

Form of delivery: media-based only versus multiple family group sessions only versus single family sessions only versus combination of group and single sessions.

If different types of family interventions are compared in a study, they will have to differ in at least one of these categories because otherwise their comparison could not be included in the network.

Comparator

Any kind of comparator will be included. The following classification of the comparator’s content is also preliminary and may be adjusted through the process of study selection. So far, it includes no treatment versus treatment as usual (as defined by the study) versus psychosocial intervention for patients only (eg, psychoeducation, cognitive–behavioural therapy, social skills training).

Study type

RCTs will be included, defined according to the Cochrane Handbook as trials in which ‘the author(s) state explicitly (usually by some variant of the term ‘random’ to describe the allocation procedure used) that the groups compared in the trial were established by random allocation’.35 Quasi-RCTs, cluster RCTs and cross-over trials will be excluded.

Outcomes

The outcomes were selected and defined in consideration of previous reviews, pilot screening of study articles and statistical properties. However, here too, the definitions may be refined during study selection and data extraction based on the specific measures used in the included studies. The most commonly reported measures of family interventions’ efficacy will serve as main outcomes.

Global clinical state, including the occurrence of relapse, hospitalisation, crisis care service use or remission/recovery, as defined by the study.

Coping and attitudes, including endpoint scores in rating scales or questionnaires concerning EE, communicative or problem-solving skills, coping with stress and negative affect, understanding of the patient’s feelings, knowledge of psychosis or positive beliefs and attitudes towards the disorder.

Severity of symptoms, including endpoint scores in rating scales or questionnaires concerning psychotic symptoms or overall mental health.

Functioning, including any quantification of global, social, occupational functioning or living skills.

Burden, including endpoint scores in rating scales or questionnaires concerning subjective or objective burden of care, emotional response (eg, stress, anxiety, depression), mental health, well-being or quality of life.

Compliance/drop-out, including number of relatives/carers leaving the study early for any reason since randomisation.

Further selection criteria

The measures within the outcome categories were sorted according to how well they represent the category’s heading, that is, the concept of interest, and how common they are. Only established scales and subscales of questionnaires and rating scales, for which reliability and validity have been examined, will be included. Broad questionnaires or rating scales that cover further domains than those of interest will be included if the main focus of the questionnaire/scale (defined by at least 75% of the questions or subscales) pertains to the domain of interest. To be eligible, studies will have to report numerical outcome data for the calculation of SMDs for continuous outcomes and RRs for dichotomous outcomes. Change scores will be excluded because the combined analysis with endpoint scores is not recommended when using SMDs.36

If a study reports multiple eligible outcome measures with sufficient information for effect size calculation, the measure for inclusion will be selected based on the following criteria, which were developed to maximise validity and homogeneity of study outcomes:

For multiple measures within the same outcome category, measures will be preferred according to the order they are listed above (eg, for global clinical state, our first preference will be measures of relapse, if these are not reported, we will include measures of hospitalisation, then measures of crisis care service use and then measures indicating the extent of remission or recovery).

For multiple time points, outcomes at one year (after the beginning of intervention/baseline assessment) or those closest to this time point will be preferred.

For multiple methods of assessment (eg, different scales), the most commonly reported will be determined among the included studies and this will be preferred for inclusion. In cases in which more than one outcome remains, the one with the best psychometric properties will be preferred.

For multiple samples, the one with the most comprehensive and adequate outcomes or—if outcomes are equivalent—the largest N will be preferred.

The criteria may be adjusted if this is required based on the included studies. If no single outcome measure can be determined based on these criteria, a composite effect will be calculated (eg, for separate scales of positive and negative psychotic symptoms). The procedure of calculating a composite effect will follow the proposals made by Borenstein et al.37

Report characteristics

Reports will have to include an English abstract to evaluate eligibility in a first step. Conference/congress abstracts and trial registrations without results will be excluded as these reports do not present sufficient data for the assessment of eligibility, especially regarding outcomes. However, we will try to identify more comprehensive publications for relevant trials (see the Searches section). Foreign language articles will be translated. If there are multiple reports referring to the same study of which at least one presents sufficient information for our analyses in English, the reports in other languages will not be considered due to limited resources.

Searches

The bibliographic databases Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE(R), Ovid PsycINFO and Cumulative Index to Nursing & Allied Health Literature (CINAHL) were searched on 8 August 2019. There was no limit for year of publication, except that Ovid MEDLINE(R) (and Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Daily and Versions(R)) was only searched for studies published since 2005. This is recommended in the Cochrane Handbook35 to supplement but not duplicate the search for RCTs by CENTRAL.

The search strategies were created using database-specific subject headings and a variety of free-text terms reflecting the key concepts of interest. To limit results to RCTs, we developed search terms according to the ‘Cochrane Highly Sensitive Search Strategy for identifying randomized trials in MEDLINE: sensitivity- and precision-maximizing version (2008 revision)’,35 supplemented with various free-text terms from CENTRAL’s most recent search strategy for identification of RCTs in Embase (https://www.cochranelibrary.com/central/central-creation). The search strategies are included in the online supplemental material 1.

Additionally, the reference lists of the most relevant, recent and comprehensive review articles were screened to identify articles missed by the computerised search. Reviews were identified by the above searches and by searching the Cochrane Database of Systematic Reviews. We also try to identify full reports of relevant trial registrations, study protocols and congress abstracts as well as cross-references in primary articles.

Study selection

Bibliographic data of all articles retrieved by the search will be imported into the reference management software ZOTERO. They will be deduplicated using the tool of the Systematic Review Accelerator by the Centre for Research in Evidence-Based Practice (https://sr-accelerator.com/%23/), which was found to have good sensitivity and specificity,38 and additional manual screening. Titles and abstracts of the remaining articles and the additional articles identified from references will be screened by AL to identify studies that potentially meet the inclusion criteria. Of those, the full texts will be consulted. We will try to identify overlapping samples and multiple reports of the same study by comparing, for example, author names, study sites and years, sample sizes, demographic and clinical data, treatment descriptions and results. Eligibility of each study will be assessed independently by two review team members and documented in an adapted version of the Cochrane data collection form (intervention reviews—RCTs only). Discrepancies will be identified and a consensus will be formed by discussion, including a third review team member where necessary. In the case of major unclarities, we will attempt to obtain additional information from previous reviews that included the study or from the study authors if there is an accurate email address available.

Data extraction

A data collection form and tool will be created and pilot tested in consideration of the recommendations in chapter 5.3 and 5.4 of the Cochrane Handbook.39 Two review team members will extract data independently; discrepancies will be resolved through discussion, including the third review team member where necessary. In the case of missing data or unclear study information, we will again consult previous review articles or—if an accurate email address is available—ask the study authors to provide the relevant information on intervention characteristics and outcome data. Extracted information will include:

Characteristics of the study: year, country.

Characteristics of the report(s): author(s), year, type of publication.

Characteristics of the sample: number randomised per group, sex, age

for patients: diagnoses, diagnostic criteria, stage of disorder, specificities (eg, regarding symptomatology, medication).

for relatives/carers: threshold of baseline dysfunction/distress for inclusion, specificities (eg, regarding relationship to the patient).

Characteristics of the intervention:

content: category (see eligibility criteria) and further description, for multiple content components: proportion of each component in the total intervention.

timing: number of sessions, duration of each session, duration of the treatment period.

inclusion of patients: number of sessions/amount of the intervention including the patients.

form of delivery: number/amount of group and single sessions or media-based only.

co-treatment(s) for patients only.

Characteristics of the comparator: category (see eligibility criteria) and further description.

Outcome(s): type of measure, operationalisation/instrument, time point of measurement, imputation of missing data, results (intention-to-treat analyses preferred).

Risk-of-bias assessment

Risk of bias in individual studies will be assessed using the Version 2 of the Cochrane risk-of-bias tool for randomised trials (RoB 2).40 This recently developed tool uses signalling questions and an algorithm to help making judgements of ‘low risk’, ‘some concerns’ or ‘high risk’ related to five domains:

Bias arising from the randomisation process.

Bias due to deviations from intended interventions.

Bias due to missing outcome data.

Bias in measurement of the outcome.

Bias in selection of the reported result.

Additionally, an overall risk of bias will be determined by the tool’s algorithm. The assessment will be done for each of the trial’s outcomes and with respect to the assignment to the intervention (intention-to-treat effect). Two trained review team members will independently answer the signalling questions using the available Excel form. Disagreements will be identified with the discrepancy check function and will be resolved by discussion, involving the third review team member where necessary. As high risks of bias may result in an overestimation of family interventions’ efficacy, studies with an overall high risk will be excluded from sensitivity analyses.

Data synthesis

An NMA will be conducted for each of the planned outcomes and intervention characteristics for which sufficient and appropriate data are available. The R package netmeta will be used. Analyses will apply a random effects model within a frequentist framework.41 We will assume a constant heterogeneity across the comparisons within a network. A network plot will be used to display the intervention types (as nodes) and the quantity of studies for all possible treatment comparisons (as connecting lines). Estimates for all pairwise treatment effects will be obtained using SMDs for continuous outcomes and RRs for dichotomous outcomes, both with their 95% confidence interval. The effects will be displayed in a league table. Interventions will be ranked using P-scores, a frequentist analogue to the Surface Under the Cumulative Ranking curve (SUCRA).42 Some interventions may have several distinct content components. If there are appropriate data, the influence of the individual components will be evaluated in an additive NMA, assuming that the effect of an intervention with several components is the sum of the effects of its individual components.43

Diversity and transitivity

Diversity of the included populations, interventions and outcomes may lead to statistical heterogeneity. This may also threaten the assumption of transitivity, that implies that the effect of treatment A versus B can be indirectly determined via a common treatment C. For this assumption to hold, trials that investigate different types of interventions need to be comparable in regard to clinical or methodological variables that may influence the treatment effect. However, the intervention types we plan to investigate may not be independent of each other. For example, psychoeducative interventions may have fewer sessions than other content types, which would threaten the transitivity of the NMA for content and number of sessions. Other potential effect modifiers are participant or outcome characteristics. For example, the type of family intervention might differ depending on the severity of the relatives’ dysfunction/distress, which was found to be associated with the efficacy of interventions,15 or interventions with more sessions may report on outcomes with longer follow-up periods.

To evaluate the likelihood of transitivity, the distribution of potential effect modifiers will be compared between trials of different intervention types and investigated for similarity.25 44 45 Based on Cochrane recommendations,36 clinical hypotheses and empirical evidence,4 15 characteristics for investigation will include:

Intervention characteristics: content of the intervention, number of sessions, inclusion of patients, form of delivery.

Existence of co-treatment(s) for patients only (above treatment as usual).

Stage of disorder (first-episode only vs longer duration/mixed samples).

Baseline dysfunction/distress of relatives/carers (above a defined threshold vs no such selection).

Regarding outcome characteristics, such as type and time point of measurement, we aim to reduce heterogeneity and maximise the chance of transitivity by including only psychometrically established scales and selecting homogeneous measures if multiple eligible outcomes are reported (see the Outcomes section).

Assessment of heterogeneity and inconsistency

Transitivity will also be addressed by assessing the consistency of direct and indirect evidence,30 with both local and global methods.25 44 Locally, the inconsistency of a specific treatment comparison will be evaluated by splitting the network estimate into the contribution of direct and indirect evidence and checking for agreement. Global heterogeneity of the network will be assessed using a generalised Q- and I-statistic.41 A decomposition into Q-statistics for heterogeneity within designs (ie, studies with the same treatment comparisons) and between designs will help to identify sources of heterogeneity and to evaluate the inconsistency of the network as a whole.

Metabias and strength of evidence assessment

We will examine the possibility of publication bias by non-statistical considerations46 and calculation of ‘comparison-adjusted’ funnel plots to assess funnel plot asymmetry.47 The strength of evidence for the estimates of the main outcomes will be evaluated according to the proposals made by Salanti et al,46 who adapted the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework specifically for NMA. This extensive approach may be adapted to the data and resources available.

Additional analyses

In sensitivity analyses, studies with an overall high risk of bias will be excluded. If there is evidence that the assumption of transitivity is threatened within a network (see the Diversity and transitivity section), that is, if there are significant differences between the distributions of potential effect modifiers, we will try to increase consistency and homogeneity by performing subgroup NMAs for the distinct manifestations of the potential effect modifiers. For example, regarding the number of sessions, there may be separate analyses for studies with a focus on psychoeducation and studies with other foci.

Patient and public involvement

Patients or the public will not be involved in the design or conduct of the study. However, the Empower Peers to Research (EmPeeRie) Now group at the UKE (Outpatient Clinic at the University of Hamburg) will consult on issues concerning reporting, interpretation and dissemination of findings. The EmPeeRie Now group consists of members with lived experience of mental disorders, with multiple members having lived experience in psychosis.

Ethic and dissemination

There are no ethical issues apparent. Results will be published in peer-reviewed journals and be presented at practitioners’ congresses. Full data will be made available on request.