Outcomes reported in randomised controlled trials of major depressive disorder in older adults: protocol for a methodological review.

INTRODUCTION: Major depressive disorder (MDD or depression) is prevalent among adults aged 65 years and older. The effectiveness and safety of interventions used to treat depression is often assessed through randomised controlled trials (RCTs). However, heterogeneity in the selection, measurement and reporting of outcomes in RCTs renders comparisons between trial results, interpretability and generalisability of findings challenging. There is presently no core outcome set (COS) for use in RCTs that assess interventions for older adults with MDD. We will conduct a methodological review of the literature for outcomes reported in trials for adults 65 years and older with depression to assess the heterogeneity of outcome measures. METHODS AND ANALYSIS: RCTs evaluating pharmacotherapy, psychotherapy, or any other treatment intervention for older adults with MDD published in the last 10 years will be located using electronic database searches (MEDLINE, Embase, PsycINFO and the Cochrane Central Register of Controlled Trials). Reviewers will conduct title and abstract screening, full-text screening and data extraction of trials eligible for inclusion independently and in duplicate. Outcomes will be synthesised and mapped to core outcome-domain frameworks. We will summarise characteristics associated with trials and outcomes. ETHICS AND DISSEMINATION: We hope that findings from our methodological review will reduce variability in outcome selection, measurement and reporting and facilitate the development of a COS for older adults with MDD. Our review will also inform evidence synthesis efforts in identifying the best treatment practices for this clinical population. Ethics approval is not required, as this study is a literature review. PROSPERO REGISTRATION NUMBER: CRD42021244753. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Our review represents the first step towards the development of a standardised initiative to inform outcome selection for trials assessing the effectiveness of interventions used to treat major depressive disorder in older adults.

Strengths of our review include: use of rigorous methodology to conduct data screening and extraction, and grouping of outcomes according to a standardised classification system.

The generalisability of our findings may be limited by our search strategy which focuses on published, peer-reviewed, English-language trials from 2011 and onwards.

Our ability to synthesise findings may be impeded by the heterogeneity in reported outcomes, and the subjective reporting of outcomes as either effectiveness or harms/adverse events across trials.

Findings from this review will contribute to the development of a core outcome set for older adults with depression, with the aim of reducing variability in outcome selection, measurement and reporting across trials examining this clinical population.

Introduction

Mood disorders or affective disorders are the terms given to the group of psychiatric disorders which characterise the lowering or elevation of one’s mood or emotional state.1 2 There are many types of mood disorders, including: major depressive disorder (MDD), bipolar disorder and cyclothymic disorder. MDD, hereafter referred to as depression, is the most common of mood disorders, and one of the leading causes of disability for older adults worldwide, accounting for 1.7% (95% CI=1.3% to 2.3%) of global disability-adjusted life years.3 Older adults with depression often experience adverse health outcomes, including a reduced quality of life,4 disability5 and mortality,6 in addition to placing a high burden on caregivers7 and increased costs for healthcare systems.8

Common interventions for treating MDD in older adults include but are not limited to: pharmacotherapy (antidepressants, eg, selective serotonin reuptake inhibitors, tricyclic antidepressants monoamine oxidase inhibitors and other forms of medication such as antipsychotics and lithium in treatment-resistant depression),9 psychotherapy (eg, cognitive–behavioural therapy, interpersonal therapy and psychodynamic therapy),10 11 light therapy,12 13 exercise therapy14 and neurostimulation treatments (eg, electroconvulsive therapy and repetitive transcranial simulation).15 16 Randomised controlled trials (RCTs) are widely considered the gold standard for assessing comparative effectiveness of therapeutic interventions.17 These trials are essential to understanding the unique clinical profiles of this population, given that comorbid mental and physical illnesses which often accompany ageing must be considered during the selection of treatment course. For instance, the use of antidepressants, although effective in treating depressive symptomatology, may result in adverse events due to comorbidities and drug–drug interactions if patients are on concomitant medications.9 Furthermore, there is a need to consider other social factors which affect non-adherence to prescribed treatment regimens among older adults with MDD, including but not limited to: socioeconomic status and insurance coverage, educational level and health literacy.18 Unfortunately, recent meta-analyses synthesising trials for depression among older adults have exhibited high heterogeneity, both in the use of dissimilar outcomes across studies, as well as differences in instruments used to measure the same set of outcomes,19 20 which creates challenges for the interpretation of trial results and limits their utility in clinical decision-making.

The inconsistency in the selection and reporting of outcomes for RCTs is a well-established issue in medical research,21–24 as little thought has been given to what constitutes a successful treatment outcome and measurement of outcome domains. In particular, the rationale for the use of a particular outcome and evidence to support the rationale,25 are infrequently, if ever, provided in studies which evaluate effectiveness of depression interventions.26 Although it is considered that different outcomes assess the severity of depression similarly and can therefore be used interchangeably, a content analysis revealed only a moderate mean overlap (Jaccard index=0.41; 0=no overlap, 1=complete overlap) between clinical rating scales, which have been commonly used as outcome measures in depression trials.26 27 Researchers and clinicians alike have also questioned the suitability of outcome measurements used in depression trials to gauge treatment effectiveness,28–30 and this variability in outcome measurements between studies makes it difficult to first determine the effectiveness of interventions and second, poses challenges for the generalisability of research conducted on depression.31 32

Furthermore, there is a need to consider which outcomes are important to patients suffering from MDD. Prior research on patients with depression has identified that outcomes which patients consider as markers of treatment success largely pertain to improvements in functional capacity, and include domains pertaining to social life, that is, reductions in social isolation, improved quality of interpersonal relationships and family life and the ability to resume professional responsibilities.33 These patient-important outcomes (PIO) are often neglected in RCTs which evaluate treatment success from clinician and policymaker perspectives, for example, improvements in depression symptoms as measured by standardised clinical scales. Establishing a common set of treatment outcomes for depression needs to consider the diverse perspectives of all stakeholders involved. Specifically, there is a clear gap in research in identifying measurable treatment outcomes that have a significant impact on improving the lives of the patient population they are designed to treat, and including PIO as measures of effectiveness in RCTs.

Core outcome sets (COS) have been proposed as a viable solution to addressing the heterogeneity of outcome selection in RCTs,21 22 24 and represent a minimum set of outcomes that must be measured and reported in trials pertaining to a particular illness (what to measure).34 COS are developed through systematic scans of the literature and consensus meetings to determine the outcomes used in these sets, for example, Delphi surveys and in-person meetings with clinicians, patients and policymakers.35 After a core set of outcomes has been identified, the outcome measurement instrument (how) and details of the measurement (when) are subsequently assessed through similar means, that is, literature scans and Delphi studies.36 The Core Outcome Measures in Effectiveness Trials Initiative is a repository of COS for different medical conditions.37 Although there have been initiatives towards the development of adolescent38–40 and general adult33 depression, there is no COS which exists for use in studies for older adults with MDD. Thus, trialists, clinicians and other evidence users lack an evidence-based means of selecting, measuring and reporting outcomes for this clinical population.

Our methodological review is the first step towards the development of a COS for RCTs evaluating treatments for older adults with depression. In this protocol, we outline the methods we will use to identify and describe outcomes reported in published trials for this clinical population. The findings from our methodological scan of the literature will be used to evaluate the extent of heterogeneity present in trials, assess whether PIO have been used in these studies, and identify a preliminary list of outcomes to consider for developing a COS for older adults with MDD.

Objectives

Our overarching goal is to identify and characterise outcomes reported in published trials of older adults with MDD. Our specific objectives are as follows:

To describe outcomes that have been reported in RCTs for older adults with depression (primary study objective).

To map reported outcomes to core outcome areas/taxonomic classifications which have been used in biomedical research: physiological/clinical, life impact, resource use, adverse events and death.41

To identify whether reported outcomes are consistent with relevant outcomes for depression which matter to patients, informal caregivers and healthcare providers33 (secondary study objective).

To identify how many trials use a single discernible primary outcome and report on the rationale for their use (tertiary study objective).

Findings from our study will be used to evaluate the extent of the heterogeneity present in outcome selection, and present an initial list of outcomes to consider towards the development of a COS for this clinical population, which are aligned with PIO.

Methods and analysis

Study design

We will conduct a methodological review of outcomes reported in published RCTs of interventions used to treat MDD in older adults. This is the most appropriate type of review for our study objectives, which seek to identify and map outcomes in this trial, and has been used in other domains.42–44

Protocol

Our protocol includes elements commonly used in protocols for other methodological reviews and/or methodological surveys of trials.45–49 Any amendments to this protocol will be detailed in the final publication. Our review will commence after peer review of this paper, and is anticipated to be completed by December 2021. This methodological review has been registered on PROSPERO.

Eligibility criteria

Studies included in our review will satisfy the following eligibility criteria, which have been based on the Population, Intervention, Comparators, Outcomes, Time, and Studies (PICOTS) framework.50–52

Population (P)

Older adults aged 65 years and older with a diagnosis of MDD, as defined by the Diagnostic and Statistical Manual of Mental Disorders (4th or 5th editions)53 or depressive disorder as defined by the International Statistical Classification of Diseases (10th edition) criteria.54 This age cut-off is a common threshold used in geriatric psychiatry.9 55 Diagnoses for depression will include a diagnosis by a psychiatrist, through administrative health records or clinical charts, or by use of a validated diagnostic tool, including but not limited to: the Beck Depression Inventory (BDI),56 Hamilton Rating Scale for Depression57 or the Center of Epidemiological Scale (CES-D).58 We will focus our review on unipolar and non-psychotic MDD, given that intervention choices may differ for these aforementioned forms of depression. Studies of patients with comorbid mental disorders, for example, anxiety and depression or schizophrenia and depression will be considered for inclusion, as comorbid mental illness is common among older adults.59 60 RCTs which include participants with different ages will be included only if there is a subgroup analysis containing adults aged 65 and older (eg, trials analysing a subgroup of adults aged 70–80 years would be eligible, but a subgroup analysis of ages 55–70 would not be eligible).

Intervention (I)

All interventions (pharmacological and non-pharmacological) will be considered for inclusion.

Comparators (C)

There will be no restrictions or specifications regarding comparators.

Outcomes (O)

All outcomes specified in published studies of randomised group comparisons will be eligible for inclusion. These may include outcomes pertaining to: delivery of care (eg, treatment adherence, intervention acceptability to patients), health status (eg, severity of depression) and/or health service use (eg, numbers of psychiatric appointments, emergency department visits or general practitioner visits).41 All outcomes will be collected in order to provide a comprehensive assessment of outcomes in trials of older adults with MDD and in accordance with recommendations followed by accepted taxonomy towards development of a COS.41 We will not include adverse events that emerge as a result of treatment and are detected through site visits, for example, headaches, as these are not planned outcomes of interest to assess the effectiveness of interventions and are pertinent to an intervention in a particular study.

Time (T)

Studies published within the last 10 years will be considered for inclusion in our review, so that we are reporting on trials which have been recently conducted. This approach is consistent with other methodological reviews.38–40 45 46 48 49 Furthermore, COS development guidelines suggest that overly large reviews are not necessary to yield additional outcomes, and that shorter timeframes for reviews may suffice in reaching saturation of outcome terminology.35 There will be no restrictions imposed on timing of outcome measurement, for example, when they have been measured post-enrollment, or on duration of follow-up post administration of the intervention.

Studies (S)

English-language RCTs will be eligible for inclusion, the language restriction in place for feasibility considerations. We will consider trials which have been conducted in all countries and settings (eg, inpatient, outpatient and community) in our review. We will not impose restrictions on RCT design, that is, single-arm, parallel, multi-arm and crossover trials will be eligible for inclusion. We will also include pilot and feasibility trials only when the full-scale RCT is not available for inclusion, so as to prevent duplication of reported outcomes.

Information sources and search strategy

Our search strategy and information sources have been developed in consultation with a health sciences librarian (SS), and will include the following databases: MEDLINE, Embase, PsycINFO and the Cochrane Central Register of Controlled Trials. Comprehensive search strategies (see table 1) will be used in the above databases to obtain articles using searches from 1 January 2011 until the date of the conducted search. Our search strategy has been developed through analysis of Medical Subject Headings terms and keywords identified though a preliminary search of the literature for relevant publications. Table 1 illustrates terms used in the most encompassing database, however final search strategies for all databases will be included in the review. De-duplication of records will be performed in the Covidence software platform.61

Table 1

Search strategy for methodological review

Database	Search terms
MEDLINE	1. Depression/ (130054)
	2. Depressive Disorder, Major/ (32723)
	3. (depressed or depressive or depression*).ti, ab, kf. (473472)
	4. or/1–3 (500071)
	5. Geriatrics/ (30556)
	6. geriatric*.ti, ab, kf. (68556)
	7. Aging/ (236743)
	8. exp Aged/ (3274084)
	9. (old* or elder* or senior* or aging).ti, ab, kf. (1923830)
	10. or/5–9 (4715917)
	11. exp randomized controlled trial/ (538724)
	12. exp Randomized Controlled Trials as Topic/ (149783)
	13. Random Allocation/ (105602)
	14. random*.ti, ab, kf. (1239648)
	15. double-blind method/ or single-blind method/ (195389)
	16. ((singl* or double or triple or treble) adj3 (method* or mask* or blind*)).ti, ab, kf. (229423)
	17. placebo*.ti, ab, kf. (227707)
	18. randomized controlled trial.pt. (537521)
	19. or/11–18 (1604401)
	20. 4 and 10 and 19 (19784)
	21. limit 20 to yr=‘2011 -Current’ (10004)
	22. (Treating anxiety and depression in older adults).m_titl. (1)
	23. Telephone-based cognitive behavioral therapy for depression in Parkinson disease.m_titl. (2)
	24. from 23 keep 1 (1)
	25. 22 or 24 (2)
	26. 21 and 25 (2)

We will also conduct manual searches of reference lists of all included studies to identify potentially missed trials. Articles identified from the reference lists will be reviewed in duplicate (detailed below) and included if they meet the aforementioned eligibility criteria. Conference reports, abstracts, unpublished reports or trial registries and other forms of literature that have not been peer-reviewed will not be included, because our objectives are to assess the measurement of outcomes in published trials.

Data management

We will import articles identified through the search strategies to the Covidence software platform, in order to conduct all phases of the review process. Covidence has the ability to store citation information imported from reference management software, and will be used to manage (1) title and abstract and (2) full-text screening, as well as (3) data extraction of included studies. Covidence permits study team members to review citation information independently and in duplicate, and extract data directly into forms on the software platform.61

Source selection

We will include a flow diagram (see figure 1) to summarise the screening process in our systematic reviews and meta-analyses. This will be structured according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) - 2020 guidelines,62 and detail numbers of studies and reasons for inclusion/exclusion. Included studies will also be described and detailed in a table.

Figure 1

Flow diagram of included studies.

Title and abstract screening

We will conduct title and abstract screening of every citation identified by the literature searches to assess eligibility of studies for inclusion in our review. Two reviewers will screen each citation independently and duplicate using the aforementioned PICOTS eligibility criteria at the title and abstract stage. Study team members will be instructed to err towards inclusion, that is, include studies if they are unsure about whether or not it meets eligibility criteria. Disagreements about study inclusion between reviewers will be resolved by a third reviewer to reach consensus. We will also assess level of agreement between study reviewers using the weighted kappa (κ) statistic63 calculated by Covidence. Studies will move on to the full-text screening stage if they have been selected for inclusion by both reviewers or have unresolved discrepant decisions regarding inclusion eligibility.

Full-text screening

During the full-text screening stage, two reviewers will screen each full-text article independently and duplicate. Disagreements about study inclusion between reviewers will be resolved by a third reviewer to reach consensus. We will contact authors, if necessary, if there is confusion regarding eligibility criteria. The level of agreement between study reviewers will be assessed at this stage using the weighted κ statistic63 calculated by Covidence. Reasons for study exclusion will be logged used Covidence and detailed in the PRISMA study flow diagram (figure 1). Included studies will move to the next stage into data extraction.

If there are multiple publications from a single trial, for example, several studies using the same patient population, we will select a single trial for inclusion in our review. In order to prevent duplicate study entries, we will select a trial: (1) with the largest sample size, (2) that has been published most recently, in case the sample sizes are the same across publications, (3) has the most number of outcomes, in case the sample size and publication years are the same or (4) applying a combination of the above criteria for inclusion. Decisions about study inclusion for duplicate trials will also be made independently and by two reviewers, with disagreements resolved by a third reviewer as needed.

Data extraction

Two members of the study team will perform data extraction independently and in duplicate for every included unique trial. After data charting has been independently completed by two reviewers, they will resolve disagreements through consensus, and with a third reviewer as needed. A standardised charting form will be developed a priori (see Online Supplementary Appendix A), and we will extract data pertaining to study and outcome characteristics. We will extract study-specific data pertaining to: study identifiers (eg, lead author surname, year of publication, name of journal), trial design (eg, single-arm, parallel, multi-arm or crossover), trial type (eg, full-scale, pilot or feasibility), study characteristics (eg, total sample size, intervention type—pharmacological or non-pharmacological, mean participant age, follow-up duration, study setting region and source of funding—industry-funded or not).

We will also extract the following information pertaining to outcomes: definition of the outcome or endpoint used in the trial, definition of meaningful change specified in the study, type of outcome measure (eg, single or composite outcome) and instruments used to measure outcome(s) (eg, BDI, CES-D or other standardised clinical scales). For instance, trials of older adults with depression often use the BDI (measurement instrument) to assess severity of depressive symptomatology (outcome). We will also extract whether study outcomes have been classified as: (1) primary, (2) secondary or (3) neither. The ‘primary’ outcome classification will apply to an outcome which: (1) studies have explicitly referred to using this language, (2) have been clearly specified in the objectives, that is, when studies aim to examine the effect of an intervention on a particular outcome or (3) in instances where the outcome data has been used to power the sample size for the trial.39 64 Given that multiple primary outcomes are often reported in trials of depression,65 we wish to capture the extent of this occurrence in our methodological review.

For studies which report a single, discernable primary outcome, we will extract: the source of information of the outcome (who), description of the outcome (what), location of assessment/setting of the outcome (where), timing of outcome measurement (when), rationale for selection of the outcome (why) and description of the means by which the outcome has been measured (how). We will also extract how outcome data has been managed and analysed, in addition to strategies employed in the analysis and interpretation of missing data.38 39

Data synthesis

Grouping of outcomes

After data extraction, outcomes will be grouped and synthesised using a thematic framework. This approach has been commonly used in other reviews and COS initiatives.38–40 45–49 64 66 67 For instance, we will use the term ‘social functioning’ to group outcomes relating to self-reported improvement in marital/spousal relationships and ‘delivery of care’ for patient satisfaction and treatment adherence. We will group individual components of composite outcomes under the appropriate terms. All outcome terms will be assigned to an outcome domain framework in consultation with psychiatrists (AD and ZS) and methodological experts (LT and SP).

Outcomes will be assigned or mapped according to two existing frameworks, the first of which has been used frequently in biomedical research. We will first assign all outcome terms to the taxonomic classification system proposed by Dodd and colleagues41, which comprise five domains: physiological/clinical, life impact, resource use, adverse events and death.41 We will also map outcome terms to outcomes which were mapped by a recent content analysis and identified outcomes for depression which matter to patients, informal caregivers and healthcare providers, including: symptoms, functioning, intervention safety, healthcare organisation and social representation.33

Synthesis of results

Our data synthesis of included studies will largely be descriptive. We will include quantitative measures (eg, counts, frequencies) of study characteristics (eg, the number of single-arm, parallel, multi-arm and crossover trials) and outcome-related information (eg, total number of outcomes per trial, total number of primary, secondary and other types of outcomes, median number of outcomes per study and the total number of outcome measurement instruments per trial). Tables will be used to display the characteristics of included studies, as well as outcome characteristics (ie, types and measurement), in addition to detailing the variation in outcome definitions present across the included trials. We will also report whether outcomes reported in trials that focus on older adults specifically, are different from those reported by trials which focus on other age groups, but include subgroup analyses of older populations.

We will map outcome terms to the two aforementioned frameworks using a modified outcome matrix which was developed by the Outcome Reporting for Brief Intervention Trials initiative68 and adopted for use by other COS reviews.39 40 64 66 67

Pilot testing

The lead author (MR) will conduct online tutorials to train study team members on use of the Covidence software. We will also conduct a calibration exercise, and pilot the full-text screening and data extraction of 10 articles by each research team member for response accuracy and adherence to the protocol and inclusion/exclusion criteria before the full-text screening and data extraction stages commence.

Risk of bias assessment or quality appraisal

We will not conduct risk of bias assessment or quality appraisal of included studies, since our aim is to report on outcome measurements. This approach is consistent with that employed by other methodological reviews.38–40 45–49

Patient and public involvement

Given that this is a methodological review, there was no patient or public involvement in its conception, and they will not be involved in its conduct, either.

Discussion

Strengths and limitations

Our methodological review is the first of its kind to our knowledge that aims to systematically assess reported treatment outcomes for MDD in trials of older adults. Our review will follow rigorous methodology, including a search strategy developed in consultation with an experienced librarian and duplicate screening and data extraction. We will also use the expertise of both psychiatrists and methodologists in grouping and mapping outcome terms, in accordance with a standardised classification system.41

However, our review is not without limitations. First, our search strategy is limited to RCTs published in the English language from 2011 onwards, and does not include abstracts, reports or trial registries. Although this restriction is aligned with our objectives and commonly practiced by other methodological reviews, it may nonetheless reduce the generalisability of our findings. Furthermore, prior research suggests that outcome reporting in RCTs not published in English may also be heterogeneous,69 about which our review cannot infer. Second, use of the COS taxonomy may not be ideal to classify all outcomes in reported trials. For instance, a methodological review on adolescent depression RCTs found that all outcomes were easily classified into the core domains suggested by the COS framework.38 40 In particular, within the context of depression, outcomes may be subjectively reported as either effectiveness or harms/adverse outcomes, further contributing to challenges in interpreting and synthesising trial results in our review.

Implications

The conduct of high-quality RCTs is crucial for the measurement and definition of outcomes which are aligned with knowledge synthesis approaches to gauge effectiveness of interventions among older adults with MDD. Both intervention effectiveness as well as generalisability of research are limited when choice of outcome measures are arbitrary,31 32 yet research on older adults with depression has rarely given thoughtful consideration to selection of an outcome measure, as very few studies provide the rationale and/or evidence to support their choice of a treatment outcome.25 26 Furthermore, PIO elicit insight on treatment effectiveness from patient perspectives and are critical to informed decision-making, but have been largely ignored in health research.

Our methodological review will identify the heterogeneity of outcomes in published trials for older adults with depression, and conclusions from our review will have implications for a core set of treatment outcomes used in studies which assess effectiveness of interventions for depression, including incorporation of patient perspectives on outcomes valued as markers of treatment success. Our findings will influence recommendations on future research conducted among people with depression, with respect to possible gaps and deficiencies in selection, rationality and reporting of treatment outcomes. Our review will also contribute towards the development of a COS used in trials for older adults with depression, with the aim of improving and standardising outcome selection. Furthermore, our results will also be of interest to a diverse group of stakeholders, including authors of methodological and systematic reviews, developers of clinical care guidelines, funding agencies and decision-makers in healthcare, who rely on information regarding treatment outcomes to assess effectiveness of various interventions in treating older adults with depression.

Ethics and dissemination

We do not require ethics approval for this study. The results from our methodological review will be disseminated in peer-reviewed journals and conference presentations to knowledge users, clinicians and other researchers.