Methodological recommendations for cognition trials in bipolar disorder by the International Society for Bipolar Disorders Targeting Cognition Task Force.

OBJECTIVES: To aid the development of treatment for cognitive impairment in bipolar disorder, the International Society for Bipolar Disorders (ISBD) convened a task force to create a consensus-based guidance paper for the methodology and design of cognition trials in bipolar disorder.
METHODS: The task force was launched in September 2016, consisting of 18 international experts from nine countries. A series of methodological issues were identified based on literature review and expert opinion. The issues were discussed and expanded upon in an initial face-to-face meeting, telephone conference call and email exchanges. Based upon these exchanges, recommendations were achieved.
RESULTS: Key methodological challenges are: lack of consensus on how to screen for entry into cognitive treatment trials, define cognitive impairment, track efficacy, assess functional implications, and manage mood symptoms and concomitant medication. Task force recommendations are to: (i) enrich trials with objectively measured cognitively impaired patients; (ii) generally select a broad cognitive composite score as the primary outcome and a functional measure as a key secondary outcome; and (iii) include remitted or partly remitted patients. It is strongly encouraged that trials exclude patients with current substance or alcohol use disorders, neurological disease or unstable medical illness, and keep non-study medications stable. Additional methodological considerations include neuroimaging assessments, targeting of treatments to illness stage and using a multimodal approach.
CONCLUSIONS: This ISBD task force guidance paper provides the first consensus-based recommendations for cognition trials in bipolar disorder. Adherence to these recommendations will likely improve the sensitivity in detecting treatment efficacy in future trials and increase comparability between studies.

INTRODUCTION

There is a pressing need for treatments targeting cognitive impairments in bipolar disorder to enhance functional recovery and reduce societal costs.1 Specifically, cognitive impairments in sustained attention, verbal memory and executive function often persist after clinical remission from mood episodes 2, 3 and may be influenced further by mood‐stabilizing and antipsychotic treatments.4, 5 Emerging evidence points to cognitive heterogeneity in the remitted phase of bipolar disorder, with 12%‐40% of patients displaying global cognitive deficits, 29%‐40% showing selective decline in attention and psychomotor speed, and 32%‐48% being relatively cognitively intact.6, 7, 8 Notably, the subgroups with neurocognitive impairments present reduced functional capacity, more stress and poorer quality of life than patients who are cognitively intact, despite similar degrees of subsyndromal mood symptoms.7, 8 This highlights a need to target cognition treatments specifically to the cognitively impaired subgroups.

There is, however, a lack of clinically available effective treatments for cognitive impairments in bipolar disorder. This is partially a result of major methodological challenges in this relatively new field, as highlighted in a previous systematic review of clinical studies of candidate cognition treatments in bipolar disorder1 and a recent expert opinion paper.9 In particular, the review noted that the majority of trials used no systematic pre‐screening for cognitive impairment. Further, criteria used for selecting trial participants, tests used for assessing treatment efficacy on cognition, allowance of non‐study medications, and statistical procedures differed between the studies.1, 9 Such broad methodological discrepancies provide the impetus for greater consensus in the field, on the design and methodology of cognition trials in bipolar disorder, to optimize the chances of demonstrating treatment efficacy on cognition and to aid comparability across trials. In addition, a key challenge in the clinical management of bipolar disorder is the absence of recommendations for physicians on how to assess and manage cognitive impairments in their patients. The International Society for Bipolar Disorders (ISBD) therefore convened an expert task force under the lead of Dr Miskowiak with the aim of developing (i) a consensus‐based guidance paper for the methodology and design of cognition trials in bipolar disorder for pharmacological as well as non‐pharmacological interventions, (ii) a clinical recommendations paper for physicians on how to address cognitive impairments in their patients, and (iii) an educational patient booklet with information about cognitive impairments and pragmatic strategies for compensating for these in daily life.

This specific paper addresses goal (i). As the field is at an early stage in its development, defining a set of fixed guidelines for cognition trials would be premature; the aim of the present paper is instead to provide consensus‐based guidance and recommendations from international experts in the field that can support decisions regarding the methodology and design of future cognition trials in bipolar disorder. While the task force recommendations are primarily intended for randomized controlled cognition trials, the principles can also be applied to naturalistic or quasi‐experimental studies. These consensus‐based methodological recommendations build on insights from a recent systematic review of trials targeting cognition in bipolar disorder1 and an expert opinion paper,9 which were reviewed, discussed and expanded on by members of this ISBD task force.

MATERIALS AND METHODS

The ISBD Targeting Cognition Task Force

The ISBD Targeting Cognition Task Force was initiated by Drs Miskowiak, Kessing, and Vieta and consisted of 18 international experts in cognition in affective disorders from the following nine countries (in alphabetical order): Brazil, Canada, Colombia, Denmark, Japan, New Zealand, Spain, the UK, and the USA. The members of the ISBD Targeting Cognition Task Force were selected based upon their expertise in cognition in bipolar disorder and include several members of a previous ISBD Cognition Task Force led by Dr Yatham.10

The process of the task force

A series of major methodological challenges in the field were identified from a recent systematic review of cognition trials in bipolar disorder1 and a methodological expert opinion paper on the assessment of cognition in this patient group.9 An introductory face‐to‐face meeting was then held at the European College of Neuropsychopharmacology (ECNP) annual congress (Vienna, September 2016), during which the overall work timelines for accomplishing the goals were discussed, a tentative list with the key methodological issues in cognition trials was reviewed and agreed upon, and possible solutions to the issues were discussed. This was followed up by a telephone conference with the members of the task force who were unable to attend the introductory meeting. During the call, the identified methodological challenges and possible solutions were discussed and agreed upon, and additional challenges were added to the list. Consensus on the methodological challenges and recommendations with regard to how these challenges may be tackled was reached through subsequent email exchanges. Any differences in opinion between task force members were resolved in all cases through a telephone call between the particular task force members and the task force chair (Dr Miskowiak). The use of formal consensus methods such as the Delphi method was deemed unnecessary given a high degree of agreement among the members of the task force.

RESULTS

How can we enrich trials with cognitively impaired patients?

One‐third to half of patients with bipolar disorder do not show clinically relevant objectively measured cognitive impairments during remission. A major reason for the overall lack of progress in development of new treatments for cognitive impairment in bipolar disorder may have to do with the fact that most (80%) cognition trials do not pre‐screen patients for cognitive impairment, thus including a percentage of cognitively intact patients, which affects signal detection.1 The task force therefore considers it critical in randomized controlled trials (RCTs) to pre‐screen patients for cognitive impairment. This will increase statistical power and thus the likelihood of signal detection, minimize the risk of unnecessary exposure of cognitively intact patients to novel investigational treatments, reduce treatment development costs, and result in a shorter time to bring novel treatments to patients with cognitive deficits.

Among the 20% (n = 23) of cognition trials in bipolar disorder that did include pre‐screening for cognitive impairment, a vast majority used subjectively self‐reported rather than objective neuropsychological screening tools.1 Since up to half of remitted patients are objectively cognitively intact despite subjective cognitive complaints,6, 7 this can lead to enrollment of patients with little or no scope for cognitive improvement and thus a high risk of type II errors.11, 12, 13 Indeed, recent evidence from several cognition trials suggests that patients with objectively assessed cognitive impairment have substantially greater chances of achieving treatment efficacy on cognition than those who are non‐impaired.11, 12, 14, 15, 16 While subjectively perceived cognitive difficulties may also increase the likelihood of treatment efficacy on cognition, this association is weak14 and not consistently observed.15 It therefore seems insufficient to rely only on patients’ subjectively reported cognitive difficulties, although these are arguably important for patient participation in trials and for clinical meaningfulness of cognition treatments. Indeed, the correlation between subjective and objective measures of cognition is poor,17, 18 indicating that it is not always the patients with most subjective complaints who show greatest objective deficits and vice versa. This discrepancy seems to be influenced by patients’ clinical characteristics, with disproportionately more subjective than objective cognitive impairment in patients with greater subsyndromal depression or mania symptom severity, bipolar disorder type II (vs type I), and greater illness chronicity.19

Given the poor correlation between subjective and objective cognition measures and some evidence for greater treatment benefits in objectively impaired patients, it seems reasonable to enrich for objective cognitive impairment in future trials to achieve more positive outcomes of RCTs that utilize objective primary outcome measures. Subjective cognitive impairment is also important to ensure that the treatment is meaningful and that patients comply with trial requirements. However, some patients are unaware of their deficits and do not report them, but may nevertheless experience difficulties with retaining normal functioning at work and in daily life due to their cognitive impairments.20 Given this, it would be advisable to enroll patients that present with both (i) subjective cognitive difficulties and/or socio‐occupation problems and (ii) objectively measured cognitive deficits. For efficiency purposes, a sequential screening process may be considered. First, a large number of patients could be given easy‐to‐administer subjective measures (questionnaires) to pre‐screen for cognitive complaints and socio‐occupational problems. This will aid recruitment and retaining of participants in the trial. Second, patients with self‐reported cognitive complaints and/or socio‐occupational difficulties can then be assessed with an objective tool (i.e., brief neuropsychological test) to substantiate that they also display measurable cognitive deficits and thus ensure indication enrichment.

Several self‐report and neuropsychological measures may be implemented in this screening process. Two new screening tools with documented sensitivity to cognitive impairment in bipolar disorder may be particularly feasible: (i) the self‐report measure Cognitive Complaints in Bipolar Disorder Rating Assessment (COBRA),21 for assessment of subjective impairment, and (ii) the brief neuropsychological tool Screen for Cognitive Impairment in Psychiatry (SCIP),22 for detection of objective cognitive impairment. An alternative approach to screen for objective cognitive impairment is to use two single neuropsychological tests that tap into different cognitive domains, such as verbal memory and executive function.16 The COBRA has been developed for bipolar disorder21; using a cut‐off of >14 on this scale—which indicates moderate to severe self‐reported difficulties—provides adequate specificity (74%) but somewhat suboptimal sensitivity (68%) for objective cognitive impairments. The optimal cut‐offs on the SCIP for cognitive impairment have been investigated in several studies.18, 23, 24 In the context of pre‐screening patients for cognition trials, we suggest a SCIP cut‐off of <75 (for details, see section 3.2).

What is a reasonable threshold for cognitive impairment?

There is no consensus definition of “clinically significant” cognitive impairment in terms of standard deviations (SDs) for neuropsychological test performance from the mean of a normative group.25 These definitions therefore vary across studies of neuropsychiatric patients, with some using a cut‐off corresponding to performance scores ≥2 SD below the normative mean, and others using less conservative cut‐offs of ≥1.5 or ≥1 SD under the norm.25, 26, 27 It is also unclear whether such thresholds should be based on single or several neuropsychological tests.25 Notably, it is relatively common that cognitively intact people, defined by their overall composite score, perform 1 SD under the norm on a single neuropsychological test.25 It thus seems necessary to apply such a cognitive impairment threshold to two or more cognitive tests. In a secondary analysis of data from the erythropoietin (EPO) trials, participants with a score of ≥1 SD below the norm on at least two cognitive tests had substantially greater chances of treatment efficacy than unimpaired individuals.15 Notably, this did not represent simple regression towards the mean, since such influence of baseline deficits was not observed in the placebo group.15 However, application of a threshold of ≥1 SD for cognitive impairment to a global cognition measure based on a broad neuropsychological assessment is likely to be too conservative, since only 12%‐40% patients display global impairments.6, 7, 8 It would therefore be advisable to employ a less conservative threshold on a global cognition measure, such as performance ≥0.5 SD below the norm. Indeed, it has been shown in a receiving operator characteristic (ROC) analysis of the SCIP that using a cut‐off of <75—corresponding to ≥0.5 SD under the mean of healthy age‐matched controls—provides adequate sensitivity and specificity (81% and 76%, respectively) for detection of cognitive impairments in bipolar disorder.18 Nevertheless, the selection of a cut‐off should be based on a balance between the desired specificity for cognitive impairment and practical recruitment considerations in the individual trial. While a high cut‐off optimizes specificity and increases statistical power, it limits inclusion of trial participants and generalizability of the findings.

Why is consideration of cognitive reserve important?

If logistically feasible, it may be worth establishing cognitive impairment with reference to patients’ premorbid cognitive reserve, as reflected by their premorbid IQ, educational level and occupational attainment.28 Cognitive reserve reflects the capacity of the brain to tolerate neuropathology, minimize symptom manifestations and slow down the clinical presentation of neurocognitive decline.29, 30 Consideration of cognitive reserve is relevant since neuropsychological tests alone may be less sensitive to cognitive change in patients with substantial above‐normal premorbid function. Useful IQ or premorbid IQ tests include the National Adult Reading Test (or locally equivalent tests), the Advanced Clinical Solutions (ACS) Test of Premorbid Functioning, the Wide Range Achievement Reading Recognition Test and the 2‐subtest IQ from the Wechsler Abbreviated Scale of Intelligence. Educational attainment may be a suboptimal measure of premorbid cognitive reserve and for matching patients and controls, since patients with bipolar disorder tend to complete fewer years of education than controls despite comparable IQ levels.31 Obtaining estimates of premorbid IQ would also enable detection of participants with intellectual disability (i.e., IQ < 70‐75) whose poor cognitive performance is unlikely to represent illness‐associated impairment and thus be more resistant to cognitive‐enhancing treatments. Such assessments could inform decisions on (i) whether individuals with remarkably high cognitive reserve but only small impairments (in comparison with norms) should be allowed in the trial and (ii) whether individuals with very low cognitive reserve (e.g., IQ < 80) should be excluded as in recent studies.13, 32

Which criteria should be used to select trial participants?

There is great disparity in the inclusion and exclusion criteria across RCTs targeting cognition in bipolar disorder. The task force therefore carefully evaluated inclusion and exclusion criteria for cognition trials, taking into account study validity, generalizability, chances for signal detection, and recruitment feasibility. In addition to improving indication enrichment by screening for objective cognitive impairment, the following recommendations may also significantly aid the methodology of future trials.

A particular challenge for cognition trials in bipolar disorder is the influence of mania and depression symptoms, and their episodic nature, on cognitive function.33 Recommendations on whether or not to allow for mood symptoms depend on the particular study aim and type of treatment under investigation. For trials with cognition as a primary outcome, patients should generally be partially remitted or euthymic, depending on the profile of the drug effects. Specifically, trials investigating candidate treatments with no known efficacy on mood should allow for subsyndromal symptoms in the interest of recruitment feasibility and generalizability of the results, since persistent subsyndromal symptoms (particularly of depression) are common in periods of mood stability. In contrast, trials investigating medications with well‐documented effects on mood should preferentially include euthymic patients to rule out “pseudo‐specificity” (i.e., nonspecific cognitive improvement due to treatment‐related decrease in mood symptoms). We propose euthymia defined as Hamilton Depression Rating Scale (HDRS) or Montgomery–Åsberg Depression Rating Scale (MADRS) and Young Mania Rating Scale (YMRS) scores ≤7. Some studies have included prospective verification of euthymia, where patients are monitored for up to 1 month prior to assessment.34, 35 If such an approach is logistically feasible, it offers a more rigorous method of ensuring the euthymic period is stable. However, it should be noted that the proposed distinction in inclusion criteria between euthymic and partly remitted patients may be somewhat artificial, since patients who are euthymic at inclusion may develop mood symptoms during the trial period. Further, euthymic patients included in a study of drugs with known antidepressant effects could run a higher risk of switching to mania than patients with subsyndromal depressive symptoms. Finally, for trials with mood symptoms as the primary outcome and cognition as a secondary outcome, patients will be presenting with moderate to severe mood symptoms. In these trials, potential treatment‐related cognitive improvement can provide only preliminary evidence for efficacy on cognition and necessitates replication in a study with cognition as the primary outcome.

Concomitant medications are a confounder in cognition trials due to their effects on cognition but must be allowed for ethical and clinical reasons to maintain mood stability and to ensure the representativeness of the patient sample (and thus generalizability of the findings). The cognitive side effects of commonly prescribed medications for mood symptoms and anxiety are not fully elucidated. Nevertheless, benzodiazepines as well as some antipsychotics, mood stabilizers and antidepressants have documented cognitive side effects, partially due to their anti‐histaminergic, anti‐dopaminergic, and anticholinergic actions.4, 5 It is therefore advisable to avoid high doses of concomitant antipsychotics and anticholinergic medications whenever possible. Decreasing doses of medication should nevertheless be traded off against the risk of relapse of mood symptoms, which may invalidate results of the trial. It would also be advisable to taper benzodiazepines to a maximum dose equivalent to ≤22.5 mg oxazepam or ≤7.5 mg diazepam per day (cut‐offs for doses with estimated limited cognitive side effects) and to restrict use of benzodiazepine and other hypnotics for at least 6 hours prior to cognitive testing. Serum lithium dose should be monitored and be kept within the therapeutic range throughout the trial to avoid confounding cognitive side effects of lithium toxicity. In general, patients should be on stable medication for at least 2‐6 weeks before trial start, depending on whether the treatment change is merely a dose adjustment or commencement of a new treatment. Concomitant medication should be carefully recorded and, if possible, kept stable throughout trial participation. If a candidate treatment is found to improve cognition, post hoc analysis should include adjustment for concomitant medications to determine whether the significant effects prevail (and are thus independent of non‐study medication). Finally, electroconvulsive therapy (ECT) in the past 6 months should be an exclusion criterion since cognitive impairments may in such cases be ECT‐ rather than illness‐associated.

There is a need for consensus on how to tackle medical and psychiatric comorbidity in cognition trials. In general, patients should be excluded if they have a history of moderate or more severe brain injury, neurological disease, current uncontrolled thyroid condition, unstable medical illness, current or recent (i.e., within the past 1‐3 months) alcohol and substance use disorders, or intellectual disability, since these are likely to hamper the chances of treatment efficacy on cognition. Patients should also be excluded if they have a current comorbid diagnosis of attention deficit hyperactivity disorder verified during the diagnostic screening interview, since the investigational treatment aims target cognitive deficits associated more specifically with bipolar disorder. Comorbid anxiety and sleep disorders may exacerbate patients’ cognitive impairment. However, they should not be exclusion criteria since this would impede recruitment and generalizability, given the high comorbidity between bipolar disorder and anxiety disorders 36 and high rates of circadian rhythm disturbance and abnormal sleep in bipolar disorder.37 However, careful recording of anxiety symptoms and sleep quality during the trial is advisable as this enables investigation of their potential influence on treatment effects on cognition in exploratory post hoc analyses. Finally, patients recruited into cognition trials should have mastery of the language in which the neuropsychological tests are validated/administered, as this is necessary for obtaining valid cognitive assessments during the trial.

How should efficacy on cognition be assessed?

There is a lack of consensus on which cognition measures to define as primary outcomes and on a priori hierarchy between cognition measures in most trials in bipolar disorder.1 In some instances, the choice of the cognitive outcome measure may be driven by the specific brain or cognitive mechanisms that might be targeted by a particular agent or treatment under study. For example, if an agent is proposed to act on a neurotransmitter or neural system that subserves a specific cognitive ability, it may be desirable to select the primary cognitive outcome based on this demonstrated link between brain and behavior.38, 39 In other instances, there may not be a clear specific cognitive target and therefore a key recommendation is to select one primary broad cognitive composite score spanning sustained attention, verbal memory, and executive functions that is sensitive to the cognitive deficits in bipolar disorder, such as the ISBD ‐ Battery for Assessment of Neurocognition (ISBD‐BANC) composite10 or the “speed of complex cognitive processing” composite15, 40 which are implemented in completed and ongoing cognition trials (ClinicalTrial.gov ID: NCT01470781).15, 16, 40 Given the cognitive heterogeneity in bipolar disorder, a broad cognitive composite score may—by summarizing the change across several domains—be a more robust outcome than a single cognition test as it can pick up small cumulative treatment effects across several cognitive domains.9 The recommended procedure for deriving such a common metric for cognition is to calculate the composite score as the mean of the individual component z‐scores for attention, memory, and executive function tests standardized against a healthy normative sample of average intelligence. Ideally, the various individual measures would have been co‐normed, i.e., the norms would have been collected from the same normative population, and demographic‐corrected norms (e.g., age, IQ/education and gender) would be available. Alternative strategies may be considered if there is no available norm group for the particular tests, such as use of meta‐norms, or calculating z‐scores by referencing the scores to the study baseline or control group performance.

The use of a specific set of neuropsychological tests for the primary cognition outcome across all future cognition trials would maximize comparability between studies. However, the task force is mindful of the international research community and that not all countries have the exact tests of the ISBD‐BANC or the “speed of complex cognitive processing” composites available. The task force therefore recommends that trials match the neuropsychological tests in their primary cognition outcome as closely as possible to the tests in the ISBD‐BANC or “speed of complex cognitive processing” composites. Broadly equivalent tests probing attention, verbal memory and executive function would thus be adequate. While not developed for bipolar disorder, the MATRICS Consensus Cognitive Battery (MCCB) is translated into >50 languages and normed in many countries and it has been validated in bipolar disorder in several independent samples.41, 42, 43 Given that the ISBD‐BANC includes many of the MCCB subtests, there should be a reasonable ability for trials to include most tests of the ISBD‐BANC. A degree of flexibility is also advisable in cases where prior studies have shown benefits of a compound on specific cognitive tests; in such cases, it would be meaningful to include these tests in the primary cognition outcome in subsequent replication trials. Important next steps that are prerequisites for developing consensus on a specific test battery would be: (i) a factor‐analytic study of cognitive impairments in bipolar disorder based on existing cognitive data sets, and (ii) validating the factor structure against external validators such as functional capacity.

Individual cognition outcomes of interest for bipolar disorder and for a particular candidate intervention should be included as secondary outcomes (together with a functional outcome/co‐primary measure; see later). For example, social cognition (such as facial expression recognition) may be a clinically meaningful secondary outcome, given the often persistent and debilitating social cognition deficits in bipolar disorder.44, 45 Finally, individual cognitive tests comprising the primary composite cognitive outcome should be specified separately as tertiary explorative outcomes. This enables exploratory analyses of the profile of the treatment effects to assess which tests (i) respond most to the particular treatment and/or (ii) show greatest sensitivity to treatment in particular subgroups. Such insights can provide hypothesis‐generating evidence which—if confirmed in additional trials with these particular measures as primary outcomes—could eventually pave the way for more personalized treatments of cognitive impairments.

What is a “clinically relevant” cognitive improvement?

The lack of consensus on what defines a “clinically relevant” cognitive change is problematic for two reasons. First, such information is critical for sample size estimations to ensure adequate statistical power for detection of efficacy on the primary outcome. Second, this information is relevant for extrapolating the clinical importance of potential treatment effects on cognition. If the treatment goal were to correct a deficit, we would anticipate effect sizes equivalent to the known deficits. However, smaller treatment effects may also be clinically meaningful—such as an improvement that is half‐way towards the normal function in healthy age‐matched individuals.46

The available evidence so far indicates that it is probably unrealistic to expect large effect sizes for treatment effects on cognition. A key reason is the well‐documented learning or practice effect with repeated testing (i.e., nonspecific improvement across all participants), which reduces the difference in cognitive change between active and control groups and thus the magnitude of the treatment effects. To optimize the signal to noise ratio for cognitive change, trials should implement parallel equivalent forms of the neuropsychological tests for the pre‐ and post‐treatment assessments if available. However, since learning effects are almost impossible to eliminate, it is of critical importance to estimate the “clinically relevant” effect on cognition with reference to the cognitive change in the control group rather than baseline performance. Given the issue of learning effects, small to medium effect sizes for between‐group differential change in cognition (such as Cohen's d of 0.2‐0.4) may arguably represent clinically relevant treatment effects. We recommend that all trials also include measures of socio‐occupational function so that the impact of improvement in cognition on day‐to‐day functioning can be assessed.

How should functional implications be evaluated?

A major criticism of cognition trials has been that it is unclear from change in neuropsychological performance alone whether the treatments actually have real‐life benefits for the patients in terms of work and social function. It is therefore critical in cognition trials to gain insight into the functional implications of potential treatment efficacy on cognition. Indeed, the Food and Drug Administration (FDA) stipulates that cognition trials in schizophrenia must provide evidence for functional benefits of cognitive improvement.47 Given this legacy from schizophrenia research, cognition trials in bipolar disorder will need to make decisions about how to define their key secondary (if not co‐primary) measure of functional change. Inclusion of a functional measure would also enable assessment of the potential interaction between functional status and the cognition benefits of an intervention, which would be interesting in light of preliminary evidence for greater treatment‐related cognitive improvement in high‐ vs low‐functioning patients,48

Two promising existing tools are the observer‐based measure, the Functional Assessment Short Test (FAST),49 and the performance‐based test, the Brief University of California, San Diego (UCSD) Performance‐based Skills Assessment (UPSA‐B).50 Both tools are sensitive to functional impairments in bipolar disorder,49, 50 and the FAST has shown sensitivity to treatment effects.13 The FAST assesses aspects of everyday functioning through self‐report and clinical observations, while the UPSA is a performance‐based measure of skills (e.g., handling finances and planning shopping) associated with functional outcomes. However, these functional measures are not exempt of limitations. While the UPSA‐B is available in several languages, it retains some transcultural problems. A more general limitation of both measures is that they do not directly assess patients’ real‐world function. New virtual reality tools are therefore being developed, including the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). This tool presents participants with realistic simulated environments to recreate daily living activities (e.g., cooking, using public transportation and food shopping) and seems to be a valid and sensitive assay of functional capacity.51 Taken together, the FAST, UPSA‐B and VRFCAT all seem adequate measures for tracking changes in functional capacity associated with cognitive improvement in bipolar disorder, although additional validation data in the disorder are currently a research priority. Additional “harder measures” of functional capacity, such as occupational and/or academic achievement, should also be assessed since some measures may correlate more with cognition or mood symptoms than with real‐world functioning.

When should pre‐ and post‐assessments be conducted?

The treatment periods in RCTs targeting cognition range from 1 to 21 weeks but are most commonly between 6 and 12 weeks.1, 16 Short study durations have the benefit of limiting the confounding effects of mood cycling, whereas longer trial durations may be more likely to produce robust and enduring efficacy on cognition and reduce potential practice effects. The optimal duration of a particular trial would depend on the presumed onset of efficacy for the particular intervention based on its putative mechanisms. Specifically, relatively pronounced effects may be seen rapidly, depending on the agent. For example, a single dose of modafinil was recently found to have acute effects on some aspects of cognition in remitted major depressive disorder.52 However, a general recommendation would be to administer pharmacological and other biological interventions for 6‐12 weeks and psychological interventions for 10‐21 weeks, depending on the specific treatment program with pre‐ and post‐treatment assessments of cognition at baseline and immediately after treatment completion (primary time for assessment of efficacy). When possible, a follow‐up assessment after 3‐6 months would be desirable. This recommendation is based on (i) the most common practice in RCTs of biological and psychological candidate treatments in mood disorders targeting mood symptoms or cognitive impairment and (ii) the assumption that the functional implications of treatment‐related increase in neuroplasticity would begin to emerge after 4‐6 weeks and presumably increase further with continued treatment.53 The follow‐up assessment 3‐6 months after treatment completion would serve to determine not only whether potential treatment‐related cognitive benefits persist long‐term, but also whether they translate into functional improvements, which are likely to occur with a time lag. An analog is the typical delay between the physical healing of a sprained ankle and a person's resumption of habitual levels of daily activity and physical exercise. It is thus likely that it takes time for objective cognitive improvement to translate into better functioning in daily life. Preliminary evidence from recent EPO trials supports this notion. Here the observed correlation between EPO‐treated patients’ objective cognitive improvement and subjective cognitive change in daily life was mediated by change in depressive symptoms in the acute treatment phase but was direct 6 weeks after trial completion.15

How should “pseudospecificity” be addressed?

Lack of control for changes in mood symptoms in the analysis of treatment‐associated cognitive change (i.e., the “pseudospecificity” issue) is generally problematic in trials of cognition in bipolar disorder as some patients may develop mood symptoms during the trial period. It is hence important to (i) adjust the analysis of treatment‐related cognitive change for symptom fluctuation by covarying for change in mood symptoms from baseline to post‐treatment at the group level and/or (ii) conduct path analysis, which can provide an estimate of whether the effects on cognition were direct or simply mediated through symptom improvement. In trials aiming to improve mood symptoms as a primary goal and include cognition as a secondary treatment target, such analyses can provide valuable hypothesis‐generating evidence for an indication for cognitive impairment that may be further explored in a subsequent trial with cognition as the primary outcome (as exemplified by the vortioxetine trials in unipolar disorder54). In addition to the primary intention‐to‐treat (ITT) strategy, exploratory post hoc analyses can also be conducted to examine whether treatment effects differ between patients who remain stable throughout the study and patients with emerging mood symptoms during the trial.

What are the methodological recommendations for specific classes of agents?

Some methodological considerations are tied to the properties of the particular candidate treatment under investigation, and therefore differ between pro‐dopaminergic drugs, atypical antipsychotics, anti‐inflammatory agents, and neuroprotective drugs. In particular, drug‐specific considerations include decisions about what clinical state patients should be in, which secondary cognition outcomes to select, whether to use an adjunctive or monotherapy design, which comparator to choose (placebo or an active treatment with no known effects on cognition), and, for adjunctive studies, which concomitant medication to allow. Monotherapy should only be used if the candidate cognition treatment under investigation has mood‐stabilizing effects for ethical reasons and to ensure generalizability of the findings. Such trials would necessitate an active comparator drug with mood‐stabilizing effects, since withholding mood‐stabilizing treatment from patients could trigger new illness episodes.

Pro‐dopaminergic drugs such as pramipexole and modafinil should be investigated in an adjunctive design given some concern about potential risk of mania switch.55, 56 Such trials should ideally restrict recruitment to euthymic patients, given the antidepressant effects of these compounds,57, 58 which would confound the interpretation of potential cognitive benefits in symptomatic patients. Alternatively, they could include depressed patients in a head‐to‐head adjunctive superiority design with a comparator without pro‐cognitive effects, although euthymic patients are better to rule out pseudospecificity. Preliminary evidence suggests that the cognitive benefits of pramipexole are restricted to strictly euthymic patients.12 It could therefore be hypothesized that the cognitive benefits of increasing prefrontal dopamine tonus with pro‐dopaminergic drugs may be confined to patients with relatively low baseline dopamine tonus (as in euthymia). Given the drug effects on dopaminergic neurotransmission, inclusion of a reward processing or emotional decision‐making cognition measure as a secondary outcome could aid mechanistic insight into the clinical effects of the treatment, as illustrated in the pramipexole trial.59

Cognition trials investigating anti‐psychotic or antidepressant drugs with efficacy on depressive symptoms, such as lurasidone and vortioxetine, should include euthymic patients to rule out pseudospecificity. Such trials may wish to include an emotional processing test as a secondary outcome to assess the mechanisms of treatment efficacy on depression. For antidepressants, the trial design should be adjunctive and mood state carefully monitored as antidepressant monotherapy is not usually recommended in bipolar disorder, given concern about potential mania switch in response to vortioxetine.60 In contrast, trials investigating anti‐inflammatory agents or neuroprotective drugs (such as EPO) with no documented effects on bipolar depression would arguably benefit from expanding the inclusion criteria to partial remission (i.e., allowing for more subsyndromal depressive symptoms) in the interest of recruitment feasibility and generalizability of the findings. For the same reason, the trial design should be adjunctive and involve a placebo control arm. These recommendations (i.e., inclusion of partially remitted patients and using an adjunctive design with a control group) also apply to psychological interventions for cognitive impairment such as cognitive and functional remediation.

How should statistical issues around missing data be handled?

Strategies for handling missing data differ between cognition trials despite the general recognition that intention‐to‐treat (ITT) analyses should be implemented to prevent bias caused by participant dropout. However, the definition of ITT is vague and involves several approaches for handling missing data in longitudinal trials. Last observation carried forward (LOCF) is one approach that involves imputation of the missing values with the last observed value, assuming that the outcomes would be unchanged from the last observed value. Although LOCF minimizes the number of participants eliminated from the analysis, it has been criticized for underestimating the variability in the data, particularly if there is a large amount of missing data. More feasible ways to handle missing data with repeated assessments are multiple imputation or mixed models, which are more robust and take account of missing values, including whether data are missing at random, and inter‐individual changes over time. These statistical methods are increasingly used in clinical research because of their availability in many statistical software packages.

In cognition trials, there will often be only two cognition data points in the primary analysis (i.e., baseline and post‐treatment) to minimize learning effects. If the primary outcome is cognitive change from baseline, inclusion of data from patients with only baseline assessments will provide very limited information in the statistical model, adding only to the estimation of the between‐subject variance in the data. The results of the analysis will therefore be highly similar to those of a modified ITT analysis, in which data are analyzed for patients with both data points. However, mixed models or multiple imputation procedures are highly feasible for the (secondary) analysis of long‐term cognitive change that includes more data points. Nevertheless, there are no universal standards, as the most appropriate method for handling missing data depends on the goals, endpoints and context of the particular trial.

FUTURE PERSPECTIVES

Need for insight into the neurobiological targets of cognitive enhancement

The FDA Critical Path Initiative has highlighted neuroimaging in human populations as a key tool to accelerate the screening and selection of new candidate central nervous system (CNS) treatments.61 For cognition trials, assessment of neuroimaging biomarkers will increase insight into the neurobiology of cognitive improvement and thereby lead to identification of common biomarkers of pro‐cognitive interventions. Specifically, the application of neuroimaging in cognition treatment discovery carries the potential to identify early change in key neuronal networks that predicts subsequent cognitive improvement. Detection of such neuro‐circuitry target engagement and dose−response findings could guide the development of new mechanism compounds for cognitive impairment as a conceptually important middle step between investigation of such treatments in preclinical models and large‐scale clinical phase III trials. This is in line with the National Institutes of Health (NIH) statement around target engagement as a potential regulatory pathway.62 Electrophysiological and psychophysiological methods may also aid insight into the neurobiological underpinnings of cognitive impairments and neuronal target engagement. In particular, quantitative electroencephalography (EEG) seems to provide sensitive biomarkers for cognitive impairments and treatment‐related cognitive change in neurological disorders63, 64 and would be interesting to include in trials in bipolar disorder.

Emerging evidence points to aberrant activity in the dorsal prefrontal cortex (PFC) and related circuitries as well as failure to suppress default mode network (DMN) activity as common neuronal correlates of cognitive impairment across distinct neuropsychiatric disorders.65, 66, 67, 68, 69, 70, 71, 72 In a series of randomized placebo‐controlled functional magnetic resonance imaging (fMRI) studies, a single dose of EPO and long‐term EPO treatment produced target engagement in the dorsal PFC and DMN during strategic encoding and working memory across healthy individuals and patients with affective disorders, and this activity change correlated with cognitive improvement.73, 74, 75 Meta‐analytic findings also point to an increase in dorsal prefrontal activity as the most reliable marker of cognitive improvement in response to cognitive remediation interventions in schizophrenia.76 There is also emerging evidence for structural hippocampal volume increase as a key neurobiological target for cognition treatments. Indeed, structural MRI assessments of patients with affective disorders in the RCTs of long‐term EPO vs saline treatment revealed increased subregional volume in the left hippocampus in EPO‐treated patients, which correlated with EPO‐related verbal memory improvement.77 Similarly, MRI assessment of an RCT of cognitive remediation in schizophrenia showed treatment‐associated preservation of left hippocampal gray matter volume over 2 years, which mediated the improvement of cognition.78 Taken together, this emerging neuroimaging evidence points to PFC and DMN target engagement as well as hippocampal volume increase as potential biomarkers for pro‐cognitive effects of distinct pharmacological and psychological interventions across several psychiatric disorders—possibilities that warrant investigation in future trials.

Finally, it would also be of key interest to include assessments of potential blood‐based biomarkers for cognitive improvements in future trials, given emerging evidence for a putative role of inflammation and oxidative stress in patients’ cognitive deficits.79 Specifically, assessments of changes in such biomarkers early in the course of treatment can lead to identification of markers that predict subsequent treatment efficacy on cognition.

Targeting of treatments according to patients’ illness stage

There is increasing evidence for clinical progression in bipolar disorder80 and growing consensus that bipolar disorder involves “clinical staging,” a progression from prodromal (at‐risk) stages to more severe and resistant presentations.81, 82 In line with the staging model, interventions seem to have differential efficacy depending on patients’ illness stage and interventions should therefore be tied to the particular illness stage.81, 82 Extrapolating from this, inclusion of a heterogeneous group of patients at various stages of their illness could potentially preclude treatment efficacy on cognition. This raises questions about whether future cognition trials should stratify patients for their illness stage and, if so, which stages to target with cognition treatment.

In the EPO trials, there was a small but significant increase in patients’ chances of achieving treatment efficacy on cognition with greater illness chronicity (16% for every year of illness).14 If the finding is generalizable, cognition treatments may be more beneficial at later illness stages that are accompanied by greater cognitive and functional disability. On the other hand, cognitive impairment at later stages may be more treatment‐resistant and be accompanied by greater functional disability, which could have been prevented with an early intervention. Indeed, preliminary evidence from a schizophrenia trial suggests that cognitive remediation is more effective in younger, less functionally impaired patients who use less antipsychotic medication.83 Given the paucity of evidence for stage‐specific effects on treatment efficacy on cognition in bipolar disorder, this question should be addressed in future cognition trials.

Potential for a multimodal treatment approach

The combination of pharmacological and non‐pharmacological (i.e., psychological or neurostimulation) interventions is likely to produce synergistic effects on brain function that translate into more robust efficacy on cognition than either treatment modality alone. Importantly, the translation of treatment‐related cognitive improvement into greater functional capacity in chronically ill patients may be more difficult due to fewer environmental opportunities to apply regained cognitive skills. In this respect, it may be necessary in chronically ill patients to apply a multi‐modality approach. There is currently a paucity of evidence for synergistic effects of multimodal interventions. A recent preclinical study revealed that the EPO‐associated increase in hippocampal pyramidal neurons and oligodendrocytes was only maintained long‐term (≥6 months) in mice that also received continuous cognitive challenges.83 This finding is consistent with the demonstration in a cognition trial in unipolar depression that working patients displayed greater cognitive benefits of vortioxetine than those who were unemployed.48 Taken together, these observations are suggestive of stronger treatment effects on neuroplasticity and cognition in individuals who receive continuous cognitive challenges. Multimodal treatment approaches should therefore be considered a key next step for trials that demonstrate cognitive improvement in response to unimodal interventions.

CONCLUSION

This guidance paper from the ISBD Targeting Cognition Task Force provides the first broad consensus‐based recommendations for future cognition trials in bipolar disorder, which may help overcome some of the methodological challenges in the field. The recommendations are summarized in Table 1. Key recommendations are to enrich trials with cognitively impaired patients by screening them with a brief neuropsychological test battery, to generally select a broad cognitive composite score as the primary outcome and a functional measure and key cognitive tests as secondary outcomes, to include partially or fully remitted patients, to exclude patients with current substance or alcohol use disorder, to disallow certain non‐study medications if possible and to keep all other concomitant medication stable. While the ideal design will vary to some degree depending on the mechanism being targeted and the hypothesized onset of effects, most recommendations are generally applicable for cognition trials in bipolar disorder. Following these recommendations will increase the internal validity of cognition trials by limiting confounding factors and the external validity by ensuring generalizability of the findings and assessment of their translation to real‐world outcomes. Finally, neuroimaging and electrophysiological assessments in future trials may identify the neurobiological targets for pro‐cognitive interventions that can aid future drug discovery strategies. Studies are also warranted to explore the potential synergistic effects of multimodal treatment approaches. Implementing the recommendations is likely to advance our understanding of which cognition treatments work, for whom and why. Specifically, optimizing the trial design and methodology across trials so findings become more replicable and comparable will advance the understanding of which treatments do—or do not—improve cognition. While the field is still in its infancy, the assessment of whether cognition treatments should be targeted to particular illness stages will clarify for whom these treatments have particular benefits. Finally, neuroimaging and neurophysiological assessments will clarify why certain treatments work by elucidating their neurobiological mechanisms.

Table 1

Quick guide with a summary of the International Society for Bipolar Disorders (ISBD) Targeting Cognition Task Force recommendations

Methodological recommendations for cognition trials in bipolar disorder by the International Society for Bipolar Disorders Targeting Cognition Task Force: quick guide
How can we enrich trials with cognitively impaired patients?
Assess subjective cognitive difficulties and functional capacity
Screen for cognitive impairment with a brief, feasible neuropsychological test battery
Two new screening tools for cognitive impairment may be particularly feasible: the COBRA (self‐report measure) and the SCIP (brief neuropsychological test battery)
What is a feasible threshold for cognitive impairment?
≥ 0.5 SD below the normative mean on a short neuropsychological screening test or, alternatively, ≥1 SD below the mean on at least two single neuropsychological tests
If logistically feasible, cognitive impairment may be established with reference to general IQ
Which criteria should be used to select trial participants?
Generally include partially or fully remitted patients in trials where cognition is the primary outcome to minimize “pseudospecificity” issues
Exclude patients with a history of moderate or severe brain injury, neurological disease, current uncontrolled thyroid condition, unstable medical illness, current or recent alcohol and substance use disorders, intellectual disability, or ECT within the past 6 months
Allow concomitant medications. These should be carefully recorded and, if possible, kept stable
In possible, disallow certain medications (high‐dose antipsychotics and anticholinergic medications)
Taper benzodiazepines to a maximum dose equivalent to 22.5 mg oxazepam/7.5 mg diazepam per day and restrict use of benzodiazepine and other hypnotics 6 hours prior to cognitive testing
Keep serum lithium within the therapeutic range
How should efficacy on cognition be assessed?
In general, select a broad cognitive composite score spanning sustained attention, verbal memory, and executive functions as the primary outcome
Use tests that are broadly equivalent to those included in the ISBD‐BANC
Select key cognitive tests of interest and a functional measure as secondary outcomes
What is a “clinically relevant” cognitive improvement?
Since learning effects are almost impossible to eliminate, a “clinically relevant” effect on cognition should be estimated with reference to the cognitive change in the control group
Given the issue with learning effects (which reduce the difference between the active and control groups), small to medium effect sizes for treatment effects may be considered clinically meaningful
How should functional implications be evaluated?
The FAST, UPSA‐B and VRFCAT are among the best measures to date for tracking changes in functional capacity associated with cognitive improvement in bipolar disorder
When should pre‐ and post‐assessments be conducted?
In general, administer biological interventions for 6‐12 weeks and psychological interventions for 10‐21 weeks with pre‐ and post‐treatment assessments of cognition at baseline and immediately after treatment completion. If feasible, perform follow‐up assessments after 3‐6 months
How should “pseudospecificity” be addressed?
Adjust the statistical analysis of cognitive change for symptom fluctuation and conduct path analysis
What are the methodological recommendations for specific classes of agents?
Monotherapy should only be used if the candidate treatment has mood‐stabilizing effects for ethical reasons and to ensure generalizability. Use an active comparator drug with mood‐stabilizing effects.
Cognition trials investigating anti‐psychotic, pro‐dopaminergic or antidepressant drugs with efficacy on depressive symptoms should ideally include euthymic patients to rule out pseudospecificity. Alternatively, they can include depressed patients in a head‐to‐head adjunctive superiority design with a comparator without pro‐cognitive effects
Trials investigating anti‐inflammatory or neuroprotective drugs with limited effects on mood would benefit from expanding the inclusion criteria to partial remission in the interest of recruitment feasibility and generalizability. Use an adjunctive study design with a placebo control
How should statistical issues around missing data be handled?
Intention‐to‐treat analyses should be implemented to prevent bias caused by dropout
Feasible ways to handle missing data with repeated assessments after treatment start are multiple imputation or mixed models

COBRA, Cognitive Complaints in Bipolar Disorder Rating Assessment; ECT, electroconvulsive therapy; FAST, Functional Assessment Short Test; SCIP, Screen for Cognitive Impairment in Psychiatry; SD, standard deviation; UPSA‐B, Brief UCSD Performance‐based Skills Assessment; VRFCAT, Virtual Reality Functional Capacity Assessment Tool.

DISCLOSURES

KWM reports having received consultancy fees from Lundbeck and Allergan. KEB has served on advisory boards for Sunovion, Sumitomo Dainippon, Takeda‐Lundbeck, and Neuralstem. AMA has received funding for research projects and/or honoraria as a consultant or speaker for the following companies and institutions: Otsuka, Pfizer, AstraZeneca, Bristol‐Myers Siquibb, Lundbeck, Brain and Behaviour Foundation (NARSAD Independent Investigator), the Spanish Ministry of Economy and Competitiveness and Instituto de Salud Carlos III. CRB has been a consultant or advisor for Boehringer Ingelheim, Lundbeck, Otsuka, and Takeda and has received grant money from Pfizer and Takeda. RSM is a consultant for and/or receives honorarium for speaking from Sunovian, Johnson & Johnson, Otsuka, Lundbeck, Pfizer, Allergan, BMS, Shire, and Purdue. CLJ has received grants from COLCIENCIAS, Universidad de Antioquia‐CODI and NIMH. He has served as a consultant, advisor or Continuing Medical Education (CME) speaker for the following companies: AstraZeneca, Eli Lilly, Glaxo‐SmithKline, Janssen, Lundbeck and Pfizer. AS has been a consultant for or received honoraria from Allergan, BMS, Lundbeck, Otsuka, and Sunovion. IJT has received consultant fees from Lundbeck and Sumitomo Dainippon. RJP uses software for research at no cost from Scientific Brain Training Pro. TS has received honoraria for advisory board, consultations, and/or speaker’s role from Dainippon Sumitomo Pharmaceutical, Meiji Seika Pharma, Novartis, Otsuka Pharmaceutical and Takeda. LNY has been on speaker/advisory boards for, or has received research grants from, Alkermes, Allergan, AstraZeneca, Bristol Myers Squibb, CANMAT, CIHR, Dainippon Sumitomo Pharma, Janssen, Lundbeck, Otsuka, Sunovion, and Teva. AHY is employed by King's College London; Honorary Consultant SLaM (NHS UK). He has received paid lectures and is on advisory boards for all major pharmaceutical companies with drugs used in affective and related disorders. AHY has no shareholdings in pharmaceutical companies. He has conducted investigator‐initiated studies from AZ, Eli Lilly and Lundbeck. LVK has within the preceding 3 years been a consultant for Lundbeck, AstraZenica and Sunovion. EV has received grants, CME‐related honoraria, or consulting fees from Alexza, Almirall, AstraZeneca, Bristol‐Myers Squibb, Cephalon, Eli Lilly, Ferrer, ForestResearch Institute, Gedeon Richter, GlaxoSmith‐Kline, Janssen, Janssen‐Cilag, Jazz, Johnson & Johnson, Lundbeck, Merch, Novartis, Organon, Otsuka, Pfizer, Pierre‐Fabre, Qualigen, Roche, Sanofi‐Aventis, Schering, Plough, Servier, Shire, Solvay, Takeda, Teva, CIBERSAM, the Seventh European Framework Programme (ENBREC), the Stanley Medical Research Institute, United Biosource Cooperation, and Wyeth. All other authors report no biomedical financial interests or potential conflicts of interest.