Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far?

In light of the number of studies conducted to examine the treatment of cognitive impairment associated with schizophrenia (CIAS), we critically reviewed recent CIAS trials. Trials were identified through searches of the website "www.clinicaltrials.gov" using the terms "schizophrenia AND cognition," "schizophrenia AND neurocognition," "schizophrenia AND neurocognitive tests," "schizophrenia AND MATRICS," "schizophrenia AND MCCB," "schizophrenia AND BACS," "schizophrenia AND COGSTATE," and "schizophrenia AND CANTAB" and "first-episode schizophrenia AND cognition." The cutoff date was 20 April 2011. Included trials were conducted in people with schizophrenia, the effects on cognition were either a primary or secondary outcome, and the effect of a pharmacologically active substance was examined. Drug challenge, pharmacokinetic, pharmacodynamic, or prodrome of psychosis studies were excluded. We identified 118 trials, with 62% using an add-on parallel group design. The large majority of completed trials were underpowered to detect moderate effect sizes, had ≤8 weeks duration, and were performed in samples of participants with chronic stable schizophrenia. The ongoing add-on trials are longer, have larger sample sizes (with a number of them being adequately powered to detect moderate effect sizes), and are more likely to use a widely accepted standardized cognitive battery (eg, the MATRICS Consensus Cognitive Battery) and MATRICS guidelines. Ongoing studies performed in subjects with recent onset schizophrenia may help elucidate which subjects are most likely to show an effect in cognition. New insights into the demands of CIAS trial design and methodology may help increase the probability of identifying treatments with beneficial effect on cognitive impairment in schizophrenia.

Introduction

Neurocognitive impairments are a core component of schizophrenia. They include significant deficits in memory, attention, working memory, problem solving, processing speed, and social cognition.1–3 These impairments have been shown to be associated with various impaired functional outcomes.4,5 The severity of cognitive impairment predicts poorer treatment adherence6,7 and increased relapse risk in first-episode patients.8 Furthermore, imaging studies have demonstrated relationships between cognitive deficits and structural or functional brain abnormalities.9–13 Due to the clinical relevance of neurocognitive impairment in schizophrenia and in particular its relationship to poor functional outcomes, development of new therapies to enhance cognition in schizophrenia remains one of the most pressing challenges in psychopharmacology.1,14–16

The joint academic, government, and industry Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative facilitated the development of guidelines for the design of clinical trials of drugs for neurocognitive impairment in schizophrenia17 and created the MATRICS Consensus Cognitive Battery (MCCB)18,19 for measuring cognitive treatment outcomes in schizophrenia. Based on experience from early studies that used the MATRICS clinical trial guidelines, the MATRICS investigators recommended revision of the inclusion criteria to enhance recruitment while maintaining sufficient methodological rigor.20 The proposed revisions relaxed the symptom inclusion criteria for hallucinations and delusions, removed the negative symptom criterion, and revised the antipsychotic medication inclusion criterion to include first generation antipsychotics in the context of no concomitant anticholinergic agents and minimal extrapyramidal symptoms. Antipsychotic polypharmacy is now allowed in the absence of pertinent pharmacodynamic/pharmacokinetic considerations.

In view of the activities related to the treatment of cognitive impairment associated with schizophrenia (CIAS), and in particular, the number of clinical trials involving pharmacological treatments of CIAS, we critically reviewed recent CIAS trials to answer the following questions: (a) What has been learned so far? (b) Which factors may contribute to negative study results? (c) What are we likely to learn from ongoing studies? and (d) How may these lessons help shape future trials?

Methods

To identify trials for inclusion in the analysis, we performed a search of the website “www.clinicaltrials.gov.” Trial registration on this trial registry started in 2000 for National Institute of Health grants and in 2002 for industry-sponsored trials. We used the following search terms “schizophrenia AND cognition,” “schizophrenia AND neurocognition,” “schizophrenia AND neurocognitive tests,” “schizophrenia AND MATRICS,” “schizophrenia AND MCCB,” “schizophrenia AND BACS,” “schizophrenia AND COGSTATE,” and “schizophrenia AND CANTAB” and “first-episode schizophrenia AND cognition.” The cutoff date for these searches was April 20, 2011. Trials identified through this initial search were individually screened, and ones fulfilling the following criteria were included in the analysis

Conducted in people with schizophrenia

The effects on cognition were either a primary or secondary outcome

The effect of a pharmacologically active substance was examined (as a monotherapy, add-on therapy, or in combination with other nonpharmacological therapeutic method, eg, cognitive remediation)

Not a drug challenge, pharmacokinetic, pharmacodynamic, or prodrome of psychosis study

Identified trials were grouped into completed ongoing and terminated trials and then subsequently classified by the following study designs: (1) trials using add-on, placebo-controlled parallel group design and (2) trials using other designs (crossover open-label monotherapy parallel-group and monotherapy single group, post switch open-label design).

For each trial included in the analysis, the following information was manually retrieved from the trial description at www.clinicaltrials.gov and tabulated

Pharmacological agent under study

Putative mechanism of action

Trial NCT registration number

Sponsor

Start date

Current status of the trial

Indication

Dose

Participant population characteristics as per inclusion and exclusion criteria

Baseline cognitive impairment as an inclusion criterion and its definition (where applicable)

Study design

Biomarkers examined in the study (if applicable)

Other outcome variables

For each completed trial, information on results was retrieved from www.clinicaltrials.gov, and supplemental searches were performed on PubMed, Google, and Google Scholar to identify additional results available in the public domain (eg, publications, published abstracts, or press releases). For the completed trials with available results, a summary of those results along with age-related variables (mean age, years since diagnosis and age at onset of illness) and baseline cognitive test scores were presented. For terminated trials, a brief description of the reason(s) for termination was also extracted based on available information. The results were displayed in tabular format and summarized using descriptive statistics where appropriate. For double-blind add-on trials, we also included information on whether the study had sufficient statistical power (beta = .80) to identify true treatment differences based upon the following assumptions: 2-tailed alpha of .05, test-retest of the primary outcome of intraclass correlation (ICC) = .90 (consistent with the MCCB composite score in multisite studies)21 and true effect sizes of d = 0.5 (medium effect) and d = 0.8 (large effect), which resulted in observed effect sizes of 0.47 and 0.76, respectively. Note that these assumptions were not strictly met in some cases, especially for studies using outcome measures with lower reliability and study designs with multiple treatment arms. We present in figure 1, the sample size requirements to achieve power of beta = .80 for various effect size estimates and test-retest reliabilities.

Fig. 1.

An illustration of sample size requirements to achieve sufficient power (beta = .80) based upon estimated effect size (cohen’s d) and test-retest reliability (intraclass correlation [ICC]).

Results

Our analysis included 118 studies that satisfied inclusion criteria.

Terminated Trials

We identified 11 terminated trials (6 add-on, placebo-controlled double-blind trials; 1 cross-sectional prospective add-on trial; 1 open-label trial; and 3 monotherapy trials) (see online supplementary table 1). The MCCB18,19 was the primary outcome measure in 3 trials; the neuropsychological assessments varied across the remainder of the terminated trials. Two trials were terminated due to difficulties with recruitment, 1 trial at the sponsor’s request due to adverse animal toxicology data, 4 trials due to a lack of any apparent clinical benefit, and the sponsor of 1 trial closed their neuroscience program. The results of the last trial are being analyzed by an academic consortium (Keefe, personal communication). For the remaining trials, the sponsor terminated the study prior to recruitment start without providing any reasons for the decision, in 1 trial, the compound was withdrawn from the market in European Union, and for 1 trial, no information was provided. We will not discuss terminated studies in any further detail in this article.

Completed and Ongoing Trials

Trial Design.

A randomized, double-blind placebo-controlled add-on design was recommended by the MATRICS panel17 and is the primary design used in CIAS trials (55.7% of all completed and 63.9% of all ongoing trials) (figure 2). Crossover or monotherapy trials were less frequent, possibly due to the concern of the utility of crossover study designs in cognition studies22 and the lack of a clear regulatory path to obtaining approval for a monotherapy that would also treat cognitive impairment in schizophrenia.20

Fig. 2.

Trial designs used in completed and ongoing trials.

Mechanisms of Action Studied.

The studies included many pharmacological treatments with diverse pharmacological mechanisms of action (MoA) (see online supplementary tables 1– 5). The following MoAs have been or are currently being examined in 2 or more studies: NMDA receptor modulation, NMDA glycine site agonism/partial agonism, NMDA glycine site antagonism, H3 antagonism, selective activation of hypothalamic regions associated with wakefulness noradrenergic receptor reuptake inhibition, acetylcholine esterase inhibitors, α7 receptors agonism/partial agonism, α4β2 nicotinic receptors partial agonism, cannabinoid receptor antagonism, D2 partial agonism + 5-HT2A antagonism, and D1/D2 agonism. Agents acting at the NMDA receptor were the most frequently examined class of agents.

Completed Trials.

Among the 61 completed trials (figure 2), 34(55.7%) used double blind, randomized, placebo-controlled add-on design; 7(11.5%) used randomized, double-blind, placebo-controlled crossover design; 3(4.9%) used add-on open-label design; and 17(27.9%) were monotherapy trials. No clear preference or consistency in the primary neurocognitive outcome measure was observed. Results of 50.0% of add-on trials and 22.0% of trials using other designs were available in the public domain.

Ongoing Trials.

Among the 57 ongoing trials, 31(68.4%) are add-on trials; 5(8.8%) are crossover trials, 7(12.3%) are open label, and 6(10.5%) are monotherapy trials. In 79.5% (N = 31) of the ongoing add-on trials and in 88.9% (N = 16) of ongoing trials using other designs, recruitment began since 2007. The MCCB18,19 (either alone or in combination with another neurocognitive assessment battery) is the primary outcome measure in 53.8% (N = 21) of ongoing add-on trials and in 38.9% (N = 7) of ongoing trials using other designs.

Characteristics of Completed and Ongoing Add-on Trials

Sample Size.

A similar distribution of sample sizes was observed for both completed trials and ongoing trials (figure 3). However, only 17.6% of completed and 35.9% of ongoing trials report a sample size that was or is anticipated to be sufficient to produce statistical power to detect a medium (d = 0.5) effect size, which requires 71 subjects per group (using 2-arm trial with drug and placebo) assuming the primary outcome measure has excellent test-retest reliability (ICC = .90) as with the MCCB composite score21 (see online supplementary tables 2 and 4).

Fig. 3.

Total sample size in completed and ongoing add-on trials.

Trial Duration.

While the trial duration was ≤8 weeks in the majority of completed trials (58.8%), there is a pattern of longer duration among ongoing trials, with 66.7% being >8 weeks long. Nevertheless, despite a moderate shift toward longer trial duration, the length of 33.3% of ongoing trials is ≤8 weeks.

Outcome Variables Used to Assess Cognitive Impairment.

Among the completed add-on trials, no clear preference or consistency in the primary outcome measures was observed (see online supplementary figure 1). No specific information about the primary outcome variable was available for 29.4% of completed add-on trials, while the MCCB18,19 (alone or in combination with other cognitive tests batteries, eg, Brief Assessment of Cognition in Schizophrenia [BACS23]) or Cambridge Neuropsychological Test Automated Battery (CANTAB24) was used in 14.7% of these trials (see online supplementary figure 1). Other neuropsychological batteries (eg, BACS23 or CANTAB24) were used in 20.6% trials and various cognitive domain-specific tests in 35.3% of completed trials.

In 79.5% of the ongoing add-on trials, recruitment began since the development of the MCCB.18,19 In 53.8% of ongoing add-on trials, the MCCB (alone or in combination with other neurocognitive test batteries such as BACS,23 CANTAB,24 or CogState Schizophrenia Battery—CSSB25) is the primary outcome measure (see online supplementary figure 1). Use of other batteries or domain-specific test is limited in the ongoing add-on trials (each in 12.8% of trials).

Functional Endpoints.

No precise information on functional outcome or functional capacity measures was available for the majority of completed (25/73.5%) and ongoing (35/89.7%) trials. In those that reported a functional outcome, the University of California San Diego Performance Skills Assessment26 was the most frequently used functional capacity measure; it was included in 4/11.8% of the completed and 4/10.3% of the ongoing trials. Other measures included the Global Assessment of Functioning27 and the Strauss-Carpenter Level of Functioning Scale.28

Participant Population Characteristics.

Age

Baseline characteristics of subjects included in the completed studies with available results (N = 17) are summarized in table 1. In general, completed add-on studies recruited stable subjects with schizophrenia between 18 and up to 60–65 years of age. Mean age ranged between 25.1 and 52.7 across the treatment groups; in the majority of studies it was between 40 and 49 years. The only outlier with respect to the age inclusion criterion was a trial of minocycline, 29 which included subjects with recent onset schizophrenia between 18 and 35 years. Among these 17 completed studies, the mean age of illness onset was reported in only 2 studies (11%–8%; onset at 22.7–25.9 y of age across the treatment groups) and mean duration of illness in 7 studies (41.2%; ranging between 13.0 and 25.5 y across the treatment groups). In general, ongoing add-on trials are recruiting subjects typically 18 to 55/60/65 years of age (see online supplementary table 4). The exceptions are 5 trials (8.8%), 3 of which included subjects ≤35 years old, one between 18 and 45 years and one between 18 and 50 years.

Table 1.

Summary of Completed Add-On Studies with Results in Public Domain

Compound (in bold)/NCT Identifier/Study Description as per www.clinicaltrials.gov	Source	Number of Subjects Per Treatment Group/Power Calculations Provided (Yes/No)	Sufficient Sample Size to Detect a Large Effect Size	Sex Distribution (Males/Females)	Mean Age, years (SD); Duration of Illness, years (SD)	Cognition as a Primary Outcome	Baseline Cognitive Impairment	Primary Cognitive Outcome Variable and Mean Score (SD) on Primary Cognitive Outcome Variable	Mean Score (SD) on Symptom Severity Scale	Results Summary
AL-108/Davunetide	Javitt et al30	Not reported	No	Not reported	Not reported; Not reported	Yes	Performance less than the maximum cutoff for one of the following MCCB tests: letter-number span (20), HVLT total (31), and CPT d-prime (3.47)	MCCB, Not reported	Not reported	• No statistically significant separation attained vs placebo on MCCB
NCT00505765										• Statistically significant separation attained on the UPSA
Phase II/interventional
Ampakine	Goff et al31	Ampakine, N = 51; Placebo, N = 54	Yes	Ampakine, 44/7; Placebo, 43/11	Ampakine, 43.7(11.0); Placebo, 42.0(9.3); Not reported	Yes	Not used	NAART, TMT, DS-CPT, California Verbal Learning Test, faces, and family pictures subtests from WMS-III, WCST, letter, and category fluency, letter-number span-grooved pegboard	PANSS total: Ampakine, 66.1(16.5); Placebo, 69.7(16.3)	• No difference from placebo when added to clozapine, olanzapine, or risperidone.
NCT00235352		Yes, the sample size was calculated to provide an 80% likelihood of detecting a between-groups effect size of 0.6.						Composite score values at baseline not reported
Phase II/interventional
Armodafinil	Kane et al32	Armodafinil 50 mg/day, N = 15; Armodafinil 100 mg/day, N = 15; Armodafinil 200 mg/day, N = 15; Placebo, N = 15; None described	No	Armodafinil 50 mg/day, 11/4; Armodafinil 100 mg/day, 10/5; Armodafinil 200 mg/day, 11/4; Placebo, 12/3	Armodafinil 50 mg/day, 44.8(8.9); Armodafinil 100 mg/day, 40.4(9.6); Armodafinil 200 mg/day, 41.4(9.8); Placebo, 46.0(7.8); Not reported	Yes	Not used	MCCB: Armodafinil 50 mg/day, 27.8(8.6); Armodafinil 100 mg/day, 20.6(8.5); Armodafinil 200 mg/day, 22.1(16.4); Placebo, 22.3(14.6)	Not available	• No improvement in cognitive measures
NCT00487942										• Add-on armodafinil appeared to mitigate negative symptoms, as assessed by changes in the PANSS negative subscore but not on the SANS
Phase II/interventional
Atomoxetine	Kelly et al33	Atomoxetine, N = 10; Placebo, N = 12; None described	No	Atomoxetine, 8/2; Placebo, 8/4	Atomoxetine, 48.9(5.7); Placebo, 49.1(8.5); Not reported	No (changes in positive symptoms and negative symptom measures)	Participants were required to have a score ≤90 on the RBANS	Reaction time, processing speed and efficiency, working memory-digit symbol, number sequencing, letter number sequencing, mental arithmetic, grooved pegboard, simple reaction time, complex reaction time, delayed match to sample from the automated neuropsychological assessment metric; (2) sustained attention and resistance to distractibility-Gordon's Continuous Performance Test; (3) learning and memory-California Verbal Learning Test and Brief Visual Memory Test; (4) executive functioning- Planning Test and Phonemic Fluency.	BPRS total: Atomoxetine, 32.5(9.7); Placebo, 39.8(9.9)	• No evidence of variation in treatment effects on z score changes across the individual cognitive tests
NCT00161031								Overall mean z score: Atomoxetine, 0.14 (0.52); Placebo, −0.13(0.72)		• No between-group differences in symptom changes
Interventional										• The authors conclude that these results are not promising, particularly as the study was powered appropriately and designed based on consensus standards for studying neurocognitive function
Atomoxetine	Friedman et al34	Atomoxetine, N = 10; Placebo, N = 10; None described	No	Not reported	Not reported; Not reported	Yes	Presence of definable cognitive deficits of interest including visuospatial working memory, CPT, and WCST (eg, at least 1 SD below average).	BACSBACS composite standardized zscoreAtomoxetine−1.13(0.61)Placebo−1.22(0.66)	PANSS total: Atomoxetine, 30.9(7.0); Placebo, 35.7(7.5)	• No significant improvement on the BACS composite score
NCT00488163										• Only significantly greater improvement with atomoxetine on the Work Skills domain of the SLOF
Pilot study; phase IV interventional										• Atomoxetine was associated with significantly greater increases in working memory-related activation of the left dorsolateral prefrontal and left posterior cingulate cortices.
										• The negative results of this study conflict with the effectiveness of amphetamine in enhancing the cognitive abilities of schizophrenic patients and may be related to the differential pattern of cortical activation and deactivation produced by amphetamine.
AZD3480	Press release,35 10 December 2008	Estimated enrollement: 400; None described	Yes	No details provided	No details provided; Not reported	Yes	Not used	IntegNeuro computerized test battery; No details provided	No details provided	• No improvement on various cognitive domains of the IntegNeuro computerized test battery
NCT00528905
Proof of concept; phase II, interventional trial
Clozapine+Risperidone	Honer et al36	Clozapine, N = 34; Clozapine + Risperidone, N = 34	No	Clozapine, 24/34; Clozapine + Risperidone, 25/34	Clozapine, 39.4 ± 11.0; Clozapine + Risperidone, 34.9 ± 8.5; Clozapine, 16.9 ± 11.2; Clozapine + Risperidone, 13.0 ± 9.0	No (reduction in PANSS total score)	Verbal working memory index: Clozapine, 0.09 ± 0.83; Clozapine + Risperidone, −0–10 ± 0.85	Verbal Working Memory Index: Clozapine, −0.05 ± 0.99; Clozapine + Risperidone, 0.14 ± 0.93	PANSS total score: Clozapine, 89.8 ± 15.8; Clozapine + Risperidone, 84.8 ± 20.1	• No statistically significant difference in symptomatic benefit between augmentation with risperidone and placebo on the PANSS
										• The verbal working-memory index showed a small decline in the risperidone group and a small improvement in the placebo group (P = .02 for the comparison between the 2 groups in the change from baseline).
NCT00272584
Phase IV, interventional
D-cycloserine or glycine	Buchanan et al37	D-Cycloserine, N = 53; Glycine, N = 52; Placebo, N = 52; None described	Yes	Not reported	D-Cycloserine, 44.4 (10.4); Glycine, 42.6(10.8); Placebo, 43.4(11.4); D-Cycloserine, 21.8(11.1); Glycine, 20.2(10.0); Placebo, 20.2(11.1)	Yes	Not used	Neuropsychological Test Battery: Neuropsychological Test Battery Summary z score: D-Cycloserine, −0.01(0.69); Glycine, −0.11(0.64); Placebo, 0.07(0.71)	BPRS total: D-Cycloserine, 1.9(0.4); Glycine, 1.9(0.4); Placebo, 1.9(0.4)	• No difference between D-cycloserine vs placebo or glycine vs placebo in changes from baseline on the SANS
NCT00222235										• No difference between D-cycloserine vs placebo or glycine vs placebo in changes from baseline on the cognitive domain z score
Phase II/III, interventional
D-cycloserine	Goff et al38	D-Cycloserine, N = 19; Placebo, N = 19; None described	No	D-Cycloserine, 10/9; Placebo, 13/6	D-Cycloserine, 50.1(9.15); Placebo, 48.0(6.66); D-Cycloserine, 23.9(12.5); Placebo, 21.6(8.7)	Yes	Not used	Cognitive battery measuring 6 domains + LMT	SANS total: D-Cycloserine, 26.5(9.88); Placebo, 24.0(10.38)	• D cycloserine was associated with persistent improvement of negative symptoms compared with placebo and facilitated memory consolidation (thematic recall test) tested after 7 days.
NCT00455702								Cognitive composite score (calculated as the mean of all 6 standardized domain scores).		• These findings suggest that once-weekly dosing with d-cycloserine for negative and memory consolidation merits further study.
Phase IV, interventional								D-Cycloserine, −0.11 (−0.72); Placebo, −0.12(−0.60)		• As the first study of once weekly dosing in schizophrenia, this study was exploratory and so results must be considered preliminary.
Galantamine	Buchanan et al39	Galantamine, N = 42; Placebo, N = 44; None described	Yes	Galantamine, 37/5; Placebo, 37/7	Galantamine, 49.9(9.2); Placebo, 49.5(9.9); Not reported	Yes	A total score of ≤90 on the RBANS	Eight-test neuropsychological test battery; RBANS totalscore: Galantamine, 70.3(10.1); Placebo, 69.4(12.3)	BPRS total: Galantamine, 33.8(9.1); Placebo, 34.9(10.7)	• Significant improvements on the WAIS-III digit symbol and verbal memory measures with galantamine
NCT00176423										• Significant improvement on the GDS distractibility test with placebo.
Phase IV, interventional										• Galantamine may have selective benefits for aspects of processing speed and verbal memory but interferes with practice effects during the performance of an attention task.
MEM 3454/R3484	Press release,40 19 November 2008	Estimated enrollment = 160; None described	Yes	Not reported	Not reported; Not reported	Yes	Not used	MCCB; Not reported	Not reported	• In a Memory Pharmaceuticals press release dated 19 November 2008, the company announced that recruitment targets had been met and suggested top-line results of this study would be reported by the end of April 2009
NCT00604760
Phase II, interventional
Memantine	Lieberman et al41	Memantine, N = 69; Placebo, N = 67	Yes	Memantine, 31/28; Placebo, 53/14	Memantine, 40.9(9.8); Placebo, 40.111.3); Memantine, 16.6(9.6); Placebo, 16.4(10.6)	No (changes in the PANSS total score)	Not used	BACS; BACS: Memantine, 0.19(0.71); Placebo, 0.01(0.67)	PANSS total: Memantine, 73.7(16.1); Placebo, 74.3(15.9)	• At endpoint, total PANSS scores did not differ between the memantine and the placebo group
		The required sample size was determined using the assumption that a clinically meaningful difference between the 2 treatment groups would be 8.5 points in total PANSS score with a pooled SD of 14.7.								• A similar outcome was observed for all secondary measures.
NCT00097942										• Memantine showed no efficacy as an adjunctive therapy in schizophrenia patients with residual psychopathology and was associated with a higher incidence of AEs than placebo.
Phase II, interventional										• Incorrect calculation of BACS composite score made primary cognition analysis uninterpretable
Minocycline	Levkovitz et al29	Minocycline, N = 36; Placebo, N = 18; None described	No	Minocycline, 25/11; Placebo, 15/3	Minocycline, 25.14(4.77); Placebo, 24.67(4.24); Minocycline, 20.94(4.54); Placebo, 21.36(4.34)	No (changes in the SANS score)	Not used	CANTAB; composite CANTAB score values at baseline not reported	PANSS total: Minocycline, 42.54(18.66); Placebo, 43.56(18.12)	• Minocycline showed a beneficial effect on negative symptoms and general outcome (evident in SANS, Clinical Global Impressions scale).
NCT00733057										• A similar pattern was found for cognitive functioning, mainly in executive functions (working memory, cognitive shifting, and cognitive planning).
Phase III, interventional										• Overall, the findings support the beneficial effect of minocycline add-on therapy in early-phase schizophrenia.
MK0777	Buchanan et al42	MK0777, 3 mg BID, N= 18; MK0777, 8 mg BID; N = 21; Placebo, N = 21	No	Males: MK0777, 3 mg BID, 61.1%; MK0777, 8 mg BID, 61.9%; Placebo, 77.2%	MK0777, 3 mg BID, 43.3 (9.3); MK0777, 8 mg BID, 44.9 (8.7); Placebo, 40.0 (19.9); Not reported	Yes	Performance less than the maximum cutoff for of the following MCCB tests: (1) Letter-number span (20); (2) HVLT total (31); and (3 ) CPT d-prime (3.47); able to complete the baseline MCCB validly as assessed by Chief Neuropsychologist or NP tester; raw score of 6 or greater on the WTAR	MCCB; MCCB mposite core, MK0777, 3 mg BID, 31.0(12.6); MK0777, 8 mg BID, 27.9 (12.2); Placebo, 30.1 (13.1)	BPRS otal core, MK0777, 3 mg BID, 28.9 (5.2); MK0777, 8 mg BID, 29.8 (6.2); Placebo, 26.8 (6.4)	• No significant group differences on the MCCB composite score.
	The sample size was determined with the ANCOVA power formula, n_2(z__z_)2 s2 (1_R 2)/d2, with z__ 2.24, z__.842 (corresponding to power_.80), R_the correlation between baseline and end of study measures of the primary outcome (estimated to equal .6 for the MCCB composite score), d the difference between groups, and s the SD of the primary outcome.									• Participants randomized to placebo performed significantly better on visual memory and reasoning/problem-solving tests than participants assigned to either MK-0777 dose.
	Planned toenroll 30 participants/group, which would have enabled detecting an effect size _ .73 with power _ .80. The actual recruitment was only approximately 20 participants/group, but the observed R approximately_.9, suggesting power to detect an effect size of .49.									• There were no significant group differences on the AX-Continuous Performance Test or N-Back d prime scores or UCSD Performance-Based Skills Assessment-2 and Schizophrenia Cognition Rating Scale total scores.
NCT00505076
Phase II, interventional
Modafinil	Freudenreich et al43	Modafinil + clozapine, N = 19; Placebo, N = 18; None described	No	Modafinil, 14/4; Placebo, 12/4	Modafinil, 44.2(12.0); Placebo, 46.4(6.4); Modafinil, 18.9(11.2); Placebo20.2(8.2)	Yes	Not used	COGBAT composite score; COGBAT (Slope)(SE), Modafinil, 0.018(0.01); Placebo, 0.028(0.01)	PANSS otal: Modafinil, 63.8(15.5); Placebo, 70.3(13.7)	• Modafinil did not reduce negative symptoms or wakefulness/fatigue or improve cognition compared with placebo.
NCT00573417										• Given the limited power to detect a treatment effect and the clear possibility of a type II error, larger trials are needed to resolve or refute a potential therapeutic effect of uncertain magnitude
Pregnenolone	Marx et al44	Pregnenolone, N = 9; Placebo, N = 9; None described	No	Pregnenolone, 8/1; Placebo, 9/0	Pregnenolone, 52.68(6.31); Placebo, 49.43(12.19); Not reported	Yes	Composite BACS score 0–3 SD below the mean	BACS and MCCB; BACS Composite z score: Pregnenolone, −1.54(0.96); Placebo, −1.28(0.96); MCCB composite T score: Pregnenolone, 29.00(10.06); Placebo, 27.33(12.96)	SANS total: Pregnenolone, 50.75(12.21); Placebo, 47.56(12.09)	• Mean changes in composite BACS and MCCB scores were not significantly different in patients randomized to pregnenolone compared with placebo
NCT00560937										• Serum pregnenolone increases predicted BACS composite scores at 8 weeks in the pregnenolone group,
Interventional										• Increases in allopregnanolone, a GABAergic pregnenolone metabolite, also predicted BACS composite scores
										• Baseline pregnenolone, pregnenolone sulfate allopregnanolone levels were inversely correlated with improvements in MCCB composite scores, further supporting a possible role for neurosteroids in cognition.
										• Pregnenolone may be a promising therapeutic agent for negative symptoms and merits further investigation for cognitive symptoms in schizophrenia.
Pregnenolone + DHEA	Ritsner et al45	PREG 30 mg, N = 16; PREG 200 mg, N = 10; DHEA, N = 16; Placebo, N = 16; None described	No	PREG 30 mg, 9/5; PREG 200 mg, 5/1; DHEA, 10/3; Placebo, 8/3	PREG 30 mg, 38.3(9.2); PREG 200 mg, 34.3(9.9); DHEA, 35.5(9.2); Placebo, 34.6(5.3); PREG 30 mg, 15.1(8.0); PREG 200 mg, 11.7(7.7); DHEA, 10.3(7.3); Placebo, 11.1(6.5)	Yes	Not used	CANTAB; CANTAB Composite z score at baseline not reported	PANSS Positive Subscale: PREG 30 mg, 17.6(5.6); PREG 200 mg, 20.5(9.2); DHEA, 19.2(5.1); Placebo, 16.0(5.2)	• PREG 30 mg group experienced significant reduction in positive and extrapyramidal symptoms and improvement in attention and working memory performance
NCT00174889										• The authors note that a replication trial is needed with a larger sample and longer duration
Phase I, interventional

Note: BACS, Brief Assessment of Cognition in Schizophrenia; BPRS, Brief Psychiatric Rating Scale; CANTAB, CAmbridge Neuropsychological Test Automated Battery; CPT d-prime, Continuous Performance Task (d is a measure of subjects ability to discriminate sound from background noise); GDS, Gordon Diagnostic System; HVLT, Hopkins Verbal Learning Test; MATRICS, Measurement and Treatment Research to Improve Cognition in Schizophrenia; MCCB, MATRICS Consensus Cognitive Battery; NAART, North American Adult Reading Test; NCT, National Libabry of Medicine began to include the ClinicalTrials.gov registry number in the MEDLINE record when the number is published as part of the original paper. The format for the ClinicalTrials.gov registry number is “NCT” followed by an 8-digit number, e.g.: NCT00000419; PANSS, Positive and Negative Syndrome Scale; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status; SANS, Scale of Assessment of Negative Symptoms; SLOF, Self-Assessment of Functional Status in Schizophrenia; TMT, Trail Making Test; UCSD, University of California San Diego (UCSD) Performance Based Skills Assessment; UPSA, University of California San Diego Performance Skills Assessment; WCST, Wisconsin Card Sorting Test; WMS-III, Wexler Memory Scale -III.

Sex

Participants of both sexes were recruited in the completed studies; male subjects were in a clear majority, with 68% men across all study samples. Ongoing add-on trials are recruiting subjects of both sexes (see online supplementary table 4).

Baseline Cognitive Impairment as Inclusion Criterion

A defined level of cognitive impairment was used as an inclusion criterion in 20.6% of the completed and 15.4% of the ongoing add-on trials (see online supplementary tables 2 and 4). However, there was no consistency in the choice of definitions of cognitive impairment used across the studies.

Baseline Cognitive Impairment of Subjects Included in the Completed Trials

Based on the available data (table 1), the subjects included in the completed trials had at least a minimal level of cognitive impairment. The MCCB18,19 was used in 2 trials with available results. In the armodafinil trial, 31 mean baseline MCCB composite scores (SD) ranged between 20.8(8.5) and 27.8(8.6) across the treatment groups. In the MK-0777 trial, 37 the mean scores ranged between 27.9 (12.2) and 31.0(12.6). These numbers are consistent with the MCCB screening scores of 323 patients in the 29-site lurasidone vs risperidone trial, for which treatment results are not yet available.21 The mean baseline score for the entire study sample in that trial was 24.7 (12.1).21 In 6 ongoing trials (15.4%), inclusion criteria defined an acceptable level of baseline cognitive impairment (see online supplementary table 4).

Results of Completed Add-on Trials.

The summary of available trial results is presented in table 1.

Only one of the above studies with available results (see table 1) had sufficient power to detect a medium (d = 0.5) effect size, while several studies had sufficient power to detect a large (d = 0.8) effect. None of the 346–48 completed crossover trials with available results reported any significant effects of drug on cognitive performance compared with the placebo (see online supplementary table 3). Only 2 of 6 completed monotherapy trials are currently published49,50 but without results of neurocognitive testing.

Discussion

Our search of www.clinicaltrials.gov identified 118 CIAS trials that satisfied criteria for inclusion in our analysis. It should be noted that, despite its size, www.clinicaltrials.gov is only one among many national and international clinical trial registries, and therefore while the data obtained on CIAS clinical trials are informative, they are not exhaustive. In addition, some older trials may not be registered at www.clinicaltrials.gov, or some trials may not have been identified in the search and therefore omitted from this analysis. Our analysis showed that add-on, placebo-controlled, double-blind design is predominant across terminated, completed, and ongoing trials and that the use of widely accepted standardized cognitive batteries is increasing. However, other critical methodological issues, such as sample sizes to achieve standard statistical power, appear to be suboptimal in many studies. The studies included pharmacological treatments with many diverse mechanisms of action, though agents acting at the NMDA receptor have been examined most frequently. The lack of available results for many trials precluded any assessment of the most promising mechanisms of action.

What Has Been Learned So Far?

In general, a majority of completed and ongoing trials follow the MATRICS guidelines17 for studying an adjunctive/cotreatment agent in a randomized, placebo-controlled trial consisting of clinically stable participants with schizophrenia.17,51–53

Compared with the ongoing trials, the completed add-on trials were shorter, had smaller sample sizes, and were less consistent in choice of cognitive outcome measures. Since published results were available for only half of the completed trials, the conclusions we can draw about the overall success of these trials are limited. In general, the completed trials, as well as the subset with available results, were predominantly of ≤8 weeks duration. The sole exception was a trial29 that assessed the effects of a 6-month add-on treatment with either minocycline or placebo on cognitive impairment in young subjects in early phase schizophrenia, aged 18–35 years. In this study, minocycline, a tetracycline antibiotic with a distinct neuroprotective profile, was found to be significantly superior to placebo at endpoint in improving cognitive functioning as well as negative symptoms and general outcome.29 The original MATRICS guidelines17 specify that a phase 3 registration trial should be of sufficient duration to show an enduring effect on cognition (ie, at least 6 months), and this recommendation remained unchanged in their recent revision.20 The majority of trials reviewed here appear to have been proof-of-concept trials with a shorter duration. This approach is reasonable given the associated expense of longer studies that entail significant financial risk without conferring sufficient guarantee of an efficacy signal. In addition, there is insufficient data to support the proposition that longer study duration is associated with better results in neurocognition.

Typically, completed trials tended to have a sample size of <100 subjects, with many enrolling <50 subjects. The authors of several recently published add-on trials have commented that their negative results could have been related to limited sample sizes and the resulting low statistical power to detect changes in cognitive scores.38,43,44 Our analysis shows that only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size, thus increasing the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.54,55 Some of the studies included in this analysis were conducted in the early phases of drug development and in keeping with their exploratory nature, involved small sample sizes. Given the large investment in time and resources required to run larger trials, small-sample studies and those that do not meet the rigorous requirements for a full-scale pivotal trial, such as open-label studies, can contribute valuable evidence of efficacy and safety. These early-phase studies assist drug developers in making important decisions about whether to invest financial resources in a drug’s development. The application of nonstandard thresholds for statistical significance is reasonable in these circumstances, as is targeting patient populations that may be the most responsive to treatment. However, the results of underpowered studies such as those reviewed here, especially when negative, should be interpreted with caution.56,57 The fact that the overwhelming majority of studies reviewed here have not had sufficient statistical power challenges the field to draw accurate general conclusions about the potential for drug development for CIAS.

Most of the studies reviewed recruited clinically stable people with schizophrenia, aged between 18 and 55/60/65 years with a mean age between 40 and 49 years. Other age-related variables, such as age at disease onset or years since diagnosis, which may serve as a proxy for schizophrenia chronicity, were rarely reported. Although cognitive impairments are generally stable over relatively short periods of time in the longitudinal course of schizophrenia58–60 cognitive function is age dependent in healthy controls and schizophrenia samples. Larger deficits in working memory have been shown to exist in elderly vs first-episode people with schizophrenia, while worse recall of material in episodic memory, changes in select time-based measures of problem solving and fine motor dexterity have all been associated with greater length of illness.61,62 In some studies of behavioral interventions not reviewed comprehensively in this report, younger people with schizophrenia and those early in the course of illness appeared to benefit more from cognitive remediation than older people.63–65 Although the sample of chronic, stable participants with schizophrenia may be relatively convenient to recruit, these may not be the individuals who are most likely to show improvements in cognition.66 A trial performed in younger subjects with recent illness onset29 may yield more positive results than those conducted in patients with an average illness duration of >20 years (See table 1). It is reasonable to expect that younger patients with greater potential neuroplasticity may be optimal candidates for pharmacological intervention, but surprisingly, few data are available that address this question empirically.

What Are We Likely to Learn From Ongoing Studies?

Compared with the completed-add-on trials, the ongoing add-on trials are longer, have larger sample sizes, and are more likely to use a widely accepted standardized cognitive battery (eg, MCCB18,19). This suggests that the MATRICS recommendations are being implemented in the more recent trials. The ongoing trials generally allow for recruitment of subjects aged between 18 and 55–65 years, and it is likely that the actual age range will be comparable to the completed studies. However, 3 ongoing add-on studies are recruiting recent onset subjects; their results may help to determine which subjects are most likely to experience cognitive benefits.66 The level of baseline cognitive impairment was defined in few ongoing trials. The MATRICS guidelines17 noted that in general, it is not necessary to exclude subjects with a high level of cognitive functioning in whom further improvement in cognition would not be expected to be demonstrated because with a properly constructed cognitive test battery, this level of performance is very rare. However, the question remains whether severely cognitively impaired subjects should be enrolled into CIAS trials. We still do not know the extent to which the presence of a “floor” effect on a cognitive test indicates minimal capacity for cognitive improvement. Analysis of extant cognitive data from previously completed studies could address whether severely impaired patients are negating an overall clinical benefit of a treatment, and surprisingly, little work in this area has been completed.

Among the ongoing add-on trials, 30.7% estimated an expected enrollment of ≥100 subjects. Such large samples imply multi-site trials, which present investigators and sponsors with specific challenges (for review, see ref.54). For international clinical trials, it is also important to ensure the cross-cultural and linguistic adaptability of primary67 and coprimary outcome measures.68 Several large-scale multisite studies, including international trials, are currently underway, and upon their completion, it may be possible to determine whether specific recommendations for performing multisite trials with neurocognitive assessments have been successfully implemented, and how these recommendations may impact trial results. Early results suggest that good psychometric characteristics of the cognitive outcome measure are possible in large multisite studies if sufficient care is given to training and data quality.21 Our results also show that the percentage of ongoing add-on trials with sufficient statistical power to detect medium effect sizes has doubled in comparison to the completed add-on trials. However, more than half of ongoing trials still may have inadequate sample sizes. Although adequate sample size is an important determinant in the estimation of statistical power in each study, an important and underappreciated component of statistical power calculations is that relatively small changes in the reliability of the neurocognitive endpoints can have a strong effect on the sample size needed, especially when the magnitude of the expected effect is small to medium. It has been demonstrated that the MCCB18,19 has excellent reliability, minimal practice effects, and significant correlations with measures of functional capacity. These favorable psychometric properties have also been observed in the context of a large multisite industry trial for which it was designed.21 However, it is still not confirmed whether the MCCB is sensitive to change during pharmacological treatment. As its use has substantially increased in ongoing add-on trials, a wealth of data regarding this issue will be available in the next few years. To accommodate the needs of multinational trials, the MCCB has been translated into a range of languages.67 Recently, the reliability, validity, and practicality of functionally meaningful co-primary measures was established,68 and the MCCB impairment profile for schizophrenia outpatients became available.69 Our search revealed the use of neuropsychological test batteries other than the MCCB (eg, BACS,23 CANTAB,24 and CSSB25), though their use is somewhat limited in the ongoing trials. Future results may contribute to our understanding of their psychometric characteristics within the context of large multisite trials.

No clear pattern could be established in the choice of co-primary outcome related to functioning or functional capacity in either completed or ongoing trials. While the MATRICS initiative made clear recommendations regarding the cognitive outcome measure, it did not make strong recommendations about the choice of functional capacity measures, and several different strategies were considered to be acceptable. The recent VIM study68 suggests that there are no optimal co-primary measures, although several of them have reasonable psychometric characteristics. The performance of these measures in currently ongoing trials will provide additional practical information about their utility for future work.

Possible Reasons and Contributing Factors Related to Negative Study Results

Since half of the completed trials do not have results in the public domain and even fewer in the peer-reviewed literature, it is unfortunately challenging to make any reliable appraisals of the factors that may be associated with negative trial results. The possible methodological reasons for the lack of effect of co-treatment with potentially cognitive-enhancing drugs are varied. The main conclusion from our review of these studies is that most of the studies had woefully inadequate statistical power. Ongoing studies appear to have greater statistical power and the opportunity for positive results is greater. While crossover designs for cognitive outcomes are appealing due to their capacity to enhance power through within-subjects analyses, these trials may obscure effects that could be seen in parallel group studies because practice effects and treatment effects may be confounded.22 Although the possibility of genetic mediation of improvements in neurocognitive deficits is intriguing, the initial findings to date will require replication and functional validation.70

How May These Lessons Help Shape Future Trials?

Based on the information gathered in our analysis, we summarized the key issues pertinent to the implementation and conduct of trials assessing potential cognitive-enhancing drugs in schizophrenia and possible solutions (table 2).

Table 2.

Key Methodological Issues in Trials Assessing Potential Cognitive-Enhancing Drugs in Schizophrenia

Key Methodological Issues
Design choice	Randomized, double-blind, placebo-controlled add-on design is preferred
	Crossover design may confound practice and treatment effects
Power	Reliability of outcome measure and sample size need to be sufficient to ensure adequate statistical power
Endpoint (cognitive testing and intermediate measure) selection	Validation of endpoints in patients with schizophrenia is essential
	Experience with the endpoint in clinical trials setting is important
	For international trials, endpoints should be adequately translated and culturally adapted; language and culture specific norms may enhance sensitivity to treatment effects
Patient population	Younger patients in the earlier phases of illness may have greater potential for benefit

Conclusions

A review of the trials listed on www.clinicaltrials.gov suggests that a substantial number of clinical trials of potential treatments for cognitive enhancement in schizophrenia are currently ongoing. The studies completed to date have not had sufficient statistical power to state confidently that a particular treatment does not have potential efficacy. Further, the predominant patient population in these studies has been older, chronic, and mostly male patients with schizophrenia, who may be the least likely to benefit from cognitive enhancement. Many ongoing studies have larger and more diverse samples and are likely to shed a brighter light on the challenges of CIAS trial design and methodology. These ongoing efforts may increase the probability of identifying treatments with beneficial effect on cognitive impairment in schizophrenia.

Funding

All authors contributed substantially to the conception and design, or acquisition of data, or analysis and interpretation of the data. All authors were involved in drafting and revising the manuscript for important intellectual content and all authors approved the final version to be published. Milana Zivkov, MD was a paid consultant to Pfizer, Inc, New York, NY, in connection with the development of this manuscript. None of the other external authors received funding from Pfizer, Inc. for their efforts. Funding to pay the Open Access publication charges for this article was provided by Pfizer, Inc.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.oxfordjournals.org.