The hidden variables problem in Alzheimer's disease clinical trial design.

As the leading cause of dementia worldwide, Alzheimer's disease has garnered intense academic and clinical interest. Yet, trials in search of a disease-modifying therapy have failed overwhelmingly. We suggest that, in part, this may be attributable to the influence of disruptive variables inherent to the framework of a clinical trial. Specifically, we observe that everyday factors such as diet, education, mental exertion, leisure participation, multilingualism, sleep, trauma, and physical activity, as well as clinical/study parameters including environment, family coaching, concurrent medications, and illnesses may serve as potent confounders, disruptors, or sources of bias to an otherwise significant drug-disease interaction. This perspective briefly summarizes the potential influence of these hidden variables on the outcomes of clinical trials and suggests strategies to abate their impact.

Introduction

Clinical trials evaluating disease-modifying therapies for Alzheimer's disease (AD) have faced unprecedented failure with over 99% of compounds entering phase I trials never reaching approval [1]. This exceeds all other disease indications of comparable scope, including complex disorders such as cancer, where success rates approach 20% [1]. Analyses typically blame the innate complexity of AD and its constituent pathologies for these failures. However, the repeated frustration of independent and diverse therapeutic strategies may also suggest an intrinsic weakness in the architecture of traditional clinical trial design as applied to AD. Accordingly, we propose that a series of hidden (in plain sight) variables may have the potential to obscure the perceived efficacy of a therapy within the standard clinical trial paradigm. We have delineated these into two broad classes of variables: (1) “everyday” variables, such as diet, intellectual stimulation, and physical exertion; (2) “clinic day” variables including clinic environment, concomitant illnesses, unrelated drug interactions, and variations in cognition testing. Cumulatively, these may alter drug efficacy, disease progression, and/or perception of disease to a clinician. This perspective provides a brief review of these variables, as relevant to the conduct of clinical trials for AD, and outlines their potential role in impeding the observation of an intervention/disease interaction. It further suggests broad strategies to abate these influences in future clinical trials. (Fig. 1)

Fig. 1

Summary of potential hidden variables in Alzheimer's disease clinical trials. The interaction of a drug on AD and cognition/memory may be modulated by a variety of variables. The efficacy of an experimental therapy may be altered by diet/nutrient status (malnutrition). The pathologies and course of AD may be altered by concurrent medications and disease, as well as diet, trauma, physical activity, and sleep. Perception of cognition/memory may be altered by caregiver coaching, clinic environment, participant stress, and cognitive reserve, which itself may be modulated by fulfillment, education, leisure participation, and multilingualism. Clinical variables are denoted in dark gray and daily variables are denoted in light gray. Amplifying effects are denoted by pointed arrows (←), whereas impeding effects are denoted by flat arrows (⊢); variables with amplifying and impeding effects are denoted by both symbols.

Everyday life variables

The everyday lives of study participants are understandably dissimilar and unalike. In AD, seemingly trivial factors such as diet or mental exertion can significantly influence disease progression and response to an experimental therapeutic. While studies typically rely on the inclusion of an equally diverse control group to mitigate the variability introduced by these parameters, the number and scope of everyday life variables pertinent to AD is sufficiently large as to potentially overwhelm the mitigating capacity of the control group and obscure a drug-disease interaction. Efforts to recognize and clarify the influence of hidden variables in everyday life may thus be important to AD clinical trial design.

Diet

Extensive nutritional studies have established a role for diet in moderating the pathogenesis, symptoms, and progression of AD [2], [3]. Cholesterol was among the first of these associations, implicated when mutations in the apoE lipoprotein transporter emerged as a major genetic predisposition for AD [4]. Subsequent studies revealed that cholesterol may directly modulate β-amyloid (Aβ) oligomerization [5] and may further drive the pathogenic misfolding of Aβ responsible for its aggregation into plaques [3], [6]. Saturated and trans fats are similarly associated with deleterious risk and more rapid disease progression. In an observational study, Morris et al. demonstrated that a high-fat diet may double the risk of AD and even modest increases of trans fat consumption were sufficient to elevate AD risk significantly [7]. Mechanistic studies have linked these fats to the escalation of inflammatory and oxidative stress, as well as increasing the risk for metabolic and vascular disorders, also independently associated with AD [2], [3].

Conversely, omega-3 polyunsaturated fatty acids have been associated with lowering AD risk and delaying symptoms, possibly due to their anti-inflammatory and neuroprotective effects [8]. Generally, consumption of nutrients with anti-inflammatory properties (such as forms of vitamins B, C, and E) lowers risk of AD and preserves cognition after disease onset [2]. Dietary patterns rich in these nutrients, the best studied of which is the Mediterranean diet, are also associated with a reduced risk profile and improved prognosis [9]. Although interventional trials of isolated nutrients have yielded largely insignificant results, a modulatory role for diet in AD is evident.

In addition, some drug models have demonstrated that diet may affect the absorption and metabolism of therapeutic compounds [10]. Several studies, discussed by Walter-Sack and Klotz, have identified that specific dietary patterns may diminish drug efficacy, either by weakening biotransformation processes or by elevating excretion and clearance [10]. Foods such as protein-rich meats and cruciferous vegetables have also shown associations with the functional levels of therapeutic compounds [11], [12]. This suggests a bilateral role for diet in a clinical trial, affecting AD risk and course, as well as the bioactivity of a therapeutic agent.

Controlling for diet

Although not routinely considered in trials of neurology, strategies to account for diet are well established in other disciplines. These range from passive recommendations, to self or caregiver reporting, to rigorously controlling diet and food intake. Naturally, intervening in diet will dramatically raise study costs as well as both participant and caregiver burden; adherence among patients with advanced dementia will also likely be poor. Studies occurring within institutionalized populations (such as long-term care facilities) may be well placed to standardize diets among other environmental variables, yet this may limit generalizability of study outcomes and conclusions. An observational approach may therefore be preferential in most cases. Here again, studies have several options which range from invasive but rigorous to passive but more error-prone. An obvious strategy is to track food intake either by caregivers or study personnel then account for significant discrepancies in dietary preferences among participants during study analyses (Section 2.5). Such an effort could clarify whether, for example, a high-fat diet or a low food intake is associated with observed outcomes. Alternatively, studies could interrogate the influence of specific nutrients already associated with AD (cholesterol, fats, vitamins, etc.) by measuring physiological concentrations (via blood or urine tests) and correlating these to outcomes during analyses. Observations of dramatic changes in nutrient levels could confound a drug effect and/or disease course and thus advise the inclusion/exclusion of patients or other analyses. For trials using a pair-matched design, incorporating baseline dietary preferences among the matching criteria for cases and controls may also help alleviate confounding or disruption, although it is noteworthy that these baseline preferences may evolve as disease progresses and may not prove relevant.

Cognitive reserve and fulfillment

The concept of cognitive reserve (CR) arose as an explanation for the weak relationship between neural pathologies and their manifestation in parameters like cognition, function, and memory [13]. CR can be defined abstractly as the multifactorial buffer between an underlying brain disease and the emergence of clinical symptoms, or as the brain's capacity to cope with, and overcome damage [14]. Initially, the size of the cerebral parenchyma was exclusively thought to define CR; however, the concept has since evolved to encompass a variety of neural processes including neurogenesis, neuroplasticity, and the regulation of neurotrophic factors [14]. In AD, CR has shown robust associations with disease risk and cognitive decline after onset [14]. As a direct mediator between pathology and symptoms, CR may be highly relevant to clinical trials.

Controlling for cognitive reserve

Quantification of CR as a clinical parameter is challenging. Studies have developed and corroborated multiple scales of CR, which are associated with neuropsychological performance [15]. In theory, CR (either baseline or changes in CR) could, as with diet, be used in selecting and matching patients, or in subsequent analyses. Yet, without a clear understanding of the biologic mechanisms of reserve, a direct translation of this manner into a clinical trial will prove controversial.

An alternative approach is to account for CR's known contributors, including education, occupation, and participation in leisure activities. In a 1994 cohort study, Stern et al. noted that low education (less than 8 years of formal instruction) and lower lifetime occupational attainment could each double the risk of AD in nondemented seniors [16]. It is noteworthy that this exceeds the widely reported relative risk for diabetes and various other traditional risk factors. Involvement in leisure activities has similarly been associated with a protective effect on AD risk. Although the precise impact is dependent on the frequency and type of activity, reports have estimated that high leisure participation may offer a 38% reduction of AD risk [14]. Multilingualism may also share similar associations to CR and AD, potentially by enforcing the development of neuroprotective executive circuitry. Bialystok et al. observed that lifelong bilingualism could delay the symptoms of dementia by an average of 4 years [17]. As with other parameters of CR, this altered prognosis may have implications for clinical trials, as it may directly obfuscate the impact of a drug/disease interaction (which also seeks to alter symptoms and prognosis).

There are multiple conceivable strategies to account for education, life fulfillment, and language status within the clinical trial design. In studies of sufficient power, each variable could be reviewed independently as baseline participant characteristics, either as binary or categorical systems, and used in data models. Alternatively, tests such as the Cognitive Reserve Scale, which quantify participation in various domains, including daily, professional, and social activities, could be used to proxy and account for CR holistically [13]. Scores from these tests could again be used in matching and selection of patients or in data analyses.

Physical activity and trauma

The role of physical activity in disease is well-documented, and sedentary lifestyles have been implicated in a variety of disease states. In the context of AD, data suggest that continued physical activity in old age may reduce overall risk by 40% [18]. After disease onset, regular exertion may preserve brain volume, aid in reducing Aβ levels, and may lessen or rescue cognitive decline [18], [19]. These multifaceted interactions highlight the relevance of exercise to AD; moreover, as cognition, Aβ levels, and cerebral volume often serve among the primary outcomes of clinical trials, the potential for obscuring a drug-disease interaction is apparent.

Controlling for physical activity

The rise of wearable fitness monitors, which continuously track the heart rate, motion, and exertion, offers a practical strategy to account for physical activity in daily life. Sylvia et al. have reviewed similar strategies and report on techniques ranging from minimally invasive monitors to direct assays of energy expenditure, which may provide observational data to account for overall physical activity among study participants [20]. Studies may also consider prescribing standardized exercise regimens; however, adherence among patients with advanced dementia and caregiver burden would favor an observational rather than interventional approach.

Controlling for trauma

A further important (although often ignored) physical variable is exposure to trauma. In Tg2576 murine models, repeated brain injuries were directly associated with a significant increase in neurofibrillary tangles, cerebral atrophy, and cognitive deficits [21]. Other studies have demonstrated that trauma is associated with Aβ aggregation and cerebral inflammation and that this translates directly to an elevated risk of AD [22]. We therefore suggest that studies also examine and/or question patients for evidence of traumatic experiences during the course of a standard clinical workup. Significant episodes of trauma, particularly near or during the study period may confound the course of AD, and should therefore be considered in evaluating the inclusion and exclusion of patients.

Sleep

Sleep disturbances are endemic among patients with dementia, and the magnitude of sleep pathology frequently parallels disease progression [23]. This association has recently been clarified with a potential mechanism when Mander et al. noted that elevated Aβ may lead to the fragmentation of nonrapid eye movement sleep [24]. Moreover, they and others found that lack of nonrapid eye movement sleep can also lead to the elevation of Aβ aggregation [23], [24]. Investigations have further observed that clearance of Aβ may be compromised when sleep is diminished [25]. This suggests a possible self-amplifying feedback mechanism, whereby lack of sleep may lead to increased Aβ, which in turn can reduce effective sleep. As with other hidden variables found in the day-to-day lives of study participants, variations in sleep habits may thus disrupt the observation of a drug/disease interaction.

Controlling for sleep

As an ostensible mediator of Aβ levels, an effort to quantify and account for sleep duration and quality should be considered, especially in trials of agents seeking to diminish cerebral Aβ. Existing, well-corroborated strategies include retrospective questionnaires such as the Pittsburg Sleep Quality Index, or direct actigraphic (motor activity) measures, which assay nighttime movement to proxy sleep duration and quality [26]. Many commercial fitness monitors can also track nighttime motion to quantify sleep patterns. Studies should further standardize or account for the use of sleep aids, sedatives, or anti-insomnia agents, as these may bias outcomes by artificial modulation of sleep.

Statistical evaluation of confounders

Although a detailed review of statistical methodology is beyond the scope of this perspective, several broad approaches exist to account for the potential influence of a confounding variable during study analyses, namely, stratification and multivariate data modeling. Stratification is a simple and often effective statistical strategy in which the strength of an outcome measure (typically the onset/severity of AD symptomology) is assessed in subgroups of the study population where a given confounder is constant. The results can then be adjusted using various estimators to both gauge the strength of the effect and the influence of the confounder. However, stratification can dramatically reduce the effective sample size of a given exposure and/or cohort, thus severely compromising study power. Multivariate modeling is a more complex solution and relies on linear or logistic regressions to examine the influence of multiple associations on an outcome. However, these models are often ineffective without large samples. Furthermore, imprudent assumptions can alter outcomes substantially, and thus skilled biostatistical analyses are required.

Other strategies, such as adopting pair matched, case-control studies, could also be used to reduce the influence of confounders. However, adopting a matched study framework may complicate patient recruitment and the study design. Moreover, improper matching can dramatically increase the influence of confounders and introduce biases, particularly if matches are chosen from unreasonably close or small populations. Broadly, there is no ideal strategy to negate the impact of confounders in study analyses. However, this should not disregard their consideration from a trial and should not discourage the collection of data relevant to assess their influence.

Clinic day variables

Under the conventional clinical trial paradigm, the efficacy of a therapy is determined over an interval of extended follow-up with periodic clinical assessments. In AD, this standard approach poses several limitations. For one, AD symptomology is highly variable from day to day, with patients known to experience good and bad days at random. An isolated clinic visit is therefore subject to AD's daily fluctuations and may fail to capture the true capacity of a patient. Furthermore, a clinic visit is a significant deviation from a patient's typical routines and schedules. The travel, unaccustomed setting, and unfamiliar personnel may perturb their mental state and thus artificially alter measurements of cognitive outcomes. In this way, AD trials risk inducing the observer effect, in that they alter the phenomenon they seek to observe, by the very attempt to observe it. This section summarizes facets of clinical studies which may further deviate a patient's cognitive parameters and thus inadvertently disrupt observations.

Variables during cognition and memory testing

A major challenge in studying AD is reliably quantifying abstract phenomena like cognition and memory. Some consistency has been established by using extensively validated tools, including the Mini-Mental State Examination or the Montreal Cognitive Assessment; however, a patient's performance on these tests can be dramatically influenced by a variety of factors. Bechtel et al. demonstrated that simply administering an assay of cognition in an unfamiliar setting was sufficient to significantly diminish perceived performance [27]. Other studies have noted similar reductions of cognition, memory, and attention when subjects were stressed, depressed, fatigued, or lacked sleep [28], [29], [30], [31]. Data have also suggested that seemingly trivial external factors like ambient lighting [32] and background noise [33] may play critical roles in mediating the perceived levels of cognition. Researchers should therefore endeavor to standardize study environment/administration and minimize factors which may distress participants. Ideally, trials should be conducted in a patient's home setting, where possible, with familiar personnel. Patients should also be screened for concomitant mood or psychological disorders, such as abnormal depression or fatigue, which may complicate measurements of cognition.

Studies should also be attentive to evidence of coaching or undue preparation of subjects. Our group previously noted that over 40% of caregivers admitted to rehearsing contents of a cognition assay before arrival in clinic and that 17% of participants showed clear evidence of preparation [34]. Caregivers and patients have a vested interest in optimizing performance, which contributes to a perceived and rewarding sense of recovery. Yet, simply learning the date, or practicing drawing a cube, may significantly elevate cognitive scores and skew an outcome measure. It is further noteworthy that patients and caregivers entering trials tend to be familiar with most standard assays of cognition; common assessments including the Mini-Mental State Examination and the Montreal Cognitive Assessment are also readily available to the public online. We suggest that trials continuing to use common assays of cognition use systematic variants of the standard questions or enact procedures to detract coaching of subjects.

The validity of these tests may be further compounded by the fact that they rely heavily on subjective evaluations to quantify the competence with which a provided task is completed. This inherently risks the introduction of variation or bias depending on the rigor with which raters are trained and monitored. Prior studies have shown that over 50% of raters in clinical trials may not fully comply with the prescribed protocols of their evaluation and that this may diminish the observed significance of an interaction [35]. As yet, the effects of coaching, clinical bias, and inconsistent testing have not been definitively established for an AD trial, although the emerging trends warrant consideration in the future study design.

Concomitant illnesses

Complex disorders, particularly age-associated diseases like AD, are often accompanied by other concomitant physical limitations and illnesses. The loss of higher sensory function is one well-established complication of AD's progressive neurodegeneration, with visual [36] and hearing [37] impairments occurring with high prevalence in AD. Koronyo-Hamaoui et al. and others have recently clarified that protein misfolding may be directly responsible for this decline, observing Aβ accumulation in the eyes of multiple AD mouse models and in postmortems of AD patients [36], [38]. Omata et al. demonstrated that the auditory system may be similarly susceptible by observing significant hearing impairments in a transgenic mouse model expressing Aβ in cochlear hair cells [39]. Hearing and sight are requisite for evaluations of cognition, which rely directly on the responses to either auditory or visual stimuli. Impaired sensation may therefore masquerade as diminished cognition, potentially confounding observations. Studies may overcome this by quantifying the visual and auditory competence of a subject before proceeding to higher assays of cognition.

Malnutrition, dysphagia, and their associated disorders are additional complications faced by AD patients. Tombini et al. have proposed that 95% of AD patients may be or are at risk of malnourishment and that this may correlate with reduced cognition [40]. Although other studies have identified a wide and occasionally contradictory array of digestive symptoms, malnutrition is consistently associated with aggravated AD symptomology and mortality [41]. In the context of a drug trial, an often overlooked factor is that drug efficacy may be compromised if patients are malnourished [10]. Although a clear mechanism has not been discerned, chronic deprivation of essential nutrients may impede requisite drug activation or transformation reactions [10]. This may considerably weaken a drug effect, as observed in a study of antimalarial agents, where treatment efficacy was halved in severe cases of malnourishment [42]. Although the model bears little relevance to AD, the potential association of malnourishment to drug efficacy is alarming.

Accounting for concomitant illnesses

Broadly, AD patients are often faced with multiple concomitant disorders. The advanced age of most patients alone predisposes them to a variety of other disorders ranging from vascular diseases to cancer. AD's risk factors (found at high prevalence among AD cohorts), including glucose/insulin dysregulation, diabetes, hypertension and dyslipidemia, further diversify AD patients into highly idiosyncratic disease profiles [2], [43]. It is as yet unclear what role many of these concurrent illnesses may have on AD progression, although it is reported that complications associated with vascular disorders generally exacerbate AD symptomology [43]. In the context of a clinical trial, it is further unclear how experimental therapies may respond in patients with complicating disorders and to what degree major outcome measures may be influenced by the concurrent illness. Pending definitive studies on the influence of these concurrent disorders, it is advisable that multiple major disease indications should be defined as an exclusion criterion. Studies may also consider accounting for the presence of concomitant disorders through statistical modeling, although the variety and prevalence of them in AD cohorts would pose challenges.

Concurrent medications

An interaction between drugs may offer yet another source of interference in a clinical trial. The proton pump inhibitor lansoprazole is a popular example. As a medication for gastritis, it raised little concern as a mediator of neurodegeneration. Yet, multiple reports have suggested that lansoprazole may enhance Aβ production, and that patients on consistent proton pump therapy may be at a higher risk of dementia [44], [45]. The claim has faced contradiction, yet the potential for exacerbating Aβ production may counteract a benefit from a novel compound.

Alternatively, some unrelated drugs may work to combat AD pathologies. For example, phosphodiesterase-5 inhibitors (including sildenafil, vardenafil, tadalafil), used to treat pulmonary hypertension, have demonstrated an ability to reduce expression of amyloid precursor protein cleaving enzymes, leading to sustained reduction of Aβ levels [46]. In a transgenic murine model expressing human amyloid precursor protein and presenilin-1, phosphodiesterase-5 inhibitors halted and even rescued memory deficits, in addition to alleviating AD pathologies [47]. In other studies, antihypertensive agents including angiotensin-1 receptor blockers, angiotensin converting enzyme inhibitors, and diuretics were associated with a diminished risk for AD [48]. A comprehensive catalog of drug interactions is again beyond the scope of this perspective; however, these exemplary interactions highlight the potential for unrelated drugs to interact with AD pathologies and thereby disrupt the observation of a novel drug/disease effect.

Accounting for concurrent medications

Accounting for concurrent drugs will pose significant challenges in a typical drug trial. Without a clear appreciation of the degree and influence of concomitant agents, typical strategies for managing confounding variables are of limited utility. As with major concomitant illness, excluding patients with medication regimens with known AD indications or interactions is likely the most feasible strategy, pending further study on the influence of unrelated drugs on AD.

Alzheimer's disease heterogeneity

AD is traditionally conceptualized as a homogenous disorder, where patients are classified as either having AD or not. However, AD's shared hallmarks and symptomology may conceal diverse origins; indeed, AD may be more of a syndrome than a disease, representing a collection of different diseases. The most obvious of these differences arises when considering the influence of genetic load, which has dichotomized AD into early- and late-onset diseases. In the former, AD is characterized by aggressive neurodegeneration, culminating in earlier mortality. Late-onset AD is more prevalent and is associated with the typical, slower progressive decline of function. Other AD subtypes have been suggested based on the distribution of AD pathologies; these include limbic predominant and hippocampal sparing [49]. Differing forms of AD display characteristic prognoses and etiologies, and it is conceivable that an effective therapy against one may be ineffective against the other. Trials which pool AD patients into homogenous cohorts may therefore compromise their ability to discern a specific drug-disease effect. As suggested by Ferreira et al., efforts should be made to recognize clinical subtypes of AD, perhaps by magnetic resonance imaging, and stratify or match cohorts into analogous subtypes [49].

As a disorder causing global decline in brain function, AD also has the ability to mask other neuropathologies. Accordingly, some AD cases are subclassified as mixed dementia, in which AD pathologies coincide with other pathologies such as vascular disorders or the accumulation of Lewy Bodies. Although studies have been limited, and diagnostic criteria remain in contention, differences in cognition may be significant in pure versus mixed AD [50]. Until further is ascertained about the interactions of other pathologies with AD, we suggest that evidence of mixed dementia serves as an exclusion criteria for clinical trials of AD therapies, although further research into treatments and complications of mixed dementia should be a priority.

Conclusions

The repeated failure of promising therapies against AD suggests either an inadequacy in the search for new treatments or a failure of clinical trials to detect significant efficacy. This perspective has outlined the evidence for the presence of various hidden variables, which may detract from the observation of correlations or confound drug-disease interactions in a clinical trial. These include diet, education, occupational fulfillment, leisure participation, multilingualism, trauma, sleep, cognitive testing, coaching, stressors in clinics, concomitant illnesses, medication interactions, and the fundamental heterogeneity of AD. Although none of these variables have been definitively implicated in altering the outcome of an AD trial, their cumulative effects may be sufficient to alter the measurement and perception of drug efficacy. We therefore suggest that future clinical evaluation of AD therapeutics consider the potential impact of the outlined variables, a priori, and make due consideration in study design in analyses. Rigorously accounting for the suggested variables may dramatically increase the costs of a clinical trial as well as burden patients and caregivers—all of which may be premature, lacking definitive evidence of any interference in the outcome of a trial. We further recognize that accounting for all the variables raised may be unfeasible. However, considering the role of these variables in some capacity, be it in the formulation of participant guidelines, study analyses, or in the architecture of the trial itself, may begin to address some of the insufficiency thus far observed in AD trials.

Systematic Review: Trials for Alzheimer's disease have faced unprecedented failure for a major research discipline. Here, we review potential confounding and disruptive variables that may impede the observation of a significant drug-disease relationship.

Interpretation: Every day variables including diet, cognitive reserve (and its associated constituents), sleep, and physical activity, as well as clinical variables including family coaching, clinic environment, concurrent medications and illnesses may act as potential confounders and disruptors, if unaccounted for.

Future Directions: In spite of higher costs and study burden, future studies should consider accounting for these variables (as outlined) in their study design as they may clarify potentially significant drug-disease interactions.