Participant outcomes and preferences in Alzheimer's disease clinical trials: The electronic Person-Specific Outcome Measure (ePSOM) development program.

INTRODUCTION: Current pharmacological interventions for Alzheimer's dementia delay symptom progression for about a year. Although the outcomes in earlier disease states may include changes in biomarkers, the clinical effectiveness of any intervention can ultimately only be assessed by a patient's self-reported well-being. A better understanding of earlier manifestations of Alzheimer's disease and the drive for relevant outcome measures, allied to technological advances in artificial intelligence, have mediated the electronic Person-Specific Outcome Measure (ePSOM) development program.
METHODS: There are 4 sequential stages in the ePSOM development program-(1) literature review, (2) focus group study, (3) national survey, and (4) development of an app for capturing person-specific outcomes. Here, we report the overall approach to the program incorporating our literature review on patient-reported outcome measures and patient preferences in the Alzheimer's disease population.
RESULTS: Alzheimer's disease trials do not use any patient-reported outcome measures. Quality of life measures are often used as proxies for this, but they do not capture individual needs. Therefore, trials currently fail to reflect the participant's aspirations for effect but rather default to clinicostatistical measure of cognition and function. There is no implementation of patient preferences despite evidence that understanding preferences may influence adherence to treatment. DISCUSSION: It is important to consider preferences for an intervention and use PROMs for the measure of effectiveness given that both risk and benefit are judged by the recipient of the treatment. The ePSOM development program will deliver the methodology for incorporating meaningful outcomes in clinical trials to expand upon current biological and clinical measurements of effectiveness.

Introduction

Dementia is a substantial public health concern, and the number of people affected is projected to increase substantially in the future [1]. There are currently four medications used to treat the symptoms of Alzheimer's disease (AD) with cholinesterase inhibition remaining the predominant target in AD pharmacology [2], [3]. Current pharmacological interventions for AD delay clinical symptom progression in people with a dementia syndrome for no longer than about a year, mediating a significant and ongoing drive toward finding effective pharmacological interventions that would prevent or delay the onset of dementia more significantly. To date, a drug's measure of effect has been based on cognition and function.

It is known from other fields of medicine that in developing new interventions and offering treatment options to patients, patient preferences toward the treatment being tested impact on treatment compliance and ultimately longer-term positive outcomes [4], [5], [6], [7]. Throughout this article, we refer to patient preferences as the thought processes patients or trial participants undertake when making health care decisions—the risk versus benefit analysis focuses on the personal expectations for desired benefits and the risks individuals are willing to take to achieve these benefits.

Outcome measures in clinical trials should reflect treatment preferences to assess whether an intervention offers a real benefit to patients while keeping potential risks at a level that trial participants would tolerate. The “real benefit to patients” should ultimately and ideally be set by the patients themselves using patient-reported outcome measures (PROMs). It is noteworthy that risks versus benefit evaluations are currently determined on behalf of patients by third parties, for example, study sponsors, investigators, ethics committees, payers, or regulators without systematically taking into account the individual's perspectives or being able to refer to data in the trial to inform these evaluations. Even if some patient input is included on committee levels, patient advisors are not reflective of the patient population more widely. This ignorance of the views of the research participant in clinical trials (who will be the future target population for the intervention) need not continue ad infinitum if a solution was found to effectively and accurately measure the participants' choice of outcomes and how these balance against how much risk they are willing to accept to achieve these.

As interventions move earlier in the disease course (e.g., via programs like the European Prevention of Alzheimer's Dementia) [8], disease-modifying trials now employ more sensitive cognitive and functional outcomes that are expected to correlate more closely with relevant early biological changes [9] and may also be more closely correlated to a participant's ability to maintain important patient-reported outcomes or even recovery of these. The misconception that cognitive or functional outcomes cannot be measured at such an early disease stage to reflect treatment success has led to a tendency to seek surrogate biomarker outcomes. As shown in Fig. 1, the traditional understanding has been that biomarker changes precede changes in cognition and function, which in turn precede changes in PROMs. One example of a PROM is a tool we are developing—an electronic Person-Specific Outcome Measure (ePSOM), which would allow us to look at correlations in specific elements of a PROM and link it with a specific biomarker. Fig. 1 highlights that different outcomes are best applied at different stages of illness. While in oncology and other disease areas there is a close correlation between disease staging and concurrent symptoms and prognosis (as measured using, e.g., biopsy), then this is lacking in AD. We note that currently used biomarker outcomes are “two steps” away from PROMs. In Fig. 1, we illustrate the two steps by which biomarker outcomes are away from PROMs outcomes in terms of surrogacy. An ideal surrogate outcome gives confidence that a change in the biomarker will be associated with high likelihood of a later change in the main outcome of interest. If we consider PROMs as the ideal outcomes of interest, then currently biomarker change will give little assurance of a later change in well-being of the patient. Given that outcomes are so loosely tied to each other, it is impractical for a patient to base a preference decision on such indistinct outcomes such as biomarkers and cognition. Using outcomes in AD trials that reflect the biological and cognitive consequences of disease may gain in objectivity but lose out in terms of relevance to the person participating in the trial; and although using PROMs in preclinical or prodromal disease may seem challenging, it is in this very population where there is a clear and immediate need to understand the patient's perspective and the risks they would be prepared to take to delay dementia onset especially when the interventions being currently trialled may have more safety issues than existing treatments.

Fig. 1

Relationship and challenges between preferences and PROMs at varying stages of AD. (A) Assumption that disease-modifying therapies are more likely to have impact on earlier stages of neuropathology but (without clear cut evidence linking biology to symptoms) least likely for person to tolerate risk of intervention. (B) indicates that the ideal measure of effect, that is, a PROM as measured now, is poorly correlated with the target of a disease-modifying therapy—making it impossible to use a PROM as a primary outcome for trial BUT also means that an effect on a biomarker may have little relevance to the patient. (C) The temporal sequence refers to the fact that a change in biomarkers, and therefore, risk status may not yield any observable benefit to the patient for several years after the intervention commenced. Abbreviations: AD, Alzheimer's disease; PROM, patient-reported outcome measure.

It is highlighted in the recent United States Food and Drug Administration (FDA) guidelines that as AD drug trials move to an earlier phase in the disease, we must now develop optimal measures to assess treatment benefit [10], [11] as well as necessarily incorporate patient preferences in the process [12]. So far, however, technical and statistical limitations have hampered the use of PROMs in clinical trials. It is critical therefore that purposeful, large scale research is undertaken to identify person-specific outcome priorities, understand patient preferences, and develop a system for capturing these that can be incorporated routinely in clinical trials with the necessary validity for regulatory acceptance.

Measuring meaningful benefits in AD clinical trials

PROMs reflect an individual's views on what they define as an effective treatment and, importantly, capture benefits specific and meaningful to that particular person. Other fields of medicine already incorporate PROMs developed specifically for the target population, for example, oncology (see [13], [14]). However, a recent review looking at outcome measures used in disease-modifying trials in AD [15] concluded that current AD clinical trials do not use PROMs and rely solely on other domains to measure drug efficacy. There may be several reasons for this. Some research studies accept activities of daily living and quality of life (QoL) measures as a proxy for PROMs. In our view, QoL measures are too generic and not specific to a person as well as often incorporate a carer perspective rather than relying on the person. We also note that QoL measures are developed through population-level research and therefore lose the specificity to an individual. Furthermore, relevant to our objective, QoL measures are predominantly used in the Alzheimer's dementia populations so would not be appropriate for use in the prodromal stages of AD. The use of other proxy measures of PROMs (e.g., cognition) may reflect a propensity or tradition for measuring effectiveness from a clinical as opposed to a patient perspective. Finally, health-related outcomes measures or health utility indexes are often used as proxies for patient preference measures, although the validity of these measures to accurately report patient views on benefit versus risk analysis and experience in treatment efficacy is uncertain (e.g., [16]). One can therefore see that there is a pressing need to apply scientific rigor and purpose to find a means to measure PROMs in people across the AD spectrum and to use these outcomes to link with treatment preferences.

In 2007, the International Psychogeriatric Association produced a consensus statement on defining and measuring treatment benefits in dementia trials [17]. The statement concludes that outcomes can include effects on people with dementia regarding cognition, behavioral and psychological symptoms, QoL, global assessments, and activities of daily living and that these measures should account for the cultural context and education of the population. PROMs were not mentioned in this statement produced over 10 years ago. However, given recent developments and regulatory shifts, we consider that effectiveness of an intervention should be defined by maintenance of well-being. In AD, this concept could be defined specifically by the patient as a PROM. We are not alone in this view. A recent review notes that newly developed treatments across the AD spectrum for approval by the FDA should provide evidence that the treatment translates into “meaningful benefits for the treatment group” [18]—a position supported by the EMA [19].

Although empirical evidence of patient preference in AD with a real-world clinical trial population is very limited, the value of this information may be substantial regarding decisions on drug development and later marketing. A study of healthy older Americans' perception of AD explored preferences were they to undergo hypothetical AD treatment, showing that participants were willing to accept serious adverse-event risks in return for effective disease-modifying benefits of new drugs [20]. A recent meta-analysis summarizing evidence from 33 cohort surveys about the general public's attitude toward AD found that belief in the value of seeking treatment was high [21]. However, despite these positive attitudes to intervention in the general public, reviews involving a clinical population who are already symptomatic have found that adherence to AD medication is low [22], which may reflect a lack of clear understanding of preferences in the originating clinical trials and lack of relevance of clinical outcome measures to reflect the patient's perspective. In essence, a lack of understanding of the clinical trial about the research participant's view of the risks and benefits of intervention and how they may meaningfully benefit from it could mediate low adherence after the drug is launched.

Given (1) the requirements now being established for clinical trials in AD, (2) the desire to improve long-term adherence and compliance with new medications, and (3) technology advancements in artificial intelligence creating opportunities for the assessment of PROMs, we established the ePSOMs program.

The aim of the ePSOM development program

In this article, we present the background underpinning the ePSOM development program. There are four sequential stages in the program: (1) literature review, (2) focus groups to elicit what matters to people when developing new treatments for AD [23], (3) design and launch of a national survey, and (4) incorporation of all knowledge gained in steps (1–3) to ultimately lead to the development of an app for capturing longitudinally person-specific outcomes and preferences.

The objective of the ePSOM development program is to deliver a valid and reliable technical solution that has regulatory compliance for use in clinical trials in people with AD, optimized for people with preclinical or prodromal Alzheimer's dementia. An interactive dialog system [24] for gathering patient feedback is envisaged, which will be deployed as an app on networked mobile devices (smartphones, tablets) and conventional Web browsers. This dialog system will include methods for data gathering through different input modalities, including typing, touch screen, and speech interaction. Speech interaction allows the software to capture not only what patients said but also how they said it. Once these data have been collected, statistical learning methods will be employed to extract outcome preferences information, as well as analyze the patient's voice for cues related to these preferences. Although usability and implementation aspects need special attention in the deployment of these technologies [25], it has been acknowledged, in other fields of medicine, that electronic methods for collection of PROMs have the potential to minimize costs and improve the effectiveness of capturing patient feedback [25], [26].

Stage 1: Literature review

Methodology

As the first step in the program, we conducted a thorough literature review on PROMs and patient preferences as they were used in clinical trials or other research settings in the AD population. We specified AD because that is the context where our ePSOM will be implemented. As much of this literature dates back over the last 20 years, the term mild cognitive impairment (MCI) is used to describe the prodromal stage in the AD population.

The first search was to identify papers referring to PROMs in the AD and MCI population in clinical trials (see Appendix 1); the second search was to identify papers referring to patient preferences in the AD and MCI population (see Appendix 2). EMBASE database was searched from 1980 until 06 August 2018 using OVID SP. Reference lists of included papers and review articles were also checked, and an additional search was done on Google Scholar. Articles published in English were included.

Search I—PROMs

Inclusion criteria

Primary research paper

Included participants with MCI or subjective cognitive impairment or Alzheimer's disease—OR healthy volunteers hypothesizing about MCI or Alzheimer's disease

Used a PROM or goal-setting measure

Exclusion criteria

Study population without neurocognitive disorders

No PROM or goal-setting measures used

Search II—preferences

Primary research papers

Include participants with MCI or subjective cognitive impairment or mild Alzheimer's disease—OR healthy volunteers hypothesizing about MCI or Alzheimer's disease

Refer to patient preferences in the context of health care decisions.

Study population without neurocognitive disorders

No explicit patient preferences (e.g., only comparison between patient and caregiver ratings in various scales)

Results

Search I—PROMS

The search yielded 41 results. The titles and abstracts of the 41 papers were reviewed against the inclusion/exclusion criteria, which left three papers. The majority of papers excluded were either review papers, opinion pieces, or were not relevant to the AD population, for example, focusing on HIV or a rehabilitation setting. Additional search strategy was then adopted by reviewing the reference lists of the remaining papers—this resulted in 2 new papers though they both used a PROM, which was already represented in the main search result. After a review of full texts was carried out, five papers referring to three different PROMs were included in the review.

Therefore, a total of three different PROMs used in the Alzheimer's disease or MCI population were identified ([27], [28], [29], [30] Appendix 3: PROMs literature review summary). None of these have been approved by regulatory bodies as a PROM in AD trials. A recent review and consensus statement we identified [15] looked at outcome measures currently used in AD trials up to 2016 and concurred with our observations. Their search yielded 125 previous and ongoing trials with 81 different outcome measures broadly fitting into six domains. However, the review did not identify any PROMs currently used in clinical trials. These findings were corroborated by another review looking at dementia studies [31] where no PROMs were identified.

Search II—preferences

The search yielded 625 results. The titles and abstracts of the 625 papers were reviewed against the inclusion/exclusion criteria, which left 31 papers. The majority of papers excluded were not relevant to the AD population focusing on health care decisions generally or had no primary data. Reviewing the reference lists of the 31 papers yielded three new papers. After a review of full texts was carried out, 15 papers referring to patient preferences in the Alzheimer's disease or MCI population were included ([32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46] Appendix 4: Preferences literature review summary).

In the literature search for patient preferences in the AD population, the 15 papers we identified were both quantitative and qualitative in study design, using a variety of methods for eliciting patient preferences. All the papers identified explored patient preferences in a hypothetical scenario, and none of the papers used patient preferences in conjunction with enrolling or enrolled participants in a clinical trial.

Summary of literature review

The reviews conducted highlighted the paucity of evidence relating to the direct use of PROMs or preferences in clinical trials in AD. Three measures of PROMs have been developed which, on the basis of the importance afforded to these measures by regulators and others, reflects a very small number compared to, for example, the number of cognitive or functional measures that have been developed and are routinely used in AD clinical trials [14]. Although more papers were included in our summary of preferences work, there was no evidence of any of this work being conducted in a clinical trial—rather all the work was either of hypothetical scenarios being tested using qualitative or quantitative methods in a variety of nontrial populations such as questionnaires, surveys, and, predominantly, focus group discussions. We note that there is no analysis of the statistical challenges in measuring patient preferences in the AD population. There was no clear indication from our review of the barriers to implementation of this work into clinical trials though one could speculate that ease of use, psychometric insufficiencies, and bias toward or familiarity with using traditional outcome measures all play a part.

Discussion

A need for electronic Person-Specific Outcome Measure (ePSOM)

The current review identified a lack of PROMs being used in AD clinical trials despite the fact that incorporating outcome measures to reflect clinically meaningful benefits are recommended by regulatory bodies.

PROMs and cognition

Cognitive impairment is the key diagnostic criterion in existing definitions of AD though multiple other symptomatic domains exist; for example, neuropsychiatric, behavioral, and functional. Cognition though predominates in the conceptualization of neurodegeneration and is therefore used as the primary outcome measures in clinical trials to measure success. The use of cognitive outcomes as the primary measure of success may have been reinforced as a consequence of the first successful pharmacological interventions showing a particular benefit in minimizing decline in cognitive symptoms, which has endorsed this outcome domain in all subsequent drug development. However, AD and other dementia syndromes are experienced in a more varied manner by the patient and caregiver with impact of the disease on a range of cognitive, functional, behavioral, and neuropsychiatric outcomes all of which will interplay and impact on well-being. Therefore, in addition to measuring cognition, trials in an early AD and MCI population would ideally need to incorporate sensitive measures that capture treatment efficacy or clinically meaningful benefit as experienced by the individual trial participant. Elsewhere, clinically meaningful benefit has been defined as a favorable, statistically robust effect on how a patient feels, functions, and/or survives [47]. It could be argued that a PROM would accord with this definition more closely than, for example, memory outcomes.

PROMs and biomarkers

Recently, in addition to cognitive measures, there has been an emergence of the use of biomarker outcomes in clinical trials, but as suggested earlier (Fig. 1), biomarker changes are probably even less well linked to PROMs than cognitive and functional symptoms. The emergence of biomarker-driven decision-making leads to the patient being communicated benefit (or lack of) via a hard to contextualize outcome, for instance, a change of value in a biomarker. Becoming amyloid “positive” from originally “negative” though may have very little direct impact on, for example, ability to drive in a preclinical or prodromal patient. With the ePSOM program, we aim to develop and understand PROMs at multiple stages of disease, that is, in the at-risk, MCI, and mild dementia populations and use this information to inform the development of a psychometrically robust outcome measurement platform for use in clinical trials. We will within the program also be able to explore the relationship between ePSOM-derived PROMs and disease states through application of this outcome measure in, for example, the European Prevention of Alzheimer's Dementia [8] or PREVENT Dementia [48] projects where much detail is collected on biomarkers and sensitive measures of cognitive change in preclinical and prodromal populations (Table 1). This will allow us to determine the degree of surrogacy between biomarkers, cognition, and PROMs (Fig. 1). One example we could explore would be the degree to which changes in hippocampal subfields (biomarker) are associated with allocentric visuospatial memory (cognition) and ability to drive (PROM/ePSOM).

Table 1

Barriers to implementation of PROM outcomes and preference analysis in AD clinical trails with solutions from the ePSOM development program

Current problems with PROMs in AD research	ePSOM solution
1. Psychometric properties	We are working with regulatory bodies (EMA) and biostatistical expertise to develop a valid and reliable PROM
2. Applicability in clinical trials	The ePSOM app will have an ease of application through various stages of development including field-testing in EPAD.
3. Tradition of using cognitive and functional measures	Delivery of (1) and (2) addressing those concerns leading to incorporation of meaningful outcomes to the trial participant in clinical trials
Current problems with patient preferences in AD research	ePSOM solution
1. Measuring “benefits” using biomarkers despite poor association with clinical condition and PROMs	Applying the ePSOM app to deeply phenotyped cohort studies to investigate the association between biomarkers and cognitive outcomes and PROMs (cross-sectionally and longitudinally)
2. Qualitative similarities in cognitive and functional decline in normal aging and those due to AD pathology
3. Patient's perception of illness	Incorporate patient's perception of illness measures in the ePSOM app

Abbreviations: AD, Alzheimer's disease; ePSOM, electronic Person-Specific Outcome Measure; PROM, patient-reported outcome measure.

PROMs and preferences

Relating PROMs to other outcome measures used is a prerequisite in the ePSOM program whereby we create preferences methodologies that can be applied in clinical trials. Our review has demonstrated a distinct lack of previous research that has as its goal the implementation of PROMs or preferences in clinical trials. We argue that the preference scales that exist outside of AD while being generic for clinical trials do not address the complexities and intricacies of AD pathology. In AD, there remains a high probability that people who test positive in certain biomarker assessments do not decline to dementia [49]. In contrast to, for example, cancer assessments, this leads to greater uncertainty in what is conveyed to the patient and how treatment decisions and trade-offs are made. Other fields benefit from categorical outcomes like death or tumor growth/resolution/spread. Therefore, in AD, the ability to correlate a biomarker or cognitive outcome to a PROM is critical but currently lacking and therefore undermining the continued use of biomarkers and cognition as the “benefit” to balance against the “risk” of the intervention from a patient perspective.

A further consideration in the use of preferences in dementia is the challenge of distinguishing changes between function associated with the development and increase of AD pathology and those associated with aging-related changes or other clinical conditions. It would be necessary to establish whether decline in any outcome measure in an individual occurs as a consequence of normal aging or due to pathological changes consistent with neurodegenerative disease. This again is different in, for example, cancer where the presence of illness is unambiguous and it is possible to clearly determine what constitutes as pathology and how tightly correlated this is to symptoms. Therefore, because of the underlying uncertainty around the diagnostic accuracy of biomarkers of AD (particularly at an earlier preclinical or prodromal stage of the disease continuum) as well as the overlap in “symptoms” with normal aging and neurodegenerative disease, it is not appropriate to fully replicate the methods used in other disease areas to elicit patient preferences in AD. The application of our ePSOM in deeply phenotyped cohort studies, for example, European Prevention of Alzheimer's Dementia will help to address empirically the association between novel biomarkers, sensitive cognitive measures, and specific PROMs.

Conclusions

To date, efficacy outcomes used in AD trials have been uniquely cognitive or functional. However, there remains debate about the clinical meaningfulness of these outcomes, which are chosen by third parties, for example, study sponsors, investigators, or regulators. PROMs use is conceptually preferable and generally advocated as is how these drive preferences for treatment (benefit versus risk), but methodological problems have hampered their use, so there is an expedient default back to cognitive and functional proxy measures of brain disease.

Here, we report the first stage (literature review) and the fundamentals of a wider research program developing and validating an electronic person-specific outcome measure. This will be in the form of an app to capture clinically meaningful treatment efficacy to be used in a prodromal AD population. Concurrently, we are developing the biostatical approaches to assessing preferences learning from other branches of medicine but adapting these to the specific challenges outlined above in the AD field. Ultimately, we will be applying the ePSOM in cohort studies as an end point to support other outcome domains.

So far, the use of PROMs has been limited due to time burden in a clinical trial being a key factor. Moreover, the psychometric properties of these measures need to be such that regulators, investigators, and payers can agree on their validity and reliability as outcomes in clinical trials. In the light of these concerns, the ePSOM development program will result in a bespoke app with a flexible multimodal interface that will generate valid and reliable outcomes regarding the effectiveness of an intervention to trial participants. This will innovate the outcome measures used in AD clinical trials to match the scientific advances in our understanding of the disease pathology and ensure that the interventions that come to market in the future benefit patients in a way that they both recognize and value.

Systematic review: As the first step of the electronic Person Specific Outcome Measure (ePSOM) development programme, we carried out a literature review on patient reported outcome measures (PROMs) and patient preferences used in clinical trials or other research settings in the Alzheimer’s disease (AD) population. EMBASE database was searched from 1980 until 06 Aug 2018 using OVID SP. We aimed to identify papers referring to PROMs or patient preferences in the AD and mild cognitive impairment (MCI) population in clinical trials in order to evaluate whether PROMs or patient preferences are currently used in the AD population.

Interpretation: Currently, AD clinical trials do not use PROMs though the need for clinically meaningful outcomes is supported by regulatory bodies e.g., European Medicines Agency and FDA. Quality of Life and Activities of Daily Living scales are currently used as proxies for PROMs though there are inconsistencies in caregiver-patient ratings and the scales’ validities as PROMs. To assess treatment preferences regarding the trade-offs in desired benefits versus acceptable risks, no specific scales exist in the Alzheimer’s disease population. However, it is critical to understand how individuals make health care decisions and include patients’ preferences for treatment options in order to reflect meaningful results in clinical trials and yield better compliance to treatment. This would mean accounting for the risks versus benefits analyses trial participants or patients make for each option.

Future directions: The ePSOM development programme aims to explore meaningful outcomes and preferences that matter to patients in assessing drug efficacy in AD. This paper outlines the framework for a large research programme ultimately aimed to develop an ePSOM app employing artificial intelligence technologies that can be incorporated in AD clinical trials.