Hemophilia trials in the twenty-first century: Defining patient important outcomes.

Treatment for hemophilia has advanced dramatically over the past 5 decades. Success of prophylactic therapy in preventing bleeding and decreasing associated complications has established a new standard of care. However, with the advent of gene therapy and treatments that effectively mimic sustained coagulation factor replacement, outcome measures that worked well for assessing factor replacement therapies in past clinical trials need to be reassessed. In addition, while therapies have advanced, so has the science of outcome assessment, including recognition of the importance of patient important and patient reported outcomes. This manuscript reviews strengths and limitations of outcome measures used in hemophilia from both a provider and patient perspective.

Hemophilia treatment has advanced dramatically over the past 5 decades.

Standard outcome measures of factor activity and ABR have strengths and limitations.

Studies of therapies that provide sustained hemostasis require reassessment of outcome measures.

Inclusion of patient important and patient reported outcomes is critical for meaningful studies.

INTRODUCTION

Treatment options for hemophilia have advanced dramatically over the past 5 decades, prompting changes in the choice of outcome measures to assess the value of newly proposed treatments. Outcome measures initially used in studies of early plasma‐derived and recombinant factor concentrates focused on raising factor activity (FA) levels and stopping joint bleeding. The usual design was a bioequivalence study, with a (cross‐over) pharmacokinetic assessment on 12‐15 patients and cases series showing that replacement therapy was allowing surgery without bleeding complications.

The success of prophylactic therapy in preventing bleeding and decreasing the progression of joint disease has established a new standard of care, progressively adopted since early childhood. As a consequence, most new products coming to the market sought an indication for prophylaxis, prompting the adoption of annualized bleeding rate (ABR) as the primary outcome. In addition, the study design evolved to include some form of comparative effectiveness to established treatment modalities, usually via randomization to different prophylaxis modalities.

In parallel, the advent of evidence‐based medicine has prompted a dramatic evolution in the science of outcome assessment, including the importance of patient important outcomes (PIOs) and patient reported outcomes (PROs), and their implication on patient‐centered research and care.

The advent of gene therapy and coagulation mimetics, producing unprecedented results in terms of level and durability of their clotting effect, necessitates a complete reassessment of outcomes measured in hemophilia trials in light of the progress in the science of outcome measurements, to ensure appropriate assessment of the value contributed by these advanced therapies.

In this manuscript, we review strengths and limitations of outcome measures used in hemophilia and how advanced therapies impact the validity of these measurements from both a provider and patient perspective.

BACKGROUND: AN HISTORICAL VIEW ON OUTCOME MEASURES IN HEMOPHILIA

Early studies of plasma‐derived and recombinant factor replacement products were assessed for their ability to increase the FA level,1, 2, 3 which was the primary efficacy endpoint. Other efficacy outcome measures included cessation of bleeding and surgical hemostasis.4, 5 Safety end‐points were viral safety, particularly important for plasma‐derived products studied in the 1980s and 1990s,6, 7, 8 and the development of neutralizing antibodies, termed inhibitors, which was progressively standardized as an outcome measure starting from clinical trials of recombinant FVIII (rFVIII) products in the 1990s,9, 10 to become an important secondary outcome of all therapeutic trials.

Factor activity level

FA reflects the genetic defect in F8 or F9 and is directly linked to the pathogenesis of the disease.11 There is a strong correlation between endogenous factor activity levels and bleeding tendency in hemophilia A and B.12 Almost all patients with severe hemophilia (<1% FA) have spontaneous bleeding episodes unless they are receiving prophylactic therapy and patients with mild hemophilia (>5% FA) rarely have spontaneous bleeding.13 This is why FA was historically used as a natural surrogate outcome in hemophilia studies. A surrogate endpoint has been defined as “a biomarker intended to substitute for a clinical endpoint,” the latter being “a characteristic or variable that reflects how a patient feels, functions, or survives.”14 Therefore, factor activity level fulfills the characteristics of a surrogate end point. This does not devalue a treatment able to restore normal factor activity levels, which would likely be a very good surrogate outcome strongly associated with clinically relevant end points (absence of bleeding, long‐term preservation of joint function, capacity of enjoying a normal life). On the other hand, treatment achieving lower than normal factor levels or bleeding despite a normal factor activity level would require demonstration of the strength of the association with clinical outcomes, and results of bleeding despite a normal factor activity level would require further study of the goodness of factor activity as a surrogate outcome.

FA has limitations as well. First of all, FA is not a patient relevant outcome per se. Patients, blinded to their FA level, may not experience different health statuses associated with different factor levels. Second, FA level is an imprecise measure, dependent on laboratory technique and performance quality (such that a coefficient of variation below 15% is considered optimal),15 although variation may not have relevant clinical impact at high factor levels. Third, FA measurements in samples from patients who have received modified recombinant proteins can vary by the laboratory reagents used or the type of assay, be it one‐stage or chromogenic.16, 17, 18, 19 Therefore, there is still need to prove that consistently high levels of replaced factor activity will impact long‐term outcomes in terms of joint and overall outcomes. Therefore, FA remains for now a surrogate outcome and measurement of patient important clinical outcomes is still be needed to ensure that measured FA reflects in vivo clotting capacity in a manner that is consistent with what would be expected from unmodified therapies (i.e, prevention of bleeding). Thus, while FA is an important and appealing outcome measure, particularly with therapies where higher FA levels are achieved for a sustained period of time, measures of clinical outcome are critical in assessing drug efficacy and, even more, safety.

Annualized bleeding rate

As prophylaxis has become the standard of care, and particularly with younger generations receiving prophylaxis since early childhood,20, 21 retaining better joint health became the main goal of care. Consequently, ABR has become the primary outcome in studies of new hemophilia therapies.22, 23, 24 Beyond aligning research and care goals, other reasons for this evolution in the choice of study outcomes were supporting the indication for prophylaxis, claims for premium value for engineered concentrates allowing more flexibility in the administration modalities while retaining full antihemorrhagic activity and safety,25 and in general attempts to measure some form of (clinical or convenience) benefits beyond simple bioequivalence. Not all bleeds are the same: therefore while “all bleeds” is usually set as primary outcome, joint‐specific ABR is often measured as a secondary outcome. Furthermore, the theoretical base for measuring ABR during prophylaxis is the concept of “break‐through” bleeding, which was initially proposed to reflect the overall hemostatic efficacy of treatment characterized by recurrent peaks and troughs of activity (i.e, a variable level of protection at different times). However, with overall goals of care changing to reflect patients and doctors seeking and recommending a more fulsome enjoyment of life and higher level of physical activity, bleeding (and ABR) was often distinguished in clinical trials between spontaneous and traumatic bleeding, the latter indicating bleeds not really caused by accidents, but by some form of physical activity more intense than routine. ABR is essentially a patient‐reported outcome, where the patient records the occurrence of bleeding events, their location, severity, and whether there was a precipitating event. Also, the success of treatment of a bleed (i.e, the number of doses needed to stop the bleed/provide relief from symptoms) is essentially a patient reported outcome, even if many studies had adopted a medical provider‐rated efficacy of treatment, based on number of factor concentrate infusions required.

ABR has some strengths: bleeding is patient important, both as a single event, and even more if one considers its strong association with long‐term join function (preservation of which is the ultimate goal of hemophilia treatment in the opinion of most patient and treaters). More direct measures of joint function, like the hemophilia joint health score (HJHS) or other join function scales, may be more reliable and objective,26, 27, 28 but require a much longer observation period, which is why ABR has been adopted across the board for hemophilia studies, as a patient‐important outcome per se and the most valid surrogate of long‐term joint function.

ABR has limitations too. Patient reports of bleeding, particularly joint bleeding, is by nature subjective, as joint pain and swelling may reflect arthritis and inflammation more than bleeding.29 Patients are advised clinically to treat if there is the suspicion of bleeding because under‐treatment or delayed treatment are considered to lead to faster joint deterioration, and thus may end in over‐reporting of bleeding to avoid long‐term complications. For that reason, hemophilia providers accept the likelihood of over‐treatment, particularly in patients with established joint disease. On the opposite end of spectrum, the ABR is an imperfect surrogate measure of future joint deterioration, in that it only captures clinically recognized bleeding events and does not capture subclinical bleeds,23 that regularly occur notwithstanding intensive prophylaxis. The last limitation of ABR in studies of hemostatic replacement therapies stems from their usual duration: ABR was conceived as annual rate of event to compare studies of different duration; however, while it is obvious one can reduce to 1 year studies longer than 12 month, measuring bleeds for a few months and then extrapolating to a longer period introduces a potential error due to bleeding events varying over time.30 Methods are available to annualize bleeding rate while accounting for these fluctuations,31 and applied by regulators in their internal analyses, but are not always applied when publishing study results.

THE EVOLUTION OF OUTCOME MEASUREMENT THEORY, PATIENT‐REPORTED OUTCOMES, AND PATIENT INPUT INTO RESEARCH

The importance of defining,32 choosing,33, 34 measuring,35, 36 analyzing,37, 38 and reporting39 outcomes appropriately has always been at the core of health care practice and research since evidence‐based medicine came into play.40, 41 In particular, concepts like patient important outcomes (as opposed to physiopathological outcomes), clinically important difference (in its many definitions)37, 38, 42 and the observation that important outcomes are often patient‐reported have progressively gained traction in the health care43, 44, 45, 46, 47, 48, 49, 50 and hemophilia51, 52, 53, 54, 55, 56, 57 communities. More recently, the value of direct involvement of patients in research32, 58, 59 and the need for specific strategies to harmonize patient‐relevant outcomes across studies have been proposed and adopted.60, 61, 62, 63, 64, 65, 66, 67

In brief, choosing and collecting PIOs and PROs is becoming an essential component of clinical trials.68, 69, 70 Each trial should measure PIOs, which can be clinical or PROs. It is now recognized that patients have a unique perspective and will consider issues differently than clinicians, scientists, regulators, and manufacturers. What matters to patients are outcomes that encompass the whole cycle of care: survival, functional status, and quality of life.71, 72 The value of patient participation in research design as well as with participation has been recognized.70

The potential impact of patient input is broad. Patient perspective can be integrated into clinical trial design by considering which burdens of disease and treatment matter most to patients and what aspects of trials could be better tailored for the patient subject. As drugs are considered for approval,25 or reimbursement decisions,73 patient‐important outcomes or preferences should be integrated into benefit–risk assessment. Patients can also inform how approved drug information should be communicated to patients and prescribers.49

Moving to the specific hemophilia field, O'Mahony and colleagues developed a patient‐centered framework with global applicability for assessing value in hemophilia care based on Porter's model for assessing value.74 They adapted that framework to hemophilia in the three tiers, Tier 1 addresses health status achieved or retained, including bleeding frequency, musculoskeletal complications and life‐threatening hemorrhage. Tier 2 addresses process of recovery, including timing of treatment and recovery, missing school and work, and disutility of care including inhibitor development, pathogen transmission, orthopedic interventions and venous access challenges. Tier 3 addresses sustainability of health as measured by bleed avoidance, maintenance of productive lives and good health over time.

There is currently no standard approach to PIO and PROs in hemophilia trials.53 Hemophilia‐specific health quality of life tools have been developed and are used, although not in a uniform way across studies.75, 76 These have recently been evaluated in a systematic review of their measurement properties.75 There is lack of uniformity in populations used in validation studies and in access to tools. Available translations and cultural adaptations of measurement tools further limit applicability in international studies.77 Multiple generic tools are also used in studies and are useful to help anchor hemophilia with other disease and with healthy populations. These may be more applicable as “normalcy” is the goal, although whether they still address the issues important to the patient will need to be addressed.

Among the many patient reported outcome instruments in the hemophilia space, the Patient Reported Outcomes, Burdens and Experiences (PROBE) project was developed by focus group methodology using both content experts and persons living with hemophilia.78 Through this methodology outcomes of outcomes of importance and metrics to consider for measurement were determined (Table 1). The PROBE questionnaire is comprised of four major sections (demographic data, general health problems, hemophilia‐related health problems and health‐related quality of life). The feasibility of using PROBE was assessed through a network of patient organizations in 17 countries who collected 656 surveys using 20 localized language versions. Seventy‐one percent of the participants completed the questionnaire within 15 minutes. Validation studies for the PROBE questionnaire have been completed. The questionnaire was assessed for face validity, relevance, clarity and completeness78; test‐retest reliability (reproducibility) confirmed79; a core analytic framework (psychometric properties) established80; and cross‐cultural validation demonstrated,81 and is now being incorporated into clinical trials ( https://clinicaltrials.gov/ct2/show/NCT03370913; https://clinicaltrials.gov/ct2/show/NCT03392974; https://clinicaltrials.gov/ct2/show/NCT03615053).

Table 1

Summary of outcomes of importance and metrics to consider from PROBEa

Outcomes of importance	Relevant metrics to consider
Reduced burden of living with hemophilia
Life	Family life, marital status, children
Family	Educational attendance, attainment
Education/School	Employment duration, underemployment, attendance
Employment	Impact on daily living, activities of daily living, mobility impairment, assistance required
Activities	Current health status (HRQol)
Reduced complications associated with hemophilia and treatment
Joint disease	Joint status
Pain/depression/anxiety	Pain (chronic, acute, interference with activity, timing medication)
HIV/HCV	Depression
Obesity	Resource utilization
Other comorbidities	Mortality, longevity

HCV, hepatitis C virus; HIV, human immunodeficiency virus; HRQoL, health quality of life.

From the patient‐reported outcomes, burdens, and experience (PROBE) project.78

DO ADVANCED THERAPIES REQUIRE FURTHER CHANGES ON USE OF OUTCOME MEASURES IN HEMOPHILIA?

In discussing why FA and ABR got to be selected and widely adopted in hemophilia trials, and in discussing their strength and limitations, we have unsurprisingly found the modality of treatment and the attainable goals played an important role. As advanced therapies, that is treatment moving beyond factor replacement, are expected to provide a more sustained factor activity level over time,82, 83 the relative strength and limitations of FA and ABR (as surrogate measures of long‐term effect of treatment) may change. When higher levels of FA are achieved and sustained, bleeding will more likely be the consequence of a traumatic event and not a spontaneous event, and thus not as good a measure of overall bleeding risk. Also, new outcomes may become relevant to measure new (beneficial or harmful) effects of new treatment modalities. A higher and sustained level of factor or factor like activity may introduce the need to monitor patient for thrombotic events. One can divide advanced therapies into two categories. The first are therapies that provide hemostasis without specific factor replacement. The second are gene therapies, aiming to repair the genetic defect causing hemophilia, and restore the capacity of the patient to produce the clotting factor he needs for normal clotting.

Coagulation mimetics

These include the bispecific antibody emicizumab and “rebalancing” drugs such as those inhibiting antithrombin or the tissue factor pathway inhibitor.83 For these drugs there is not yet a laboratory measure that directly correlates their administration with hemostatic activity to be used as a surrogate endpoint. Bleeding events remain the primary and important endpoints. However, as these drugs may have improved efficacy when compared to standard factor concentrates, fewer patients are experiencing spontaneous bleeding episodes. Thus, other measures are needed to reflect efficacy and we would posit should be those with meaningful functional outcomes for patients and their families.

Gene therapy

Gene therapy trials are now reporting factor activity levels in the mild hemophilia range, with a few achieving normal range,84, 85 virtually normalizing the risk of bleeding. This would indeed apply to “otherwise” healthy subjects, which we could considered being “cured” by gene therapy. Their risk of bleeding should be as low as the normal subject. However, this might not be true for older patients with establish arthropathy. For them, one may assume gene therapy would “cure” the deficiency, but not the carrying individual, as the established joint damage will still determine a limitation in function and a risk of bleeding higher than normal. In theory, for a patient with normal joints we would expect that raising the factor activity level to 0.5 IU/mL or more would completely stop bleeding. As a matter of fact, if that will be or not the case for patients with established hemophilic arthropathy is not known. Trials on tertiary prophylaxis, that is prophylaxis begun after joint damage has occurred, have shown beneficial effect, but certainly not zeroing of bleeds.86, 87, 88 However, trough levels in those trials were far from above 0.5 IU/mL at all times. With factor activity levels in the normal range, one would expect that even if bleeding was to happen, the trigger cause for the bleed is the preexisting local damage in the joint, and not the insufficient factor level. If a patient has such bad joints he will bleed no matter the factor level, he may benefit from gene therapy but less than others. The practical complication here is that different hemophilia populations may require different primary outcomes for gene therapy hemophilia trials, as the observed efficacy in terms of ABR (and long‐term effect on joint function) of the same FA may different for patient with and without preestablished joint damage. For the foreseeable future, it is unlikely that pediatric patients with pristine joint will be enrolled in gene therapy trial, and therefore the choice of outcomes will need to adapt to feasible trials on available populations. Data from ongoing gene therapy trials seem to confirm that the expressed factor is hemostatically similarly to endogenous factor, and most of the treated patient did present zero to very few bleeds and required minimal to no additional factor replacement, but data are needed to confirm this finding on a larger scale. Also, trialists may want to stratify patients based on established arthropathy in assessing the effect of gene therapy.

What are the best outcome measures in trials with high and sustained clotting activity levels?

Bleeding events in patients with mild hemophilia provide insight into outcomes that may be relevant in more effective and sustained therapies, where peaks and troughs of FA are not seen. den Uijl et al assembled observational data showing that a baseline level >12% FA would make non‐traumatic bleeding very unlikely.89 In another study, Soucie et al found more inter‐individual variability, but found similar results in that individuals with levels above 15% FA had minimal joint bleeding.90 Therefore, if gene therapy results in high and sustained hemostatic factor activity, as preliminary trials suggest, we would not expect spontaneous bleeding in patients with levels over 15% FA. On the other hand, we cannot really predict how many traumatic bleeds one would observe; first, because published data are scanty; second, because traumatic bleeds depend on level and intensity of physical activity, which may vary a lot patient to patient. In essence, ABR or any other measure of bleeding may not be able to finely discriminate the efficacy of new therapies producing sustained high level over a short period of time and achievable samples of patients. However, ABR will remain an important “safety” outcome. By this we mean that one would expect no spontaneous bleed and a very minimal number of traumatic bleeds linked to increased levels of activity. Failure of achieving these results, irrespective of any measured FA level, would require serious investigation of why bleeding events are observed.

Indeed, it may well be that a few bleeds will be viewed as being more than offset by a much deeper and complete enjoyment of the full range of life experience, but it would still be very important for decision making to know what one can expect in terms of protection from bleeding, and which additional treatment would be required for those bleeds. A FA level of 20% versus 80%, for example, would have different implications in terms of bleeding risk with major trauma or surgery.

A minor concern for using FA as a primary endpoint is variability in FVIII and FIX assay results in patients receiving gene therapy and whether levels measured reflect FA as measured in the general population. In patients receiving a B domain‐deleted FVIII replacement, FVIII measured by one‐stage assay are lower than those measured by chromogenic assay.18 On the other hand, in patients receiving F8 gene therapy with a B‐domain deleted transcript, the FVIII activity measured by one‐stage assay appears to be higher than by chromogenic assay. With the use of a gain‐of‐function F9 variant in the gene therapy construct, effects on FIX activity would be anticipated.91 While it is critical that we understand mechanisms responsible for observed discrepancies, and that bleeding activity be collected and correlated with activity, given issues with the ABR and the high sustained FA levels being achieved with gene therapy, it is reasonable to conclude that FA is the best primary short term surrogate endpoint. Long‐term (5‐10 year time horizons) assessments of efficacy (HJHS, quality of life [QoL]) and safety (thrombotic events, viral integration, neoplastic diseases, etc.) will tell us how good of a surrogate FA was. Thus measuring both factor activity and bleeding events/bleeding risk and their association are important outcome measures for patient management going forward.

Renewed need for more global patient important outcomes

Directly stemming from the above considerations is why PIOs are critical outcomes through which to assess gene therapy, beyond FA and joint function. Hopefully, gene therapy will impact the life experience of patients so deeply, that not only the number of bleeds will change, but the intrinsic “value” of a bleed will need to be redefined. With this perspective in mind, a core outcome set termed coreHEM was developed as a multi‐stakeholder project, involving patients, clinicians, researchers, regulators, drug developers, and payers.63 The frequency of bleeds, factor activity level, duration of expression, chronic pain, healthcare resource use, and mental health were identified as important core outcomes. With gene therapy there are known and unknown risks that differ from factor replacement therapy.92 It is important that patient benefits be measured well to inform benefit/risk ratios. In addition, globally harmonized mechanisms for long term follow up and data collection are needed to assess ongoing safety of this new technology.

GLOBAL PERSPECTIVE ON THE IMPACT OF NEW THERAPIES AND IMPLICATIONS FOR ASSESSING PATIENT OUTCOMES

Diagnosing and treating hemophilia in less developed countries remains a challenge, and only a small minority of patients has access to care. Measures to improve care of patients with hemophilia in less resourced countries are being sought. Recent studies have documented efficacy of prophylaxis, compared to episodic treatment, in children with hemophilia using doses lower than recommended for standard prophylaxis.93, 94 However, effective dosing in all patients is needed and use of factor replacement therapy, particularly for trauma and surgical procedures, requires laboratory expertise not available in many countries. If a safe curative approach exists, this may be a better use of resources.95 We should look to the best therapeutic approaches worldwide. This may not be repeating the drug development pathways that have been used to date in hemophilia. However, if coagulation mimetics and gene therapy are introduced into less resourced countries, we will still need efficacy and safety measures in these populations, including patient‐reported outcomes, to ensure their overall benefit in hemophilia.

CONCLUSION

The goal of hemophilia treatment is to prevent life‐threatening bleeding and long‐term bleeding‐related complications, thus allowing a normal life expectancy and quality of life. Advanced therapies are demonstrating improved efficacy with decreased disease burden, and the potential for cure. However, whilst we cannot claim a “cure” to last for a lifetime based on a 6 months trial data required for regulatory approval, we cannot call for life‐long trials in patients with perfect joint at enrollment to document the full effect of gene therapy. Over the span of a trial, FA will be an important surrogate outcome and will serve best as the primary outcome; bleed data will remain critical secondary outcomes to appraise clinical efficacy, as well as long‐term safety. Systematic collection of PIOs is critical in this process, as achieving near normal factor activity levels may change the perceived “impact” of traumatic bleeds. Fortunately, a number of tools have recently been, or are being, developed to use for this purpose. It is an exciting time in hemophilia. As new advanced therapies are introduced, we need a long‐term plan to truly capture safety and efficacy in a patient relevant manner.

RELATIONSHIP DISCLOSURES

Dr. Konkle reports grants and personal fees from Bioverativ/Sanofi, personal fees from Genentech, grants from Pfizer, grants from Sangamo, grants and personal fees from Spark, grants from Shire, outside the submitted work. Dr. Skinner reports grants, personal fees and other from BioMarin, other from Spark Therapeutics, other from Pfizer, grants and personal fees from Bayer, grants from Baxalta, now part of Shire, grants from Bioverativ, a Sanofi company, grants from CSL Behring, grants and personal fees from Novo Nordisk, grants and personal fees from Roche/Genentech, grants from Sobi, other from uniQure, outside the submitted work; and serves as Principal investigator for the Patient Reported Outcomes Burdens and Experiences (PROBE) study. Dr. Iorio's Institution has received project based funding via research or service agreements with Bayer, CSL, Grifols, NovoNordisk, Octapharma, Pfizer, Roche and Shire (formerly Baxter and Baxalta).

AUTHOR CONTRIBUTIONS

B.A. Konkle, M. Skinner, and A. Iorio contributed to the conception of the manuscript, manuscript writing and revisions and approval of the final version.