Nosebleeds in hereditary hemorrhagic telangiectasia: Development of a patient-completed daily eDiary.

OBJECTIVE: A prospective, qualitative study was conducted to develop a patient-reported outcome measure (PROM) for daily administration via electronic diary (eDiary) to assess the severity of nosebleeds in patients with hereditary hemorrhagic telangiectasia (HHT), in accordance with Food and Drug Administration (FDA) PROM guidance criteria.
METHODS: Three expert clinicians who treat patients with HHT provided input during instrument development, which comprised: 1) Peer-reviewed literature and instrument review; 2) Development of draft Nosebleed Diary items; 3a) Three rounds of qualitative interviews (two with a paper-based diary, one with an eDiary) with patients with documented severe epistaxis related to HHT, for concept elicitation and cognitive debriefing; 3b) Face validity and translatability assessment; 3c) Patient evaluation of the usability and acceptability of the eDiary device; and 4) Preparation of the final Nosebleed eDiary and conceptual framework.
RESULTS: No existing instruments were identified that evaluate HHT-related nosebleed severity daily and meet FDA PROM guidance criteria. Frequency, duration, and/or speed of flow (i.e., intensity) were reported by most participants with HHT when asked to describe their nosebleed severity. The Nosebleed eDiary was refined based on 17 patient interviews, clinical expert input and the face validity and translatability assessment. The final four-item eDiary was acceptable to patients with HHT.
CONCLUSION: The Nosebleed eDiary is "fit for purpose" to assess the severity of HHT-related nosebleeds, and has established face and content validity. Further adaptation may be required for use in mild or moderate HHT populations. Psychometric testing to evaluate construct validity and reliability are recommended next steps. LEVEL OF EVIDENCE: 2c "Outcomes research".

INTRODUCTION

Hereditary hemorrhagic telangiectasia (HHT) is a dominantly inherited genetic vascular disorder with an estimated worldwide prevalence of ∼1 in 5000 individuals.1, 2 HHT is characterized by vascular malformations with direct artery‐to‐vein connection and loss of intervening capillaries. These arterio‐venous malformation (AVM)‐type lesions can occur in multiple vascular beds, but typically occur in the oral, nasal, and gastrointestinal mucosa.

Beyond the infrequent catastrophic vascular accidents associated with major organs that generally occur in childhood, the primary adult phenotype is nosebleeds (epistaxis). Recurrent epistaxis, caused by the presence of numerous mucosal telangiectasias, is the most common symptom, developing in ∼95% of individuals with HHT by 50 years of age.1

Epistaxis episodes disrupt the lives of patients with HHT physically, emotionally, socially, and professionally.3, 4, 5 Individuals with HHT often develop anemia, with its attendant symptom of fatigue, as a consequence of recurrent epistaxis and GI bleeding.1 In previous studies, the health‐related quality of life (HRQoL) of patients with HHT was significantly poorer than that of reference populations, largely due to epistaxis.5, 6, 7, 8, 9, 10 Previous attempts to quantify the extent and impact of epistaxis include the development of an epistaxis‐specific quality of life questionnaire (EQQoL)11 and the Epistaxis Severity Score (ESS).12 The EQQoL focuses on the psychosocial impact of epistaxis. It was originally intended as a disease‐specific measure of HRQoL to supplement data from the 36‐item Short Form Health Survey in clinical research,11 but no published evidence on its use was found. The ESS, developed as a patient‐reported outcome measure (PROM) of epistaxis severity, evaluates the “typical” frequency, intensity, and duration of epistaxis episodes, as well as the need for medical attention and blood transfusions, over the past 3 months.12 However, HHT‐related epistaxis varies frequently such that the ESS, with a three‐month recall period, may lack the required sensitivity to capture small, but potentially important, changes in HHT‐related epistaxis following treatment, which may limit its use in clinical research.

In December 2009, the US Food and Drug Administration (FDA) released guidance on the use of PROMs in medical product development to support labeling claims.13 This document provides recommendations for the development and psychometric evaluation of a PROM, as well as the required documentation to support the use of PROMs in regulatory approval or promotional claims. Based on recent discussions regarding self‐reported assessment of recurrent HHT bleeds, the FDA urged for daily recording of epistaxis events to accurately capture the patient experience and to minimize recall bias.

This manuscript describes the development of a diary‐based electronic PROM, the Nosebleed eDiary, to assess daily the severity of nosebleeds in patients with HHT, in accordance with FDA PROM guidance criteria13 and the International Society for Pharmacoeconomics and Outcomes Research Patient Reported Outcomes (ISPOR PRO) Good Research Practices Task Force reports.14, 15

MATERIALS AND METHODS

Qualitative research with patients with severe epistaxis associated with HHT was performed by PRO instrument development experts (MC, SM, NH) in conjunction with feedback from clinical experts (JH, DP, and SO from the fields of pulmonology, otolaryngology, and ophthalmology, respectively) who treat patients with HHT (Fig. 1).

Figure 1

Overview of the Nosebleed eDiary development process.

Targeted Literature Review

A targeted review of the literature was performed to confirm patient‐reported constructs and identify PROMs assessing the severity or intensity, frequency, and duration of epistaxis associated with HHT. Using a detailed search strategy (see Supplementary Methods), we reviewed peer‐reviewed literature in the PubMed and PsychInfo databases, studies listed on the ClinicalTrials.gov website, and abstracts presented at the 2011 HHT 9th International Scientific & Medical Conference, focusing on publications in English between 2004 and 2014.

Recruitment

The HHT Foundation International, a patient advocacy organization, assisted with patient recruitment for the qualitative interviews. A sample size of 18 participants was targeted to achieve conceptual saturation, i.e., the point at which no new information is elicited from subsequent interviews. Efforts were made to recruit a patient sample that was diverse in sex, education, and race.

Individuals between 18 and 75 years of age reporting a physician diagnosis of HHT were included if they had severe epistaxis in the previous 4 weeks, defined as an average of at least three nosebleeds per week of a total duration greater than 15 minutes per week that required iron therapy (oral and/or intravenous). Participants had to be willing and able to participate in a 1‐hour interview, be able to read, speak, and understand English, and to provide informed consent.

Relevant concepts and a preliminary item pool for the Nosebleed Diary were developed based on the ESS,12 results from the targeted literature and instrument review, and feedback from the FDA regarding an investigational new drug application (including results from a phase 2 study; Clinical Study Identifier NCT02204371). The draft focused on core epistaxis elements, daily recording, elements describing intensity (e.g., gushing or pouring), and use of an eDiary format.

Interview Procedure—Concept Elicitation and Cognitive Debriefing

To evaluate and refine the Nosebleed Diary items, qualitative, combined concept elicitation/cognitive debriefing interviews of participants with documented HHT‐related severe epistaxis, were conducted at research facilities in Michigan (round 1), North Carolina (round 2), and Massachusetts (round 3).

A protocol was developed for this qualitative research study in collaboration with three clinical experts. All study materials were reviewed and approved by RTI International's institutional review board committee.

All patient interviews followed a semi‐structured interview guide. Participants were asked to describe their experiences with HHT‐related nosebleeds using a series of open‐ended questions, for example, participants were asked to describe the most bothersome aspect of their nosebleeds. The objective of this component of the interview was to elicit spontaneous descriptors of nosebleed severity (e.g., frequency, duration, and intensity) and how they were determined (i.e., concept elicitation).

Participants were then given a draft version of the Nosebleed Diary and asked to engage in cognitive debriefing of the instructions, questions, response options, and recall period. A “think‐aloud” format was used to gather information about participants’ interpretations of each item and its instructions, and about the process they used to select each response. Participants were also asked which response wording was most applicable to their experience, and to identify the most and least important items for assessing their nosebleeds, as well as items that were missing or could be omitted.

During the first two rounds of interviews, participants reviewed the Nosebleed Diary on paper. For the third round of interviews, participants reviewed the Nosebleed Diary in an electronic format (i.e., eDiary) using a handheld device with a touchscreen. The usability and acceptability of the electronic device (i.e., whether respondents could use the electronic device and software appropriately) was assessed. Transcripts were verified through an iterative process of technical and editorial review.

Face Validity and Translatability Assessment

Following revisions to the draft diary after round 1, a face validity and in‐depth translatability assessment was conducted to facilitate future translation and wider regional use of the Nosebleed eDiary. Translation and linguistic validation experts assessed content validity (i.e., the extent to which the instrument measures the concept of interest) of the original English source text. Sixteen linguists from six different language groups then reviewed the items to identify concepts, phrases, or components of the items that might be difficult to translate or could be culturally‐specific. The assessment targeted the following six geographic regions: Northern Europe, Southern Europe, Eastern Europe, Eastern Asia, the Middle East, and Australia. Based on the results of this assessment, the draft diary was modified to improve translatability, prior to conducting round 2 of the interviews.

Analysis

After each round of interviews, the interviewers conducted qualitative analysis of the field notes and transcripts, focusing on whether the content of the draft Nosebleed eDiary was relevant, appropriate, understandable, and interpreted consistently by participants. The draft questionnaire was subsequently revised and evaluated in rounds 2 and 3.

Based on input from round 3 interviews and feedback solicited from the three clinical experts, the Nosebleed eDiary was revised and finalized.

RESULTS

Peer‐Reviewed Literature and Instrument Review

In total, 36 unique abstracts were identified from PubMed and PsychInfo, of which 12 full‐text articles were reviewed for patient‐reported constructs and/or instruments assessing the severity/intensity, frequency, and duration of epistaxis associated with HHT in the context of qualitative research. Ten interventional studies listed on ClinicalTrials.gov included measures evaluating one or more aspect of HHT‐related epistaxis. Five abstracts describing interventional studies presented at the 2011 HHT 9th International Scientific & Medical Conference were identified and reviewed.

Ten epistaxis‐specific PROMs for use in patients with HHT were identified (Supplementary Table 1).12, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33 Many of the PROMs assessed severity by the intensity, frequency, and/or duration of epistaxis. However, only a few assessed all three concepts of interest. A diary implemented by de Gussem and colleagues assessed all concepts of interest, including severity defined as the nosebleed intensity, ranging from 1 = drops of blood to 3 = large gush.29 The ESS, the most recently developed epistaxis‐specific PROM, assessed all three epistaxis concepts of interest,12 and was the most frequently cited PROM in the literature12, 16, 17, 18, 19, 20 and in HHT trials listed on ClinicalTrials.gov (NCT01408732, NCT01397695, NCT01314274, NCT01408030).

Table 1

Participant Characteristics

	Sites
Characteristic	Round 1 Michigan (n = 5)	Round 2 North Carolina (n = 6)	Round 3 Massachusetts (n = 6)	All (N = 17)
Sex, n (% for all patients)
Female	2	5	4	11 (64.7)
Male	3	1	2	6 (35.3)
Age, years
Mean	58.8	50.7	56.2	55
Range	46–69	37–66	41–64	37–69
Race/ethnicity
White	4	4	5	13 (76.5)
African American/Black	1	2	0	3 (17.6)
Hispanic	0	0	1	1 (5.9)
Education, n (% for all patients)
High school diploma or equivalent	0	0	1	1 (5.9)
Some college	2	1	0	3 (17.6)
College degree	1	3	4	8 (47.1)
Professional or advanced degree	2	2	1	5 (29.4)
Employment status, n (% for all patients)
Full time	3	3	2	8 (47.1)
Unemployed/disabled/retired	2	3	4	9 (52.9)

No existing PROMs were identified that evaluated HHT‐related nosebleed severity daily and were developed and validated consistent with the FDA PROM Guidance.12

Nosebleed eDiary Development

Participants

In January, February, and March 2015, 17 adults with severe HHT‐related epistaxis completed combined concept elicitation/cognitive debriefing interviews. The mean age of participants was 55 years, 11 (65%) were female, and 4 (24%) were black or Hispanic. Most participants had a college or graduate degree (77%), and roughly equal proportions of the sample did not work (53%) or worked full time (47%) (Table 1).

Concept Elicitation

Description of nosebleed severity

When describing the severity of their HHT‐related nosebleeds, the majority of participants across all three rounds of interviews spontaneously reported frequency (how often nosebleeds occur), duration (how long the nosebleed lasts), and/or speed of blood flow (i.e., intensity) (Table 2). Duration was the most frequent spontaneously reported descriptor of nosebleed severity (n = 15), closely followed by intensity (n = 13) and frequency (n = 12). When probed, intensity and duration (n = 1) and frequency (n = 1) were reported as related to severity for two participants who did not spontaneously report those concepts. In round 1, four of the five participants stated that they considered the amount of blood during each nosebleed when evaluating the severity or intensity of the nosebleed. Four participants associated the need to go to hospital with the level of nosebleed severity. Finally, the formation or size of blood clots was factored into nosebleed severity for two of the 17 participants.

Table 2

Patient‐Reported Descriptions of HHT‐Related Nosebleed Severity

Round #‐ Patient #	Intensity	Frequency	Duration	Other terms for severity (S/P)
1–1		S	S	Amount— i.e., “towels full of blood,” “stomach full of blood,” “sink full of blood” (S)
1–2	S	S	S	Amount—i.e., “I don't feel it was uncommon that I lost 2 pints of blood or more” (S)
1–3	S		S	Amount and need to go to hospital—i.e., “I had to call 911 because I lost so much….” (S)
1–4	S	S	S	“Nothing that has landed me in a hospital, so nothing I would classify as traumatic or severe, along those lines”(S)
1–5	S	S	S	Amount—i.e., “a few drops” (S)
2–1	S		S	None
2–2	S	S	S	None
2–3		S	S	“Blood clotting in the back of my throat” (S)
2–4	S		S	None
2–5		S	S	None
2–6	S	S		None
3–1	S	P	S	“Mild to moderate” (S)
3–2	S	S	S	“Hospitalized” (S), “passing out” (S)
3–3	P	S	P	“longer it goes, more anxiety, losing more blood” (S)
3–4	S	S	S	“ping pong‐sized” clots (S)
3–5	S	S	S	None
3–6	S		S	“I've had one where I actually had to go to the hospital” (S)
Total	S (n = 13) P (n = 1)	S (n = 12) P (n = 1)	S (n = 15) P (n = 1)

HHT = Hereditary Hemorrhagic Telangiectasia; P = mentioned on probing; S = mentioned spontaneously.

When asked to rank the aspects of nosebleed severity (e.g., intensity, frequency, or duration) that were most bothersome (from 1 = the most bothersome to 3 = the least bothersome) based on individual aspects each participant had previously reported, nosebleed duration was most commonly reported as most bothersome (8/14 participants), followed by intensity (6/12), and frequency (3/12) among those who responded to this question.

Conceptual coverage

Participant feedback on the Nosebleed Diary items during cognitive debriefing corroborated the results of the concept elicitation, and confirmed that the items of the Nosebleed Diary encompassed all aspects considered by participants to be relevant and important for the assessment of HHT‐related epistaxis severity. Completion of the Nosebleed Diary each time a nosebleed occurs measures the frequency of nosebleeds. During the first round, participants reviewed a three‐item draft Nosebleed Diary assessing severity, duration, and intensity. For round 2 of the participant interviews, an additional item was added asking if participants had needed to seek medical care for the nosebleed; this item was retained during round 3 of the interviews and in the final Nosebleed Diary.

Nosebleed Diary understanding and comprehension

Overall, the Nosebleed Diary items and instructions were generally well‐understood and consistently interpreted across the three groups of participants. As a result, few changes were implemented throughout the three rounds of interviews, with most of the items and instructions retaining their wording throughout. Participants noted that daily recall was better than 7 days, as they may have forgotten the previous days’ elements.

Participants found the eDiary easy to complete using the handheld device and reported that their HHT‐related nosebleed experiences would be easy to report on a daily basis using the device. It was generally reported that the burden of completing the items each day, or each time they had a nosebleed during the day or night (as could be required within a clinical trial) would be low. Round 3 participants’ interpretation of the questions and concepts were consistent with participants from the previous two rounds of interviews, confirming the comparability of the electronic and paper versions of the diary.

Face validity and translatability

A multilingual team considered this newly formulated diary to be amenable to translation into different languages, with a low probability for misinterpretation.

Nosebleed eDiary finalization

Before finalizing the Nosebleed eDiary, feedback was solicited from the three expert physicians on the version created following round 3 interviews. JH and DP had no further comments and endorsed the final version (Fig. 2). The third expert (SO) endorsed the final version and suggested the addition of the following item to provide insight into the severity of the bleed: “How did you stop the bleeding?” SO believed that the method used to stop the bleeding may differentiate a moderate/severe bleed from a mild one. However, accrual of such data was considered difficult to collect uniformly, and was not pursued.

Figure 2

Final version of the Nosebleed eDiary.

The results of the patient interviews were also used to develop a conceptual framework for measuring change in epistaxis severity using the Nosebleed eDiary. A conceptual framework typically describes the relationships among concepts and items measured and the scores produced by a PROM.13 Although the overall concept assessed by the Nosebleed eDiary is patient perception of the nosebleed severity, a total score based on all items assessed is not calculated. Rather, as shown in Figure 3, the five key nosebleed characteristics important to patients with HHT‐related nosebleeds are measured by the Nosebleed eDiary with each represented as a single, individually scored item: nosebleed frequency, duration, intensity (speed of flow), overall severity, and need for medical attention.

Figure 3

Nosebleed eDiary conceptual framework.

Scores on the individual items addressing nosebleed frequency, duration, and intensity are currently expected to support key endpoints for evaluation of treatment in clinical trials. Scores on the medical attention and overall severity items are included to gather additional, supportive evidence of treatment efficacy. Whether a total score representing overall nosebleed severity for an individual patient based on all five items may be possible remains to be determined. Psychometric analysis, including factor analysis or principal component analysis are recommended next steps to determine whether the individual items are measuring the same underlying construct and may be grouped together.

DISCUSSION

A Nosebleed eDiary was developed by a multidisciplinary team (involving representatives from a pharmaceutical firm, experts in PROM development, language translation and electronic data capture, and expert clinicians who regularly treat patients with HHT) in close alignment with the best current standards for PROM development described in the FDA PROM Guidance13 and the ISPOR PRO Good Research Practices Task Force reports.14, 15 Importantly, this research included comprehensively documented qualitative research involving a demographically diverse and clinically representative sample of adults with severe HHT‐related epistaxis.

The results of 17 interviews conducted with participants with severe HHT‐related nosebleeds suggest that the final version of the Nosebleed eDiary is “fit for purpose” to assess the frequency, duration, intensity, and overall severity of HHT‐related nosebleeds. The Nosebleed eDiary demonstrated content validity (i.e., the items adequately reflect the intended measurement concept13 in patients with severe HHT‐related epistaxis.

Research supporting the development of the Nosebleed eDiary highlights consistency in the way adults with HHT‐related epistaxis describe and assess epistaxis severity (irrespective of demographic or clinical differences). A review of English‐language publications from the past 10 years found patient‐reported constructs of frequency, duration, and/or speed of flow (i.e., intensity) were reported by the majority of participants with HHT asked to describe their nosebleed severity. Based on the experience of one of the authors (JH) of patient feedback during routine clinical practice, epistaxis frequency is the most important aspect for patients with HHT who have frequent nosebleeds, while duration is the most important aspect for patients who have less frequent but prolonged nosebleeds. From the literature review, it was not initially clear that intensity and severity were separate concepts from the patient perspective, but this distinction is reflected in the four items comprising the final Nosebleed eDiary.

The Nosebleed eDiary seeks to address issues with existing instruments, including assessing epistaxis severity in terms of inadequate or incomplete concept coverage,6, 23, 25, 28, 30 use of recall periods that require participants to average over time or think back to an earlier state,12, 21, 31 and use of response options that may limit the ability to demonstrate change in symptoms over time.12, 21 It is also the first PROM to be developed with patient input throughout. The ESS was based on input provided at the beginning of instrument development through an internet survey of patients with HHT, and the concepts selected for the questionnaire were not based on feedback from in‐depth patient interviews. Instead, concepts were selected for inclusion in the ESS based on the six factors found to be independent predictors of self‐described epistaxis severity based on the survey. The draft ESS was not further tested or revised based on input from patients with HHT (i.e., cognitive debriefing interviews). The Nosebleed eDiary advances the ESS scoring and, after establishing its measurement properties, the Nosebleed eDiary could potentially be used to support the registration of products to treat HHT‐related epistaxis, as well as inform the optimal management of HHT‐related epistaxis in the clinic.

The FDA has stated that while there are many instances where therapeutics can decrease mortality or serious morbidity, there are cases where patient symptoms, and the impact these symptoms have on patient functioning, are the most relevant clinical outcomes.34 Given the impact of epistaxis in HHT on patient QoL, there is an excellent case study for the value of PROMs to evaluate the clinical benefit of novel therapeutics.

The small number of mostly Caucasian participants and the focus on those with more severe bleeds limit the generalizability of these results. The impact of gastrointestinal bleeds, heart failure due to liver AVMs, or hypoxia due to lung AVMs were not determined, which may further limit the value of the results to the wider HHT population. While compliance with daily completion of the eDiary will need to be evaluated, it is common practice in clinical research for reminders to be programmed onto the devices on which subjects complete the diaries. These reminders support better compliance and reduce the risk of a bias towards recording primarily the worst bleeds, an otherwise overly conservative representation of bleeding. severity. Psychometric testing with a more racially diverse population of patients with varying levels of severity of HHT to evaluate construct validity and reliability are the recommended next steps.

CONCLUSION

In summary, a PROM has been developed, in line with FDA guidance, that is ‘fit for purpose‘ to assess the frequency, duration, intensity and overall severity of HHT‐related nosebleeds in patients with severe HHT‐related epistaxis. The Nosebleed eDiary has demonstrated face validity and translatability, and is unique in its ability to specifically define the clinical status of a patient with epistaxis on a day‐to‐day basis.

CONFLICT OF INTEREST

Marci Clark, Susan Martin, and Nimanee Harris are employees of RTI Health Solutions, a nonprofit organization that provides research consulting services to pharmaceutical and biotechnology companies. Dennis Sprecher is a former GSK employee with stock provisions and a current employee of Bioview Consulting LLC. Pamela Berry is a former GSK employee and a current employee of Janssen Global Services LLC, with stock provisions from each.

Scott Olitsky and Jeffrey B. Hoag have no conflicts of interest to declare.