Development of a questionnaire to assess experience and preference of intranasal corticosteroids in patients with allergic rhinitis.

BACKGROUND: Allergic rhinitis affects 10%-20% of the US population. Its chronic nature, combined with patients' perceptions of safety/efficacy, administration, and sensory attributes of nasal sprays (corticosteroids), impact patient adherence to therapy. The purpose of this study was to develop a measure of experience with and preference for corticosteroid therapy for treatment of allergic rhinitis.
METHODS: Questionnaire development was conducted through qualitative research including concept elicitation and content testing in 153 patients with allergic rhinitis. Patient focus groups (n = 66), in conjunction with content confirmation and saturation in additional groups (n = 87), provided research data. A literature-based conceptual framework was incorporated into the interview guide. An iterative process of data collection, analysis, and theory development yielded the conceptual framework.
RESULTS: Consistent comments from the focus groups combined with those from cognitive debriefing interviews led to the incorporation of 14 finalized attributes into the Experience with Allergic Rhinitis Nasal Spray Questionnaire (EARNS-Q) items. Between the first and second cognitive debriefing interviews, researchers revised the EARNS-Q for retesting. Face and content validity tests indicated that the items, responses, and instructions were understood by study participants. The EARNS-Q is comprised of two modules that measure patient experience with nasal sprays (experience module), and patient preference for a nasal spray relative to another (preference module).
CONCLUSION: The EARNS-Q accurately measured patient experience with and preference for nasal sprays used in treating allergic rhinitis. A potential application of this questionnaire may be as a patient-reported outcomes endpoint in clinical trials of intranasal corticosteroids in patients with allergic rhinitis.

Introduction

Allergic rhinitis is a highly prevalent and chronic condition that affects an estimated 10%–20% of the US population.1–4 According to the National Center for Healthcare Statistics, up to 40 million people have allergic rhinitis and generate an estimated $2.7 billion in both direct and indirect costs.1,5,6 Allergic rhinitis is also associated with daily life burden, including 52.5 symptomatic workdays and 3.6 workdays missed each year, and 2.3 hours of unproductive work for every symptomatic day. The resulting annual economic burden on employers is calculated to be an average of $593 per patient.7

Regular prophylactic use of intranasal corticosteroids is recommended for long-term management of the symptoms of allergic rhinitis.8–11 Despite the guidelines, the practical issues around patients’ treatment compliance may significantly impact the success of therapy; patients’ acceptance and/or satisfaction with their treatment may also play important roles in compliance. Recent studies have shown that patient perceptions of intranasal corticosteroids are influenced by the efficacy and safety of an agent, ease and comfort of administration, and by sensory attributes.12,13 Because of the chronic, long-term nature of allergic rhinitis, adherence to therapy is difficult and may result in inadequate disease control. According to the Allergies in America survey, most patients (61%) stopped their prescribed intranasal medication because of its attributes rather than allergic rhinitis itself. Patients reported lack of efficacy (37%), diminished effectiveness over time (35%), lack of 24-hour relief (32%), and side effects (25%) from the medication as reasons why they discontinued treatment.14 Mahadevia et al reported that patients’ willingness to adhere to physician advice would significantly improve with more favorable sensory attributes.15

Therefore, it is important to have sound measurement of the concepts of experience and preference for allergic rhinitis therapy to track patients’ acceptance of therapies and to optimize treatment outcomes. Thus, a project to develop (and validate) measures of experience with and preference for allergic rhinitis therapies was undertaken. The results of the qualitative research that comprised the development phase of a promising instrument, ie, the Experience with Allergic Rhinitis Nasal Spray Questionnaire (EARNS-Q), are presented here.

Methods

Preference

Preference can be defined as the action that a patient would choose in a particular medical situation at a particular time, given a set of alternatives.16–18 Hypothetically, a patient’s experiences with attributes of different products is evaluated (“A” in Figure 1), and they make a determination about a preference for one product over the other (with respect to those attributes, “B” in Figure 1). In addition, the patient’s preference for one product over the other can be evaluated using a single “global preference” item, which hypothetically reflects his or her consideration of all attributes, experience, and preferences in a single expression of product preference. These hypothesized relationships are illustrated in Figure 1. A thorough review of the preference literature in allergic rhinitis demonstrates that the most appropriate method is direct assessment of preferences among medications by using questionnaires.19

Figure 1

Operationalization of preference elicitation.

Development and validation of the EARNS-Q

In order to evaluate whether the EARNS-Q is a valid measure to assess experience and preference of intranasal corticosteroids in allergic rhinitis patients, a qualitative and quantitative research methodology was implemented for its development and psychometric validation. This development approach followed guidelines described in the US Food and Drug Administration (FDA) patient-reported outcomes guidance document.20 The development and validation program included three major phases, ie, qualitative research to determine concepts of importance to patients, development of a draft instrument, and patients’ comprehension of the items (methodologies 1, 2, and 3), quantitative evaluation of the validity of each module of the EARNS-Q, and qualitative research to guarantee representation of allergic rhinitis patients from low socioeconomic circumstances. The development phase of the EARNS-Q is summarized in Figure 2. Phases 2 and 3 are reported in a separate paper.

Figure 2

Allergic Rhinitis Nasal Spray Questionnaire: qualitative and quantitative research phases.

Abbreviations: EARNS-Q, Experience with Allergic Rhinitis Nasal Spray Questionnaire; SES, socioeconomic status.

Patient samples and analytic procedures

Methodology 1: Focus groups for concept elicitation and item generation

The goal of this phase was to elicit patients’ experiences with nasal sprays for the treatment of allergic rhinitis. This process, described as concept elicitation, was performed in a US focus group consisting of 10 allergy sufferers who previously used or were currently using nasal sprays to alleviate allergy, and recruited through a newspaper advertisement. Patients with a wide representation of demographic characteristics (eg, age, gender, and social status) were invited to participate in the study. Inclusion criteria included patients aged 18–65 years and a confirmed diagnosis of allergic rhinitis (perennial, vasomotor, or seasonal). Exclusion criteria included a diagnosis of a life-threatening condition.

The focus group utilized a structured discussion guide. Patients were invited to share recollections of their condition and treatment history, followed by discussions aimed at eliciting nasal spray attributes.

Focus group interview data were tape-recorded and transcribed, and coding schemes were developed and iteratively refined to translate descriptions of patient characteristics into thematic trends for data analysis using grounded theory methods.21

Guided by the verbatim patient comments in the focus group, a set of items was developed that reflected the allergic rhinitis-related patient concerns, while ensuring item clarity and interpretability. Thus, all major areas of concern that had been voiced by patients were represented in the initial instrument. The questionnaire was subsequently reviewed by a researcher and expert in psychometric analysis (who was not involved in the initial development) to search for potentially problematic item content. The questionnaire was divided into two modules, ie, an experience module and a preference module which investigated patient preference of medication, because conceptually, preference is linked to the patients’ experience with medication and ratings of its attributes.

The experience module was designed to capture patients’ evaluation of specific characteristics of nasal sprays, and was intended for use after patients’ respective treatment administration. It consisted of two questions that examine the attributes, ie, to rate their experience on each attribute and to capture their importance. The rationale for this approach was to investigate developing an importance-weighted rating of each attribute, such that attributes deemed more important would receive greater weight.

The preference module addressed the comparison of two products, and established the study participant preference for a product based on individual product attribute (and offered the choice of a neutral category in the event of no clear preference). It was specifically developed for use at the end of a crossover study in which participants receive each study treatment in a random order during the trial, and features a global preference item, which is supported by individual preference domains.

Methodology 2: Cognitive debriefing interviews

Cognitive debriefing aims to verify the comprehensiveness and patient understanding of a questionnaire, as well as its clinical applicability and acceptability. A series of individual interviews (n = 5) with allergic rhinitis sufferers, who had been prescribed a nasal spray, was conducted after the first draft of the experience module was developed. These interviews examined whether the draft questionnaire was appropriate to measure the target population (who were treatment candidates). The identical inclusion/exclusion criteria from methodology 1 were applied to the target population.

The interviews lasted about 2 hours and comprised a two-step process. The first step followed an open-ended, exploratory format that allowed patients to express spontaneously what attributes should be included in the questionnaire (spontaneous elicitation). Patients were then asked to comment systematically on the content and relevance of the questionnaire (cognitive debriefing).

Transcription, coding, and analysis by grounded theory methods proceeded as in methodology 1, with the addition of a saturation grid to document the adequacy of the sample size to yield consistent and stable results.

Responses from the cognitive debriefing section of the interviews were reviewed and patterns and inconsistencies in patient perceptions were identified. Items that patients found difficult to understand and/or interpret were rephrased. Ten additional interviews were conducted, following the same methodology, to ensure that the modifications were well understood.

Methodology 3: Focus groups testing the concept of “experience”

The content of the EARNS-Q experience and preference modules was explored in four focus groups (n = 41) of allergic rhinitis patients, with a focus on the meaning of the term “experience”. Data captured patients’ evaluation of specific characteristics of nasal sprays, and was intended for use after patients’ respective administration of the products. The same recruitment criteria and methods as in methodology 1 were utilized for this phase of the development.

Results

Results from methodology 1: Concept elicitation

The mean age of study participants was 43 (range 30–50) years. Most patients were female (70%), and a majority (70%) of participants had achieved a college or a graduate degree. The remaining patients had a diploma or attended some high school. Patients who participated in the focus groups were diagnosed with allergies for an average of nine (range 0–28) years prior to the focus groups.

Results from the analysis of qualitative data from the initial focus group revealed key attributes that were consistently reported by study participants (Table 1).

Table 1

Experience with Allergic Rhinitis Nasal Spray Questionnaire attributes

Attributes consistently mentioned in patient focus groups	Final EARNS-Q attributes
Ability to deliver the same dose	Ability to deliver the same dose
Size/comfort of the nose tip	Comfort of the nose tip
Ease of operation	Ease of operation
Ability to deliver the right dose	Ability to deliver the right dose
Bottle transparency	Ability to assess how much medication is left in the bottle
Absorption by the nose (leakage)	Leakage (nose or throat)
Efficacy (how well it worked)	Efficacy
Smell	Smell
Onset of action	Onset of action
Aftertaste	Aftertaste
Symptom relief	Symptom relief
	Burning
	Mist
	Ease of transport

Abbreviation: EARNS-Q, Experience with Allergic Rhinitis Nasal Spray Questionnaire.

Certain attributes identified by patients were beyond the scope of the concept of interest. The excluded attributes were weight of the product, ease of use in the elderly, price or financial issues, and size of the bottle/frequency of refill. Note that the impact of price/financial issues as an attribute could not be determined in an industry-sponsored trial where the sponsor covers the cost of therapy. The first draft of the questionnaire included an experience module (illustrated in Figure 3) and a preference module.

Figure 3

Conceptual framework for the Experience with Allergic Rhinitis Nasal Spray Questionnaire (illustrated with items from experience module).

A four-point Likert scale was selected for the experience module because it was easily understood, offered response choices that discriminate between respondents’ perceptions, and was readily interpretable. A “neutral” category was not required, and an even number of options was offered.22 Responses to the preference model were collected on five-point Likert scales.

Results from methodology 2: Cognitive debriefing interviews

The mean age of the interviewed patients was 38 (range 20–72) years. Seventy-three percent of the study population was female. Sixty percent of the population worked either full-time or part-time, and 27% were full-time or part-time students. Participants had been diagnosed with allergies for an average of 15 years prior to the focus group, ranging from 8 months to 60 years.

All patients found the attributes in the questionnaire relevant, and the most important characteristics were that the nasal spray worked (ie, it treated their allergies), and that it was convenient (ie, easy to carry, easy to use). Participants thought the questionnaire was of a proper length, comprehensive, informative, and consistent (data not shown).

Because study participants did not have comparative products (on hand) during debriefing, researchers instructed them to imagine their current nasal spray for their allergies as “product A” and others as “product B”. However, some patients had difficulty because they were unable to imagine a comparison of two products without actual sprays being present. Nevertheless, all patients found the instructions clear and had no difficulty understanding them. They did not suggest rewording them, and the study participants indicated overall that the answer choices were clear, detailed, and appropriate for the questions (data not shown).

Between the first and second cognitive debriefing interviews, researchers revised the EARNS-Q for retesting, as an iterative, qualitative-research process. The rationale (summarized in Table 2) was determined by comparing comments from the initial focus group with those from the first round of cognitive debriefing. The relevance/importance of three items from the initial focus group (eg, convenience of the bottle [transport], direction of the spray [straight line or mist], and intensity of the spray) were supported and added to the questionnaire.

Table 2

Summary of questions added and rationale for inclusion

Elicited attribute	Rationale for inclusion	Added question
Convenience of the bottle	During attribute elicitation, focus group participants mentioned the portability of nasal sprays. The size of the product was also a concern, but preference for size was not universal (some liked bigger bottles, some liked smaller bottles); therefore, convenience in the sense of portability was added, and size was left out.	Overall, how easy to carry is the nasal spray you are using for your allergies?How important is it to you that a nasal spray is easy to carry?Which product do you prefer because it is easier to carry?
Direction of spray – straight line or mist	In both the focus group and the first set of face and content validity interviews, patients mentioned the power of the spray and how much the spray bothered them.“How much does it hurt” refers to how bothersome the spray is to the patient. Thus, a question about whether pain is felt when the spray is released into the nose was added to the questionnaire.	Overall, how much does it hurt when you release the spray into your nose?How important is it to you that a nasal spray does not hurt?Which product do you prefer because it hurts less?
Strength of spray (powerful versus gentle)	In both the focus group and the first set of face and content validity interviews, patients mentioned characteristics of the power of the spray. “Gentle” refers to the quality of the stream of the spray. Thus, a question about whether the gentleness of the spray was added to the questionnaire.	Overall, how gentle is the mist that is released from the nasal spray you are using for your allergies?How important is it to you that a nasal spray is gentle?Which product do you prefer because it is more gentle?

The second set of interviews consisted of a thorough review of the revised questionnaire in 10 patients. When asked for their overall opinion of the questionnaire, three patients indicated it was thorough and consistent, while six believed it was too long. Study participants believed that the questionnaire was easy to understand (n = 8), easy to answer (n = 9), and comprehensive (n = 10). Although a majority of the questions were easily understood and relevant, some items were perceived as having awkward wording or seemed irrelevant. Consistent comments discerned from the initial focus group combined with that from cognitive debriefing interviews led to 14 finalized attributes in the EARNS-Q (Table 1).

Results from methodology 3: Testing the concept of “experience”

Subjects judged “experience with nasal sprays” to be the most suitable term that summarized their opinions about nasal sprays; they noted that the term was broad and not leading. Experience included efficacy, side effects, form of administration, onset and duration of action, convenience, ease of use, and comfort. Patients also indicated a connection between experience and preference. Moreover, because the preliminary content of the instrument encompassed only the wording/ elements mentioned in these focus groups (described in detail above), the responses from the focus groups corroborated the ability of the EARNS-Q to capture relevant attributes of nasal sprays for patients with allergic rhinitis.

The participants’ spontaneous responses to questions regarding their likes/dislikes with nasal sprays confirmed the findings of the initial focus group, as well as the first set of item elicitation/cognitive debriefing interviews. Saturation was achieved (data not shown), such that the subjects’ responses to the interviews remained consistent across all items, and items were similar to those reported in other preference studies.17

Discussion

Due to the influence of patient perceptions of safety/efficacy, ease of administration, and favorable sensory attributes, combined with the chronic nature of allergic rhinitis, adherence to therapy may be difficult. Therefore, development of a sound measurement of the concepts of patient experience and preference in allergic rhinitis is warranted. To meet that need, the EARNS-Q, a 43-item questionnaire that assesses patients’ experience with and preference for nasal sprays used to treat allergic rhinitis, has been developed. It should be noted that the manuscript by Crawford et al found only 29 items were necessary (14 for experience and 15 for preference), thereby allowing the user to drop the importance items.23

These data presented herein were a result of rigorous research into the measurement of patient preferences that are associated with nasal sprays. The EARNS-Q comprises two modules that measure patient experience with nasal sprays (“experience module”, 28 questions), and patient preference for a nasal spray relative to another (“preference module”, 15 questions).

During the focus groups, patients were also asked to describe attributes for a “perfect nasal spray” (data not shown). The top four responses included bottle size, dosage, efficacy, and long-lasting. Following these, top responses were decreased side effects, frequency of use, onset of action, and pump design. The EARNS-Q captures most of these directly (see Figure 3) or through the domain structure. Modules include domains on efficacy, spray delivery (dosage concerns), sensory perceptions (eg, burning, smell, taste), and device characteristics (including ease of carrying the spray). Utilizing the EARNS-Q in practice may help further define the “perfect nasal spray” by evaluating treatment experiences and the patients’ resultant preference for one treatment over another.

A limitation of this study was the large percentage of women and college-educated respondents during the early development. However, during the focus groups with the lower economic status groups, no new concepts arose. The large percentage of women may reduce the questionnaire’s generalizability to the extent that women report different concerns with respect to the use of nasal sprays. However, differences between the attributes reported by men versus women were not noticed. Therefore, we do not expect this to have much impact on the questionnaire. Another limitation to this study is that due to the nature of the questions comprising the EARNS-Q experience and preference modules, no explicit recall period existed. The recall period that might be considered in the responses was linked to the treatment that study subjects received while on study. With regard to patient ability to formulate a judgment of preference weeks (or months) after receiving a medication while on study, it is unclear whether a patient’s stated preferences might differ with access to his or her own historical reports of symptoms during the study. However, it was demonstrated in another therapeutic area that patient perception of subjective overall preference for a medication was not different when assessed with and without access to their diaries where they recorded their symptoms. In this regard, instrument developers might present study subjects with a clear method to judge preference between two competing therapies, and not to define the window of time for carrying out this cognitive task.24

The use of the EARNS-Q in different age groups might be considered in future research. Current epidemiologic estimates and challenges in treatment support the importance of such studies.

Allergic rhinitis is the most common chronic condition in the US, affecting 10%–30% of adults and up to 42% of children. Allergic rhinitis is also a common rhinitic condition among the elderly.25 In addition, steroid nasal sprays may be subject to poor compliance among pediatric patients, because the treatment is not suited for the relief of immediate reactions to seasonal allergies.26 In the current study, we developed the EARNS-Q based on experiences among the adult population with a mean age of 38–43 years. However, there may be added value to evaluating the experiences and treatment preferences in allergic rhinitis among pediatric and elderly populations, and incorporating them into the EARNS-Q. Additional qualitative research with pediatrics would be warranted, whereas with the elderly, a two-stage process of concept elicitation and cognitive debriefing may be sufficient.

The qualitative research summarized herein reflected a thorough, iterative process of instrument development, as described in the FDA Guidance for Industry.20 As a result of the rigorous application of qualitative research to the development of the EARNS-Q, we have a high level of confidence that the instrument’s content is relevant among patients with allergic rhinitis. The psychometric properties of the instrument will be reported in a separate article, and together these data provide evidence that the EARNS-Q instrument is a sound measurement of the concepts of experience and preference in allergic rhinitis. The data also indicate that the EARNS-Q may play a role in measuring treatment benefit in the context of a clinical trial, and also allow researchers to track patients’ acceptance of therapies and optimize treatment outcomes. Further research may also be warranted on the use of the EARNS-Q in the community treatment setting for allergic rhinitis.