Development of a New Tool for Evaluating the Benefit of Preventive Treatments for Migraine on Functional Outcomes - The Migraine Functional Impact Questionnaire (MFIQ).

OBJECTIVE: The objective of this study was to develop a method for evaluating patient-relevant outcomes of interventions for preventing migraine attacks, followed by an assessment of the content validity of a new patient-reported outcome (PRO) instrument: the Migraine Functional Impact Questionnaire (MFIQ).
BACKGROUND: The aim of preventive treatments for migraine is not only to reduce migraine frequency, but also to restore patients' ability to function and improve quality of life.
METHODS: A multi-stage process based on best practice methods and regulatory guidelines for ensuring content validity of PRO instruments for evaluating treatment benefit was followed. Qualitative concept elicitation interviews conducted to understand the experiences of adults with migraine underpinned the development of the instrument. The initial stage included the development of a conceptual disease model (CDM) based on information from these interviews. This CDM was used to identify the concepts of interests (COI) to evaluate outcomes of preventive treatments. The results of the interviews were also used in stage 2, to develop a measurement framework for collecting data about these COI. In the third stage, existing instruments were reviewed for coverage of the concepts in the framework and evidence of concept elicitation to the point of saturation, to support content validity. In the fourth stage, an instrument was drafted to evaluate concepts in the framework, based on the qualitative data collected from the interviews. Following a review by clinical and translation experts, the new instrument was tested in adults with migraine in the fifth stage using 2 rounds of cognitive interviews, and was modified based on interview feedback. In the last stage, the instrument was linguistically adapted, using methods recommended for PRO instruments, to ensure conceptual equivalence of language versions for use in international studies. Each language version was tested in at least 5 native speakers using cognitive interviews.
RESULTS: Results from the concept elicitation interviews suggested that migraine had an impact on various aspects of functioning. A conceptual framework for evaluating functional outcomes was developed for the concept selected based on a review of the CDM - physical and emotional functioning, every day activities, and social/leisure activities. Existing PROs lacked coverage of some concepts in the conceptual framework, had recall periods that were inappropriate for capturing the experience of COI or did not have evidence of content validity. A novel PRO instrument, the MFIQ, was developed to address these gaps. Cognitive interviews with 9 adults with migraine resulted in minor changes to the items of the MFIQ, and a final round of 8 interviews confirmed the changes were acceptable and supported its validity. The interviews conducted to test linguistic adaptations confirmed conceptual equivalence in the 25 countries evaluated.
CONCLUSIONS: Development of the MFIQ followed best measurement practices to ensure content validity and followed regulatory guidelines for PRO instruments to evaluate benefits of treatments. The MFIQ was developed for use in clinical trials or clinical practice settings to track outcomes of preventive treatments that are most relevant to adults with migraine.

conceptual disease model

concept elicitation

chronic migraine

concept of interest

consolidated criteria for reporting qualitative research

case report form

episodic migraine

European Medicines Agency

Food and Drug Administration

Headache Impact Test

International Classification of Functioning, Disability, and Health

International Classification of Headache Disorders 3

International Headache Society

institutional review board

International Society for Pharmacoeconomics and Outcomes Research

Migraine Functional Impact Questionnaire

Migraine Disability Assessment Questionnaire

Migraine Physical Function Impact Diary

Migraine Specific Quality of Life Questionnaire

National Committee for Quality Assurance

National Quality Forum

patient‐reported outcome

standard deviation

United Kingdom

United States

World Health Organization

Background

Epidemiological research shows that migraine, a disabling, recurrent primary headache disorder is a commonly occurring disease, affecting as many as 18% of women and 6% of men.1 The impacts of migraine are experienced both during attacks of migraine and in the inter‐ictal period between attacks. Associated symptoms, such as photophobia, phono‐phobia, nausea, and vomiting are also experienced in the pre‐ and postdromal periods and between attacks, with anxiety and avoidance of activities due to fear of exacerbating or causing an attack.2, 3, 4 The overall prevalence of migraine is greatest between 30 and 50 years of age,1, 5 and substantially impacts the workplace through absenteeism (missed work) and suboptimal performance (presenteeism).6 Migraine could also result in missed family events and vacations, difficulty with routine responsibilities, and impaired relationships with spouses, children, and other family members.4, 7 A review of literature to identify the most relevant psychosocial difficulties related to migraine suggested that reduced vitality and fatigue, emotional problems, pain, difficulties at work, general physical and mental health, social functioning and global disability were the most frequently reported.8

Stakeholders from all sectors of healthcare recognize the importance of functioning as a key metric for evaluating both the burden of disease and the benefit of treatments. Regulators like the United States (US) Food and Drug Administration (FDA) and European Medicines Agency (EMA) have stated their interest in tracking clinical trial outcomes using endpoints that capture how patients feel and function, but have cautioned against the use of outcome measures that have not been validated.9, 10 Agencies measuring quality and outcomes in health care, like the National Committee for Quality Assurance (NCQA) and the National Quality Forum (NQF) are increasing their inclusion of patient‐reported outcomes (PROs) as quality measures.11, 12, 13, 14 Reimbursement agencies who are framing decisions about the value of new health technologies are asking for evidence about the burden of illness from a humanistic perspective, setting the stage for describing treatment benefit in that same way.11 While some of these stakeholders differ in how they operationalize or assess patients’ functioning and well‐being, there is a clear endorsement of its importance.

Guidelines for the management and evaluation of treatments for migraine emphasize the importance of collecting data about the impact of migraine on functioning.15 The aims of preventative treatments for migraine are to: reduce attack frequency, severity, and duration; improve responsiveness to treatment of acute attacks; and improve function and reduce disability.16

To interpret the benefit of preventive treatments, it is important to examine whether a reduction in migraine frequency (the traditional primary endpoint for clinical trials) also translates to a reduction in the impact of migraine on patients’ lives. A number of PRO tools are used in clinical studies to evaluate migraine therapy outcomes, including the Migraine Disability Assessment questionnaire (MIDAS),17 Headache Impact Test (HIT‐6™),18 and the Migraine Specific Quality of life questionnaire (MSQ).19 These instruments were developed before the FDA guidance for PRO instruments to support labeling claims10 was published. The FDA guidelines highlight the importance of providing evidence that a PRO instrument comprehensively reflects the perspectives of the target sample (ie, evidence of content validity).

Objective

This paper describes the development of a novel migraine instrument for adults, the Migraine Functional Impact Questionnaire (MFIQ), which captures the impact of migraine on physical, social, and emotional functioning.

It should be noted that a separate study was conducted for collecting data for item analyses, to inform item reduction to develop the final version of the MFIQ, scoring, and the assessment of reliability, validity, and responsiveness to change. Results will be reported separately. A related daily diary instrument, the Migraine Physical Function Impact Diary (MPFID), which assesses the impact of migraine on physical functioning in the past 24 hours was developed in parallel to the MFIQ20, 21 and has been included in global clinical trials. Both measures were developed following methods described in the FDA PRO guidance.10

Methods

A multi‐stage process outlined by the FDA Roadmap to Patient‐focused Outcome Measurement in Clinical Trials was used.22 Initial stages focused on understanding the condition through qualitative concept elicitation (CE) interviews, with subsequent stages aimed at conceptualizing treatment benefit to help select/develop the outcome measure and then to test the measures in the target sample of adults with migraine using cognitive interviews (CI) (in the United States to test the initial US English version and in target countries to test translated versions). An overview of the stages of the study is summarized in Figure 1.

Figure 1

Stages in the development of the new PRO instrument. *CE interviews and Stage 1 has been described in detail by Mannix and colleagues20 and only summarized in this paper.

The development of the new PRO tool was underpinned by qualitative research in adults with migraine as illustrated in Figure 1. The conduct of the studies was guided by the methods described in the US FDA Guidance for PRO Measures10 used to support labeling claims about treatment benefit and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force Reports on best practice for establishing and reporting on content validity of a new PRO instrument.23, 24

Overview of the Data Collection Approach

The concept elicitation and cognitive interviews in the United States were conducted following ethics approval from a central Institutional Review Board (IRB) (Schulman Associates IRB, Inc). The research team for these interviews included a PRO strategy team of PRO tool development experts, co‐authors AH, SM, and AS – who led the development of the strategy and implementation of the activities during the stages described below. Four experts, including two neurologists with expertise in treating adults with migraine, one clinical psychologist with expertise in headache and pain management (co‐author DB), and one methods expert with expertise in migraine‐specific PRO tools (co‐author MB), were consulted throughout the study process. A team of interviewers (including authors AS and SM) conducted the concept elicitation interviews and another team of interviewer (including co‐author SG) conducted the cognitive interviews. A convenience sample from US‐based clinical sites, were recruited (details of eligibility criteria are shown in the online supplement). One‐on‐one in person interviews were conducted at the clinical sites in a private room using a semi‐structured interview guide. No others were present during the interviews. Participants were provided information about the objective of the interviews and provided written informed consent to participate. Interviews started with open ended questions, followed by probes. Interviews were audio‐recorded and field notes were captured. Each interview lasted approximately 90 minutes. Demographic and clinical characteristics were collected via case report forms (CRFs) developed for the study. All participants completed interviews and were compensated for participation. A professional third‐party service transcribed each interview verbatim and each transcript was reviewed by Evidera staff for quality assurance purposes, to remove any personal identifying information, and to correct obvious transcription errors. Transcripts were not returned to participants for comment. Demographic and clinical characteristics were collected via CRFs developed for the study.

Cognitive interviews for testing translated versions did not collect any personal health information and did not require ethical review. One‐on‐one in person interviews were conducted by health care professional interviewers who were fluent in the target language. Interviewers were contracted by the organization (FACITtrans) conducting the translation. Interviews were conducted at clinical sites in a private room or at participants’ place of residence using a semi‐structured interview guide. No others were present during the interviews. Participants provided verbal consent to participate. Each interview lasted approximately 1 hour. All participants completed interviews. Each interviewer took detailed field notes and typed the notes into a testing report. No audio recordings were made. Basic demographic and clinical characteristics were collected as part of the cognitive interview. Table 1 and sections below provide additional details of the research team and the study design for these qualitative interviews.

Table 1

Details of the Research Team and Designs of Studies for the Qualitative Interviews

Concept Elicitation Interviews in the US	Cognitive Interviews in the US	Cognitive Interviews in the Target Countries
Research team – Interviewers
Health outcomes research professionals trained in qualitative interview techniques who held a minimum of an undergraduate or master’s degree		Healthcare professionals such as physicians, psychologists, study nurses, and research assistants with regular patient contact. They held a minimum of an undergraduate degree and most were MDs, PhDs, or RNs. The interviewers were fluent in the target language. Interviewers were contracted by the organization (FACITtrans) conducting the translation
Interviewers were employees of a research consulting organization – Evidera – contracted for the study
5 women (including co‐authors AS and SM).	1 man and 3 women (including co‐author SG)	22 women and 7 men. None of the authors were interviewers
Interviewers and participants had no exposure to each other prior to the interview session. The study was explained to all participants in a consistent manner by the site staff during recruitment and by the interviewer during the interview session		Some interviewers (clinicians) and participants had exposure to each other prior to the interview session. In other cases, the clinician or site introduced the patient to the interviewers
Study design
One time qualitative interviews to collect data about the experience of adults with migraine	One time qualitative interviews to test the initial US English versions of the MFIQ	One time qualitative interviews to test the translated versions of the MFIQ
Conducted‐October–November 2013, N = 32	Conducted January–February 2014, N = 17	Conducted March–July 2015, N = 146
Recruitment and participant selection
Participants were recruited through community‐based clinical sites in the United States using convenience sampling and approached in person or via telephone		Clinicians at clinical sites recruited participants through a convenience sampling and approached in‐person or via telephone
Participants had to be between the age of 18 and 60; have had a history of migraine headaches (with or without aura) for ≥12 months prior to screening meeting IHS criteria.25 Additional criteria to ensure that the sample reflected the target sample of a clinical trial for preventive treatments for migraine were used and are listed in the online supplement		Participants were between the age of 18 and 65; had a history of migraine (with or without aura) in the 12 months prior to screening based on medical records and/or patient self‐report of clinician confirmed diagnosis of migraine; migraine frequency ≥4 migraine headache days in the past month; and at least one patient in each country should have chronic migraine

Overview of Analytical Approach

A content analysis approach was used to analyze the qualitative data. For the CE and CI interviews, analyses integrated information from interviews’ notes and coded transcripts of audio recordings. An analytic coding dictionary was developed based on the interview guide, and later revised as needed during qualitative interviewers contributed to the coding process. Transcripts were coded using ATLAS.ti, a qualitative data analysis software.26 Six trained qualitative researchers (including co‐author SG) formed the coding team who were involved in coding the transcripts (coding was also reviewed by co‐author AS). The specific coding methods for the CE, CI and translation interviews are provided in the respective section below.

Descriptive statistics (eg, n, mean, standard deviation, and/or frequency) were calculated and used to summarize quantitative data to characterize the sample. All statistical analyses were performed using SAS 9.2 (SAS Institute Inc., Cary, NC, USA).

Qualitative Concept Elicitation (CE) Interviews

The methods and results of the CE study conducted to understand the experiences of adults with migraine have been reported previously.20 The CE study involved in person, one‐on‐one, qualitative interviews with adults with migraine recruited from 5 neurology and clinical research clinics in the US. A phenomenological approach 27 was used to collect concept elicitation data. Interviews elicited concepts about the subjects’ current experiences with migraine, such as the impact of symptoms on functioning. Interviews started with open ended questions followed by probes on the concepts identified from the literature review and clinician interviews. Interviews were conducted to the point of saturation, defined as the point at which no substantially new information can be gained from conducting further interviews.28 Thirty‐two adults with migraine were interviewed. AS and SM conducted the initial interviews to pilot test the guide.

Codes were used to label participant quotes within the transcripts by concept. Participants’ descriptions of the impacts of migraine in relation to their day‐to‐day functioning were coded. Their descriptions of magnitude and variability, level of impact or bother, were also coded. Emerging concepts and concept endorsement were tracked to determine when saturation had been met (ie, no new concepts were being identified). In a saturation grid interview transcripts were grouped by sets of 5 successive EM or CM participants to evaluate themes/concepts reaching saturation. Details of the interview team and study design are summarized in Table 1.

Stage 1: Development of the Conceptual Disease Model (CDM) to Identify Concepts of Interest

Based on guidelines for instrument development,23, 24, 29 a Conceptual Disease Model (CDM) was developed as a pictorial representation of the disease processes, guiding decisions about what to measure and how to measure it. The CDM was used as a visual tool by the PRO strategy team and 2 other clinical experts in migraine, to help identify and select the most relevant COIs for assessing treatment benefit of preventive treatments for migraine as described by Mannix and colleagues20 to identify the concepts of interest (COIs) to evaluate the outcomes of preventive treatments for migraine.

Stage 2: Developing a Conceptual Framework

A preliminary conceptual (measurement) framework (CF) was developed to measure each COI. The CF is an explicit diagram that helps to illustrate the relationship between the item‐concepts contained in an instrument and the disease concepts measured. The World Health Organization’s (WHO) International Classification of Functioning, Disability and Health (ICF) provided a conceptual basis for assessing functioning, and was used as a guide for developing the CF.30, 31 In the ICF framework: Acts (things that an individual can do independent of context or purpose) and Tasks (things people do in daily life in a specific context, with purpose) are key facets of assessing functioning. An assessment of the ideal recall period and response scale was considered for each COI (details are described in the results section.)

Stage 3: Review of Instruments to Inform Selection

Next, the content of existing PRO instruments (the details of the instruments and review to identify these instruments is described by Mannix and colleagues20) were examined (1) for coverage of concepts in the CF (to evaluate the broader outcomes of the impact of migraine) and (2) for the availability of documented evidence of the relevance of concepts to the target sample (eg, item generation based on insights from adults with migraine).

Stage 4: Item Generation and Development of the Draft MFIQ Instrument

Items for the new instrument were generated to represent the concepts and sub‐concepts in the conceptual framework, using participants’ descriptions from CE interviews of the impact of migraine on functioning. The pool of items from previously validated questionnaires20 was also reviewed to inform item construction. The development of instructions, questions, recall period, and response scale was guided by best practices outlined for the development of PRO instruments.23, 24 Specific language used by participants from the CE interviews was considered during the development process to develop the MFIQ version 0.1.A translatability assessment32 was conducted to evaluate whether the items can be meaningfully translated to other languages for use in global studies. The aim was to ensure that item wording was suitable for culturally diverse populations and appropriate for harmonized translation into multiple languages and to identify potential issues in source wording. A lexibility (grade or reading level) assessment was also conducted to ensure that the reading level was appropriate for the target audience. The assessor also provided recommendations to address any issues identified.

Stage 5: Cognitive Interview for Testing the Initial Drafts of the MFIQ

Cognitive interviews were conducted to test the initial draft (version 0.1) of the US English version of the new instrument with a new sample of adults with migraine – a new convenience sample of adults with migraine recruited from 1 neurology site and 1 clinical research clinic in the US. The eligibility criteria (see online supplement for criteria), recruitment, and interview procedures used for the CE interviews were followed (See Table 1 for details of the study design). Cognitive interviews were conducted using a pragmatic approach to address the specific objectives. Interview participants were interviewed at the clinical sites, and provided information about the objective of the interviews and asked to complete the MFIQ. Interviewers assessed the coverage of domain‐concepts (eg, social functioning impacts) and explored participants’ overall understanding of the item structure, instructions, recall period, and response options. Interviews were conducted in 2 rounds. After Round 1 of interviews, an interim analysis was conducted using interviewer field notes, and the MFIQ version 0.1 was revised based on participants’ feedback, followed by a translatability assessment of potential item revisions. A second round of cognitive interviews was conducted to further confirm the content validity of the revised MFIQ instrument. Codes were used to label participant statements about comprehension of the MFIQ instructions, items, recall period, and response options as well as overall comprehensiveness of the MFIQ. Six researchers coded the data using the coding dictionary. Data on concept‐level debriefing questions were also coded. Codes in the coding tree pertained to items or concepts and the objectives of cognitive testing. Details of the interview team and study design are summarized in Table 1.

Stage 6: Translations and Testing of Translated Versions

Following development of the US English version of the MFIQ, conceptually equivalent language versions were developed for use in clinical studies in 25 countries (see online supplement for a list of the countries and languages). The ISPOR guidelines for linguistic validation of PRO measures33 using the universal approach34 were followed. The aim was to develop versions that were conceptually equivalent to enable pooling and comparisons of data.

To adapt the US English version for use in the United Kingdom (UK) and Canada, the US English source items were reviewed by language experts from the United Kingdom and Canada and adaptations were made. For each non‐English language translation, 2 forward translations by native translators, reconciliation of forward translations, back‐translation by 1 English‐speaker fluent in the target language, and a final review by a native‐speaking language coordinator were conducted. Harmonization was performed to ensure conceptual equivalence across languages.

Each linguistic adaptation was tested with a convenience sample of at least 5 adults with migraine in the target country (ie, native speakers of the language). Interview participants were asked to complete the translated version of the MFIQ and were subsequently engaged in a cognitive interview about the new instrument. To assess comprehension, participants were asked to provide feedback on relevance of items, instructions, recall period, items, and response scales. Interviewer notes about participants’ comprehension and relevance of the instructions, items, recall period and response scales were coded. The coded data was evaluated to assess the participant responses between language versions for conceptual equivalence. Table 1 shows additional details about the research team and study design.

Results

Concept Elicitation Interviews

Results of the concept elicitation interviews have been described previously by Mannix and colleagues.20 Only additional results used to develop the MFIQ are summarized here.

Thirty‐two adults with migraine were interviewed. Most (n  =  27; 84%) were female and their mean (SD) age was 40.3 (11.3) years (Table 2). Participants reported experiencing 7.5 (4.1) (mean [SD]) migraine attacks each month. Characteristics of the sample of adults with migraine who participated in concept elicitation interviews are also included in Table 2.

Table 2

Participant Characteristics: Stage 1 (Concept Elicitation) and Stage 5 (Cognitive Interview) Samples

Participants Characteristics (self‐report)	CE Sample Overall	CI Sample Overall
	N = 32	N = 17
Episodic Migraine	21 (65.6%)	12 (70.6%)
Age (years), Mean ± SD	40.3 ± 11.3	39.5 ± 9.4
Range	18–58	20–53
Female gender, n (%)	27 (84.4%)	14 (82.3%)
Ethnicity, Not Hispanic or Latino, n (%)	28 (87.5%)	15 (88.2%)
Race (White), n (%)	26 (81.3 %)	11 (64.7%)
Migraine diagnosis duration (years), mean (SD)	14.3 ± 9.7	12.4 ± 11.3
Migraine with aura, n (%)	10 (31.2%)	5 (29.4%)
Taking prescription treatment for migraine to treat migraines when they occur	28 (87.5%)	17 (100%)
Employment status, n (%) *
Employed, full‐time	19 (59.4%)	10 (58.8%)
Employed, part‐time	10 (31.3%)	3 (17.7%)
Student	2 (6.3%)	1 (5.9%)
Unemployed	1 (3.1%)	2 (11.8%)
Disability	1 (3.1%)	2 (11.8%)
Highest level of education, n (%)
Secondary/high school	3 (9.4%)	2 (11.8%)
Some college	14 (43.8%)	6 (35.3%)
College degree	13 (40.6%)	7 (41.2%)
Postgraduate degree	2 (6.3%)	2 (11.8%)
Migraine interference with daily activities in the past week, n (%)
Not at all (0)	1 (3.1%)	5 (29.4%)
Mildly (1–3)	5 (15.6%)	2 (11.8%)
Moderately (4–6)	14 (43.8%)	5 (29.4%)
Markedly (7–9)	9 (28.1%)	5 (29.4%)
Extremely (10)	3 (9.4%)	0 (0%)
Missed work or school due to migraine‐related symptoms in the past week, n (%)	9 (28.1%)	5 (29.4%)

*Not mutually exclusive.

Major concepts discussed by 50% or more of the participants (outlined in Table 3) included physical function (acts), physical function (tasks), social function, leisure activities, and emotional function. Concepts discussed by fewer participants were considered as minor concepts and included: confidence, self‐esteem, sense of well‐being, health care resource use, work, eating, sleeping, concentrating, and thinking. Participants endorsed functioning concepts in each grouping of 5 successive interviews, indicating that saturation was achieved after ten interviews per participant group. Concept saturation for each functioning‐related concept was reached after the first 15 interviews (ie, no new concepts emerged in the final group of interviews) as shown in Table 4.

Table 3

Major Concepts of Interest Emerging from Stage 1 Concept Elicitation Interviews

Major Concepts Impact on …	% Sample Reporting	Example Quote	Concepts of Interest	Comment/Potential for Sensitivity to Change in Context of Use
Physical function – Acts	100	… If I get up in – if I’m laying in bed … my head will just start throbbing worse. (005–011 – 53 year old male with EM)	Impact of migraine on: movement of head movement of body Need to rest or lie down Walking Activities needing physical effort	High sensitivity to change
Physical function – Tasks	100	My children wanted to come over … and I just texted “headache.” I don’t even talk. (001–010 – 55 year old female with CM)	Impact of migraine on: usual daily activities (ie, home, school, work) activities requiring concentration or thinking clearly Making self‐presentable/getting ready for the day Activities in extreme sensations Routines, schedules and plans	High sensitivity to change
Social function	91	I just want to be left alone in the dark, by myself (005–012 – 44 year old female with EM)	Impact of migraine on: activities and usual social interactions with family, friends, and co‐workers being around other people talking with others intimacy with partner	Medium to low sensitivity to change
Leisure	91	I mean sometimes it’s hard to do family functions … if you just don’t feel well (005–007 – 33 year old female with EM)	Performance of hobbies, sports, community/religious‐related	Medium to low sensitivity to change
Emotional	97	It’s hindered our relationship … I mean he’s actually had to undress me, which is debilitating and it can be embarrassing (005–012 – 44 year old female with EM)	Feeling frustration Worry in response to migraine Disappointment in response to migraine Feeling like a burden Lack of control	Medium to low sensitivity to change

Table 4

Stage 1 Interviews: Evidence of Saturation of Concepts

Stage 1 Interviews (n = 35)†		Saturation of Concepts
		Impacts to Physical Function	Impacts to Social and Leisure Function	Impacts on Emotions
Episodic migraine	Group 1: interviews 1–5	New concepts = 9	New concepts = 8	New concepts = 9
	Group 2: interviews 6–10	New concepts = 0	New concepts = 0	New concepts = 0
	Group 3: interviews 10–15	New concepts = 1‡	New concepts = 2§	New concepts = 0
	Group 4: interviews 16–21	New concepts = 0	New concepts = 0	New concepts = 0
Chronic migraine	Group 5: interviews 22–27	New concepts = 0	New concepts = 0	New concepts = 0
	Group 6: interviews 28–35	New concepts = 0	New concepts = 0	New concepts = 0

†Saturation of concepts was evaluated in chronological interviews grouped by EM and CM participants.

‡Ability to keep a schedule; §Impact on social function related to (1) being around bright light and (2) being around loud noises.

When describing their experiences with migraine attacks and the effects on functioning, participants provided detailed accounts of how they often “powered through” the difficulties and how long they experienced difficulties with activities, or the level of difficulty in performing certain activities. Subjects with frequent and severe migraine attacks reported great difficulty with functioning, to the extent that they were sometimes unable to perform many everyday activities.

Stage 1. Development of the Conceptual Disease Model and selection of COI

The CDM (see online supplement) developed based on the results of these interviews was discussed with experts and revised to focus on concepts that were relevant to evaluate outcomes of preventive treatments for migraine, in the context of changes that may be perceived over 4‐ to 6‐month timeframe as described by Mannix and colleagues.20 The CDM illustrates the key concepts that reflect participants’ experiences of the disease (migraine symptoms and impacts of migraine on physical, social, and emotional functioning), and shows that some impacts are more temporally proximal to migraine symptoms. The CDM was further refined based on discussions with the PRO strategy team and in consultation with experts, as described by Mannix and colleagues,20 to ensure that the impacts of migraine discussed by patients were the result of migraine symptoms (and not comorbid conditions), and were relevant to the target sample. The PRO strategy team determined that to ensure that impacts relevant to most patients with migraine were selected, concepts from the CE interviews endorsed by at least 50% of the participants should be considered as major concepts, and therefore, considered as COI to be measured to evaluate functional outcomes.

The COI domain concepts are represented in 4 domains: impact on physical function, impact on usual activities, impact on social function, and impact on emotional function; the domains included in the conceptual framework are shown in Table 5. The physical function domain was crafted to include Acts and Tasks concepts in the Activity domain of the WHO ICF framework. Clinicians suggested that concepts, such as perceived difficulty concentrating, fatigue, and sleep disturbances, reported in the concept elicitation interviews were difficult to attribute solely to migraine. However, the impacts of these symptoms were important to CM and EM participants, and the item‐concepts were crafted to ensure that the impacts of these symptoms were included in the framework (eg, impact on activities requiring concentration).

Table 5

Content of the Conceptual Framework of the MFIQ version 0.1

Concept – Domain	Item Concepts
Impact on physical function (Acts)	Frequency of impacts on: ability to move ahead, ability to move body, ability to get in and out of bed, ability to stand up, ability to bend over, ability to walk around inside home, ability to walk at normal speed, ability to do activities needing physical effort, needing to rest or lie down
Impact on usual activities (Tasks)	Frequency of impacts on ability: to keep routine/schedule, to do activities with family or friends, to do chores outside of the home, to do usual activities, impact on plans, to do usual daily activities
	Level of difficulty/intensity of impact: to do usual household chores, to make self presentable, to do chores outside of the home, school or work, to take care of family, to do activities that require concentration, to do activities that require thinking clearly, impact on activities in extreme sensations – sound, smell, light
Impact on social and leisure activities	Frequency of impact on: being around other people, participating in social activities, talking with others, intimacy with partner, hobbies, on usual leisure activities
	Intensity of impact on usual social interactions
Impact on emotional functioning	Frequency of feeling: feeling worried because of migraine; like a burden on others because of migraine, feeling lack of control of life because of migraine
	Intensity/level of being: frustrated because of migraine, disappointed because of a migraine

Patient descriptions of the severity and frequency of concepts were considered for developing the scale for measuring these concepts (eg, frequency of occurrence, level of difficulty). CE interviews revealed that adults with migraine ‘powered through’ some essential activities like caring for others but limited activities like activities outside the home. It was therefore considered that the evaluation of functional outcomes of treatments should include concept not just to evaluate a reduction in the frequency of impact on functioning but also to potentially evaluate a reduction in the level of difficulty.

The variability of the patients experiences reported and the recommendation by Patrick and colleagues24 to select a recall interval that is as short as possible were taken in to consideration for determining the ideal recall period of the past 7 days, balancing recall bias and respondent burden.

The conceptual framework was reviewed and refined by the study team (AH, AS, SM), as well as the clinical experts and PRO development experts (including co‐authors MB and DB).

Stage 3: Review of instruments to select for use in trials

Twenty‐one PRO instruments were first reviewed20 for concepts covered in the conceptual framework. Instruments reviewed included 8 generic PRO tools and 11 headache specific tools.35 Three headache specific instruments were short‐listed as potentially relevant for measuring the impacts of migraine in the context of measuring outcomes of interventions to prevent migraine. These instruments were also frequently used in clinical studies evaluating outcomes of migraine therapies: HIT‐6,18 MIDAS,17 and MSQ.19

Reviewing the content of these 3 instruments found that none of the 3 short‐listed instruments reviewed, collected data about the impact of migraine on the following concepts: difficulty with movement during a migraine attack: difficulty with moving one’s head or body, bending over, walking, getting out of bed, and doing activities requiring physical effort (acts). Some impacts on everyday activities (tasks) reported by interview participants as occurring during and between migraine attacks (difficulty caring for others, impact on relationships, and being unable to do activities outside the home) were also missing in some of these instruments. Also, though the MSQ included items covering a majority of the concepts in the CF, all the items aimed to collect data only about the frequency of experiences. None of the items measured the level of difficulty or intensity of the experience. Some of the items included multiple impacts in the same item, which could lead to response errors since during CE interviews participants mentioned that they were able to do some essential activities with difficulty but were unable to do or avoided doing other activities. For example, one item asks about needing help for handling routine tasks, such as everyday household chores, doing necessary business, shopping or caring for others. A table mapping coverage of the concepts of interest in the 3 tools is available as an online supplement. The HIT‐6, MIDAS, and MSQ instruments have recall periods of 4 weeks or longer. Allowing a longer window in which attacks may occur is helpful in the case of lower frequency migraine. However, this could potentially compromise subjective recall about day‐to‐day variations in the impact of migraine and related symptoms.

Evaluating evidence of content validity (eg, items generated based on input from patients), the HIT‐6 and MIDAS were not developed based on patient input. While the MSQ publications mention interviews with patients, saturation of concepts was not reported.  Item generation was simply described as being based on, “a review of the migraine literature; items that, from patients’ responses, appeared to be sensitive to migraine treatment were selected from a questionnaire that was developed for administration to adults with migraine participating in a clinical trial; and items were generated based on discussions with migraine specialists and patients.”19 One‐on‐one patient interviews were conducted to evaluate patients’ responses to the questionnaire and further revisions were made based on psychometric analysis and additional one‐on‐one patient interviews, resulting in the MSQ v.2.0.19, 36

Because none of the existing instruments were considered fit‐for‐purpose for the context of use, a decision was made by the PRO development team (including SM, AS, and AH) in consort with experts (DM, MB), to design novel instruments to evaluate the impact of migraine on functioning. Two draft instruments were developed, one to collect data on the day‐to‐day variability of physical impacts: the Migraine Physical Function Impact Diary (MPFID), designed for completion everyday; and one to focus on physical, social, and emotional impacts, the Migraine Functional Impact Questionnaire (MFIQ). Additional information about the MPFID is reported elsewhere.20, 21

Stage 4. Item Generation and Development of the Draft MFIQ Version 0.1

Instructions, item stems, and response options were derived from patient language elicited during interviews to ensure appropriateness, relevance, understanding, and clarity of the items included in the measure and are described below.

Items were generated by the PRO development team using participants’ descriptions of the impact of migraine on functioning from the transcribed interview data. Participant coded concepts were drafted into items to collect data about the concepts in the conceptual framework. An iterative process was used, which involved reviewing the item concepts from the qualitative interviews, consulting the existing PRO item pool, and obtaining feedback from clinical and PRO experts. Issues identified during the translatability assessment were addressed based on recommendations to enhance readability and translatability. For example, the response options for a few items evaluating the severity of impact were revised from "Mildly" to "Slightly" to enhance readability and appropriateness for the concept being evaluated.

The 31 items that comprised version 0.1 of the MFIQ assess the impact of migraine on physical function, usual activities, social and leisure activities, and emotional functioning. An overall impact item was also added. Content includes concepts shown in Table 5 and a copy of this version is available for review as supporting information, in the online supplement.

The 7‐day recall period for this module was selected to help evaluate the perspectives of adults with migraine about the overall impact of the disease, ie, on days with and without (interictal) migraine. Item response choices for the function items were derived by examining how participants described the severity of their physical, social and emotional functional impacts over a period of time (in terms of duration or frequency or intensity of the experience over a period of time). For most items, a Likert‐type response scale approach using a 5‐point ordinal scale was chosen. Response options were chosen to reflect dimensions that were salient to all participants and relevant for recall over the past week. Four types of response options were used to tailor the options to evaluate the sub‐concept being measured by an item. The response option set ranging from never to always was used for items that were designed to collect data about how often a migraine impact was experienced; eg, impact on daily routine. The response option set not difficult to extremely difficult was used for items evaluating the level of difficulty; eg, doing or ability to do specific physical activities. The response option set not at all to extremely was used for items measuring the degree of limitation experienced due to migraine or effect of migraine; eg, impacts on usual activity. This set was also used for items measuring the degree of frustration about not being able to do what they needed to do. A few items include a does not apply option, in addition to the 5 response choices; eg, if the item about the impact on work/school does not apply to them.

Stage 5. Cognitive interviews (CI) Testing the Initial Drafts of the MFIQ

Seventeen adults with migraine participated in the cognitive interviews to assess the content validity of the MFIQ version 0.1. Of the 17 adults with migraine interviewed, (9 participants in Round 1 and 8 participants in Round 2), most (82 %) were female and their mean (standard deviation; SD) age was 39.5 (9.4) years. Participants reported experiencing 7.5 ± 5.3 (range: 1–20) migraine days each month. Sample characteristics are shown in Table 2.

The majority of cognitive interviews were conducted at domain‐concept level rather than item level to reduce participant burden. The cognitive interviews about the MFIQ were conducted after the item‐level debriefing of 17 items of the MPFID.

Examples of domain‐concept level questions included “Was there anything missing about physical functioning?,” “Were the questions about emotional functioning applicable to you?,” “What were the instructions asking you to do in your own words, and “What time period were you thinking about when you answered these questions?” Item‐level debriefing was conducted where time permitted using questions such as “Using your own words, what were you thinking when you read this question?” and “How did you pick your answer?”

Twelve of the seventeen participants (71%) reported that the questions on the MFIQ were relevant to their migraine experience. Two subjects (17%) felt some of the questions were irrelevant to them in the past week, because they had not had a severe migraine during the 7 days prior to the cognitive interview. Overall, participants who were probed on the sections of the MFIQ reported being able to understand the content of the items, instructions, and response options. After Round 1, 2 sentences were added to the instructions as a clarification, in response to patient feedback about the other instrument (MPFID) that was debriefed. To ensure that respondents considered the impact of migraine symptom and guiding them to think about the impacts of symptoms in addition to their pain, the new instructions stated, “We would like to understand how a migraine affects your day‐to‐day activities. Symptoms of migraine can include headache pain, nausea, vomiting or sensitivity to light or noise. We want you to think about the migraine symptoms that you experience and how they impact your day‐to‐day activities.” These instructions were well‐received and no further changes were recommended after Round 2.

Only a subset of participants reviewed each of the 31 items, as the interviews were onerous and followed item‐level debriefing of 17 items of the other instrument included in the study. A few changes were made to item stems of version 0.1 of the MFIQ, based on the feedback received from participants in Round 1, and a few examples are described in Table 6. Revisions were related to punctuation or were revisions to or the addition of examples to enhance clarity. The items about difficulty making oneself presentable and impacts on intimacy with a partner were substantially revised, to develop the MFIQ version 0.1, as shown in Table 6. No further changes were made to the version tested in Round 2, so the version was finalized as version 1.0, following translatability assessment and discussion with clinicians.

Table 6

Example of Changes Made to MFIQ Version 0.1, Following Round 1 Stage 5 Cognitive Interviews (n = 9)

Version 0.1 of Item Tested in Round 1 Interviews	Rationale for Change	Version 0.2 of Item Tested in Round 2 Interviews
In the past 7 days, how often did a migraine limit your ability to move your body? (eg, standing up, walking, bending?)	Punctuation changed to correspond with other items on the instrument, specifically the examples after the end of the item question were put in parenthesis	In the past 7 days, how often did a migraine limit your ability to move your body? (eg, standing up, walking, bending?)
In the past 7 days, how difficult was it to make yourself presentable? (eg, brushing hair, shaving, applying make‐up?)	Round 1 results indicated some participants were considering washing and dressing as examples. A review of concept elicitation interview results showed participants endorsed several aspects of this concept (eg, difficulty/avoiding brushing hair, difficulty/avoiding putting on makeup, not feeling attractive, avoiding shaving; impacts on showering/bathing, getting dressed/ready). Expert feedback suggested trying to add examples may result in too many examples, and suggested making item less specific. The item was modified to a generic term based on Round 1 CI results, review of CE results, and discussion with experts	In the past 7 days, how difficult was it to get yourself ready for the day?
		New item: In the past 7 days, how often did you have difficulty completing specific personal grooming activities? (eg, brushing hair, shaving, applying make‐up)
	Experts suggested that a separate question to assess allodynia specific impacts on grooming is needed. Especially since allodynia doesn’t disappear in between migraines. Experts suggested including 2 grooming questions, 1 allodynia specific and 1 general. CE and CI discussion of this concept by participants endorsed both allodynia and non‐allodynia related impacts on grooming, which conceptually supported the recommendation of the experts. Therefore, a new item was added to capture allodynia specific impacts of grooming activities
In the past 7 days, how often did a migraine prevent you from being intimate with your partner or spouse?	The concept was revised based on participant feedback to include other aspects of a relationship, in addition to intimacy, that can be affected by migraine	In the past 7 days, how often did a migraine interfere with your relationship with your partner or spouse?

Stage 6. Translations and testing of conceptual equivalence

A total of 146 participants with a mean age of 43 years (range: 18–79) were interviewed across all languages being tested; 73% were female. The following language versions were tested: German, Dutch, French, Bulgarian, Czech, Danish, Finnish, Greek, Hungarian, Spanish, Norwegian, Polish, Portuguese, Romanian, Russian, Slovak, Swedish, Italian, Turkish, and English.

Five native‐speaking participants were interviewed for each language/country combination, with the exception of Portugal where 6 adults with migraine were interviewed. The countries and translations completed for each are listed in the online supplement. The English adaptation process resulted in minor changes: “doing errands” was changed to “running errands,” and “inside the home” was changed to “inside your home” to create a universal English version. The translations were well‐understood and considered relevant, although some participants raised minor issues during interviews and required changes. For example, in Bulgarian, Italian, Norwegian, Polish, and Portuguese, the term “keep to your daily routine or schedule” posed challenges as “routine” was interpreted as a “schedule” or a “plan.” A more descriptive phrase, “the typical course of your day,” was used to address the issue.

Discussion

As migraine can be a debilitating disease that interferes with an individual’s ability to carry out everyday tasks, reducing the impact of migraine on functioning is an important objective of preventive treatment. In qualitative interviews, adults with migraine reported multiple impacts of migraine on their physical, social, and emotional functioning. They discussed how physical impacts from migraine attacks were immediate, resulting in inability to move or limiting movement (acts). Similarly, participants reported that the frequency and severity of attacks directly impaired their ability to do activities of daily living (tasks). The migraine attacks also limited social interaction and affected emotional function.

The MFIQ was developed following the methods recommended in the FDA PRO guidance10 and the ISPOR Good Research Practice23, 24 methods for instrument development to monitor outcomes of treatments. By including a range of items addressing the impact of migraine attacks on physical functioning, social and leisure activities, and emotional functioning, this new instrument offers a more comprehensive measurement of the impacts of migraine compared to existing instruments (such as the HIT‐6 or MIDAS).

The content of the MFIQ was generated and tested in adults with migraine and has been developed specifically to evaluate outcomes of preventive treatments for this disease that are relevant to adults with migraine. The 31 items in the MFIQ were developed following the methods recommended for the development of PRO instruments evaluating treatment benefit10, 23, 24 and includes items evaluating the impact on acts as well as tasks, as defined in the ICF framework,30, 31 demonstrating robust evidence of its content validity. The instrument also includes questions that assess the level of difficulty performing some tasks, as reported by adults with migraine during the CE interviews, in addition to assessing the frequency of impacts on other aspects of functioning unlike the MSQ that only assessed the frequency of impacts. Items were generated to tease out experience of impact on specific acts and tasks to avoid response error to items that aim to capture impacts on multiple tasks.

While patient involvement was reported for the development of the MSQ v.1.0;19 the authors have not reported methods for ensuring evidence of content validity as required by recent best practice methods and regulatory guidelines for PRO instrument development,10, 23, 24 eg, saturation of concepts in the target sample of migraine patients.

To avoid recall bias, shorter recall periods have been recommended for instruments measuring outcomes of conditions that can vary day‐to‐day.37, 38 Instruments like the MPFID21 are designed to be completed every day to support regulatory endpoints, however, daily administration is not practical for capturing impacts from migraine attacks which happen less frequently. The MFIQ offers the flexibility for completion once a week, and is less burdensome and more appropriate for capturing impact outcomes in the 4‐ to 6‐month time frame of medical follow‐up in clinical practice.

Cognitive interview participants confirmed that the instrument was comprehensive and covered the impacts of migraine that were most relevant to them. Translatability assessments at the time of item generation and subsequent revisions facilitated the development of conceptually equivalent linguistic adaptations for use in global studies.32 After development of the MFIQ in US English, 20 language versions were developed and tested for use in for 25 countries following methods recommended for ensuring linguistically and culturally valid versions.33, 34 Cognitive interviews in each country supported content validity and the process of harmonization ensured conceptual equivalence with the English version. The availability of these translated versions addresses a significant need for conceptually equivalent patient‐focused instruments that can be used in different countries. The methods support pooling and comparisons of data collected from different countries.

While the sample sizes for qualitative interviews are small, typical for qualitative research studies of this nature, the evidence of saturation demonstrated in the CE interviews supports the validity of the findings and relevance to adults with migraine. Though few participants were interviewed about comprehension of each of the 31 items of the MFIQ for the US English version, a systematic item‐level debrief was conducted with at least 5 migraine participants for each of the translated versions. The sample for the studies was limited to target clinical trial samples to reflect the context of use for which it was developed. The sample was predominantly female, reflecting the prevalence of migraine in the US.39 The cognitive interviews for the translated versions were however, conducted in a less restrictive sample (Table 1). While most participants were, non‐Hispanic white, the samples aimed to include some diversity of race and ethnicity.

Psychometric validation – including item analyses to inform item reduction, scoring, and the assessment of reliability, validity, and responsiveness to change – and additional linguistic adaptation are underway and will be reported separately. A version of the MFIQ (version 2.0, following item reduction) has been included in recent clinical trials of preventive treatment for migraine and in observational studies (NCT02456740 and NCT02483585). The MFIQ has also been mentioned as a PRO tool that can be used to support secondary endpoints in clinical trials of preventive treatment, in the recent ‘Guidelines of the International Headache Society for controlled trials of preventive treatment of chronic migraine in adults’.40 Requests for copies of the final version of the instrument may be addressed to the corresponding author.

Conclusion

A review of current clinical outcome assessments for use with people with migraine showed gaps in the coverage of functional outcome concepts that are important to adults with migraine. An instrument for evaluating patient‐relevant outcomes of migraine preventive therapies was developed using guidelines for best practice for PRO instruments to evaluate treatment benefit. In response to the gaps in the literature and best practice evaluation: two draft instruments were developed; one to collect data on the day‐to‐day variability of physical impacts, the MPFID, designed for completion every day; and one to focus on physical, social, and emotional impacts, the MFIQ. The MFIQ, a new PRO instrument with a recall period of the past 7 days, was developed for evaluating outcomes relevant to adults with migraine, and specifically assesses the impacts of migraine on physical functioning, everyday activities, social and emotional functioning, filling an unmet need among preexisting instruments for use with people with migraine. Evidence supporting content validity has been documented. All linguistic adaptations of the MFIQ were confirmed to be conceptually equivalent and well understood in the 25 countries evaluated. Research to evaluate the measurement properties of the MFIQ is ongoing. This will help ensure that the instrument is considered fit for purpose for use in global clinical trials as well as in clinical practice, to monitor outcomes of interventions aimed at preventing migraine.

Statement of Authorship

Category 1

(a) Conception and Design

Asha Hareendran, Sally Mannix, Sara Lavoie

(b) Acquisition of Data

Sally Mannix, Anne Skalicky, Asha Hareendran

(c) Analysis and Interpretation of Data

Sally Mannix, Anne Skalicky, Asha Hareendran

Category 2

(a) Drafting the Manuscript

Anne Skalicky, Sara Lavoie, Asha Hareendran, Sally Mannix

(b) Revising It for Intellectual Content

Sally Mannix, Anne Skalicky, Sara Lavoie, Asha Hareendran, Andrew V. Thach, Pooja Desai, Daniel D. Mikol, Dawn C. Buse, Martha Bayliss

Category 3

(a) Final Approval of the Completed Manuscript: Mannix, Skalicky, Gleeson, Hareendran*, Thach, Desai, Mikol, Buse, Bayliss

Category 4

(a) Final Approval of the Completed Manuscript

Sally Mannix, Anne Skalicky, Sara Lavoie, Asha Hareendran, Andrew V. Thach, Pooja Desai, Daniel D. Mikol, Dawn C. Buse, Martha Bayliss

Conflict of Interest

Asha Hareendran is an employee of Evidera and owns Pfizer stock; Sally Mannix, Anne Skalicky, and Sara Lavoie are employees of Evidera. Evidera received financial support from Amgen Inc. in connection with the implementation of the study; Andrew V. Thach, Pooja Desai, and Daniel D. Mikol are employees and shareholders of Amgen Inc; Martha Bayliss is an employee of Optum, a division of UnitedHealth Group, which has consulting engagements with many pharmaceutical companies, including Amgen; Dawn C. Buse has received grant support and honoraria from Allergan, Amgen, Avanir, Eli Lilly, Teva, and Promeius. She is on the editorial board of Current Pain and Headache Reports, the Journal of Headache and Pain, Pain Medicine News, and Pain Pathways magazine.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.