Psychometric parameters of food allergy quality of life during an allergen immunotherapy trial.

BACKGROUND: The Food Allergy Quality of Life Questionnaire-Parent Form (FAQLQ-PF) is a commonly used patient-reported outcome measure in food allergy (FA) research. It was developed before FA treatment clinical trials were commonplace and is used as a secondary outcome measure in pivotal FA treatment trials. We examined the psychometric properties of the FAQLQ-PF and its relevance to children with peanut allergy engaged in an epicutaneous immunotherapy (EPIT) clinical trial.
METHODS: Analysis was performed on 26 universally answered items of the FAQLQ-PF, from assessments undertaken during the phase 3 PEPITES study (baseline, Month 12), which examined the safety and efficacy of EPIT for children with peanut allergy aged 4-11 years. Item response theory (IRT) was used to assess psychometric parameters of the FAQLQ-PF (i.e., discrimination, difficulty, and information). Confirmatory factor analysis was also employed; reliability was assessed using McDonald's omega (ω) and Cronbach's alpha (α).
RESULTS: A total of 23 of 26 items presented very high discrimination levels (>1.7), and all 26 fell within the recommended difficulty threshold (between -1.5 and 1.5). The items contributed a reasonable information level for their respective factors/subdomains. The measure also presented a marginally acceptable model fit for the 3-factor structure (e.g., comparative fit index = 0.88, Tucker-Lewis index = 0.87) and good reliability levels across time points (ω and α > 0.90).
CONCLUSIONS: Herein, we present a novel reanalysis of the FAQLQ-PF items using IRT. The longitudinal performance of individual items and subscales was corroborated, and items with the highest discrimination were identified, showing that the tool is suitable for longitudinal measurements in FA treatment trials.

INTRODUCTION

Peanut allergy is associated with reduced health‐related quality of life, , , , which can be assessed with disease‐specific instruments such as the well‐validated Food Allergy Quality of Life Questionnaire (FAQLQ) instruments. , , These instruments were designed and validated for the cross‐sectional assessment of food allergy quality of life (FAQL). Now, with rapidly expanding treatment options being investigated for individuals with food allergies, these instruments are increasingly being used to examine potential changes in FAQL over time during clinical trials of food allergy immunotherapy. , However, these tools were not explicitly designed or validated to be used in this setting, given their development predated all but one food therapy trial (TNX‐901).

Because a scale's measurement properties can influence the interpretation of clinical trials and research results and determine its appropriateness for use in such contexts, it is essential to understand the psychometric properties of the FAQLQ more precisely. Item response theory (IRT) is a method for nuanced item analysis that analyzes the performance of individual items within patient‐reported outcomes (PROs) assessments (such as those measured by the FAQLQ). Using IRT compared with traditional classical test theory (e.g., exploratory and confirmatory factor analysis [CFA]) provides more detailed and more accurate descriptions of the item and scale‐level performance. This is because classical test theory focuses on the structure of the measure and its external/internal validity, whereas an IRT approach offers more specific information about the items (e.g., discrimination, difficulty, and information). This information is obtained through mathematical formulas, expressing the relationship between observed and hypothetical variables, called latent traits. IRT assumes that the latent construct (in this case, FAQL) and items of a measure (e.g., the FAQLQ‐PF [FAQLQ–Parent Form]) are organized along a continuum. One of the technique's primary purposes is to establish the individual's position on that continuum. The idea of a continuum is helpful in the context of PROs for 2 main reasons: (1) The intervals between scores on a measure's response scale cannot necessarily be assumed to be equivalent (e.g., 1–2 vs 4–5 may not represent equal degrees of change); (2) the weight, importance, or impact of each question in the questionnaire may differ depending on the particular respondent. It also demonstrates equivalence across different subgroups, which is valuable for measures that are used in different settings.

In this post hoc analysis, we assessed the discrimination, difficulty, and information levels of each item of the FAQLQ‐PF using IRT to understand its relevance to children with peanut allergy involved in immunotherapy trials. The longer‐term objective is to provide data to improve and streamline the assessment of health‐related quality of life during food allergy treatment. Datasets were derived from PEPITES, a global phase 3, pivotal, double‐blind, placebo‐controlled trial of epicutaneous immunotherapy for peanut allergy in children aged 4–11 years that evaluated the safety and efficacy of Viaskin™ Peanut (DBV712) 250 µg for peanut allergy.

METHOD

Details of study designs, subject characteristics, and primary outcomes of the PEPITES study have been published previously. Briefly, PEPITES was a phase 3, multicenter, randomized, double‐blind, placebo‐controlled clinical trial of Viaskin Peanut, conducted between January 2016 and August 2017 in 4‐ to 11‐year‐old children with peanut allergy. Three hundred fifty‐six participants (238 Viaskin Peanut 250 µg, 118 placebo) were enrolled from 31 sites in Australia, Canada, Germany, Ireland, and the United States. Overall, 79.5% of subjects were from North America, 13.8% from Europe (Ireland and Germany), and 6.7% from Australia. A total of 81.5% of subjects identified as Caucasian, 7.6% as Asian, and 0.8% as African American. Among other inclusion criteria, participants were required to react to an eliciting dose of ≤300 mg peanut protein on baseline double‐blind, placebo‐controlled food challenge (DBPCFC) and meet dual minimum peanut skin prick test and serum peanut immunoglobulin E criteria.

Participant FAQL was assessed using the FAQLQ‐PF for all subjects and the FAQLQ–Child Form (FAQLQ‐CF) for subjects 8 years and older at entry and following 12 months of treatment as a prespecified endpoint. The FAQLQ‐CF is only validated in persons 8 years of age and older. The FAQLQ assessment at Month 12 was performed prior to the DBPCFC that was used to assess the desensitization effect of Viaskin Peanut treatment, meaning that all assessments were obtained both prior to unblinding of study arm and challenge outcome.

We chose to assess 26 of the 30 possible items of the FAQLQ‐PF because items 1–26 are completed by the parent (from the child's perspective) in children aged 4–12 years and applied to all participants in the sample. The remaining items, 27–30, are completed only on behalf of children aged 7–12 years and were only completed by the older age‐group in the sample. The FAQLQ‐PF has previously been shown to have 3 domains (food‐related anxiety [FRA], social and dietary limitations [SDL], and emotional impact [EI]) derived through exploratory factor analysis during the initial validation of the index.

In this study, we assess item quality within each of these domains, considering the following parameters: (1) Discrimination parameter (a) is an index of how well an item in the FAQLQ can differentiate participants with varying levels of FAQL; the higher the value, the higher the item is discriminative between those with low and high FAQL. For discrimination, Baker's classification was adopted: very high (>1.70), high (1.35–1.69), moderate (0.65–1.34), low (0.35–0.64), and very low (0.01–0.34) discrimination. (2) Difficulty parameter (b) indicates the FAQL level that the individual must endorse to select the next higher response option category on the response scale. More difficult items tend to be endorsed only by those individuals who report higher scores of FAQL. In contrast, easier items tend to be endorsed by a broader range of individuals. Items that are neither too easy nor too difficult are recommended (i.e., means across b1–b6 between −1.5 and 1.5). , (3) Item/test information curves evaluate how much information an item shares with the total information of the measure. Items with a higher level of information are more informative, indicating higher measurement precision, less measurement error, and higher scale reliability.

Data were analyzed using R software. All the analyses were performed twice—at baseline and Month 12. IRT analysis (i.e., discrimination, difficulty, and informative curves) was performed using the multidimensional IRT package. The graded response model was considered in this package, as the FAQLQ uses a scale with more than 2 response categories (e.g., 0–6 scale). Additionally, we performed CFA with the weighted least square mean and variance adjusted (WLSMV) estimator, considering the following model fit indices , : the comparative fit index (CFI) and Tucker–Lewis index (TLI), which are both recommended to be higher than 0.90, and the root mean square error approximation (RMSEA), which must be lower than 0.08. Also, we assessed the reliability levels of the measure using McDonald's omega (ω) and Cronbach's alpha (α). Both ω and α are considered adequate when above 0.70. Missing data were not imputed as in IRT; a missing response does not contribute to the item's measurement.

Ethics approval for the post hoc analysis of prespecified data from the PEPITES study was not required.

RESULTS

Subjects

Data from fully completed FAQLQ‐PF assessments were available for 326 (106 placebo and 220 Viaskin Peanut 250 µg subjects) of 356 randomized subjects' parents/caregivers at baseline and for 306 subjects (97 placebo and 209 Viaskin Peanut 250 µg subjects), which were completed in full at both time points (baseline and Month 12). The median (interquartile range) age of the 326 subjects included in this analysis was 7 years (6–9), and 62% were men. The analysis population consisted of 91% and 86% of the randomized study population.

Item parameters: discrimination, difficulty, and information

Twenty‐three FAQLQ‐PF items assessed at baseline showed very high levels of discrimination (α > 1.7) (Table 1) both at baseline and Month 12. The other 3 items had either high (1.35–1.69) or moderate (0.65–1.34) discrimination. At baseline, Item 20 (FRA) and Item 18 (SDL) were the most discriminative, whereas Item 26 (EI) and Item 08 (SDL) were the least discriminative. At Month 12, Items 20 and 21 (FRA) were the most discriminative, whereas, once again, Items 26 and 08 were the least discriminative. Only Item 26 showed a difficulty level below the recommended threshold at baseline and Month 12. The full discrimination and difficulty values tables are available as Appendix S1 (See Tables S1 and S2).

TABLE 1

Item parameters of the FAQLQ–PF at baseline and Month 12

Item	Discrimination—a		Difficulty—b(m)
	TM = 0	TM = 12	TM = 0	TM = 12
Because of food allergy, my child…
Food‐related anxiety
1. … feels anxious about food	2.769	2.405	−0.08	0.14
4. … feels afraid to try unfamiliar foods	2.105	2.253	−0.26	−0.12
5. … feels concerned that I am worried that he/she will have a reaction to food	1.839	1.955	0.08	0.39
16. … feels anxious when going to new places	3.334	3.189	0.50	0.68
17. … feels concerned that he/she must always be cautious about food	2.912	3.06	−0.33	−0.15
20. … feels anxious about accidentally eating an ingredient to which he/she is allergic	3.516	3.751	−0.21	−0.03
21. … feels anxious when eating with unfamiliar adults/children	2.966	3.443	0.10	0.15
Social and dietary limitations
3. … feels frustrated by dietary restrictions	2.025	1.998	−0.02	0.07
8. … has a lack of variety in his/her diet	1.095	1.117	0.86	1.07
12. … 's social environment is restricted because of limitations on restaurants we can safely go to as a family	2.509	2.217	−0.60	−0.42
13. … 's social environment is restricted because of limitations on holiday destinations we can safely go to as a family	1.891	2.169	−0.24	−0.10
14. … 's ability to take part has been limited in social activities in other people's houses (sleepovers, parties, playtime)	2.067	2.121	−0.43	−0.07
15. … 's ability to take part has been limited in preschool/school events involving food (class parties/treats/lunchtime)	2.18	2.254	−0.39	−0.24
18. … feels “left out” in activities involving food	3.454	3.162	−0.22	−0.10
22. … feels frustrated by social restrictions	2.781	3.369	0.14	0.23
Emotional impact
2. … feels different from other children	1.974	1.906	0.02	0.24
6. … experiences physical distress	1.9	1.946	1.25	1.45
7. … experiences emotional distress	3.128	3.025	0.41	0.54
9. … has been negatively affected by receiving more attention than other children of his/her age	2.151	2.045	0.70	0.91
10. … has been negatively affected by having to grow up more quickly than other children of his/her age	2.055	2.195	0.61	0.71
11. … has been negatively affected by his/her environment being more restricted than other children of his/her age	2.24	2.188	−0.02	0.22
19. … feels upset that family social outings (eating out, celebrations, days out) have been limited by food allergy	2.024	2.113	0.31	0.48
23. … is more anxious in general than other children of his/her age	2.232	2.713	0.47	0.61
24. … is more cautious in general than other children of his/her age	1.681	1.661	−0.33	−0.08
25. … is not as confident as other children of his/her age in social situations	1.795	2.188	1.04	1.12
26. … wishes his/her food allergy would go away	1.418	1.22	−1.63	−1.77

a = discrimination levels; b(m) = means between b1–b6.

Next, we assessed each item's information level, as shown in the Appendix S1 (See Figures S1–S6). The information curves with a higher I(θ) (i.e., higher curve) are more informative. With a few exceptions (Items 08 and 26, at both baseline and Month 12), results showed that the items shared reasonable information with their respective domains. Also, the sum of the contribution of all items that compose the FAQLQ factors generates the total information curve (Figure 1A–C). The EI and FRA domains were the most informative, with a reasonable spread of information across their respective factors. All the domains were associated with reasonable (and very similar) information levels across the 2 time points, which show them to be reliable factors.

FIGURE 1

(A) Test information curves (food‐related anxiety). Unbroken line = baseline; dashed line = Month 12. (B) Test information curves (social and dietary limitations). Unbroken line = baseline; dashed line = Month 12. (C) Test information curves (emotional impact). Unbroken line = baseline; dashed line = Month 12

Confirmatory factor analysis

Using the WLSMV estimator, baseline FAQLQ‐PF responses had the following model fit: CFI = 0.88, TLI = 0.87, and RMSEA = 0.074, indicating a marginally acceptable fit for the 3‐factor structure. The results were similar for Month 12 responses: CFI = 0.88, TLI = 0.87, and RMSEA = 0.077. All factorial weights (lambdas) were statistically different from zero (λ ≠ 0; z > 1.96, p < .01).

Reliability

We used McDonald's omega (ω) and Cronbach's alpha (α) to assess the reliability levels. Results indicated good internal consistency for all domains across both stages : FRA (stages 1 and 2, ω, and α = 0.93), SDL (stage 1, ω, and α = 0.90; Stage 2, ω, and α = 0.91), and EI (stage 1 and 2, ω, and α = 0.92). The full measure also was associated with high reliability levels (stage 1, ω, and α = 0.96; stage 2, ω, and α = 0.97).

DISCUSSION

Item response theory, used extensively in educational testing applications, has gained popularity and acceptance in PRO‐related research, which is vital to assessing treatment outcomes. Although the US regulatory authorities will not accept PRO data as primary outcome measures for product approval, they are increasingly recognized by regulators, clinicians, and patients as valuable tools to collect patient‐centered data. , Indeed, the US Food and Drug Administration has encouraged their incorporation in clinical trials and has recently issued guidance for the industry on the use of core PRO measures in clinical cancer trials in 2020 and 2021.

Item response theory analyzes the individual parameters of items to ensure measurement quality, providing a deeper and more accurate description of item‐level PRO questionnaire data. In particular, the technique creates a link between the items (or questions) on a measure (i.e., the FAQLQ), the individuals responding to these items, and the underlying construct being measured (i.e., FAQL). As a result of this synergy, in any particular context (e.g., food immunotherapy), items with the best parameters will provide more robust results. The technique has been applied successfully in many settings, including in health outcomes research in oncology, multiple sclerosis, , and diabetes. This success led the Patient‐Reported Outcomes Measurement Information System initiative to recommend the use of IRT as a practical approach to examining the construct validity of PROs.

We considered 3 crucial individual parameters for our analyses: discrimination, difficulty, and information. More specifically, for an item to be adequate to the composition of the measure, it must (1) be able to discriminate participants between different levels on the latent trait or construct (i.e., FAQL) ; (2) be neither too easy nor too difficult (suitable for patients with different levels of FAQL, from poor to excellent) , ; and (3) provide sufficient information to the total score of the measure (i.e., the FAQLQ). Unlike classical test theory (which is typically used to develop or refine health questionnaires), IRT considers the number of questions answered correctly and the difficulty of the questions answered. For example, 2 respondents (A and B) received a total score of 3.2 but had higher scores for some of the questions and lower scores for others. Although they had the same score, their food allergy burden may not be the same if the questions that make up the total score are “easier” for A and more “difficult” for B.

The results were consistent across both baseline and Month 12 datasets. Most items had difficulty within the recommended level and contributed a reasonable amount of information to their respective domains. Also, 23 of the 26 items in the FAQLQ‐PF had very high discrimination levels. Item 08 had moderate discrimination levels at baseline and Month 12, and Item 26 had difficulty levels slightly beyond the recommended threshold. It is essential to highlight that these results are expected due to the general content of these items. For instance, Item 26 states that “… wishes his/her food allergy would go away,” which is something with which all those affected by the disease would likely agree, logically. Therefore, this item would be too “easy.” Despite the results in this item's parameters, it is unlikely that its inclusion to measure FAQL will influence the research outcome when all findings are taken together.

Furthermore, CFA showed a marginally acceptable model fit for the 3‐factor structure. Such findings confirm that the 3 domains of the FAQLQ (FRA, SDL, and EI) provide a solid theoretical framework, making the FAQLQ‐PF a robust tool for assessing the impact of immunotherapy on the concept of FAQL. The reliability levels provide further evidence of the quality of the measure across time points for both the factors in isolation and for the full measure itself. Limitations of the analysis include that not all parents completed in full the FAQLQ at either baseline or follow‐up at Month 12. However, as our sample is originated in one of the most extensive food allergy immunotherapy trials conducted to date, the results are likely generalizable across the whole study population. Given that this population sample was obtained in the context of a phase 3 clinical treatment trial, offered only at particular institutions and geographic regions, which may cluster to more severe or difficult‐to‐manage cases, there may be some selection bias. Furthermore, the majority of the subjects were from North America, and the relatively smaller contribution of subjects from Europe and Australia did not allow for analysis of variability of findings across regions. Additional selection bias may result from a clinical trial population that may differ considerably from the general population with food allergy. IRT is a theory of content analysis and not an exclusive method of doing so, and, thus, other analytic techniques may not lead to the same conclusions reached herein. This IRT was performed in the context of peanut allergy for this index, and, in theory, this may differ when used in the context of other allergens.

FINAL CONSIDERATIONS

The FAQLQ was developed before most current research into treatments was performed. The index themes may be more reflective of the world before recent labeling laws, stock epinephrine legislation, and the advent of more treatment trials and clinical practice oral immunotherapy. Thus, the clinical impact of certain items in the final rating of the initial sample at the time may differ from how a current sample may rate the item's clinical impact, which would affect item inclusion/exclusion. The use of IRT may be a strategy to ensure the items included in a PRO measure have suitable parameters that can be reliably used to assess the FAQL construct, ensuring robust results. Items with unacceptable parameters may bias the analyses, leading to questionable results. IRT can also help find causative agents underlying the score differences and match them to the subgroup/group characteristics.

We plan to use the current analysis to validate further a short form of the FAQLQ that may be more easily administered across treatment settings than the current full‐length tool. Our findings provide a basis for selecting the most sensitive items to streamline the FAQL assessment process and create a more sensitive and robust tool for evaluating the population with peanut allergy in future clinical trials and clinical practice of food allergen immunotherapy. Moreover, our IRT findings might support the development of computerized adaptive testing (CAT) applications using the FAQLQ. Whereas questionnaires use the same questions for all respondents, CAT uses an algorithm to tailor the items presented based on respondents' level of FAQL. That is, they will answer a specific initial question and, based on their level of agreement with it, will be presented with different (and reduced) sets of items. The use of IRT to support CAT has been increasing over the years, especially for PRO domains.

CONFLICT OF INTEREST

Dianne E. Campbell and Todd D. Green are employees of DBV Technologies. Audrey DunnGalvin has received research grants from Aimmune Therapeutics, National Children's Research Centre, DBV Technologies, and Food Allergy Research and Resource Program, and other research support from Safefood Ireland, and has served as a consultant and/or advisory board member for Aimmune Therapeutics, Atlantia Clinical Trials in Food Ireland, and Anaphylaxis Ireland. Matthew Greenhawt is a consultant for Aquestive, is a member of physician/medical advisory boards for DBV Technologies, Sanofi/Regeneron, Nutricia, Novartis, Aquestive, Allergy Therapeutics, AstraZeneca, ALK‐Abello, and Prota, with all activity unrelated to vaccines/vaccine development or COVID‐19 treatment, is an unpaid member of the scientific advisory council for the National Peanut Board and medical advisory board of the International Food Protein‐Induced Enterocolitis Syndrome Association, is a member of the Brighton Collaboration Criteria Vaccine Anaphylaxis 2.0 working group, is the senior associate editor for the Annals of Allergy, Asthma, and Immunology, and is member of the Joint Taskforce on Allergy Practice Parameters. He has received honorarium for lectures from ImSci, MedLearningGroup, RMEI Medical Education, and multiple state/local allergy societies. He received past research support ending in 2020 from the Agency for Healthcare Quality and Research (K08‐HS024599). Jonathan O'B Hourihane reports advisory board fees, Aimmune Therapeutics; speakers bureau, Aimmune Therapeutics, Nutricia, and Mead Johnson; grants to institution/research funding; and clinical trials within past 2 years, Aimmune Therapeutics, Johnson& Johnson, National Childrens' Research Centre Ireland, Dublin Skin and Cancer Hospital Charity, Temple St Foundation, and Clemens von Pirquet Foundation. David M. Fleischer has received research support to his institution from DBV Technologies and Aimmune Therapeutics; reports clinical medical advisory board membership with DBV Technologies; has served as a consultant for DBV Technologies, Allergenis, Aquestive Therapeutics, Aravax, Genentech, Nasus Pharma, and Intrommune Therapeutics; and has received honorarium for lectures from DBV Technologies and royalties from UpToDate. Marcus Shaker has participated in research funded by DBV Technologies, is a member of the Joint Task Force on Practice Parameters, is an associate editor for Annals of Allergy, Asthma, and Immunology, and serves on the Editorial Boards for the Journal of Allergy and Clinical Immunology In Practice and the Journal of Food Allergy. Philippe Bégin has received grants through his institution from DBV Technologies during the conduct of the study; personal fees from ALK, Novartis, Sanofi, Bausch Health, Aralez, and Pfizer; and grants from Regeneron, Sanofi, Novartis and the Canadian Allergy, Asthma, and Immunology Foundation outside the submitted work. The following authors have no conflicts of interest to disclose: Gang Chen and Gabriel Lins de Holanda Coelho.

AUTHOR CONTRIBUTIONS

Gabriel Coelho and Audrey DunnGalvin contributed to study design, analysis, and manuscript preparation. Todd Green and Dianne Campbell contributed to study design and manuscript preparation. All the other authors contributed to manuscript preparation and review.

App S1

Click here for additional data file.