Development of Clinician-Reported Outcome (ClinRO) and Patient-Reported Outcome (PRO) Measures for Eyebrow, Eyelash and Nail Assessment in Alopecia Areata.

BACKGROUND: Eyebrow and eyelash hair loss and nail damage-in addition to scalp hair loss-are important signs/symptoms of alopecia areata (AA) to patients and deserve assessment in AA clinical trials.
OBJECTIVES: Our objective was to develop clinician-reported outcome (ClinRO) and patient-reported outcome (PRO) measures and accompanying photoguides to aid in the assessment of AA-related eyebrow, eyelash and nail signs/symptoms.
METHODS: Iterative rounds of qualitative, semi-structured interviews were conducted with US expert dermatologists and North American patients with AA. Patients with eyebrow, eyelash and nail involvement were purposefully sampled. Interview transcripts were qualitatively analyzed.
RESULTS: Dermatologists (n = 10) described eyebrow and eyelash loss as concerning for affected patients and, along with nail appearance, as deserving assessment. Dermatologist data informed the development of single item, 4-point Likert-type ClinRO and PRO measures of current eyebrow loss, eyelash loss and nail appearance and a PRO measure of eye irritation. Patients (n = 45, age 15-72 years) confirmed the importance and relevance of these signs/symptoms. Interim revision resulted in measures that were understood by and relevant to patients. Dermatologists (n = 5) and patients (n = 10, age 21-54 years) participated in the development of the eyebrow, eyelash and nail photoguides and confirmed that they included photos that appropriately represented different severity levels and were helpful to derive and standardize ratings across raters.
CONCLUSIONS: The ClinRO and PRO measures for eyebrow, eyelash and nail appearance, with their accompanying photoguides and the PRO Measure for Eye Irritation provide clear and meaningful assessments of outcomes important to patients with AA.

Key Points

Introduction

Understanding and evaluating the various manifestations of diseases and patients’ treatment experiences is critical for the development and evaluation of new therapies [1]. Current clinical assessment guidelines for alopecia areata (AA) focus on scalp hair loss, with secondary assessment of “body hair loss” and “nail involvement” [2-4]. However, it is known that AA can also cause eyebrow and eyelash hair loss [5, 6] and that the loss of these defining facial features is physically and psychologically bothersome to patients [7, 8]. Eyebrows are also important for facial recognition [9]. Additionally, nail changes are not uncommon, affecting 7–66% (average 30%) of patients with AA [10].

Fit-for-purpose clinical outcome assessments (COAs) are required to evaluate the efficacy of treatment on these key non-scalp signs/symptoms in AA clinical trials and to complement clinician and patient assessments of scalp hair loss [11, 12]. However, to our knowledge, no publicly available outcome measures to evaluate eyebrow hair loss, eyelash hair loss and nail damage have been developed in conjunction with industry guidelines [13].

This series of cross-sectional, non-interventional, qualitative interview studies aimed to develop clinically relevant and usable: (1) clinician-reported outcome (ClinRO) measures, (2) patient-reported outcome (PRO) measures and (3) accompanying photoguides to guide clinicians and patients to derive scores related to these measures and help standardize ratings by providing clear visual instruction for assessment of these non-scalp signs of AA.

Methods

Qualitative Approach and Research Perspective

This study was designed to develop novel PRO and ClinRO measures through methodology aligned with the US FDA PRO Guidance for Industry [13] and good research practice recommendations (Fig. 1) [14]. Draft measures were initially developed by expert clinicians (FPN, BAK) with input from COA experts (KWW, HK, SK, NVJA). Iterative sets of qualitative interviews and a small panel meeting (SPM) comprising expert dermatologists were then conducted to develop the measures and accompanying photoguides.

Fig. 1

Iterative development of clinician-reported outcome (ClinRO) and patient-reported outcome (PRO) measures and accompanying photoguides. AA alopecia areata

Interview Conduct

The study protocol was approved by Western Institutional Review Board (ref. #20171820). All interviews were conducted between October 2017 and September 2018 by a qualitative interviewer trained in COA development techniques. Interviews were audio-recorded and transcribed verbatim. Separate semi-structured interview guides were used in each interview set/round and included concept elicitation (open-ended and probed questioning to systematically explore dermatologists’ and patients’ insights into AA) and/or cognitive debriefing [14, 15] to evaluate acceptance and interpretation of the ClinRO and PRO measures.

Sampling

Expert dermatologists practicing in the USA were identified by their involvement in contemporary research and treatment of patients with AA. Eligible patients in North America were identified by dermatologists at the University of California Irvine, Yale University, Northwest Dermatology Research Center, SKiN Centre for Dermatology and Dawes Fretzin Dermatology. Patients were eligible for inclusion if they had severe AA, determined by present or recent experience of ≥ 50% scalp hair loss. Purposive sampling targeted the inclusion of patients with current or previous eyebrow, eyelash and/or nail involvement to ensure that the PRO measures and accompanying photoguides were reviewed by patients with relevant AA experience. Full eligibility criteria for each round of the study are provided in the Online Resource 1.

Analysis

Interview transcripts were coded using ATLAS.ti version 7.5. Thematic analysis [16] took a phenomenological approach, focusing on the perceptions, feelings and lived experiences of participants [17]. Conceptual saturation, the point at which no new concept-relevant information emerges [18], was explored to guide patient interview sampling and analysis. Saturation was achieved in round 1, indicating understanding of the elicited sign/symptom concepts [14, 19]; round 2 interviews then sought to refine and confirm the PRO and ClinRO measures. Cognitive debriefing data were subject to framework analysis [20] to identify the relevance and appropriateness of item wording, response options and recall period.

Results

Development of Clinician-Reported Outcome (ClinRO) and Patient-Reported Outcome (PRO) Measures

Sample

Ten expert dermatologists, with 6–38 years (mean 21.2) of experience in managing patients with AA, and 45 patients with AA, aged 15–72 years (mean 33.5), participated in these interviews (round 1 [n = 30]; round 2 [n = 15]). A diverse patient sample was recruited: 58% (n = 26) of participants were female, 44% (n = 20) were non-Caucasian, 53% (n = 24) reported a high school diploma or lower education level and 78% (n = 35) had partial/full eyebrow and/or eyelash hair loss. Full clinical and demographic characteristics are provided in Online Resource 2.

Identification of Measurement Concepts

Dermatologists described eyebrow and eyelash hair loss as facial defining features that changed patients’ appearance and were difficult to conceal. In clinical practice, dermatologists assessed eyebrow hair loss primarily through consideration of coverage/continuity (i.e., absence of gaps), although some dermatologists considered density (fullness/thickness) and symmetry (across both eyes). Similarly, dermatologists assessed eyelash hair loss through consideration of continuity and symmetry.

Patients reported eyebrow hair loss (n = 28/35 [80%]) and/or eyelash hair loss (n = 25/35 [71%]) among their three most bothersome signs/symptoms of AA. Patients described feeling embarrassed and self-conscious about their eyebrow hair loss and/or eyelash hair loss, expressing a desire to look “normal” or cosmetically acceptable. Some described concealing their eyebrow loss using makeup or hats/glasses. Patients who experienced eyebrow hair loss (n = 38) primarily described their eyebrow hair loss in terms of gaps/patches, thinness, density and/or symmetry. Patients who experienced eyelash hair loss (n = 32) described the gaps as the most important aspect of their eyelashes to consider in evaluation.

Many (n = 22) patients also described functional impacts of eyebrow and/or eyelash hair loss, including problems with sweat (n = 6) or debris (n = 13) getting into their eyes, irritation from the wind (n = 2) or difficulty seeing because of glare or sunlight (n = 4). These patients described the sensation as “itching,” “burning,” “dry,” or “like something being stuck in the eye.”

All dermatologists described AA-related nail involvement as “pitting/pits” (n = 6), “breaking”/“splitting”/“brittleness” (n = 4), “dystrophy” (n = 2), “roughness” (n = 2), and “shedding” (n = 1). Patients with nail damage (n = 14) described their nails as pitted, split, bumpy, grainy, indented and/or feeling brittle, thin and/or weak. For some participants, nail damage was an aesthetic issue, whereas for others it caused functional problems (most did not experience any nail pain). The SPM identified that the severity of nail damage was more important than the number of nails that were damaged.

From these insights, it was identified that PRO and ClinRO measures to assess eyebrow hair loss, eyelash hair loss, eye irritation and nail appearance should be developed as both dermatologists and patients considered these to be important signs/symptoms of AA. Patient and dermatologist quotes that support measurement concepts are presented in ESM 3.

Development of ClinRO and PRO Measures to Assess Eyebrow and Eyelash Involvement

The ClinRO measures for eyebrow hair loss and eyelash hair loss underwent three iterative revisions based on feedback provided in the dermatologist interviews (ESM 4 and 5). A four-option response scale was confirmed by dermatologists as appropriate to allow assessment of incremental severity on each ClinRO measure. The final version of each ClinRO measure was confirmed during the SPM and is presented in Table 1. These measures were also reviewed by patients, time permitting. In round 2, almost all patients who were consulted considered the final ClinRO Measure for Eyebrow Hair Loss™ (n = 9/11) and the final ClinRO Measure for Eyelash Hair Loss™ (n = 9/11) to be important for evaluating treatment outcomes.

Table 1

Final versions of ClinRO measures for eyebrow and eyelash hair loss, and PRO measures for eyebrows and eyelashes

The ClinRO Measure for Eyebrow Hair Loss™, ClinRO Measure of Eyelash Hair Loss™, PRO Measure for Eyebrows™, PRO Measure for Eyelashes™ and accompanying photoguides (ESM 10) are subject to copyright owned by Eli Lilly and Company. Permission to use is granted under Creative Commons attribution-No derivatives 4.0 International License. Contact Eli Lilly and Company in the event of any proposed modification

ClinRO clinician-reported outcome, PRO patient-reported outcome

The PRO measures were reviewed in dermatologist interviews and in the SPM; several changes were suggested to more closely align PRO measure wording with that of the ClinRO measures. Versions of the PRO measures were then tested with patients over two rounds of interviews with interim revision between rounds. The changes resulting from patients’ feedback are reported in ESM 4 and 5. In round 2 interviews, all patients could provide responses to each PRO measure and generally understood the item wording and severity gradations within the response scale options. Therefore, no further changes or testing were required. The final version of each PRO measure is presented in Table 1.

Development of a PRO Measure to Assess Eye Irritation

A PRO measure to assess eye irritation was developed; a ClinRO measure was not developed because only a patient can report on the sensation of eye irritation. Dermatologist and patient feedback on the PRO measure to assess eye irritation is summarized in ESM 6. The final version of the PRO measure (Table 2) was confirmed by patients following round 2 interviews; all 15 patients could understand and interpret the item consistently with its intended meaning, and a few commented that the example descriptors provided in parentheses (itching, stinging, burning, dry) had helped them interpret the concept of eye irritation.

Table 2

Final version of the PRO Measure for Eye Irritation™

Please rate how irritated (e.g. itching, stinging, burning, or dry) either of your eyes have been in the past 7 days. Please select one answer. ❏0 My eyes have not been irritated ❏1 My eyes have been a little irritated ❏2 My eyes have been moderately irritated ❏3 My eyes have been severely irritated

The PRO Measure for Eye Irritation™ is subject to copyright owned by Eli Lilly and Company. Permission to use is granted under Creative Commons attribution-No derivatives 4.0 International License. Contact Eli Lilly and Company in the event of any proposed modification

PRO patient-reported outcome

Development of a ClinRO and PRO Measure to Assess Nail Involvement

Dermatologist and patient feedback on the ClinRO and PRO measures to assess nail involvement is summarized in ESM 7. The final version of the PRO measure (Table 3) was confirmed by patients following round 2 interviews; all patients could understand and interpret the item consistently with its intended meaning, and the response options were generally understood and used appropriately. Patients consulted (n = 10/11) also confirmed the relevance of the PRO and ClinRO measure, reporting that it was important for clinicians and patients to rate nail appearance.

Table 3

Final versions of the ClinRO and PRO Measures for Nail Appearance

ClinRO Measure for Nail Appearance™ Examine the fingernails and toenails and select the best response below. ❏0 Nails are not at all damaged (e.g. pitted, rough, brittle, split) ❏1 At least one nail is a little damaged (e.g. pitted, rough, brittle, split) ❏2 At least one nail is moderately damaged (e.g. pitted, rough, brittle, split) ❏3 At least one nail is very damaged (e.g. pitted, rough, brittle, split) or subject has lost at least one nail

PRO Measure for Nail Appearance™ Examine your fingernails and toenails. Please rate your fingernails and toenails, as they look today. Please select one answer. ❏0 Nails are not at all damaged (e.g. pitted, rough, brittle, split) ❏1 At least one nail is a little damaged (e.g. pitted, rough, brittle, split) ❏2 At least one nail is moderately damaged (e.g. pitted, rough, brittle, split) ❏3 At least one nail is very damaged (e.g. pitted, rough, brittle, split) or you have lost at least one nail

The ClinRO Measure for Nail Appearance™ and PRO Measure for Nail Appearance™ and accompanying photoguides (ESM 10) are subject to copyright owned by Eli Lilly and Company. Permission to use is granted under Creative Commons attribution-No derivatives 4.0 International License. Contact Eli Lilly and Company in the event of any proposed modification

ClinRO clinician-reported outcome, PRO patient-reported outcome

Development of Accompanying Photoguides

Five of the expert dermatologists who participated in the COA development interviews (AA experience range 8–30 years [mean 17.7]) participated in interviews to develop and confirm the photoguides. Ten patients with AA with current eyebrow hair loss (n = 9 [90%]), eyelash hair loss (n = 8 [80%]) and/or nail damage (n = 3 [30%]), aged 21–54 years (mean 37.0) were also interviewed (see ESM 8).

Drafting, Review and Finalization of Accompanying Photoguides

The photoguides were developed to help raters derive their score, standardize ratings across raters and provide clear visual instructions for assessment of the observable signs of AA (eyebrow hair loss, eyelash hair loss and nail damage). A photoguide to accompany the PRO Measure for Eye Irritation™ symptom measure was not developed.

High-resolution photographs of patients’ eyebrows, eyelashes and nails were obtained, with patient consent, from a specialist dermatology clinic. The eyebrow, eyelash and/or nail damage of these patients were rated by their dermatologists using the relevant ClinRO measure. Selected photographs were then edited using Adobe® Photoshop® software to depict greater or lesser severity (i.e., more or less hair loss/nail damage) corresponding to each measures’ response options. The edited images were reviewed by dermatologists and iteratively modified throughout the dermatologist interview process to incorporate their feedback. Images were then approved by the lead AA expert and included in the photoguides, resulting in a final set of four severity images depicting three different patients/presentations for eyebrow loss and two different patients/presentations for eyelash hair loss and nail damage (photographs for eyelash hair loss and nail damage required enlargement for raters to be able to clearly view the detail presented). See ESM 9 for an illustrative figure of the development methodology and ESM 10 for the final photoguides.

The final images received a 98% approval rating from the dermatologists and patients who reviewed them, confirming that the images appropriately depicted the severity levels they represented and that the photoguides provided a clear visual instruction for assessment. Nearly all dermatologists and patients felt that the photoguides helped them to derive ratings and would help to standardize ratings.

Discussion

This qualitative study confirmed that dermatologists and patients with AA consider eyebrow and eyelash hair loss as bothersome, necessitating treatment evaluation. Patients and dermatologists in this study also highlighted eye irritation as a physical impact of eyebrow/eyelash loss, endorsing eye irritation as an additional relevant concept for assessment. Both clinicians and patients also endorsed the assessment of nail appearance.

Investigator global assessment and PRO measures with accompanying photoguides have been used to assess eyebrows and eyelashes in clinical trials for other conditions [21, 22]; however, to our knowledge, these measures have not been designed to assess patients with AA. Thus, there was a need to develop appropriate measures for use in AA clinical trials.

The ClinRO and PRO measures were developed in line with FDA PRO Guidance for Industry [13]. Multiple rounds of concept elicitation and cognitive debriefing using industry standard methodology [14] were conducted with dermatologists and patients to craft measures that were informed by the intended population of use. This extensive testing resulted in measures of eyebrow hair loss, eyelash hair loss, nail appearance and eye irritation that are content valid (i.e., interpretable, relevant and usable) for their context of use and population of interest (i.e., clinical trials including patients with severe AA). However, the validation of new ClinRO and PRO measures requires a compilation of evidence [13]. Next, quantitative research will be conducted to psychometrically test and establish other aspects of validity and reliability and the ability of the measures to detect change.

Eyebrow hair loss, eyelash hair loss and nail appearance manifest differently from patient to patient, so describing different levels of severity within each ClinRO and PRO measure was a considerable challenge. To meet this challenge, and to support dermatologists and patients in providing their ratings, accompanying eyebrow, eyelash and nail photoguides were iteratively developed, incorporating feedback from expert dermatologists, and confirmed with patients. The interviewed dermatologists and patients deemed these photoguides useful in deriving their score, helping standardize ratings across raters and providing clear visual instruction for assessment.

Some limitations to the present study exist. Participants were recruited in North America and, thus, the findings may not be generalizable to other cultures and countries without further confirmatory work. Photographs for the photoguides were limited to those available with patient permissions, and the photos are of Caucasian models only.

Conclusion

The ClinRO Measure for Eyebrow Hair Loss™, ClinRO Measure of Eyelash Hair Loss™, ClinRO Measure for Nail Appearance™, PRO Measure for Eyebrows™, PRO Measure for Eyelashes™, PRO Measure for Nail Appearance™ and PRO Measure for Eye Irritation™ and accompanying photoguides were developed to provide content-valid, clinically meaningful measures that reflect patients’ and dermatologists’ perspectives of AA. In conjunction with the Alopecia Areata Investigator Global Assessment™ (AA-IGA™) [11] and Scalp Hair Assessment PRO™ [12] for scalp hair loss, these measures provide a comprehensive suite of assessments for key AA signs/symptoms.

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 4123 kb)

Iterative rounds of qualitative interviews with dermatologists and patients in North America confirmed that eyebrow hair loss, eyelash hair loss, nail damage and eye irritation are bothersome signs/symptoms of alopecia areata (AA) that warrant assessment in clinical trials.

The content validity of newly developed clinician-reported outcome (ClinRO) and patient-reported outcome (PRO) measures of these signs/symptoms, along with corresponding photoguides, was confirmed during these interviews; quantitative studies to confirm their psychometric properties are ongoing.

The ClinRO and PRO measures may be used in similar contexts and populations to provide clinically meaningful assessments that reflect patients’ and dermatologists’ perspectives of AA.