Report from the third international consensus meeting to harmonise core outcome measures for atopic eczema/dermatitis clinical trials (HOME).

This report provides a summary of the third meeting of the Harmonising Outcome Measures for Eczema (HOME) initiative held in San Diego, CA, U.S.A., 6-7 April 2013 (HOME III). The meeting addressed the four domains that had previously been agreed should be measured in every eczema clinical trial: clinical signs, patient-reported symptoms, long-term control and quality of life. Formal presentations and nominal group techniques were used at this working meeting, attended by 56 voting participants (31 of whom were dermatologists). Significant progress was made on the domain of clinical signs. Without reference to any named scales, it was agreed that the intensity and extent of erythema, excoriation, oedema/papulation and lichenification should be included in the core outcome measure for the scale to have content validity. The group then discussed a systematic review of all scales measuring the clinical signs of eczema and their measurement properties, followed by a consensus vote on which scale to recommend for inclusion in the core outcome set. Research into the remaining three domains was presented, followed by discussions. The symptoms group and quality of life groups need to systematically identify all available tools and rate the quality of the tools. A definition of long-term control is needed before progress can be made towards recommending a core outcome measure.

This report provides a summary of the third meeting of the Harmonising Outcome Measures for Eczema (HOME) initiative held in San Diego, CA, U.S.A. 6–7 April 2013 (HOME III). The HOME initiative is an international, multidisciplinary group open to anyone interested in atopic eczema (AE) outcomes research. The aim of HOME is to develop a consensus-based set of core outcome measures to be included in all clinical trials of eczema.

This meeting was preceded by two previous meetings: HOME I (Munich, July 2010)1 where the need for international consensus was agreed upon, and HOME II (Amsterdam, June 2011)2 where it was agreed by consensus that there would be a core outcome measure for each of the four domains of clinical signs, patient-reported symptoms, long-term control and quality of life that should be recorded and reported in every clinical trial investigating interventions for eczema.

The aims of the HOME III meeting in San Diego were specifically to:

Discuss and interpret new research since the previous HOME meeting (HOME II) on the four core outcome domains.

Decide which instrument(s) should be used to measure clinical signs of eczema.

Prioritize areas for further research in the four core domains.

A separate consensus statement detailing the outcome of the clinical signs domain has been published elsewhere. The full minutes of this meeting can be found at www.homeforeczema.org.

Methods

All members of HOME were invited to participate. There were a total of 56 attendees from around the world: North America (32), Europe (18), Japan (4) and South America (2). These included a mixture of prior HOME meeting attendees and new members. Details by stakeholder group can be found in Figure1. Most attendees had been involved with designing or recruiting into clinical trials (86%) and approximately half had previously used EASI (Eczema Area and Severity Index) and/or SCORAD (SCORing Atopic Dermatitis).

Figure 1

Breakdown of meeting participants by stakeholder group.

A combination of formal presentations and nominal group techniques were used at the meeting. Consensus was achieved through an iterative and cumulative process over the 2 days. The decision was taken prior to the meeting that members would need to be present in order to vote to ensure that decisions were based on the data presented and the subsequent discussions.

Each session began with presentations of relevant research, followed by whole group and breakout group discussions. Voting was anonymized by using electronic handsets and TurningPoint© software to analyse the results in real time. Voting took place as one panel; all stakeholders were included in every vote. The previously agreed consensus rule2 was applied here; that is, consensus is reached where fewer than 30% of the voters disagree.

There was no registration fee for the meeting, and no sponsorship from any commercial organization was received. With the exception of patient representatives whose costs were covered by HOME members, attendees covered their own travel and accommodation costs.

Patient representatives from France, the U.K. and the U.S.A. attended the meeting. A lay summary was produced and circulated prior to the meeting and patient representatives had time before the meeting to ask any questions about the process and content. The patient representatives were then present at all sessions and participated in the discussion and voting.

Day 1

Session 1: Introduction

Presentation 1·1. Introduction and background

Professor Hywel Williams (U.K.) opened the meeting by presenting the background to HOME and highlighted that core outcomes are an essential part of good clinical research to compare outcomes of studies. He then summarized the work undertaken by HOME members (https://www.ncbi.nlm.nih.gov/pubmed/25186228) to date and the HOME roadmap, which describes the steps needed to progress each HOME workstream. The group was reminded of the need to put aside preferences and allegiances and to work together as a global community for the benefit of patients.

Professor Williams also stressed that core outcomes do not exclude other outcomes being measured in a trial, nor does it imply that the core outcome has to be a primary outcome; they are simply the minimum set that should be measured and reported in all future eczema trials.

Presentation 1·2. The gift of the OMERACT filter

Dr Jasvinder Singh (U.S.A.) from the OMERACT (Outcome Measures in Rheumatology) Group acted as an external advisor to keep the meeting focused and fair. Dr Singh explained the similarities between rheumatoid arthritis research 20 years ago and eczema research now. He stressed that in the interests of achieving consensus the group should accept that the outcome of the consensus may not be perfect but should ask themselves the question ‘is it preferable to the status quo?’ The core set can be updated over time to reflect new findings and the measures may be different for children and adults.

Presentation 1·3. Feasibility in all settings

Professor Kim Thomas (U.K.) highlighted that core outcomes need to be feasible in all settings and trial designs. The core outcome set should (i) include at least one objective measure because not all interventions/trial settings can be blinded; (ii) be relatively quick and easy to perform with minimal training required; (iii) perform well; and (iv) measure things of importance to patients and clinicians. Discussions highlighted that there was variation in how a representative site was determined.

Presentation 1·4. The use of global assessments in eczema research: a systematic review of randomized controlled trials

Dr Eric Simpson (U.S.A.) explained that the Investigator Global Assessment (IGA) is required by regulatory agencies to be performed in clinical trials; he presented data from a systematic review showing that there is huge variation on how IGA is conducted, and called for standardization. In the discussion that followed, the group felt that although not a core domain, the HOME initiative should be involved with helping to standardize and better validate the IGA.

Session 2: Clinical signs domain

Professor Jochen Schmitt (Germany) stated the objectives of this session were to (i) achieve consensus on content validity and (ii) determine which instrument should be included in the core outcome set to assess clinical signs of eczema in trials.

Presentation 2·1. Measuring responsiveness (sensitivity to change) and minimal clinically important difference of the Three-Item Severity scale

Dr Mandy Schram (the Netherlands) explained that the Three-Item Severity (TIS) score3 is quick and easy to administer because it includes only erythema, oedema and excoriation, each measured at one representative site. Therefore, because of the simplicity of the scale, work had been done to look at the responsiveness of TIS based on previously published trials and was presented here.

Presentation 2·2. Measurement properties of outcome measurements for eczema

Professor Jochen Schmitt (Germany) and Stephanie Deckert (Germany) presented the results of a systematic review assessing the measurement properties of 16 eczema sign scales. Each included scale was evaluated using pre-defined criteria including the OMERACT filter of truth, discrimination and feasibility.5 The quality of the methodological studies was assessed using the COSMIN checklist.6 The two scales that had been the most extensively validated, and shown to be the most valid and reliable instruments to assess clinical signs of eczema were SCORAD7 and EASI.8

The main differences between these two scales were then discussed: (i) SCORAD uses a representative site whereas EASI measures the intensity of lesions at four body areas and (ii) SCORAD relies on the assumption that the extent of eczema has a linear relationship with severity, which has been shown not to be the case in all patients.9

Professor Schmitt also described a previous study10 that showed both intensity and extent of disease should be measured. There followed an in-depth discussion on the signs domain that continued into day 2.

Day 2

Session 2: Clinical signs domain (continued)

Day 2 began with putting aside named scales, and a significant amount of time was dedicated to discussing which signs are essential to reflect the construct of the disease and should therefore be included in the core outcome. With reference to the book Measurement in Medicine,11 Professor Schmitt explained that when the conceptual framework was considered, signs were a formative model, because the items of the instrument form the construct (domain) to be measured. Therefore, content validity, defined as the degree to which the domain ‘signs’ is comprehensively represented by the items in the instrument, is crucial. In other words, the items in the instrument all need to be relevant and all relevant items need to be included.

The main points arising from the discussion are summarized in Table1.

Table 1

Summary of main discussion points regarding the signs domain core outcome

The three signs that have previously been shown to be independent predictors of patient-rated disease severity (excoriations, erythema and oedema/papulation) were all considered to be important signs

In addition, lichenification should also be considered for inclusion because it is often more prominent in darker skin types and reflects the chronic relapsing nature of the disease

Other signs were discussed, including crusting/oozing, xerosis/dryness, blanching and flaking, and were considered to be less important

It was generally felt that there should be some measure of area involvement in the scale

Discussions suggested that it would be appropriate for the core outcome ‘clinical signs’ to be an investigator-assessed objective measure to reduce information bias and because the other three domains (symptoms, quality of life and long-term control) are primarily patient-reported

Discussions highlighted that it is important to remember that patient versions of scales are not necessarily the same as investigator versions (e.g. EASI and self-administered SA-EASI15) and should not be used interchangeably. However, only core outcome measures were under discussion at this meeting and investigators are free to also include any other scales in addition to the core outcome measures

In the subsequent voting session, consensus was achieved on the following points (Table S1 for detailed voting results):

The core outcome measure for the clinical signs domain should include erythema, excoriation, oedema/papulation and lichenification as a minimum to achieve content validity.

Both the intensity and extent of each clinical sign in the core set should be measured.

No other signs are essential to be included, so further individual signs were not voted on.

Only once there was consensus on the important criterion of content validity did the discussion move on to which of the named scales identified in the systematic review met the criteria for content validity AND standards for validity in other areas. Professor Schmitt referred back to the systematic review results to confirm that objective SCORAD and EASI both include the four essential clinical signs (erythema, excoriation, oedema/papulation and lichenification) so these two scales would be shortlisted for consideration. Objective SCORAD was considered from this point onwards rather than SCORAD because the purpose here was to identify a scale to measure only the domain of clinical signs. He also noted that the eczema community is very fortunate to have two good and freely available scales to choose from to measure clinical signs.

An explanation of how to use each of these two scales was then given to the group (summarized in Table2).

Table 2

Properties of objective SCORAD and EASI

Objective SCORAD, presented by Professor Jochen Schmitt	EASI, presented by Dr Eric Simpson
A representative site is selected	A score for different areas of the body for each of the essential signs is used (four signs and four body sites)
Measures six signs: the four agreed essential signs plus oozing/crusting and dryness	Measures only the four agreed essential signs
Gives more weight to intensity than extent	Signs and extent are equally weighted

Nominal group techniques were then used to achieve consensus on which scale to recommend for inclusion in the core outcome set. The group split into five smaller groups of mixed nationalities to discuss the two scales. Each small group reached independent conclusions and voted on their preferred scale; all five groups voted in favour of EASI.

The small group discussions were then shared with the group as a whole. EASI was favoured over objective SCORAD by each group because (i) there is no requirement to identify a representative site; (ii) it measures only the four essential signs; (iii) more importance is given to extent; and (iv) distinguishing between body areas may be important in future research.

The aim of HOME is to recommend one scale per domain so the whole group proceeded to vote on whether objective SCORAD or EASI should be recommended (Table S1 for detailed voting results).

The result of the vote was 90% in favour of EASI to be recommended as the core outcome measure for the clinical signs of eczema (7% in favour of objective SCORAD and 2% unsure).

The proposal to include both objective SCORAD and EASI in the core set for a period of time to collect comparative data on the two scales was not supported (36% voted against) (Table S1 for detailed voting results).

Session 3: Quality of Life domain

Presentation 3·1. Quality of Life measures in eczema clinical care and research

Dr Shehla Admani (U.S.A.) presented a focused review of the nine most commonly used Quality of Life (QoL) measurement tools in eczema trials and the properties of each instrument.

Presentation 3·2. Measurement properties of eczema-specific measures of health-related quality of life: systematic review

Dr Christian Apfelbacher (Germany) proposed a protocol to systematically assess measurement properties of eczema-specific measures of health-related QoL and identify outcome measures for eczema-specific QoL. This will follow similar methods to the signs systematic review4 and will be completed for the HOME IV meeting.

Discussion

There followed a discussion on the QoL domain. It is clear that HOME should liaise with the American Association of Dermatology (AAD) regarding systematic reviews of health-related QoL being conducted. The use of PROMIS (Patient Reported Outcomes Measurement Information System) should be considered (www.nihpromis.org) but internationalization of the terminology used would be required. It was discussed that generic scales can perform as well as disease-specific scales with regard to sensitivity to change but are often not popular with patients as they can be lengthy to complete. The fact that different scales may perform better for specific age groups was highlighted and that work on the QoL domain should include significant patient input.

Session 4: Patient-reported symptoms domain

Dr Phyllis Spuls (the Netherlands) opened the symptoms session with a definition: a symptom is any feature that is observed by the patient whereas a sign is observed by other people.

Presentation 4·1. The use of symptom outcome measures in eczema research: a systematic review of randomized controlled trials

Dr Spuls showed that symptoms were reported in three-quarters of trials published between January 2000 and May 2012, with itch and sleep loss being the most commonly reported. SCORAD was the most commonly used named scale to measure symptoms but most trials that used SCORAD (92%) did not report the symptoms separately to the overall composite score.

Presentation 4·2. Eczema signs and symptoms: what is important to patients?

Dr Laura von Kobyletzki (Sweden) presented the results of a web-based survey asking patients and parents/carers in 31 countries to rate the importance of different signs and symptoms. Itch and pain were rated the most important, with hot/inflamed skin, area affected, visible sites, sensitive sites, bleeding, weeping, cracks and sleep difficulties also considered important.

Presentation 4·3. Comparison of the visual analogue scale (VAS) and verbal rating scale in Japanese patients using VAS with 10-point end of ‘worst imaginable itch’

Dr Norito Katoh (Japan) presented a study showing that there was significant correlation between the visual analogue scale (VAS) and the verbal rating scale (VRS) in 949 Japanese patients with itchy skin diseases. Each category of VRS differed significantly from the other categories based on the VAS scoring. While not directly relating to HOME objectives, this is an example of where useful work on eczema outcomes is related to HOME.

In the subsequent discussion, the next steps towards a consensus on patient-reported symptoms outcome measure were agreed. As per the HOME roadmap, this will be to finalize the important constructs in the symptoms domain, and then a systematic review of the validation studies on currently available instruments and assessment of the quality of the scales. The overlap with quality of life should be considered carefully throughout. Patients highlighted that fatigue is different to sleep loss and that different symptoms are important to individuals of different ages.

It was then agreed by voting that itch and sleep loss were NOT sufficient on their own to reflect the construct of the disease; other symptoms should also be considered for inclusion. Table S1 for detailed voting results.

Session 5: Long-term control domain

Professor Kim Thomas began by emphasizing the need to agree on the definition of ‘long-term control’ and determine whether it is truly a separate domain or simply a repeated measurement of other core outcomes. Options for measuring long-term control include flares, escalation of treatment, well-controlled weeks and accessing health resources, but the outcome measure needs to suit trials of all durations and reflect that eczema is a chronic disease. There are parallels with asthma where researchers have recently published a consensus on using ‘escalation of therapy’ as an indicator of control.12

Presentation 5·1. Systematic review of flare definitions used in prospective studies. ‘How should eczema “flares” be defined?’ Implications for the design and conduct of trials

Professor Thomas presented data from a systematic review showing that of the 26 included studies, 21 different flare definitions were used. Flares, as currently defined, may be a good contender for the long-term control domain, particularly for trials with long follow-up or minimal patient contact, as there are difficulties in collecting the data required.

Presentation 5·2. A validation study of a flare definition based on the need to escalate treatment as a result of worsening disease control

Professor Thomas presented results of a validation study to test the previously published definition of flare: ‘escalation of therapy due to worsening of disease’. This intuitively understood concept correlated moderately well with other eczema severity scales (Patient Oriented Eczema Measure, POEM),13 Three-Item Severity (TIS)3 and Six Area, Six Sign Atopic Dermatitis (SASSAD).14 However, it does require resource-intensive completion of daily diaries so may not be suitable for all settings.

Presentation 5·3. Validation study of well-controlled weeks as a way of measuring long-term control

The final presentation from Professor Thomas focused on assessment of well-controlled weeks and how well this performs in capturing long-term control. Although the concept of well-controlled weeks is intuitive and well understood, the data collection and management is complex and resource-intensive so may not be suitable in all trial settings. Nevertheless, similar to flares, well-controlled weeks correlated moderately well with other eczema severity scales (POEM, TIS and SASSAD).

The relative merits of the two broad approaches to measuring long-term control of eczema were discussed and are summarized in Table3.

Table 3

Summary of discussion on different ways of measuring long-term control

Measure of flares/well-controlled weeks	Repeated serial measurements of the other three domains (signs, symptoms and QoL)
Measurement of flares should be a patient-reported outcome (PRO) because a flare is a significant event for patients and so they are well placed to determine when a flare has occurred. Trials often do not have enough clinic visits to enable flares to be measured by the investigator	This could be an attractive and efficient way of measuring long-term control if the other domains are already being captured in the trial
There is variability between patients in how they define a flare so individual flare definitions may be needed in a trial. It should be remembered that patients can have poor control without a flare	Scales would need to be completed frequently enough to capture the fluctuations in the eczema which may impact on the feasibility of this approach
A definition of a flare (including the end of a flare) needs to be agreed on if it is to be used	The EASI scale captures only the eczema at that moment, so to capture signs over the long term, frequent trial visits with the investigator would be required
Parents can sometimes get confused about whether questions on well-controlled weeks refer to the eczema or the child's behaviour	There are many ways these data can be analysed to measure long-term control (e.g. mixed models, fixed-effect models)
Need to determine whether or not the floor effect is important because trials are measuring a benefit	Does not reflect any changes in treatment
Existing trial data where a mixture of daily, weekly and monthly data have been collected should be used to establish the frequency of data collection needed

There is great variation in how people define long-term control including stable disease over a period of months, lack of need for escalation of treatment/rescue therapy, number and duration of flares, duration of the trial (long-term) and repeated measurement of the other three core outcome domains. It was agreed by voting that the next stage is to conduct a systematic review to establish how long-term control has been captured in other trials (and possibly other long-term prospective studies) followed by a systematic review of validation studies. Voting suggested that long-term control is a unique concept that requires its own outcome tool but there was mixed opinion regarding whether long-term control should be completed exclusively by patients and how often long-term control should be measured. It was agreed this domain should apply to trials of more than 3 months' duration. Details of the voting results can be found in Table S1.

Meeting close

Professor Hywel Williams closed the meeting by summing up the progress of each domain and commented that the group had worked well together with everyone contributing. The meeting benefited from input of patients and caregivers, and indicated that the independent and expert input of Jas Singh from OMERACT was invaluable. He concluded by reminding the group that the ultimate aim of the HOME initiative is to help patients and this should always be remembered.

Limitations and dissenting opinions

Limitations of these findings include the possibility that the HOME meeting did not fully represent all stakeholders interested in eczema. The HOME initiative attempts to reduce bias by holding meetings that include a diverse group of researchers, clinicians, regulatory and industry representatives, patients and patient advocates. Meetings are held in different areas of the world with no registration fees and are open to anyone interested in eczema. Researchers who are not able to participate are encouraged to send a representative for voting purposes. Only those participating in the meeting are able to vote as HOME meetings involve presentation of new and emerging evidence, and employ nominal group techniques and electronic voting to achieve consensus. These meeting procedures were developed after consultation with key leaders of OMERACT – a core outcome group with over 20 years of experience in core outcome set development.

In achieving global consensus it is important to listen to the voice of ‘dissenters’ as well as those in agreement with the consensus view. After HOME III, the HOME group received feedback from some colleagues with a strong interest in SCORAD that they did not agree with the recommendation to use EASI as the preferred outcome scale for assessment of clinical signs.

We would like to stress that both the EASI and objective SCORAD were evaluated during the HOME III meeting as being valid and reliable tools for the objective assessment of eczema clinical signs, and would like to reassure readers that use of EASI as the recommended core outcome measure does not preclude the use of other outcome measures in future trials alongside the EASI. It should also be stressed that patient-reported outcomes (including symptoms and quality of life) will be discussed at future HOME meetings with a view to establishing the best instruments for evaluating these domains. Consensus was reached that clinical signs should be assessed using an objective outcome measure, in order to protect against reporting bias in trials where participants are aware of their treatment allocation.

Next steps

Although HOME III focused on the clinician-reported signs, the patient-reported domains of symptoms and quality of life, and the long-term control domain will be the focus of future HOME meetings. Prior to the HOME IV meeting, individual working groups will conduct systematic reviews and further validation studies to inform discussions over the most appropriate tools for measuring these remaining core domains. In relation to the EASI, efforts will now focus on disseminating and sharing the consensus view with researchers, journal editors and regulatory authorities. Standardized training resources will be developed and made available via the HOME website, along with summary data to inform sample-size calculations in different patient populations.

The HOME group will continue to encourage members from nationalities currently not represented, and other clinicians involved in the management of eczema, to join the group and attend meetings. The HOME project has always been, and will remain, open to views from all stakeholders interested in the development of core outcome sets for eczema.