Evaluation of the psychometric properties of the Early Morning Symptoms of COPD Instrument (EMSCI).

Background: Early morning respiratory symptoms impact quality of life and are often the most troublesome for patients with COPD. Reduction in symptoms and their impact are important treatment outcomes for COPD. The Early Morning Symptoms of COPD Instrument (EMSCI) is a daily diary designed to collect patients' report of the occurrence, severity, and impact of morning COPD symptoms.
Methods: To assess the psychometric properties of the EMSCI, a split-half sample of data from a COPD clinical trial where participants completed the EMSCI daily was used for conducting descriptive statistics, factor analyses, and Rasch model analyses to examine item performance and inform scoring. Once the final scoring algorithm was determined, data from the second split-half sample were used to examine the properties of the EMSCI. Test-retest reliability was assessed using intraclass correlation coefficient (ICC). Correlations with other study assessments were used to evaluate convergent and known-groups validity.
Results: Data from 1,663 patients with COPD aged 40-93 years were analyzed. Factor analysis and Rasch analysis confirmed a one-factor structure for the 6 individual symptom items. Item analyses supported the generation of 4 scores. All scores demonstrated good test-retest reliability: 6-item symptom severity (ICC, 0.84); overall morning symptom severity (ICC, 0.84); activity limitation (ICC, 0.85); and rescue medication (ICC, 0.62). Significant correlations between EMSCI scores, St George's Respiratory Questionnaire scores, and EXAcerbations of Chronic pulmonary disease Tool (EXACT)-Respiratory Symptoms scores supported the tool's convergent validity. Significant differences (p<0.0001) in all EMSCI domain scores were found between known-groups based on median split St George's Respiratory Questionnaire and EXACT-Respiratory Symptoms scores.
Conclusion: The EMSCI consists of 4 scores: 6-item symptom severity, overall symptom severity, activity limitation, and rescue medication. The EMSCI is a reliable and valid instrument for evaluating patients' experience of early morning COPD symptoms.

Introduction

COPD is a chronic lung condition characterized by airflow obstruction.1 Common symptoms include shortness of breath, cough, wheeze, and phlegm production, but symptoms can vary across individuals. Patients with COPD often report the morning as a particularly difficult time2 when symptoms can limit their ability to perform morning activities3,4 and negatively impact their quality of life (QOL).5 A pan-European, cross-sectional, observational study conducted in 17 countries found early morning symptoms (EMS)3 were the worst symptoms experienced in patients with COPD.3 EMS can also potentially predict worsening of the disease, increasing the risk for exacerbations and use of rescue inhalers.6 Reducing EMS and their impact on patients’ QOL is a key goal of treatment, and there is a need to assess EMS to evaluate the necessity and benefits of treatments.

Disease symptoms and their impact on patients are best measured using patient-reported outcome (PRO) measures, which are developed using methods that ensure the measure reflects the patient’s perspective; ensuring content validity.7

A review of the literature at the time of the development of the Early Morning Symptoms of COPD Instrument (EMSCI; 2011) revealed only 2 measures reporting EMS: the Capacity of Daily Living during the Morning questionnaire (CDLM)8 and Global Chest Symptoms Questionnaire (GCSQ).8 However, the symptoms measured in the GCSQ include only shortness of breath and tightness in the chest, and the development of the instruments involved only a small sample of individuals. Furthermore, there was no evidence of the saturation of concepts to support the content validity of the tools, that is, to ensure that the instrument evaluated all the concepts that were relevant to patients’ experience of COPD symptoms in the early morning.

A more recent review revealed that other measures of EMS are being developed in parallel to the development of the EMSCI, for example the COPD morning symptom diary (COPD-MSD).9 The 19-item measure is a daily diary where patients are asked about “this morning”; the diary includes items on frequency and severity of COPD symptoms in the morning, wheezing the previous night, level of shortness of breath related to specific activities in the morning, and feeling tired and rested. The 10-item Manchester Early Morning Symptoms Index (MEMSI)10 was developed based on focus groups and cognitive debriefing and has been validated among Global initiative on Obstructive Lung Disease (GOLD) grade II patients with COPD. The measure uses a 1-week recall period to evaluate EMS and impaired activity in patients with COPD.

The EMSCI11 was designed as a daily diary to collect data on the occurrence and severity of EMS and the impact of the symptoms on patients with COPD. The EMSCI was developed according to the US Food and Drug Administration Guidance for Industry on PRO measures to support labeling claims12 and developed based on a literature review of existing COPD instruments, interviews with COPD clinical experts, and qualitative research involving patients with COPD.11 The qualitative research was conducted to the point of saturation to ensure the content validity of the instrument.11

Patient involvement in the development of the instrument, a clear rationale for item generation and reduction, and strong measurement properties (reliability and validity) are necessary to ensure that an instrument captures data that are both meaningful and relevant to the target sample.13–15

Objective

The objective of this study was to conduct item analyses to inform the deletion or retention of items, determine the scoring algorithm, and examine the cross-sectional psychometric properties (reliability and validity) of the EMSCI.

Methods

Study design

The data used for psychometric evaluation in this study were from a prospective Phase III, multicenter, multinational, randomized, parallel-group, active and placebo-controlled clinical trial of a bronchodilator (NCT01437397)16 with a 24-week treatment period. The original clinical trial protocol was approved by the institutional review board at each study center (Supplementary material), and all participants provided written informed consent.

Study sample

The trial randomized 1,692 participants aged ≥40 years with moderate-to-severe stable COPD (defined using the criteria of the GOLD [2011]: post-albuterol/salbutamol forced expiratory volume in 1 second [FEV1] ≥30 to <80% predicted and FEV1/forced vital capacity [FVC] of <0.7). Patients had a smoking history of 10 pack-years or more.

Measures

During the trial, patients completed a daily electronic diary that recorded the EMSCI questions and the Evaluating Respiratory Symptoms (E-RS).17 Patients were also asked to complete the St George’s Respiratory Questionnaire (SGRQ)18 and the Patient Global Impression of Change (PGIC),19 and spirometry tests at study visits. Data from the E-RS and SGRQ were used to evaluate convergent validity and known-groups’ validity.

EMSCI

The EMSCI was designed to measure the occurrence and severity of EMS in patients with COPD, and the symptoms’ impact on activity limitation and rescue medication use.11 Patients completed the EMSCI each morning, between 7 and 11 am, using an electronic daily diary. The EMSCI asks patients to indicate symptoms between the time they got out of bed to start their day and the time of diary completion. The EMSCI asks about the presence of specific COPD symptoms (cough, wheezing, shortness of breath, tightness in chest, chest congestion, and difficulty bringing up phlegm) and their severity on a scale from 1 to 4 (1= mild, 4= very severe). A score of 0 is assigned if a patient had no symptoms. Single items assess overall COPD symptom severity in the early morning (0= no symptoms, 4= very severe), early morning activity limitation due to COPD symptoms (0= not at all, 4= a very great deal), and the number of puffs of rescue medication required in the morning.

SGRQ

The SGRQ18 is a validated measure of impaired health in diseases of chronic airflow limitation, and has been widely used in clinical trials in COPD. It contains 50 items divided into 3 subscales: “symptoms”, “activity”, and “impacts”. A score is calculated for each section and a “total” score is also calculated. In each case, the lowest possible value is zero and the highest is 100. Higher values correspond to greater impairment of QOL.

E-RS

The E-RS was designed as a standardized respiratory symptom diary and utilizes 11 respiratory symptom items from the 14-item Exacerbations of Chronic Pulmonary Disease Tool.17 The respiratory symptoms total score represents overall respiratory symptom severity. Total scores range from 0 to 40, with higher scores indicating more severe respiratory symptoms. The E-RS was completed by patients throughout the course of the study in the e-Diary each night directly before going to sleep.

PGIC

The PGIC19 contains an item that assesses the patient’s perspective on how their COPD symptoms have changed since the start of the study. The PGIC has a 7-point response grade ranging from “very much worse” to “very much improved” (scores 1–3= improved, 4= no change, and 5–7= worse). The PGIC provides a subjective summary index of degree of improvement (or worsening). The PGIC was used to identify a stable sample of patients for evaluating the EMSCI’s test–retest validity.

Spirometry

Spirometry (FEV1 and FVC) was performed by clinical study site personnel during study visits. FEV1 measures how much air a person can exhale during a forced breath, and FVC is the total amount of air exhaled during the FEV1 test. The largest value of 3 technically satisfactory forced exhalation efforts was used in the current study.

Statistical analyses

Statistical analyses were conducted in 2 stages and in accordance with best practices for instrument development and psychometric evaluation.14 The analytic sample consisted of patients in the intention-to-treat population (n=1,669) with a baseline EMSCI measurement. Individuals were excluded if data were missing on PRO and clinical data at baseline, leaving a sample of 1,663 patients. The data for all treatment groups were combined and randomly split into 2 split-half samples for the analyses. Phase I consisted of item analyses and the factor analysis to inform item reduction and for defining scoring. Item analyses, factor analysis, and scoring definition utilized the first split-half sample (n=831). Using the final version of the EMSCI and scoring algorithm, Phase II was conducted on the second split-half sample (n=832) to determine reliability and validity. Statistical analyses were performed using SAS statistical software version 9.1 (Cary, NC, USA) following a pre-specified statistical analysis plan. Item analysis and psychometric validation were conducted using the average of the 7 days (Day –6 to Day 1) leading up to and including the day of randomization (referred to in this manuscript as baseline week), and a single day score; Day –7 before randomization (referred to as baseline day). For analysis of a 7-day average, a minimum of 4 (of 7) diary entries during the 7-day period was required, following the approach of Junghard et al.20 Tests of validity (construct and known-groups [discriminant] validity) were performed on samples at baseline week and Week 4/Visit 4 (the average of the week before Visit 4 or Day 29±3).

Item structure and scoring algorithm development

Item analysis was conducted to determine if any items were poorly performing and to determine the scoring algorithm for the measure. Distributional characteristics of each of the 9 items on the EMSCI were evaluated by exploring item-level descriptive statistics and item-to-item correlations.

Criteria for identifying poorly fitting items included any items that showed a floor (minimum response >30% of patients) or ceiling (maximum response >30% of patients) effect; item–item correlations >0.80; factor loadings <0.3 or misfit to the Rasch model; a high negative (<−3.0) residual number, which suggests an over-fitting item (meaning that the information provided by this item does not add any new information to the measurement); or a high positive (>3.0) residual number, which suggests that the item is under-fitting (indicating that the item has a poor fit to the model and the response categories are under-discriminating–or not discriminating–differences in severity). To evaluate the model fit, the following thresholds were used: comparative fit index ≥0.90, root mean square error of approximately ≤0.08, and root mean square residual number ≤0.05. The results from the confirmatory factor analysis (CFA) and Rasch model analysis were used to inform item deletion or retention of items through an iterative process and to determine the scoring algorithm for the measure.

After the initial item evaluation, factor analysis, and Rasch model analysis, an item analyses meeting was conducted with the PRO tool developers, statisticians, and a COPD clinical expert (co-authors of this paper, including BM, AH, and EZ). The intent of the meeting was to consider the qualitative information obtained during the concept elicitation and cognitive interviews,11 and incorporate the quantitative results to develop a scoring scheme that is consistent with the objectives of the measure. Clinical input and item performance results were used to identify well-performing items as well as items that were potentially problematic. All decisions made on the final items were with the qualitative results of patient input in mind and the consideration of clinically relevant symptoms as well as the quantitative results on item performance.

Internal consistency (reliability), test–retest reliability, construct validity, and known-groups (discriminant) validity were all examined as part of the psychometric assessment of the EMSCI.

Internal consistency was assessed using Cronbach’s α, with a target value >0.70 at baseline week. Test–retest reliability was assessed using intraclass correlation coefficient (ICC) within a group of stable patients; defined by PGIC “no change” (n=222). Patients’ scores were compared at baseline week and Visit 4 (average of week ending on study Visit 4). An ICC ≥0.7 indicates good test–retest reliability, between 0.4 and 0.7 indicates moderate reliability, and <0.4 indicates low test–retest reliability.21,22

Construct validity (ie, the extent to which a scale measures what it is hypothesized to measure) was evaluated at baseline by examining the relationship between each domain score with SGRQ total and subscale scores, E-RS total score, and FEV1. Construct validity was considered supported when the EMSCI domain items/scores were substantially correlated (>0.40; using Spearman rank order correlation coefficients) with items or scales measuring similar concepts.23

Known-groups validity was evaluated using analysis of variance models to explore the relationship between the EMSCI scores and measures of disease status: severity of airflow obstruction from GOLD stage I (mild) to stage IV (very severe), SGRQ total score (≤ sample median versus > sample median), SGRQ symptoms score (≤ sample median versus > sample median), and E-RS total score (≤ sample median versus > sample median). EMSCI scores were compared between groups differing on severity of their airflow obstruction (both were collected during screening), SGRQ scores (at baseline), and E-RS total score (at baseline).

Results

The demographic and clinical characteristics of the full sample (n=1,663) are presented in Table 1. The mean age was 63.9±8.9 years (range: 40–93 years old) and the majority of participants (n=885; 53.2%) were male. As expected, given the inclusion criteria, most participants were categorized as GOLD stage II (n=946; 56.9%) or III (n=696; 41.9%), with only 4 participants classified as GOLD stage I (0.24%) and 12 as GOLD stage IV (0.72%). The most common COPD-related concomitant medication was a short-acting β2-adrenergic agonist (n=940; 56.5%), followed by inhaled corticosteroids (n=549; 33.0%), and systemic corticosteroids (n=243; 14.6%). Long-acting bronchodilators were not allowed for the duration of the clinical trial. Participants had an average total score of 9.6 on the E-RS and SGRQ total and symptom scores of 46.5 and 65.1, respectively, suggesting moderate-to-severe COPD.24

Table 1

Sociodemographic and clinical characteristics

Characteristics	Overall sample Day 1,a N=1,663
Demographic
Age, mean (SD)	63.9 (8.9)
Gender, female, n (%)	778 (46.8)
Race/ethnicity, %
White	93.2
Black or African American	5.7
Other	1.1
FEV1 mean (SD)	1.4 (0.5)
FEV1 % predicted mean (SD)	49 (14.0)
SGRQb (n=1,648)
Total score mean (SD)	46.5 (18.5)
Symptoms domain score mean (SD)	65.1 (21.4)
Activity domain score mean (SD)	60.6 (22.3)
Impact domain score mean (SD)	32.7 (19.4)
E-RS (n=1,658) totalc mean (SD)	9.6 (6.3)
GOLD stage, n (%)
I	4 (0.2)
II	946 (56.9)
III	696 (41.9)
IV	12 (0.7)
BDI mean (SD)	6.4 (2.2)

Notes:

For patients who were missing the EMSCI diary entry on Day 1 (day of randomization), data from the closest day before Day 1 were used.

SGRQ scores range from 0 to 100; higher score = more severe health status.

The E-RS total score is an aggregate of 3 domains identified on the E-RS. Scores range from 0 to 40; higher score = more severe respiratory symptoms.

Abbreviations: BDI, Baseline Dyspnea Index; EMSCI, Early Morning Symptoms of COPD Instrument; E-RS, Evaluating Respiratory Symptoms; FEV1, forced expiratory volume in 1 second; GOLD, Global initiative for chronic Obstructive Lung Disease; SGRQ, St George’s Respiratory Questionnaire.

Patients’ experiences with COPD symptoms in the early morning

Within the full sample, at baseline day, more than 50% of patients experienced coughing and shortness of breath, and between 23% and 42% of patients experienced the other symptoms of COPD (Figure 1). Approximately 29% of the full sample experienced EMS (“Overall, how severe were your COPD symptoms this morning?”) that were at least moderate (score ≥2) in severity on baseline day.

Figure 1

Percentage of patients experiencing EMS of COPD evaluated using the EMSCI on baseline day.a

Note: aBaseline: Day –7 prior to randomization.

Abbreviations: EMS, early morning symptoms; EMSCI, Early Morning Symptoms of COPD Instrument.

Item analysis for item reduction

The full range of response options (0–4) was observed for all items on the EMSCI. The percentage showing ceiling effects was low for all items, ranging from 0.1% to 1.1%. Floor effects were observed for all the 6 symptom items (>30% not experiencing the symptom in the time period) on the baseline day, ranging from 42.0% to 76.9%. The single overall symptom severity item did not show floor effects (17.5%). These findings suggest that patients did not experience all symptoms on any given day; however, floor effects decreased for each of the 6 symptom items when using the average at baseline week (Table 2).

Table 2

Summary of item analyses for decision making

N=831	Floor effect (baseline week)	Highest inter-item correlation (baseline week)	CFA loading (baseline week)	IRT fit	Decision to reject or accept
Cough	21.0%	0.660	0.736	Yes	Accept
Wheezing	36.8%a	0.612	0.745	Yes	Accept
Shortness of breath	22.2%	0.803a	0.757	Yes	Accept
Tightness in chest	48.0%a	0.582	0.762	Yes	Accept
Chest congestion	48.3%a	0.598	0.843	No	Accept
Difficulty bringing up phlegm	59.6%a	0.577	0.669	Yes	Accept
Overall symptom severity	6.8%	0.803a	NA	No	Accept
Activity limitation	34.8%a	0.697	NA	NA	Accept
Rescue medication	NA	0.445	NA	NA	Accept

Note:

Indicates they are above the threshold criteria that were set a priori to flag items for potential problems, that is, floor effect minimum response >30% and ceiling effect maximum response >30%.

Abbreviations: CFA, confirmatory factor analysis; IRT, item response theory; NA, not applicable.

Item–item correlations for the 6 symptom items and the overall symptom severity item were between 0.28 and 0.80 at baseline week and statistically significant (p<0.0001). There were high correlations between the single item measuring overall symptom severity and each of the 6 individual symptom items at baseline week (>0.51, p-values all <0.0001). In particular, the overall symptom severity item was very strongly correlated with severity of shortness of breath (0.80). The predetermined cutoff for identifying potential item redundancy was a correlation of 0.80, a threshold reached at baseline week. Excluding the correlation between these 2 items, the correlations between EMSCI symptom items ranged from 0.28 to 0.70 (baseline week). However, the high item-to-item correlation suggests that shortness of breath may be an important cardinal symptom of COPD. The items measure different concepts; one assesses a specific symptom (shortness of breath), the other assesses the patients’ overall experience of COPD symptoms in the morning, and may also include symptoms other than the 6 key COPD symptoms specifically noted in the EMSCI. Therefore a decision was made to retain both items. In addition, clinical input suggested that all individual symptom items were clinically meaningful and should be retained for a full understanding of the patient experience with COPD EMS. A summary of the item analyses for decision making can be found in Table 2.

The Rasch model analysis conducted at baseline day included both the overall symptom severity item and the 6 specific symptom items. In this model, the former did not fit the model (chi-square probability p<0.001; fit residual: −5.29). A second Rasch model analysis was then conducted using just the 6 symptom items.

Most items were identified as having correctly ordered response categories, with the exception of the misfitting item on chest congestion (fit =−3.09). All other items fell within the acceptable −3.0 fit residual range. The overall model-fit chi-square was 94.81 (p<0.001). A decision was made to retain the item on chest congestion as patient interviews and insight from clinical experts suggested that it is an important symptom.

The threshold parameters for the items on tightness in the chest and bringing up phlegm suggested response option misordering. This was likely due to a small percentage of subjects who chose the response option “mild”, relative to the large percentage who indicated they did not experience the symptom at all (15.3% versus 69.5% for tightness, and 6.5% versus 76.9% for difficulty bringing up phlegm). Given that the location of the “mild” (scored as 1) response option was in between “no symptoms” (scored as 0) and “moderate” (scored as 2) as intended, and because it assesses a clinically relevant symptom (as determined by qualitative interviews and clinical confirmation during the meeting), the decision was made to retain the item.

Determining scoring of the EMSCI

The standardized coefficients of the 6 symptom items on the EMSCI ranged from 0.65 to 0.791, with good model fit. CFA suggested a 1-factor structure for the 6 symptom severity items. It was decided that the 6 symptom severity items should be scored as one domain score, separate from the single item measuring overall symptom severity, and the activity limitation and rescue medication items would each be scored separately. The revised conceptual (measurement) framework of the EMSCI reflecting these changes is shown in Figure 2.

Figure 2

Final EMSCI conceptual framework.

Abbreviation: EMSCI, Early Morning Symptoms of COPD Instrument.

Psychometric properties

The 4 EMSCI scores – 6-item symptom severity score, overall symptom severity score, activity limitation score, and rescue medication score – were tested for their psychometric properties: reliability and validity.

Reliability

Internal consistency was high for the 6-item symptom severity score (Cronbach’s α=0.82), indicating good reliability.

Among stable patients (those classified as “no change” in symptoms on the PGIC from Weeks 1 to 4; n=222), good test–retest reliability (ICC >0.7) was demonstrated for the 6-item symptom severity score (ICC 0.84), overall symptom severity score (ICC 0.84), and activity limitation score (ICC 0.85). The rescue medication score (ICC 0.62) was moderately reliable.

Validity

Convergent validity results are shown in Table 3. The 4 EMSCI scores were correlated with scores on other PRO instruments that measure similar concepts (SGRQ and E-RS). Moderate to strong correlations were observed between both the SGRQ and E-RS, and the 6-item symptom severity (all >0.54), overall symptom severity (all >0.52), and activity limitation (all >0.52) scores. Smaller, but significant correlations were observed between the rescue medication score and both the SGRQ and E-RS scores (0.32–0.35). Correlations with FEV1 were low and mostly not significant; this was expected as COPD symptoms have been shown to have a poor relationship with lung function parameters.

Table 3

Correlationa of EMSCI domain scores with SGR Q, E-RS total, and FEV1b at baseline week

Outcome measure	6-item symptom severityd	Overall symptom severityc	Activity limitation	Rescue medication
SGRQ total score	0.59***	0.56***	0.64***	0.33***
SGRQ symptoms score	0.67***	0.58***	0.52***	0.33***
SGRQ impact score	0.54***	0.52***	0.60***	0.32***
E-RS total score	0.83***	0.80***	0.73***	0.35***
FEV1 (trough)b	−0.04	−0.10	−0.13**	−0.13

Notes:

Spearman rank order correlation coefficients:

p<0.0001;

p<0.001.

Morning pre-dose value.

Single item measuring overall early morning COPD symptom severity.

Average score of 6 symptoms (cough, wheezing, shortness of breath, tightness in your chest, chest congestion, and difficulty bringing up phlegm).

Abbreviations: EMS, early morning symptoms; EMSCI, Early Morning Symptoms of COPD Instrument; E-RS, Evaluating Respiratory Symptoms; FEV1, forced expiratory volume in 1 second; SGRQ, St George’s Respiratory Questionnaire.

Known-groups (discriminant) validity

A pairwise comparison of EMSCI domain scores was made between GOLD stages II and III as the sample contained primarily patients with moderate and severe symptoms. Significant differences were found between these 2 groups for all EMSCI scores (Figure 3).

Figure 3

EMSCI domain scores by GOLD severity stages II and III at baseline week.

Note: Significance level for comparison ***p<0.0001; *p<0.05.

Abbreviations: EMSCI, Early Morning Symptoms of COPD Instrument; GOLD, Global initiative for chronic Obstructive Lung Disease.

A comparison was made between the EMSCI domain scores and SGRQ and E-RS scores. Significant differences in all EMSCI domain scores were found between known-groups based on median-splits of SGRQ total and symptoms scores (Figure 4) and E-RS total score (Figure 5).

Figure 4

EMSCI domain scores by SGRQ total score at baseline week.

Note: Significance level for comparison ***p<0.0001.

Abbreviations: EMSCI, Early Morning Symptoms of COPD Instrument; SGRQ, St George’s Respiratory Questionnaire.

Figure 5

EMSCI domain scores by E-RS scores at baseline week.

Note: Significance level for comparison ***p<0.0001.

Abbreviations: EMSCI, Early Morning Symptoms of COPD Instrument; E-RS, Evaluating Respiratory Symptoms.

Discussion

EMS of COPD can significantly impact patient functioning.3,4 The EMSCI was developed based on patients’ experiences of COPD and was designed as a daily diary to measure the variability of occurrence and severity of EMS and their impact on morning activities and rescue medication use.

CFA and Rasch model analyses confirmed a 1-factor structure for the 6 symptom severity items, and following psychometric analyses and discussions with clinical experts, it was suggested that these items be scored separately to the overall symptom severity item. The final EMSCI measure is composed of 9 items and these can be used to generate 4 scores: 1) the 6-item symptom severity score; 2) the overall symptom severity score; 3) the rescue medication score; and 4) the activity limitation score.

This study demonstrates that the EMSCI is a reliable and valid instrument to evaluate EMS of COPD and their impact on patients. Test–retest reliability was confirmed for all 4 scores. The 4 EMSCI scores were correlated with scores on other PRO instruments measuring similar concepts (SGRQ and E-RS) demonstrating convergent validity. In addition, the measure is able to distinguish between groups known to differ clinically (GOLD stages II and III, and between median-splits of SGRQ, and E-RS total scores).

The validity assessments in the current study suggest that either the 6-item symptom severity score or the overall symptom severity score could be used as an assessment of COPD EMS presence or severity. The 6-item score is based on more items than the overall score; it is therefore likely to be more precise because it combines scores on a range of symptoms, whereas the overall score does not take into account that some symptoms might be more bothersome than others. However, the overall symptom severity score may provide a simple single item measure for evaluating symptom-free days, which may be an important endpoint for trials evaluating mild COPD. The American Thoracic Society has provided a statement that an overall symptom measure is more appropriate than individual symptom items for evaluating symptom-free days. Whenever possible, we recommend using both scores, as together they would provide the most comprehensive picture of EMS in patients with COPD. However, there may be cases where participant burden is a concern and a single item measure could be used to reduce this burden. The overall severity item has proven to be robust; however, it was evaluated in the context of its application alongside the 6 individual symptom severity questions and therefore responses may have been influenced by the responses given to the individual items. Further testing of this single item on its own is required to confirm these findings.

While there are other instruments that have been developed to examine EMS in patients with COPD, they have limitations that have been addressed with the EMSCI. For example, both the CDLM and the GCSQ were developed using a small sample of individuals.5 The EMSCI was developed with patient input to the point of saturation, providing evidence of content validity.11 The COPD-MSD and MEMSI also assess EMS of COPD; however, the former has not been validated, and the latter asks patients to recall their symptoms over a 1-week period. The results of this study demonstrate the reliability and validity of the EMSCI. Additionally, the EMSCI is designed to collect data every day and is therefore able to gather information about symptoms and limitations that typically vary on a day-to-day basis in COPD patients. Additionally, a daily recall period decreases the likelihood of recall bias.

As a daily diary, EMSCI can be used to evaluate outcomes that are relevant to COPD patients and reflect the patients’ perspective: average severity as well as variability of COPD patients’ experiences over time, as well as calculate symptom-free time periods. A related instrument to the EMSCI is the Nighttime Symptoms of COPD Instrument.25 It assesses nighttime symptoms of COPD and has been validated.26 The 2 measures were developed at the same time and have been used alongside one another in clinical trials and observational studies (such as NCT01462942, NCT01462929, and NCT03031769). These instruments would be suitable for use in a clinical trial. While the EMSCI could also be used in clinical practice to monitor patient outcomes, this has not been tested.

Study limitations and future research

A limitation of this study is the somewhat homogenous population. The patient population taken from the clinical trial consisted mainly of patients with COPD classified in GOLD stage II or III. Sample sizes of patients in higher (IV) or lower stages (I) were not large enough to be analyzed. Additionally, the study population was mostly White and did not provide a representative sample of patients with COPD. In this study, there was a weak relationship between FEV1 and symptoms. Although this is in line with previous research,27 it is possible that at higher levels of severity, this relationship could differ. However, it was not possible to test this with this sample size. Finally, this study used scores from the SGRQ and E-RS to examine convergent validity of the EMSCI. Other known measures which look at symptoms throughout the day in COPD were not used during the trial. Future research should examine the relationship between EMSCI scores and other known measures of COPD symptoms (for example, the Medical Research Council Dyspnea Scale).28

Conclusion

The EMSCI is a reliable and valid instrument to evaluate early morning COPD symptoms and activity limitation, and shows good internal consistency and test–retest reliability. Its robust psychometric properties suggest that it can be used for the evaluation of COPD patients’ experience of EMS and as an outcome measure for evaluating the benefit of interventions. Further research is being conducted to determine definitions of clinically meaningful change to enable the interpretation of change scores on the EMSCI when evaluating interventions to reduce EMS.

Name and address of institutional review boards/independent ethics committees

Australia

Bellberry Human Research Ethics Committee229 Greenhill Road, Dulwich, SA 5065, Australia

Flinders Clinical Research Ethics CommitteeBedford Park, SA 5042, Australia

Gold Coast Health Service District Human Research Ethics Committee108 Nerang Street, Southport, QLD 4215, Australia

Human Research Ethics Committee – DSt Vincent’s Health Melbourne, PO Box 2900, Fitzroy, VIC 3065, Australia

Hunter New England Human Research Ethics CommitteeLocked Bag 1, New Lambton, NSW 2305, Australia

Royal Adelaide Hospital Research Ethics CommitteeNorth Terrace, Adelaide, SA 5000, Australia

Canada

Concordia Hospital Ethics Committee (local)Concordia Hospital, 1095 Concordia Avenue, Winnipeg, MB, Canada R2K 3S8

IRB Services (Institutional Review Board Services) (central)372 Hollandview Trail, Suite 300 Aurora, ON L4G 0A5, Canada

University of British Columbia, Clinical Research Ethics Board Office (local)Room 210, Research pavilion 828 West 10th Avenue, Vancouver, BC, Canada, V5Z 1L8

University of Saskatchewan Biomedical Ethics Review Board (local)Research Ethics Office, University of Saskatchewan, Box 5000 RPO University, Saskatoon, SK S7N 4J8, Canada

New Zealand

Multi-region Ethics Committee c/- Ministry of HealthPO Box 5013, 1 the Terrace, Wellington

USA

Creighton University IRB (local)2500 California Plaza, Omaha, NE 68178, USA

GLA-VA IRB (local)11301 Wilshire Blvd, Building 114 Room 325 Los Angeles, CA 90073, USA

Human Studies Subcommittee Bay Pines VA Healthcare System (local)Research and Development Service (151), 10000 Bay Pines Blvd, Bay Pines, FL 33744, USA

Institutional Review Board University of Texas Health Science Center at Tyler (local)11937 US Highway 271, Tyler, TX 75708, USA

Kansas City Veterans Affairs Medical Center IRB (local)4801 East Linwood Blvd, ≤151 Kansas City, MO 64128, USA

Mayo Clinic Office for Human Research Protection Institutional Review Board (IRB) (local)201 Building, Room 4–60 200 1st Street, SW Rochester, MN 55905, USA

Mercy St Vincent Medical Center Adult Investigational Review Board (local)2213 Cherry St, Toledo, OH 43608, USA

Quorum Institutional Review Board (central)1601 Fifth Avenue, Suite 1000, Seattle, WA 98101, USA

Sparks Institutional Review Board (local)1001 Towson Avenue, PO Box 2406, Fort Smith, AR 72902-2406, USA

St Luke’s IRB (local)232 S Woods Mill Rd, Chesterfield, MO 63017, USA

UC Davis Medical Center IRB (local)CTSC Building 2921 Stockton Blvd, Suite 1400, Room 1429, Sacramento, CA 95817, USA

University of California San Diego Human Research Protections Program (local)9500 Gilman Drive, Mail Code 0052 La Jolla, CA 92093-0052, USA

University of Illinois College of Medicine of Peoria Institutional Review Board (local)One Illini Drive Box 1649, Peoria, IL 61656-1649, USA

Wake Forest University Health Sciences Office of Research/Institution Review Board (local)Medical Center Blvd, Winston-Salem, NC 27157, USA

Western Institutional Review Board (local)3535 7th Avenue SW, Olympia, WA 98502, USA