Multidimensional severity assessment in bronchiectasis: an analysis of seven European cohorts.

INTRODUCTION: Bronchiectasis is a multidimensional disease associated with substantial morbidity and mortality. Two disease-specific clinical prediction tools have been developed, the Bronchiectasis Severity Index (BSI) and the FACED score, both of which stratify patients into severity risk categories to predict the probability of mortality.
METHODS: We aimed to compare the predictive utility of BSI and FACED in assessing clinically relevant disease outcomes across seven European cohorts independent of their original validation studies.
RESULTS: The combined cohorts totalled 1612. Pooled analysis showed that both scores had a good discriminatory predictive value for mortality (pooled area under the curve (AUC) 0.76, 95% CI 0.74 to 0.78 for both scores) with the BSI demonstrating a higher sensitivity (65% vs 28%) but lower specificity (70% vs 93%) compared with the FACED score. Calibration analysis suggested that the BSI performed consistently well across all cohorts, while FACED consistently overestimated mortality in 'severe' patients (pooled OR 0.33 (0.23 to 0.48), p<0.0001). The BSI accurately predicted hospitalisations (pooled AUC 0.82, 95% CI 0.78 to 0.84), exacerbations, quality of life (QoL) and respiratory symptoms across all risk categories. FACED had poor discrimination for hospital admissions (pooled AUC 0.65, 95% CI 0.63 to 0.67) with low sensitivity at 16% and did not consistently predict future risk of exacerbations, QoL or respiratory symptoms. No association was observed with FACED and 6 min walk distance (6MWD) or lung function decline.
CONCLUSION: The BSI accurately predicts mortality, hospital admissions, exacerbations, QoL, respiratory symptoms, 6MWD and lung function decline in bronchiectasis, providing a clinically relevant evaluation of disease severity. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

What is the comparative predictive ability and clinical utility of the two recently developed disease-specific severity tools in bronchiectasis—the Bronchiectasis Severity Index (BSI) and the FACED score—in assessing clinically relevant disease outcomes?

This study of 1612 patients across seven European cohorts shows that both tools accurately predict mortality in bronchiectasis, but that the BSI is superior to FACED in predicting multiple clinically useful outcomes including hospital admissions, exacerbations, quality of life, respiratory symptoms, exercise capacity and lung function decline.

This study demonstrates that the BSI provides an accurate assessment of disease severity enabling ‘real-world’ decision-making in terms of identifying high-risk patients who may benefit from aggressive treatment and low-risk patients who could receive non-specialist follow-up or simpler treatment regimes.

Introduction

Bronchiectasis is a heterogeneous, poorly understood, multidimensional disease with recent UK and German data demonstrating increasing prevalence and hospitalisation rates.1–3 Management is challenging as there are no licensed therapies. Given that the majority of available treatments are antibiotic-based, it is important for antimicrobial stewardship to target treatments to patients at severe risk of complications and avoid over-treatment of mild patients at low future risk.4–6

Two composite disease-specific prognostic indices have been developed for bronchiectasis to aid clinical decision-making: the Bronchiectasis Severity Index (BSI) and the FACED score.7 8 Both attribute points according to age, value of FEV1 % predicted, presence of chronic colonisation by Pseudomonas aeruginosa, radiological extension and type of bronchiectasis and degree of dyspnoea. The BSI also considers body mass index, exacerbation frequency, prior hospitalisation for severe exacerbation and chronic colonisation with bacteria other than P. aeruginosa. Both scores classify patients into low, moderate and high risk groups, using different thresholds. The BSI was derived from a large Edinburgh-based study in the UK and was subsequently validated in four international cohorts.7 The FACED score was derived from a retrospective Spanish study and was recently independently evaluated in a single centre UK cohort of 74 patients for long-term prediction of mortality.8 9 FACED was developed specifically to predict mortality, while the BSI was developed to predict mortality, severe exacerbations, frequency of exacerbations and quality of life (QoL).

The data currently available suggest that both scores can predict future mortality in bronchiectasis. Bronchiectasis is not, however, a disease whose impact is primarily measured in terms of mortality. Outcomes other than mortality are likely to be more important in terms of patients priorities, clinical decision-making, healthcare utilisation and socioeconomic costs.2 3 Clinicians face two major challenges in the management of bronchiectasis: (1) identifying patients with a high symptom burden or those at risk of frequent exacerbations or rapid lung function decline who may benefit from aggressive treatment and (2) identifying low-risk patients who could be suitable for non-specialist follow-up or simpler treatment regimes. No therapy has been developed that can reduce mortality in bronchiectasis, but existing and developing therapies are designed to improve QoL, reduce symptoms, reduce exacerbations and slow disease progression.4–6 10–13 Therefore for clinical trial design and subsequent ‘real world’ decision-making, these are the key outcomes to identify.13 The concept of ‘severe’ bronchiectasis should therefore reflect patients with impaired QoL, severe symptoms, frequent exacerbations and progressive disease.

There are limited published data on predictors of outcomes other than mortality in bronchiectasis. This study aimed to evaluate the predictive ability of the two bronchiectasis tools, the BSI and FACED score, in assessing clinically relevant disease outcomes across seven European cohorts.

Methods

Study population

Seven European centres participating in the European Bronchiectasis registry project contributed to the study. Detailed descriptions of these cohorts have previously been published.7 14–19 All cohorts used a standardised protocol and provided data in a standardised case report form. Inclusion criteria were consecutive adult patients with a high resolution CT-confirmed diagnosis and clinical history consistent with bronchiectasis. Patients were excluded if they had active malignancy, cystic fibrosis or a primary diagnosis of pulmonary fibrosis/sarcoidosis with secondary traction bronchiectasis. Patients were assessed and managed according to a standardised protocol based on the British Thoracic Society (BTS) guidelines.13 Ethical approval was obtained from each individual centre's Research Ethics Committee. In order to ensure statistical independence, the derivation cohorts of the BSI and FACED scores were not considered for inclusion in the present analysis.7 8

Clinical assessments and calculation of severity scores

Patients were followed up at outpatient clinic assessments. The underlying aetiology of bronchiectasis was determined following testing recommended by BTS guidelines.13 The BSI and FACED scores were calculated using baseline data as described.7 8 Patients were classified according to severity cut-offs described in original publications as mild, moderate or severe.

Study end points

Mortality

Data on all-cause mortality were collected for up to 5 years in keeping with follow-up periods of the original BSI and FACED derivation cohorts.

Exacerbations

An exacerbation was defined as the requirement for antibiotics in the presence of one or more symptoms of increasing cough, increasing sputum volume, worsening sputum purulence, worsening dyspnoea, increased fatigue/malaise, fever and haemoptysis.13 14 Exacerbations were recorded for 12 months after severity score calculation.

Hospitalisation for severe exacerbations

Severe exacerbations were defined according to BTS guidelines as unscheduled hospitalisations or emergency department visits for exacerbations or complications as recorded from patient histories and verified using administrative databases.13 Hospital admissions were recorded up to 5 years follow-up post-score calculation.

FEV1% predicted decline

This was analysed for up to 4 years of follow-up.

QoL

This was assessed by the QoL Bronchiectasis (QoL-B) Questionnaire (V.3.1) and the St Georges Respiratory Questionnaire (SGRQ), with minimum clinically important differences (MCIDs) of 8 and 4 points, respectively.20 21

Symptoms and function

Cough symptoms were evaluated using the Leicester Cough Questionnaire (LCQ) (MCID of 1.3 units).22 Exercise capacity was evaluated with the 6 min walking distance (6MWD) test which has an MCID of 84 m.23

Definition of validity

The validity of each scoring system was evaluated in terms of discrimination- the degree to which groups are different from each other, and calibration, the degree to which the scores perform as expected across different healthcare systems. Good discrimination and calibration are required for a score to be considered valid.

For mortality and hospitalisations, a scoring system was considered to be valid if the area under the receiver operator characteristic curve (AUC) exceeded 0.7. A value of >0.7 is generally regarded as clinically useful and was therefore chosen as a cut-off for valid prediction.7 8 AUC evaluates discrimination. As mortality was the end point in both the BSI and FACED derivation studies, we also evaluated calibration compared with the original derivation. Scores were considered valid if there was no statistically significant difference between pooled event rates in the validation cohorts and the original predicted event rate.24 For QoL and symptom scores, scores were considered to be valid if the differences between the three groups exceeded the reported MCIDs.20–23

Statistical analysis

Simple descriptive statistics were used to compare baseline data according to data distribution. Mean differences were compared using t-test (or analysis of variance (ANOVA) for more than two groups) and medians using the Mann-Whitney U test or Kruskal-Wallis test as appropriate. The AUC was used for discrimination analysis and differences between AUC values were compared using the method of DeLong et al.25 For calibration analysis, ORs and 95% CIs were calculated comparing event rates in the validation cohorts with the original derivation cohorts. Exacerbations and hospital admissions during follow-up were evaluated by Poisson regression analysis with data presented as rate ratio (RR) with 95% CI.

Effect estimates were pooled using a random effects meta-analysis to determine overall discrimination and calibration. Statistical heterogeneity between cohorts was assessed using the Higgins I2 test.

Analyses were conducted using SPSS (V.21) for Windows (SPSS, Chicago, Illinois, USA), Graph Pad Prism V.6 (Graph Pad Software, San Diego, California, USA) and Metadisc (V.1.4; Hospital Universitario Ramon y Cajal, Spain).

Results

Baseline characteristics

Seven international cohorts were included comprising 1612 patients. The characteristics of each cohort are shown in table 1. Cohorts were heterogeneous in keeping with the fact that they were derived from different healthcare systems. However, all cohorts had a female predominance. The average annual rate of exacerbations ranged from 1 to 3/year. The cohorts were primarily classified as moderate-to-severe bronchiectasis based on mean BSI scores (6.0–9.7); however, in contrast, the majority were considered to be mild bronchiectasis according to the FACED score (mean 1.5–2.3). Data for mortality, hospitalisations and exacerbations were available in all seven cohorts. Additional data were available for lung function decline, 6 min walk test, QoL and LCQ from two cohorts (Dundee and Monza) comprising 744 patients.

Table 1

Characteristics of included cohorts

Characteristics	Dundee, Scotland	Galway, Ireland	Monza, Italy	Leuven, Belgium	Athens, Greece	Newcastle, England	Vojvodina, Serbia
N	494	280	250	190	159	126	113
Study dates	2011–2015	2008–2015	2011–2015	2006–2012	2010–2015	2009–2013	2010–2015
Age, mean (SD)	65.3 (12.9)	60.5 (14.6)	65.1 (12.2)	66.4 (16.0)	59.3 (16.2)	59.1 (14.5)	62 (13.0)
Female, n (%)	300 (60.7)	188 (67.1)	147 (58.8)	97 (51.0)	101 (64.0)	75 (59.5)	80 (70.8)
FEV1% predicted, mean (SD)	71.6 (24.7)	80.3 (25.9)	79.2 (27.5)	69.3 (25.3)	70.1 (24.9)	64 (26.9)	64.8 (26.2)
BMI, mean (SD)	25.9 (5.2)	27.1 (5.6)	23.7 (4.4)	23.9 (4.3)	24.6 (3.4)	26.2 (5.1)	25.1 (4.9)
Pseudomonas colonisation, n (%)	63 (12.8)	39 (13.9)	54 (21.6)	16 (8.4)	58 (36.5)	13 (10.3)	1 (1%)
Reiff score, mean (SD)	4.4 (3.0)	3.4 (3.0)	5.5 (2.7)	4.5 (1.3)	4.8 (2.5)	2.8 (1.4)	4.7 (2.4)
MRC dyspnoea score, mean (SD)	2.3 (1.1)	2.0 (1.0)	2.0 (1.3)	2.3 (1.2)	2.4 (1.5)	2.5 (1.1)	2.5 (1.4)
Exacerbations/year, mean (SD)	2.1 (2.6)	2.9 (1.3)	1.9 (2.0)	1.9 (2.1)	2.4 (1.5)	3.4 (1.7)	1 (1.25)
LTOT, n (%)	8 (1.6)	9 (3.2)	35 (14.0)	10 (5.3)	26 (16.4)	0 (0)	12 (10.6%)
Prior hospitalisations, n (%)	118 (23.9)	62 (22.1)	34 (13.6)	55 (28.9)	83 (52.2)	75 (59.5)	15 (13.3%)
BSI, mean (SD)	7.3 (4.4)	6.8 (4.5)	7.3 (4.1)	7.6 (4.6)	9.1 (5.4)	9.7 (4.9)	6.0 (3.7)
FACED, mean (SD)	2.1 (1.6)	1.5 (1.5)	2.3 (1.6)	1.9 (0.9)	2.1 (1.8)	1.6 (1.6)	2.3 (1.6)
End points evaluated	MortalityHospitalisationsExacerbationsSymptomsQuality of lifeFEV1 decline	MortalityHospitalisationsExacerbations	MortalityHospitalisationsExacerbationsSymptomsQuality of lifeFEV1 decline	MortalityHospitalisationsExacerbations	MortalityHospitalisationsExacerbations	MortalityHospitalisationsExacerbations	MortalityHospitalisationsExacerbations
Follow-up time for mortality, years	4	5	4	5	5	4	5

BMI, body mass index; BSI, Bronchiectasis Severity Index; LTOT, long-term oxygen therapy; MRC, Medical Research Council.

Mortality

Outcomes across each of the cohorts are shown in table 2. There was a clear difference in classification of the scores, with the BSI identifying 16.7%–38.9% of patients as low risk versus 52.6%–72% with FACED. A much larger number of patients were identified as having severe bronchiectasis with the BSI, 21.2%–63.5%, compared with 3.6%–13.2% with FACED.

Table 2

The overall distribution of patients, number of deaths and number of hospital admissions for severe exacerbations in mild, moderate and severe groups across the seven cohorts

	No. of patients per group			Mortality			Hospital admissions
	Mild, n (%)	Moderate, n (%)	Severe, n (%)	Mild, n (%)	Moderate, n (%)	Severe, n (%)	Mild, n (%)	Moderate, n (%)	Severe, n (%)
Dundee, Scotland (n=494)
BSI	136 (27.5)	211 (42.7)	147 (29.8)	1 (0.7)	13 (6.2)	28 (19.0)	3 (2.2)	24 (11.4)	75 (51.0)
FACED	303 (61.3)	145 (29.4)	46 (9.3)	12 (4.0)	15 (10.3)	15 (32.6)	44 (14.5)	45 (31.0)	13 (28.3)
Galway, Ireland (n=280)
BSI	109 (38.9)	92 (32.9)	79 (28.2)	8 (7.3)	11 (12.0)	25 (31.6)	2 (1.8)	6 (6.5)	30 (38.0)
FACED	217 (77.5)	53 (18.9)	10 (3.6)	23 (10.6)	19 (35.8)	2 (20.0)	18 (8.3)	15 (28.3)	5 (50.0)
Monza, Italy (n=250)
BSI	67 (26.8)	104 (41.6)	79 (31.6)	0 (0)	3 (2.9)	11 (13.9)	10 (14.9)	27 (25.9)	55 (69.6)
FACED	135 (54.0)	88 (35.2)	27 (10.8)	3 (2.2)	4 (4.5)	7 (25.9)	40 (29.6)	34 (38.6)	18 (66.7)
Leuven, Belgium (n=190)
BSI	51 (26.8)	63 (33.2)	76 (40.0)	2 (3.9)	16 (25.4)	26 (34.2)	6 (11.8)	18 (28.6)	34 (44.7)
FACED	100 (52.6)	65 (34.2)	25 (13.2)	9 (9.0)	19 (29.2)	16 (64.0)	23 (23.0)	22 (33.8)	13 (52.0)
Athens, Greece (n=159)
BSI	36 (22.6)	43 (27.0)	80 (50.3)	0 (0)	0 (0)	9 (11.3)	1 (2.8)	7 (16.3)	27 (33.8)
FACED	104 (65.4)	35 (22.0)	20 (12.6)	0 (0)	4 (11.4)	5 (25.0)	17 (16.3)	9 (25.7)	9 (45.0)
Newcastle, England (n=126)
BSI	21 (16.7)	25 (19.8)	80 (63.5)	0 (0)	1 (4.0)	15 (18.8)	0 (0)	5 (20.0)	52 (65.0)
FACED	91 (72.2)	27 (21.4)	8 (6.3)	6 (6.6)	7 (25.9)	3 (37.5)	37 (40.7)	15 (55.6)	5 (62.5)
Vojvodina, Serbia (n=113)
BSI	41 (36.3)	48 (42.5)	24 (21.2)	0 (0)	12 (25.0)	8 (33.3)	0 (0)	1 (2.1)	12 (50)
FACED	60 (53.1)	44 (38.9)	9 (8.0)	4 (6.7)	13 (29.5)	3 (33.3)	2 (3.3)	10 (22.7)	1 (11.1)
Total (n=1612)
BSI	461 (28.6)	586 (36.4)	565 (35.1)	11 (2.4)	56 (9.5)	122 (21.6)	22 (4.8)	88 (15.0)	285 (49.6)
FACED	1010 (62.7)	457 (28.3)	145 (9.0)	57 (5.6)	81 (17.7)	51 (35.1)	181 (17.9)	150 (32.8)	64 (44.1)

BSI, Bronchiectasis Severity Index.

There were no deaths in the low-risk BSI group in the Newcastle, Monza, Athens and Vojvodina cohorts and there were only 11 deaths in the low-risk groups overall (2.4%), compared with 57 deaths in the ‘mild’ bronchiectasis group according to FACED (5.6%). Mortality rates were higher for patients classed as severe using FACED (35.1%) compared with the BSI (21.6%) (shown in table 2).

The scores had very different characteristics in identifying patients at high risk of death. The FACED score had poor sensitivity ranging from 5% to 56% with a pooled sensitivity from meta-analysis across all seven cohorts of 28% (22%–36%). In contrast, the BSI was more sensitive (ranging from 40% to 100%, pooled value 65% (57%–71%) but less specific (range 41%–83%, pooled value 70% (66%–72%)) compared with FACED 93% (92%–95%).

Comparing AUC scores for mortality, the BSI had a numerically higher AUC in the Dundee, Galway, Monza, Athens, Newcastle and Vojvodina cohorts; the FACED score had a numerically higher AUC in Leuven. The only statistically significant difference was a superior AUC for the BSI in the Newcastle cohort. All AUC values for BSI were above 0.7 and six AUC values were above 0.7 for FACED, suggesting that both scores are valid in terms of discrimination for mortality (table 3).

Table 3

Area under the receiver operator characteristic curve values for mortality and hospital admissions across seven European cohorts

Cohort	Scores	AUC mortality	AUC hospital admissions
Dundee,Scotland (n=494)	BSI	0.78 (0.71 to 0.85)	0.84 (0.80 to 0.88)*
	FACED	0.76 (0.70 to 0.83)	0.68 (0.63 to 0.73)
Galway,Ireland (n=280)	BSI	0.73 (0.64 to 0.81)	0.87 (0.80 to 0.93)
	FACED	0.71 (0.62 to 0.80)	0.79 (0.73 to 0.86)
Monza,Italy (n=250)	BSI	0.86 (0.75 to 0.96)	0.79 (0.73 to 0.85)*
	FACED	0.77 (0.64 to 0.90)	0.62 (0.55 to 0.70)
Leuven,Belgium (n=190)	BSI	0.73 (0.65 to 0.81)	0.71 (0.63 to 0.78)
	FACED	0.78 (0.70 to 0.86)	0.65 (0.57 to 0.74)
Athens,Greece (n=159)	BSI	0.93 (0.87 to 0.98)	0.76 (0.67 to 0.84)*
	FACED	0.87 (0.80 to 0.94)	0.62 (0.51 to 0.74)
Newcastle,England (n=126)	BSI	0.82 (0.73 to 0.91)*	0.80 (0.72 to 0.87)*
	FACED	0.68 (0.52 to 0.84)	0.56 (0.46 to 0.66)
Vojvodina,Serbia (n=113)	BSI	0.75 (0.65 to 0.85)	0.97 (0.93 to 1.00)*
	FACED	0.74 (0.64 to 0.85)	0.69 (0.58 to 0.81)
Pooled cohort (n=1612)	BSI	0.76 (0.74 to 0.78)	0.82 (0.78 to 0.84)*
	FACED	0.76 (0.74 to 0.78)	0.65 (0.63 to 0.67)

*Indicates p<0.05 compared with FACED. BSI, Bronchiectasis Severity Index.

In the calibration analysis, both scores were well calibrated in terms of identifying patients at low risk of death. For identifying severe patients, however, the FACED score failed calibration with a pooled OR of 0.33 (0.23 to 0.48, p<0.0001), suggesting that the true mortality rate is 70% lower than that predicted by FACED. A meta-analysis of all available data for mortality, including one previously published validation study by Ellis et al,9 is shown in the online supplementary table S5(a) and (b).

Hospital admissions

The BSI appeared to be superior to FACED in terms of predicting hospital admissions. Rates of hospitalisation in mild patients according to BSI were 0%–14.9% and increased proportionately across mild, moderate and severe risk groups. Hospitalisation rates were much more variable for ‘mild’ patients according to FACED, with rates from 3.3% to 40.7% and, in the Dundee and Vojvodina cohorts, there was a paradoxically higher percentage of hospital admissions in the ‘moderate’ group compared with ‘severe’.

Comparing AUC values for hospitalisations, BSI showed statistically significant superiority in six of seven cohorts when analysed individually. FACED was only predictive of hospital admissions in the Galway cohort using the cut-off of 0.7 for valid discrimination.

Rates of hospital admissions comparing the mild and severe groups with the moderate group for each severity score are shown in table 4. Notably, the RR for low-risk patients using the BSI was numerically lower than FACED in all seven cohorts. Similarly, RR for hospitalisation in the severe group was numerically higher in five of seven cohorts.

Table 4

Hospitalisations meta-analysis data. RR of hospitalisation for severe exacerbations during follow-up according to different severity scores

Cohort	Scores	RR—mild	RR—moderate	RR severe
Dundee,Scotland (n=494)	BSI	0.19 (0.06 to 0.64)*	1.00 (reference)	4.46 (2.82 to 7.07)*
	FACED	0.47 (0.31 to 0.71)*	1.00 (reference)	0.91 (0.49 to 1.69)
Galway,Ireland (n=280)	BSI	0.28 (0.06 to 1.39)	1.00 (reference)	5.82 (2.42 to 14.0)*
	FACED	0.29 (0.15 to 0.58)*	1.00 (reference)	1.77 (0.64 to 4.86)
Monza,Italy (n=250)	BSI	0.50 (0.26 to 0.97)*	1.00 (reference)	4.45 (3.08 to 6.42)*
	FACED	0.75 (0.52 to 1.08)	1.00 (reference)	3.69 (2.55 to 5.34)*
Leuven,Belgium (n=190)	BSI	0.56 (0.29 to 1.30)	1.00 (reference)	1.68 (0.93 to 3.04)
	FACED	0.68 (0.38 to 1.22)	1.00 (reference)	1.54 (0.77 to 3.05)
Athens,Greece (n=159)	BSI	0.17 (0.02 to 1.39)	1.00 (reference)	2.07 (0.90 to 4.76)
	FACED	0.64 (0.28 to 1.43)	1.00 (reference)	1.75 (0.70 to 4.41)
Newcastle,England (n=126)	BSI	0 (no events)*	1.00 (reference)	3.25 (1.30 to 8.14)*
	FACED	0.75 (0.40 to 1.41)	1.00 (reference)	1.18 (0.47 to 2.95)
Vojvodina,Serbia (n=113)	BSI	0 (no events)*	1.00 (reference)	24.5 (3.2 to 188.4)*
	FACED	0.15 (0.03 to 0.67)*	1.00 (reference)	0.49 (0.06 to 3.82)
Pooled cohort (n=1612)	BSI	0.41 (0.27 to 0.62)*	1.00 (reference)	3.71 (2.95 to 4.66)*
	FACED	0.55 (0.41 to 0.75)*	1.00 (reference)	1.58 (0.92 to 2.71)

*Indicates statistical significance at p<0.05. Pooled data were obtained through random effects meta-analysis.

BSI, Bronchiectasis Severity Index; RR, rate ratio.

Exacerbations

Exacerbation rates were strongly linked to clinical severity scores. Using Poisson regression, low-risk patients, determined by the BSI, had fewer exacerbations in all cohorts and differences were statistically significant in four out of seven cohorts (table 5). Severe patients according to the BSI had significantly more exacerbations than the moderate group across all cohorts. In contrast, FACED was less consistent for predicting exacerbations. Paradoxically, we observed numerically more exacerbations occurred in the moderate versus severe FACED group in the Newcastle and Vojvodina cohorts, while ‘mild’ patients had numerically more exacerbations than moderate patients in the Monza and Athens cohorts, suggesting that the FACED score did not reliably and consistently identify patients at high risk of exacerbations. Differences between mild and moderate FACED groups were only statistically significant in two out of seven cohorts (Galway and Vojvodina) and differences between moderate and severe were only statistically significant in four out of seven cohorts. In the pooled analysis, the differences between mild and moderate FACED groups were not statistically significant, while overall differences between severe and moderate were significant (table 5).

Table 5

RR for exacerbations during follow-up according to different severity scores

Cohort	Scores	RR—mild	RR—moderate	RR severe
Dundee,Scotland (n=494)	BSI	0.66 (0.53 to 0.82)*	1.00 (reference)	2.14 (1.84 to 2.50)*
	FACED	0.91 (0.77 to 1.06)	1.00 (reference)	1.44 (1.15 to 1.80)*
Galway,Ireland (n=280)	BSI	0.70 (0.55 to 0.88)	1.00 (reference)	1.44 (1.16 to 1.78)*
	FACED	0.74 (0.59 to 0.92)*	1.00 (reference)	1.25 (0.82 to 1.92)
Monza,Italy (n=250)	BSI	0.76 (0.59 to 0.99)*	1.00 (reference)	1.38 (1.11 to 1.70)*
	FACED	1.06 (0.86 to 1.31)	1.00 (reference)	1.64 (1.24 to 2.17)*
Leuven,Belgium (n=190)	BSI	0.78 (0.52 to 1.18)	1.00 (reference)	2.08 (1.52 to 2.84)*
	FACED	0.86 (0.64 to 1.14)	1.00 (reference)	1.51 (1.05 to 2.16)*
Athens,Greece (n=159)	BSI	0.67 (0.34 to 1.33)	1.00 (reference)	1.92 (1.21 to 3.04)*
	FACED	1.20 (0.73 to 1.96)	1.00 (reference)	2.36 (1.33 to 4.21)*
Newcastle,England (n=126)	BSI	0.47 (0.32 to 0.71)*	1.00 (reference)	1.29 (1.02 to 1.65)*
	FACED	0.81 (0.63 to 1.04)	1.00 (reference)	0.95 (0.64 to 1.41)
Vojvodina,Serbia (n=113)	BSI	0.14 (0.05 to 0.38)*	1.00 (reference)	2.61 (1.69 to 4.04)*
	FACED	0.22 (0.13 to 0.37)*	1.00 (reference)	0.93 (0.49 to 1.77)
Pooled cohort (n=1612)	BSI	0.63 (0.52 to 0.78)*	1.00 (reference)	1.73 (1.42 to 2.12)*
	FACED	0.78 (0.60 to 1.01)	1.00 (reference)	1.40 (1.16 to 1.68)*

*Indicates statistical significance at p<0.05. Pooled data were obtained through random effects meta-analysis.

BSI, Bronchiectasis Severity Index; RR, rate ratio.

QoL-B questionnaire

QoL and symptom data were available in two cohorts (744 patients) which were pooled for analysis. Using the QoL-B, statistically significant differences were observed for respiratory symptoms, physical functioning and role functioning domains between mild/moderate and moderate/severe groups using the BSI. All differences were above the MCID (table 6).

Table 6

QoL-B questionnaire domains

	Mean difference(mild vs moderate)	p Value	Mean difference(moderate vs severe)	p Value
Respiratory symptoms (MCID=8)
BSI	−10.4 (3.0)	0.0006	−8.8 (2.6)	0.0009
FACED	−6.3 (2.5)	0.01	−2.9 (3.4)	0.4
Physical functioning (MCID=10)
BSI	−14.8 (4.8)	0.002	−20.7 (3.9)	<0.0001
FACED	−20.1 (3.7)	<0.0001	−4.5 (4.6)	0.3
Vitality (MCID=10)
BSI	−7.4 (3.2)	0.02	−6.8 (2.9)	0.02
FACED	−7.5 (2.6)	0.005	−2.2 (3.6)	0.5
Role functioning (MCID=8)
BSI	−11.1 (4.0)	0.006	−13.9 (3.5)	0.0001
FACED	−12.6 (2.2)	<0.0001	−8.8 (4.0)	0.03
Health perceptions (MCID=8)
BSI	−3.5 (3.5)	0.3	−8.6 (2.9)	0.003
FACED	−7.7 (2.7)	0.005	5.2 (4.0)	0.2
Emotional functioning (MCID=7)
BSI	−5.0 (3.3)	0.1	−6.1 (3.2)	0.05
FACED	−2.2 (2.9)	0.5	−1.2 (4.0)	0.8
Social functioning (MCID=9)
BSI	−5.0 (4.0)	0.2	−7.7 (3.7)	0.03
FACED	−7.6 (3.2)	0.02	1.6 (4.8)	0.7
Treatment burden (MCID=9)
BSI	−6.6 (4.9)	0.2	−5.8 (3.7)	0.1
FACED	−3.6 (3.6)	0.3	−2.2 (4.6)	0.6

Values show mean difference with SD for between-group differences. Differences that are statistically significant and above the MCID are shown in bold.

BSI, Bronchiectasis Severity Index; QoL-B, quality of life-bronchiectasis; MCID, minimum clinically important difference.

There were also significant differences for BSI in the vitality domains across both mild/moderate and moderate/severe groups but at levels below the MCID. Health perceptions were not significantly different between mild/moderate groups but were statistically and clinically different between moderate/severe groups.

FACED demonstrated less discrimination in terms of QoL-B domains. There were clinically and statistically significant differences between the mild/moderate FACED groups for physical functioning (table 4) but the only other difference exceeding the MCID was for role-functioning.

Similar data were observed for the SGRQ (figure 1). The differences between mild, moderate and severe groups for the BSI were significant (p<0.0001) with a difference of 11 points between mild/moderate groups and 12 points between moderate/severe groups, both above the MCID of four points. There was a 10-point difference between the mild/moderate FACED groups (p<0.0001) but no difference between moderate/severe groups, p=0.9. We further validated these findings in the only other published validation study from the Royal Brompton Hospital (76 patients).9 In this cohort, there were clear differences between BSI groups in terms of SGRQ score, with mean differences of 9 and 11 points comparing mild/moderate with moderate/severe, respectively (p=0.003). In contrast, there were no statistically significant differences between FACED groups and the between-group difference for moderate/severe was below the MCID, p=0.2 (see online supplementary figure S1).

Figure 1

Bar graphs showing quality of life, symptoms and lung function decline. For SGRQ and FEV1 decline, bars show mean with SEM. For Leicester Cough Questionnaire and 6 min walking distance, mean, SD and range are shown. BSI, Bronchiectasis Severity Index, SGRQ, St Georges Respiratory Questionnaire.

Symptoms and function

As above, symptom and functional data were only available in two cohorts (744 patients). Cough severity, evaluated by the LCQ was different between BSI groups (p<0.0001 by ANOVA) with mean differences of 1.5 and 1.4 between groups, both above the stated MCID of 1.3.

There was a significant difference between mild and moderate FACED groups but this did not reach the MCID (mean difference 1.1, p=0.005); there was no difference between moderate and severe groups (mean difference 0.26, p=0.7).

Data for 6 MWD were available for 471 patients. The differences in 6 MWD between groups for the BSI were 115 m (mild/moderate) and 83 m (moderate/severe), the latter just failing to reach the MCID. For FACED, differences were 53 m (mild/moderate) and 21 m (moderate/severe), neither of which is statistically significant or above the MCID (p=0.2).

There was a weak but statistically significant association between BSI and annual FEV1 decline (figure 1), driven by a higher rate of decline in the severe group as there was no difference between mild and moderate groups (p=0.7). FACED did not predict lung function decline with the lowest rate of decline observed in the moderate group.

Discussion

This study is the first multicentre study critically appraising whether prognostic indices can predict clinically meaningful outcomes broader than just mortality. If bronchiectasis severity tools are to be used in clinical practice to guide escalation of therapy, they need to predict outcomes that are relevant to these decisions.13 This prospective international observational study suggests that the BSI is superior to FACED in predicting clinically important disease-related outcomes, including hospital admissions, exacerbations, QoL, respiratory symptoms, 6 MWD and lung function decline in bronchiectasis.

The primary outcome in most intervention trials in bronchiectasis has been exacerbations or QoL; these are also considered to be the most important triggers for changes in treatment.4–6 13 According to our analysis, the BSI consistently stratified patients as having low, moderate and high risk of exacerbations and severe exacerbations requiring hospitalisation, while there were lesser differences between FACED risk groups for exacerbations. Furthermore, FACED had a very poor overall ability to predict hospitalisations with AUC scores below 0.7 in six of the seven included cohorts. It is perhaps not surprising that FACED predicts exacerbations poorly, as although risk factors for exacerbations in bronchiectasis have not been well defined, data from COPD show that the strongest predictor of future exacerbations is a previous history of exacerbations and anecdotally the same is true in bronchiectasis.26 27 The BSI incorporates prior history of mild and severe exacerbations, while the FACED score does not.

The BSI was also a valid predictor of respiratory symptoms and physical functioning using the QoL-B, as well as passing validity testing against the SGRQ, the 6 MWD and the LCQ. This is consistent with multiple studies published over the past 18 months where the BSI has been correlated with the SGRQ, the COPD assessment test—another measure of symptoms, the capsaicin cough sensitivity, impulse oscillometry, the 6 MWD and activity measured using accelerometers.28–31 Finally, Dente et al32 published a significant association between BSI and sputum neutrophilic inflammation, while no relationship with FACED was found. In the present analysis, the BSI accurately categorised different severities of bronchiectasis according to these parameters, whereas, although sometimes showing trends towards differences in these parameters, the FACED score did not pass validity for the majority of QoL-B domains, the 6 MWD or the LCQ.

Our analysis also suggests a relationship between the predictive ability of the BSI and lung function decline, which again was not evident with FACED. There have been few studies of lung function decline in bronchiectasis. In their analysis of 76 patients, Martínez-García et al found P. aeruginosa colonisation and severe exacerbations to be the strongest predictors.33 It is perhaps therefore not surprising that the BSI predicts lung function decline as it awards a high proportion of points to P. aeruginosa colonisation (3 points) and hospitalisation for severe exacerbations (5 points), while FACED awards 1 point to P. aeruginosa colonisation and does not consider severe exacerbations. Our results are therefore in keeping with previous studies.

There are clearly advantages and disadvantages to both scores. The BSI is slightly more complex than the FACED score, requiring the measurement of 8, rather than 5, clinical parameters with a variable weighting that awards different points for each. Evidence suggests that clinicians may find weighted scores more difficult to apply leading to an underutilisation in clinical practice.34 However, in the current era of telemedicine and online medical calculators that assign a total on inputting the relevant data in sequential order, this is potentially a concern of the past.34 35 Although FACED is an acronym that may be easier for clinicians to remember, the score is subject to the same limitations, also awarding different weights to different variables, while having a lower accuracy for the majority of clinical outcomes.

Our analysis identified several potential limitations of the FACED score. It failed calibration analysis, suggesting that its prediction performance varies across different healthcare settings and requires local recalibration before use. We further confirmed this finding by incorporating the cohort of Ellis et al9 into the analysis (see online supplementary material), which independently confirms the failure of calibration. FACED consistently had a sensitivity of <50% for prediction of mortality and hospitalisations. The FACED score appears to prioritise specificity over sensitivity. This is potentially problematic for clinical decision-making, as it is counterintuitive to say that only a quarter of patients dying of bronchiectasis or being hospitalised for severe exacerbations have ‘severe’ bronchiectasis. There are some circumstances where a high specificity, that is, a high confidence that a high-risk patient will die, is desirable, the most obvious being in assessment of mortality risk in potential lung transplant recipients and the FACED score appears to be well adapted to this. However, in view of these limitations and the finding that the ‘severe’ FACED group was not associated with increased exacerbations or differences in most other morbidity parameters, we suggest that FACED should not be viewed as a severity assessment tool but rather as a mortality risk tool and therefore the terms mild, moderate and severe be replaced with low, intermediate and high risk of mortality when describing the FACED score.

It is widely accepted that prognostic model development is a three-stage process, comprising derivation (creating the rule), validation (applying the rule to new populations of patients to confirm its accuracy) and impact analysis (applying the rule and determining if it can improve clinical outcomes for patients).36 Our results suggest the BSI is superior in identifying patients at low risk of mortality, hospitalisation, exacerbations and morbidity who may benefit from primary care or nurse-led follow-up, which have the potential to either improve access or reduce healthcare costs and improve patient satisfaction.37 The BSI is also more sensitive in identifying patients at high risk of mortality, hospitalisation and exacerbations who may benefit from more aggressive treatment early on in their disease course to reduce associated complications, as well as closer follow-up in specialist bronchiectasis clinics. Our results do not, however, prove that implementation of the BSI may improve clinical outcomes and an impact analysis is now required.

A great strength of the present study is the inclusion of multiple cohorts across Europe. However, limitations of the study must be acknowledged. So far these scores have only been evaluated in European cohorts with similar demographics. Further validation would be desirable in populations significantly different from the original derivation studies, for example, in the USA where there is a high prevalence of non-tuberculous mycobacteria bronchiectasis or in Asian populations, where the overall prevalence of bronchiectasis is suspected to be higher than in Caucasians.38 A large amount of additional data is likely to be generated by ongoing international registry projects which incorporate calculation of the BSI.39 40

Conclusion

In conclusion, the BSI accurately predicts mortality, hospital admissions, exacerbations, QoL, respiratory symptoms, 6 MWD and lung function decline in bronchiectasis, providing a clinically relevant evaluation of disease severity.