Improving the Reliability Between the BODE Index and the BODS Index in Which the 6-Min Walk Test Was Replaced with the Five-Repetition Sit-to-Stand Test.

Purpose: The BODS index has been confirmed to have predictive properties similar to the original BODE index for mortality in COPD. We evaluated the agreement between the BODS index and the BODE and explored with an updated BODS how this agreement could be improved and its ability to correctly discriminate individual participants' mortality in a prospective cohort study. Patients and
Methods: We included prospectively a consecutive sample of 137 patients with COPD, between 40 and 80 years, during 2014 and followed for 5 years (2014-2019) in the Pneumology section of a public university hospital in Spain. They participated in the baseline data collection, which included BODE- and BODS-related measurements and prognostic factors, and were followed up for 5-year mortality. We used Bland-Altman plots and the kappa coefficient to analyze the agreement between both the original and updated BODS and the BODE index, and we used the areas under ROC curves (AUC) to compare their discriminative abilities for 5-year all-cause mortality.
Results: The original BODS index scores and quartiles had weak agreement with the BODE index, and our updated BODS strengthened these agreements (a small, statistically nonsignificant mean bias [<0.03] with LoAs<2 points, and a substantial Kappa coefficient [k =0.63; IC 95%: 0.53-0.73]). In addition, the updated BODS index scores had better summarized ability than the BODS index in discriminating participants' mortality during the following 5 years (AUC: 0.768 versus 0.736; p=0.04).
Conclusion: The updated BODS index scores and quartiles may provide prognostic information similar to that provided by the BODE index in COPD. Future research should focus on index improvement through external validation, as well as the assessment of safety and effectiveness in clinical practice by means of impact studies.

Introduction

Chronic obstructive pulmonary disease (COPD) is associated with significant premature mortality.1,2 Thus, substantial efforts have been made to build, validate, and/or update composite prognostic indices such as the BODE index.3–6 This index can be used to make treatment choices and follow-ups.7,8

Although the original BODE index received much attention for heavy smoker patients with COPD, its application in practice has been limited because the 6-minute walk test (6MWT) is difficult to assess in routine clinical and/or primary-care settings.6 To overcome this practical problem, new proposals have emerged based on replacing the 6MWT with alternative factors prognostic of mortality. Age and acute exacerbations were initially selected to simplify the BODE index, leading to several indices.9 Later, interest in functional tests grew.10 The BODS index is a representative index from this last tendency, and it has also demonstrated prognostic value for predicting mortality risk in patients with COPD.11 It replaces the four stratified categories of the 6MWT in the BODE index with others from the five-repetition sit-to-stand test (5-STS). This test alone has also demonstrated prognostic value for mortality in subjects with COPD.12,13 It has also been used to evaluate exercise tolerance.14–16 Although the BODS index showed association with time to death,11 validation of its ability to correctly discriminate individual participants’ mortality within a time horizon is lacking. Moreover, to our knowledge, no one has yet assessed the degree of agreement between the BODE index and the BODS index.

The main objective of our study was to evaluate the agreement between the BODS index and the BODE and its ability to discriminate mortality within a time horizon in patients with COPD, and to explore whether an updated BODS could improve this agreement and discriminative ability. A second aim was to test the prognostic value of this updated BODS to predict time-to-death.

Materials and Methods

Participants and Study Design

A consecutive sample of eligible patients with stable COPD was prospectively recruited from a hospital in Spain during 2014. All study participants provided written informed consent. The study complied with the Declaration of Helsinki and the study protocol was approved by the institutional review board of the hospital, called the “Ethical Committee of Clinical Research of the General University Hospital” (approval number: EST-35/13). Inclusion criteria were a diagnosis of COPD according to the Global Initiative for COPD (GOLD)8 recommendations,17 and age between 40 and 80 years. Exclusion criteria were unstable cardiac condition within 4 months of the start of the study, cognitive deterioration, and inability to walk. The cardiac criterion was established to avoid the possibility of an adverse event during physical functional tests (eg 6MWT, 5-STS). The cognitive and walking criteria were based on the need to ensure that patients could provide reasonable answers and performance in the physical functional tests.

BODE and BODS Measurements

We measured the four BODE components and calculated scores as expressed by Celli:3 body mass index as mass [kg]/height [m2]; dyspnea by the modified British Medical Research Council (mMRC) scale;18 the forced expiratory volume in the first second or FEV1 in percentage by means of a post-bronchodilator spirometry with a Master Scope Spirometer (version 4.6, Jaeger, Würzburg, Germany) according to the American Thoracic Society guidelines;19 and the 6MWT as described previously.20

For calculation of the BODS, we first measured the 5-STS (in seconds), which required participants to rise from a chair with their arms across their chest, and then sit back down, five times.21 Then, we assigned points (0–3) to patients’ performance according to four categories (<10.89, 10.90–13.64, 13.65–19.06, and >19.06 s).12 For other BODS parameters, we used the same point-assignment system and cut-offs as the original BODE index.11 The BODS index scores (range 0–10) were also categorized into the same four scales of the BODE: one to four with scores of 0–2, 3–4, 5–6, and 7–10, respectively. As these scales were initially based on quartiles,3 we called them BODE quartiles.

Outcome Measure and Prognostic Factors

The primary outcome measurement used to examine prognostic values was 5-year all-cause mortality (2014–2019). We focused on all-cause mortality since the determination of causes of death is prone to misclassification and is difficult to perform, even if centrally adjudicated.22 Deaths were captured at 5 years from patients’ electronic files.

Age (years), sex, and three clinical factors with high prognostic value for mortality were collected to be used as covariates in multivariate Cox regression analysis: grade of dyspnea (measured on the mMRC scale); history of heart disease, which accounts for a third of deaths in COPD;22 and the number of severe exacerbations in the previous year since patients admitted to hospital show a 22–43% mortality rate.23,24 Other clinical measures were collected to describe participants: current smoking status (yes/no), history of smoking in pack-years, number of comorbidities (measured with a functional comorbidity index), and GOLD-index stage.8

Statistical Analysis

Participants’ characteristics at baseline were summarized. The t-test or Mann-Whitney U-test, and chi-square tests were used to compare baseline characteristics between surviving and deceased participants. We analyzed the agreement between the BODS index and the BODE index regarding both scores and quartiles scales. For scores, Bland–Altman plots were used, including the upper and lower limits of agreement (LoA) and the mean difference between indices, which we called “mean bias”. For quartiles, we calculated the kappa coefficient (k). If the kappa was poor (<0.4)25 or mean bias was statistically significant, an updated BODS was generated and again compared with the BODE index in the same manner.

To update the BODS index we defined new four categories for the 5-STS based on their discriminative ability for 5-year mortality, and generated a new point assignment system for them as reported previously ( and ).4,26,27 We kept the method of point assignment and cut-offs used in the original BODS index for its other parameters.

The ability of the updated BODS, the BODS, and the BODE index scores to correctly discriminate individual participants’ mortality was summarized by the area under the ROC curve (AUC) and compared by DeLong’s test.

Cox univariate and multivariate regression analyses were used for covariate selection and subsequently for estimating the hazard ratios (HRs) and 95% confidence intervals (CIs) of both the updated BODS scores and the quartile scales. These multivariate analyses were first adjusted for age and sex (as reported previously),11 and second by the addition of the selected covariates. Time-to-death of the groups defined by the quartiles scale of the BODS were described by Kaplan-Meier survival curves and compared by the Log rank test.

Regarding missing values from patients who failed to complete the 5-STS test, we imputed an estimated time, as recommended.21 No imputations were made for missing values from patients unable to do any tests (ie, who had zero repetitions).

A sample size of 110 subjects was selected to achieve a power of 0.80 when using Cox regression analysis at a significance level of 0.05. We assumed an anticipated event rate of 0.30 and an R-squared equal to 0.18 of the variable of interest for other covariates. All analyses were performed using SPSS, version 24.0 (IBM SPSS).

Results

Participants

The 137 patients finally included at baseline had a mean age of 67 years; most of the subjects were male (87.6%) and 29.9% were current smokers (Table 1). All patients were able to perform all tests, except 3 who did not complete the 6MWT. Six failed to complete the 5-STS, but no one was unable to do at least two repetitions. Of the 134 patients with complete data, 77 (57.5%) were classified in the first BODE quartile, 37 (27.6%) in the second, 16 (11.9%) in the third and 4 (3%) in the fourth.

Table 1

Baseline Characteristics of the 137 Patients with COPD, and by Groups of 5-Years Mortality

Characteristic	AllN=137	LivingN=99	DeadN=37	p-value
Sociodemographic variables
Age, years	66.92 (±8.33)	65.58 (±8.32)	70.81 (±7.08)	0.001
Males	120 (87.60%)	84 (84.85%)	35 (94.59%)	0.126
Clinical and pulmonary variables
Current smoker	41 (29.90%)	31 (31.31%)	10 (23.03%)	0.628
Pack-years	55.00 (33.00)	56.55 (24.72)	65.86 (25.60)	0.055
Body mass index, kg/m2	28.92 (±5.06)	29.05 (±5.07)	28.57 (±5.15)	0.620
Comorbidity index	3.00 (2.50)	2.88 (1.53)	3.89 (1.76)	0.001
History of heart disease, yes	27 (19.70%)	10 (10.10%)	17 (45.95%)	0.000
Number of severe exacerbations in the previous year	1.00 (1.00)	0.59 (0.62)	1.00 (0.67)	0.002
mMRC; scale dyspnoea
0–1	88 (64.23%)	75 (75.76%)	12 (32.43%)	0.000
2	33 (24.10%)	17 (17.17%)	16 (43.24%)
3–4	16 (11.68%)	7 (7.07%)	9 (24.32%)
FEV1, %	50.21 (±16.50)	52.21 (±15.95)	44.78 (±17.16)	0.019
GOLD stage
A	24 (17.50%)	22 (22.22%)	2 (5.41%)	0.014
B	22 (16.10%)	17 (17.17%)	5 (13.51%)
C	12 (8.80%)	11 (11.11%)	1 (2.70%)
D	79 (57.70%)	49 (49.49%)	29 (78.38%)
Functional tests
6MWT, m	365.00 (90.75)	365.92 (76.07)	300.60 (90.73)	0.000
5-STS test, s	14.53 (4.52)	14.27 (4.71)	22.91 (13.76)	0.000

Note: Data are reported as mean (±standard deviation), median (IQR), or n (%).

Abbreviations: FEV1, forced expiratory volume in 1 s; 5-STS, five-repetition sit-to-stand test; GOLD, Global Initiative for Chronic Obstructive Lung Disease; mMRC, modified British Medical Research Council; 6MWT, six-minute walking test.

A total of 37 patients (27.0%) died within the 5 years of observation (7 within the first 2 years). Of the remaining 100 patients, only 1 was lost to follow-up. Participants who died were older and had worse clinical and pulmonary characteristics (higher numbers of pack-years, comorbidities, and previous exacerbations; higher presence of heart disease) than patients who were alive at 5 years.

Agreement Between the BODS Index and the BODE Index

Figure 1 shows the Bland-Altman plots comparing the BODE and BODS scores. Often the original BODS provided higher scores than the BODE index (Figure 1A), and mean bias (1.05) was statistically significant (p=0.001). LoA+ was >2 points. The kappa coefficient between quartiles was low (k =0.34; IC 95%: 0.22–0.45). These discrepancies were also seen regarding quartiles scales. Figure 2A shows that almost 50% of patients classified in each one of the first three BODE quartiles were placed in a higher quartile by the BODS index (35/77, 15/37, and 7/16 cases in Q1, Q2, and Q3, respectively). Regarding all patients, the BODS classified 104 (77.61%) in a higher quartile than the BODE index (ie overrated).

Figure 1

Bland–Altman plots for BODE index scores and the two BODS indices: the original (A) and the updated (B).

Figure 2

Number of subjects within the quartiles of the original (A) and the updated BODS (B) in each BODE index quartile.

Because of the poor agreement of the original BODS index with the BODE index, we updated the BODS index. Table 2 shows its scoring system and the new categories for the 5-STS test.

Table 2

Assignment of Points for the Updated BODS Index

	0 Points	1 Point	2 Points	3 Points
Body mass index, kg/m2	>21	≤21
FEV1, % predicted	≥65%	50–64%	36–49%	≤35%
Dyspnea, mMRC scale	0–1	2	3	4
5-STS test, seconds	<16	16–18.9	19–22	≥22

Abbreviations: FEV1, forced expiratory volume in 1 s; 5-STS, five-repetition sit-to-stand test; mMRC, modified British Medical Research Council.

The updated BODS index scores provided a small, statistically non-significant mean bias (<0.03) with the BODE index, with LoAs<2 points (Figure 1B). The kappa coefficient between quartiles was substantial (k =0.63; IC 95%: 0.53–0.73). The updated BODS index overrated 15% of 134 patients, and the major discrepancies lay in the BODE 5–6 quartile (Figure 2B).

Ability of the Indices to Discriminate Mortality Within a Time Horizon

Table 3 shows that the discriminative abilities of both the updated and original BODS were at least as good as the BODE index for 5 and 2-year mortality (p>0.10). Nevertheless, the updated BODS index was statistically better than the original BODS index at discriminating 5-year (p=0.041 in the DeLong’s test) and 2-year all-cause mortality (p=0.007 in the DeLong’s test).

Table 3

Discriminative Ability for Mortality at 5 and 2 Years

	5 Years*		2 Years*
Characteristics	AUC	95% CI	AUC	95% CI
Updated BODS index	0.768a	0.68–0.86	0.661b	0.51–0.82
Original BODS index	0.736a	0.64–0.83	0.619b	0.44–0.79
BODE index	0.730	0.63–0.83	0.646	0.46–0.83

Notes: *Only p-values for significant statistically differences among AUCs are showed in this table. ap-value was 0.041 and bp-value was 0.007 for differences between the original and updated BODS.

Abbreviations: CI, confidence interval; AUC, area under the curve; BODS, body mass index, airflow obstruction, dyspnea, five-sit-to-stand test (5-STS); BODE, body mass index, airflow obstruction, dyspnea, exercise capacity.

Prognostic Value of the Updated BODS Index to Predict Time-to-Death

The Kaplan-Meier plot (Figure 3) shows that increases in the updated BODS quartiles were associated with a reduction of time-to-death (X2[3]=43.81 log rank; p<0.001). The mortality rates within the 5-year follow-up in Q1–Q4 of the updated BODS index were 13.3% (n=10/75), 32.4% (n=12/37), 35.5% (n=5/13), and 90.9% (n=10/11), respectively.

Figure 3

Kaplan–Meier survival curves of all patients divided into four groups, from the updated BODS index.

Cox multivariate analysis of Table 4 adjusted for all covariates (age, sex, heart disease and exacerbations) revealed that only the BODS 5–6 and 7–10 quartiles, compared with the 0–2 quartile, were independently associated with increased all-cause mortality risk. The updated BODS index scores (0–10) were also associated with mortality.

Table 4

Summary of Cox Regression Analysis for Predicting Time to Death

	Univariate Model		Multivariate Modela		Multivariate Modelb
Characteristics	HR (95% CI)	p-value	HR (95% CI)	p-value	HR (95% CI)	p-value
Updated BODS quartiles - per increase to next stage (reference 0–2)
3–4	2.58 (1.11–5.97)	0.027	2.34 (1.01–5.45)	0.048	2.24 (0.95–5.03)	0.066
5–6	3.35 (1.15–9.81)	0.027	3.03 (1.02–8.97)	0.045	3.03 (1.01–9.13)	0.048
7–10	12.40 (5.07–30.35)	0.000	8–99 (3.57–22.61)	0.000	6.85 (2.58–18.17)	0.000
Updated BODS scores (0–10)	1.47 (1.28–1.68)	0.000	1.41 (1.22–1.62)	0.000	1.38 (1.19–1.60)	0.000

Notes: aAdjusted by age and sex. bAdjusted by age, sex, number of severe exacerbations in the previous year, and history of heart disease.

Abbreviations: CI, confidence interval; HZ, Hazard ratio; AUC, area under the curve; BODS, body mass index, airflow obstruction, dyspnea, five-sit-to-stand test (5-STS).

Discussion

Our study showed that the original BODS index scores and quartiles had weak agreement with the BODE index scores, and that our updated BODS strengthened this agreement. Moreover, we showed that the updated BODS had better summarized ability than the original BODS index in discriminating individual participants’ mortality. Finally, we also found that increases in the updated BODS index quartiles and scores were associated with an earlier time-to-death, even after adjusting for relevant prognostic factors.

This study had several strengths. First, the present study expands previous work to develop the BODS by determining its agreement with the BODE and its discriminative ability. Second, our final point assignment system (0–3) for scoring the cut-offs of the 5-STS test within the updated BODS was determined with a methodology widely used for updating prognostic indices.4,26,27 Third, while the original BODS used Cox multivariate analysis adjusted for age and sex, we added more prognostic indicators (the history of heart disease, number of severe exacerbations, and dyspnea), which are strong predictors of mortality as cited above.22–24

In our study, the original BODS index scores showed a pattern of over-scoring with respect to the BODE index, with many patients scoring two or more points higher and a mean bias of 1 point, which have proven to be a prognostic difference.28 Thus, clinicians should not use interpretations from the scores of these two indices interchangeably. In contrast, the updated BODS index showed better agreement for two of these quartiles and no mean bias. As the cut-offs in the 5-STS test were the only differences between both BODS indices, the original and the updated, they were the main source for that differential pattern of agreement between the two BODS and the BODE index. Interestingly, the first cut-off point used to classify the patients with the fastest times in that test was set in the original BODS to be five seconds below that in the updated BODS index (11 vs 16s). Thus, while patients who performed within that interval scored zero points in the updated BODS, they scored one or more in the original BODS. While we generated these cut-offs for the updated BODS by using a method often used26 for choosing cut-offs with discriminative ability, the authors of the original BODS generated its cut-offs by using quartiles of the 5-STS values within their study population.12 Some authors have suggested that this last method may generate cut-offs that are not necessarily transferable to other populations.29

In our study, we tried to improve the agreement of the indices without sacrificing the prognostic value of the original BODS index. Fortunately, our study found that both the original and the updated BODS indices showed acceptable discriminative ability for individual participants’ 5-year and 2-year mortality. Nevertheless, the updated index showed slightly better discrimination than the original BODS. This slight improvement was partially expected because, as cited above, we used a method that searched for cut-offs for the 5-STS test that had discriminative ability. On the other hand, our study also showed that the updated BODS index scores and some quartiles (5–6 and 7–10) were associated with time-to-death. Similar predictive equivalence with time to death was previously reported with the original BODS.11

Limitations

This study had also some limitations. First, although we included strong prognostic factors for confounding-control purposes, some important baseline factors might have been omitted (eg, physical activity, participation in rehabilitation programs, etc.). Second, only baseline values were available despite the fact that many of our predictive factors may vary over time. Consequently, in this line, we also omitted other time-dependent factors (such as smoking cessation or longitudinal loss of fat-free-mass), which have showed relevance on quality of life and mortality, respectively.30,31 Third, the precision of the HRs of the highest quartiles of the updated BODS index was low because of the small number of patients in those quartiles. Finally, as our study was developed in a single center and only a small number of women participated, its results should be generalized with caution.

Implications for Research and/or Practice

The agreement between the BODE and the updated BODS index supports the latter as an accurate tool that provides results very close to BODE scores and quartiles in the absence of the 6MWT. Furthermore, our data suggest that the updated BODS is significantly better in the discriminative ability to classify individual participants’ mortality compared with the original BODS. In addition, as the 5-STS requires only little resources (ie a standard chair, a stopwatch and less than 2 minutes), clinicians might reduce costs by substituting the difficult-to-obtain 6MWT for the 5-STS test. Future research should focus on index improvement through external validation, as well as the assessment of safety and effectiveness in clinical practice by means of impact studies.

Conclusion

To conclude, both BODS indices, the original and the updated, may provide similar prognostic information in COPD, but the scores and quartiles from the updated index may be closer to those from the BODE index.