Smallest worthwhile effect of land-based and water-based pulmonary rehabilitation for COPD.

This study aimed to determine the smallest worthwhile effect of land-based and water-based pulmonary rehabilitation on 6-min walk distance among people with chronic obstructive pulmonary disease (COPD). Using a benefit-harm trade-off method, people with COPD who had completed two baseline 6-min walk tests at the commencement of outpatient pulmonary rehabilitation were presented with two scenarios: 8 weeks of land-based and 8 weeks of water-based pulmonary rehabilitation. Participants were guided through an iterative process allowing them to progressively refine their estimates of the smallest improvement due to each form of rehabilitation that would outweigh the associated costs, risks and inconvenience presented in the scenario. 100 people with COPD participated (mean±sd age 72±9 years, forced expiratory volume in 1 s 54±16% predicted and baseline 6-min walk distance 377±101 m). For land-based pulmonary rehabilitation, the median smallest worthwhile effect was 20 m (95% CI 15-37 m). For water-based pulmonary rehabilitation, the median smallest worthwhile effect was 26 m (95% CI 15-33 m). These estimates did not differ significantly (p=0.10). People with COPD typically perceive that pulmonary rehabilitation would be worthwhile if it increased the 6-min walk distance by about 6%. The smallest worthwhile effects of land- and water-based pulmonary rehabilitation were similar.

Introduction

Chronic obstructive pulmonary disease (COPD) is defined as chronic limitation of expiratory airflow that is not fully reversible. It is characterised by chronic dyspnoea, reduced exercise tolerance and reduced quality of life [1]. COPD is a leading cause of morbidity and early mortality worldwide [2], and results in an economic and social burden that is substantial and increasing [3, 4]. In Australia alone, COPD costs the community approximately $800 million annually [5]. The worldwide prevalence of COPD is anticipated to increase until at least 2020 [6].

Reduced exercise tolerance in people with COPD has been attributed to abnormal respiratory mechanics, gas-exchange disturbances and deconditioning of skeletal muscle [7]. Pulmonary rehabilitation involves exercise training, education and psychosocial support [8], traditionally in a group format in a dedicated exercise area, such as a hospital gymnasium. It is recognised as an essential component of the management of people with COPD because it improves exercise capacity, reduces hospital admissions and improves health-related quality of life [9, 10]. However, due to the increasing prevalence of COPD, only a small proportion of people receive pulmonary rehabilitation [8]. Therefore, other modes of training are needed in order to meet the growing demand [11, 12].

An alternative mode of delivering pulmonary rehabilitation is to deliver the exercise training in water. Pulmonary rehabilitation that incorporates aquatic exercise training has equivalent benefits to land-based pulmonary rehabilitation in people with COPD [13], and produces significantly greater improvements than land-based training in the incremental shuttle walk test (mean difference 39 m, 95% CI 5–72 m) and endurance shuttle walk test (228 m, 95% CI 19–438 m) in people with COPD with physical comorbidities [12]. In addition, some people with COPD have greater ability to participate in and progress exercise in water than on land due to physical comorbidities such as obesity and lower-limb musculoskeletal conditions [12].

The smallest worthwhile effect of an intervention is the smallest beneficial effect of the intervention that justifies its costs, risks and inconvenience [14]. Estimates of the smallest worthwhile effect are relevant to both clinicians and researchers. When researchers plan a clinical trial, the smallest worthwhile effect can be used in the sample size calculation to ensure that the trial has enough statistical power to detect an effect of that magnitude. Once trial data have been analysed, clinicians can decide whether patients are likely to consider a treatment is worthwhile, based on whether the average (i.e. expected) effect of an intervention in a clinical trial exceeds the smallest worthwhile effect.

Ferreira et al. [14] argue that three criteria must be met for a valid estimate of the smallest worthwhile effect of an intervention. 1) The estimate must be derived exclusively from the patient: the decision about what magnitude of an effect makes an intervention worthwhile must be made entirely by the patient [15]. They must not be influenced by a researcher or clinician. 2) The estimate must be generated with respect to a specific intervention: decisions about whether an intervention's effect is worthwhile must consider whether the benefit of the intervention outweighs its costs, risks and inconveniences [15]. Therefore, estimates of smallest worthwhile effects must be specific to a particular intervention. 3) The estimate must be expressed in terms of an effect, not an outcome: the effect of an intervention is the difference in outcomes that would occur with and without that intervention. The outcome that occurs without intervention can be influenced by natural recovery, regression to the mean and placebo effects [16]. So, estimates of the smallest worthwhile effect of an intervention must be expressed in terms of the difference in outcomes with and without the intervention, rather than in terms of outcomes or changes in outcome over the course of treatment [14].

Thresholds known as “minimum clinically important differences” [17] are widely used to judge whether the average effect of pulmonary rehabilitation would be considered worthwhile by people with COPD. The minimum clinically important difference in 6-min walk distance (6MWD) for people with COPD has been estimated at 54 m [18], 80 m [19], 35 m or 10% [20], 25 m [21] and 26 m [22]. The method used to estimate the minimum clinically important difference differs between these five studies. However, all five studies used some form of anchor-based and/or distribution-based methods to derive their estimates.

Anchor-based estimates are derived by asking patients to link the amount of change in an outcome with a descriptor on a scale such as “a little better”, “somewhat better” or “moderately better”. The estimates may therefore appear to be derived from the patient. However, it is almost always the researcher, not the patient, who decides which descriptor on the scale corresponds to the “clinically worthwhile” threshold [14]. Moreover, these decisions are not explicitly linked to a particular intervention.

Distribution-based estimates are actually clinimetric properties of an outcome measure in a particular population, like the minimum detectable change, a measure of variability or a standardised effect size. They do not reflect patients' perceptions of the magnitude of effect and they are not linked to any intervention. Therefore, they cannot tell us the magnitude of effect that patients consider outweigh the costs, risks and inconvenience of an intervention.

Therefore, although anchor- and distribution-based methods are widely used to generate estimates of minimum clinically important differences, they do not provide valid and interpretable estimates of the smallest worthwhile effect of an intervention [14]. In contrast, the benefit–harm trade-off method, described in detail in the Methods section, potentially provides valid and interpretable estimates of the smallest worthwhile effect [15, 23].

As discussed above, pulmonary rehabilitation with water-based exercise training has similar benefits to land-based pulmonary rehabilitation [12, 13]. However, land- and water-based exercise training may differ with respect to their costs, risks and inconvenience. For example, water-based exercise training may have greater costs (e.g. swimsuit), risks (e.g. reaction to chlorine or falls on wet surfaces) and inconvenience (e.g. increased work of breathing due to lung function restriction with water immersion or exposing more of one's body to other patients). Therefore, patients may justifiably consider that greater benefit is required to make pulmonary rehabilitation in an aquatic setting worthwhile. Conversely, patients may consider these issues to be minor and may enjoy the novelty, the buoyant environment or the variety of water-based exercises. Therefore, it is particularly important for aquatic-specific estimates of smallest worthwhile effects to be generated using valid methods and to compare these to equivalent estimates for land-based pulmonary rehabilitation.

The present study had two aims. First, we sought to identify the smallest worthwhile effect of pulmonary rehabilitation on 6MWD in people with COPD, using valid and interpretable methods. Second, given the potential differences in costs, risks and inconveniences of land- and water-based pulmonary rehabilitation, we sought to determine if people with COPD perceive that land- and water-based pulmonary rehabilitation have different smallest worthwhile effects.

Methods

Participants

Adults with COPD who were referred to and attended pulmonary rehabilitation at the Prince of Wales Hospital or Royal Prince Alfred Hospital (Sydney, Australia) were recruited. The inclusion criteria were: a diagnosis of COPD (i.e. Global Initiative for Chronic Obstructive Lung Disease (GOLD) stage I–IV) [24]; and completed spirometry and two 6-min walk tests as part of a baseline assessment for pulmonary rehabilitation. The exclusion criteria were: inability to understand English; an unstable clinical condition (defined as change in medication or hospitalisation within the previous 4 weeks); significant comorbidity that would hinder exercise training, affect the ability to exercise in water or alter the response to exercise training; or unwillingness or inability to provide consent. Ethical approval was obtained by the Human Research Ethics Committees of the Sydney Local Health District and the South Eastern Sydney Local Health District.

Baseline data collection

After eligibility was confirmed and written consent to participate was obtained, each participant's baseline characteristics were recorded. These details included age, sex, height, weight, lung function, GOLD stage and St George's Respiratory Questionnaire score to characterise the study cohort, and the best result from two 6-min walk tests performed at the initial pulmonary rehabilitation assessment. Both recruitment sites used a ≥30-m track for the 6-min walk test [25].

Benefit–harm trade-off analysis

The investigator entered the participant's baseline characteristics into a custom-designed computer application. The application used these details to generate two scenarios (land-based and water-based pulmonary rehabilitation). The application then guided the participant through an iterative process (known as benefit–harm trade-off analysis) to progressively refine the participant's estimate of the amount of benefit that would make the pulmonary rehabilitation programme in each scenario worthwhile. To demonstrate the process, the following example presents the scenario that was presented to a participant who: was enrolled at Prince of Wales Hospital; had a baseline 6MWD of 200 m; and was randomised to the land-based pulmonary rehabilitation scenario first.

Existing research suggests that, with current management of your condition, in 8 weeks from now, the distance you can walk in 6 minutes will be 200 metres. However, the distance you can walk in 6 minutes may be altered by an 8-week course of pulmonary rehabilitation.

If you were to participate in the course of pulmonary rehabilitation, you would attend 1-hour classes twice per week at Prince of Wales Hospital. The exercise would be supervised by a physiotherapist. The exercises would be undertaken in the hospital gym with other patients. The course of pulmonary rehabilitation would include education about management of breathlessness. You would also be prescribed exercise to perform independently at home on other days of the week. You would need to pay for transport and/or parking, but the program is otherwise free. You might feel some muscle soreness in the days after doing the exercise.

If undertaking pulmonary rehabilitation meant that in 8 weeks the distance you can walk in 6 minutes would be 1000 metres (instead of 200 metres, without pulmonary rehabilitation), would you think the treatment was worthwhile?

The highest value of 1000 m was nominated as beyond the maximum plausible improvement that could be anticipated from pulmonary rehabilitation. Therefore, if the participant clicked “no” on the computer screen, the program thanked the participant for their response and no higher improvement was presented.

If the participant clicked “yes”, the computer application presented the scenario again, halving the difference between the two most extreme outcomes.

If undertaking pulmonary rehabilitation meant that in 8 weeks the distance you can walk in 6 minutes would be 600 metres (instead of 200 metres, without pulmonary rehabilitation), would you think the treatment was worthwhile?

The program would then continue to present the scenario, narrowing the participant's estimate by halving the range between the nearest acceptable and unacceptable outcomes, until the difference between the estimates was ≤5 m, when data collection ended, and the average of the two final acceptable and unacceptable estimates was stored by the application as the smallest worthwhile effect for that participant.

When this participant subsequently underwent the benefit–harm trade-off procedure for the water-based pulmonary rehabilitation scenario, the intervention was altered to the following.

…you would attend 1-hour classes twice per week at Prince of Wales Hospital. The exercises would be supervised by a physiotherapist and undertaken in warm water with other patients. The course of pulmonary rehabilitation would include education about management of breathlessness. You would also be prescribed land-based exercise to perform independently at home on the other days of the week. You would need to pay for transport and/or parking, but the program is otherwise free. There is a very small risk of injury and some risk that you may react to chlorine in the pool. You might feel some muscle soreness after doing the exercise.

Each participant underwent the benefit–harm trade-off procedure for each of the two scenarios in random order.

Where necessary, participants were encouraged to consider the nominated distance either in terms of the 6-min walk test course (“Instead of walking 4 laps of the course in 6 minutes as you did in your last test, what if pulmonary rehabilitation meant you could walk 7 laps in 6 minutes? Would this make pulmonary rehabilitation worthwhile?”) or in terms of the local landmarks (“Your walk test result means you could walk from here to the florist in front of the hospital in 6 minutes. If pulmonary rehabilitation meant you could instead walk to the convenience store down on the corner in 6 minutes, would this make pulmonary rehabilitation worthwhile?”).

Data analysis

The data generated by the benefit–harm trade-off method from multiple participants were used to generate estimates of the smallest worthwhile effect of land- and water-based pulmonary rehabilitation on 6MWD. The distributions of the estimates of smallest worthwhile effects were plotted as frequency histograms. A binomial method described by Mood and Graybill [26], which makes no distributional assumptions, was used to generate 95% confidence intervals for effects that are considered to be worthwhile by 20%, 50% and 80% of participants [27]. Our simulations indicated that, at least with normally distributed data, a sample size of 60 participants would provide expected CI widths of ≤0.4 standard deviations. However, as many participants were willing to undergo the procedures, we recruited 100 participants. The estimates of the smallest worthwhile effects of land- and water-based pulmonary rehabilitation on 6MWD were compared using a paired t-test. A test for linear correlation was performed between the baseline 6MWD and the smallest worthwhile effect for land-based pulmonary rehabilitation.

Results

Flow of participants through the study

100 participants were recruited: 49 at Prince of Wales Hospital and 51 at Royal Prince Alfred Hospital. All participants completed data collection for both land- and water-based pulmonary rehabilitation scenarios as randomly allocated. Complete data were obtained from all participants for both baseline characteristics and estimates of the smallest worthwhile effects for both scenarios. Baseline characteristics of the participants are presented in table 1.

TABLE 1

Baseline characteristics of study participants

Characteristic	Participants
Participants	100
Age years	72±9
Males/females	43/57
BMI kg·m−2	27 (6)
Pulmonary function
FEV1 L	1.34±0.51
FEV1 % predicted	54±16
FVC L	2.50±0.77
FVC % predicted	77±19
FEV1/FVC %	54±11
6MWD m	377±101
GOLD stage n
I	2
II	60
III	28
IV	10
SGRQ score#,¶
Total	45±17
Symptoms	56±20
Activity	65±20
Impact	31±19

Data are presented as n or mean±sd. All values are post-bronchodilator. BMI: body mass index; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity; 6MWD: 6-min walk distance; GOLD: Global Initiative for Chronic Obstructive Lung Disease; SGRQ: St George's Respiratory Questionnaire. #: on a scale from 0 to 100; ¶: 12 missing data points.

Estimates of the smallest worthwhile effect

The estimates generated for the two scenarios are summarised in table 2, with data from individual participants presented in figures 1 and 2. The distribution of the differences in smallest worthwhile effects of land-based and water-based pulmonary rehabilitation on 6MWD is presented in figure 3.

TABLE 2

Estimates of smallest worthwhile effects of pulmonary rehabilitation in people with chronic obstructive pulmonary disease

Percentile	Smallest worthwhile effect on 6MWD m
	Land-based	Water-based
80%	64 (50–80)	72 (51–100)
50%	20 (15–37)	26 (15–33)
20%	4 (3–6)	4 (4–7)

Data are presented as mean (95% CI). 6MWD: 6-min walk distance.

FIGURE 1

Distribution of the smallest worthwhile effects (SWEs) of land-based pulmonary rehabilitation on 6-min walk distance (in metres) nominated by people with chronic obstructive pulmonary disease (n=100).

FIGURE 2

Distribution of the smallest worthwhile effects (SWEs) of water-based pulmonary rehabilitation on 6-min walk distance (in metres) nominated by people with chronic obstructive pulmonary (n=98). Two outlying values (564 and 744 m) are not shown.

FIGURE 3

Distribution of the differences in smallest worthwhile effects (SWEs) of land-based and water-based pulmonary rehabilitation on 6-min walk distance (in metres) nominated by people with chronic obstructive pulmonary disease. Difference values are water-based SWE minus land-based SWE, so a value to the right of zero indicates that the participant required a greater improvement from pulmonary rehabilitation when the exercise component was water-based to feel that the benefit would outweigh the cost, risk and inconvenience (n=98). Two outlying values (525 and 584 m) are not shown.

Land-based pulmonary rehabilitation scenario

All participants generated a smallest worthwhile effect <1000 m for the land-based pulmonary rehabilitation scenario. Participants stated that they would need to see a median of 20 m (95% CI 15–37 m) improvement in 6MWD (in contrast to the lack of any change in 6MWD over 8 weeks anticipated without pulmonary rehabilitation), to make the costs, risks and inconvenience of land-based pulmonary rehabilitation worthwhile.

Water-based pulmonary rehabilitation scenario

Two participants stated that they would not participate in water-based pulmonary rehabilitation, even if it would achieve the maximum plausible benefit of 1000 m. Participants stated that they would need to see a median of 26 m (95% CI 15–33 m) improvement in 6MWD (in contrast to the lack of any change in 6MWD over 8 weeks anticipated without pulmonary rehabilitation), to make the costs, risks and inconvenience of water-based pulmonary rehabilitation worthwhile.

Difference between scenarios

A paired t-test indicated no significant difference in smallest worthwhile effect between the land- and water-based pulmonary rehabilitation scenarios (p=0.10). The differences between the smallest worthwhile effects for individual participants are presented in figure 3.

Correlation between baseline 6-min walk test and smallest worthwhile effect estimate

The test for linear correlation between the baseline 6MWD and the smallest worthwhile effect for land-based pulmonary rehabilitation showed no statistically significant correlation, with the baseline 6MWD accounting for only 3% of the variance in the smallest worthwhile effect.

Discussion

This is the first study using the benefit–harm trade-off method to determine the smallest worthwhile effect of the 6-min walk test following pulmonary rehabilitation by people with COPD. This study showed that, from a health consumer's perspective, there is no significant difference in the smallest worthwhile effect for the 6MWD between land-based pulmonary rehabilitation and water-based pulmonary rehabilitation.

The median estimates for 6MWD determined in this study are the same or very similar to two of the minimum clinically important difference estimates obtained with anchor-based or distribution-based methods [21, 22]. However, the previous estimates of the minimum clinically important difference span quite a wide range – from 25 m [21] to 80 m [19] – so the fact that the estimates determined in this study fall within this range is not surprising. This means that some previous studies may have overestimated the smallest worthwhile effect of pulmonary rehabilitation, when compared to values nominated by people with COPD [18, 19]. The results of our study also indicate that what people with COPD perceive as the smallest worthwhile effect of treatment on 6MWD is in fact very close to what recently published studies report as important gains from a pulmonary rehabilitation programme [21, 22]. The effect that is seen as worthwhile was not associated with the baseline 6MWD.

A substantial proportion of participants were satisfied with a very small improvement in 6MWD. This could be a “polite patient” effect, or alternatively, it might indicate that some participants do not consider the costs, risks and inconvenience of pulmonary rehabilitation to be very onerous. This behaviour has been observed in previous benefit–harm trade-off studies [15, 28] and might also represent the fact that patients often perceived additional benefits of intervention not captured by improvements in clinical outcomes. These may include the education component of pulmonary rehabilitation programmes, the opportunity to ask questions about their condition without the time pressure of a physician's appointment, or even the reassurance and encouragement gained during encounters with healthcare professionals and other patients. Therefore, patients may perceive the improvement in 6MWD as just one of many potential areas of benefit from pulmonary rehabilitation. If distance has little importance to patients, future research may need to employ outcomes measures seen as more relevant to patients such as quality of life, or the ability to stay out of hospital or to perform day-to-day tasks.

Most participants nominated similar smallest worthwhile effects for the land-based and water-based pulmonary rehabilitation scenarios. This suggests that, in the opinions of people with COPD, the costs, risks and inconvenience of water-based exercise training as part of pulmonary rehabilitation are not substantially greater than those of land-based exercise training. Given there is no significant difference between the two approaches in terms of effect on the 6MWD [12], the current findings support the conclusion that water-based exercise training could be offered as part of a pulmonary rehabilitation programme, particularly for people with physical comorbidities who find land-based exercise training difficult [12].

A recent Cochrane systematic review [29] has shown that, when compared to no treatment, water-based exercise training can improve the 6MWD by a mean of 62 m (95% CI 44–80 m). This indicates that water-based exercise training is clearly worthwhile in the opinion of the typical person with COPD. Indeed, both the point estimate of the treatment effect and the lower limit of its 95% confidence interval are greater than the median smallest worthwhile effect for water-based exercise training. Likewise, a recent Cochrane review on pulmonary rehabilitation involving land-based physical exercise components has shown that the programme can improve the 6MWD by a mean of 77 m (95% CI 12–143 m) when compared to conventional community care [30], with the point estimate of the effect substantially being greater than the smallest worthwhile effect.

A limitation of this study is that participants had moderate COPD with a low 6MWD. This limits the generalisability of the study findings to the broader group of people with COPD who have a higher level of function and more or less severe COPD.

Furthermore, people who had failed to complete pulmonary rehabilitation were not included as participants in this study. It is possible that this subgroup of people with COPD might feel that the costs, risks and inconvenience of land- or water-based pulmonary rehabilitation outweigh any possible benefits. However, this may not be true for all noncompleters of pulmonary rehabilitation as there are many reasons for drop-out, such as illness or transport issues [31].

Conclusion

The smallest worthwhile effect for the 6-min walk test in people with COPD is 20 m for land-based pulmonary rehabilitation and 26 m for water-based pulmonary rehabilitation. These estimates provide the most valid and interpretable estimates to date of the smallest worthwhile effect of the 6-min walk test following pulmonary rehabilitation.