Economic Evaluation Alongside a Randomized Controlled Crossover Trial of Modified Group Cognitive-Behavioral Therapy for Anxiety Compared to Treatment-as-Usual in Adults With Asperger Syndrome.

Background: There is a growing interest in using group cognitive-behavioral therapy (CBT) with people who have Asperger syndrome (AS) and comorbid mental health problems. This study aims to assess the cost-effectiveness of modified group CBT for adults with AS experiencing co-occurring anxiety compared to treatment-as-usual.
Methods: Economic evaluation alongside a pilot, multicenter, single-blind, randomized controlled crossover trial. Costs from the UK public sector (National Health Service and Social Services) and societal perspectives, quality-adjusted life years (QALYs), incremental net (monetary) benefit (INB), expected value of perfect information, expected value of sample information, expected net gain of sampling, and efficient sample size of a future trial are reported.
Results: Over 48 weeks, from the societal perspective, CBT results in additional costs of £6,647, with only a 0.015 incremental gain in QALYs, leading to a negative INB estimate of £6,206 and a 23% probability of cost-effectiveness at a threshold of £30,000/QALY. Results from sensitivity analyses support the unlikely cost-effectiveness of CBT but indicate the potential for cost-effectiveness over longer time horizons. Eliminating decision uncertainty is valued at £277 million, and the efficient sample size for a future trial is estimated at 1,200 participants per arm. Limitations: Relatively small sample size and prevalence of missing data present challenges to the interpretation of the results. Conclusions: Current evidence from this small pilot study suggests that, on average, modified group CBT is not cost-effective. However, there is much decision uncertainty so such a conclusion could be wrong. A large, full-scale trial to reduce uncertainty would be an efficient investment for the UK health economy.

RCT Entities:   Population Interventions Outcomes

Cognitive–behavioral therapy (CBT) that specifically targets anxiety symptoms in children with Asperger syndrome (AS) has been designed and shown to be effective,[1] but limited evidence exists concerning its effectiveness in adults.[2] Furthermore, there are no data on the cost-effectiveness of CBT in an adult AS population. To address these concerns, the People with Asperger Syndrome and Anxiety disorders (PAsSA) treatment trial was conducted: a pilot, multicenter, single-blind, randomized controlled crossover trial of modified group CBT compared to a wait-list control arm receiving treatment-as-usual (TAU) in adults with AS experiencing clinically significant anxiety.[3,4]

CBT offers a highly structured approach, where treatment goals and interventions are planned and undertaken in a gradual manner, making it suitable for AS patients who usually find it hard to tolerate surprise or uncertainty in their care. High rates of comorbid anxiety in AS patients are reported,[5-8] and due to the lack of evidence of (cost-) effective nonpharmacological interventions, patients may not be offered formal help or may be given suboptimal treatments, potentially increasing the burden on health and social services that care for AS patients.

Following good research practices for cost-effectiveness analysis alongside clinical trials,[9] the results of an economic evaluation based on the PAsSA trial are presented to help inform resource allocation decision making. The cost-effectiveness of modified group CBT compared to TAU in adult patients with AS experiencing clinically significant anxiety is estimated from both the perspective of the UK public sector (defined as National Health Service and Social Services; NHS-SS) and society (defined as NHS-SS, other public and voluntary services, out-of-pocket expenditure, and lost productivity) over a 24- and 48-week time horizon. The expected return on investment to UK society from a future full-scale trial is also determined.

Methods

Overview of the Trial

Full details of the PAsSA trial study protocol and its clinical findings are reported elsewhere.[3,4] Briefly, 52 adults from communities in the south and east of the United Kingdom, between 16 and 65 years of age, fulfilling diagnostic criteria for AS, high functioning autism, or pervasive development disorder–not otherwise specified, and in whom a diagnosis of anxiety was confirmed when assessing their eligibility for the trial were randomized to a CBT treatment arm or a wait-list TAU control arm and crossed over after 24 weeks of CBT treatment.

Participants in the treatment arm received three initial sessions of 1:1 CBT, followed by 21 group CBT sessions during the first 24 weeks of the trial. Upon crossover at week 24, participants initially randomized to TAU also received the CBT regimen during weeks 24 to 48 of the trial. All participants received TAU for the entire duration of the trial (i.e., both before and after crossover), comprising normal NHS access to primary care, which would have included medication, potentially counselling, and some were receiving mental health care in secondary care, excluding CBT (none of the participants reported having concurrent CBT, but some had received such treatment in the past). A favorable ethical opinion for the trial was obtained from the Cambridgeshire 4 NHS Research Ethics Committee (Reference: 10/H0305/42).

Overview of the Economic Evaluation

While crossover trials are valuable in encouraging enrolment into a trial, they are problematic for economic evaluations due to the lack of a washout period (indeed, it is hoped that the treatment effect will continue after the intervention period). Therefore, we analyze the results of CBT versus TAU over a 24-week within-trial period from both the perspectives of the UK NHS-SS and society. The cost-effectiveness over a 48-week (~1 year) time horizon is also explored through follow-up data observed in the CBT arm and extrapolation of TAU data. Results are presented using two generic preference-based measures of health-related quality of life (EQ-5D-3L and SF-6D) and using both a complete case analysis (i.e., all participants with incomplete data removed from the analysis) and a multiple imputation analysis, adjusted for baseline utility and costs.

To avoid misleading conclusions as a result of cost shifting between budgets rather than genuine changes in resource consumption, it is argued that the most appropriate analytic perspective should be that of society.[10] The National Institute for Health and Care Excellence (NICE) in England prefers health effects in adults are measured and valued using the EQ-5D and that time horizons are long enough to reflect all important differences in costs or outcomes between the interventions being compared.[11] Finally, accounting for baseline values takes into account any differences in patient characteristics at baseline[12] and multiple imputation avoids discarding informative data.[13] Therefore, the imputed and adjusted, 48-week, EQ-5D-3L cost-utility analysis conducted from the societal perspective was selected as the primary analysis. In addition, areas of uncertainty are explored and the value of future research to UK society is quantified to determine if a larger, definitive trial is worthwhile to reduce decision uncertainty.

Resource Use and Costs

Health (secondary and primary care services for both the patient and family members as well as medications) and social services (community care, respite, and voluntary organization services as well as day activities) use, patient/family out-of-pocket costs, informal care by family members, and the value of any lost productivity to society by carers were collected using an adapted version of the Client Socio-Demographic and Service Receipt Inventory.[14] Data collected at baseline pertained to the 3 months prior to randomization, which were then used as a predictor of future costs in the analysis. Resource data were divided into six categories: NHS, social services, other public services, voluntary/charity services, out-of-pocket expenses, and lost productivity. Costs involved in delivering the intervention (i.e., conducting the 1:1 and group-based sessions over the 24-week treatment period) were derived from the Unit Costs of Health and Social Care[15] and were based on the cost of three 55-minute 1:1 CBT sessions plus the cost per participant per 1-hour session for 22 sessions of group CBT assuming 12 people per session.

Cost per patient was calculated by multiplying unit costs identified from standard reference sources (Table S1) by resource quantities (Table S2). Inpatient hospital visits were costed using the National Schedule of Reference Costs.[16] Prescriptions were assigned appropriate prices from the British National Formulary.[17] Unit costs for other resource use counts were mainly derived from the Unit Costs of Health and Social Care.[15] All costs are expressed in 2014 British Pounds Sterling. No discounting was used in the 24-week within-trial or 48-week extrapolation analyses as the follow-up periods were less than 1 year. However, costs and quality-adjusted life years (QALYs) in the 48- to 72-week extrapolation period used in sensitivity analyses were discounted by 3.5% as per NICE guidelines.[11]

Utility Measurement and Valuation

Utility values were measured at baseline, 24 weeks, and 48 weeks. Although the EQ-5D-3L was identified as the primary utility measure for the economic evaluation (as per NICE guidelines),[11] for comparative purposes SF-6D utility values (derived from the SF-36 UK version 1)[18] were also collected on each patient over time for use in sensitivity analysis. Both have been shown to be valid for use in common mental health problems such as anxiety and show some evidence of responsiveness to change in health status.[19] The UK tariffs for both instruments were used to derive the utility values.[18,20] QALYs were calculated as the area under the utility curve for each patient.[21]

Imputation Model for Missing Data

Missing cost or quality-of-life data were estimated using multiple imputation (MI) implemented separately by treatment allocation.[13] Imputations were performed at the level of total costs and health state utility values for baseline and the two follow-up points. The use of log MI with predictive mean matching (PMM) has been shown to result in reliable results with large amounts of missing data and also performs well for data with a large amount of zero values.[22] Therefore, this was selected as the MI strategy for the total cost variables. Before log-transformation a constant was added to zeros in the data, which was subtracted again following imputation and re-transformation.

Health state utilities were not transformed prior to imputation and only the follow-up values required imputation using PMM. Covariates included in the imputation model were health state utility values for the EQ-5D-3L and SF-6D at 24 and 48 weeks; baseline, 24-, and 48-week NHS-SS and social costs; age; and highest education level. Five imputed data sets were obtained using chained equations (MI-MICE) in STATA/IC version 14.0 (see the statistical analyses section for more details). Five imputed data sets were chosen as even with 50% missing data, estimates from multiple imputation based on five imputations only have a standard deviation that is about 5% wider than one based on infinite imputations.[23] After imputation, the complete data were transformed back to their original scale prior to any analyses.

Extrapolation Methods

The PAsSA trial used a crossover design and therefore the longest any patient received CBT was 24 weeks. In order to determine the cost-effectiveness of a 24-week course of CBT over a 1-year time horizon, extrapolation of the within-trial analysis was required. A detailed schematic of the crossover and extrapolation methods is provided in Figure 1. For patients initially randomized to CBT, costs and QALYs from weeks 24 to 48 were added to the initial 24-week data to establish an approximate 1-year time horizon. Note that these patients only received CBT during the initial 24-week period. For patients initially randomized to TAU their costs and QALYs were assumed to be the same over weeks 24 to 48 as during the initial 24 weeks before crossover.

Figure 1

Schematic of the trial crossover and extrapolation methods. For patients initially randomized to CBT, costs and QALYs from weeks 24 to 48 were added to the initial 24-week data to establish an approximate 1-year time horizon. Note that patients only received CBT during the initial 24-week period. For patients initially randomized to TAU their costs and QALYs were assumed to be the same over weeks 24 to 48 as during the initial 24 weeks before crossover. Extrapolation of the time horizon to 72 weeks was also tested in sensitivity analyses by carrying forward the costs and QALYs from weeks 24 to 48 for the CBT arm for an additional 24 weeks. Costs and QALYs for TAU patients were assumed to be the same over weeks 24 to 48 and weeks 48 to 72 as during the initial 24 weeks before crossover. CBT = cognitive–behavioral therapy; QALY = quality-adjusted life years; TAU = treatment as usual.

The PAsSA trial protocol[3] proposed that trial data would be extrapolated to a 5-year time horizon. Due to the amount of missing data and resultant small sample size, it was deemed inappropriate to extrapolate any trial findings this far into the future. The extrapolation for the main analyses was therefore limited to only 48 weeks so that minimal assumptions could be made but at the same time the cost-effectiveness of CBT beyond the within-trial period (i.e., 24 weeks) could be explored. Extrapolation of the time horizon to 72 weeks was also tested in sensitivity analyses by carrying forward the costs and QALYs from weeks 24 to 48 for the CBT arm for an additional 24 weeks. Costs and QALYs for TAU patients were assumed to be the same over weeks 24 to 48 and weeks 48 to 72 as during the initial 24 weeks before crossover.

Statistical Analyses: Stochastic and Methodological Uncertainty

Mean differences in costs and QALYs were estimated for each of the five imputed data sets using ordinary least squares (OLS) regressions. The models accounted for both baseline costs and health state utility values,[12] and 1,000 bootstrapped samples were created for each of the five data sets. The five bootstrapped samples were then combined to estimate the means of the 5,000 mean differences in costs and QALYs, using the percentile method to estimate 95% confidence intervals (CIs). To avoid the difficulties associated with interpretation of confidence intervals for incremental cost-effectiveness ratios (ICERs) that cross quadrants of the cost-effectiveness plane,[24] incremental net (monetary) benefit (INB)[25] was chosen as the main outcome of the analysis. The willingness-to-pay (WTP) threshold for estimating the point estimates of INB was set at £30,000 as this is the upper limit set by NICE for cost-effective interventions.[11] Cost-effectiveness acceptability curves were calculated by determining the proportion of INB point estimates that were greater than zero from the bootstrapped samples for a range of WTP values.

To address methodological uncertainty a number of sensitivity analyses were conducted, including different cost perspectives (public sector [NHS-SS] vs. societal), different instruments for measuring health state utility values (EQ-5D-3L vs. SF-6D), extrapolation of the time horizon to 48 and 72 weeks, and combining cost and health state utility data for all patients receiving CBT regardless of the time period. Baseline to 24-week data of the group initially receiving CBT was combined with 24- to 48-week data of the group initially receiving TAU to leverage all possible information on participants receiving the intervention.

Value of Information Analyses

Per-person expected value of perfect information (EVPI) was estimated both parametrically[26] and nonparametrically[24] to confirm equivalence. If equivalent, this suggests that a parametric approach to estimating the expected value of sample information (EVSI) may yield a reasonable approximation and therefore save on processing time. Population EVPI was estimated using a prevalence of 0.98% of the UK adult population (of the 64.6 million people in the United Kingdom, 64% are aged 16–64 years)[27] being affected by an autism spectrum disorder (ASD),[28] 59% of which have experienced multiple anxiety disorders over their lifetime,[5-8] resulting in an affected population of 239,501. The time horizon for CBT to be a relevant intervention was assumed to be 5 years.

Separate from the main analyses, an OLS regression of net (monetary) benefit (NB)[25] was used to estimate parameters required for the calculation of the EVSI using a parametric approach. The regression was estimated for only the primary analysis (imputed/adjusted, 48-week, EQ-5D-3L cost-utility analysis conducted from the societal perspective). EVSI was estimated by first calculating the expected reduction in variance of mean INB from a trial of sample size ns per arm and then multiplying by the unit normal loss (i.e., the proportion of the INB distribution that is associated with negative values) and the potentially beneficial population (total eligible population less those enrolled in the study).[26] The expected net gain of sampling (ENGS) for a wide range of sample sizes (ns) was then estimated by subtracting the cost of sampling from the EVSI to determine the optimal ns that maximizes the ENGS. The cost of sampling was estimated based on the research costs incurred during the conduct of the PAsSA trial and categorized by fixed and variable costs (£218,375/trial and £966/participant respectively).

Results

Baseline Characteristics of the Study Participants

Fifty-two individuals, with a mean age of 33.1 years, SD 14.6, 54% women, were recruited and enrolled in the pilot trial. The two treatment arms (e.g., initial CBT for 24 weeks then crossover to TAU for 24 weeks and initial TAU for 24 weeks then crossover to CBT for 24 weeks) were well matched on IQ, age, and sex. Participants were predominately white British (96%), single (73%), and without children (77%). Most participants had at least a secondary education (96%), with 38% of participants holding a university degree. Further details of the baseline characteristics of the study participants are provided by Langdon and others.[4]

Point estimate analyses of resource use at both 24 and 48 weeks (Table S2) show very little difference for the majority of resource use between the CBT and TAU groups. As this pilot trial is limited in sample size, it is unclear if these reported differences are significant, but the relatively high unit cost of some of these resources (e.g., acute psychotic crisis admission—£376) could account for a large portion of the cost difference between the CBT and TAU groups. From the perspective of the NHS-SS, on average CBT is associated with an additional cost of £721 (SE £503) per patient at 24 weeks and £980 (SE £686) over 48 weeks. Imputed analyses yield similar results (£673 and £1,028 per patient at 24 and 48 weeks respectively, Table 1).

Table 1

Summary Costs by Type of Expense at 24 and 48 Weeks, Mean (SE) (2014 £)

Expense Category	CBT			TAU			24-Week Difference (CBT − TAU)	48-Week Difference (CBT − TAU)
	N (24, 48 Weeks)	24 Weeks	48 Weeks	N (24, 48 Weeks)	24 Weeks	48 Weeks
CBT intervention	(26, 26)	429.50 (0)	429.50 (0)	(26, 26)	0 (0)	0 (0)	429.50 (0)	429.50 (0)
Prescription medicines	(25, 25)	38.09 (20.98)	101.53 (40.26)	(25, 25)	47.30 (30.60)	94.60 (61.20)	−9.21 (37.10)	6.92 (73.26)
Primary care	(22, 19)	72.95 (29.78)	90.93 (29.83)	(21, 21)	97.10 (53.71)	194.21 (107.41)	−24.16 (60.67)	−103.28 (116.56)
Secondary care	(25, 25)	994.64 (675.67)	1271.32 (709.81)	(26, 26)	23.85 (11.28)	47.69 (22.57)	970.79 (662.38)	1223.63 (696.12)
Other health professionals	(22, 21)	29.20 (14.23)	55.38 (23.54)	(24, 24)	76.00 (63.91)	152.00 (127.81)	−46.80 (68.18)	−96.62 (138.59)
Social services	(22, 22)	36.66 (20.98)	57.38 (39.79)	(25, 25)	50.88 (28.17)	101.76 (56.34)	−14.22 (35.92)	−44.38 (70.75)
Other public services	(24, 24)	2.17 (2.17)	4.33 (4.33)	(24, 24)	3.25 (3.25)	6.50 (6.50)	−1.08 (3.91)	−2.17 (7.81)
Voluntary/charity services	(19, 18)	94.16 (78.46)	169.87 (157.00)	(23, 23)	264.75 (157.04)	529.50 (314.07)	−170.59 (187.19)	−359.63 (381.97)
OoP costs	(24, 24)	54.30 (40.23)	108.61 (80.45)	(22, 22)	9.60 (5.65)	19.20 (11.30)	44.71 (42.40)	89.41 (84.80)
Indirect costs	(21, 20)	12773.14 (5514.18)	18311.10 (6403.21)	(25, 25)	3738.70 (1327.96)	7477.40 (2655.92)	9034.44 (5247.63)	10833.70 (6439.61)
Total NHS-SS costs[a]	(19, 17)	1067.82 (466.07)	1672.70 (590.99)	(19, 19)	346.48 (188.56)	692.96 (377.12)	721.33 (502.77)	979.73 (686.19)
Total societal costs[a]	(12, 10)	8884.28 (4030.08)	15909.84 (6448.88)	(13, 13)	7272.38 (2362.56)	14544.76 (4725.11)	1611.90 (4583.11)	1365.07 (14544.76)
Imputed total NHS-SS costs[b]	(26, 26)	1021.65 (400.94)	1725.04 (473.39)	(26, 26)	348.28 (177.10)	696.56 (354.20)	673.38 (421.35)	1028.48 (581.28)
Imputed total societal costs[b]	(26, 26)	16617.28 (7416.79)	22260.45 (7700.91)	(26, 26)	5631.90 (1926.09)	11263.79 (3852.18)	10985.38 (7769.55)	10996.65 (8648.15)

Note: CBT = cognitive–behavioral therapy; NHS-SS = National Health Service and Social Services; OoP = out-of-pocket; SE = standard error; TAU = treatment as usual. NHS-SS costs include prescription medicine, primary, secondary, other health care, and social services costs. Societal costs include all NHS-SS costs, plus other public services, voluntary/charity services, out-of-pocket, and indirect costs.

Mean costs by expense category and totals across categories do not include multiple imputations for missing or incomplete resource use and cost information.

These total costs include multiple imputations (see methods section for more details of the imputation approach).

From the societal perspective, on average CBT was associated with an additional cost of £1,612 (SE £4,583) per patient at 24 weeks and £1,365 (SE £14,515) over 48 weeks (Table 1). When analyzing the imputed data, the additional per patient costs associated with CBT were much larger (£10,985 and £10,997 at 24 and 48 weeks, respectively). These costs are, however, derived from a small number of patients and therefore highly uncertain, illustrated by the associated large standard errors.

Quality of Life and Health State Utilities

Imputed, adjusted analyses yielded a loss of QALYs for CBT patients compared to TAU patients, with the exception of the EQ-5D-3L at 48 weeks, which resulted in a 0.015 gain (Table 2). EQ-5D-3L and SF-6D QALYs were very similar at 24 weeks, and the loss of SF-6D QALYs at 48 weeks was similar to the 24-week period (0.006; Table 2).

Table 2

Mean Costs, Quality-Adjusted Life Years, and Cost Utility

Analysis	N		Cost		QALYs		Unadjusted, Complete Case Analysis					Imputed, Adjusted for Baseline Utility and Cost
	CBT	TAU	CBT	TAU	CBT	TAU	Inc £ (95% CI)	Inc QALYs (95% CI)	INB[a] (95% CI)	P(CE)[a] (%)	PP-EVPI[a]	Inc £ (95% CI)	Inc QALYs (95% CI)	INB[a] (95% CI)	P(CE)[a] (%)	PP-EVPI[a]
NHS-SS, 24 weeks, EQ-5D	16	15	£1,159	£155	0.307	0.301	£1,004 (£345, £2,136)	0.006 (−0.064, 0.086)	−£833 (−3,236, £1,742)	24.50	£194	£568 (−£29, £1,364)	−0.008 (−0.036, 0.024)	−£814 (−£2,085, £399)	9.26	£27
NHS-SS, 24 weeks, SF-6D	16	16	£1,159	£154	0.296	0.301	£1,005 (£314, £2,185)	−0.005 (−0.036, 0.027)	−£1,137 (−£2,492, £75)	3.30	£8	£568 (−£29, £1,364)	−0.007 (−0.018, 0.004)	−£777 (−£1,625, −£123)	0.86	£1
Societal, 24 weeks, EQ-5D	9	9	£11,652	£6,402	0.300	0.278	£5,250 (−£3,964, £17,705)	0.022 (−0.077, 0.118)	−£4,764 (−£17,004, £5,348)	22.40	£608	£7,501 (−£3,906, £23,171)	−0.008 (−0.036, 0.024)	−£7,747 (−£23,554, £3,710)	12.68	£292
Societal, 24 weeks, SF-6D	9	10	£11,652	£7,449	0.294	0.299	£4,203 (−£5,841, £15,030)	−0.004 (−0.042, 0.032)	−£4,291 (−£15,314, £5,825)	22.60	£644	£7,501 (−£3,906, £23,171)	−0.007 (−0.018, 0.004)	−£7,709 (−£23,280, £3,627)	12.78	£284
NHS-SS, 48 weeks, EQ-5D	13	15	£1,891	£311	0.650	0.601	£1,581 (£381, £3,230)	0.049 (−0.099, 0.191)	−£119 (−£5,192, £4,655)	49.70	£941	£864 (−£131, £1,915)	0.015 (−0.048, 0.085)	−£423 (−£2,816, £2,005)	36.00	£303
NHS-SS, 48 weeks, SF-6D	13	16	£1,891	£308	0.613	0.602	£1,583 (£353, £3,381)	0.011 (−0.059, 0.081)	−£1,196 (−£3,921, £1,389)	19.00	£129	£864 (−£131, £1,915)	−0.006 (−0.033, 0.020)	−£1,056 (−£2,300, £161)	4.56	£13
Societal, 48 weeks, EQ-5D	6	9	£18,133	£12,803	0.656	0.555	£5,330 (−£10,897, £24,986)	0.101 (−0.069, 0.274)	−£2,275 (−£21,525, £15,146)	41.40	£2,585	£6,647 (−£7,290, £23,689)	0.015 (−0.048, 0.085)	−£6,206 (−£23,802, £8,209)	22.98	£903
Societal, 48 weeks, SF-6D	6	10	£18,133	£14,897	0.623	0.597	£3,236 (−£13,256, £22,058)	0.025 (−0.057, 0.114)	−£2,254 (−£22,362, £14,606)	42.20	£2,783	£6,647 (−£7,290, £23,689)	−0.006 (−0.033, 0.020)	−£6,839 (−£23,829, £6,896)	19.32	£700

Note: CBT = cognitive–behavioral therapy; CI = confidence interval; INB = incremental net (monetary) benefit; Inc = incremental; NHS-SS = National Health Service and Social Services cost perspective; P(CE) = probability that CBT is cost-effective at a willingness-to-pay threshold of £30,000/QALY; PP-EVPI = per-person–expected value of perfect information; QALY = quality-adjusted life years; Societal = societal cost perspective; TAU = treatment as usual.

Calculated using a willingness-to-pay threshold of £30,000.

Cost Utility and Stochastic Uncertainty

In all analyses, the 95% CIs for INB did not exclude zero, with the exception of the SF-6D, 24-week cost-utility analysis from the NHS-SS perspective, which had a negative 95% CI (Table 2). For all complete case analyses (CCAs) and imputed/adjusted analyses INB was consistently negative.

The probability of CBT being more cost-effective than TAU at a WTP of £30,000 per QALY gained ranged from 3% to 25% at 24 weeks and from 19% to 50% over 48 weeks. These probabilities were smaller for the imputed/adjusted analyses ranging from 1% to 13% at 24 weeks and 5% to 36% over 48 weeks. In both complete-case and imputed/adjusted analyses, the 48-week, EQ-5D-3L cost-utility analyses from the NHS-SS perspective had the highest probability of CBT being more cost-effective than TAU (Table 2). Figure 2 highlights that the probability of CBT being cost-effective remains relatively constant from the societal perspective across different WTP thresholds. In contrast, the probability of CBT being cost-effective increases with larger WTP thresholds from the NHS-SS perspective but remains below 50%, with the exception of the CCA at 48 weeks when using the EQ-5D-3L and WTP thresholds greater than £30,000 per QALY gained.

Figure 2

Cost-effectiveness acceptability curves. The cost-effectiveness acceptability curve represents the probability that CBT is cost-effective given a threshold of X. There is between 3.00% to 49.70% and 0.86% to 36.00% probability that the ICER is below £30,000, depending on the analytic perspective for the complete case and imputed/adjusted analyses respectively. NHS-SS = National Health Service and Social Services; QALY = quality-adjusted life year.

Methodological Uncertainty

When combining cost and health state utility data over weeks 24 to 48 from patients who crossed over to CBT at 24 weeks with data over the first 24 weeks from patients who initially received CBT, the two analyses from the NHS-SS perspective were consistent with the original within-trial results. The results of the two societal analyses were, however, reported to be more favorable with positive INB estimates in the CCAs, but remained negative in the imputed/adjusted analyses. This change is the result of a decrease in the social costs associated with CBT when using a combined sample treatment arm (£11,652 vs. £6,157-£6,406; Table S3).

Overall, the results from the 72-week extrapolation were slightly more favorable in terms of cost-effectiveness (Table S4). INB became positive for the EQ-5D-3L analyses from the NHS-SS perspective in both complete case and imputed/adjusted analyses. INB also became positive for the two analyses from the societal perspective for the CCAs, but remained negative in the imputed/adjusted analyses. The probability of CBT being cost-effective at a WTP of £30,000 also increased for all analyses ranging from 25% to 58% and 8% to 49% for the complete case and imputed/adjusted analyses, respectively.

Value of Future Research

Per-person EVPI estimates calculated parametrically and nonparametrically were confirmed to be equivalent and are reported in Table 2. Relatively large estimates of value in eliminating decision uncertainty are observed once converting the per-person EVPI estimates to population values. Population EVPI estimates for the CCAs ranged from £3.1 million to £415 million for the 24-week period and increased for the 48-week period, ranging from £4.9 million to £855 million. Population EVPI estimates decreased for the majority of the imputed/adjusted analyses, ranging from £614,369 to £277 million.

For the 48-week, EQ-5D-3L, cost-utility analysis from the societal perspective estimates of EVSI were quite large, even at small sample sizes (Figure 3). EVSI reached a maximum of £192,787,534 at a sample size of 2,500, and remained relatively constant for all larger sample sizes up to 5,000. The total cost of sampling was much smaller than estimates of EVSI and increased slightly for larger sample sizes. The ENGS was therefore similar to the estimates of EVSI and reached a maximum (£181,068,595) at a sample size of 1,200 per arm. Any sample size greater than 10 was associated with a positive return on investment.

Figure 3

The most efficient sample size for a future trial. The optimal sample size per arm of a future trial is derived by first determining the expected value of sample information (EVSI) for a range of sample sizes per arm and the associated cost of sampling. The difference between these values is the expected net gain of sampling (ENGS). The sample size that maximizes the ENGS is the most efficient sample size for the conduct of a future trial. ENGSn = expected net gain of sampling for a given sample size n per arm; EVSIn = expected value of sample information for a given sample size n per arm; TCn = total cost of sampling for a given sample size n per arm.

Discussion

Interpretation of Results

In the primary analysis CBT results in additional costs of £6,647, with only 0.015 gain in QALYs, leading to a negative INB estimate of £6,206 at a WTP of £30,000 per QALY gained. From these results, CBT is unlikely to be considered a cost-effective treatment, but analyses of uncertainty suggest a 23% probability of cost-effectiveness and population EVPI of £277 million over a 5-year time horizon at a WTP threshold of £30,000 per QALY gained, indicating that future research could be efficient to confirm the cost-effectiveness of CBT. The most efficient sample size of a confirmatory trial is likely to involve 1,200 participants per arm, but almost any positive sample size has a positive return on investment. The results of a number of additional analyses representing different analytic perspectives (NHS-SS and societal), different instruments for deriving health state utility values (EQ-5D-3L and SF-6D), time horizons (24, 48, and 72 weeks), and alternative approaches for handling missing data also support the unlikely cost-effectiveness of CBT, but indicate the potential for cost-effectiveness over longer time horizons.

Policy Implications: Intervention Adoption and Future Research Design

This pilot study was intended to inform the design and potential efficiency of a future full-scale randomized controlled trial (RCT) and so was not powered to detect significant differences in outcomes. It is therefore unsurprising that the outcomes analysis did not detect such differences in participants’ anxiety levels,[4] and it would be unwise to base a definitive policy decision on these data. The INB point-estimate from the economic evaluation conducted alongside the PAsSA trial indicates that CBT is not cost-effective from the societal perspective and resulted in additional costs (£6,647) with only a small gain in QALYs (0.015). Despite this, the majority of participants had positive views of the intervention with just over half agreeing or strongly agreeing that CBT improved their anxiety and 73% agreeing or strongly agreeing that they would recommend therapy to others and that it was helpful. This apparent paradox is likely due to random noise from the small sample size. It should, however, also be noted that our scenario analyses suggest a potential improvement in cost-effectiveness with a longer (72 weeks) time horizon of analysis, especially for the EQ-5D-3L cost-utility analysis from the NHS-SS perspective.

The relatively small sample size and prevalence of missing data present further challenges to the interpretation of the results. Due to the multidimensional nature of data required for economic evaluation, missing data are frequently an issue. Our selected most appropriate analysis was derived from only 15 complete case participants (6 CBT; 9 TAU) with the remaining 37 having one or more components of data missing for costs and/or health state utility values (Tables S5 and S6). Confidence intervals in such scenarios will be very wide, potentially rendering the point estimate INB uninformative. This is reflected in the “large” EVPI, suggesting that future research could be a worthwhile investment, but all effort should be made to maximize the completeness of data that are collected.

Our analysis has indicated that the most efficient sample size of a full-scale RCT would be 1,200 participants per arm, when ENGS reached a maximum of £181,068,959. A future trial of this nature would be quite large, possibly impractical to conduct and traditional power calculations might indicate much smaller sample sizes for a reasonably power study. It should therefore be noted that even for small sample sizes (e.g., 50 participants per arm), although not the most efficient, it would still result in relatively large ENGS estimates (£73,522,457). This signifies that investment in full-scale trials much smaller than the most efficient sample size identified in our analysis would still be worthwhile.

Comparison With Other Studies

There is limited evidence concerning the cost-effectiveness of CBT in adults with AS experiencing clinically significant anxiety, but there are two studies that provide some evidence of the cost-effectiveness of CBT in related populations.[29,30] van Steensel and others reported the cost-effectiveness of individual CBT versus TAU for anxiety disorders in children with ASD from a societal perspective in the Netherlands.[29] The study used a quasi-randomized design to assign 49 children to either 15 sessions of CBT or TAU with 6 months of follow-up. Costs and QALYs were not statistically different between the two interventions, although the point estimate ICER suggested CBT dominated TAU. In contrast, the point estimates of INB from the societal perspective in our study were relatively unfavorable for similar follow-up periods (i.e., 24 weeks; Table 2, rows 3 and 4), as well as for longer follow-up periods (48 weeks; Table 2, rows 7 and 8).

More recently, Visser and others developed a probabilistic decision-analytic Markov model to estimate the cost-effectiveness of group CBT compared with a wait-list control for adult patients with unexplained physical symptoms from a societal perspective in the Netherlands over a 4-year time horizon.[30] RCT data from 162 patients were limited to 3 months posttreatment but the patients receiving group CBT were also followed to 6 months and 1 year in an uncontrolled phase. At 18 months group CBT passed a threshold of €30,000/QALY; at 33 months it was dominant and remained dominant at 4 years. This indicates that longer time horizons may be required for CBT to be cost-effective in our analysis, which is apparent from the improved estimates of cost-effectiveness in our scenario analyses using a 72-week time horizon (Table S4).

Study Limitations

The analyses in our study were based on data from a pilot RCT with crossover and were therefore limited by a small sample size and limited follow-up period. Due to the absence of prior data from the literature concerning the effectiveness of CBT in an AS population,[2] it was not possible incorporate additional sources of evidence into our analysis and hence it was based solely on data from a pilot RCT. This potentially complicates the interpretation of the reported value of information estimates as these figures are based on the assumptions that the pilot study population would be the same as the population to be included in a larger trial as well as the same as the population for which we would want to make treatment decisions in clinical practice.[31] Furthermore, it is well known that pilot studies may not be designed and conducted with the same rigor as larger RCTs, potentially biasing the results.[32] Large amounts of missing data were also observed for total social costs (27% to 42% of patients missing total social costs depending on time point and treatment allocation; Tables S5 and S6), which are likely to be an important factor in estimating the cost-effectiveness from the most appropriate perspective. To account for these shortcomings, both complete-case and imputed/adjusted analyses have been presented, with the results remaining relatively consistent but associated with considerable uncertainty.

Despite these weaknesses the analyses presented are based on good research practices for cost-effectiveness analyses alongside clinical trials[9] and conform to the good reporting practices for economic evaluations.[33] Due to the pilot nature of the RCT data used in the analyses and therefore the considerable uncertainty associated with the results, a number of sensitivity analyses have been conducted, all of which point to the general conclusions of the unlikely cost-effectiveness of CBT over short time horizons, but the potential for cost-effectiveness over longer time horizons from the NHS-SS perspective and to a lesser extent the societal perspective. Furthermore, the value in reducing decision uncertainty and the most appropriate sample size for the conduct of a full-scale trial has also been quantified. We have highlighted that there may be value in reducing decision uncertainty through the conduct of a larger trial with longer term follow-up.