Does the use of patient decision aids lead to cost savings? a systematic review.

OBJECTIVES: To update a previous systematic review to determine if patient decision aid (PDA) interventions generate savings in healthcare settings, and if so, from which perspective (ie, patient, organisation providing care, society).
DESIGN: Systematic review. DATA SOURCES: MEDLINE, CINAHL, PsycINFO, Web of Science, Cochrane Library, Embase, Campbell Collaboration Library, EconLit, Business Source Complete, Centre for Reviews and Dissemination: NHS Economic Evaluations Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA) from 15 March 2013 to 25 January 2019. The references of studies that met the eligibility criteria and any publications related to conference abstracts or registered clinical trials were reviewed to increase the sensitivity of the search. ELIGIBILITY CRITERIA: Full and partial economic evaluations with an experimental, quasi-experimental or randomised controlled design were included. The intervention had to satisfy the pre-determined minimum conditions necessary to be defined as a PDA, and (for full evaluations) provide details on the comparator used. DATA EXTRACTION AND SYNTHESIS: All study outcomes and economic data were extracted. The reporting and quality of the economic analyses were independently assessed by two health economists.
RESULTS: Of 5066 studies, 22 studies were included, including the 8 studies from the previous review. Twelve studies reported cost-savings (range=US$10 to US$81 156; US dollars in 2020), primarily from the organisational or health system perspective, and 10 studies did not. However, due to the quality of the economic analyses, and the related issues with the interpretative validity of results it would be inappropriate to say that PDAs will generate savings, from any perspective.
CONCLUSIONS: It is unclear whether PDAs will generate savings. Greater consensus on what constitutes a PDA and the need to compare them against usual care over a sufficient time horizon to allow valid assessment of costs and outcomes is required. PROSPERO REGISTRATION NUMBER: CRD42019118457. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A strength of this study was the adherence to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting and best practice guidelines for conducting a systematic review.

We employed a very broad inclusion criteria to provide more insight into the challenges of undertaking economic evaluations of patient decision aids.

Two health economists independently assessed the reporting and quality of the economic analyses using two checklists and evaluated the interpretative validity of results.

A weakness of our systematic review is that the intervention descriptions often lacked sufficient detail, so it was difficult to determine if they met the definition of a patient decision aid.

The heterogeneity of the methods and results did not enable a meta-analysis.

Introduction

The use of patient decision aids (PDAs) tends to shift patients’ preference towards either non-surgical interventions or more towards risk-averse treatment options and this has led to the presumption that these tools lead to cost-savings.1 We define ‘cost-saving’ as a positive net monetary benefit to an alternative. A previous systematic review found that the evidence to support that claim was weak: half of the eight economic analyses included found that PDAs generated significant savings, but they were considered to be of low or moderate quality.2 Since then, PDA research has increasingly focussed on the implementation of these tools in clinic workflow, offering more opportunities to study their effects on cost from various perspectives.3–5 What conclusions can we draw from reviewing the totality of the evidence regarding the effects of PDAs on cost in healthcare settings?

PDAs provide evidence-based information in a comparative format to help patients make decisions that align with their preferences.1 Randomised trials have shown that these tools have increase knowledge and awareness of treatment options, engagement in the decision-making process, improve risk perception and reduce decisional conflict.1 A couple of systematic reviews have focussed on the cost-effectiveness of PDAs in clinical practice.6 7 In 2014, Trenaman et al7 found considerable variation regarding the costs of administering PDAs. The review concludes that encouraging PDA implementation to reduce spending is ‘inappropriate’ considering that short-term costs incurred may actually be higher when using a tool, with only one randomised trial providing evidence for cost-savings beyond 1 year.7 This evidence echoes Légaré et al6 which found that there was insufficient information (or uncertainty) to make any claims regarding costs.

Assessing whether or not PDAs generate savings is challenging not just due to the lack of methodological rigour (eg, insufficient length of follow-up which is typically 2 years) or high risk of bias in studies, but because there is no consensus on an economic evaluative framework.8 Butt8 argues that the absence of an established framework and disagreement on how researchers attribute value (monetary or otherwise) to PDAs has left us unable to draw any meaningful conclusions.

Since our previous systematic review, more economic analyses of PDAs have been conducted which may potentially provide insight into whether or not these tools generate savings—a question that is increasingly relevant for policymakers and organisations who wish to implement these tools to improve healthcare communication and delivery. Our aim in this study was to update the previous systematic review2 to determine if PDA interventions generate savings in healthcare settings, and if so, from which perspective (ie, patient, organisation providing care, society).

Methods

We updated a previous systematic review2 according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (online supplemental material table A) - PROSPERO - registration # CRD42019118457, 21 January 2019.

Inclusion criteria

We employed the population, intervention, control, outcomes, study design (PICOS) criteria to assess study eligibility with no restrictions.9 To be eligible for inclusion, studies had to:

meet the following definition of decision support intervention—a definition that is synonymous with PDA: tools that “help people make specific and deliberative choices among options (including the status quo) by providing (at minimum) information on the options and outcomes relevant to a person’s health status and implicit methods to clarify values”. We expanded the intervention criteria to include tools that provide personalised patient probabilities of an event or outcome (eg, the risk of stroke or death for an individual based on their family history, age and so on). Expanding the inclusion criteria enables us to provide additional insight into the challenges of undertaking economic evaluations of patient decisions, as reported by Ara et al10 as part of the Policy Research Unit in Economic Evaluation of Health and Care Interventions (EEPRU) 2015 report. The EEPRU report undertook research to develop a framework to evaluate the economics associated with the use of PDAs.10 They limited their review only to those studies which assessed both the costs and benefits associated with any shared decision-making process involving PDAs in any indication or setting (ie, limiting the search to only full economic evaluations).10 The emergence of both physiological and preference-based personalised healthcare has questioned whether this conventional economic evaluation framework is sufficient to capture a range of non-health benefits and process outcomes which are emerging as key drivers of ‘value’. Given that the true ‘value’ of PDAs most likely extends beyond this conventional paradigm, we employed a broad definition to capture not only full evaluations, but also partial evaluations, and other studies which have measured the resource implications of involving patients in shared decision making;10

include a control group such as usual care, the absence of a PDA that meets the above definition, or an alternative PDA;

meet our expanded study design criteria to include randomised controlled trials (RCTs), economic evaluations, observational and experimental or quasi-experimental designs which contained a control group, partial economic evaluations where patients may have acted as their own control (pre–post studies) or where the authors appealed to evidence of ‘no difference in effect’ and looked at differences in cost between a control and intervention group, and economic studies that used trial data (eg, an economic evaluation conducted alongside an RCT);

specify the primary secondary outcomes.

We excluded Markov models and economic models that used computer-simulated data, hypothetical data or data estimates based on expert opinion because they often lack transparency, or the quality of such data may be open to debate. Including these models may have also led to ‘double-counting’ if the data used to populate the model is derived from a trial that is already included in our review.

Data sources and search strategy

Two information scientists (PJB and HBB) updated the previous systematic review2 search strategy and adapted it for the following databases: MEDLINE, CINAHL, PsycINFO, Web of Science, Cochrane Library, Embase, Campbell Collaboration Library, EconLit, Business Source Complete, Centre for Reviews and Dissemination: NHS Economic Evaluations Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA). The previous review searched databases from their inception to 15 March 2013. Therefore, in this update, the search dates covered 15 March 2013 to 25 January 2019 and identified studies related to: decision support interventions/PDAs and cost benefit (online supplemental material table B). We did not impose language restrictions. We reviewed the references of included studies and also searched for related conference abstracts or registered clinical trials.

Study selection

After removing duplicate study titles, PS reviewed the titles and abstracts of identified studies. A second author (PJB) reviewed 10% of randomly selected titles and abstracts to increase the rigour of the study selection process. Uncertainty or disagreement on study selection for full-text review were resolved by a third reviewer (GE). Full-text review of selected studies was conducted by PS and GE, and disagreements resolved by a health economist (CON).

Data extraction

Data were extracted into a standard case report form that included: author and year of publication, study design, location and setting, description of patient population, study sample size, sample demographics, data collection period, intervention description (including timing and mode of delivery), type and perspective of economic analysis (societal, healthcare system or organisation, patient, clinician), time horizon, estimated cost/resource use to implement the PDA, study-specified primary and secondary outcomes and estimated savings.

Assessment of study quality

A number of quality assessment tools were used to assess various study designs. We used the Cochrane ‘risk of bias’ tool to assess the quality of the included RCTs (PS and GE).11 For every one of the seven domains we judged high, low or unclear risk of bias.11 Disagreements were resolved by PJB.

We used the National Heart, Lung and Blood Institute (NHLBI) scoring checklist12 to assess the quality of the pre–post studies. The NHLBI is a 12-item questionnaire with a response format of yes (1), no (−1) or can’t tell (0). A score between −12 and −4 indicates the study quality is poor, −3 to 4 is considered fair quality and >5 represents a good quality study.12 The Newcastle-Ottawa Quality Assessment scale13 was used to evaluate observational studies and contains three domains judged to be poor, fair or good quality: the selection of the study groups, the comparability of the groups and the ascertainment of either the exposure or outcome of interest.

Two health economists (CON and GEC) independently used the Drummond checklist and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist to assess the quality of the economic evaluations.14 15 The Drummond checklist is 10 items and indicates the quality of the economic evaluation,14 and the CHEERS’ 24 items assesses the quality of how the economic evaluation was reported.15 Use of the two checklists was recommended by our health economists to ensure a comprehensive assessment of both the quality and reporting of the economic evaluations in our review. In particular, the CHEERS checklist enabled us to identify the components of the economic evaluation that were missing. For the Drummond checklist, reviewers gave a rating of ‘yes’ (1), ‘cannot tell’ (0.5) or ‘no’ (0).14 Higher ratings (maximum of 10) indicates higher quality.14 Similarly, for the CHEERS checklist, reviewers provided a score of ‘1’ if present, ‘0’ if absent or in some cases, ‘not applicable’.15 Reviewers (CON and GEC) resolved any disagreements by discussion.

Analysis

Due to the heterogeneous nature of the reporting of economic data, we examined various indicators such as, but not limited to, the mean PDA cost per user (from any perspective), the incremental cost-effectiveness ratio or the incremental net monetary benefit. Currency data were converted to US dollars on 30 August 2019. We did not conduct a meta-analysis due to the heterogeneity of both research methods and economic analyses.

Patient and public involvement

This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Results

Characteristics of included studies

Our search identified 5066 studies, with 3539 remaining after the removal of duplicates. After review of their titles and abstracts, we identified 72 articles for full-text assessment; 58 did not meet the inclusion criteria. We therefore included 14 articles in addition to the 8 articles from the previous review2 for a total of 22 studies.16–37 See figure 1.

Figure 1

Flow diagram outlining the study selection process.

Of the 22 included studies, 16 were RCTs, 5 pre–post studies and 1 retrospective cohort study. Over 100 000 participants were recruited across the studies, covering the following health conditions: breast cancer, hip and knee osteoarthritis, menorrhagia, benign prostatic hyperplasia (BPH), chest pain, cardiovascular disease, back pain, lumbar spinal stenosis, perimenopausal women, couples waiting for in vitro fertilisation, women with previous caesarean operations, residents in a geriatric health facility, decisions regarding mechanical ventilation and about the MMR (measles, mumps and rubella) vaccine acceptance. The format of the PDAs included paper (n=4), web-based applications (n=6) or the tool was embedded as a component of a larger intervention (ie, coaching, telephone call, DVDs, interviews) (n=11). Most studies (n=12) were conducted in the USA, four from the UK, two in the Netherlands and one study from each of the following countries: Australia, Canada, Finland and Japan. See table 1.

Table 1

Characteristics of studies included in the review of patient decision aids

Study	Population	Sample size	Setting	Study design	Mode and timing of delivery	Description of the intervention	Outcomes*
Kennedy (2002), USA16	Women with menorrhagia	894	Recruited from six hospitals	Three arm RCT	Mode: Videotape and bookletsTiming: 6 weeks pre-consultation interview with nurse immediately pre-consultation with specialist	The 28-page booklet included chapters describing menorrhagia and its causes, investigations, treatment options (medical and surgical), and the benefits and risks of surgery. It also included a section in which the reader was prompted to write down her preferences in response to a series of questions. The 30 min video included clips of interviews with women who had experienced different treatments for menorrhagia. Also, a structured interview to enable patients to clarify their values	↑ Role Physical of the health status measure∅ Reported treatment (except for hysterectomy rate which was lower for the intervention group)↑ Satisfaction with decision making involvement
Wennberg (2010), USA17	Patients with preference sensitive conditions	18 351	Not applicable.	RCT	Mode: Telephone calls from health coaches, booklets, and videosTiming: Health coaches contact patients who have been discharged from the hospital in order to review, explain and reinforce discharge instructions.	A team of health coaches delivered the intervention. Health coaches were trained to give study participants knowledge and awareness of their treatment options, engage them in discussions to help them sort out their treatment preferences, and encourage them to communicate those preferences to their healthcare providers.	↓ Hospital admission rate↓ Surgical procedures
Van Peperstraten (2010), Netherlands18	Couples on waiting list for in vitro fertilisation treatment	308	Five IV clinics in the Netherlands	RCT	Mode: Booklet and in-person interviewTiming: 3 elements (including PDA) pre-consultation +1 phone call post consultation	Standard in vitro fertilisation care, including a session in which the number of embryos transferred was discussed. In addition, couples received a multifaceted empowerment strategy by post. Couples were sent a PDA about the number of embryos transferred. The couples also received the offer of reimbursement of an additional fourth cycle. The content of the PDA and the reimbursement offer were discussed in person with a trained in vitro fertilisation nurse	↑ Chose single embryo transfer∅ Ongoing pregnancies after the second cycle∅ Twin pregnancies after the second cycle↑ Knowledge↑Empowerment∅ Decisional conflict∅ Anxiety↑ Subclinical depression
Arterburn (2012), USA19	Patients with hip osteoarthritis	1788	Group Health Service Line includes 27 staff surgeons, 15 physician assistants in 5 practices within western Washington State	Before–after observation	Mode: DVD and booklets; onlineTiming: Prior to speciality consultation	Orthopaedic providers were instructed to order a decision aid for every patient with knee or hip osteoarthritis seen in their practice, regardless of disease severity.	↓ Rate of surgery
	Patients with knee osteoarthritis	7727
Cox (2012), USA20	Self-identified as being most involved in medical decision making for each patient mechanically ventilated for ≥10 days	111	Three intensive care units: Duke University, Durham Regional Hospital and the University ofNorth Carolina	Prospective, pre–post study	Mode: PaperTiming: Unclear	The main decision about prolonged mechanical ventilation was presented as a continuum of options ranging from maximising life prolongation to maximising comfort. It is a printed version that was 10 pages in length, written at a sixth grade reading level and made generous use of simple diagrams to illustrate key points	↓ Physician-surrogate discordance↑ Quality of communication↑ Medical comprehension↓ Decisional conflict∅ Trust∅ Ventilator days, ICU and hospital days
Veroff (2013), USA21	Patients with preference sensitive conditions	60 185	Not applicable.	Subanalysis of data from Wennberg et al (2010)17	Mode: Telephone calls from health coaches, booklets and videosTiming: Health coaches contact patients who have been discharged from the hospital in order to review, explain, and reinforce discharge instructions.	A team of health coaches delivered the intervention. Health coaches were trained to give study participants knowledge and awareness of their treatment options, engage them in discussions to help them sort out their treatment preferences and encourage them to communicate those preferences to their healthcare providers.	↓ Hospital admission rate∅ Emergency department visits∅ Surgeries and imaging
Wilson (2013), USA22	Women with breast cancer	68	The Cancer Resource Centres of Mendocino County	Cost-benefit analysis using trial results from Belkora et al (2012) RCT	Mode: Telephone or in-personTiming: Pre-consultation	The visit preparation consisted of two components: consultation planning (CP) and a consultation visit recording and summary (RS). The entire intervention is referred to as CPRS. In CP, trained facilitators (CPRSers) elicited questions and concerns from patient’s pre-consultation. A written ‘consultation plan’ was then provided to the patient as a visual aid for their upcoming appointment. For the RS portion of the intervention, the CPRSers accompanied the patient to the appointment and created an audio-recording of the visit.	↑ Self-efficacy∅ Anxiety∅ Satisfaction with the intervention∅ Preparation for decision-making
Keyserling (2014), USA23	Patients with no known cardiovascular disease, and at moderate-to-high risk for CHD	385	Five diverse family medicine practices in North Carolina	RCT	Mode: Web-basedTiming: 4, sessions at monthly intervals, followed by three maintenance sessions delivered at 2 month intervals.	The PDA (1) calculated participants’ 10 year FRS, (2) educated participants about their CHD risk factors and the pros and cons of risk-reducing strategies, and (3) showed participants how much their CHD risk might be reduced by one or more of the following: changes in diet, increased physical activity, smoking cessation, initiation of aspirin (for men only) or initiation or intensification of treatment with statins or hypertension medication.	↓ Framingham Risk Score↓ Weight loss↓ A1c level (in the counsellor group)↑ Appropriate use of and adherence to medication↑ Increase physical component measure of quality of life
Tubeuf (2014), UK24	First-time parents whose first child was offered the first MMR vaccine dose	203	Urban general practices in the North of England	Three arm RCT	Mode: Web-basedTiming: Pre-consultation via the post	MMR decision aid plus usual practice. Parents were sent a web link for the MMR decision aids and log-in instructions by post.	↓ Decisional conflict↑ Vaccine uptake↓ Resource utilisation
Volandes (2016), USA25	Adult inpatients with late stage disease	3119	Hilo Medical Centre in Hawaii	Pre–post	Mode: VideoTiming: Unclear	A single, 1- to 4-hour training and access to the advanced care planning (ACP) video decision aids. The videos attempt to provide a general framework in which to understand ACP including the broad questions that patients should reflect on and how individual preferences can be translated into actionable medical orders and interventions.	↑ ACP documentation↑ Rate of discharge to hospice∅ Death rate
Trenaman (2017), Canada26	Patients with moderate or severe hip or knee radiographic osteoarthritis	343	Two orthopaedic screening clinics in the Ottawa area	RCT	Mode: Video+booklet, and a one-page surgeon preference reportTiming: At home, pre-consultation	Standard patient education, a PDA (treatment choices for hip osteoarthritis and Treatment choices for knee osteoarthritis) and a preference report for the surgeon.	∅ Wait time↑ Decision quality↑ Realistic expectation of outcomes↑ Knowledge∅ Decisional conflict∅ Surgical rates
Parkinson (2018), Australia27	Adult breast cancer patients	222	Eight breast clinics in Australia	RCT	Mode: Web-basedTiming: Unclear	The Breast RECONstruction Decision Aid (BRECONDA)—an evidence‐based online intervention that supports women through their breast reconstruction decision making including information on strategies for managing emotions related to the reconstruction decision, values clarification components and video segments detailing other patients’ experiences. It takes ~45 min to review all sections of the website.	↓ Decisional conflict↑ Satisfaction with the information∅ Decisional regret
Murray (2001a), UK28	Men with benign prostatic hypertrophy	112	33 general practices from two urban areas, one suburban and one semirural area in the UK	RCT	Mode: Interactive multimedia video with bookletTiming: Unclear	An interactive multimedia programme with booklet and printed summary. Treatment options discussed were surgery, balloon dilatation of the prostate, drugs and watchful waiting. Information comprised probabilities of the risks and benefits of each treatment, calculated on the basis of information on age, severity of symptoms and general health. After viewing the programme, the patients were given a summary of the information; a copy was also sent to their general practitioners.	↓ Decisional conflict↑ Perceived that treatment decisions had been made by patients∅ Anxiety∅ Prostatic symptoms∅ Health status and physical function∅ Resource utilisation
Murray (2001b), UK29	Perimenopausal women	205	26 general practitioners, two urban, one semi-urban and one semi-rural	RCT	Mode: Interactive multimedia video with bookletTiming: Unclear	The intervention comprised an interactive multimedia programme, with booklet and printed summary. Information comprised quantified probabilities of the risks and benefits of hormone replacement therapy. After viewing the programme, the patients were given a summary of the information; a copy was also sent to their general practitioners.	∅ Treatment preference↓ Decisional conflict∅ Patient perception of who should make the treatment decision∅ Anxiety∅ Health status or state∅ Menopausal symptoms
Vuorma (2004), Finland30	Women with menorrhagia	363	Gynaecology outpatient clinics in 14 hospitals	RCT	Mode: BookletTiming: 7 days prior to outpatient appointment	PDA booklet (25 pages) about menorrhagia and its treatment options. Based on scientific literature (not systematic review) and clinical guidance	∅ Health status∅ Anxiety∅Psychosomatic symptoms∅ Sexuality∅ Menstrual symptoms∅ Satisfaction with outcome∅ Healthcare services
Hollinghurst (2010), UK31	Pregnant women who have had one previous caesarean delivery	524	Three maternity units in South West England and one unit in Scotland	Three arm RCT	Mode: Web-basedTiming: Pre-visit	The information programme provided descriptions of the risks and benefits for vaginal birth after caesarean, elective caesarean, emergency caesarean, including possible health outcomes for mother and baby	∅ Resource utilisation↓ Decisional conflict
Patel (2014), UK32	Adult participants who had been referred to a single community physiotherapy department for treatment of non-specific low back pain	148	Physiotherapy service at National Health Service (NHS) Coventry Community Physiotherapy	RCT	Mode: BookletTiming: Pre-consultation	A patient booklet that details the available treatment options (exercise, manual therapy, acupuncture and a cognitive behavioural approach). The booklet also provided answers to the frequently asked questions associated with each option. Space was provided to enable patients to note any points they wanted to discuss in the consultation.	↓ Satisfaction with treatment
Arterburn (2015), USA33	Patients with benign prostatic hyperplasia (BPH)	3778	The Group Health urology service line which includes 14 staff surgeons in five speciality clinics.	Pre–post observational	Mode: DVD or onlineTiming: Pre or post-consultation	12 high-quality video-based PDAs with accompanying written information in booklet format. PDAs were distributed primarily by mail in DVD format; clinical staff could order the DVD versions through the electronic health record. Patients could also view the PDAs online, and providers could embed a link in the patient’s after-visit summary.	↓ Transurethral prostate procedures among men who had previously received pharmacological treatment for BPH↓ Reduction in actively treating prostate cancer
Nagayama (2016), Japan34	Residents in a geriatric health facility	54	Geriatric health service facilities	RCT	Mode: iPAD applicationTiming: Unclear	The Aid for Decision-making in Occupation Choice (ADOC). The participants and occupational therapists each used the ADOC to identify meaningful occupations from 95 illustrations of daily occupations. Then, the participants and occupational therapists set goals and prioritised the occupations. The occupational therapists observed each participant performing the selected occupations and assessed their occupational performance.	∅ Quality of Life and Quality Adjusted Life Years↑ Activities of Daily Living
Klaassen (2018), Netherlands35	Adult breast cancer patients	100	Six hospitals	Pre–post	Mode: Web-basedTiming: During the consultation	The PDA consisted of four sections: (i) intro and generic info on late side effects of primary treatment It was emphasised that the PDA solely focussed on aftercare, whereas follow-up is standard for all patients; (ii) assessment of preferences and values; (iii) presentation of available aftercare options and intuitive preference assessment; (iv) overview of the (mis)match between aftercare options and individual preferences: Data on patient’s preferences and intuitive response was combined into an overview resembling an Option Grid PDA.	∅ SDM∅ Decisional conflict↑ Chose less intensive aftercare↑ Consultation time
Ogink (2018), US36	Patients with lumbar spinal stenosis	10 858	18 different practices in the USA	Retrospective cohort study	Mode: UnclearTiming: Unclear	Health Dialogue Decision aid	Not applicable
Schaffer (2018), USA37	Emergency department (ED) physicians and mid-level providers caring for patients with chest pain + patients presenting to the ED with a chief complaint of chest pain	898	Five USA EDs: The University of Pennsylvania; the Mayo Clinic in Rochester, Minnesota; the Mayo Clinic in Jacksonville, Florida; the University of California Davis; and Indiana University Health Methodist Hospital	RCT	Mode: PaperTiming: During the clinical encounter	The decision aid describes for patients the rationale for, and results of, the initial emergency department evaluation (ECG, initial cardiac troponin level) and the potential utility of additional cardiac testing. The decision aid also provides explicit management options (admission with urgent cardiac stress testing, follow-up with a cardiologist and so on) for the clinician and patient to consider when reaching a shared decision.	∅ Healthcare utilisation∅ ED length of stay∅ Billing-based utilisation (except testing)

∅=non-significant difference between control and intervention, ↑=outcome is statistically significant in favour of the intervention, ↓=statistically significant decrease in outcome.

*Cost outcomes are reported in table 2A.

CHD, coronary heart disease; MMR, measles, mumps and rubella; RCT, randomised controlled trial.

Table 2B

Concerns with methods and/or conclusions of studies published since the previous systematic review

Study	Methods	Conclusions
Kennedy (2002)16	No major issues	These seem reasonable
Wennberg (2010)17	Insufficient duration of follow-up; the PDA is delivered concurrently with behavioural change and motivational counselling	It is not possible to disentangle the effect of the PDA, behavioural change and motivational counselling on resource utilisation
Van Peperstraten (2010)18	Including payment for an extra cycle of IVF in conjunction with the PDA may have affected patient’s choice	As the PDA included an inducement, the conclusions should be treated with caution
Arterburn (2012)19	Observational study with no concurrent control population; short follow-up period (180 days); no evidence on compliance; concurrent introduction of PDA and quality improvement initiative	The relatively short follow-up suggests caution may be warranted in the claimed effect size
Cox (2012)20	PDA costs were not included and as this is a pilot study it was not powered to detect differences.	The conclusions are expressed conservatively.
Veroff (2013)21	Insufficient duration of follow-up; the PDA is delivered concurrently with behavioural change and motivational counselling	It is not possible to disentangle the effect of the PDA, behavioural change and motivational counselling on resource utilisation
Wilson (2013)22	The incremental net benefit is based on the difference between two options (in-person vs telephone delivery) for which the net benefit is negative; this is wrong.	Both methods of delivery had a negative willingness to pay yet the difference between these is presented as a positive, while telephone delivery is better than in-person, to suggest that it should be invested in is wrong.
Keyserling (2014)23	No usual care arm.	The absence of the comparator makes the conclusion misleading.
Tubeuf (2014)24	The study used intended resource use not actual resource use.	As actual resource use is not gathered, the conclusions are questionable but seem reasonable as expressed.
Volandes (2016)25	Pilot study not powered to detect significant differences and the intervention coincided with changes to reimbursement, multiplicity of comparisons.	The conclusions over-reach; significant conflicts of interest also appear to exist.
Trenaman (2017)26	There were no substantive issues, though longer term follow-up would be required to confirm findings.	There were no substantive issues though the incremental cost effectiveness ratios were not statistically significant.
Parkinson (2018)27	Intervention costs per patient are based on the assumption that every person available for the intervention would receive it for the next 3 years which seems optimistic; healthcare provider time with the PDA was based on the literature, which is a big assumption.	The conclusions are expressed conservatively.
Murray (2001a)28	The assessed technology delivery modality was redundant by completion of the study	Possibly reasonable comments regarding Internet delivered PDA go beyond the evidence in the study
Murray (2001b)29	The assessed technology delivery modality was redundant by completion of the study	Possibly reasonable comments regarding Internet delivered PDA go beyond the evidence in the study
Vuorma (2004)30	No major issues	These seem reasonable
Hollinghurst (2010)31	The costs associated with complications for the child beyond 6 weeks post-delivery (risks that may relate to delivery mode) were excluded and the use of weighted average costs for delivery mode complications.	The conclusions of the paper don't really follow from the analysis. The authors say there is no cost to the NHS associated with the intervention but allow for a midwife consultation to guide the mother through the decision aid which would have a cost.
Patel (2014)32	Pilot study. It is therefore not powered to find definitive outcomes. PDA cost not included in the analysis.	The conclusions are expressed conservatively
Arterburn (2015)33	The cost of developing the PDA appear not to have been included and the potential for a trend in method of treatment (falsification test) appears not to have been conducted.	The conclusions are expressed conservatively.
Nagayama (2016)34	This is a feasibility study and therefore likely underpowered to detect real differences; intervention costs appear not to be included in the analysis.	The conclusions state cost-effectiveness was shown and are therefore overstated.
Klaassen (2018)35	This was a pilot study and likely underpowered to detect differences; the exclusion of PDA costs from the analysis presents another issue.	The conclusions are expressed conservatively.
Ogink (2018)36	PDA costs were excluded from the analysis, outcomes other than costs were not examined. Less than 1% of participants may have received the intervention.	The conclusions are expressed conservatively.
Schaffer (2018)37	The study did not examine costs (nor by implication what those associated with the intervention were) nor are outcomes actually reported here.	The results are misleading as outcomes are not actually reported here.

IVF, in vitro fertilisation; NHS, National Health Service; PDA, patient decision aid.

For the 16 RCTs, over 70% had low risk of selection bias, over 65% had low risk of attrition bias and nearly 40% had low risk of detection or performance bias (online supplemental table l C). Pre–post studies were rated to be of fair or good quality (range=4 to 8) (online supplemental table D), and the one observational study was deemed to be of fair quality (online supplemental table E).

Synthesis of evidence to determine whether PDAs generate savings

PDA interventions that generated cost savings

Twelve studies reported that the PDA intervention generated cost-savings which ranged from US$10 to US$81 156 (when adjusted to US dollars in 2020). Despite reporting that the PDA intervention generated savings, 7 of the 12 studies contained methodological issues or overstated results which impacted the interpretative validity of conclusions (see the next section for more details).

To summarise, Kennedy16 found that the interview group had lower mean costs than the control group or information-only group. Wennberg17 found that an enhanced coaching intervention for patients with preference-sensitive conditions led to a decrease in hospitalisations and significant monthly savings to the payer of healthcare services (US$8 per member). van Peperstraten’s18 empowerment strategy which included a PDA to help couples decide how many embryos should be transferred during the in vitro fertilisation process, led to mean cost savings of US$219 per couple from the healthcare system’s perspective due mainly to the lower rate of twin pregnancies in the intervention arm of the trial. Arterburn’s19 decision support intervention for knee and hip replacement surgery was associated with decreased surgery rates and a reduction in arithmetic mean costs for the Group Health organisation of 17% and 19% per patient respectively. Cox’s20 decision aid for mechanical ventilation were associated with a significant decrease in cost for the intervention group compared with usual care (US$110 609 vs US$178 618) which authors presume is due to fewer days in the intensive care unit, fewer hospital and ventilator days. Wilson’s22 cost-benefit analysis found that the incremental net benefit (INB) of eliciting questions/concerns of breast cancer patients pre-visit by telephone compared with doing so in person was positive (US$65). Keyserling’s23 web-based intervention to educate patients on their risk of coronary heart disease cost significantly less to implement compared with the counsellor format (US$220 vs US$393 less per participant) from the societal perspective when accounting for the labour (wages and market value for staff time) and non-labour (postage, printing, laptops and so on) costs. Tubeuf24 concluded that a web-based tool for first time parents whose first-born was offered the MMR vaccine had an approximate 72% chance of being cost-effective from the National Health Service perspective. Volandes25 concluded that advanced care planning videos ‘decreased healthcare costs in the last month of life for decedents’ relative to the control group. Trenaman’s26 cost-effectiveness analysis found that a PDA for total joint arthroplasty had a high probability of being cost-effective, ranging from 88% to 99% across willingness to pay values of US$0 to US$100 000 per quality-adjustedlife year (QALY). Parkinson’s27 online tool for breast reconstruction surgery cost less compared with usual care from the healthcare systems perspective mainly due to lower hospitalisation costs.

PDA interventions that did not generate cost savings

Ten studies reported that the PDA intervention either did not generate significant cost-savings or actually cost more than the comparator. In contrast to the previous paragraph, however, the majority of these studies (7 out of 10) expressed their conclusions conservatively based on our assessment.

To summarise, Murray’s28 29 multimedia PDAs (benign prostatic hypertrophy and hormone replacement therapy) increased costs from the healthcare system perspective mainly due to the cost of the video technology. The cost of Vuorma’s30 decision aid intervention to help women with menorrhagia make decisions did not generate significant savings when compared with usual care (€2760 vs €3094), respectively from the societal perspective when accounting for surgical procedures or other medical treatments, visits, tests, the cost of producing the intervention and the personal costs to the participant. Hollinghurst’s31 information programme which provided the risks and benefits for vaginal birth after caesarean cost more than usual care or decision analysis group (£2069, £2033 and £2019, respectively) from the National Health Service (NHS) perspective mainly due to the cost associated with the mode of baby delivery. Patel’s32 decision support package to help patients with low back pain was not cost-effective in comparison to usual care (£264.7 vs £271, respectively) from the healthcare systems perspective when considering the cost of NHS services, tests, drugs and the cost of the intervention.32 Arterburn’s33 video-based PDA for BPH and prostate cancer lowered surgery rates but was not linked to significant changes in healthcare costs from the healthcare system perspective. Nagayama34 found that the iPAD application for Occupation Choice did not generate savings compared with standard occupational therapy (US$11 643 vs US$11 393, respectively) from the participants’ perspective. Klaassen’s35 breast cancer aftercare decision aid did not significantly reduce costs (€92) compared with usual care (€123) from the hospital’s perspective when taking into account tests, days in the hospital, emergency room visits, physiotherapy and social work sessions though this excluded consultation time, when significantly increased. Ogink’s36 retrospective cohort study found that a decision aid received by 82 patients with lumbar spinal stenosis did not change healthcare costs from the payer’s perspective. Schaffer37 evaluated the impact of the Chest Pain Choice PDA on healthcare utilisation, finding this to be lower, however the authors did not examine costs.

Quality assessment of the economic analyses

There was considerable variation in the quality of the economic analyses reporting. CHEERS scores (online supplemental table F) ranged from 50% to 100% (mean=78%). Key elements that were not generally reported include the incremental costs and outcomes (mean values or the main categories of estimated costs and outcomes of interest, as well as the mean differences between the comparator groups), and the characterisation of uncertainty and heterogeneity. The quality of the studies also varied widely. Drummond quality scores ranged from 0 to 10 (mean=5.4). The majority of studies did not identify the incremental analysis or relevant costs and consequences for each alternative, the allowance for uncertainty in the estimates of costs or the issues and concerns with the results. See table 2A and online supplemental table G.

Many studies contained methodological problems the most common of which were a short duration of follow-up, underpowered analyses due to small sample size or lack of a control group which affects the interpretative validity of their conclusions. To summarise: in the Wennberg17 study, and the subanalysis by Veroff,21 there was insufficient duration of follow-up. It was not possible to isolate the impact of the PDA on cost-savings as it was delivered concurrently with behavioural change and motivation counselling. A longer time horizon (2 years compared with 1 year) was reported by Trenaman26—the only study with a perfect Drummond score—in order to assess the value for money of patient decisions aids. The observational study by Arterburn19 had no concurrent control population, no evidence on whether patients who received the decision aid via mail actually viewed them or whether patient–clinician conversations changed as a result of the decision aid, a short follow-up period (180 days) for patients undergoing elective knee and hip replacement surgery and so the association between the implementation of the decision aid and the decrease in surgical procedure rates needs to be interpreted with caution. The Keyserling23 study lacked a usual care arm, so to suggest that the web-intervention is cost-effective is misleading. Wilson22 presented the incremental net benefit as positive despite the net benefit for the two methods of delivering the intervention (telephone and in-person) being negative. Due to this presentation, the wrong impression was created that one method of delivery (telephone) was not as costly as the other (in-person), and thus warrants caution.22 In the Tubeuf24 study, costs were estimated based on intended rather than actual resource use. Conclusions from the van Peperstraten18 RCT should be treated with caution as well because the offer of reimbursement for an additional fourth cycle was only applicable if couples chose single embryo transfer in the first and second cycle and no pregnancy occurred, which may have made the option more attractive. In addition, the costs associated with complications for the child beyond 6 weeks post-delivery (risks that may relate to delivery mode) in the Hollinghurst31 study were excluded and weighted average costs were used for delivery mode complications. See table 2B for details.

The four pilot studies,20 25 33 35 included in this review lacked power to provide compelling evidence of cost-effectiveness. The pilot studies and some other20 33–36 RCTs did not include the cost of decision aids in their analyses. These reasons provide a basis on which to question any findings related to cost-savings given that the accuracy of estimated incremental costs is unclear. For instance, in the study by Parkinson,27 PDA costs assumed that everyone eligible to access the service would use the tool over the course of 3 years. This seems overly optimistic and could result in an underestimate of incremental costs.27 The methodological concerns and interpretative validity of the results are reported in table 2B.

Discussion

Statement of principal findings

Based on our assessment of these 22 studies, and their methodological weaknesses, we conclude that there is contradictory evidence as to whether PDA use will lead to cost savings and that it is not possible to arrive at a firm conclusion. A lack of consensus exists with respect to which resources should be included when costing a PDA and apportioning these appropriately (the development, updating and maintenance of, for example, a web resource). There is also a high degree of variability with respect to: the mode of delivery for the PDA (web-based only, web-based plus interaction with a healthcare professional), the timing of delivery (before, during or after the consultation), whether the tools were actually used by the patient, the complexity of the tools’ content, the subsequent compliance with the initial decision (follow-up times did not allow assessment of whether the decision merely postponed treatment) which could have positive or negative downstream cost implications, and the lack of data availability regarding the long-term impact on health-related quality of life and survival. Essentially, the heterogeneity of the methods employed due to the lack of consensus on an evaluative economic framework to assess PDAs complicates the task of determining whether or not these tools generate savings or the context in which they could generate savings.

Strengths and weaknesses of the study

Our systematic review had a number of strengths: we followed the PRISMA reporting guidelines and best practice guidelines for conducting a systematic review; two information scientists (PJB and HBB) adapted our search strategy for each database; we included a broader inclusion criteria in comparison to the EEPRU report by Ara et al10 which resulted in more studies and additional insight into the challenges of undertaking economic evaluations of PDAs; two health economists (CON and GEC) independently used two checklists to assess the quality of the economic analyses and how they were reported and evaluated the interpretative validity of the studies.

In terms of limitations, despite our best efforts we may not have found every relevant study. We believe this limitation was mitigated, however, by our search strategy which included all papers from the previous review2 and our broad inclusion criteria which included studies with non-randomised designs and the review of the studies’ reference lists. The intervention descriptions were often limited, so it was difficult to ascertain whether they were PDAs. The heterogeneity of the methods and results did not enable a meta-analysis.

Results in context

Our findings mirror the conclusions of the previous systematic review (2014) which determined that the evidence as to whether or not PDAs generate savings is inconclusive due to the heterogeneous nature of the methods, the lack of quality economic analyses and the issues related to the study design and interpretation of results.2 The variation in the economic models used to assess cost in these studies, however, suggests that an agreed on economic evaluative framework needs to be developed.

The lack of consensus on an economic evaluative framework that is best applied to this research question makes it difficult to determine the context in which PDAs could generate savings. Researchers may therefore need to re-think how to approach these evaluations and how we attribute monetary value to PDAs. Butt described the limitations of the current economic frameworks because they focus on ‘health gain’ when in fact the PDAs may have a much broader impact.8 Butt states that ‘by excluding other benefits within the broad umbrella of process of care, the utility of decision aids is likely to be undervalued’.8 Other variables such as the duration of the patient–clinician relationship, the health literacy of the patient, the timing and mode of delivery may impact monetary value and clinical encounter duration.8 For this reason, the consultation time trade-off (CTTO) has been proposed as a new evaluative framework to assess the cost-effectiveness of PDAs.8 The CTTO would represent ‘the number of minutes the patient would be willing to trade for use of the tool’, so the number of consultation minutes saved due to the PDA would be converted to a monetary value using the clinician’s wage rate.8

Furthermore, the EEPRU (2015) suggested that the ‘the scope of an economic evaluation of PDAs needs to be extended beyond the health-related QALY’ to include the wider societal benefits of health services.10 The report suggests that finding ways to express the benefits such as increased knowledge, reduced decisional conflict and improved patient–clinician communication in terms of QALYs or another metric in which value is captured is important in any cost estimation.10 Ultimately, the development of a framework which could potentially include non-health outcomes in the valuation of PDAs could lead to more well-designed economic evaluations and more definitive conclusions about the impact of these tools on cost and outcomes. The adoption of a welfarist cost-benefit approach in which stated or revealed preferences are expressed in monetary terms and related to costs may offer a way forward

Implications

The lack of definitive evidence that PDAs lead to cost-savings can be attributed to a number of factors. Existing studies lack sufficient methodological rigour and are of limited duration given the potential consequences of these tools may extend over time.38 Current evaluative frameworks may be too narrow, and therefore limit the potential to consider the full scope of impact on savings.

Ultimately, economic evaluations provide a comparative analysis of alternative uses of resources in terms of cost and outcomes. The technique is well established, and clear guidance on the conduct and reporting of evaluations exists including identification of appropriate comparators, study perspective (which will determine costs included) and choice of appropriate outcome. However, the sine qua non for an evaluation of a PDA is consensus around the definition of a PDA, including its delivery modality. While there remains ambiguity in the evaluation literature around what constitutes a PDA, assessments of their cost-effectiveness and comparisons of these will continue to be problematic. Ideally, an evaluation should first set out how the intervention described meets accepted criteria for its consideration as a PDA. Only then are the questions of which PDAs, and under which circumstances, provide good value for money likely to be answered in a convincing manner.

It is also important to voice the argument that supporting patients and clinicians to arrive at informed, well-considered decisions has value in and of itself. This value has been recognised by many organisations including NICE (National Institute for Health and Care Excellence) in the UK who are committed to placing patient involvement at the fore of the treatment decision-making process.39 Similarly, the Affordable Care Act advocated that shared decision making be supported by the development of certified PDAs—an indication of the value of engaging patients in decisions.40 What monetary value do we place on high quality decisions—on doing the right thing? It is a question that lies at the heart of current health systems.

Conclusion

It is unclear based on the quality of the economic evidence available to date, whether PDAs generate cost savings in healthcare settings, regardless of the perspective being analysed. Nevertheless, the evidence that these tools improve patient outcomes and the overall quality of decision-making should encourage their implementation in practice by organisations who wish to practice patient-centred care. Going forward it is important to conduct robust and well-designed economic evaluations which have sufficient follow-up to enable costs and outcomes (both health and non-health) to be assessed over longer time horizons.

Table 2A

Review of economic analysis of the patient decision aid interventions

Study	Perspective	Time horizon	Mean PDA cost per user*	Incremental cost effectiveness ratio (ICER)/incremental net monetary benefit (INMB)*	Probability of being cost effective	Quality†
Kennedy (2002)‡16	Healthcare system	2 years	Information pack: US$29Interview: US$38	None reported	None reported	6
Wennberg (2010)‡17	Payer	1 year	Control: US$275PDA: US$265	None reported	None reported	7§
Van Peperstraten (2010)18‡	Healthcare system	Not clear	Intervention: US$587	None reported	None reported	8§
Arterburn (2012)19‡	Organisation	6 months	HipControl: US$19 989PDA: US$16 284KneeControl: US$12 122PDA: US$9707	None reported	None reported	0§
Cox (2012)20	Healthcare organisation	4 months	PDA: US$131 992 (SD=106 630)Control: US$213 148 (SD=137 415)	None reported	None reported	4
Veroff (2013)21‡	Payer	1 year	Usual support: US$541Enhanced support: US$512	None reported	None reported	6§
Wilson (2013)22	Societal, resource centre and participant	Unclear	Per participantResource centre:US$221 (telephone)US$241 (in-person)Participant:US$31 (telephone)US$78 (in-person)Societal:US$252 (telephone)US$319 (in-person)	The INMB of providing telephone compared with in-person CP was US$79 (societal perspective).	None reported	6§
Keyserling (2014)23	Payer, participant and social	1 year	Counsellor:Payer: US$237 (SE=US$3.9)Web-based:Payer: US$126 (SE=US$4.0)	ICER per QALY gainedSocietal cost: US$6577	None reported	6§
Tubeuf (2014)24	Healthcare system (National Health Service (NHS)) and societal	1 year	Payer: PDA US$60 (SD=US$10.9)Control: US$76 (SD=US$8.96)Societal: PDA US$72 (SD=US$13.7)Control: US$83 (SD=US$10.73)	Not reported	The PDA has ~~72% chance of being cost-effective based on the NHS perspective	8
Volandes (2016)25	Patients	2 years	Unclear	None reported	None reported	4§
Trenaman (2017)26	Health system	2 years	PDA: US$6748 (95% CI 6162 to 7271)Control: US$7198 (95% CI 6595 to 7668)	Dominant	The PDA arm has a high probability of being cost-effective, ranging from 88% to 99% across willingness to pay values of US$0 to US$89 609 per QALY.	10
Parkinson (2018)27	Healthcare system	6 months	PDA: US$2475 (95% CI 1657 to 3413)Control: $3058 (95% CI 2087 to 4109)	Dominant	BRECONDA has an 87% probability of being cost-effective at US$60 000 QALY gained	8
Murray (2001a)28‡	Healthcare system	9 months	Control: US$408 (SD=US$650)PDA: US$1285 (SD=US$1302)	None reported	None reported	5
Murray (2001b)29‡	Healthcare system	9 months	Control: US$197 (SD=US$85)PDA: US$663 (SD=US$93)	None reported	None reported	4
Vuorma (2004)30‡	Societal	1 year	Control: US$8046 (SD=US$665)PDA: US$7178 (SD=US$672)	None reported	None reported	3
Hollinghurst (2010)31	Healthcare system (NHS)	37 weeks gestation and 6 weeks postnatal	Usual care: US$3885 (SD=1293.8)Usual care + information programme: 3954 (SD=1410)Usual care + decision analysis programme: US$3858 (SD=1416.1)	None reported	None reported	7§
Patel (2014)32	Healthcare system	4 months	PDA: US$443 (SD=74.5)Control: US$454 (SD=602.7)	ICER of US$3181 (US$63.6/0.02) per QALY gained for usual care compared with the PDA	The PDA is unlikely to be cost-effective: with only 16% probability of being cost-effective at a threshold of US$33483/QALY gained.	7
Arterburn (2015)33	Healthcare system	6 months	No prior medical treatmentControl: US$9250 (95% CI 8099 to 10 400)PDA: US$9770 (95% CI 8460 to 11 079)Prior medical treatmentControl: US$10 558 (95% CI 9019 to 12 098)PDA: US$8948 (95% CI 7755 to 10 141)Prostate cancer cohortControl: US$22 762 (95% CI 19 725 to 25 801)PDA: US$20 386 (95% CI 17 576 to 23 198)	None reported	None reported	5.5
Nagayama (2016)34	Participant	4 months	PDA: US$14 949 (SD=1266)Control: US$14 628 (SD=1959)	US$81 per change in Barthel Score	None reported	5§
Klaassen (2018)35	Hospital	6 months	PDA: US$126 (SD=US$218.6)Control: US$168 (SD=US$203.4)	None reported	None reported	4
Ogink (2018)36	Payer	12 years	PDA: US$1373 (SD=1309)Control: US$1378 (SD=1441.5)	None reported	None reported	2
Schaffer (2018)37	Healthcare system	45 days	None reported	None reported	None reported	2.5§

*Results were expressed as US dollars adjusted if necessary for purchasing power parity and inflated to 2020. Purchasing Power Parity conversion and inflation adjustment rates were taken from the International Monetary Fund through a tool developed with support from the Cochrane group (https://eppi.ioe.ac.uk/costconversion/default.aspx). Where original data were reported in $ following conversion from another currency using reported exchange rates, they were converted back to the original currency before adjusting for inflation and Purchasing Power Parity.

†Possible range 0 to 10 with higher scores indicating higher quality.

‡Study is carried over from the previous systematic review published in 2014.

§See table 2B for commentary on studies’ method and interpretative validity.

PDA, patient decision aid; QALY, quality-adjusted life year.