Translating Evidence to Facilitate Shared Decision Making: Development and Usability of a Consult Decision Aid Prototype.

AIM: The purpose of this study was to translate evidence from Cochrane Reviews into a format that can be used to facilitate shared decision making during the consultation, namely patient decision aids.
METHODS: A systematic development process (a) established a stakeholder committee; (b) developed a prototype according to the International Patient Decision Aid Standards; (c) applied the prototype to a Cochrane Review and used an interview-guided survey to evaluate acceptability/usability; (d) created 12 consult decision aids; and (e) used a Delphi process to reach consensus on considerations for creating a consult decision aid.
RESULTS: The 1-page prototype includes (a) a title specifying the decision; (b) information on the health condition, options, benefits/harms with probabilities; (c) an explicit values clarification exercise; and (d) questions to screen for decisional conflict. Hyperlinks provide additional information on definitions, probabilities presented graphically, and references. Fourteen Cochrane Consumer Network members and Cochrane Editorial Unit staff participated. Thirteen reported that it would help patient/clinician discussions and were willing to use and/or recommend it. Seven indicated the right amount of information, six not enough, and one too much. Changes to the prototype were more links to definitions, more white space, and details on GRADE evidence ratings. Creating 12 consult decision aids took about 4 h each. We identified ten considerations when selecting Cochrane Reviews for creating consult decision aids.
CONCLUSIONS: Using a systematic process, we developed a consult decision aid prototype to be populated with evidence from Cochrane Reviews. It was acceptable and easy to apply. Future studies will evaluate implementation of consult decision aids.

Key Points for Decision Makers

Background

Health policy in several countries calls for using patient decision aids to facilitate shared decision making, provide more patient-centered care, enhance health literacy, and improve patient safety [1-4]. This demand for patient decision aids is part of a larger paradigm shift from doctor-centric decision making to shared decisions with patients which is achievable if effective implementation interventions such as patient decision aids are used [5]. Patient decision aids make explicit the decision, provide high-quality evidence on the outcomes of options including watchful waiting, and help patients clarify what is important to them based on their personal circumstances [6]. Longer formats have been developed and used by patients in preparation for the consultation with briefer formats used between patients and clinicians during the consultation. Compared with usual practice, patients exposed to decision aids have higher decision quality, increased participation in decision making, and lower personal uncertainty concerning the decision to be made (i.e., decisional conflict) [6]. Those decision aids designed for use in the consultation are more likely to facilitate shared decision making discussions between patients and their clinicians [6, 7]. Despite strong evidence on the effectiveness of patient decision aids, there are only about 300 up-to-date ones publicly available for a limited number of health conditions and most are restricted to English [8].

Evidence from Cochrane systematic reviews could be used to develop decision aids to inform patient-level decisions and reduce waste in health care by creating tools that promote the integration of research into everyday clinical practice [9]. There are over 6000 Cochrane Reviews that provide high quality evidence, free from industry interests, on the benefits and harms of options [10, 11]. Most reviews provide probabilities that can be used to communicate the chances of both benefits and harms for options; this is known to improve patients’ realistic expectations [6]. Another advantage to using evidence from Cochrane Reviews is the commitment to updates every 2 years which can ensure patient decision aids are consistent with the latest evidence [12]. As well, plain language summaries, available in multiple languages, provide a simpler description of the health condition and evidence from the Cochrane Review [6, 11]. However, current Cochrane Reviews do not present the evidence in an interactive format that meets the definition of a patient decision aid and are not in a user-friendly format to help patients with understanding the complex probabilistic nature of the evidence [10]. Current formats contain the full Cochrane Review, scientific abstract, and consumer summary. We argue that translation of Cochrane Review evidence into clinical practice would be greatly enhanced if it was embedded in patient decision aids.

Aim

Our overall aim was to translate evidence from Cochrane Reviews into a format that can be used to facilitate shared decision making during the consultation, namely patient decision aids. The specific objectives were to (a) develop and evaluate a prototype for a consult decision aid that is acceptable and usable; (b) apply the prototype to develop a series of 12 consult decision aids; and (c) identify considerations for embedding consult decision aids in Cochrane Reviews.

Methods

Study Design

We used a systematic development process that was guided by the Ottawa Decision Support Framework and the International Patient Decision Aid Standards (IPDAS) [11, 13, 14]. The process involved (a) establishing an expert committee of stakeholders; (b) developing the prototype for a consult decision aid; (c) conducting alpha testing with potential users; (d) applying the prototype to 12 Cochrane Reviews; and (e) reaching consensus on considerations for including consult decision aids in a Cochrane Review. Ethics approval was obtained from the Ottawa Hospital Research Ethics Boards (20130926-01H).

Theoretical Frameworks

We used IPDAS and the Ottawa Decision Support Framework. The IPDAS Collaboration established an evidence-informed framework for enhancing the quality and effectiveness of patient decision aids [15]. Over 100 stakeholders (e.g., researchers, policy makers, patients, clinicians, patient decision aid developers) from 14 countries reached consensus on 74 items in the IPDAS checklist [10]. The IPDAS checklist was transformed into a 47-item instrument [16]. More recently, IPDAS criteria were triaged into proposed qualifying (6 items), certifying (6 items plus 4 items for screening decisions), and quality criteria (28 items) [11]. For example, criteria for qualifying as a patient decision aid are explicitly stating the decision, providing information on the health condition and options, describing the positive and negative features of options, and helping patients clarify their values.

The Ottawa Decision Support Framework is an evidence-based mid-range theory for supporting patients making health or social decisions [14, 17, 18]. It was informed by theoretical concepts from psychology, decision analysis, decisional conflict, social support, and economics. This framework is most commonly used for developing and evaluating patient decision aids [18]. It has also been used to develop clinician decision support resources and tools to evaluate the quality and outcomes of providing patient decision aids [18, 19].

Stakeholder Committee

Our committee had a range of stakeholders including a consumer with regular exposure to a variety of consumers from different settings, clinicians (e.g., nurse, primary care physician, and rheumatologist), information technology specialists, and health services researchers with expertise in patient decision aid development and evaluation, knowledge translation to patients, shared decision making, systematic reviews, and web-based applications.

Consult Decision Aid Prototype

We developed a prototype for a patient decision aid that could be informed by evidence taken directly from Cochrane systematic reviews. Cochrane Reviews were chosen because the Cochrane Library has the largest international set of systematic reviews providing a robust source of regularly updated evidence free from industry interests. The patient decision aid was titled a consult decision aid given a brief 1-page format designed for facilitating discussion in the consultation. The prototype includes (a) a title that specifies the decision; (b) information on the health condition and the eligible users; (c) a summary of the options being considered; (d) evidence on the benefits and harms of each option with probabilities (evidence quality rated using GRADE); (e) an explicit values clarification exercise; (f) a question soliciting the preferred option; and (g) the SURE Test to screen for decisional conflict and thus identify decision making needs (Fig. 1). The prototype has several hyperlinks to additional information including definitions, graphic representation of probabilities, and references. The consult decision aid meets all IPDAS qualifying and certifying criteria except the criterion for using an implicit values-clarification method requiring detailed descriptions of the options and outcomes [11]. Instead, given there is “no single ‘best’ practice” for values clarification [20], an explicit values-clarification method was included using a rating scale. Probabilities are displayed using words, numbers and diagrams: formats known to be easier for patients to understand [21, 22].

Fig. 1

Consult decision aid prototype indicating IPDAS criteria

Alpha Testing with Potential Users

The consult decision aid prototype was populated with evidence from a Cochrane Review focused on alendronate as a treatment option for osteoporosis (Appendix, see electronic supplementary material). Alpha testing of this consult decision aid on alendronate was conducted with members of the Cochrane Consumer Network (potential patients) and staff from the Cochrane Editorial Unit to determine acceptability and usability (Table 1) [13]. Cochrane Consumer Network members and the Editorial Unit staff are familiar with Cochrane protocols and reviews but unlikely to have had previous exposure to values clarification exercises and tools to support decision making. Eligible participants needed to be able to read English and have computer access. Participants were interviewed in person or via Skype. During the audio-recorded interviews, the research assistant guided the participant through the consult decision aid in a manner similar to which it could be presented in a consultation and then asked questions using a structured interview guide. Prior to the interview, participants received a copy of the consult decision aid and interview guide. They were told not to review it before the interview and that it would be used during the interview.

Table 1

Demographic characteristics of participants (n = 14)

Characteristics	Frequency
Age (in years)
30–39	1
40–49	2
50–59	10
60–69	1
Sex
Male	2
Female	12
Highest level of education
High school	3
Community college	4
University undergraduate degree	4
University graduate degree	3
Employment status
Employed as a health professional	3
Employed as a manager	2
Retired	2
Receiving disability	5
Unemployed	2
Experience with the health condition
No	4
Yes	10
<2 years	1
>2 years	9

The interview guide had seven questions from the Preparation for Decision Making Scale [23], ten questions from the acceptability questionnaire that has been extensively used for evaluating patient decision aids during development [24], and four common questions for usability testing [25] (Table 2). The Preparation for Decision Making Scale, previously validated, was shown to have high internal consistency with alpha coefficients of 0.92–0.96, and discriminated between patients who reported the patient decision aid was or was not helpful [23].

Table 2

Consult decision aid acceptability and usability

Items	Frequency (N = 14)
Amount of information in the consult decision aid
Much less than I wanted	2
A little less than I wanted	4
About right	7
Little more than I wanted	0
Much more than I wanted	1
Balanced presentation of information in the consult decision aid
Clearly slanted towards taking the treatment	0
Slightly slanted towards taking the treatment	4
Completely balanced	10
Slightly slanted towards not taking the treatment	0
Clearly slanted towards not taking the treatment	0
Icons readable
Yes	12
No	2
Space for data entry
Yes, enough	5
Not enough	8
Not necessary	1
Words in the PtDA make sense
Yes	12
No	2
PtDA fit with patients’ discussions with physician, nurse or pharmacist
Yes, as it is	12
Yes, but with some alteration	1
No	1
Willingness to use PtDA and/or tell someone about it
Yes	13
No	1

PtDA patient decision aid

Participant quantitative responses were entered into an Excel database and analyzed descriptively using frequency distribution. Audio recordings of interviews were transcribed verbatim and qualitative comments were analyzed using content analysis. More specifically, the transcript was read line by line to identify common themes that participants identified as issues affecting acceptability and/or usability of the patient decision aid. The research team reviewed findings and revised the prototype.

Applying the Prototype to Cochrane Reviews

The revised prototype was used to create a series of 12 consult decision aids populated with data from 12 different Cochrane Reviews. Eligible reviews had a summary of findings table and included two or more reasonable options (that may have included no treatment/screening). A clinician from the stakeholder committee with expertise in patient education reviewed the description of the health condition and options and verified accuracy of the probabilities presented. As well, the clinician verified use of plain language. Time to create the consult decision aids was monitored.

Considerations for Embedding a Consult Decision Aid in a Cochrane Review

Using a Delphi process [10], the stakeholder committee was tasked with reaching consensus on the considerations necessary for selecting which Cochrane Reviews should have an accompanying consult decision aid. This Delphi process involved three iterations: (1) in the first round, we gathered suggestions based on our experiences developing patient decision aids and creating the series of 12 consult decision aids; (2) in the second round, committee members individually rated each consideration on a scale from 1 to 9 in an Excel spread sheet (1 not important, 9 extremely important) [26]; and (3) in the third round, committee members discussed ratings having median differences of three or more points and reached consensus on those that were rated as important (e.g. rated as 7–9) [27].

Results

Participant Characteristics

Eleven members of the Cochrane Consumer Network and three staff from the Cochrane Editorial Unit consented to participate (Table 1). Two were male and 12 were female; ages 30–60 years old with a range of education levels (e.g., high school only to university graduate degrees). Participants were employed (e.g., health professionals, managers), retired, or unemployed with several receiving disability insurance. Although not required, ten had previous experience (personal, professional or family member) with the health condition in the consult decision aid (i.e., osteoporosis).

Acceptability and Usability

Most participants rated the details provided in each section of the consult decision aid as good or very good (Table 2). Ten of 14 rated the presentation of information as balanced and four rated it as slightly slanted toward taking treatment. Twelve participants thought the words made sense and icons were understood. Regarding information content, seven indicated that there was the right amount, six wanted more information, and one indicated there was too much. On the Preparation for Decision Making scale, most participants indicated it was helpful for preparing to make decisions with the clinician (Table 3). Thirteen participants indicated that it was suitable for use in consultations with their clinician and were willing to use it and/or tell someone else about it (Table 2). One consumer participant consistently rated the questions about acceptability and usability as low (Table 2); describing inadequate time with the clinician in the consultation to be able to use the consult decision aid as the main reason influencing his responses.

Table 3

Participants’ ratings on the Preparation for Decision Making Scale [23]

Items	Frequency (N = 14)a
	Not at all	A little	Somewhat	Quite a bit	A great deal
1. Help patients recognize a decision needs to be made	0	0	1	3	10
2. Prepare patients to make a better decision	0	0	2.5	4.5	7
3. Help patients think about the pros/cons of options	0	1	0	6	7
4. Help patients think about which pros/cons are most important	0	1.5	1.5	8	3
5. Help patients know that the decision depends on what matters most to them	0	0	0	7	7
6. Help patients organize their own thoughts about the decision	0	0	3	5.5	5.5
7. Help patients ask questions to their doctor	0	2	0	3	9

aHalf numbers were used when participants rated between items

Qualitative Feedback with Actions Taken (Table 4)

The suggestion to use more plain language (e.g., decline, contraindication, bisphosphonate, harm–benefit ratio) was addressed by changing to simpler words. Where words could not be changed, a link to the definition was added (e.g., best estimate, GRADE, osteoporosis). Topics/questions for patients to ask their clinician were added including safety with other drugs, long-term use of drugs, costs, changes in side effects over time, how to manage side effects, and allergies. Although the font was described as too small, an electronic version would allow the font size to be scalable. The font size in the prototype was chosen to ensure a one-page format when printed. More white space was added. The X icon used for the option of declining the active option was described as too strong and was changed to a size more consistent with other icons. Suggestions for using the consult decision aid in the consultation were developed (see Box 1).

Some suggestions in qualitative comments were not addressed. For example, in step 2 we did not add a full list of options because the options were limited to those described in the Cochrane Review. If Cochrane Reviews provided a full list of options, this information could consistently be added into the consult decision aid. Visual aspects and images would need to be improved by a graphic designer during the publishing process. Confidence intervals were not added as they make the figures too busy and are already acknowledged within the GRADE rating.

Applying the Prototype to a Range of Reviews

In March 2014, there were 975 Cochrane Reviews with summary of findings tables. Twelve Cochrane Reviews were purposefully chosen to represent a range of decisions about surgery, screening, or medications. The average time for creating a consult decision aid was 3 h by a research assistant plus 1 h for verification by a clinician on the stakeholder committee.

Considerations for Embedding Consult Decision Aids in Cochrane Reviews

The stakeholder committee reached consensus on three characteristics of Cochrane Reviews that make it easier to develop a consult decision aid and seven circumstances indicating when it is relevant to include a consult decision aid. It is easier to create a consult decision aid when a Cochrane Review provides a summary of findings table, describes the health condition and interventions in the plain language summary, and is focused on a topic involving two reasonable options (including no treatment/screening). It is relevant to include a consult decision aid when the Cochrane Review (a) provides evidence on benefits and harms; (b) is focused on an important topic to patients and clinicians; (c) is about a topic that occurs commonly in clinical practice; (d) involves trade-offs between benefits and harms; (e) is a controversial topic (e.g., strong views for and against the options); (f) involves a decision that is irreversible (e.g., surgery, genetic testing); and/or (g) has recently been updated (e.g., <5 years old).

Discussion

Using a systematic process, we developed a consult decision aid prototype that was found to be acceptable for facilitating shared decision making during the consultation and feasible to use for translating evidence from Cochrane Reviews. We showed that the Ottawa Decision Support Framework and IPDAS approaches for developing patient decision aids were flexible enough to develop this prototype [11]. Although the participants rated the consult decision aid as acceptable and usable, most wanted more information. In addition, the prototype was easily applied to create a series of 12 consult decision aids from 12 different Cochrane Reviews with the process taking on average 4 h per consult decision aid. Finally, the stakeholder committee determined three characteristics of Cochrane Reviews making it easier to create consult decision aids and seven circumstances indicating when it is relevant to create one. These results lead us to four main points of discussion.

First, our findings demonstrate that the Ottawa Decision Support Framework provides a clear approach for producing these briefer patient decision aids for use in consultations [17, 18, 28]. In previous trials of patient decision aids developed using the Ottawa Framework, patients had, for example, improved knowledge, improved realistic expectations, decreased decisional conflict, and rated the decision aid as acceptable [17, 28–34]. A toolkit for developing patient decision aids and consult decision aids using the Ottawa Framework is available in a free-of-charge online training tutorial [35]. Similar to other newer approaches to building decision aids for use in consultations (e.g., decision box, option grids), further research is required to determine their effectiveness [6].

Second, briefer one-page patient decision aids with more limited information on options shift the focus from providing detailed information on options to being more oriented to the process of decision making. The ultimate aim of the consult decision aid is to facilitate discussion of the evidence on the options and on the patients’ values during a shared decision making process. Of eight randomized controlled trials evaluating previously developed patient decision aids designed for use within the consultation, seven reported improved patient–clinician communication and one trial found no difference when compared with usual care [6]. Hence, the new consult decision aid should stimulate patient–clinician discussion within the consultation on the options, prompt them to consider other information tailored to the patient’s personal situation, provide the patient opportunities to ask other questions; for example, how does this new medication fit with their other medications and health conditions (e.g., allergies), plus discuss possible side effects and ways they could be managed. A systematic review has indicated that question prompt lists are effective interventions for improving communication and cognitive outcomes [36].

Third, expanding the information in the consult decision aid to respond to participants’ feedback must be considered cautiously. The consult decision aid needs to provide enough information to improve patients’ knowledge while avoiding too much information that can interfere with its usability within clinical consultations. When more detailed decision aids were compared with briefer decision aids in trials, patients had slightly higher knowledge but the differences were small [37]. A unique feature of this consult decision aid is its layered presentation of information with one main page and links to additional information. The main page provides brief information on options, helps clarify patients’ values for outcomes, and screens for decisional conflict to indicate unmet decisional needs. Current links provide information on definitions but could be expanded to provide links to more detailed sources of information. Providing layers of information allows patients after the consultation to obtain the amount of information they desire. Using internet-based delivery of the consult decision aids would enable easier access to other relevant information resources [38].

Fourth, efficient or automated processes could be developed to transfer information from Cochrane Reviews into the consult decision aid prototype and facilitate updates. After applying the prototype to 12 Cochrane Reviews, we established a set of considerations for selecting Cochrane Reviews suitable for creating consult decision aids. Next, these considerations need to be applied by people less familiar with patient decision aid development and they could be applied to non-Cochrane systematic reviews. Given current initiatives to automate systematic reviews [39], it may also be possible to automate the creation of these consult decision aids. The Cochrane Library is in the process of establishing automation for some processes. For example, Evidence Pipeline was initiated to redirect primary studies automatically to their respective review groups, in order to facilitate and streamline Cochrane Review updates [40]. As well, the PICOtron is being used to extract information from reviews to produce summaries of the evidence [41]. Given that Cochrane evidence is currently reported in 12 languages, there is the potential to create consult decision aids in multiple languages which would better address health policies mandating their use in clinical practice [42].

There are a few limitations and strengths to consider. The consult decision aids developed from Cochrane Reviews are limited by the amount of information presented in the reviews. For example, many Cochrane Reviews identified a limited number of options but this could easily be changed in reviews by, at a minimum, including a fuller list of options. Second, the evaluation from potential users (who happened to be familiar with the topic) was simulated and conducted as part of the development process. Hence, further evaluation is required to determine their effectiveness in clinical consultations and how it compares to a more detailed decision aid. Third, participants evaluated mocked-up word documents and ideally the consult decision aids should be hosted online to provide easy access to hyperlinked information and scaling of fonts. Fourth, the 12 consult decision aids were not reviewed by a panel of topic experts. Given that Cochrane Review teams include topic experts, it is possible to incorporate a topic panel review into the development process. A subsequent study could evaluate the added value of topic expert review by determining the accuracy of information presented in the consult decision aid with and without expert review. These 12 consult decision aids require further evaluation by potential users, ideally including users with a broader range of characteristics. Finally, the acceptability evaluation was based on the consult decision aid prototype applied to one Cochrane Review and some participants’ comments were specific to this clinical decision. Strengths were the various characteristics of participants including several with secondary school education only and the relevance of most of their feedback across decisions.

Conclusions

The consult decision aid prototype uses a one-page brief format for presenting evidence from Cochrane Reviews to facilitate shared decision making in the consultation. This prototype meets the proposed minimal qualifying and certifying criteria from the IPDAS Collaboration. Health consumers and Cochrane editorial staff found it was acceptable and rated it positively for usability. Several suggestions were used to strengthen the amount and display of information. Application of the prototype to a series of Cochrane Reviews demonstrated the ability to apply it to a range of different health topics and efficiencies in creating consult decision aids. Issues to consider are the primary aim of the decision aid (e.g., information plus support versus process-oriented), the amount of information to provide and/or link to, and ways to automate the process to further improve efficiencies in creating patient decision aids. Consult decision aids were easier to create when there was a summary of findings table, a plain language summary, and a treatment/screening decision with two or more reasonable options (including no treatment/screening). Seven circumstances were identified for determining Cochrane Reviews relevant for including consult decision aids. These considerations for embedding consult decision aids in Cochrane Reviews are likely transferable across other types of high quality systematic reviews. Future studies will evaluate implementation of consult decision aids.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM1 (PDF 133 kb)

It is feasible to develop a brief patient decision aid prototype to be used to facilitate shared decision making in the consultation and also meets the International Patient Decision Aid Standards qualifying and certifying criteria.

Several key characteristics of Cochrane Reviews make it easier and some circumstances make it more relevant for creating consult decision aids.

Developers need to balance providing adequate information on the options, benefits and harms while keeping the consult decision aid brief enough that it can be used in clinical practice.

Table 4

Summary of qualitative findings on the consult decision aid about alendronate as a treatment option for osteoporosis

Categories	Comments/suggestions	Response
Plain language	Use plain language (e.g., harm–benefit ratio, contraindications, decline)	Changed to simpler words
	Alendronate ‘may’ … is unclear	No change because this is accurate information
	Remove ‘reasons to’	No change; features are not always benefits
	Unsure if all will understand the information	No change; for use in the consultation
	Unclear what ‘best estimate’ means	Added hyperlink to more information
Print size	Add more space	Added more space on internet version
	Enlarge font for GRADE rating	No change; information is for the clinician
	Enlarge font overall	No change; can modify font with browser settings
	Make footer easier to read	Changed color from gray to black font
Logical layout/format	Need printable version	Electronic and paper versions are planned
	Improve designs for images	Images to be improved during publishing
	X in ‘decline the option’ is too strong	X used for years in DAs; changed color/font of X to balance with other icons
	Move ‘SURE test’ sooner	No change; selecting kept after comparing options
	Write from patient perspective (e.g. ‘I’ instead of ‘you’)	No change; for use in the consultation
Clarity of information provided	GRADE should be explained	Added more details in a hyperlink
	Unclear for ‘how much does this matter’	Added ‘to you’
	Osteopenia does not equal osteoporosis	Removed osteopenia as it was not necessary
Missing information	Interaction with other drugs	Added to list of other information to consider
	Long-term use of medicine: benefits/harms, changes in side effects over time	Added to list of other information to consider
	Significance of harms and outcomes	Added to list of other information to consider
	Confidence intervals	No change; not necessary for IPDAS criteria
	Allergies	Added to list of other information to consider
	Cochrane and development of DA	Added in user manual
	More information on side effects	Added to list of other information to consider
	Full list of options	Identified as a limitation
	List of questions to discuss with doctor	Added based on other information to consider
Too much information	If they have osteoporosis, do you need description in Part I	Required by IPDAS but not necessary to go through in a consult; added in manual on how to use it
	Need for data entry: if they are doing it with the doctor, why write it down?	For values clarification, it is better if the tool is interactive; made explicit in user manual
	Remove references	Required by IPDAS criteria; remained as hyperlink
Instructions	Be more explicit on how to use the links	Added to user manual and provided blue underlines
	Provide instructions on using it	Added in user manual

DA decision aid, IPDAS International Patient Decision Aid Standards