Reducing complexity of patient decision aids for community-based older adults with dementia and their caregivers: multiple case study of Decision Boxes.

OBJECTIVES: To identify patient decision aids' features to limit their complexity for older adults with dementia and their family caregivers.
DESIGN: Mixed method, multiple case study within a user-centred design (UCD) approach.
SETTING: Community-based healthcare in the province of Quebec in Canada. PARTICIPANTS: 23 older persons (aged 65+ years) with dementia and their 27 family caregivers.
RESULTS: During three UCD evaluation-modification rounds, participants identified strengths and weaknesses of the patient decision aids' content and visual design that influenced their complexity. Weaknesses of content included a lack of understanding of the decision aids' purpose and target audience, missing information, irrelevant content and issues with terminology and sentence structure. Weaknesses of visual design included critics about the decision aids' general layout (density, length, navigation) and their lack of pictures. In response, the design team implemented a series of practical features and design strategies, comprising: a clear expression of the patient decision aids' purpose through simple text, picture and personal stories; systematic and frequent use of pictograms illustrating key points and helping structure patient decision aids' general layout; a glossary; removal of scientific references from the main document; personal stories to clarify more difficult concepts; a contact section to facilitate implementation of the selected option; GRADE ratings to convey the quality of the evidence; a values clarification exercise formatted as a checklist and presented at the beginning of the document to streamline navigation; involvement of a panel of patient/caregiver partners to guide expression of patient priorities; editing of the text to a sixth grade reading level; UCD process to optimise comprehensiveness and relevance of content and training of patients/caregivers in shared decision-making.
CONCLUSIONS: The revised template for patient decision aids is designed to meet the needs of adults living with dementia and their caregivers better, which may translate into fewer evaluation-modification rounds. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The multiple case design allows suggesting general features for adapting patient decision aid templates to user needs.

Integration of user feedback in a user-centred design approach allowed an in-depth study of decision aid features influencing adoption of shared decision-making.

Caregivers offered their feedback on the Decision Boxes in the presence of the person in their care, and this could have influenced our conclusions.

The presence of caregivers may also have caused some of the seniors with dementia to be less spontaneous, particularly if their caregivers played a dominant role in the dyad.

Introduction

In 2015, 46 million adults were living with dementia worldwide. This number is expected to increase to 131.5 million by 2050.1 The medications available to treat dementia are of limited efficacy and can cause major side effects.2 Non-pharmacological alternatives may help with some symptoms, but patients, their caregivers and their primary healthcare professionals are less familiar with their benefits and harms.3 Moreover, patients experience adverse health outcomes and reduced autonomy and capacities, which result in difficult life management or non-medical decision.4 In such clinical situations, the shared decision-making model calls for healthcare professionals and patients to work together to reach joint decisions based on the best evidence available regarding the benefits and harms of all available options (including watchful waiting) as well as patient values and preferences with regard to those options.5

However, involving older adults living with dementia in decision-making may be a challenge given their cognitive decline. Their older age also puts them at greater risk that other factors limit their participation in decision-making compared with the general population, such as lower levels of literacy and numeracy,6 7 the presence of caregivers,8–10 sensory deficits such as deafness or visual impairment and a greater propensity to rely on healthcare professionals to make health decisions.11 Healthcare professionals may also perceive older persons with dementia as being too vulnerable to participate in decision-making, and thus exclude them from the process.12

Patient decision aids are standardised evidence-based interventions designed to help people make informed and deliberated choices among options.13–16 At a minimum, they provide information about the options and their associated relevant outcomes.15 An earlier systematic review described some features of patient decision aids to support understanding and values clarification in adults with limited health literacy skills, namely presenting essential information more prominently, adding videos to verbal narratives, presenting numerical information (1) in tables or pictographs (2) with the same denominator and (3) using higher numbers to display improvements.17 Health communication research also suggests several features to limit the complexity of health information, such as simple language and the use of pictures,18 and presenting actionable health information.19 20 However, no decision aid feature is known to support communication between adults with limited literacy and their healthcare professionals.17 There is also a lack of evidence regarding the features of decision aids to communicate uncertainty to adults with limited literacy,21 and health literacy is still rarely considered in the literature to date.17

Moreover, only a few studies so far have described the development of a decision aid for people living with dementia and their healthcare team.22–24 Hence, knowledge gaps remain on the most efficient design strategies and on the specific features of patient decision aids to meet the decision-making needs of this population. To begin to fill this gap, we have prioritised difficult decisions that older adults with neurocognitive disorders (NCDs) and their caregivers frequently face.4 We then synthesised the evidence of potential benefits and harms for all the options involved in some of those decisions and integrated them into Decision Boxes (DBoxes), which are the patient decision aids template developed at Laval University (Quebec City, Canada).25 26

Our previous results suggested that including user feedback is instrumental in designing DBoxes better adapted to their needs.25 Preliminary evidence indeed suggests that user-centred design (UCD) may enhance the implementation in practice of patient decision aids.27–32 UCD builds on an explicit understanding of users, tasks and environments to address the whole user experience.33 It is driven and refined by iterative user-centred evaluation, and involves a design team with interdisciplinary skills and perspectives.33 User experience looks broadly at the individual’s interaction with a product, as well as the thoughts, feelings and perceptions that result from that interaction.34

In the current study, we used a UCD approach to tailor three DBoxes to the literacy level of older adults with dementia and their caregivers, and improve their experience using them. As we observed users interacting with the DBoxes, we sought to identify patient decision aid features that reduced their complexity and prepared patients and caregivers to participate in shared decision-making.

Methods

Study design and approach

We used a multiple case study evaluation across three DBoxes. The evaluation comprised interviews and questionnaires within an iterative UCD approach. We used three rounds of questionnaires and semi-structured interviews to have patient and caregiver dyads evaluate the three DBoxes (figure 1). After a first evaluation round with a subsample of nine dyads, we analysed the data and, based on the findings, refined the DBoxes to limit their complexity and improve the user experience. We then used the same evaluation/tailoring process again in two more rounds, with new participants each time.

Figure 1

User-centred design process of three Decision Boxes (DBoxes) for seniors with dementia and their caregivers.

Case selection

Based on an earlier Delphi study,4 and using a rapid review approach,35 we created five DBoxes to support decision-making regarding five difficult and frequent decisions faced by older adults with NCDs and their caregivers: (1) choosing a non-pharmacological treatment to manage agitation, aggression or psychotic symptoms; (2) deciding whether or not to stop driving following diagnosis; (3) deciding whether or not to prepare a power of attorney, called a Protection Mandate in Quebec (Canada) covering health, property and financial matters; (4) choosing a support option to decrease caregiver burden and (5) choosing an option to improve quality of life. For the current study, we used a maximum variation sampling strategy to select three of these five DBoxes as different cases. We chose #1 because it was the longest of the five and compared several options, whereas #2 and #3 compared only two options each. We chose #2 because it covered a very sensitive topic and thus allowed identifying features of decision aids to facilitate shared decision-making in emotionally charged contexts. We selected #3 because it used a more technical and complex vocabulary than any of the others. We excluded #4 because caregivers were the target users.

Theoretical framework

We chose the Health Literacy Skills framework36 to structure data collection and allow comparability across cases. This framework hypothesises the relations between health literacy and health-related outcomes and reflects how factors external to the individual (eg, family, setting, community, culture and media) influence the constructs represented. The framework incorporates health-related stimuli that people receive in their daily life, such as the DBox. According to the framework, after exposure to a stimulus, the health literacy demand of the stimulus interacts with a person’s health literacy skills to influence comprehension of the message. Health literacy demand is defined as the complexity and difficulty of a stimulus, and it was the focus of the data collection in the current study.

Population and sampling strategy

All healthcare professionals from 11 outpatient geriatrics clinics in the Quebec City area, Canada, were invited to participate in this project. Those who agreed were asked to identify patients (aged 65+ years) diagnosed with dementia of any severity among their clientele, and the patients’ informal caregivers. They contacted those patients or their caregiver, asking permission for the research team to contact them and explain the project. The research team then followed up with each willing patient or caregiver. We aimed to recruit 27 patient/caregiver dyads, a large enough sample size for this type of testing.37

Study procedure

Decision Box prototypes development

We used a rapid review approach35 and the Ottawa decision support framework38 to create DBoxes prototypes that respected the international standards for patient decision aids.14 The DBoxes provided information on the health problem of interest, included an exercise to help patients and caregivers clarify what mattered most to them, explained the probabilities of experiencing benefits or harms for each of the available options and listed resources to guide those experiencing decisional conflict. Between two and four experts in the care of older adults with dementia (among healthcare professionals, informal caregivers, managers, representatives of community-based organisations devoted to these seniors or clinical researchers involved in the organisation of primary care or services delivered to seniors with dementia) reviewed and validated each DBox.

Data collection

Nine patient/caregiver dyads were randomly selected as a subsample of all participants at each round and randomly assigned one of the three DBoxes (DBox #1 to #3) studied, for three dyads/DBox at each round (figure 1). Copies of the DBox were sent to study participants about 1 week prior to the interviews for them to review first. A trained moderator, a nurse trained in geriatric care (GB)—the same for all participants—then met them at their homes for data collection. Patients and the caregivers initially completed a questionnaire comprising questions on sociodemographic characteristics (age, gender, race, marital status, education level, income) and the type and duration of the relationship between caregiver and patient. In cases when patients were unable to complete the questionnaire, the caregiver completed it on their behalf. Then, using an interview guide, the moderator assessed participants’ opinions of the strengths/areas of improvement of the DBoxes in fostering a shared decision-making behaviour, which was ‘to express their priorities to their healthcare professional regarding the decision to be made’. The moderator also asked participants for suggestions to improve the DBoxes.

The moderator initially addressed all questions to the patient. If the patient did not participate actively in the discussion, then the moderator systematically sought the caregiver’s suggestions (1) on how to get the older person to express their opinion and (2) how to modify the document to facilitate use by the older person. The caregivers’ own perspectives on the strengths/weaknesses of the DBox were also welcomed.

At the end of the session, able patients and caregivers completed a self-administered questionnaire, comprising: (1) the Chew three-item health literacy scale39 adapted to French (personal communication, Holly Witteman, Laval University), (2) the patient version of the Information Assessment Method (IAM) for assessing the value of information,40 (3) eight items built from the Technology Acceptance Model (TAM-2),41 to assess how useful and easy to use the DBox was in expressing their priorities to their healthcare professional regarding the decision to be made and (4) level of satisfaction with the DBox on a 5-point smiley-face rating scale ranging from 1 (sad face) to 5 (smiling face).

The moderator took written notes during and after the interviews to describe non-verbal communication and interactions between the older person and his or her caregiver.

The interviews were recorded and transcribed verbatim.

Analysis of participants’ experience between rounds

To tailor the DBoxes between rounds, the transcripts and notes were entered as project documents into specialised software (NVivo V.10, QSR International, Cambridge, Massachusetts, USA). Two researchers (GB, AMCG) analysed the data using deductive/inductive thematic qualitative data analysis, first by searching for factors set out in the Health Literacy Skills framework,36 then by integrating any new themes that emerged from the data. To this end, the two researchers reviewed the interview transcripts separately. They then compared their results and came to a consensus on a list of themes. They noted these themes in a codebook, labelled and defined them, and entered them in NVivo as nodes. One of the researchers (GB) then applied these preliminary codes to all the interview transcripts. Coding was updated as necessary, and the second reviewer (AMCG) checked the new codes to ensure consistency with the chosen framework.

Tailoring the DBoxes

To tailor the DBoxes, we assembled an expert panel that comprised graphic designers, a healthcare professional specialised in the care of older adults (GB), knowledge translation researchers (AMCG, HOW, GB), a human factors engineer (HOW) and on one occasion, a health literacy expert (EF). Between each round, this expert panel met to review the qualitative and quantitative findings and tailor the DBoxes to improve the participant experience, that is, (1) limit their complexity, (2) add any missing information and (3) ensure that participants felt more empowered to express their priorities to the healthcare professional regarding the decision to be made.

We used the same evaluation/tailoring process after each of the three round.

Quantitative analysis and triangulation

We completed a descriptive statistical analysis of the questionnaire data at the end of the study using SAS (V.9.4, SAS Institute). We then interpreted the results in light of the qualitative findings to understand further which factors would make it easier for patients and caregivers to express their priorities to their healthcare professional. We further synthesised insights from individual case studies in a cross-project analysis to reveal a pattern of findings across all cases.

Patient and public involvement

A caregiver to a person living with dementia (JB) participated in the study as a coinvestigator. This person participated to the study design and contributed in the development of the DBoxes by providing critical feedback before user testing.

Ethical issues pertaining to human subjects

When persons living with dementia were deemed able to provide consent by their healthcare professionals, they were directly invited to participate. To ensure minimal risk to the health of incapacitated adults, the research team sought informed consent from the caregivers of patients who could not themselves provide informed consent, in conformity with the Civil Code of Quebec.

Results

Participant characteristics

Healthcare professionals from six ambulatory geriatric clinics out of the 11 invited to participate recruited patients and their caregiver. These healthcare professionals invited 34 patients to take part, 23 of whom accepted (74%). Either one or two caregivers accompanied the participating patients, for a total of 27 participating caregivers.

Most of the 23 patients were aged 85 years or more and had a high school education (table 1A). They reported a mean level of health literacy competency of 2.0±SD of 1.5 (on a scale of 0–4, with 4=low literacy). We did not have access to their medical records, but the moderator—a registered nurse—qualitatively classified the severity of their dementia as moderate (n=9), severe (n=9) or very severe (n=5).

Table 1

Demographic characteristics of participating (A) patients, and (B) caregivers

(A) Patients
Characteristic	Frequency (%) n=23
Female	13 (57)
Age (years)
65–74	0
75–84	8 (35)
85 and more	15 (65)
Education
No education	0
Elementary	5 (22)
High school	11 (48)
College	2 (8.7)
University	5 (22)
Income (US$)
0–24 999	12 (52)
25 000–34 999	6 (26)
35 000–49 999	1 (4.3)
50 000–74 999	3 (13)
75 000–99 999	1 (4.3)
100 000–150 000	0 (0)
150 000+	0 (0)
Health literacy
Self-reported frequency of having someone helping read medical materials
(0) Never	7 (30)
(1) Occasionally	4 (17)
(2) Sometimes	4 (17)
(3) Often	4 (17)
(4) Always	4 (17)
Mean (±SD)	1.7 (±1.5)
Self-reported confidence with forms
(0) Extremely	4 (17)
(1) Quite a bit	4 (17)
(2) Somewhat	3 (13)
(3) A little bit	3 (13)
(4) Not at all	9 (39)
Mean (±SD)	2.4 (±1.6)
Self-reported problems learning about medical condition because of difficulty reading medical materials
(0) Never	5 (22)
(1) Occasionally	7 (30)
(2) Sometimes	2 (8.7)
(3) Often	4 (17)
(4) Always	5 (22)
Mean (±SD)	1.9 (±1.5)
Marital status
Single	1 (4.4)
Married or common-law partner	11 (48)
Widow	11 (48)
Separated	0 (0)
Divorced	0 (0)
Living with caregiver, n (%)	11 (48)

Most of the 27 caregivers were aged between 45 and 85 and had completed undergraduate degrees (table 1B). Caregivers reported a mean level of literacy competency of 0.8±SD of 1.1.

Research processes

Before the interviews, several caregivers offered suggestions on how to reduce the emotional burden of the information on the patient, such as changing some words in a sentence or adding pictures. As caregivers were the ones who knew the person best, they were able to warn the moderator to avoid certain subjects to limit the person’s distress or anger (eg, driving abilities).

Factors influencing adoption of shared decision-making

Despite the main interview focus on identifying patient decision aid features influencing their complexity, participants reported additional factors influencing their adoption of shared decision-making. These factors were divided among individual moderators, professional practice mediators, social environment mediators and healthcare organisation mediators (figure 2) in keeping with the Health Literacy Skills framework.36 Additionally, despite the main interview focus on participants’ intention to express their priorities to their healthcare professional regarding the decision to be made, participants spontaneously discussed several other shared decision-making behaviours, which are listed in figure 2.

Figure 2

Factors influencing adoption of shared decision-making behaviours by patients/caregiver dyads, adapted from Squiers’ Health Literacy Skills framework.

Factors influencing the complexity of patient decision aids

Factors that were found to influence the complexity of patient decision aids were structured under three main themes: informational content, visual design and values clarification (listed in figure 2). The next sections describe these factors as they were brought up during the interviews and the features or strategies that were proposed to limit complexity and improve the user experience. A detailed description of these features and strategies are also listed in table 2. Supplementary files #1 and #2 respectively present samples of the initial and final versions of one of the three DBoxes studied.

Table 2

Features of patient decision aids and design strategies to limit their complexity and improve users’ experience

Finding	Feature or design strategy to limit issue
Unclear purpose/topic of the patient decision aid	Clear statements in larger fonts describing who the decision aid is aimed at, and what it aims to achieve. Pictogram or images showing people using the patient decision aid to represent its purpose. Personal story displaying the context of use, and purpose of the patient decision aid.
Irrelevance of patient decision aid to people with dementia	Recognising caregivers’ role in decision-making through explicit statements that the patient decision targets caregivers equally to patients.
Arduous read or unclear content	Systematic and frequent use of high-quality pictograms to illustrate text. Glossary to define complex terminology. Write out the text at a sixth grade reading level. Removal of the references within the text; reference list included on the DBox website. Use ‘priority’ instead of ‘preference’.
Missing information on the options	Detailed and comprehensive description of each option. For the more complex options: propose personal stories displaying a person going through the option.
Missing information on the next steps to implement the selected option following decision-making	Contact section listing contacts, resources and available services to implement each of the option.
Missing topic-specific information, irrelevant content	Use a user-centred design process until information needs are met and all sections are perceived as relevant.
Quality of the evidence	Offer information on the quality of the evidence to those interested.
Challenge using the Likert rating scales in the values clarification exercise	Asking users to select a single preference in a checklist; avoid rating scales.
Proposing meaningful priorities in the values clarification exercise	Involve a panel of patient/caregiver partners at start of development with mandates to (1) propose a list of priorities to guide the literature review and (2) revise the wording of priorities extracted from the scientific evidence (further evaluation required).
Navigation challenges	When the patient decision aid comprises more than two options, use the values clarification exercise to streamline navigation by inviting users to read more on the options meeting their priorities. Use textual cues to refer to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) and to the glossary. Use visual cues (eg, pictogram of the options) and colours to structure the general layout.
Irrelevance of scientific evidence to patients/caregivers	Explain the targeted shared decision-making behaviours in text and, if possible, with pictures. Train patients/caregivers in shared decision-making to prepare them to review information on the benefits and harms of the options (further evaluation required).

Informational content of the DBoxes

Purpose/topic

Clarifying the purpose of the DBoxes was a more important concern in the first and second rounds, as participants devoted more time to commenting on these aspects. They often did not understand what the DBox was intended for, as illustrated in this comment from a caregiver:

My father was starting to experience mild dementia, and when he read the document he got stressed thinking he would be evaluated. I knew he had read it, because he talked about it to me and I knew this is what he was talking about. (Caregiver #6, DBox #2)

Several participants could not understand who the DBoxes were aimed at:

That has nothing to do with us. It’s not for us (caregiver) or the patient. So who is it for? (Caregiver #9B, DBox #1).

The team prioritised this issue and consequently added two statements to improve understanding—in large font at the top of the first page—describing whom the document was aimed at and what it was supposed to achieve. A pictogram was also added showing a person reading a printed document to represent the purpose of the DBox. In the last version (Supplementary file #2), these features were emphasised even more by isolating them on a separate cover page, but we could not test this new layout as it was added in the last version. Personal stories were also added to the prototypes, to provide context for the DBox and its use and purpose. Participants appreciated the stories, which increased their interest in the content. They also mentioned that the stories helped them relate the content to their personal situations, as described by this patient:

The disorder that this man has (note from the author: the patient is referring to a character in the personal story), the memory loss and other memory problems… I thought about all that, my memory’s slipping away, I hope it’s going to be awhile before I lose it completely. (Patient #23, DBox #2)

Missing information

In the first evaluation round, several participants formulated requests, often several, for very specific information missing from the DBox. They asked for more information on the health problem itself, on how it was assessed, on options that were not quite clear, on specific outcomes to an option or on how to implement an option. Some of these questions were too specific to generate a change in the DBox template, and we could generally address them easily by adding to or modifying the text. For example, one caregiver asked for more information on driving skills assessment:

When you’re driving and you can’t see the street name, then you can get lost. It’s a physical issue (that makes driving dangerous), because your eyes are not working properly. But when does dementia make it dangerous to drive? (Caregiver #11, DBox #2)

Most of these issues were resolved after the first round.

Some of the requests for more information could be grouped together into broader categories and the template modified accordingly, so that future DBox will have the missing information, before user evaluation. First, we added a detailed description of each option to the DBox template to address the needs for more information on the options. Second, we added a section listing contacts and resources (Supplementary file #2), to provide information on the services related to implement the options. People regularly asked whom to contact about the less familiar options, such as music or massage therapy, as illustrated in the following discussion between two caregivers:

Caregiver #1: ‘[…] Perhaps something you could add here would be ‘refer to such and such a social worker, psychologist, psychotherapist’ but for psychosocial needs, maybe you should indicate how to find those resources, which resources and where. That would be really important. (Caregivers #9A, DBox #1)

Participants reported fewer issues with missing information as the rounds progressed. Most mentioned that they would use the information and that they were better equipped to discuss the health issue with someone, as demonstrated by this quote:

But I think that document is more to know what’s best for the person’s well-being: drugs or no drugs. … And there are options if you don’t want to be taking drugs. That’s the purpose of the document. … In that regard I find the document to be complete. (Patient #1, DBox #1)

Relevance

In every round, several participants mentioned that they found the DBoxes to be relevant, credible, interesting or useful. They explained how the DBoxes provided good information and made them aware of the options available:

It’s well presented because we can see the advantages and disadvantages next to each other. It would definitely help someone to decide who has not made their mind up yet. It gives you all sides of the coin for the various options we have. It’s important to plan ahead as you get older. (Caregiver #15, DBox #3)

They also mentioned that the DBoxes gave them something to think about:

It gets you thinking. It really gives you a good idea. If you think ‘I want more protection, I don’t want to be taken advantage of or something’, then that tells you to draw up advance directives. It points you in the right direction.(Patient #8, DBox #3)

On the other hand, participants found some parts to be of little relevance, use or interest. A number of caregivers mentioned that the DBox was of no use to patients who could not participate in the decision-making process because of cognitive decline. To address this issue, the DBox template was modified to give the caregiver equal billing as a target user, specifically on the cover page in the bottom line (Supplementary file #2, p. 2) and in the values clarification exercise (Supplementary file #2, p. 4).

Other participants’ comments were to the effect that the information on evidence, GRADE ratings and benefit and harm probabilities was for clinicians and was irrelevant to them, as this quote shows:

Anyway, I mean this page, to be practical—page 4—the harms, I would take it out altogether. I would leave it up to the clinicians to read that. We have no use for the studies that’ve been done. (Caregiver #9, DBox #1)

No strategy was found to address this in the DBox template. In the first prototype, the DBox already recommended that the healthcare professional share the decision with the patient, so we emphasised this in the last version of the template by using a visual representation of SDM and by putting the message in a box in the last version to highlight it (Supplementary file #2, p. 3).

Other comments were more topic-specific and could not be addressed by adjusting the DBox template. For example, the introduction to DBox #2 (Stopping driving) described the healthcare professional’s responsibilities with regard to driving assessment. Patients and caregivers considered this irrelevant.

Take the section aimed at the healthcare professionals […], in any event, I think you probably shouldn’t put the emphasis on that, or at least you should make the part that really concerns the patient bigger so that they can really concentrate on what’s essential. (Caregiver #6, DBox #2)

Clarity of content

The participants consistently mentioned the complex terminology and sentence structure as important barriers, as this quote shows:

Physical activity, touch therapy, music therapy, aroma therapy: for me, sure, but for them at their age, I’m not sure they know what they mean. Those who don’t have much education definitely don’t know what these are. (Caregiver #13, Dbox #1)

To address this issue, we added a step to the development process, before user testing. It consists in checking the text reading level with online freeware (eg, https://www.webpagefx.com/tools/read-able/, WebpageFX USA) so that it corresponds to a sixth grade level. We also added a glossary to the DBox template, to define more complex terms for which we could not find simpler synonyms. The notation ‘see definition in the glossary on p. x’ in parentheses after the term referred users to the glossary. In the last version (Supplementary file #2), the word was also highlighted and underlined.

Participants also commented on the risk communication strategies used. One person thought that percentages were obvious and did not understand that the DBox also used natural frequencies:

It reads ‘Out of 100 people, 33 will experience stress symptoms'. People are going to think 33% rather than to keep the two figures separate. … It depends on the person, but a dog is a dog. You don’t need to draw it, you can just write ‘dog’. (Caregiver #20, DBox #3)

In addition, participants were often surprised by the harms (eg, that stopping driving led to an increased risk of dying). As mentioned above, some respondents thought that this type of content should be provided to clinicians and that it was irrelevant to them.

Several participants did not understand the GRADE scale. They noted the lack of explicit link between the GRADE rating beside the evidence and the legend at the bottom of the page. To address this, the DBox template now has an invitation to ‘see legend’ in parentheses after each GRADE symbol. One participant suggested using green, yellow and red instead of the ‘+’ for a more intuitive representation of the quality of the evidence. This idea was not implemented because the DBox needed to be usable in a black and white version.

Several participants grasped the essence of the GRADE scale displaying the quality of the evidence, as this quote shows:

No, it’s all very clear. And also when you look at the legend, it’s even easier to see how advanced the research is from that viewpoint. Some are quite obvious, but others you can see don’t have a lot of data from what is shown. (Caregiver #16, DBox #3)

Visual design

Participants shared several recommendations on the DBox visual design, which we further categorised into layout, pictures, colour and font (figure 2).

Layout

‘Layout’ comprised participants’ comments on navigation, organisers, length and density. Navigation proved difficult in the first rounds as several participants became lost when they moved from one section to another, such as between the health options and the contact section (located at the end of the document). We thus adjusted the DBox template by adding organisers and textual cues to the related section (eg, ‘see glossary’, ‘see the list of resources on page x’).

We also removed the reference list, as the reference numbers in the text caused confusion. We chose to provide the reference list on the DBox website instead for those (eg, caregivers) who might be interested in reviewing the sources of evidence.

Participants also recommended adding a table of contents at the beginning of the document to help navigation, and to allow choosing the sections relevant to them, as this quote shows:

Maybe say, ‘There are a number of activities available. Choose those that interest you’. Or have an index like in a book: animal therapy, see page 3. Because maybe they felt obligated to see all the activities and read them all. Maybe it was confusing. Maybe give them the option, or say, ‘Here are several activities that can reduce aggressiveness…’ (Caregiver #7, DBox #x)

To meet this need, we moved the values clarification exercise to a position right after the ’Introduction' section. This allowed people to choose their priority first, and then read only about the options meeting those priorities, instead of having to read about all the options. In addition to improving navigation, this also shortened the time required to read the document.

DBox #3 (power of attorney) received several comments on how complex the information was and how hard it was for respondents to compare the three legal options. After receiving several questions from participants on the legal concepts in the first version of the DBox, we added more information in the second round, which upped the number of pages from 5 to 9 in round #2. Users disliked this longer version, as the quantitative finding below shows, despite the fact that it was much more comprehensive. In the third version, we therefore clarified this information in a large table setting out the legal implications of the various options (Supplementary file #2). This addition improved users’ experience, made the DBox shorter and reduced their complexity.

Pictures

In the first round, several people found the DBoxes hard to read because they consisted mainly of text. Some people, such as this caregiver, suggested adding pictures to make things more engaging:

I don’t think you can get away from having text, but maybe not so much detail… Maybe you could add some pictures… maybe a little drawing, a car in an accident, to provide illustration and so that people don’t have to concentrate so hard to read the sentence right to the end. (Caregiver #6, BD #2)

In the second round, we therefore added pictograms strategically to draw attention to the most important text. These generated positive comments, but we did not use them systematically, nor consistently, and some participants found them unclear. In the last DBox template that has not yet been evaluated (Supplementary file #2), we systematically added pictograms specially designed by a graphic designer to illustrate (1) the purpose of the DBox on the cover page, (2) each of the activities covered in the DBox (Introduction, Clarifying priorities, Exploring the options, Choosing an option, Glossary), (3) the bottom line on page 2, (4) each option, and (5) the Contacts section.

Colours

Participants found the DBoxes’ colours attractive and helpful to facilitate reading:

It’s well presented. Colourful documents like that are attractive and nice to consult. It’s also easy to spot information from page to page. Benefits and harms can be compared side by side and the other. I really liked the document. (Caregiver #15, DBox #3)

Values clarification exercise

The values clarification exercise underwent major changes throughout the study. Most participants did not understand the first version of the exercise, which was designed using the Ottawa Decision Support Framework.38 We initially observed that a number of participants did not understand the meaning of the word ‘preference,’ so we replaced it with ‘priorities’. This resolved the issue and improved participants’ understanding of the purpose of the exercise.

Additionally, the exercise asked users to assess the extent to which each priority mattered to them, using a 6-level Likert scale. Some people perceived it as too detailed, and some caregivers mentioned that patients might have difficulty using this scale, as they were unfamiliar with rating scales:

Well, it’s my generation but not his [speaking of the person in his care], so when I look at this form, I look which side is important and which side is not important, then Bingo I complete it…. But for him… (Caregiver #6, DBox #2)

After the first round, we thus removed the Likert scales and used a checklist instead, asking people to select a single priority from a list (Supplementary file #2, p. 4). For each item on the list, the DBox then proposed a list of options for meeting that priority, and the page number to view evidence of that option’s benefits and harms. These modifications improved participants’ understanding of the purpose of the exercise—participants reflected more on the benefits versus harms, and before and during the interview they talked more about their priorities for that decision. For example, with this new format, some people understood perfectly well that they had to choose what mattered most to them among the list of items, but they were reluctant to do so considering the decisional conflict they had to face:

If I have to choose only one—live longer or maintain a good health—who wouldn’t pick to live as long as possible? For sure, I would. Or stay in good general health, of course, I would pick that. I wouldn’t go saying I don’t want to drive anymore. So I don’t know. It’s a strange question. (Caregiver #16, DBox #2)

We revised the sentences expressing each of the priorities several times, as participants considered some to be unclear or illogical.

Quantitative results and triangulation with qualitative findings

The quantitative data from the questionnaires suggest that participants’ perceptions of the DBoxes were generally positive, with TAM-2 mean scores mostly above 4 (scale of 1–7, with 7 indicating positive perceptions), and satisfaction mean scores above 3 (scale of 1–5, with 5 indicating positive perceptions) (figures 3A, B, 4A, B and 5A, B).

Satisfaction, ease of use and usefulness of the Decision Box #1 (agitation, aggression, psychotic symptoms) as evaluated by (A) older people with dementia and (B) their caregivers.

However, the patients who evaluated DBoxes #2 (driving) and #3 (power of attorney) in round #2 reported lower satisfaction and lower perceptions of DBox usefulness than did the patients in rounds #1 and #3 or the caregivers (figure 4A, B, figure 5A, B). They also gave lower understanding and relevance scores on the IAM questionnaire (table 3). During the interviews, these five patients commented repeatedly on the lack of clarity of the content and on the inappropriate terminology in the DBoxes, for example:

Figure 4

Satisfaction, ease of use and usefulness of the Decision Box #2 (driving) as evaluated by (A) older people with dementia and (B) their caregivers.

Figure 5

Satisfaction, ease of use and usefulness of the Decision Box #3 (power of attorney) as evaluated by (A) older people with dementia and (B) their caregivers.

Table 3

Patients’ report of the value of Decision Boxes per round: ratings based on the patient version of the Information Assessment Method40

	Frequency (n/N)
	Round 1	Round 2	Round 3	All round
Relevance
This information is…
… totally relevant	3/9	0/7*	0/5*	3/21
… relevant	6/9	4/7*	5/5*	15/21
… somewhat relevant	0/9	1/7*	0/5*	1/21
… irrelevant	0/9	2/7*	0/5*	2/21
Understanding
They understood this information…
… completely	3/9	3/7*	1/5*	7/21
… mostly	4/9	1/7*	2/5*	7/21
… poorly	1/9	2/7*	2/5*	5/21
… not at all	1/9	1/7*	0/5*	2/21
Cognitive impact of the information
They learnt something new	4/9	0/8	2/6	6/23
This information allowed them to confirm what they do, or did	5/9	3/8	2/6	10/23
They are reassured	6/9	1/8	3/6	10/23
They were reminded of something they already knew	5/9	1/8	1/6	7/23
They are motivated to learn more	4/9	2/8	4/6	10/23
There is a problem with the presentation of this information	5/9	1/8	2/6	8/23
They disagree with the content of this information	0/9	0/8	0/6	0/23
This information is potentially harmful	1/9	1/8	0/6	2/23
Information use
They will use this information  This information will…	5/9	4/8	2/6	11/23
…help them improve their understanding of a particular issue and make a decision	3/5	1/4	0/2	4/11
…help them do something when they did not know what to do	2/5	0/4	0/2	2/11
…convince them to do something that they already wanted to do	1/5	1/4	0/2	2/11
…allow them to change the way they do something	2/5	1/4	0/2	3/11
…allow discussing something with someone else (a relative or a healthcare professional)	5/5	2/4	1/2	8/11
Expected benefits of the information
They expect the information to help…
…be less worried	2/5	0/4	0/2	2/11
…be better equipped to discuss something with someone else (a relative or a healthcare professional)	4/5	3/4	2/2	9/11
…have more confidence in deciding something with someone else (a relative or a healthcare professional)	3/5	1/4	1/2	5/11
…handle an issue	2/5	1/4	0/2	3/11
…prevent an issue (or prevent it from getting worse)	4/5	0/4	0/2	4/11
They expect no benefits	2/5	0/4	0/2	2/11
Expected consequences
They expect that the use of this information can have a negative impact on their well-being of their health	1/5	1/4	0/2	2/11

*One missing data.

I have trouble understanding. When you start talking about legal stuff, you lose me. (Patient #15, BD #3)

The participants in the first round and second rounds frequently reported problems with the information, but these were mostly resolved in the third round (tables 3, 4: cognitive impact of the information). Only three people—one caregiver (#23) and two patients (#22, #19)—still reported problems with the information in round 3. The interview transcripts reveal that these patients only talked about how hard they found the text to understand, how long it was or how much trouble they had remembering what they had just read. As for the caregiver, he commented that he thought it was inappropriate for the DBox to list taxicabs or public transit as resources for people who need to stop driving due to dementia:

It’s okay for people who don’t have driver’s licences anymore, but still want to go out and still have the ability. But there’s no way she could ever take a cab or assisted transport. It depends on how serious your cognitive loss is. … At first, things were fine. She would take cabs to go places. She would travel to and from her sister’s by cab. But I’m not sure I’d let her do that now. There are lots of options, and they help lots of people, but it depends on how bad your memory loss is. (Caregiver #23, BD #2)

Patients and caregivers most frequently reported in the questionnaire that they expected the information to help them be better equipped to discuss something with someone else, to have more confidence in deciding about something with someone else and to prevent an issue (tables 3 and 4).

Caregivers’ perceptions of DBox #3 (power of attorney) were also generally observed to improve as the rounds progressed (figure 5B).

Discussion

We studied which features of patient decision aids may limit their complexity and improve the experience of older adults living with dementia and their caregivers as they prepare for shared decision-making with healthcare professionals. We described a series of practical features and design strategies to improve the user experience of patient decision aids and limit the number adjustments required during user testing. Some participants considered scientific evidence to be for healthcare professionals’ use only and thus had no interest in such content. Other participants understood concepts related to the quality of scientific evidence and methodological biases. Quantitative measures allowed collecting distinct feedback from patients and caregivers.

Strength and weaknesses of the study

A strength of this research project lies in its multiple case design that allowed suggesting general features for adapting patient decision aid templates to user needs. Integration of user feedback in a UCD approach also represents a strength of this study. However, caregivers may have been less spontaneous in some of their opinions because of the presence of the person in their care, and this could have influenced our conclusions. The presence of caregivers may also have caused some of the seniors with dementia to be less spontaneous, particularly if their caregivers played a dominant role in the dyad. On the other hand, inviting caregivers to reflect on the changes needed for the patient decision aids to address the needs of the person in their care provided an opportunity to benefit from caregivers’ own expertise in dementia while still receiving direct input from the patients themselves.

Relation to other studies

This is the first study focused on identifying general design features to tailor patient decision aids to the needs of seniors with dementia. It is also unique in its focus on the primary care offered to older persons living with dementia. Indeed, we found only three other reports on the impacts of patient decision aids, which targeted caregivers of older adults with advanced dementia recruited in nursing homes or acute care settings.23 24 42 Their development and evaluation were guided by the Ottawa Decision Support framework,38 which the authors adapted minimally. In one study, they added a tutorial regarding study design and the strength of evidence to the patient decision aid, due to limited high-quality evidence regarding the proposed options.24 In the other, they specified revising the printed decision aid to a sixth grade reading level, and sizes 16–20 font,23 which agrees with our proposed features.

Other web-based support tools for this population have been studied, but did not qualify as patient decision aids.43–45 For example, the FIT and DEM-DISC tools addressed a major gap, by helping caregivers of older adults living with dementia in clarifying their most pressing needs and identifying supportive products and services meeting those needs.43 44 This converges with the current study findings that patient decision aids for this population should provide a list of the available resources to support users in their implementation of the selected option. Interestingly, among the three designs evaluated for FIT, caregivers and healthcare professionals appreciated the exercise that used ticking of checkboxes the most, as it was more familiar to them and allowed people to view all the choices at once.43 Another such tools worth mentioning is the web-based DecideGuide that served to support communication and step-by-step shared decision-making among the network of people involved in the care of the person living with dementia.45 However, most of the challenges reported by users of the DecideGuide concerned web-based and interactive aspects of the tool, and are not applicable to a printed patient decision aid.

Interestingly, some of the current study participants understood the GRADE level of confidence display, which provides a deeper understanding of the uncertainty associated with each risk estimate. GRADE ratings communicate one of the types of uncertainty related to the outcomes of medical interventions, which is the ambiguity about the strength or validity of evidence about risks,46 also named ‘epistemic uncertainty’.47 Despite the influence of uncertainty on patients’ choices, there is considerable variation in approaches to communicate it in the patient decision aids currently available, and more than half do not communicate epistemic uncertainty.21 The current project thus adds observations on the potential of the GRADE strategy to communicate epistemic uncertainty to patients/caregivers. Future research should look into how GRADE ratings are understood by patients, and how they influence patient preferences.

Participants requested that we enliven the documents by adding pictures. Research on health communication suggests that combining well-designed pictures with written or spoken text enhances attention, recall of health education information and understanding, especially among groups with lower literacy.48 Pictures should illustrate key points, be accompanied by text using simple language, be simple to minimise distracting details and be selected with people from the intended audience to ensure cultural relevance.48 While several studies focus on the inclusion of graphs in patient decision aids to improve risk understanding,49 50 fewer focus on the impacts of other types of pictures on user experience of these tools. Importantly, a recent qualitative study described how pictures conveying important and detailed information combined with icon arrays in a patient decision aid were perceived as explanatory and easy to understand by women of low socioeconomic status.27 There is also some evidence on the pictures characteristics to support informed decision-making,51 but high-quality evidence on the impacts of pictures—and their key features—on decision quality is still lacking.

Our findings also provide new evidence on the factors potentially influencing the health literacy demand of a stimulus, recognised in the Health Literacy Skills framework36 as influencing comprehension of the message. Our practical descriptions of content and visual design features of health-related stimuli allow shifting the focus away from the person targeted, towards improvement of the stimuli to reach wider audiences.

Meaning and implications of findings

Some comments by participants were useful to suggest features to be used systematically so that the same issues would not come up again when new patient decision aids are developed. On the other hand, some of the reported issues cannot be addressed generally, as they were specific to a given health problem, intervention or outcome. UCD thus remains essential to help users clarify their needs. For example, some topics require the use of technical terminology, such as BD #3 on the power of attorney that included a lot of legal terminology. Some topics may also elicit a very strong emotional response, which might be impossible to foresee and may lengthen the design process.

The glossary proved essential for patient decision aids targeting older adults with dementia and their caregivers, but navigation to access it was a challenge. Web-based decision aids should use pop-ups or other methods to provide definitions without further navigation.

When patient decision aid templates require major changes to address topic-specific issues, developers should plan one or several additional evaluation rounds. In the current study, the DBox on the power of attorney required adding much information on the various legal options in the second round, generating lower scores before the table comparing the various options and their features were incorporated in the third round.

The wording used to express priorities required several rounds before we were able to arrive at the best expression, which suggests that the list of priorities would benefit from the input from a patient/caregiver expert panel. Moreover, the panel could be involved early in the rapid review process, to help identify those priorities, accelerate the review process and streamline the literature search to identify patient priorities early on.

Having questionnaires made it possible to collect patients’ perceptions independently from their caregivers’. This might be a good way to ensure that patients with caregivers nevertheless make their preferences known and questionnaires could be validated in this regard specifically with people with dementia.

Unanswered questions and future research

Several participants questioned the provision of scientific evidence on the benefits and harms of the available options in the DBoxes. Such comments might reflect a negative attitude toward the shared decision-making approach, but more likely demonstrates a lack of understanding of shared decision-making principles, namely that patients should be informed of the benefits and harms of each option to partake in decision-making. This suggests that patient/caregiver education is required to change attitudes towards shared decision making (SDM) before patients and caregivers can really partake in shared decision-making. Training is a promising strategy to address this issue.52

To improve understanding of risks, the DBoxes provide probabilities formulated in two different ways, that is, in natural frequencies (eg, for every 100 persons, 30 experience an effect) and in percentages (30%). Our risk communication strategy thus conforms with the current literature to the effect that risks should be presented in several formats to ensure that a broader audience is attained.53 However, we did not use icon arrays, to avoid lengthening the DBox, which we designed as a printable pdf. With as many as 10 options and 62 health outcomes in some of the DBoxes for this population, icon arrays seemed inappropriate. A future study on a web-based DBox would be useful to assess older adults’ perceptions of icon arrays displayed in pop-up windows.

More research is required to evaluate whether the modified DBox template, which now comprises several features intended to improve understanding, will allow designing more satisfactory patient decision aids for this population in fewer UCD rounds.

Table 4

Caregivers’ report of the value of Decision Boxes per round: ratings based on the patient version of the Information Assessment Method40

	Frequency (n/N)
	Round 1	Round 2	Round 3	All round
Relevance
This information is…
… totally relevant	3/11	6/10	2/6	11/27
… relevant	8/11	4/10	3/6	15/27
… somewhat relevant	0/11	0/10	1/6	1/27
… irrelevant	0/11	0/10	0/6	0/27
Understanding
They understood this information…
… completely	4/11	8/10	3/6	15/27
…mostly	6/11	2/10	3/6	11/27
…poorly	1/11	0/10	0/6	1/27
…not at all	0/11	0/10	0/6	0/27
Cognitive impact of the information
They learnt something new	2/11	5/10	5/6	12/27
This information allowed them to confirm what they do, or did	6/11	5/10	2/6	13/27
They are reassured	1/11	4/10	4/6	9/27
They were reminded of something they already knew	3/11	3/10	2/6	8/27
They are motivated to learn more	4/11	6/10	2/6	12/27
There is a problem with the presentation of this information	4/11	2/10	1/6	7/27
They disagree with the content of this information	2/11	0/10	0/6	2/27
This information is potentially harmful	1/11	0/10	0/6	1/27
Information use
They will use this information	7/11	9/10	6/6	22/27
This information will…
…help them improve their understanding of a particular issue and make a decision	1/7	2/9	4/6	7/22
…help them do something when they did not know what to do	0/7	0/9	1/6	1/22
…convince them to do something that they already wanted to do	2/7	2/9	3/6	7/22
…allow them to change the way they do something	0/7	1/9	2/6	3/22
…allow discussing something with someone else (a relative or a healthcare professional)	2/7	6/9	2/6	10/22
Expected benefits of the information
They expect the information to help them…
…be less worried	0/7	4/9	4/6	8/22
…be better equipped to discuss something with someone else (a relative or a healthcare professional)	6/7	7/9	4/6	17/22
…have more confidence in deciding something with someone else (a relative or a healthcare professional)	2/7	3/9	2/6	7/22
…handle an issue	1/7	3/9	2/6	6/22
…prevent an issue (or prevent it from getting worse)	2/7	3/9	6/6	11/22
They expect no benefits	0/7	0/9	0/6	0/22
Expected consequences
They expect that the use of this information can have a negative impact on their well-being of their health	0/7	1/9	0/6	1/22