User-testing an interactive option grid decision aid for prostate cancer screening: lessons to improve usability.

OBJECTIVE: To user-test a web-based, interactive Option Grid decision aid 'prostate-specific antigen (PSA) test: yes or no?' to determine its usability, acceptability and feasibility with men of high and low health literacy.
DESIGN: A semi-structured interview study.
SETTING: Interviews were conducted at a senior centre, academic hospital or college library in New Hampshire and Vermont. PARTICIPANTS: Individuals over 45 years of age with no history of prostate cancer who voluntarily contacted study authors after viewing local invitations were eligible for inclusion. Twenty interviews were conducted: 10 participants had not completed a college degree, of which eight had low health literacy, and 10 participants had high health literacy. INTERVENTION: An interactive, web-based Option Grid patient decision aid for considering whether or not to have a PSA test.
RESULTS: Users with lower health literacy levels were able to understand the content in the tool but were not able to navigate the Option Grid independent of assistance. The tool was used independently by men with high health literacy. In terms of acceptability, the flow of questions and answers embedded in the tool did not seem intuitive to some users who preferred seeing more risk information related to age and family history. Users envisioned that the tool could be feasibly implemented in clinical workflows.
CONCLUSION: Men in our sample with limited health literacy had difficulty navigating the Option Grid, thus suggesting that the tool was not appropriately designed to be usable by all audiences. The information provided in the tool is acceptable, but users preferred to view personalised risk information. Some participants could envision using this tool prior to an encounter in order to facilitate a better dialogue with their clinician. ETHICS APPROVAL: The study received ethical approval from the Dartmouth College Committee for the Protection of Human Subjects (STUDY00030116). © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The strength of this study was the use of the think-aloud technique in conjunction with in-depth interviews to examine the usability, acceptability and feasibility of the Option Grid.

The sample includes participants with lower levels of health literacy and education who are not computer-savvy.

Due to two study authors’ (GE and M-AD) involvement in developing the prostate-specific antigen interactive Option Grid, PS conducted the interviews and analysis of the data to mitigate the risk of bias.

A weakness of our study is the fact that we did not use a validated framework to user-test patient decision aids.

We recognise that including a more diverse sample of men (ie, ethnicity and geographical location) could have provided us with a different user-testing perspective.

Background

A growing number of developers are creating web-based, interactive decision aids and this includes tools for men considering the prostate-specific antigen (PSA) test.1 Patient decision aids are designed to provide users with evidence-based information on a particular health condition or screening test, and many men are using these tools when needing to make a preference sensitive decision – such as whether they should have a PSA test. Part of the development process is to user-test these web-based PSA decision aids to ensure they are easy to use and meet the needs and desires of men.2–4

The International Patient Decision Aid Standards indicate that users should be involved in the development process of these interventions.2 Elwyn et al proposed a four-step process map for developing web-based tools which recommends usability testing and field testing.3 Once a prototype is deemed ready by developers, it should be user-tested by stakeholders, including patients, to provide their perspective on elements of the tool that need to be improved (ie, navigation and design). This is a key step in the process prior to introducing the web-based decision aid in clinical settings for further evaluation.3 A review conducted by Coulter et al outlined similar processes to evaluate patient decision aids all of which recommend user-testing or determining the needs of patients who will eventually use these tools.4 The Ottawa Decision Support Framework, for example, described an iterative process to decision aid development which includes ‘assessment of determinants of decisions’ through the lens of the patient and provider.4 5 Patient input enables developers to create robust tools that are acceptable to potential users while addressing their needs and concerns.6 However, involving patients is time-consuming and costly.4 Many developers bypass the user-testing step and post tools that may not be suitable for the target audience.

According to a recent explanatory study by Dugas et al, of the patient decision aid development projects that have been conducted, less than 20 per cent include vulnerable populations (ie, patients with low health literacy).7 Health literacy is defined as ‘the ability to perform basic reading and numerical tasks required to function in the healthcare environment’.8 9 A clear theme regarding web-based tools that have been user-tested by low health literacy populations is that they are helpful (improve knowledge and reduce decisional conflict), but improvements are needed to resolve navigation issues and make them easier to use.10–15 Developers tend to see the opportunity to use online platforms to provide more information than they otherwise would on paper versions, but this increases the cognitive load, particularly for individuals unfamiliar with medical jargon.11 12 Evidence also exists that vulnerable populations may need assistance when using web-based applications.16 Fine-tuning the information elements and layout of these web-based tools to provide simplicity are frequently suggested alterations by low health literacy users regardless of health topic.

In the case of web-based PSA screening tools, it is clear that the majority available to the public have not been user-tested or tested with men of low health literacy – a patient population that is in need of access to evidence-based information regarding the screening test. We searched for, and found, only one web-based PSA screening tool that has been user-tested. It was designed for men with a family history of prostate cancer who provided positive feedback related to navigation, the amount of information and time required to complete the decision aid.17 However, this tool was customised for a certain segment of the male population and did not take into account the experience of men with lower levels of health literacy. Our study fills this gap by user-testing a web-based interactive tool known as Option Grid patient decision aids, with men of various education and health literacy backgrounds.

The aim of our study is to user-test the ‘PSA test: yes or no?’ web-based, interactive Option Grid patient decision aid to determine its usability, acceptability and feasibility with men of higher and lower health literacy.

Methods

Design

A semi-structured interview study was designed and reported according to the Consolidated criteria for Reporting Qualitative research checklist using purposive sampling, with embedded think-aloud methods to evaluate the user experience with the ‘PSA test: yes or no?’ interactive Option Grid decision aid. The think-aloud technique (see data collection section) was employed to examine the activities and comments of the user and get feedback on the usability and feasibility of the tool. Semi-structured interviews were conducted following users’ completion of the web-based Option Grid to get a better understanding of how to improve the interactive version.

Recruitment

We posted advertisements at bus stops throughout the Upper Valley region describing the study in order to invite men to use the PSA Option Grid and provide feedback. Also, coordinators at community centres and senior centres in the Upper Valley granted us permission to post advertisements at their centres. The advertisements encouraged men who were interested in participating to contact PS to agree on an interview time. The advertisement included information that participants would receive a $20 gas card for their participation.

Settings

Interviews were conducted at three locations in the Upper Valley of New Hampshire and Vermont: (i) the Dartmouth-Hitchcock Medical Centre, (ii) Dartmouth College library and (iii) the Upper Valley Senior Centre in Lebanon, New Hampshire.

Participants

Individuals over 45 years of age living in the Upper Valley of New Hampshire and Vermont with no history of prostate cancer, who voluntarily contacted study authors via phone or email, and indicated their willingness to participate, were eligible for inclusion. Participants were excluded if they could not read English. Participants were not excluded if they underwent a PSA test at any point prior to their participation in the study. Study aims and procedures were explained to the participants using an information sheet. Participants were informed, prior to their consent, that any information provided for the study would be kept confidential and stored securely with restricted access, thereby minimising risks to the privacy of those involved in the study.

Patient and public involvement

The development of the research question, study design and procedure are employed to improve the patient experience using a web-based patient decision aid like Option Grid. Patients were not involved in designing or conducting the study. Results were not disseminated to study participants.

Development of the PSA option grid decision aid

Option Grid patient decision aids are available in both a paper-based and web-based, interactive version and provide brief, evidence-based information on the available treatment options for various health conditions or screening tests, including the PSA.18 An editorial team comprised of researchers and physicians, and led by GE, developed the web-based, interactive PSA Option Grid based on the International Patient Decision Aid Standards. The team, in consultation with patients, determined the frequently asked questions (FAQs). Evidence-based answers to those FAQs were provided by conducting a literature review. The search focused on systematic review papers to determine the risk information embedded in the tool. A document listing the references used to formulate the answers for each FAQ accompanies the interactive PSA Option Grid. Users have the option of printing a paper-based version of the tool or using the interactive, web-based version of the PSA Option Grid. Printed Option Grid decision aids use a tabular format to identify the available treatment options for a chosen healthcare topic (columns) and the questions that patients most frequently ask (rows).18 The content embedded in the tool consists of the evidence-based answers to the FAQs.19

Users who select the interactive version are asked to provide demographical information such as their role (patient/patient caregiver or healthcare professional), race, age group, gender, ethnicity and geographical region. Users are asked: ‘Do you already have a preferred option in mind?’ Users can select one of three options: having a PSA test, not having a PSA test or not sure. They are then asked how strong their preference is – weak, moderate or strong. Next, users fill out a four-item, validated SURE (Sure of myself; Understand information; Risk-benefit ratio; Encouragement) survey which helps measure their level of decisional conflict,20 21 followed by a series of five questions to check their knowledge of the PSA test. A series of 10 FAQs are then presented, always in the same order. Users select their preference leaning based on the answer given for each treatment option. They then identify, on a scale of 0 (not important at all) to 5 (extremely important), the importance of the information presented. Once all 10 steps are completed, the user indicates their final preference, identifies their preference strength based on the scale and completes the SURE survey and knowledge questions. Finally, the user is presented with a summary of the choices they make for each FAQ.

Data collection

All interview sessions were conducted face-to-face with the participant and occurred in a private room at one of the three study settings. The Chew et al health literacy measure was completed by all participants.9 It is a validated three-item measure that uses a Likert scale (always, often, sometimes, occasionally, never) to determine the confidence with which individuals fill out forms and medical forms independent of any help, and how often they require help reading hospital materials.22 The study author queried each participant on their highest level of education attained. For the purposes of this study, participants who did not graduate with a college degree were considered to be of lower educational attainment.

Participants were provided with a laptop. Participants in this study were not provided the option to view or print the paper-based version of the PSA Option Grid. The interactive PSA Option Grid webpage was open, so participants could begin the interactive journey when ready. As participants were using the interactive tool they were asked to verbalise their thoughts - a technique known as think-aloud. Think-aloud was used because it enabled us to have a better understanding of the usability, feasibility and the impact on user ‘cognition and emotion’.23 24 It provided participants with the opportunity to voice their satisfaction or dissatisfaction with various elements of the interactive Option Grid, recommend changes and identify any difficulties related to navigation or understanding.

Following the completion of the interactive PSA tool, we conducted semi-structured interviews (interview guide can be viewed in online supplementary appendix 1). Questions (derived from Morville’s framework – see analysis section) were focused on five elements: (1) usability (including design and navigation), (2) usefulness (including questions on content), (3) desirability (positive or negative features), (4) value and (5) credibility. Interviews were audio-recorded and duration of the interviews was noted. Data was collected until saturation was achieved – sampling more data would not have led to new information regarding usability of the tool as users kept repeating the same concerns or ideas. Transcripts were not returned to participants for comment or correction.

Analysis

We searched for, but could not find, a validated framework to user-test a web-based patient decision aid. This is a relatively new field, and although there does not seem to be a well-used or recognised framework, we opted to use Morville’s ‘honeycomb’ framework to provide structure to our data (see figure 1).25 It has been used in the past to test oral health education applications, mobile device applications, business process management tools and social software.26–29 This framework has also been employed once before to user-test a decision aid known as the ‘decision-box’, and by the Cochrane Collaboration to test user’s experience with their systematic review library.30 31 We applied this framework to analyse our data since it contains all the features that we wanted to evaluate in our interviews. It was developed to assess seven different elements associated with the user’s experience with an intervention: usability, usefulness, credibility, desirability, value, findability and accessibility.28 Usability refers to the ease of use, and the degree of satisfaction attained from using the PSA interactive Option Grid. Usefulness relates to the value of the tool to the user’s knowledge-base or potential decision-making regarding the PSA test. Credibility explores the user’s level of trust for the evidence-based information embedded in the tool. We used the desirability element of the framework to examine the type of changes the user recommends making to improve the user experience. Value explores whether the interactive PSA Option Grid ‘advances the mission’, which in this case, means improving knowledge of the PSA test so a user can make a more informed decision, or can have a more efficient conversation with their provider in future encounters. For the purpose of this study, findability and accessibility were not assessed because study authors introduced the interactive PSA Option Grid to the user. We did not test the ease with which one could find the interactive tool online, or the barriers to access. We applied five of the seven (usability, usefulness, credibility, desirability, value) ‘honeycombs’ to evaluate any issues the user may encounter with the PSA interactive Option Grid which may be the basis for re-design.

Figure 1

Morville’s user-experience ‘honeycomb’ framework.

Qualitative data analysis

We employed Wallace et al’s recommendation to use the ‘sometimes’ response on Chew’s Likert scale as the cut-off to determine the number of participants with ‘limited or marginal health literacy skills’.32

User-testing sessions were recorded and transcribed by PS. The framework analysis method was used to analyse and code the data.33 We conducted a deductive thematic analysis using Morville’s framework, and the data was coded independently by two researchers (PS and JS). We used this type of qualitative analysis because the coding and theme development were directed by the framework which enabled us to best achieve our study objectives. The ATLAS.ti software was used to assign codes, generated based on the five facets of the framework, to sections of participant statements, and additional codes were created to group codes together. All codes were reviewed to determine the themes associated with usability and feasibility of the PSA interactive Option Grid. We also highlighted specific changes or alterations suggested by the participant to improve the tool.

Results

Twenty interviews were conducted with men in the Upper Valley region. Ten participants interviewed had not completed a college degree (low educational attainment) and 10 participants completed at least one college degree (high educational attainment). Of the 10 participants in the low educational attainment group, eight had low health literacy skills. All participants who achieved a college degree had high health literacy. All participants were white and their ages ranged from 49 to 81. See table 1 for details.

Table 1

Participant characteristics

User	Age	Ethnicity	Education level	Health literacy level
1	65 to 70	White	Low	Low
2	70	White	Low	Low
3	70 to 75	White	Low	Low
4	79	White	Low	Low
5	76	White	Low	Low
6	70 to 75	White	Low	Low
7	50 to 55	White	Low	Low
8	50 to 55	White	High	High
9	49	White	High	High
10	66	White	High	High
11	45 to 64	White	Low	High
12	69	White	High	High
13	75	White	High	High
14	81	White	High	High
15	70	White	High	High
16	50 to 55	White	Low	High
17	50 to 55	White	Low	Low
18	65 to 70	White	High	High
19	65	White	High	High
20	62	White	High	High

Thematic analysis

We identified four key themes: (1) Perceived usability was contingent on familiarity with digital interfaces (and on health literacy), (2) Desire for revisions to improve user experience, (3) The value of communicating risks and tailoring content to individual attributes like age and family history and (4) Credible source of information, useful for pre-visit use. See table 2 for themes and sub-themes.

Table 2

Themes and sub-themes derived from user-testing data

Themes	Sub-themes
Perceived usability was contingent on familiarity with digital interfaces (and on health literacy).	Participants with low health literacy have difficulty using computers. A solution to alleviate computer intimidation is to provide low health literacy users with a paper-based decision aid. Participants with high health literacy skills recognised that navigating the PSA Option Grid could be challenging for participants with low health literacy, so they suggested having a coach accompany them through the tool. Increase the size of the words throughout the interactive Option Grid.
Desire for revisions to improve user experience.	The layout of the FAQ pages need to be changed. Preferably stating a fact about the PSA test, and then asking: are you more or less inclined to have the PSA test? Based on that fact. Embed the data viewer icon array on the FAQ page, so the user does not have to click on an option to view the pictorial. Remove the importance ratings at the bottom of each FAQ page.
The value of communicating risks and tailoring content to individual attributes like age and family history.	Men desire a comprehensive source of information, and appreciate the risk information embedded in the tool. Participants believed that the PSA Option Grid is an ideal foundation for a more collaborative discussion with their physician. Users would prefer to see age—specific information, and information related to family history and its impact on prostate cancer risk.
Credible source of information, useful for pre-visit use.	Completing the interactive PSA Option Grid is not burdensome. It is best to complete the tool prior to the clinical encounter and bring the results to the physician to have a discussion. The PSA Option Grid represents a credible source of knowledge that can be used by men to be more involved in their healthcare decision making.

FAQ, frequently asked question; PSA, prostate-specific antigen.

Theme 1: perceived usability was contingent on familiarity with digital interfaces (and on health literacy)

According to Morville’s framework, usability encompasses the ease of use, and the degree of satisfaction attained from using the intervention. Throughout their journey, participants with lower levels of health literacy (according to the Chew health literacy measure) described their difficulty with using and navigating the tool which affected their level of satisfaction with the intervention. Participants with higher health literacy levels used the tool with ease but envisioned potential challenges for men with lower levels of education.

Six of the eight users who had lower levels of health literacy and low educational attainment were unable to use or navigate the PSA interactive Option Grid due to their stated unfamiliarity with the computer interface. The interviewer (PS) had to read and select the various options for the participant. Using an electronic device proved to be an intimidating factor in their user-testing experience and decreased satisfaction:

I’m not computer literate. That’s obvious…I’m intimidated by computers to be honest. (Participant 2 – age 70, low educational attainment and health literacy)

I don’t like this. You go ahead and use it. (Participant 3 - age 70 to 75, low educational attainment and health literacy)

According to multiple users, the majority of the population at the senior centre would not be computer literate. We were advised to simplify the tool for a lower health literacy population:

I’d say about 40% of the people that come here are computer literate. But, for older people that aren’t, and I’m thinking about people that come here…you need a more simplified version. (Participant 5 – age 76, low educational attainment and health literacy)

The men with higher health literacy, who were also part of the high-education bracket, had no issues with using or navigating the tool, but did express their concern for men who lack a higher level of education. They could envision how this interactive tool may not be usable for those men and hypothesised that a coach would be needed to guide them through the tool.

I think they [participants with low educational attainment] would need a coach to do this. (Participant 19 - age 65, high educational attainment and high health literacy)

Lastly, twelve users recommended we increase the size of the words to make the content easier to read for all users. Regardless of health literacy levels, elderly men being the target population of the PSA Option Grid have specific needs in terms of usability:

The font needs to be bigger. Particularly for people like me who may have vision problems… everything needs to be bigger. (Participant 3 - age 70 to 75, low educational attainment and health literacy)

The discrepancy between men with higher and lower health literacy levels regarding usability was evident and is a factor in the ability to use and navigate the PSA interactive Option Grid.

Theme 2: desire for revisions to improve user experience

The desirability component of the framework highlights the desire to change various attributes of the tool to improve the user experience. What type of changes do users want to see? We describe these changes in detail.

Participants had some difficulty understanding the actions they needed to take when faced with the FAQ portion of the interactive tool. For instance, the first FAQ poses the question: what does the test involve? The having a PSA test option states that the PSA is a blood test that measures the antigen level in the blood from the prostate gland, and the not having a PSA test option simply states ‘does not apply.’ For the second FAQ, the same information was associated with each option – ‘15% of American men will develop prostate cancer in their lifetime’. Eight participants felt like the information associated with each option were unclear, leading to confusion on which screening option they should choose.

So, I’m confused as to why I have to check this box as opposed to this box because they both have the same information. What are you trying to tease out here in this question with the two choices? (Participant 12 - age 69, high educational attainment and health literacy)

To alleviate confusion, users suggested we alter the organisation of the material for each FAQ. Participants indicated we should state a fact related to the PSA test, and then ask: does this information make you more or less inclined to have, or not have, a PSA test? Then, the participant can check the box based on their preference. This adjustment would also decrease the amount of material embedded in each FAQ page.

Start out with this bit of information up here, and then that’s when you get the two options. Put the statement you just made, and then add…does this information make you more or less inclined to have the test or not? (Participant 10 - age 66, high educational attainment and health literacy)

I’m suggesting that you have the piece of information, and then you’re given two boxes, yes/no, click one. (Participant 12 - age 69, high educational attainment and health literacy)

I would be more…it would seem a little more logical to me to have a number of those facts laid out, and then say: do these make you feel more or less likely to want to have a PSA? (Participant 18 - age 65 to 70, high educational attainment and health literacy)

For some FAQs, the having a PSA test option provided risk information. Below that risk information was a ‘data viewer’ option that, if clicked, provided users with an icon array to help them better understand the frequencies or percentages presented (see figure 2). Only two participants in the sample recognised the data viewer due to its small presence on the page. Once we directed their attention to the icon array, users indicated that it should be embedded with the rest of the information:

Figure 2

Icon array embedded in the ‘data viewer’ option in the PSA interactive Option Grid. PSA, prostate-specific antigen.

Yeah, pictorials are good. I didn’t have problems with the 15%, but I would incorporate this (the icon array) in here (on the FAQ page). (Participant 15 - age 70, high educational attainment and health literacy)

The third recommended adjustment was the removal of the importance ratings at the bottom of each FAQ. The majority of users did not understand the purpose or usefulness of that exercise, considering their attention was focused mainly on the information. The information was generally considered to always be of high importance to the user:

So, to me the importance ratings of each question were less important than the stuff above. I thought the top part was informative, and then the zero to five scale was informative to you, the researcher, but not to me. (Participant 8 - age 50 to 55, high educational attainment and health literacy)

Ultimately, users suggested to re-organise the material for each FAQ by stating a piece of information, and then asking whether or not that information made the user more or less inclined to have the PSA test, thus clarifying the action required. Moving the icon array embedded in the data viewer to accompany the information for each FAQ would also help users visualise and understand the percentages and frequencies presented.

Theme 3: the value of communicating risks and tailoring content to individual attributes like age and family history

Value, according to the framework, refers to the attributes that participants deem most valuable. Viewing the tool through this lens, we identified that the risk information was most valued. In particular, users wanted to see prostate cancer risk information related to age and family history.

Participants felt that the greatest asset of the PSA interactive Option Grid was the level of high-quality, evidence-based information. It was also easy to understand, and much of the numerical information was new to users:

There were a couple of things that I wasn’t aware of regarding the PSA test, which came up in here. About 30% of men with a high PSA have prostate cancer. I didn’t know that. (Participant 2 – age 70, low educational attainment and health literacy)

Even for participants who already possessed knowledge of the risks and benefits associated with the PSA test, they felt that the tool provided the foundation for a discussion with a physician:

I think this is a good introduction because you have people out here who haven’t considered or worried about it, and need to be educated. They need to be brought up to speed… as an introduction this will work. You may have had some people today that didn’t know what the heck they were doing or haven’t really paid attention… (Participant 3 - age 70 to 75, low educational attainment and health literacy)

In particular, participants highlighted the value of the various probabilities associated with the PSA test, including: likelihood of developing prostate cancer in one’s lifetime, the chances of getting prostate cancer if the PSA test is high or normal, the percentage of prostate cancers that are aggressive and the risks associated with prostate biopsies and cancer treatments.

The immediate risk of the various procedures, or if you had a high PSA, the risk associated with having a biopsy. That was the most helpful piece of information for me. (Participant 9 - age 49, high educational attainment and health literacy)

Despite the participants’ satisfaction with the true content in the tool, they indicated that there were two key information elements missing. The first is age-specific risk information related to prostate cancer. Stratifying the risk information according to age would help men better understand when the ideal time is to consider the PSA test. Second, how does family history influence one’s risk of prostate cancer? How important is family history for those considering the PSA test? These are questions that participants felt should be addressed in order to create a more comprehensive tool for users.

I think it would be interesting to know something about age because I’ve been told, I heard, that if you are over a certain age, the speed at which the average prostate cancer moves is very slow that you’ll probably die of something else first. (Participant 13 – age 75, high educational attainment and health literacy)

Theme 4: credible source of information, useful for pre-visit use

According to the framework, credibility refers to the trust given to the tool’s contents. It is because of the trust in the evidence-based nature of the PSA Option Grid that participants found the tool useful. The intervention was useful in expanding the user’s knowledge-base and helped him make a decision regarding the PSA test. The two mechanisms of the framework are linked in the sense that the tool represented a credible source of information for users, and it could be used in a clinical setting prior to their discussion with the physician.

Participants felt that the ideal time to complete the tool would be prior to the clinical encounter due to the time pressures physicians experience. The patient can then bring the summary of their results to the physician to facilitate a better discussion about whether or not the PSA test is worthwhile.

Yes. I think people could fill this thing out before the visit, so that they don’t have to sit with the doctor and hash through it. (Participant 8 - age 50 to 55, high educational attainment and health literacy)

Participants recognised that physicians have limited time with their patients, but the need to have a more constructive discussion about the PSA test still exists. They believed that the interactive PSA Option Grid represented the basis for an improved interaction with the provider because it was a credible source of information. There is a plethora of resources available in print or online regarding the PSA test, but men indicated their difficulty deciphering which information is credible and which information is not. All participants felt that being provided with one source that can provide accurate facts can be helpful to men in their decision-making, alleviating concerns of getting misinformation.

Everyone Google’s…all the time, and so, more and more people, are going to their physician with questions they found online…I feel like a tool like this if presented within the context of a healthy conversation with your physician is valuable. (Participant 9 - age 49, high educational attainment and health literacy)

Discussion

Overall, men indicated that the interactive PSA Option Grid was useful despite the challenges some faced engaging with the tool, the information acceptable with an emphasis to add more risk information associated with age and family history and potentially feasible to implement this tool prior to a clinical consultation. The usability issues for participants with low health literacy can perhaps be attributed to an interactive tool that was not designed for such a vulnerable population. It is worth noting that many men with lower health literacy also seemed unfamiliar with computers and reluctant to use the online intervention independently. The difficulty experienced in navigating the digital interface, feelings of intimidation with having to use a laptop, preference for a paper-based version and desire for changes to the layout and presentation of information all suggested that the PSA Option Grid had not paid enough attention to usability. The risk information provided by the Option Grid was found to be highly valuable with most hoping to see content tailored to their specific attributes like age and family history in a future version of the tool. The PSA Option Grid also represented a credible source of information, and users envisioned using this tool prior to the clinical encounter to facilitate an improved discussion with their physician.

The strength of this study was the use of the think-aloud technique in conjunction with in-depth interviews to examine the usability, acceptability and feasibility of the tool. The purposive sampling method enabled us to include participants across the health literacy spectrum, including those with lower levels of health literacy and education who are not computer-savvy. Due to two study authors’ (GE and M-AD) involvement in developing the PSA interactive Option Grid, PS conducted the interviews and analysis of the data to mitigate the risk of bias. A weakness of our study is the fact that we did not use a validated framework to user-test patient decision aids. Also, we did not provide the option for participants to use a tablet or smartphone to access the PSA Option Grid. It may be that using these devices could have changed some of our results. We also recognise that interviewing a more heterogenous sample that includes men of different ethnicities and from different locations from across the USA would have provided us with a different user-testing perspective and made our results more applicable to decision aid developers across cultural contexts.

Our research highlights the importance of user-testing interactive decision aids prior to making it available to the online world. Developing a decision aid that is usable by men with lower health literacy involves an iterative interview process as described by Barton et al in their development of an encounter-based decision support intervention for rheumatoid arthritis medications.34 Patients were interviewed on two separate occasions in order to refine the tool prior to pilot testing.34 The developers of the PSA Option Grid did not user-test prior to its publication and based on this study we know men with lower health literacy struggled to navigate the application. Using a laptop proved to be a challenging exercise that provoked feelings of intimidation - an unprecedented finding that adds a layer of complexity for developers of interactive tools. The design changes recommended by users also suggests a desire for simplicity by embedding one fact about the PSA per FAQ to assess preference. Developers of decision aids should user-test their tools to ensure the layout and presentation of information meets the needs of low health literacy patients, and so that they can navigate the tool without assistance.

Our study indicates that men value risk information regarding the PSA test, and seek information tailored to their own characteristics, particularly age and family history. Survey and qualitative research echo these findings.35 36 Despite information needs being highly variable and dependent on demographics, the desire for risk information remained stable for men making a decision on whether to have the PSA test or not.35 Davison et al concluded that ‘men who had sons, a positive family history and lower levels of education ranked hereditary risk’ as a key component in their prostate screening decision-making process.37 Family history was repeatedly mentioned by participants in our sample as a factor that weighs heavily on the decision-making process. Although sexual function was not an issue raised by participants in our study, research shows that the potential effects of prostate screening on sexual function is significant to younger men considering the PSA test.38 39 In particular, participants with higher education consider sexual function to be of value when reflecting on the test.39

The source of the various facets of information regarding the PSA test is also a major theme in the literature. The internet has become the prime resource for men seeking credible information on the PSA test.40 The preferred sites are those that are recognised or ‘institutions with established reputations’.41 Also, men often resort to online patient discussion forums for information support – a platform that was not mentioned by participants in our sample.42 43

Participants felt that it would be feasible to use the tool prior to the consultation with the physician. According to the Ottawa Inventory of patient decision aids, there are eight PSA tools available for men – three of which are designed for use in the encounter. The feasibility or routine implementation of these decision aids has yet to be studied. Future work should examine how and when to best integrate these online decision support tools that have shown to increase patient knowledge and satisfaction related to the prostate cancer screening decision.44 In addition, the community of decision aid developers should set an agenda to explore how to make web-based tools accessible and usable for low literacy individuals.

Conclusion

The PSA interactive Option Grid decision aid was useful but not optimally designed to ensure usability for all men in the sample. Research is needed to determine how best to increase access to patient decision aids (and the information embedded in these tools) so individuals of all literacy, health literacy and computer literacy levels can benefit. The value of the evidence-based risk information embedded in the tool rendered it credible and acceptable by users who wish to see information associated with family history and age in a future version of the PSA Option Grid. Users envisioned the tool being feasibly implemented prior to their visit with the physician. Overall, this highlights the importance of user-testing web-based decision aids prior to their release in order to get a comprehensive understanding of user needs and priorities. Based on our results, decision aid developers can ascertain that users prefer simplicity regarding the layout and presentation of evidence-based information and prefer to view risk data that is based on their personal attributes (ie, age and family history). However, more research is needed to better understand how interactive, web-based decision aids can be better designed to be usable by individuals across the health literacy spectrum.