Qualitative user evaluation of a revised pharmacogenetic educational toolkit.

INTRODUCTION: Pharmacogenetic (PGx) testing is a leading application for personalized and precision medicine; however, there are barriers, including limited provider and patient understanding, which affect its uptake. There is a need for tools that can enhance the patient and provider experience with testing and promoting the shared and informed decision-making.
MATERIALS AND METHODS: In this study, we sought to gather additional feedback on a PGx toolkit comprised of four educational tools that had been previously evaluated through an online survey by pharmacists. Specifically, we conducted semi-structured interviews with pharmacists and members of the public regarding their understanding and utility of the toolkit and its individual components.
RESULTS: Participants found three of the four toolkit components, a test information sheet, flipbook, and results sheet, to be useful and important. The fourth component, results card, was viewed less favorably. Participants differed in their preference for medical jargon and detailed results nomenclature (namely star * alleles).
CONCLUSION: User input during the development of educational materials is essential for optimizing utilization, effectiveness, and comprehension.

Introduction

Pharmacogenetic (PGx) testing is one of the leading applications for personalized and precision medicine practices having the greatest number of early adopters.1 However, questions of clinical utility and optimum delivery models, as well as barriers such as limited provider knowledge and limitations in clinical decision support systems have inhibited universal adoption of PGx testing.2–6 Research and clinical efforts have begun to address some of these barriers to implementation,7–10 but additional support is needed, particularly regarding patient understanding and informed decision-making for PGx testing.

Although the delivery model (and health providers involved) for PGx testing may vary depending on the clinical circumstances, pharmacists will likely play an instrumental role in the delivery and/or implementation of PGx test results.11–13 Our own work and others have demonstrated that community pharmacies may be well-suited to provide the PGx testing services.14,15 Specifically, community pharmacists are well-informed about the drug interactions, often provide clinical care services,16–19 and are experienced in counseling patients about medications.20–23 Provision of PGx services in community pharmacies is feasible, particularly when offered as part of medication therapy management.24 Although we anticipate that community pharmacies are an appropriate setting for implementation of PGx services, there is a need for pharmacist-facing support tools25 as well as tools to enhance the patient experience with PGx testing. Pharmacy school curricula include instructions on PGx,26–30 and a number of continuing education opportunities are available,28,31,32 but pharmacist knowledge of PGx is still limited.32,33 Furthermore, some pharmacists may struggle to identify patients who have low health literacy and may need additional educational support to promote informed decision-making about PGx testing.34,35

As previously reported, we developed a suite of educational tools (“toolkit”) to support providers, particularly community pharmacists, when discussing about PGx testing and results with their patients.36 The toolkit comprised of a single-page test information sheet (TIS), an educational flipbook, a single-page results report, and a wallet-sized results card. The toolkit components were reviewed by 380 community pharmacists in North Carolina and were revised based on their feedback. In this paper, we present the second round of qualitative evaluations for understandability and perceived utility by both pharmacists and members of the public of an updated version of the toolkit components.

Materials and methods

Toolkit components and revision

Four educational tools are included in the toolkit: a Test Information Sheet (TIS), a “Flipbook” Guide to PGx Testing, a Result Summary Handout, and a Results Wallet card (Table 1). All the components have been previously described in detail and revised based on the feedback from an initial assessment.36 Following the assessment, we aimed to improve the toolkit by using more patient-friendly language, improving esthetics of the components, and utilizing more graphics. In summary, we revised language in the TIS and Flipbook to improve readability, we completely recreated the Flipbook to improve its design and incorporate new graphics and illustrations, and we made minor adjustments to the design of the results card. The revised toolkit components are publicly available via the Community Pharmacists Pharmacogenetics Network (CPPN; https://www.rxpgx.com/patient-resources).

Table 1

Details of revised toolkit components

Component	Format	Content	Use	Snapshot
Test information sheet	Single 8.5 × 11 inch paper, black and white print, double sided; modeled after Vaccination Information Statements	General overview of PGx testing: Purpose of PGx testing What is involved in testing Risks and benefits of testing	For patients to take home and reference
Flipbook	8.5 × 11 inch 16-page spiral bound book, color print; modeled after flipbooks were used by genetic counselors	Graphics of PGx concepts:• Outcomes when prescribing with PGx information compared to prescribing without• Impact of metabolizer types Text details about medications eligible for testing:• Associated gene• Possible test results• Impact of results on care	For providers to reference in the clinic/pharmacy when discussing PGx testing or testing results
Summary results report	Single 8.5 × 11 inch paper, black and white print	Patient results, interpretation of the result, and guidance for follow-up	For patients to take home and share with other providers
Results card	Wallet-sized card (~3.5 × 2 inches)	PGx genes tested, results (*allele/genotype and phenotype)	For patients to carry with them and share with other providers

Abbreviation: PGx, pharmacogenetics.

Semi-structured interviews

For gathering in-depth feedback regarding each toolkit component, we conducted semi-structured interviews with pharmacists and members of the public. All interviews were conducted by one investigator (RM) using an interview guide to ensure consistency (Table 2). The interview guide consisted of 15 questions about the format, content, and utility of each tool. Pharmacists were also asked about the likelihood of using one or more components in their pharmacy to discuss PGx testing with patients. “Think aloud” techniques, a standard method in cognitive interviewing,37 were used to elicit participants’ preferences and educational needs. Participants were also encouraged to circle confusing terms on a printed copy of the handouts. Suggestions for improvement were also elicited. All interviews were audio-recorded, transcribed, and coded by themes as previously described.36 The study was approved by the Duke University Health System Institutional Review Board (IRB) (Pro00069061), and all participants provided written informed consent prior to participating in the interview.

Table 2

Qualitative interview guide

Pharmacist interview	Patient interview
For each of the four components:
Format• What is your overall impression of the format?• What format might be better?• What other types of graphics/picture may be helpful to include?• What do you think is the purpose of the component? (Explain the purpose if unclear) How can the format be improved to make it easier to understand the purposeContent/information• What information in the toolkit component is understandable/not understandable?• Do the TIS (#1) and flipbook (#2) make it understandable on how PGx testing will be used?• Is there anything that does not make sense?• Any words you do not understand or that you do not think patients would understand?• Imagine you were considering ordering testing for your patient or havingtesting yourself – what else would they/you want to know?	Format• What is your overall impression of the format?• What format might be better?• What other types of graphics/picture may be helpful to include?• What do you think is the purpose of the component? (Explain the purpose if unclear) How can the format be improved to make it easier to understand the purposeContent/information• What information in the toolkit component is understandable/not understandable?• Do the TIS and flipbook make it understandable on how PGx testing will be used?• Are there any sentences/sections that do not make sense to you?• Are there any words that you do not understand? Or that you do not think other people (like your family or friends) would understand?• Imagine that you were considering having testing – what else would you want to know?
After reviewing all the four components
Utility• Would you feel better able/more confident/comfortable describing PGx testing to your patients with the toolkit?• Would you use components #1 or #2 of the toolkit with a patient who was considering having PGx testing? Why/why not?• Would you use components #3 or #4 to report results for PGx testing? Why/why not?• What changes would make it more usable?• What other educational tools or information would be useful?• What other formats would you prefer?• What other resources would you use or refer your patient to for more information on PGx?ValueWhat is the primary value/benefit of this toolkit for pharmacists and your patients?	Utility• How is your understanding of PGx testing after reviewing the toolkit? Better/worse?• Which components are most helpful? Which would like to receive/view if you had testing?• Would you recommend components #1 and #2 to someone who is considering to undergo PGx testing? Why/why not?• Would you want your pharmacist to give you components #3 or #4 after receiving results? Why/why not?• What other tools/information would be useful?• What other formats would you prefer?• What additional questions would you have for your pharmacist?• Where would you go to learn more about PGx?

Abbreviations: PGx, pharmacogenetics; TIS, test information sheet; component #1, test information sheet; component #2, flipbook; component #3, summary results report; component #4, results card.

Participants

Individuals from the public were recruited to provide feedback from the patient perspective. The study was advertised via Craigslist postings and flyers posted in and around Duke Hospital (Durham, NC). Four individuals participated: one male and three females. Pharmacist participants were recruited via CPPN. Two pharmacists, one male and one female, agreed to participate. Neither of the pharmacists participated in the initial study. Additional demographic data were not collected from the participants to protect their privacy due to the small sample size.

Results

TIS

Overall, reviewers found the TIS useful and believed it would be an important resource to refer to after discussing testing with a provider. One pharmacist indicated that he would feel more confident or comfortable offering PGx testing to patients because he would have written information to “back up” a conversation about testing. Another pharmacist said, “It’s good for communicating with the patients and provides another avenue to talk to the patient. In those cases, it is serving its purpose.” Both pharmacists indicated they would use the TIS for all patients offered PGx testing.

The text and content were positively reviewed by three patients and one pharmacist participant. With respect to the language and reading level (currently at an eighth grade reading level), one patient participant indicated, “It’s readable. Not too technical;” and a pharmacist participant said, “I thought it was simple enough language that I think anyone could understand.” However, one patient suggested making the text more concise, saying “I think it’s too much information … there’s too much overall. Everything could be shorter.” This participant, who self-identified as Spanish-speaking, indicated that if translated, the information would likely not fit into the single-page format. One pharmacist expressed concern about the inclusion of terminology like the word “pharmacogenetics” or “variation.” Two participants (one patient, one pharmacist) appreciated the inclusion of a list of online resources to view for additional information about pharmacogenetics and testing.

In order to enable usability of the TIS for a variety of testing platforms and laboratories, some details about the test, such as price and turn-around-time were intentionally left blank. However, participants indicated their desire for more specific details. In particular, specifying cost was very important to most participants (three patients and one pharmacist). As originally worded, the TIS indicated that “these tests usually cost a few hundred dollars.” The participants then expressed that it is also important to include additional information such as the range, maximum possible amount, or insurance coverage.

Participants also suggested that including information about risks is also important.

I need to know yes or no if there are risks. I don’t see anything that says yes or no. It just gives laws that protect you. Are there any risks?

With respect to the Genetic Information Nondiscrimination Act (GINA) (included in the Risks section), a participant indicated that it was confusing and that it was “jarring” to “talk about the law.” Another participant indicated it was important to include the information about GINA and its protections as well as the potential effects of testing will have on future prescriptions, “especially when it comes to money and insurance.”

The format of the TIS was generally viewed favorably. One patient praised the layout by saying, “This is good because you’ve got it set up with headings and then the facts;” a pharmacist participant said, “I like the format; I like the … sections so right away you can see what you need to know… It is a fine format that would be familiar to patients.” One patient participant did request that the font size be increased to make it easier to read.

Flipbook

Like the TIS, feedback regarding the usefulness of the flip-book was generally positive. Three participants including two patients and one pharmacist indicated that although it was a useful resource, it probably would not be necessary for all patients. One patient participant said, “A patient with a science background may not need all of this, but people with less education or who are older might need something with graphics.” Two participants indicated that although the flipbook was helpful, it would still be important for a provider to explain the concepts.

This is a little complicated … [but] if the doctor is going to go over it with you and you’re not just reading it, it is probably okay.

A number of suggestions were made on ways to improve the content and layout of the flipbook. As with the TIS, some participants expressed concern about potentially confusing terminology. Two patient participants felt the term “metabolize” needed a better definition, with one pointing out the risk of confusing it with “metabolism of food.” One also worried that some patients may not be familiar with the term “enzyme.” Although considered confusing, participants still felt the proper medical terms should be used; two participants suggested including a glossary or reference sheet to define the less-familiar terms. Two participants felt the term “adverse response” should be replaced with “side effects” because “most people have heard that; it’s what’s written in the brochure that comes with the medicine.”

Participant response was mixed regarding inclusion of details about genotype, specifically star (*) alleles. Two patient participants indicated inclusion of genotype information was not useful; however, another stated that he “wants to know everything [because] the more [information] you give, the more knowledge patients have to decide whether or not to take this drug.” One of the pharmacist participants thought the allelic information should not be included at all.

Participants made additional design requests such as increasing font sizes, turning some text that was written vertically, and minimizing the use of the color red since it seemed “bad” or stigmatizing. Three participants commented on the cover of the flipbook, which was a collage design including graphics of a DNA double helix, a medication bottle, and a patient. Specifically, they felt that some patients would be unfamiliar with the double helix and might get confused. A patient and pharmacist recommended replacing the cover with a picture of a patient talking to a pharmacist.

One patient participant requested making the flipbook available digitally, perhaps as a video or a digital book.

I think I’d like it as a video. Because if you get too much information at the doctor’s office it can cause your mind to like freeze. But if you go back home and sit down to watch it and is like ‘oh this is what he actually means’ then if the patient or the doctor has a question they can go to a video.

A pharmacist participant agreed with the suggestion, stating that paperless versions of these types of resources would be beneficial, and they would like to review this resource with patients on a computer screen or a tablet.

Summary results report

Overall, the participants liked the simplicity of the results report. We specifically asked about their perception on the clarity of the form, since the results report included all possible test results as metabolizer type, primarily for ease of use, so that the provider could check/circle the specific result and corresponding impact on care. Three patients stated that it was clear and they would have no difficulty understanding the result. One pharmacist agreed that it was clear and understandable, whereas the other felt like it may be confusing but could provide an additional talking point to ensure that the pharmacist takes time to review the results with the patient instead of just handing them the result.

Few recommendations were made on ways to improve the results report. One patient preferred having the star (*) allele included, rather than only including the phenotype (ie, poor metabolizer, reduced uptake, etc). One pharmacist suggested changing the “impact on care” box because it might appear as if the pharmacist preferred one drug to another. Instead of specifying the drug change, a more general statement was suggested as, “A change to your medication may be required. Please discuss with your doctor during your next visit.”

Results card

Compared to the other toolkit components, the results card was the resource that participants felt was the least likely to be used. One patient participant was unsure of the purpose of the card.

This information isn’t going to be in your medical records? Why do I need a card? … I don’t get it, having to have an extra thing in your pocket … I just don’t want to carry a card.

A pharmacist expressed similar sentiments saying, “I really don’t see anybody carrying a PGx test card with them and handing it out … So I don’t think this is as useful, because let’s face it, how many things do you have in your wallet? How many more things do you want to carry around? … it’s just too much.”

One patient who felt the results card was useful suggested including only the gene name and star (*) allele result since the “pharmacist and doctor is supposed to know what that means and the patient may not need to know.” The participant indicated that reducing the amount of text by eliminating the interpretation would enable the text size to be increased on the card. A pharmacist shared a similar opinion, stating, “I would include [alleles] because this is not something they’d show to their friends, but to their doctor or pharmacist.” That pharmacist also preferred to remove the interpretation of metabolizer type.

Participants suggested alternative options for storing results, such as saving as an image on their smartphone.

Revisions to the toolkit

Based on the feedback received, a number of changes were made to the toolkit before being finalized for use in a trial assessing the delivery of PGx in community pharmacies (NIH 5R01GM081416; NCT02937545). Specifically, the wording in the TIS was made more concise to reduce the amount of text. Subsequently, we were able to enlarge the text, addressing a suggestion by one of the participants. The original TIS presented to participants was 737 words in length, and the revised version was 700 words. The reading level was maintained at about an eighth grade reading level. Throughout the text, the term “genetic variations” was replaced with the term “genetic differences.” Significant changes were made to the section on risks of testing in response to comments from the participant who felt that the question was not answered understandably. Finally, the statement “Only your healthcare provider can approve changes to your prescription. You should continue taking your medication as currently prescribed until you discuss results with your provider” was reformatted to stand out from the other text as pharmacists deemed it an important statement.

A number of changes were also made to the flipbook. For example, the cover page, which had previously been a collage of graphics, was replaced with a photo of a pharmacist talking to patient. Throughout the flipbook, the term “adverse response” was replaced with “side effect,” which was deemed a more understandable term. In addition, based on comments from multiple participants, use of the color red was minimized, as it was perceived as having a negative value. For the drug-specific pages, the star alleles were removed and possible results were described as metabolizer or activity type.

The only revision made to the results report was to the “Impact on Care” section. The statement about results indicating a potential need for a drug or dose change was made more general to say: “The test results suggest that switching to a new drug may be needed. Please talk to your doctor before making any changes to your medication.” Minimal changes were made to the results card with only a revision to the description of PGx testing from “A pharmacogenetic test looks at genes that affect how the body responds to and uses medicine” to “A pharmacogenetic test looks for variants in genes that affect how to body responds to and uses medicine.”

Discussion

Patient education will be a key component for the successful introduction and implementation of PGx testing at its early stage. Patient input or participatory design for the development of educational materials is essential to optimize utilization, effectiveness, and comprehension. This follow-up study included public input and validated our initial assessment of our PGx testing toolkit,36 particularly regarding the positive perceived usefulness and importance of the TIS, flipbook, and results sheet.

There was consistency in many themes between patients and pharmacists, including the usefulness of the toolkit components, the ease of their use, and the design of the components. One area of diverging opinions was on the use of medical terminology and level of descriptive detail. Users’ reading level is one of the major challenges in developing any type of educational materials.38–41 The pharmacist participants expressed concern that the language, including the term “pharmacogenetics,” was too complicated and should be replaced. Although patient participants identified some confusing language, they believed that it was important to use proper medical terminology. We acknowledge that the reading level of the toolkit components, currently at the eighth grade level, is greater than the recommended 4th–6th grade reading level range for patient education materials, which may account for some of the comments.42 Other studies examining patient-facing materials have similarly reported mixed responses regarding the use of medical terminology43 to more general language.44,45 Though some patients may prefer inclusion of medical terminology, minimizing its use can improve patient understanding.46 As suggested by our participants and others,47 a hybrid approach where medical terminology is included with a definition or glossary may be an appropriate solution to these conflicting findings and promote comprehension.

Despite the limited enthusiasm for the patient card, “pharmacogenetic ID cards” have been utilized by other researchers, laboratories, and healthcare systems. For example, Bangkok’s Ramathibodi Hospital was one of the first to utilize a PGx results card by providing a plastic wallet card following HLA testing in order to indicate patients at risk of Stevens–Johnson syndrome and toxic epidermal necrolysis.48 In a clinical trial that included provision of a results card and explored the impact of HLA testing, treatment outcomes were improved when physicians modified treatment based on test results.48,49 In Estonia, a pilot program for personalized medicine allows residents to access medical information using a national identity card;50 however, an electronic “smartcard” is issued to Estonian residents, rather than a basic results card similar to that examined for this research and used in Tai-wan. Other “smart” technologies such as a two-dimensional barcode (QR code) that stores PGx information and links to decision support services using a Medication Safety Code system51,52 has been well-received by physicians and pharmacists.53 Use of results cards may also be beneficial in care settings that lack the ability to integrate PGx information into electronic health records or clinical decision support systems. However, to our knowledge, there has been limited exploration of patients’ attitudes about or utilization of the cards. Differences in culture, portability of health records, public trust of health providers, and availability of electronic medical records likely to affect the patient and provider attitudes about this toolkit component.

There are some limitations in the current study that must be considered. Due to the small population size and recruitment limited to individuals from Durham, NC, and surrounding areas, the findings of these interviews may not be applicable to other populations. In addition, none of the participants have undergone PGx testing or used it in their pharmacy settings; therefore, all feedback was based on participant expectations rather than their actual experience using the toolkit components. This limitation will be addressed by assessments of toolkit usage in an ongoing trial of PGx testing in community pharmacies (NCT02937545).

Conclusion

Facilitating patient engagement and comprehension about PGx testing will be essential for the foreseeable future as more and more providers offer these tests. Since the conclusion of this assessment, we have made additional revisions to the toolkit components, which include removing some of the medical jargon and genetic nomenclature, and replacing it with general or common terminology. Continued evaluation of educational tools in a practice-based setting will be important to capture the utility of tools in actual patient–provider discussions, with a wider diversity of patients and providers, time pressures, and competing tasks. We anticipate further development and transition to digital formats in the future.