Evaluation of Online Written Medication Educational Resources for People Living With Heart Failure.

Background: Patient educational resources on heart failure (HF) medications may improve patient understanding, which is critical for informed decision-making and patient self-efficacy. The purpose of our study was to evaluate the quality and readability of written medication educational resources available online.
Methods: Two investigators searched Google, Yahoo, and Bing for written patient educational resources that addressed at least one HF medication. We assessed educational quality using the Ensuring Quality Information for Patients (EQIP) tool (range 0 [worst] to 100 [best]), and we evaluated readability using the Flesch-Kincaid Grade Level.
Results: From 693 identified webpages, 39 HF medication educational resources met study eligibility. Among included resources, the median Ensuring Quality Information for Patients score was 61% (interquartile range 54%-68%), with 2 (5%) rated as high quality (score ≥ 75%). The median Flesch-Kincaid Grade Level was 8 (interquartile range 8-12), with 4 (10%) resources meeting the recommended 6th-grade reading level. Conclusions: Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.

Heart failure (HF) affects approximately 750,000 Canadians. Despite the high morbidity and mortality rates associated with HF, it can be successfully managed with a combination of healthy behaviours, medical devices, and medications. The variety and complexity of HF treatment options may result in patients and their caregivers feeling overwhelmed, which can negatively impact patient outcomes., One intervention to overcome this issue is patient education about HF, their disease condition, and available treatment options., Provision of written patient educational resources may help improve patient understanding of their condition and treatment options. Various types of educational resources exist, including those designed to enhance patient knowledge (eg, informational pamphlets, patient handouts) and specifically to support shared decision-making (ie, decision aids). Regardless of the type, ideally, written educational resources regarding HF should facilitate discussions between patients and their healthcare providers, should be based on high-quality evidence, and should be developed with the patient’s informational needs and health literacy in mind.

Emerging evidence documents the suboptimal quality and readability of patient educational resources.7, 8, 9, 10, 11, 12, 13, 14 For instance, a 2014 review assessed the quality of available patient educational resources on left ventricular assist devices, for patients with advanced HF, using the Flesch-Kincaid Grade Level score, the Fry algorithm, and a modified version of the International Patient Decision Aid Standards. This study found that most educational resources available online that address left ventricular assist devices are of suboptimal quality. Despite the fact that pharmacotherapy is the cornerstone of HF management, educational resources on HF medication have not yet undergone the same level of rigorous evaluation. Therefore, the purpose of our study was to collect, categorize, and evaluate the quality and readability of online written patient educational resources regarding HF medication.

Methods

Search strategy

From May 2021 to June 2021, 2 investigators (S.L. and F.Y.) independently conducted simultaneous, comprehensive searches of the top 3 Internet search engines—Google, Yahoo, and Bing—using Google Chrome (the most commonly used browser) to collect Internet-based educational resources about HF medications.,,,, Search queries included the following: (i) “Heart failure medication patient information”; (ii) “Heart failure medications patient information”; (iii) “Heart failure medication patient handout”; (iv) “Heart failure medications patient handout”; (v) “Heart failure drugs patient information”; and (vi) “Heart failure drugs patient handout”. The search queries used were developed through discussion and collaboration between the researchers, with the goal of replicating search terms that patients looking for this information might use (ie, more general, broad terms, in lay language). The investigators used an empirical method to minimize the influence of prior search history, which involved clearing the cookies from their web browsers and entering “incognito mode” prior to each search. To emulate typical Internet searches of patients looking for health information, we considered the results from the first 2 pages of each search.,17, 18, 19 The number of results per page was 10 for Google and Yahoo, and 8-10 for Bing, depending on ads. Additionally, we manually searched Web sites from key cardiovascular, HF, pharmacy, and patient advocacy organizations; HF medication manufacturers; and the Ottawa Decision Aid Inventory (Supplemental Table S1). Key Web sites were defined as being those that are well known Web sites of cardiovascular and HF societies, prominent Canadian cardiac centres, and health authorities. Evaluators assessed resources as a whole and multiple pages, if relevant, but hyperlinks to different URLs were not assessed. Further, S.L. and F.Y. individually removed any intra-assessor duplicate resources from their results prior to combining their lists. After the results were combined, between-assessor duplicates were removed. Lastly, although we were unable to determine the exact intention of the resources evaluated, the inferred intention was that they were developed to provide patient education. Therefore, we included resources irrespective of their stated (or unstated) intention if they met all the eligibility criteria below.

Eligibility criteria

We included educational resources that met the following criteria: (1) written resources directed at patients; (2) written in English; (3) included ≥ 10 sentences about HF medications (as per Iacovetto et al. 2014) to ensure inclusion of resources with substantive content on medications; (4) accessible without having to register and/or pay a fee; (5) specifically described ≥ 1 different HF medication treatment option (i.e. individual medication or drug class); and (6) provided medication information specific to their use in HF.

Primary outcome: EQUIP score

The primary outcome was educational resource quality, based on the Ensuring Quality Information for Patients (EQIP) score. EQIP is a validated 20-item questionnaire first described in 2004 that measures the quality of patient information on a scale from 0% (worst) to 100% (best) by assessing the domains of completeness, appearance, understandability, and usefulness (Supplemental Appendix S1). Each question has 3 possible answers with an accompanying score of 1 (yes), 0.5 (partly), or 0 (no). The total EQIP score was determined by averaging individual items’ scores using the following formula: [((# of Yes∗1) + (# of Partly∗0.5) + (# of No∗0))/20 – (# of does not apply))] ∗100 = % score. Domains that were not applicable to a given resource were excluded from the overall score calculation. Two reviewers (F.Y. and S.L.) evaluated each resource using the EQIP score, with a discrepancy between reviewers defined as a difference of ≥ 10% in EQIP score. Discrepant scores, which could occur due to the subjective nature of some of the questions, were reviewed by the 2 reviewers in order to reach a consensus, with disagreements resolved by a third author (R.T.). Following resolution of discrepant scores, the 2 EQIP scores calculated for each resource were averaged. This final averaged score was used in the analysis and to categorize each resource as one of the following: (i) well written, high-quality (score of 76%-100%); (ii) good quality with minor problems (score of 51%-75%); (iii) serious problems with quality (score of 26%-50%); and (iv) severe problems with quality (score of 0%-25%), consistent with categories recommended in the original EQIP development publication.

Secondary outcome: readability

Readability was assessed primarily using the Flesch-Kincaid Grade Level score, and secondarily, via the Flesch Reading Ease score, by copying the text of each resource into a Microsoft Word 2016 (Microsoft, Redmond, WA) document and using the software’s built-in readability function. The Flesch-Kincaid Grade Level and the Flesch Reading Ease scores have previously been validated and used to measure the readability of patient educational resources.,, In the US, an estimated 61% of adults read at a 6th-grade level, and it is therefore recommended that patient information resources be written at a 6th- grade or lower reading level.,, For this study, HF medication patient educational resources with a higher than 6th-grade reading level were considered to have low readability. We further used a Flesch Reading Ease score < 80 to indicate low readability.,, Lastly, we collected data on each resource’s total word count, for descriptive purposes.

Statistical analysis

We performed descriptive analyses, and we report data using medians and interquartile ranges (IQRs) for continuous variables, and frequencies and percentages for categorical variables. All analyses were performed using Microsoft Excel 2016 (Microsoft, Redmond, WA).

Results

From 693 search results, we identified 39 HF medication educational resources that met our eligibility criteria (Fig. 1). Despite the methods employed and the fact that the searches conducted were identical, F.Y.’s and S.L.’s search results differed slightly, likely due to differences in geographic location (Alberta vs British Columbia). Key characteristics of HF medication educational resources are described individually in Table 1, and summary statistics are provided in Table 2.

Figure 1

Study flow diagram. HF, heart failure.

Table 1

Individual characteristics and quality assessment of 39 HF medication educational resources, in descending order of EQIP score

Resource title	Developer/ funding source	EQIP Score, %	Flesch-Kincaid Grade Level	Word count
A Marvellous Guide to Medicines for Heart Failure https://pumpingmarvellous.org/wp-content/uploads/2018/05/Heart-Failure-Medicine-Guide.pdf	Pumping Marvellous	79	8.6	9039
Patient Education: Heart Failure (Beyond the Basics)https://www.uptodate.com/contents/heart-failure-beyond-the-basics	UpToDate	76	9	4778
Heart Failure - A Guide for Patients and Familieshttps://www.ottawaheart.ca/sites/default/files/uploads/heart-failure-patient-guide.pdf	University of Ottawa Heart Institute	73	8	7571
Heart Failurehttp://www.cardiacbc.ca/health-info/heart-conditions/heart-failure	Cardiac Services British Columbia	72	8	1020
Living With Heart Failurehttps://www.heartandstroke.ca/-/media/pdf-files/canada/health-information-catalogue/en-living-with-heart-failure.pdf	Heart & Stroke Foundation	71	6	20,853
Heart Failure Patient Education Handbookhttps://www.asante.org/app/files/public/2134/Heart-Failure-Handbook.pdf	Asante	71	7.5	4717
Heart Failure Diagnosis & Treatmenthttps://www.mayoclinic.org/diseases-conditions/heart-failure/diagnosis-treatment/drc-20373148	Mayo Clinic	71	9	4518
Digoxin: A Medicine for Heart Problemshttps://familydoctor.org/digoxin-a-medicine-for-heart-problems/?adfree=truetreat-heart-failure	familydoctor.org	70	6.5	929
Heart Failure Medicationshttps://www.ottawaheart.ca/heart-failure-patient-guide/heart-failure-medications	University of Ottawa Heart Institute	69	10	1638
Medications to Manage Heart Failurehttps://www.health.qld.gov.au/__data/assets/pdf_file/0030/993009/Medications-booklet-web-version.pdf	Queensland Government	69	10	3068
A Decision Aid for Entrestohttps://www.cardiosmart.org/docs/default-source/assets/decision-aid/heart-failure-drug-options.pdf?sfvrsn=aaff9c98_1	American College of Cardiology (CardioSmart)	69	8.7	1024
Introduction to Medicationshttps://ourhearthub.ca/medications/	Ted Rogers Institute for Heart Research	67	8.1	759
Heart Failurehttps://myhealth.alberta.ca/Health/pages/conditions.aspx?hwid=hw44415#tp17546	Alberta Health Services	67	6.2	6062
Congestive Heart Failurehttps://patient.info/heart-health/heart-failure-leaflet	patient.Info	65	8.7	2032
What Medications Are Used to Treat Heart Failure?https://heart-failure.net/medications	heartfailure.net	65	11.3	1077
A Quick Guide to Living With Heart Failurehttps://www.novartis.com/sites/novartis_com/files/nvs-hf-patient-booklet.pdf	Novartis / Pumping Marvellous	64	8	4070
Heart Failure Medicationshttps://my.clevelandclinic.org/departments/heart/patient-education/recovery-care/heart-failure/medications	Cleveland Clinic	64	6.4	2059
Managing Heart Failurehttps://www.coeuretavc.ca/-/media/pdf-files/canada/health-information-catalogue/en-managing-heart-failure-v3.ashx	Heart & Stroke Foundation	62	7.3	7400
What to Expect: Living With Heart Failurehttps://www.upmc.com/-/media/upmc/patients-visitors/education/documents/living-with-heart-failure-booklet.pdf	University of Pittsburgh Medical Center	61	8	5458
Commonly Prescribed Heart Failure Medicationshttps://heartfailureoxford.org.uk/patient/heart-failure-medications/	Heart Failure Oxfordshire	61	9.5	1531
Discharge Packet for Patients Diagnosed With Heart Failurehttps://www.heart.org/-/media/files/health-topics/heart-failure/hf-discharge-packet.pdf?la=en&hash=90463681A07EE6230276BC27A08F5D337D1D6D8C	American Heart Association	60	6	11,630
Medicines for Congestive Heart Failurehttps://healthy.kaiserpermanente.org/washington/health-wellness?item=/common/healthAndWellness/conditions/heartDisease/chfMedications.html	Kaiser Permanente	58	9.2	1045
Everything You Need to Know About Heart Failure Medicationshttps://www.healthline.com/health/heart-failure/heart-failure-medications	Healthline	58	10	1983
About Heart Failurehttps://www.cardiomyopathy.org/about-cardiomyopathy/heart-failure	Cardiomyopathy.org	58	10.2	890
CHF Booklethttps://www.meritushealth.com/documents/CHF-booklet.pdf	Meritus Health	56	9	5230
Heart Failurehttps://www.farxiga.com/heart-failure	AstraZeneca	56	10	3081
HF Patient Education Booklethttp://www.gov.pe.ca/photos/original/hpei_cp_hf_book.pdf	Prince Edward Island Government	55	6.7	4078
Heart Failure Handbookhttps://www.thechristhospital.com/Documents/Our%20Services/Heart%20Failure%20Handbook.pdf	The Christ Hospital	55	7	2120
Heart Failure—Treatmenthttps://www.nhs.uk/conditions/heart-failure/treatment/	United Kingdom National Health Services	54	11	1933
10 Drugs Commonly Prescribed for Heart Failurehttps://www.healthgrades.com/right-care/heart-failure/10-drugs-commonly-prescribed-for-heart-failure	HealthGrades	54	8	1428
Self-Care Guide for the Heart Failure Patient30	American Heart Association	54	9.6	1020
Heart Failure Medicineshttps://www.heartfailurematters.org/what-your-doctor-can-do/heart-failure-medicines/	heartfailuremattters.org	53	10	437
Treatment of Heart Failure?https://www.aahfn.org/mpage/treatement_hf	American Association of Heart Failure Nurses	50	7.5	2089
Understanding Heart Failure: Answers to Common Questionshttps://www.med.umich.edu/1libr/CCG/HeartFailure.pdf	Michigan Medicine	49	7.5	1328
Congestive Heart Failure Medicationshttps://www.medicinenet.com/congestive_heart_failure_medications/drug-class.htm	MedicineNet	49	12	1328
Heart Failurehttps://www.nhlbi.nih.gov/health/heart-failure	National Heart Lung and Blood Institute	49	8	5269
Heart Failure Medicineshttps://myhealth.alberta.ca/Alberta/AlbertaDocuments/Heart-Failure-Medicines-Oct-2019.pdf	Alberta Health Services	46	7.3	886
Medicationshttps://www.cardiosmart.org/topics/heart-failure/treatment/medications	American College of Cardiology (CardioSmart)	38	8.4	313
Heart Failure Treatmentshttps://www.ucsfhealth.org/conditions/heart-failure/treatment	University of California San Francisco Health	31	11	1272

EQIP scores range from 0% (worst) to 100% (best).

CHF, congestive heart failure; EQUIP, Ensuring Quality Information for Patients; HF, heart failure.

Table 2

Summary characteristics and quality assessment of heart failure medication educational resources

Characteristic	Value	First and third quartile (range)
Word count	2032	1045 to 4778 (313 to 20,853)
Format
Webpage	25 (64)	—
PDF	14 (36)	—
Country of origin
USA	22 (56)	—
United Kingdom	7 (18)	—
Canada	9 (23)	—
Australia	1 (3)	—
EQIP, median	61	54 to 69 (31 to 79)
High quality	2 (5)	—
Good quality, minor problems	30 (77)	—
Serious problems	7 (18)	—
Severe problems	0	—
Readability
Flesch-Kincaid Grade Level, median	8	8 to 10 (6 to 12)
≤6th grade	4 (10)	—
Flesch Reading Ease, median	59	52 to 65 (36 to 72)
≥ 80	0	—

Values are n (%), unless otherwise indicated.

EQIP, Ensuring Quality Information for Patients score; PDF, portable document format.

EQIP

The median EQIP score was 61% (IQR 54% to 68%; Table 2; Fig. 2A). Of the 39 included educational resources, 2 (5%) were of high quality, 30 (77%) were of good quality with minor problems, and 7 (18%) had serious problems with quality. Assessment of the specific components of the total EQIP score revealed that 36 resources (92%) used lay terms, 38 (97%) addressed the reader personally, and 39 (100%) had a respectful tone. However, only 2 resources (5%; including the highest-scoring resource) reported consulting with patients/family members during development. In terms of medication-specific information, 38 resources (97%) addressed the purpose of described HF medications, 28 (72%) qualitatively described benefits (eg, decrease mortality), 28 (72%) qualitatively described the medication risks and side-effects, and 21 (54%) addressed alternative medications (eg, angiotensin-receptor blocker if cough develops while taking an angiotensin-converting enzyme inhibitor). Further, 34 resources (87%) used generic names (instead of, or in addition to, brand names) and distinguished brand names as such, whereas the remaining 5 resources (13%) only discussed medication classes. Lastly, 24 resources (62%) did not address quality-of-life issues, with the remaining 15 (38%) only partly addressing this item. Only 5 resources (13%) included visual content to supplement the text.

Figure 2

Boxplot of (A) Ensuring Quality Information for Patients (EQIP) score and (B) Flesch-Kincaid Grade Level for 39 heart failure medication educational resources.

Readability

The median Flesch-Kincaid Grade Level was an 8th-grade reading level (IQR 8th to 10th grade) (Fig. 2B). Only 4 resources (10%) achieved the recommended 6th-grade reading level, and no resource had a 5th-grade or lower reading level. The median Flesch Reading Ease score was 59 (IQR 52 to 65), with all resources demonstrating low readability based on this score.

Discussion

HF is a chronic, complex condition that can be treated using a variety of healthy behaviours, medical devices, and medications. Provision of written resource for patients to review on their own time and at their own pace can significantly improve information retention, compared to verbal instruction., In this study, we found that most online HF medication patient educational resources were of good quality with minor problems, based on the EQIP tool. However, most HF medication educational resources were written at a grade level beyond the reading level of most people in the general population.

We identified several key areas for improvement among HF medication patient educational resources. In general, resources had a respectful tone, used generic names of medications, presented information in a logical order, and qualitatively described the purpose, side-effects, benefits (though not quantitatively,) and alternatives of medications. However, few followed a summative flow (ie, outlining the document content and covering what is stated), involved patients and caregivers in their development, or addressed quality-of-life issues.

Having a clear definition outlining the document content can help patients, caregivers, and healthcare providers understand whether the resource is suitable for the patient’s specific informational needs.,, Next, patients and healthcare providers may have different perceptions of what information is important to convey in educational materials.,,, Therefore, involving patients and caregivers in development of resources can help tailor educational materials to the needs of patients, making them more useful and relevant to their target audience. Further, describing the quality-of-life and practical impacts of medications ensures resource transparency, enables realistic patient expectations,, and allows patients to better understand and implement self-care and management strategies., A recent study established a framework for patient-oriented practical issues to be included in evidence summaries and shared decision-making tools. Such practical considerations align closely with quality-of-life issues incorporated into the EQIP questionnaire (eg, how a treatment impacts a patient’s daily routine or social life) and should be included in HF medication educational resources., These details are also more likely to align with patients’ individual goals of therapy (eg, improve symptoms, daily functioning, capacity for social interaction, etc.) and are crucial to include in patient education to engage patients in their care and medication management.

Our search identified resources with a wide range of readability, with few resources achieving sufficient readability. The readability level of HF medication educational resources thereby poses a problem, as patients may not be able to fully comprehend the included information, regardless of its quality. An additional consideration for barriers to use of educational resources is the length of time required to read through them. The average adult reads at a pace of approximately 250 words/minute. Of the top 5 highest-scoring resources based on the EQIP tool, the median word count was 4778, indicating approximately 19 minutes of reading time. For the average patient, who may be new to their diagnosis and receiving an overwhelming amount of information, resource length is likely a substantial barrier to absorbing and implementing the included information.

Limitations

Limitations to this study warrant discussion. First, we restricted resource inclusion to written materials available online. We did not capture information presented within social media communities (eg, Reddit, Facebook, etc.), video resources (eg, YouTube), or local/clinic-specific resources that are not available online. We anticipate greater variability of quality of these resources, as they may be subject to less curation. Second, the EQIP instrument was open to interpretation, which resulted in initial variability in scoring between the 2 reviewers. However, we mitigated this variability by incorporating duplicate scoring and an a priori plan to identify and resolve discrepancies. Third, the EQIP questionnaire used for our primary outcome defines quality from the perspective of healthcare professionals, rather than from the perspective of the end-user (ie, the patient). Therefore, it may not encompass all domains of quality or treatment considerations that are important to patients, or capture the relative value that patients place on each domain.

Conclusions

Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.

Funding Sources

The authors have no funding sources to declare.

Disclosures

The authors have no conflicts of interest to disclose.