Patient Information on Benign Colorectal Disease: An Assessment of the Value and Effectiveness of Traditional Methods.

Health literacy is the best predictor of health status, with patient information leaflets (PILs) commonly used to improve information access. However, they can often be inconsistent. Benign colorectal disease can be challenging for patients and ensuring they are accurate and understandable is important. Available PILs in a tertiary unit were assessed. The Flesch reading ease and Flesch-Kincaid Grade level scores were used to calculate objective readability. Subjective assessment of readability, understandability, and patient opinion was assessed using a questionnaire. All PILs had objective readability scores at age 14 or older, above recommended advice. Three hundred sixty patient questionnaires were collected. The relationship between subjective readability and understandability was significant (P < .05); the easier a patient was able to read the information the more likely they were to understand it. There was no link between objective and subjective readability-a more difficult calculated reading score didn't correspond to the patient finding it harder to read. Patients preferring paper information were significantly older than patients who preferred online information (P = .01). Patient information leaflets remain valued by patients, and PILs that patients find easier to read are then better understood; however, ease of reading is not related to objective readability scoring and there was no consensus that a shift to online information is merited.

Introduction

Health literacy is the single best predictor of an individual’s health status. A health literate individual has the knowledge to communicate with health professionals, make appropriate use of health services, and so contribute to improving the management of their own health issues. Engagement with health information is essential for shared decision-making, which itself benefits both the patient and health professionals involved. Patients with good health literacy skills can help other patients, spreading best practice and improving disease management (1).

However, it is known that patients do not retain a significant amount of information that is communicated during a consultation, which has been shown to be an important factor in noncompliance with treatment (2). Patient information leaflets (PILs) are commonly used to help (3). However, a consistent finding is that PILs require relatively high reading skills. People may then find PILs difficult to understand, especially if they are designed for the “adult” population (4).

Colorectal and anal diseases are common in the United Kingdom and 2% to 3% of the population have symptoms at any given time (5). Unfortunately there remains social taboo associated with symptoms and patient embarrassment often leads to delayed presentation and diagnosis (6). Conditions include anal fissures, diverticular disease, pruritus ani, and irritable bowel syndrome (IBS). Symptoms need to be explored with patients without eliciting embarrassment. It is therefore important to have accessible information that can improve understanding and lead to less of a delay in seeking advice (7).

Readability (how easy it is to read) is often a point of research when designing PILs. It is now a widely practiced assessment in their creation and the consideration of this would be deemed best practice (8). However, there have been challenges to the idea that readability scores are a sufficient indication of how well a patient will be able to read a PIL.

In addition, the debate over the superiority of online or paper-based information has become more pronounced. There has been a trend toward online information, but to our knowledge, there has been no research on the preferences of general surgery patients. This is particularly important as many of these patients are older; while technology uptake among older citizens is increasing, they are much less likely to be online access than younger people and this divide will continue for the foreseeable future. Internet usage has also been shown to correlate with level of income, education and social exclusion, attributes that are often associated with older people (9). It is therefore important to assess if there is still a desire for paper-based information, and what the factors that influence this are.

Aims and Objectives

Given the importance of patient information, we aimed to assess if the PILs available in a surgical department regarding benign colorectal conditions were easy to read, using both mathematical analysis and patient opinion.

To calculate readability with validated scoring systems and assess if these were comparable with reading ages of the population.

Assessing opinions of patients about PILs provided, to gauge if they find them useful and if they still value traditional methods of providing information compared to newer, online methods.

Information that was deemed difficult to read in either or both of these aspects would indicate improvement in the PILs was necessary to enhance the patient experience in our department.

Methodology

The 4 diseases included in the PILs were anal fissures, pruritus ani, diverticular disease, and IBS. The readability of PILs can be calculated by various methods. The Flesch Reading Ease score (FRE score) and Flesch-Kincaid Grade level (FKG level) are 2 validated scores. They use word and sentence length to provide scores of readability and education level of a piece of text (10 –12). This was done both manually and electronically for reliability.

The scores from the first formula, the FRE calculation, could then be compared to Table 1, which was originally created by Flesch and which has been adapted to demonstrate the equivalent UK school year.

Table 1.

The Flesch Reading Ease Score and Equivalent Educational Level.

Flesch Reading Ease score	School level (UK equivalent)	Notes
100.0-90.0	Year 6 (10-11 years old)	Very easy to read
90.0-80.0	Year 7 (11-12 years old)	Easy to read. Conversational English
80.0-70.0	Year 8 (12-13 years old)	Fairly easy to read
70.0-60.0	Year 9/10 (13-15 years old)	Easily understood by 13-15 year olds
60.0-50.0	Year 11-13 (16-18 years old)	Fairly difficult to read
50.0-30.0	University level	Difficult to read
30.0-0.0	University graduate	Very difficult to read

The FRE score and FKG level cannot be used to objectively assess the influence of illustrations or diagrams on readability. The only validated tool for this is the Suitability Assessment of Materials (SAM) (13). Various factors can be evaluated using this, including graphics and layout. The SAM tool was used for further analysis of PILs that had diagrams, which were IBS and diverticular disease/diverticulitis. Each factor is rated as superior, adequate, or not suitable. Each of these have a corresponding statement that indicates the standard it represents.

The final objective was to assess patient opinions on the PILs. Responses were gathered from all general surgical inpatients in the hospital, rather than only from patients who had the disease in question. The exclusion criteria were patients who did not have capacity to answer the questions and patients who did not speak English. The voluntary and anonymous nature of participation and answering the questionnaire was explained. It is shown in Figure 1. This was done over a period of 5 months. Patients were given an identical copy of the leaflets. These were all in black/white and A4 in size. Patients were only given one PIL to read before completing the questionnaire.

Figure 1.

The questionnaire given to patients.

Statistical analysis was performed using SPSS 23 (14). After discussion with the local ethics committee, a formal application for full approval was not required.

A further cohort of patients were asked to review online patient information on the same diseases and answer the same questionnaire. The only alteration to Figure 1 was that the use of “leaflet” was changed to “website.” The same exclusion criteria were applied.

All patient information shown to patients was from the NHS A-Z website (15). This project has a focus on the readability and understandability of patient information, and testing computer literacy in addition was not thought to be appropriate. For this reason, the patient did not have to navigate to the webpage. Analysis was performed using SPSS 23 (14).

Results

Readability Assessments

The anal fissure PIL had a calculated FRE score of 51.02 and an FKG level of year 10, which is approximately 14 years old. The pruritus ani PIL had a calculated FRE score of 40.91 and an FKG level of sixth form (17-18). The IBS PIL had a calculated FRE score of 40.18 and an FKG level of sixth form. The diverticular disease/diverticulitis PIL had a calculated FRE score of 29.06 and an FKG level of sixth form. Table 2 shows these results. The closer the score to 0, the more difficult it is deemed to read. The pruritus ani and IBS PILs both have scores that equate to university standard reading levels and would be judged “difficult to read.” The diverticular disease/diverticulitis PIL has the lowest score, equating to university graduate level.

Table 2.

Readability Scores for PILs Available in the Department.

Patient information leaflet	Flesch Reading Ease score	Flesch-Kincaid Grade level score (UK equivalent)
Anal fissure	51.02	Year 10 (age 14)
Pruritus ani	40.91	Sixth form (age 17-18)
Irritable bowel syndrome	40.18	Sixth form (age 17-18)
Diverticular disease	29.06	Sixth form (age 17-18)

Abbreviation: PIL, patient information leaflet.

Further analysis was completed using the SAM tool. The IBS PIL was rated as “adequate” for type of illustrations, “superior” for relevance of illustrations, and “not suitable” for captions, as there were none. It was “superior” for typography and “adequate” for layout. The diverticular disease/diverticulitis PIL was rated as “adequate” for type and also “adequate” for relevance. It was deemed “not suitable” for captions, as there were none. It was “superior” for typography and “adequate” for layout.

Although the SAM guide suggests 2 points per superior rating and 1 point per adequate rating, there is no overall score that this could be compared to.

Questionnaires Regarding the PILs

Three hundred twenty questionnaires were collected; 80 questionnaires were collected for each of the 4 conditions. The average overall age of patients was 54.2 years old, with a median age of 53 and an age range of 19 to 84. One hundred eighty-three patients were female and 137 were male.

Within the 4 groups for each PIL, there were some differences in age; the average age of the IBS group was 60.1 years old, 52.9 years old in the anal fissure group, 51.2 years old in the pruritus ani group, and 52.6 years old in the diverticular disease group. Dependent t test analysis showed that the difference in age between the IBS group and the other 3 groups was significant (P = .01) but was not significant in other combinations.

Likert scale analysis was carried out for patient-assessed readability and how well they felt they understood the PIL. The mean score for ease of reading for the combined 320 results was 3.89, with 5 representing “very easy” to read and 1 representing “very difficult.” When this was assessed within each of the 4 PIL groups, the mean scores were 4.31 for IBS, 3.53 for anal fissure, 3.40 for pruritus ani, and 4.30 for diverticular disease/diverticulitis.

The subjective readability results inversely correlate with the objective readability scores calculated using the FRE score. The PIL on diverticular disease/diverticulitis had the lowest score in this, but patients themselves found that it was the easiest PIL to read. The average score of the subjective understandability aspect was 3.78. The patients who read the IBS PIL gave an average score for ease of understanding of 4.36, anal fissure of 3.21, pruritus ani of 3.16, and diverticular disease/diverticulitis of 4.34. Spearman rank correlation analysis on subjective readability and understandability showed there was a statistically significant relationship (P < .05) for all 4 PILs between readability and understandability. The easier a patient was able to read a PIL, the easier they found it to understand.

Further assessment of readability and understandability from the combined results with Mann-Whitney U analysis, against if patients had had the disease before, showed that having had the disease significantly improved both readability and understandability (P< .05). There was no significant relationship between readability/understandability and age or gender.

Another focus of the questionnaire was regarding patient preference for paper or online information. In χ2 analysis, patients who had not previously had the disease were significantly less likely to prefer online information (P < .05), with the majority selecting paper as the preferred option. This is shown in Figure 2. Patients who had previously had the disease were more likely not to have a preference.

Figure 2.

Graph showing the preferred information modality of patients compared to if they had previously had the disease.

There was a significant relationship between age and preferred information modality, determined by 1-way analysis of variance (F 2,317 = 6.642, P = .001). A Tukey post hoc test showed that the patients who preferred paper information were statistically significantly older than patients who preferred online information (P = .01) or had no preference (P = .006). There was no significance when patients who had no preference were compared with patients who preferred online information (P = .873).

Previously having had the disease also made a patient more likely to be able to explain the disease to a family member after reading the PIL. Chi-square analysis showed statistical significance (P = .001). This is shown in Figure 3.

Figure 3.

Graph showing the number of patients confident to explain the disease to family members after reading the patient information leaflet (PIL), compared to if they had previously had the disease.

Questionnaires Regarding Online Patient Information

Thirty-two patients completed the questionnaire after viewing the online information, 8 patients for each disease. The average age was 59.8 years old, with a median age of 61 and an age range of 34 to 80 years old. Seventeen patients were female and 15 patients were male.

Likert scale analysis was again carried out with regard to patient-assessed readability and how well they thought they understood the online information. The mean score for ease of reading, for the combined 32 results, was 3.81, with 5 representing “very easy” to read and 1 representing “very difficult.” The mean score for ease of understanding was 3.75. Spearman rank correlation analysis on subjective readability and understandability again showed that a higher score for ease of reading correlated with a higher score for ease of understanding. This was not statistically significant (P = .21).

Both of these results are marginally lower than the scores for the paper PILs (3.89 and 3.78, respectively), but again this was not statistically significant. Results are demonstrated in Table 3.

Table 3.

Comparing the Overall Results From Questionnaires After Reading Paper PILs and Online Information About the Disease.

Comparison between the survey results based on PILs and on online information
Variable		Paper PILs (n = 320)	Online (n = 32)
Age	Median (range)	53 years (19-84 years)	61 years (34-80 years)
Gender (total number of patients)	Women	183 (57%)	17 (53%)
	Men	137 (43%)	15 (47%)
PIL group (total number of patients)	IBS	80 (25%)	8 (25%)
	Anal fissure	80 (25%)	8 (25%)
	Pruritus ani	80 (25%)	8 (25%)
	Diverticular/diverticulitis	80 (25%)	8 (25%)
Patients who have had the disease previously (total number of patients)	IBS	22 (28%)	3 (38%)
	Anal fissure	9 (11%)	1 (9%)
	Pruritus ani	7 (9%)	0 (0%)
	Diverticular/diverticulitis	16 (20%)	2 (25%)
Readability (average score)	IBS	4.31 (/5)	4.25 (/5)
	Anal fissure	3.53 (/5)	3.49 (/5)
	Pruritus ani	3.40 (/5)	3.40 (/5)
	Diverticular/diverticulitis	4.30 (/5)	4.10 (/5)
Ease of understanding (average score)	IBS	4.36 (/5)	4.11 (/5)
	Anal fissure	3.21 (/5)	3.66 (/5)
	Pruritus ani	3.16 (/5)	3.25 (/5)
	Diverticular/diverticulitis	4.34 (/5)	3.98 (/5)
Preferred modality (overall)	Paper	142 (44%)	19 (59%)
	Online	57 (18%)	9 (28%)
	No preference	121 (38%)	4 (13%)
Could the patient explain more about disease to family after reading PIL?	Yes	268 (84%)	31 (97%)
	No	52 (16%)	1 (3%)
Was the PIL useful? (total number)	Yes	250 (78%)	31 (97%)
	No	70 (22%)	1 (3%)

Abbreviations: IBS, irritable bowel syndrome; PIL, patient information leaflet.

Discussion

Assessment of Readability

Objective assessment of readability using the FRE score, FKG level, and SAM tool showed that none of the PILs were at the recommended reading level. All required a reading age of 14 or older. The PIL regarding diverticular disease required the reading ability of a university graduate. The average reading age in the United Kingdom is approximately 11 years old (4), which is exceeded by all the PILs. This may demonstrate that the reading age was not taken into account at all during the initial design phase, and that earlier patient involvement is required.

The use of diagrams is important. As it is the only validated score for this the SAM tool was used. It has limitations, as the scoring system is subjective. The SAM scores correlated with both readability scores in that the IBS leaflet was easier to read than the diverticular disease/diverticulitis leaflet.

However, the question that arises is whether standard measures of readability are appropriate in PILs. The number of syllables in a word is a key factor in the formulae, but often medical terms have a lot of syllables due to their etymological origin. For example, “diverticulitis” has 6 syllables and when used frequently in the PIL impacts the readability score. However, a patient with the disease who is reading the PIL is unlikely to be completely unfamiliar with the word. Formulas rate short difficult terms as more readable than less complicated polysyllabic words (16); for example, “perforate” has half as many syllables as “diverticulitis” but may not be a word that even someone with the disease is familiar with. This could explain findings in subsequent sections that showed no correlation with these objective scores.

Patient Questionnaires

From the 320 patients who completed the questionnaire it is clear there is a wide age range. This is not unexpected given that the department admits patients from all general surgical subspecialties, with predictable demographic changes for conditions such as appendicitis or diverticulitis (17). Of this, 57% of the patients were female. Statistics from the European Institute for Gender Equality, isolated for the United Kingdom, show that a higher proportion of hospital inpatients are female (18). Additionally, it is known that females are more likely to use health care services than males (19).

One of the more intriguing findings was that objectively assessed readability inversely correlated with subjective results from the questionnaires. The PIL about diverticular disease/diverticulitis was deemed to be the most difficult to read. However, it was thought to be the easiest to read by the patients surveyed. This aspect of the results adds further weight to concerns about the use of these formulae in the construction of a medical PIL, and although it is more time-consuming using focus groups of patients to review PILs may be a better way of assessing them.

Statistical analysis showed that the easier a patient found reading a PIL the more easily they felt they were able to understand it. Although this is not an unexpected finding, and a strong correlation has been shown in previous studies (20), it adds credence to the point that patient input is perhaps the most critical part of designing these information leaflets.

A key objective of the project was to determine whether patients still valued “traditional,” paper-based information or whether the societal shift toward online information is replicated here. Statistical analysis showed a significant relationship between age and preferred modality. Patients who preferred paper-based information, compared to online or no preference, were more likely to be older. Younger adults are known to have adapted to the use of information technology significantly quicker than older adults, with websites and online programs tending to then benefit younger adults at the expense of, and while underestimating, older people (21). These results clearly answer one of the primary research questions and show the need to retain a reliable paper-based PIL selection to cater for this population.

The results of a comparison between ease of reading and understanding for online information again showed a positive correlation between the 2, although the overall mean for both was slightly lower than for the paper PILs. There was a lack of statistically significant results from this section, and a clear limitation is the small number of responses. This is a basis for further research. A possible next step is a matched study, giving a cohort of patients both paper-based and online information to compare readability and understandability. Studies as to whether new readability scores are required, specific for online information, would also be merited.

Project Limitations

The FRE and FKG level scores were used to calculate readability, but there are other formulae and using a number of these may have demonstrated a different pattern of results.

With regard to the patient questionnaires, the 4 groups of patients were not matched for age and gender—this would have improved the strength of significant findings, but was not thought to be practical given the high turnover of inpatients on the wards in question. The small number of responses gleaned for the online information is also a limitation, demonstrated by the lack of statistically significant results. This stage was commenced at a later stage than questionnaires about the paper PILs and is a point of improvement for future work. Making alterations to the PILs and then resurveying the same patients could have given more insight into the most important aspects that need to be addressed.

A final query is if findings supporting paper-based information will continue to be applicable in future, as digitalization of information continues to rapidly progress and with evidence that shows it can also improve health knowledge, with increased user satisfaction (22).

Conclusions

Patients found that PILs they could read more easily could also be understood more easily. However, this ease of reading is not related to objective readability scoring and did not correspond with expected results based on readability formulae.

There was no clear consensus that a shift to online information is merited. Indeed, we have confirmed in our findings that older patients still prefer traditional, paper-based leaflets. Whether this will remain the case in an increasingly digital era is uncertain and should be the subject of ongoing research. The same patients should be surveyed on both modalities to allow a more accurate comparison of patient opinion.

Overall the project demonstrates important findings with clear and definite room for improvement. More research is needed to establish scoring systems and assessment methods that better reflect the need for proper PIL design.