Readability in printed education materials for Chinese patients with systemic lupus erythematosus: a mixed-method design.

OBJECTIVES: To assess the readability of printed education materials (PEMs) for patients with systemic lupus erythematosus (SLE) and to explore the perceptions of patients with SLE with different health literacy regarding the readability of PEMs.
DESIGN: A mixed-methods study, including a cross-sectional survey and semistructured interviews.
SETTING: The SLE PEMs were collected from 13 hospitals in China. The interviews were conducted in the Department of Rheumatology of a hospital in Hefei, China. PARTICIPANTS: In the cross-sectional survey, convenience sampling was used to select the Chinese SLE PEMs, with 20 PEMs included. In the qualitative study, the patients with SLE were divided into two groups based on their health literacy. Then, purposive sampling was used to select participants in each group, with 18 patients recruited. OUTCOME MEASURES: The readability of PEMs was assessed by the language analysis technology and the Chinese version of the Suitability Assessment of Materials (SAM-C) instrument.
RESULTS: For text factors of readability, the mean Chinese language difficulty coefficient was 67.09±8.03, which indicates that the text of PEMs was difficult to read. For non-text factors, the mean SAM-C score was 45.62±9.51. Eight PEMs were rated not suitable, 12 were adequate and none were superior. In the interviews, eight categories were identified: information source, content, actionability, plain language, pictures, tables, numbers and layout. Patients with different health literacy had discrepant views on the detail of basic information, the necessity of question list, the location of functional pictures and the application of mathematical symbols.
CONCLUSIONS: The readability of Chinese SLE PEMs does not perform well, and it is necessary to reduce the difficulty of words, shorten the length of sentences and improve the picture design and actionability. To develop PEMs tailored to patients' level of health literacy, patients' unique view of readability should be integrated into the design of PEMs. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The readability assessment result of systemic lupus erythematosus printed education materials (SLE PEMs) came from the researchers’ evaluation combined with views of patients with SLE patients.

Patients’ perceptions of the readability of SLE PEMs were compared across different health literacy levels, so as to give customised design for PEMs.

All the participants in this study were hospitalised women with SLE and most of them were younger, and this might lead to selection bias, as the results cannot be generalised to outpatients with milder disease states, male patients and elderly patients.

The measurement results of the language analysis technology can reflect the relative difficulty of the text but cannot indicate what level of education is appropriate for the text.

Introduction

Systemic lupus erythematosus (SLE) is a chronic multisystem autoimmune disease with a variety of clinical manifestations and lifelong recurrent flares, which may lead to extensive tissue and organ damage. In China, the prevalence of SLE ranges from 30 to 70/100 000 population with a generally higher prevalence compared with some Asian countries (such as India and Japan).1 2 SLE typically presents between the ages of 15 and 45 years, and the female-to-male sex ratio is 9:1.2 Nearly 55.5%–66% of the patients with SLE have episodes of flares or persistently active disease per year.3 4 SLE disease activity has been found to be influenced by health literacy.5

Health literacy is the degree to which individuals have the capacity to obtain, process and understand the health information needed to make appropriate health decisions.6 Low health literacy is associated with less health-related knowledge, poorer ability to demonstrate taking medications properly and interpret health information, increased use of healthcare services, and higher prevalence and severity of disease.7 Our previous work found that 55% of patients with SLE in China had low health literacy, which was associated with high disease activity and poor medication compliance.5 This result suggests that improving the health literacy of SLE patients may help them control the disease. In fact, health literacy improves when there is good interaction between the literacy skills of individuals (print literacy, numeracy, communication) and the health literacy demands of health information (the complexity and difficulty of health information).8 In other words, if the health literacy demand of information is so high that the individual is unable to process the information with his or her existing literacy skills, the individual will not understand the information and apply it during the decision making. Therefore, both Healthy China 20309 and the US National Action Plan on Health Literacy10 advocated the development of health information that can be accessed, understood and applied by people with different health literacies. Health education materials are recording carriers of health information in health communication activities, including printed education materials (PEMs), audio-visual materials and online information,11 which are commonly used during the educational process as auxiliary tools.12 Approximately 38%–58.6% of the Chinese population obtains health information through PEMs.13–15 Therefore, it is critical to ensure that the health literacy demands of PEMs match the literacy skills of readers.

Readability is the key indicator to evaluate the health literacy demands of PEMs.16 Readability refers to the degree to which a given class of people find certain reading matter compelling and comprehensible.17 The assessment of readability is the evaluation of its influencing factors, which can be divided into external factors and internal factors.18 External factors include the influence of readers (interest, motivation and familiarity with content) and the environment on reading comprehension.19 Compared with the variability and subjectivity of external factors, internal factors can be quantified objectively and accurately; therefore, they have received more attention from researchers.20 Internal factors are the sum of all elements within the materials that affect reading comprehension,18 including text factors (the difficulty of vocabulary and sentences) and non-text factors (content, structure, design).18 21 22 To evaluate the readability of PEMs comprehensively, researchers often use readability formulas with rating scales (eg, the Suitability Assessment of Materials, SAM-C).23 This multitool assessment method has been applied to PEMs for chronic kidney disease, heart failure and asthma.24–26 The evaluation results of text factors suggest that the average reading level of materials is often higher than the educational level of readers. For non-text factors, PEMs perform well in terms of layout but not well in pictures, cultural appropriateness and content.

Currently, only one study has used readability formulas to assess the text factors in the readability of SLE PEMs written in English, and the reading level of the materials was too high for most patients.27 This situation may lead SLE patients to misunderstand their condition and treatment, which in turn could cause noncompliance and unnecessary hospital admissions. In China, the rheumatology department in most hospitals provides PEMs to SLE patients, but the readability of commonly used SLE PEMs is unclear. It is the gap in health education among patients with SLE.

In most studies, healthcare providers have evaluated the readability of materials, but these results might not fully reflect readers’ view of readability because the two groups have discrepancies in medical background and reading level.22 28 Therefore, the reader’s perceptions of the readability of PEMs should be assessed and supplemented with professional evaluation.27 29–31 Several studies have provided evidence of differences in the way people with different levels of health literacy find, understand and use health information.32 33 As a result, patients with high and low health literacy may have diverse preferences for the readability of the materials. However, there are no reports of these patients’ perceptions of the readability of SLE PEMs.

To address the gaps highlighted above, this study aimed to (1) evaluate the readability of PEMs for SLE patients based on the text and non-text factors of readability and (2) explore the perceptions of SLE patients with high and low health literacy regarding the readability of PEMs.

Methods

This study used a mixed-method design34 that incorporated quantitative and qualitative survey data in two steps. In step 1, language analysis technology was used to evaluate the text factors of the readability of SLE PEMs, and the Chinese version of the SAM-C instrument was used to access the non-text factors of readability. In step 2, two materials with the lowest and highest rating scores analysed from the SAM-C were distributed to patients with SLE with different levels of health literacy before they participated in semistructured individual interviews. Then, the two groups of patients’ views and suggestions on the readability of these materials were collected and compared.

PEM collection

SLE PEMs were selected from hospitals in Hefei, Nanjing, Hangzhou, Shanghai and Changsha by convenience sampling. The included materials were Chinese-language PEMs designed for patients with SLE that were used by rheumatology departments and provided free information about health promotion, disease prevention, diagnostic procedures, treatment modalities and self-management regimens. Materials were excluded if they were duplicates or printed PowerPoint documents, had excessive information about hospital regulations (eg, instructions for safe exit) or were medicine advertisements. To record the characteristics of PEMs, we designed a general information form, including four items: forms (leaflet, folding, pamphlet), publication source (self-designed, commercial company, non-profit organisation), information source (reference, expert statement) and publication date.

Quantitative assessment of PEMs

Text factors assessment

Text factors of readability were assessed by the language analysis technology. We measured a set of linguistic features, including the average number of sentences per hundred words, the average sentence length, the average number of difficult words per hundred words and the Chinese language difficulty coefficient (table 1).35 The textual part of PEMs was included in the assessment, excluding the text from titles, catalogues and tables. The Natural Language Processing and Information Retrieval Sharing Platform is a Chinese lexical analysis system developed by the Institute of Computing Technology, Chinese Academy of Sciences (https://github.com/NLPIR-team/NLPIR). We used this system to segment words in the sentence.

Table 1

Measurement methods of Chinese linguistic features

Linguistic feature	Formula	Operation steps	Criteria
ANS	number of sentences *100number of words	1. No of sentences: Syncopate sentence by Chinese punctuation mark (eg, ‘。’, ‘?’, ‘!’, ‘; ‘, ‘……’) and count them.	>10 for simple materials; 6–10 for primary materials; <6 for intermediate and difficult materials.
		2. No of words: The total no of words in the texts excluded Arabic numerals and English letters.
ASL	number of wordsnumber of sentences	—	—
ANDW	number of difficult words *100number of words	Difficult words: The words in third, fourth and superclass grades of <Vocabulary and Characters of Different Hsk Levels>61	—
		1. Segment words by NLPIR Chinese lexical analysis system.
		2. Build word database based on <Vocabulary and Characters of Different Hsk Levels>by SQL Server software (Microsoft, Redmond, Washington, USA).
		3. Calculate the word frequency of each grade in the text with SQL Server software.
CLDC	ASL +ANDW	—	20–30 for intermediate materials; >30 for difficult materials.

ANDW, average number of difficult words per hundred words; ANS, average number of sentences per hundred words; ASL, average sentence length; CLDC, Chinese language difficulty coefficient; NLPIR, Natural Language Processing and Information Retrieval Sharing Platform.

Non-text factors assessment

Non-text factors of readability were assessed by the SAM-C instrument. In 2014, the SAM-C was translated from the original SAM, and administered to asthma PEMs to test its psychometric properties, including content validity, internal consistency, and IRR.36 The SAM-C consists of 22 items grouped into six domains: content, literacy demand, graphics, layout and typography, learning stimulation and motivation, and cultural appropriateness. Content evaluates whether the purpose is evident, the topic is behaviour focused, the scope is limited to purpose or objectives, and summaries are provided. Literacy demand evaluates whether the reading difficulty of the text is appropriate, the text uses a conversational style and active voice, the vocabulary uses common words, context is provided before new information, and topics are preceded by advance organisers such as headers. Graphics evaluates whether the cover graphic shows the purpose, the illustrations are recognisable and familiar, the illustrations are relevant to key messages and have captions, and lists and tables are explained. Layout and typography evaluates whether the organisation and printing of pages are easy to read, such as visual cueing, type size and typographic cues. Learning stimulation and motivation evaluates whether the material includes reader interaction and detailed behavioural information and whether complex topics are subdivided into small parts. Cultural appropriateness evaluates whether the text and the graphics of the material correspond with the logic, language and experiences of the intended culture. Each item is rated as superior (2 points), adequate (1 point) or not suitable (0 point). Items that do not apply to the materials are rated as not applicable. In this study, dimension scores and overall scores were calculated as a percentage of the maximum possible score [dimension score=the sum of the ratings in this dimension/(the number of items in this dimension - the number of nonapplicable items in this dimension)×2, overall score=the sum of the ratings/(the number of items - the number of nonapplicable items)×2]. The resulting percentages were classified as superior (70%–100%), adequate (40%–69%) or not suitable (0%–39%).36

In the original SAM, the item ‘reading grade level’ is used to evaluate the text factor of readability, which is measured by Flesch-Kincaid formula.22 However, as there is no suitable Chinese readability formula that can measure the reading grade level of materials, we assessed the text factors by language analysis technology which can reflect the reading difficulty of the text, and the ‘reading grade level’ item was removed. Hoffmann et al37 indicated that due to the subjective rating criteria of some items in the SAM, which may allow raters to have different interpretations, these items had low IRR. To improve the consistency of raters’ understanding of the items, we refined the scoring criteria by consulting the researcher who translated the instrument, summarising the interpretation of the scoring criteria from the literature and finally developing the draft of the SAM-C handbook. Two raters used this handbook to assess five PEMs independently. IRR was assessed using Cohen’s kappa (k) statistic, which was calculated for each item, domain and the overall scale. Substantial agreement (k=0.61–0.80) was chosen as the lowest acceptable range for the raters to achieve for all items,38 and we improved language clarity and accuracy for items with k less than 0.61. We repeated the same process for the modified instrument until k was higher than 0.61 and obtained the final edition of the SAM-C handbook, which had ‘substantial’ to ‘almost perfect’ agreement (k=0.615–1.0) across all the domains and ‘substantial’ agreement (k=0.8) for the overall scale.39

Two raters, both with medical backgrounds, used the SAM-C handbook to assess SLE PEMs independently after a training session, including the steps of using the handbook and the method of scoring. Inconsistencies were resolved by group consensus.

Statistical analyses

Descriptive statistics were used to describe the PEM characteristics and the text and non-text factors of the readability of the materials. The categorical data were summarised by number/percentages, while the continuous data were described by the mean with SD or the median with IQR. IRR was assessed using Cohen’s kappa statistic. Cohen’s kappa was calculated by Stata V.15.0 (StataCorp). The other analyses were performed using SPSS V.24.0 (SPSS).

Qualitative assessment of PEMs

We wanted to gain insight into how SLE patients with varying health literacy levels perceive the readability of PEMs and take their opinions as a complement to the assessments by healthcare providers. The Consolidated Criteria for Reporting Qualitative Research checklist was used to ensure complete and transparent reporting.40

Recruitment of participants

We recruited SLE inpatients at the First Affiliated Hospital of Anhui Medical University, Hefei, China. Potentially eligible patients met the SLE classification criteria as revised in 1997 by the American College of Rheumatology and were over 18 years old. Those who were illiterate, had neuropsychiatric SLE, had a score on the Chinese version of the Mini-Mental State Examination less than 27,41 were unable to communicate due to sensory impairment (such as patients who were deaf and mute) and were unable to read because of serious organ damage were excluded. For patients who met the inclusion and exclusion criteria, stratified purposive sampling was used to select the interviewees. First, we divided them into two groups according to their health literacy, which was measured by the Health Literacy Scale for Patients with Chronic Disease (<96 is considered ‘low health literacy’ and ≥96 is ‘high health literacy’).42 Then, purposive sampling was used to select participants with different ages, sexes, education levels and courses of disease in each group. The sampling was discontinued when saturation had been reached. To ensure data saturation, we chose another participant for verification after we found that no new information was emerging from the data.43

Data collection

We chose individual interviews rather than focus groups to collect data, because individual interviews were a discretionary choice that focus groups could potentially keep illiterate persons to conceal their opinion to avoid losing face.33 From the quantitative study, we chose two PEMs with the lowest and highest readability scores according to the assessment results of the SAM-C handbook as stimulation materials in the interview. If the participants were asked to read two PEMs in a short time, they might be bored and skim through the materials because of the large amount of reading, which might lead to inaccurate interview results. With that in mind, every participant received two individual interviews with a break of 2–3 days between the interviews. Before the first interview, the participants were given the material with the lowest score and asked to read it; the material with the highest score was provided before the second interview. The patients were asked to share their perceptions and improvement suggestions for the readability of these materials. All interviews were conducted in the conference room at the hospital, which was quiet and private. After the participants completed a short questionnaire of demographic information, the interviews were conducted and audio recorded. Field notes were used to record participants’ facial expressions and body movements, the researcher’s understanding of patients’ perceptions during the interviews and the summary of representative views after the interview. To ensure that the main issues were discussed, we developed an interview guide based on text and non-text factors of readability, which included the questions about the source of health information and the perceptions of readability (box 1). We conducted two pilot interviews using an initial interview guide and one was with high health literacy participants and the other was with low health literacy ones. Because there was no difference between the questions in the initial interview guide and the formal guide, the audio files and the field notes obtained in the pilot interviews were also used in the data analysis. All interviews were conducted by QW, who took part in some courses focusing on the theory and the practice of qualitative research, participated in the forum on the development and diffusion of qualitative research, and gained a lot of experience by conducting some interviews in other research projects. A summary of each interview was sent to each patient to make clear that if her view had been described correctly by the researcher. Finally, if there was any change in the summary, the revised version would be used in the subsequent analyses.

Major questions

How do you usually get health information? How often do you use printed education materials?

Do you like the amount of information in this material?

Do you like the order of presentation of the various topics in this material?

Do you like the expression of words and sentences in this material?

There is some numerical information in this material. Do you like this form?

What do you think of the design of pictures and tables in this material?

What do you think of the layout of this material?

How do you feel about the effect of learning after reading this material?

What further improvements can be made to this material?

Data analysis

The interview recordings were transcribed verbatim and integrated with the recording of the facial expressions and body movements of patients. The transcripts of two interviews with the same patient were combined into the overall views on the readability of PEMs. The interview data were divided into two groups based on the level of health literacy and analysed. Then, the perspectives of the patients in the two groups were compared.

The analysis used combined deductive and inductive content analysis with NVivo V.11.0 (QSR International, Burlington, USA).44 Because the purpose of the qualitative research was to collect patients’ opinions on the readability of the materials, the data analysis remained at the manifest level describing categories and did not involve more in-depth interpretation to derive latent content.44The first phase was deductive,45 and we developed a framework of broad prior categories (eg, content, graphic, layout) identified in the SAM-C handbook to guide the analysis. QW and LW independently reviewed each transcript carefully, highlighting all text that appeared to describe reading experience, which constituted the unit of analysis. All highlighted text was coded using the predetermined categories wherever possible. Text that could not be coded into one of these categories was coded with another label named ‘other views’. Then, we compared coding, reconciled differences and refined the framework. The next phase used inductive content analysis.46 QW and LW independently analysed the text in each category and in the ‘other views’ label for the first five interviews and divided them into meaning units that were condensed. They then abstracted the condensed meaning units to create a preliminary list of codes. We discussed inconsistencies in the coding process and created a refined code book with explicit definitions and examples. QW and LW subsequently coded the remaining interview materials according to the code book. Inconsistencies between the two researchers and new codes formed were discussed by the members of the research group.

The codes in the predetermined categories were compared based on their differences and similarities and sorted into subcategories, while the codes under the ‘other views’ label were sorted into subcategories and categories.

Patient and public involvement

Patients and the public were not involved in the design, recruitment or conducting of this study. The study results will be disseminated to all participants by email or WeChat.

Written informed consent was obtained from all participants. Participants could withdraw from the study at any time without reason and would not receive unfair treatment as a result. In the data collection stage, identifiable information of the participants was hidden and replaced with a number. Only the involved researchers had access to the interview data, which were kept by QW and sealed after the analysis was completed.

Result

Characteristic of PEMs

We collected 23 PEMs that met the included criteria from 13 hospitals, of which three materials were excluded for the following reasons: printed PowerPoint documents (n=1) and excessive information about medicine advertisements (n=2). Twenty PEMs were ultimately included in this study. Thirteen (65%) PEMs were created by hospitals, and seven (35%) were produced by pharmaceutical companies. Only 1 out of 20 materials provided the source of information (expert statement). The PEMs were published between 2008 and 2017.

Text factors assessment

The average number of difficult words per hundred words for materials was 36.50±3.07 (range 32.83–42.82), and approximately two-thirds of difficult words were superclass words (ie, in the superclass grade of ). The average number of sentences per hundred words of the PEMs was 3.43±0.69 (range 1.78–4.84), indicating that all PEMs were intermediate and difficult materials. The Chinese language difficulty coefficient was 67.09±8.03 (range 58.65–89.49), indicating that all PEMs were difficult materials. The difficulty coefficient of the PEMs from hospitals (66.94±9.37) was slightly lower than that of PEMs from pharmaceutical companies (67.37±5.35) (table 2).

Table 2

Text factors assessment results of PEMs used for patients with SLE

Publication source	C and D words*	Superclass words†	ANDW‡	ASL§	ANS¶	CLDC**
Self-designed (n=13)	11.60±1.80	25.47±2.84	37.07±3.22	29.87±9.44	3.57±0.80	66.94±9.37
Commercial company (n=7)	10.81±1.84	24.63±1.53	35.44±2.67	31.93±3.13	3.16±0.31	67.37±5.35
Overall (n=20)	11.32±1.80	25.18±2.45	36.50±3.07	30.59±28.64	3.43±0.69	67.09±8.03

*Number of words in third and fourth grades of per hundred words.

†Number of words in the superclass grade of per hundred words.

‡ANDW, average number of difficult words per hundred words.

§ASL, average sentence length.

¶ANS, average number of sentences per hundred words: >10 for simple materials, 6–10 for primary materials, <6 for intermediate and difficult materials.

**CLDC, Chinese language difficulty coefficient: 20–30 for intermediate materials, >30 for difficult materials.

PEM, printed education material; SLE, systemic lupus erythematosu.

Non-text factors assessment

The inter-rater reliability (IRR) for the SAM-C overall scale was 0.75, and the range was 0.69 to 0.93 for each domain. This indicated variation among the domains from substantial to almost perfect agreement between the two investigators. The mean overall SAM score for the 20 PEMs was 45.62±9.51 (range 30.56–66.67), which was deemed adequate. Among the 20 materials, eight (40%) PEMs were rated not suitable, twelve (60%) were adequate, and none were superior (table 3). With regard to the domains captured by the items, only the graphic domain had a mean rating (23.58±18.07) in the ‘not suitable’ range. The other five domains had mean ratings within the ‘adequate’ range, of which the cultural appropriateness domain scored highest (63.75±33.91).

Table 3

Evaluation results of non-text factors using the SAM-C instrument

Domains	Items	Distribution of rated points*, n (%)				Score of a domain (x− ±s)	Inter-rater reliability (kappa)†
		0	1	2	N/A‡
Content		5 (25)	15 (75)	0 (0)	0 (0)	53.13±10.63	0.69
	Purpose is evident	1 (5)	0 (0)	19 (95)	0 (0)		1
	Content about behaviours	6 (30)	13 (65)	1 (5)	0 (0)		0.82
	Scope is limited	1 (5)	6 (30)	13 (65)	0 (0)		0.90
	Summary or review included	20 (100)	0 (0)	0 (0)	0 (0)		1
Literacy demand		9 (45)	10 (50)	1 (5)	0 (0)	42.5±16.42	0.92
	Writing style	2 (10)	14 (70)	4 (20)	0 (0)		0.67
	Vocabulary	6 (30)	3 (15)	11 (55)	0 (0)		0.94
	Context is given first	18 (90)	2 (10)	0 (0)	0 (0)		0.77
	Advance organisers	3 (15)	15 (75)	2 (10)	0 (0)		1
Graphics	13 (65)	4 (20)	0 (0)	3 (15)	23.58±18.07	0.86
	Cover graphic shows purpose	10 (50)	1 (5)	0 (0)		9 (45)	1
	Type of graphic	8 (40)	8 (40)	1 (5)		3 (15)	0.95
	Relevance of illustrations	9 (45)	7 (35)	1 (5)		3 (15)	0.74
	Lists and tables explained	1 (5)	1 (5)	2 (10)		16 (80)	0.80
	Captions used for graphics	7 (35)	4 (20)	0 (0)		9 (45)	0.95
Layout and typography		6 (30)	14 (70)	0 (0)	0 (0)	43.34±16.58	0.78
	Layout factors	2 (10)	13 (65)	5 (25)	0 (0)		0.82
	Typography	3 (15)	15 (75)	2 (10)	0 (0)		0.90
	Subheadings used	11 (55)	9 (45)	0 (0)	0 (0)		1
Learning stimulation and motivation		3 (15)	15 (75)	2 (10)	0 (0)	54.17±20.14	0.90
	Interaction used	9 (45)	7 (35)	4 (20)	0 (0)		1
	Behaviours are modelled and specific	8 (40)	8 (40)	4 (20)	0 (0)		0.94
	Motivation	0 (0)	4 (20)	16 (80)	0 (0)		1
Cultural appropriateness		3 (15)	7 (35)	10 (50)	0 (0)	63.75±33.91	0.93
	Match in logic, language, experience	3 (15)	6 (30)	11 (55)	0 (0)		1
	Cultural images and examples	0 (0)	9 (45)	2 (10)	9 (45)		0.95
Overall		8 (40)	12 (60)	0 (0)	0 (0)	45.62±9.51	0.78

*Superior (2 points), adequate (1 point), not suitable (0 points).

†Agreement was deemed poor (0), slight (0.01–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80) or almost perfect (0.81–1.0).40

‡N/A=not applicable; items that do not apply to the materials.

SAM-C, Suitability Assessment of Materials.

Only one material was rated ‘superior’ in ‘presenting content in a tangible, behaviour-related context’. That is, the content of most of the materials mainly included facts about SLE, not guides for patient behaviour or decision making. Additionally, none of the materials included a summary or review at the end and were, therefore, rated ‘not suitable’. Thirty percent (n=6) of the materials used medical terms extensively and did not explain the terms using common words. Ninety per cent (n=18) of the materials did not provide context before new information to help patients learn new facts more quickly. More than half (n=11, 55%) of the materials included too much information under subheadings, and only two materials (10%) used visual cueing devices (shading, boxes, arrows) to direct patients’ attention to key content. Forty-five per cent (n=9) of the materials were rated not suitable with respect to interaction with the patient. Only four PEMs (20%) included actionable behaviour-related information.

In the graphical domain, 11 out of 20 materials (55%) contained images, among which only one material’s images met the ‘simple, clear, and familiar to patients’ criterion, and seven PEMs did not use captions to explain the images. Only one material (5%) presented key points in illustrations and allowed patients to grasp the key ideas from illustrations alone. Four materials (20%) contained tables, but none of them provided examples or explanations to help patients use the tables.

Patients’ view of the readability of PEMs

PEM selection for the interview

Two pamphlets with the lowest and highest readability scores were selected and distributed to the SLE patients before participating in the interviews. The pamphlet with the lowest score was the Manual for Patients with Rheumatism, which was rated not suitable in the content, literacy demand, graphics and layout domains. The pamphlet with the highest score was The Handbook for Patients with Lupus, which was rated adequate to superior in all domains. In addition, the handbook included a diary table to help patients record their symptoms before follow-up, a list of medication questions to give patients tips when they communicate with doctors, and patient stories to share treatment experiences.

Participant characteristics

Eighteen patients participated in the interview. All the patients were female. The mean age was 37.29 years (range 20–60 years), and the median disease duration was 6.05 years (range 0.64–21.64 years). Two patients were Hui people, while others were Han people. Seven patients had a junior high school education or below, four had a high school education and seven had a bachelor’s degree or above. The mean score of health literacy was 90.72±17.26. Eight patients had high health literacy, and the others had low health literacy (see online supplemental file 1).

Perception of SLE PEMs

The duration of the interviews varied from 12 to 44 min. As shown in table 4, eight categories were identified in the interviews: information source, content, actionability, plain language, pictures, tables, numbers and layout. In the information source category, the participants’ views on the information channels were presented. The participants’ perceptions of SLE PEMs were presented in the remaining seven categories, in which some subcategories were consistent with the items in the SAM-C or the evaluation results of text factors, while others were the patients’ unique perspectives on readability.

Table 4

Categories and subcategories with example quotations

Categories and subcategories	Quotations‡
Information source	‘When the doctor told me I got sick, I looked for information about SLE on the Internet.’ (Age: 27)
	‘I will trust the information on the Internet if it matches the messages from doctors. When I open a website with many advertisements, I will skip it.’ (Age: 39)
	‘I believe the messages from Baidupedia, because this website indicates which expert wrote this information, and I think it is authoritative.’ (Age: 39)
	‘I expect the hospital can offer the printed materials because they are more comprehensive and reliable compared with the Internet.’ (Age: 37)
Content
Summary* (Summary or review included)	‘There is so much in each chapter that I don’t know what to look at. It is better to have a summary of each chapter. If I am busy, I can only read the summaries to get the key points.’ (Age: 44)
Contradictory information†	‘I think there are some contradictions between the two materials. The first material says we can’t eat celery, and the second one says we can eat less in the evening. It makes me feel confused. You should confirm the authenticity of these messages and tell us the information sources.’ (Age: 51)
Content topics†	‘I want each part to be described in detail. Some basic medical knowledge, such as the cause and classification of disease, also needs to be explained because it can help me understand my disease, and I need to understand my health comprehensively.’ (Age: 30, high health literacy)
	‘I’m concerned that the solutions for current symptoms and medical knowledge can be briefly summarised.’(Age: 54, low health literacy)
Actionability
Behavioural information* (Behaviours are modelled and specific)	‘The plan of action is so sketchy that I do not know how to carry it out. You should tell me more detailed steps on how to manage myself. For example, according to the different stages of SLE, what kind of exercise is suitable for me and how long?’ (Age: 50)
Case introduction†	‘I like this form of case description. It was interesting and close to life. You can tell a story of a SLE patient, such as her disease process, treatment, and daily management.’(Age: 37)
Symptom recording software†	‘Paper forms record information, which is troublesome and easy to lose. I suggest creating a symptom recording app instead of the paper forms. It is good for preserving data and protecting privacy.’ (Age: 20)
List of questions†	‘I don’t think it’s necessary to list these questions. I know what I should ask when I visit the doctors.’ (Age: 37, high health literacy)
	‘These questions give me some hints about what information is important. However, I don’t think I can use them during the follow-up visit. Because the doctors are very busy, they don’t have time to answer so many questions.’ (Age: 51, low health literacy)
Plain language
Medical terms*	‘I can’t understand these medical terms and English abbreviations. You need to explain them or replace them with common words.’ (Age: 30)
Complicated sentences*	‘Several sentences here all have the same meaning, which is too verbose. I suggest to combine them into a sentence with terse language.’ (Age: 44)
Pictures
Functional pictures†	‘The written description of clinical manifestations is too abstract. Add some real photos on the side, and I can compare my symptoms with these photos. There is no need to add pictures in the daily management because these topics are easy to understand, and the purpose of pictures is to visualise text.’ (Age: 39, high health literacy)
	‘You need to add some pictures to illustrate how to exercise and some text description. I don’t have much patience to read so many words, but pictorial information can increase reading interest.’ (Age: 57, low health literacy)
Decorative pictures†	‘You can make the layout look better by adding some pictures of flowers and plants, and I will feel pleasure during reading.’ (Age: 32)
Tables
Explanations and directions* (Lists and tables explained)	‘I can’t use this table (SLEDAI-2000) independently because I don’t know what it is and how it works. If you want me to use it, you must write the steps clearly. What’s the first step and the next step? ’ (Age: 51)
Numbers
Numerical information†	‘I like these specific numbers, which make me feel convinced.’ (Age: 32)
Mathematical symbols†	‘I can understand common mathematical symbols.’ (Age: 27, high health literacy)
	‘I don’t understand what this symbol (%) means. You should use words or pictures to explain it.’ (Age: 30, low health literacy)
Layout
Marking symbol* (Layout factors)Division* (Motivation)	‘I can’t find the information I need because there is no marker to demonstrate what is important. I think these markers are necessary to help me remember the knowledge.’ (Age: 27)
Word-spacing* (Layout factors)Font size* (Typography)	‘The layout is too compact, and I need glasses to read clearly. It is better to divide the long texts into parts, increase the word spacing and line spacing, and use a large font size.’ (Age: 60)

*Subcategory is consistent with the evaluation result of text factor or the item in the SAM-C presented in the parentheses below.

†Subcategory is inconsistent or complementary with the item in the SAM-C.

‡Only the quotations, which belonged to the subcategories that participants with different health literacy had different perceptions, were marked by high or low health literacy.

SAM-C, Suitability Assessment of Materials.

Information source

The information sources for SLE patients included oral instructions from paramedics, PEMs and the Internet. A booklet on SLE patient health education was placed in each participant’s ward; the booklet was designed by the medical staff in this department. However, most respondents said that the content of the material was complicated and difficult to understand, so it was rarely used. For the network information, patients judged the credibility of information by their experiences (eg, believing the online information if it was an authoritative network platform to spread medical popular science identified by the National Health and Family Planning Commission) and did not believe information on websites for advertising purposes. Because most of the information from the Internet is fragmented and has low credibility, patients with SLE xpressed the need for PEMs developed by professional medical institutions that provided comprehensive medical knowledge.

Content

Patients reported that they found contradictory information between the two materials that we provided during the interview. This type of information is confusing and can reduce patients’ trust in PEMs. The patients suggested that the information should be reviewed by authorities and that the information source should be provided. For the content topics, all participants expressed the need to introduce medical facts before behavioural information, which could help them to understand the self-management recommendations. Patients with high health literacy wanted all types of information to be as rich as possible, while patients with low health literacy suggested that medical facts should be summarised simply and behavioural information should be described in detail.

Actionability

After looking at the PEM with the highest score, patients thought that three things could help them complete behaviours and improve the positivity of action: case introduction (describe the treatment process of some patients), symptom recording software (use software to record symptoms and laboratory results) and list of questions (list the questions that patients can ask when communicating with doctors).

Many participants reported that reading a personal story of a patient ith SLE (including the patient’s experience of illness, treatment process and self-management suggestions) was an effective strategy that could help them remember the points of knowledge in the story and improve their confidence in treatment. Advice on self-management in the stories served as a prompt to act. With regard to the symptom record form in the material, most patients believed that it was useful to record their health condition regularly, which could improve physicians’ knowledge of the patient’s physical condition. However, they suggested using software that made it convenient to record and save the data instead of paper forms. The software should remind patients to record on time and show the variation trend of each laboratory indicator by charts.

Patients with different health literacy had diverse perceptions of whether to add the list of questions in materials. Patients with high health literacy expressed that they knew what to ask during the communication, so there was no need to write these questions. In contrast, patients with low health literacy thought the questions could serve as a reminder to help them obtain more information from physicians. The patients also mentioned that it might be hard to implement in practice because the physicians were too busy to answer these questions.

Pictures

The participants stated that the PEMs should include two types of pictures: functional pictures (used to illustrate information) and decorative pictures (used to make the layout more appealing and reduce fatigue from reading).

Patients with high health literacy suggested that functional pictures should be placed near medical facts that were difficult to understand (eg, clinical features, classification of disease), and patients could compare these pictures of symptoms with their own performance. They also thought that there was no need to add functional pictures next to self-management information because the texts were easy to understand. However, patients with low health literacy wanted to add functional pictures near all vital information and preferred to obtain useful knowledge from pictures rather than words.

Numbers

Many participants thought the expression of numerical information was intuitive and convincing and wanted to add this expression to specific content, such as disease prevalence and amount of exercise. Unlike patients with high health literacy, most patients with low health literacy indicated that they could not understand the mathematical symbols (eg, percent sign, %) and suggested replacing them with words or diagrams.

Discussion

The study assessed the text and non-text factors of the readability of SLE PEMs and explored the views on the readability of materials among patients with different levels of health literacy. Overall, we found that all PEMs were difficult materials based on the Chinese language difficulty coefficient, and none of the PEMs were rated superior by SAM-C. Patients with different levels of health literacy had differing opinions about the detail of basic information, the necessity of question list, the location of functional pictures and the application of mathematical symbols. They also provided many suggestions on the content arrangement, layout design, picture design and reading interaction of the materials.

Compared with the evaluation results of online health information,47 48 PEMs for patients ith SLE have a higher Chinese language difficulty coefficient, which indicates that these materials are more difficult to understand. According to the evaluation results of text factors and patients’ suggestions, there are some areas that need to be improved. First, if more than one-third of 100 words are difficult, the reader’s cognitive burden will be increased.49 We must change complicated terminology to everyday language that is easier to understand. To standardise this process, the national institutes of health can develop guidelines to list frequently used terms in health materials and their common, everyday alternatives in plain language sentences, similar to the document titled ‘Everyday words for public health communication’ developed by the US Centers for Disease Control and Prevention.50 Furthermore, sentences need to be divided into multiple sentences by using punctuation marks (eg, “。”, “; ”, “！”), and excessive embellishment of words should be avoided.48

It is notable that some of the patients’ views are consistent with the assessment results of SAM-C, and we need to pay much more attention to these areas. First, the SLE materials do not describe the behavioural information in detail. It is difficult for patients to implement these actions. For example, the materials only tell patients, ‘If you are in stable condition, you can try to do some exercises’. Patients may wonder which indexes can be used to measure the stability of their condition, what exercises they should perform and for how long. In addition, the materials can provide specific tools to help readers take action. The participants confirmed the effectiveness of three tools: case introduction, symptom recording software and list of questions. In a study evaluating the readability of materials about paediatric human papillomavirus vaccination, it was also noted that case introductions (personal stories) could make the situation feel real and help with decision making.51 While most SLE PEMs advise the reader to ‘ask your doctor’, 89.4% of Chinese patients with SLE lacked communication and interaction abilities,5 which suggests that patients might not know what questions to ask their doctors or the doctors do not have time to communicate with patients. Therefore, the list of questions can serve as a reminder during communication. The poor actionability of these materials may be related to the developers because detailing all behaviours into an actionable message is difficult and time consuming, and it is easy to overestimate how easy something is for someone else to do.52 Second, in the assessment of SAM-C, the graphic category received the lowest score, and all participants emphasised the role of pictures in improving their comprehension and memory of texts. Educational theory suggests that presenting congruent information in multiple formats can help to increase comprehension.25 Some studies suggest that pictures can enhance patients’ understanding of the effect of medication treatment53 and self-management recommendations.54 In our study, we found that patients preferred to use pictures showing clinical manifestations to judge whether they had the same symptoms, and patients, especially those with low health literacy, preferred to obtain important knowledge through cartoon stories. In addition, there were other problems in the materials, including the lack of a summary of key information, the lack of explanations of tables, the lack of marker symbols for key points, and crowded layouts. PEM developers can modify the above problems according to the evaluation criteria for the corresponding items in SAM-C.

There are some inconsistencies between the assessment results of the SAM-C and the perceptions of patients with SLE on readability. First, the SAM-C instrument suggested that most of the information in the materials should focus on desirable behaviours rather than medical facts. However, all the participants felt that basic knowledge of the disease and behavioural information was necessary. Patients with different levels of health literacy only disagreed on how detailed the basic knowledge should be. This may be explained by the limited capacity model of mediated message processing.55 Information processing can be divided into three stages: message encoding, storage and retrieval. Each stage requires cognitive capacity to be achieved, and people’s cognitive capacity is limited. Because patients with low health literacy have insufficient skills to read and interpret health messages,12 they need to spend more cognitive capacity to encode information (ie, read information and derive meaning) when they process complex medical knowledge. As a result, the cognitive capacity used for message storage and retrieval is insufficient, which means that these patients have difficulty remembering and applying complex information. Meppelink et al also suggested that health information provided to patients with low health literacy should be reduced in complexity.56 The materials should introduce the clinical manifestations and inducing factors of SLE before proposing solutions for corresponding symptoms, which would guarantee a logical sequence that is easier for patients to understand and would help patients remember behavioural information and put it into practice. Second, if the PEM has decorative pictures, it will be rated ‘not suitable’ in the item ‘relevance of illustrations’ from SAM-C. It is recommended to include only pictures that convey information (functional pictures) in the materials because decorative images will distract readers and should not be used.57 However, most of the patients wanted some decorative images to be added to the paragraphs, which can improve the aesthetics of the layout to make it more appealing and reduce fatigue from reading.

The patients also raised questions that were not addressed in the quantitative assessment. The patients expected to obtain comprehensive and accurate PEMs written by authoritative health organisations because the SLE information on the website was fragmented and unreliable. In randomised controlled trials, PEMs have been demonstrated to enhance patients’ knowledge and understanding of health conditions.58 However, when the participants read the two PEMs we provided, they found contradictions between them. For example, according to one material, the precondition for pregnancy is that the SLE patient’s condition is inactive and stable for at least 6 months, while another material extended this time to 12 months. To solve this problem, material developers should refer to authoritative guidelines (eg, Recommendations for perinatal management of SLE in China)59 when compiling information and should indicate the source of information. In this study, only one material indicated that the information came from expert statements. Furthermore, abstract words reduce the objectivity and rigour of information,48 and specific numerical information should be used when describing degrees (eg, moderate exercise vs exercising no more than 30 min). Because SLE patients with low health literacy have difficulty understanding mathematical symbols (eg, %), text descriptions could be used instead of these symbols (eg, 7% of people vs 7 out of 100 people). In conclusion, the results of the quantitative assessment cannot completely reflect SLE patients’ views on the readability of PEMs, and patients with different health literacy have different preferences for the readability of PEMs. This suggests that in the process of material development, reader testing should be conducted to ensure that the content and design of the materials are easy for readers to understand, and we should develop different types of PEMs for patients with different levels of health literacy. In addition, evaluation items for the information authority and numbers can be added to the SAM-C according to patients’ perceptions.

There are limitations to this study. Given the resources and time constraints, all the participants in this study were hospitalised women and most of them were younger, and this might lead to selection bias, as the results cannot be generalised to outpatients with milder disease states, male patients, and elderly patients. In the future, we will include these kinds of patients in our study as possible as we can to reduce the bias. Unlike the English readability formulas whose results can be matched with readers’ educational level, the language analysis technology used in this study obtained only a relative result. In this study, we used the semi-structured interviews to collect participants’ perceptions, and it may be a more intuitive strategy to use a ‘think aloud’ format for these interviews,60 which allows for assessment of inferences made by participants while reading the material, thus reflecting the reading experience of participants and theirs understanding of the material.

Conclusion

This study used a mixed-method approach to assess the readability of PEMs for SLE patients. We suggest that all SLE PEMs should focus on the difficulty of words and sentences, actionability and image use. In the development process of PEMs, creators should pay attention to the different views of readability between patients with high and low health literacy, including the number and function of pictures, the proportion of medical facts and the use of mathematical symbols and question lists. Additionally, evaluation items for the information authority and numbers can be added to the SAM-C according to patients’ perceptions.