Readability of Patient Educational Materials in English Versus Arabic.

Little research has been done about patient educational materials (PEMs) written in Arabic. Readability of Arabic PEMs has not previously been assessed because, until recently, there was no validated Arabic readability assessment tool. A total of 207 PEMs in both Arabic and English were collected from the Medline Plus portal. Readability was assessed using Flesch-Kincaid in English and the Open Source Metric for Measuring Arabic Narratives, a new Arabic readability test. We also examined Arabic documents for other linguistic features that could lead to confusion. Mean readability grades were 6.1 and 7.1 for Arabic and English, respectively (p < .01). In 31.6% of PEMs, the English text was higher by two or more grades than the Arabic text, and the Arabic text was higher by two or more grades than the English text in 3.8% of PEMs. No diacritical marks in Arabic were used. An array of esoteric Arabic medical terms, transliterated English terms, Arabicized English terms, and written versions of spoken Arabic dialects were used. [HLRP: Health Literacy Research and Practice. 2019;3(3):e170-e173.].

More than 90 million adults in the United States have inadequate health literacy (U.S. Department of Health and Human Services, 2008). The problem of unreadable materials occurs across a broad range of topics from informed consent and notices of privacy protection to patient's “bill of rights” documents and patient educational materials (PEMs) (Paasche-Orlow et al., 2013; Paasche-Orlow, Jacob, Hochhauser, & Parker, 2009; Paasche-Orlow, Jacob, & Powell, 2005; Paasche-Orlow, Taylor, & Brancati, 2003).

The most common readability tools use rules based on factors such as the average number of syllables per word and the average number of words per sentence (Kher, Johnson, & Griffith, 2017; Meillier & Patel, 2017). Such tools fail to measure elements of syntax other than sentence length, and they only measure semantic burden by association with word length.

More than 420 million people worldwide speak Arabic, and it is the official language of 26 countries (Sawe, 2016). In the U.S., more than 1.1 million people speak Arabic (Brown, 2016). Several features of Arabic make the development of PEMs a challenging task. First, there is a large distinction between spoken and written Arabic. Second, although there is a dominant form of written Arabic, known as Modern Standard Arabic, there are multiple dialects of spoken Arabic. Third, medical Arabic is rarely used. The only country in which medical education is conducted in Arabic is Syria; in all other Arabic-speaking countries, medical education is conducted in English or French. As such, few medical Arabic terms from Modern Standard Arabic are actually in use. Fourth, there is no consistent way to render spoken medical words (which are mostly English) in PEMs. For example, these words can be written as transliterations or can be Arabicized in one manner or another. It is preferred to avoid using Arabicized terms because they are understood only by a minority of the population (Heba Shaji Sa'adeh, 2013).

A new automated tool for Arabic readability was validated in 2016 (El-Haj & Rayson, 2016). As the Medline portal contains PEMs that are translated from English to Arabic, we sought to compare the readability of the English and Arabic versions of these materials. In addition, we examined these PEMs for examples of the type of Arabic linguistic complexity that would not be captured by currently available automated readability analysis.

Methods

In this study, 209 pairs of PEMs in Arabic and English were collected from the Medline Plus portal. These were all the article pairs on the Medline Plus portal as of July 2017 that were designed to present the same information and were available in portable document file (pdf) format. They cover a broad range of medical conditions.

We measured the readability of English text with the Flesch-Kincaid tool and the readability of the Arabic text with the Open Source Metric for Measuring Arabic Narratives (OSMAN) tool (El-Haj & Rayson, 2016). OSMAN was designed as a modified version of English tools such as Flesch-Kincaid and was calibrated and validated to English measures using 73,000 parallel English and Arabic sentences from United Nations' documents. An automated approach with JavaScripting was used to extract text from the pdf documents, standardize formatting, and assess both Arabic and English readability.

The Flesch-Kincaid score represents the reading grade level for English and is reported to the nearest one-tenth of a grade level. The OSMAN score corresponds to the Flesch-Kincaid score; however, OSMAN grade levels are rounded to the nearest whole grade. In addition, the OSMAN score has not been validated below the level of 5th grade, so the tool does not report scores below 5th grade. For comparability, all scores in English below the 5th grade level were represented as 5th grade.

Results

The mean readability score was 6.1 and 7.1 for Arabic and English, respectively (p < .01). In 66 (31.6%) documents, the readability of the English version was higher than the Arabic version by at least two grade levels. In six (3.8%) PEMs, the readability of the Arabic version was higher than the English version by at least two grade levels.

In Figure , various English terms are matched with their different Arabic translations from the Arabic PEMs. These translations followed different approaches in different PEMs. For example, heart attack was translated descriptively into different Arabic words. In addition, colonoscopy was either translated descriptively or by Arabization, depending on the document. No instance of diacritical marks existed in any of these PEMs despite numerous instances where the meaning of a word could be easily confused. For example, a three-letter word in Arabic could either mean a drunken state or diabetes (see Figure ) depending on the diacritical marks.

Discussion

Our results show that, on average, the PEMs in Medline Plus have fairly good readability in English and Arabic, with Arabic documents exhibiting better readability than the English documents; however, variability is evident. As such, readability analyses should continue to be done to identify documents that require special scrutiny. The OSMAN tool is the first readability analysis tool for Arabic. It is freely available on the Internet (https://github.com/drelhaj/OsmanReadability) and can be used to help identify Arabic texts that are likely too hard for people to read.

Unfortunately, readability analyses are inherently limited. Although they identify text that is likely to be problematic, they do not prove that text with good scores will be understandable. In some respect, this phenomenon is similar between English and Arabic, especially with tools that depend exclusively on factors such as the number of syllables per word and length of sentences (Kher et al., 2017). Yet, there are some challenges for the design of effective Arabic PEMs that do not exist in English.

Arabic is structurally and grammatically different from English (El-Haj & Rayson, 2016). In Arabic, diacritical marks are used to inform the reader how to pronounce words. These markings are particularly useful for homo-graphs (i.e., words that are written the same but are pronounced differently and have different meanings), which are common in Arabic. Without diacritical marks readers must use their understanding of the context to understand what words mean (El-Haj & Rayson, 2016). Although diacritical marks are not commonly used in documents for adults, they can be important to clarify confusing words (El-Haj & Rayson, 2016). Diacritical marks are generally not used in documents for adult readers but adding diacritical marks to medical terms that have confusing heteronyms is a clear opportunity to improve PEMs (Figure ).

In addition, there is a significant difference between written and spoken Arabic. Whereas Arabic documents are written in Modern Standard Arabic, which is taught in school in all Arabic-speaking countries, spoken Arabic has multiple different regional dialects (Nair, Satish, Sreedharan, & Ibrahim, 2016). People with limited education and people who speak specific dialects of spoken Arabic may have some difficulty reading Modern Standard Arabic documents (Nair et al., 2016; Boudelaa & Marslen-Wilson, 2013).

Furthermore, Arabic medical terminology can be particularly complex because it is used rarely. Syria is the only country in which medical education is conducted in Arabic. Most Arabic countries teach medicine in English or French (Argeg, 2015). The common use of English and French medical terms among Arabic-speaking medical providers may lead these providers to believe that patients understand these words (Argeg, 2015). In addition, the dearth of medical Arabic in common usage leads to problems translating acronyms and initials (Kasprowicz, 2010). For example, AIDS, an acronym for “acquired immunodeficiency syndrome” in English, is translated literally as a word in Arabic. This means that in Arabic, it is not actually an acronym but a transliterated word (Kasprowicz, 2010). Similarly, the acronym “HIV” is sometimes transliterated literally for the reader to pronounce the sounds of the letters H, I, and V, and sometimes translated into the full Arabic name for the disease (i.e., the disease of acquired human immunodeficiency virus. Translating medical terms can be done in different ways. The term can be rendered into a descriptive phrase to capture the meaning, the term can be Arabicized in some fashion, and the term can be directly transliterated. Accordingly, translations should be vetted with Arabic-speaking populations (Heba Shaji Sa'adeh, 2013). There are many different translations for each medical term, and a unified term that is understood by most of the Arabic-speaking populations should be standardized in PEMs to avoid confusion.

In English, a direct relationship has been established between readability and understandability in some studies. For example, studies of PEMs relating to Zenker's diverticulum and vocal cord paralysis found a strong negative correlation between readability level and understandability (Balakrishnan, Chandy, Bui, & Verma, 2016). However, this has not yet been exhibited for Arabic texts.

The OSMAN tool is a readability test that can serve as a general test to highlight PEMs that may be difficult to read. Future projects could rely on the OSMAN tool to guide the development of a more cohesive understandability test that takes into account the nuances of the Arabic language. Nevertheless, the information gained from the OSMAN tool can be of significant clinical value when applied to current Arabic PEMs.

Conclusion

The availability of the OSMAN tool can help identify Arabic texts that are hard to read. Judicious use of diacritical markings can be easily employed to improve readability, but several more complex challenges need to be overcome to create effective Arabic patient education materials. Authors need to avoid esoteric Arabic medical terms and navigate multiple options when translating text including descriptive translations, transliterations, and transliteration with Arabicization. Further, all of these options are subject to substantial variations in regional dialects. Consequently, usability testing with members of the target population to ensure understandability is strongly recommended.