Readability Comparison of Pro- and Anti-Cancer Screening Online Messages in Japan

Background: Cancer screening rates are lower in Japan than those in western countries. Health professionals publish procancer screening messages on the internet to encourage audiences to undergo cancer screening. However, the information provided is often difficult to read for lay persons. Further, anti-cancer screening activists warn against cancer screening with messages on the Internet. We aimed to assess and compare the readability of pro- and anti-cancer screening online messages in Japan using a measure of readability.
Methods: We conducted web searches at the beginning of September 2016 using two major Japanese search engines (Google.jp and Yahoo!.jp). The included websites were classified as “anti”, “pro”, or “neutral” depending on the claims, and “health professional” or “non-health professional” depending on the writers. Readability was determined using a validated measure of Japanese readability. Statistical analysis was conducted using two-way ANOVA.
Results: In the total 159 websites analyzed, anti-cancer screening online messages were generally easier to read than pro-cancer screening online messages, Messages written by health professionals were more difficult to read than those written by non-health professionals. Claim × writer interaction was not significant.
Conclusion: When health professionals prepare pro-cancer screening materials for publication online, we recommend they check for readability using readability assessment tools and improve text for easy comprehension when necessary. Creative Commons Attribution License

Introduction

Cancer screening rates in Japan are lower than those in western countries. The screening rates in 2013 for breast cancer in women aged 50–69 years and for cervical cancer in women aged 20–69 years were 80.8% and 84.5% in the United States, and 75.9% and 78.1% in the United Kingdom, but 41.0% and 42.1% in Japan, respectively (OECD, 2013). Other cancer screening rates in Japan are similarly low: gastric cancer is 39.6%, lung cancer is 42.3%, and colorectal cancer is 37.9% in men and women aged 40–69 years (National Cancer Center, 2013).

Low health literacy (HL) has been recognized as one barrier to cancer screening, and is associated with low cancer screening rates (Oldach and Katz, 2014). HL is the ability to understand health information and to use that information to make good decisions about one’s health and healthcare (Nielsen-Bohlman et al., 2004). HL is determined based on interactions between an individual’s skills and the societal demands in which the individual lives, including the manner in which health information is communicated in society (Institute of Medicine, 2009). Accordingly, health information needs to be communicated in a manner that allows easy access for those with low HL.

In the field of health literacy, health communication accessibility has generally been discussed in terms of readability (Nielsen-Bohlman et al., 2004). “Readability” is defined as “the determination by systemic formulae of the reading comprehension level a person must have to understand written materials” (Albright et al., 1996). Text that is ‘‘readable’’ makes information more accessible and useful by improving comprehension, learning, and retention (Klare, 2000). Further, easy-to-read text is more liked and trusted (Schwarz, 2004; Alter and Oppenheimer, 2009), and generates a more favorable reader attitude (Claypool et al., 2015) than difficult-to-read text. Therefore, cancer screening information should be easy to read.

The internet is an important media to disseminate cancer related information and education (Shahrokni et al., 2014; Tuna et al., 2014; Mohammadzadeh et al., 2016). Approximately 91% of Japanese are regular users of the Internet (Internet World Stats, 2015), and the internet is the one of main sources of cancer screening information in Japan (Tokyo Metropolitan Government Bureau of Social Welfare and Public Health, 2013). To encourage audiences to obtain cancer screening, Ministry of Health, Labour and Welfare, cancer research centers, municipalities and individual physicians publish pro-cancer screening messages on websites, news sites and blogs. However, studies indicate that health professionals often use jargon that are unfamiliar to lay persons (Byrne and Edeani, 1984; Ley, 1998; Castro et al., 2007), and that health information written by health professionals is often written at readability levels that are too high for the majority of the intended recipients (Rudd et al., 2000; Berland, 2001). Further, regrettably, anti-cancer screening activists, who are physicians or self-proclaimed specialists lacking specialized knowledge, propagate on the internet that cancer screening has little or no efficacy, but has a high risk of side effects by radiation exposure, and warn audiences not to obtain cancer screening (e.g., Kondo, 2015; Funase, 2016; Utsumi, 2016).

Considering that over half of Internet users believe that “almost all” or “most” information on health websites is credible (Rice, 2006), the anti-cancer screening online messages can be a barrier to cancer screening. Additionally considering the importance of readability mentioned earlier, it is desirable that pro-cancer screening online messages are easier to read than anti-cancer screening online messages.

Studies have investigated readability of printed cancer materials (Friedman et al., 2004; Grewal and Alagaratnam, 2013), including in Asia-Pacific region (Montazeri and Sajadian, 2004; Akansel and Aydin, 2011; Kim and Lee, 2014; Okuhara et al., 2015). However, no study has assessed readability of cancer screening online messages. In the present study, we aimed to assess and compare the readability of pro- and anti-cancer screening online messages using a measure of readability. We proposed two hypotheses. Hypothesis 1: anti-cancer screening online messages are easier to read than pro-cancer screening online messages. Hypothesis 2: cancer screening online messages written by health professionals are more difficult to read than cancer screening online messages written by non-health professionals.

Materials and Methods

Material collection

We conducted web searches at the beginning of September 2016 using a search formula in Japanese input into Google.jp and Yahoo!.jp (the Japanese version of the search engine); “cancer screening”; “cancer screening” AND (meaningful OR meaningless); “cancer screening” AND (efficacious OR inefficacious); “cancer screening” AND (obtain OR “not obtain”); “cancer screening” AND (danger OR dangerous); “cancer screening” AND (“don’t obtain” OR “you’d better not obtain”); “cancer screening” AND “you must not obtain”. The terms “danger”, “dangerous”, “don’t obtain”, “you’d better not obtain”, and “you must not obtain” were included in these formulae for gathering anti-cancer screening online messages because only the first three formulae did not gather a sufficient number of anti-messages for examination. Google and Yahoo! were chosen because they are the most popular search engines in Japan, accounting for approximately 66% and 29%, respectively, of all Internet searches in August 2016 (StatCounter Global Stats, 2016).

The top 100 results were reviewed for each search formula. Duplications were excluded. Results about prostate cancer screening and positron emission tomography were excluded because they were not recommended as public health services by the Ministry of Health, Labor and Welfare in Japan. Results about gastric, lung, colorectal, breast, and cervical cancer screening were included for analysis if they did not meet any of the following exclusion criteria: (1) bulletin board system or listserv or newsgroup pages or twitter; (2) pages solely containing brief notices about other website content; (3) video results; (4) non-Japanese websites; (5) inactive links; (6) online messages exclusively explaining about cancer screening (e.g., Wikipedia); (7) presentation slides (e.g., Microsoft PowerPoint); (8) online messages without any claims of anti- or pro-cancer screening (e.g., exclusively about time, place and expenses for obtaining cancer screening). The URLs of the included materials were recorded in a Microsoft Excel 2016 spreadsheet.

Material classification

The included materials were classified as “pro” or “anti” depending on the claims: materials that recommended readers to obtain cancer screenings were classified as “pro”, and materials that objected to readers’ obtaining one or more of gastric, lung, colorectal, breast, and cervical cancer were classified as “anti”. Materials that referred to both claims of anti- and pro-cancer screening but did not state their own assertion were classified as “neutral”.

Additionally, materials were classified as “health professional” if they were written by physicians, nurses, pharmacists, and researchers, or were published by pharmaceutical companies, health-care companies, research centers, and hospitals. Conversely, materials were classified as “non-health professional” if they were written by alternative therapists, journalists, and lay individuals. When the materials were published by non-health professionals but their content were exclusively reprinted from articles written by health professionals, they were classified as “health professional”.

Readability assessment

Before readability assessment, materials were edited by removing any URLs and quotations written in English to ensure accurate measurement of Japanese readability. When the included materials comprised multiple pages and included topics other than cancer screening, only the relevant messages were assessed by using a measure of readability.

A validated measure of Japanese readability called the “Japanese text readability measurement system” (JTRMS) (Lee and Hasebe, 2013) was used in the present study. The JTRMS is the most authentic validated measure of Japanese readability. The JTRMS calculates Japanese readability on the basis of average length of sentences, difficulty level of words, and proportion of grammatical parts of speech and types of characters per sentence (Lee, 2011). Scores range from 0.5 to 6.4. A high score indicates that the text is relatively easy to read; 5.5–6.4, very easy; 4.5–5.4, easy; 3.5–4.4, neutral; 2.5–3.4, a little difficult; 1.5–2.4, difficult; 0.5–1.4, very difficult. Scores were determined by using the JTRMS online tool (Lee and Hasebe, 2013). Score results were recorded in a Microsoft Excel 2016 spreadsheet.

Statistical analysis

Distributions and mean readability scores for “anti”, “pro”, “neutral”, “health professional”, and “non-health professional” materials were calculated. Two-way ANOVA was conducted with readability as the dependent variable, and Claim (i.e., pro or anti) and Writer (i.e., health professional or non-health professional) as the two independent variables. Additionally, readability of “pro” materials were compared with that of “anti” materials on the first 10 search results of all search formulae using the two-sample t test, because online health information seekers usually examine the first 10 search results (Eysenbach and Kohler, 2002). P-values were set at 0.05 for all statistical tests. All statistical analyses were performed using SPSS version 21.0 (Chicago, IL, USA).

Results

Characteristics of materials

Collected materials comprised websites or blogs and three independent Facebook pages, collectively labeled as “websites”. Of the 159 total websites that were evaluated, the number of sentences varied from 11 to 953, and the number of letters varied from 232 to 19,986. The numbers of websites by cancer types were as follows; gastric cancer screening, 3; colorectal cancer screening, 9; gastric and colorectal cancer screening, 2; breast cancer screening, 23; cervical cancer screening, 20; breast and cervical cancer screening, 3; breast and colorectal cancer screening, 1; gastric, lung, colorectal, breast and cervical cancer screening, 98.

Distribution by category

Table 1 shows the distribution of websites by category. Up to 75 websites (47.2%) propagated pro-cancer screening messages, 81 websites (50.9%) propagated anti-cancer screening messages, and three websites (3%) were “neutral”. Of the 75 pro-cancer screening websites, 46 were written by health professionals, and 29 were written by non-health professionals. Of the 81 anti-cancer screening websites, 30 were written by health professionals, and 51 were written by non-health professionals.

Table 1

Distribution of Websites by Category of Claim and Writer

		n (%)
	Pro-cancer screening	Anti-cancer screening	Neutral	Total
Health professional	46 (29.0)	30 (18.8)	2 (1.3)	78 (49.1)
Non-health professional	29 (18.2)	51 (32.1)	1 (0.6)	81 (50.9)
Total	75 (47.2)	81 (50.9)	3 (1.9)	159 (100)

Readability

Table 2 and Figure 1 show mean readability of websites by category; “neutral” was excluded because its distribution was small. The mean readability of all 159 websites was 2.7 (SD=0.52). ANOVA revealed a main effect of Claim, F (1,152)=24.9, p<.001, MSE=0.201, indicating that anti-cancer screening online messages were easier to read than pro-cancer screening online messages (M=2.9 VS M=2.4). ANOVA also revealed a main effect of Writer, F (1,152)=18.1, p<.001, indicating that cancer screening online messages written by health professionals were more difficult to read than those written by non-health professionals (M=2.4 VS M=2.8). Claim × Writer interaction was not significant, F (1,152)=0.229, p=0.633. Table 3 shows mean readability of websites on higher search results. When the websites that were included from the first 10 search results of all search formulae were measured, anti-cancer screening messages were found to be significantly easier to read than pro-cancer screening messages (M=2.91 VS M=2.27, p=0.003).

Table 2

Mean Readability of Websites by Category of Claim and Writer

	Pro-cancer screening		Anti-cancer screening		Total
	M	SD	M	SD	M	SD
Health professional	2.29	0.42	2.69	0.44	2.45	0.47
Non-health professional	2.64	0.54	2.97	0.42	2.85	0.49
Total	2.42	0.5	2.87	0.44	2.66	0.52

* A high score indicates that the text is relatively easy to read.

Figure 1

Mean Readability of Websites by Category of Claim and Writer

Table 3

Mean Readability of Websites with the First 10 Search Results

Pro-cancer screening		Anti-cancer screening
(n=7)		(n=15)
M	SD	M	SD	p
2.3	0.4	2.9	0.4	0.003*

*, p<0.05; *, A high score indicates that the text is relatively easy to read

Discussion

In the present study, we found that, in Japan, anti-cancer screening online messages were easier to read than pro-cancer screening online messages. Additionally, cancer screening online messages written by health professionals were found to be more difficult to read than those written by non-health professionals. Thus, our hypotheses were supported by the study results.

Studies indicate that easy to read text is more accessible and acceptable than difficult to read text (Klare, 2000; Schwarz, 2004; Alter and Oppenheimer, 2009; Claypool et al., 2015). Therefore, ease of readability of anti-cancer screening online messages that was indicated in the present study may contribute to their acceptance by some audiences. The present study also found that, on the first 10 search results, anti-cancer screening online messages were significantly easier to read than pro-cancer screening ones. The ease of reading of anti-cancer screening messages may influence judgement of reader from the beginning of their web searches. Considering that over half of Internet users believe that “almost all” or “most” information on health websites is credible and state that the information they encounter online influences their treatment decisions (Rice, 2006), anti-cancer screening messages on the Internet and their ease of readability might increase the number of people not obtaining cancer screenings, thereby decreasing cancer screening rates.

Our result was consistent with previous studies indicating that health information written by health professionals was often difficult to read for lay persons (Rudd et al., 2000; Berland, 2001), including in Asia-Pacific region (Akansel and Aydin, 2011; Kim and Lee, 2014). Pro-cancer screening messages may tend to be difficult to read partly because technical terms are often written in Chinese characters such as saibousin (cytodiagnosis) and musakuika hikaku taishousiken (randomized controlled trial) in Japan. However, health professionals can improve pro-cancer screening online messages for easy reading using measures of readability. The JTRMS (Lee and Hasebe, 2013) shows the difficulty level for each word that is used in texts by using colored lettering on a screen. Users can improve the texts and make them easier to read by rewriting difficult words into easy ones. Additionally, long sentences are found at a glance because the JTRMS display all sentences individually in parallel. Users can improve the texts by shortening long sentences.

Of pro-cancer screening online messages, the proportion of messages written by health professionals was larger than that of messages written by non-health professionals. When non-health professionals write pro-cancer screening online messages, they may refer to pro-cancer screening websites written by health professionals. Thus, pro-cancer screening online messages written by health professionals are considered to be influential both in quantity and in quality. If pro-cancer screening online messages written by health professionals become easier to read, those written by non-health professionals may also become easier to read and therefore more easily accepted by audiences. Health professionals are recommended to make efforts to write easier-to-read pro-cancer screening online messages.

The present study has limitations. First, the measures of readability used measure only word difficulty and sentence length and complications. They do not assess impact of factors such as font type, font size, color, white space, and illustrations, which may influence processing fluency.

Second, although an extensive number of websites (n=159) were selected for analysis, it was not feasible to examine the universe of sites for reasons of availability, access, and time. This is especially difficult considering that new websites are created each day, while old ones become inaccessible.

Third, only Japanese-language websites were included in the study. To generalize the results of this study to other countries, the study should be replicated for websites written in languages other than Japanese. Finally, future study is recommended to investigate whether readability of cancer screening online messages influences readers’ attitudes, intentions, and behaviors regarding obtaining cancer screening.

Despite these limitations, this is the first study to assess and compare anti- and pro-cancer screening online messages, with significant implications as mentioned previously.

Readability of written information can be easily assessed by using measures of readability. When health professionals prepare cancer screening materials for publication online, we recommend they check message readability using measures of readability, and improve the text for easy reading if necessary. Writing and disseminating easy-to-read pro-cancer screening online messages is one means of fighting against anti-cancer screening activists for health professionals.

Abbreviation

JTRMS: Japanese text readability measurement system

Funding sources

This study was supported by a JSPS KAKENHI Grant (Number 167100000384).

Conflict of interests

None declared