SUFE and the internet: are healthcare information websites accessible to parents?

BACKGROUND: Slipped upper femoral epiphysis is an adolescent hip disorder requiring rapid surgical intervention. Faced with the prospect of their child undergoing surgery, many fearful parents will turn to the internet to provide information and reassurance. Previous studies have shown the orthopaedic information can be difficult to comprehend.
OBJECTIVE: Assess the readability of healthcare websites regarding slipped upper femoral epiphysis.
METHODS: The term Slipped Upper Femoral Epiphysis was searched in Google, Bing and Yahoo. The websites were evaluated using readability software with seven specialised readability tests including the Flesch-Kincaid Reading Grade Level, the Flesch Reading Ease Score, the Simple Measure of Gobbledygook, Coleman-Liau Index, Automated Readability Index and the Gunning Fog Index. The reading grade level (RGL) was also calculated.A Flesich Read Ease Score (FRES) score above 65 and an RGL of sixth grade and under was considered acceptable. Websites were also assessed for translation services.
RESULTS: 21 unique websites were assessed. The average FRES was 52.5±15.4. Only 3 websites scored 65 or higher (14%). There was a statistically significant difference between website scores based on affiliation, with physician websites having the overall highest mean(p=0.004).The average RGL was 8.67±1.8. Only two websites met the accepted RGL criteria (9.5%) while five websites were marked as extremely difficult to understand (23.8%). Only five websites offered translations (23.8%). There was no statistically significant difference in readability scores between websites which offered translation and those which did not.One-sample t-tests showed that both the RGL (p<0.001; CI 1.83 to 3.49) and the FRES (p<0.001, CI -19.4 to -5.4) scores were significantly different from the accepted standard.
CONCLUSION: Most websites reviewed were above the recommended RGL, making content inaccessible. Improving readability and translation services would enhance the internet's usability as a healthcare tool for parents. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Slipped upper femoral epiphysis (SUFE) is a complex adolescent hip disorder. Caregivers may turn to the internet for information but find healthcare websites offer conflicting advice, resulting in increased anxiety and reduced postoperative rehab compliance.

The study shows that for websites regarding SUFE, the average Flesich Read Ease Score score is 52 and the average reading grade level is 8.67. Both of these figures do not comply with acceptable standards for readability. It also offers suggested solutions to identified weaknesses within these websites

Introduction

Slipped upper femoral epiphysis (SUFE) is an important adolescent hip disorder encountered in orthopaedic surgical practice.1 2 SUFE can be categorised as a Salter-Harris type fracture,3 characterised by the slippage of the proximal femoral metaphysis anteriorly and superiorly relative to the epiphysis.4 The epiphysial plate is usually widened due to the presence of an unusually large hypertrophic zone. This increased area of hypertrophy effects the normal cartilaginous architecture, making it less organised and can result in weak areas, where slippage can occur.5 6

On history and clinical examination, the signs and symptoms of SUFE include hip pain with potential radiation to the knee, shortening of the affected limb, pain on internal rotation and an antalgic, out-toeing gait.2 It has an overall incidence of 10.8 per 100 000, though this may be higher in African-American and Hispanic cohorts.2 7 8

SUFE has become increasingly prevalent in the last decade due to the epidemic of obesity and the associated incline in the number of children who fall into the 95th percentile of their weight category.9 Traditionally, the age of presentation with SUFE was in children between 10 and 16 but over the last decade, the average age of presentation has dropped; this has been theorised to be associated with the faster maturation of children in modern society.8 There may or may not be a history to trauma.

SUFE is often suspected with the presentation of an acutely limping child and this diagnosis is often confirmed with clinical examination and X-ray.10 An accurate diagnosis combined with immediate treatment is critical to prevent complications such as avascular necrosis.11 Despite explanations of the condition and its treatment provided by the orthopaedic surgeon caring for the child, many fearful parents will turn to the internet as a ‘quasi-second opinion’.12 Previous studies have shown that 93% of parents in Canada will have consulted the internet about their child’s symptoms before they have even presented to the emergency department.13 14 Thus, it is of the upmost importance that the information on the internet be as inclusive and accessible to parents as possible to ensure adequate health literacy.

Health literacy is defined as the ‘ability to interpret and understand basic information with such competence as to be able to apply the information to the enhancement of health’.15 Poor standards of health literacy is associated with higher inpatient hospital service utilisation,16 increased postoperative complications,17 reduced postoperative compliance with rehabilitation,18 missed appointments and lower patient satisfaction.19 The key to improving health literacy is to ensure that health consumers have the ability to understand the materials available to them.20

According to the United States Department of Health and Human Services (USDHSS), the average American reads at an eighth grade level or lower.20 21 Previous USDHSS reports state that over 88% of Americans having a level of health literacy that is incompatible with understanding the surgical management of orthopaedic pathologies such as SUFE, resulting in negative postoperative outcomes and high economic costs.21–23 To encourage inclusivity and accessibility, the USDHSS has recommended that health education material be written at a reading grade level (RGL) of no higher than the sixth grade.23 24 However, previous studies have shown that this level is frequently exceeded, resulting in negative health outcomes.20 23–27

Based on our literature search, we have not found any previous study which has sought to determine the accessibility of information about SUFE on the internet. The aim of this study, therefore, was to assess the readability of information of the internet using specific readability scoring systems and to determine the RGL of each website analysed. Considering that modern society is multicultural, we also noted the presence or absence of translation services on each website.

Methods and materials

Patient and public involvement

Patients and the public were not involved in the designing of this study.

Search strategy

As this study was done using websites on the internet and did not involve any patient consent or contact, ethical approval was not needed. In May 2020, the term slipped upper femoral epiphysis was searched for using the three main search engines (Google, Bing and Yahoo!). The first two pages of hits from each search engine were analysed (n=60). The reason for this set limitation was that previous studies have shown that most people do not look beyond the first two pages of website hits and that the majority of people only look at the first page of hits.28 Table 1 shows the amount of hits returned for each search engine.

Table 1

Hits returned for each search engine

Search engine	Hits returned
Google	998 000
Bing	80 600
Yahoo	140 000

Duplicate websites were removed and medical journals, sites requiring logins or composed solely of videos were also excluded. This is in accordance with previous studies which felt that medical journals would be beyond the capacity of the majority of the population.29 Of the initial 60 websites, this left 21 webpages to undergo further evaluation. A breakdown of this methodology is shown in figure 1. The next step in the analysis was to categorise the websites by type; these included academic, physician, non-physician, commercial, media and news, social media and non-specified.29 Academic refers to any website linked to a university while physician described any private website owned by a doctor. Non-physician referred to websites created by other multidisciplinary team members such as physical therapists, radiographers and occupational therapists. Commercial websites denoted websites which were trying to sell products or contained advertising. Social media was added as a category to acknowledge the increased influence of Facebook, Instagram, Tinder and Tik Tok in the modern era. Sites which did not fall into any of the above categories were classed as unspecified. A list of all included sites is included in online supplemental appendix 1.

Figure 1

Flow diagram of methodology for screening websites. Internet search flow diagram, based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.

Application of the readability tool

Once classified, the websites were uploaded into the online readability software (WEB FX),23 30 producing scores for six readability tests and providing an RGL for each website. These tests included the Flesch-Kincaid Reading Grade Level (FKGL), the Flesch Reading Ease Score (FRES), the Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index (CLI), Automated Readability Index (ARI) and the Gunning Fog Index (GFI).23 30 An explanation of the different tests is presented in table 2. All the tests apart from the FRES represent different measures of RGL based on their formulae.23–26 30 The FRES is the only metric where a higher score indicates an increased readability; a score of 65 or greater is considered to be acceptable.23 A breakdown of the FRES scoring system is shown in table 3.

Table 2

Definitions for each of the readability tests used

Name of test	Scoring basis	Description of test	Formula
Flesch-Kincaid Reading Grade Level	Grade level	Developed by the Navy to assess readability of technical documents. Suitable for assessment of a vast range of documents, often used in education. It generates a score determining the equivalent level of US level education needed to understand the piece of text	(11.8 × ASW) + (0.39 × ASL) −15.59
Flesch-Kincaid Reading Ease	Index score	Index score ranging from 0 to 100, with higher scores denoting easier readability. Score over 65 is deemed to be an acceptable readability level which is accessible to most people.	(206.835 − (84.6 × ASW) − (1.015 × ASL)
ARI	Grade level	Designed to gauge the understandability of a text. Unlike other indices, relies on characters per word instead of syllables.	4.71 (characters/words)+0.5 (ASL) – 21.43
Gunning Fog	Grade level	This index estimates the years of formal education required to understand a text on first reading. Developed to assist American businesses improve the readability of their writing. Applicable to numerous disciplines.	0.4 × (ASL+((C**/W) × 100))
Simple Measure Of Gobbledygook	Grade level	Tests for 100% comprehension of a document on first reading, whereas most formulas test for around 50%–75% comprehension. Most accurate when applied to documents ≥30 sentences in length. This test is widely used in checking health messages	1.0430 × √C+3.1291
Coleman-Liau	Grade level	Designed for secondary age (fourth grade to college level) readers. Approximates the level of reading required to comprehend a text. Applicable to numerous sectors. Relies on characters instead of symbols per word	0.0588L-0.296S-15.8

ASL, number words/number sentences; ASW, number syllables divided by number of words; C**, complex words with exceptions including, proper nouns, words made 3 syllables by addition of "ed" or "es", compound words made of simpler words; C, complex words (≥3 syllables); L, average number of letters per 100 words; N, number of syllables; RGL, reading grade level; S, average number of sentences per 100 words; W, number of words.

Table 3

Breakdown of the Flesch Reading Ease Score system

Score	School level	Notes
100.00–90.00	5th grade	Very easy to read. Easily understood by an average 11-year-old student.
90.0–80.0	6th grade	Easy to read. Conversational English for consumers.
80.0–70.0	7th grade	Fairly easy to read.
70.0–60.0	8th and 9th grade	Plain English. Easily understood by 13-year to 15-year-old students.
60.0–50.0	10th to 12th grade	Fairly difficult to read.
50.0–30.0	College	Difficult to read.
30.0–0.0	College graduate	Very difficult to read. Best understood by university graduates.

A score of 65 or greater is concerned to be easily accessible to all reading levels.

The RGL is a cumulative score based on the outcomes of the other readability tests. As previously stated, it is recommended that healthcare related materials be written at no more than a sixth-grade level of education.20 21 To further determine accessibility, each website was assessed for translation services and if offered, how many translations were available.

Statistical analysis

Statistics were performed using SPSS V.26 (SPSS).31 The level of significance was set as a p value less than 0.05. To determine whether sites with translation services predicted higher readability scores, two sample t tests were used when the data were normally distributed and the Mann-Whitney U test was used when it was not. To determine the difference between categories, analysis of variance (ANOVA) testing was performed and if this achieved significance, post-hoc statistics were undertaken. RGL was compared with the sixth grade standard using a one-sample t-test. A score of 65 or higher was determined to be acceptable for the FRES test.

Results

A total of 21 unique websites were assessed during this study. Table 4 shows the categorical breakdown of the websites analysed. The majority of the websites reviewed were academic (n=10; 47.61%). Only five websites (23.8%) offered translational services and of these five websites, only two websites offered more than one additional language. Average readability scores by FRES, FKGL, SMOG, CLI, ARI, GFI and RGL for all websites were analysed and are shown in table 5.

Table 4

Breakdown of the websites included in the final analysis by type

Website type	Total websites (N)
Academic	10
Physician	3
Non-physician	2
Commercial	3
Non-profit	3
Unspecified	0
Total	21

Table 5

Descriptive statistics for each of the performed readability tests

Statistics
	FRES	FKGL	GFI	SMOG	ARI	CLI	RGL
N
Valid	21	21	21	21	21	21	21
Missing	0	0	0	0	0	0	0
Mean	52.5533	7.8729	8.6095	7.0257	6.3824	13.9690	8.6667
Median	56.2000	7.2000	8.0000	6.7000	5.4000	12.8000	8.0000
Mode	56.20	6.80	7.50*	6.20	4.50*	12.20*	8.00
SD	15.42517	2.03693	2.48855	1.84024	2.82765	3.01623	1.82574
Skewness	−0.960	0.935	−0.371	2.027	2.039	1.490	0.440
SE of skewness	0.501	0.501	0.501	0.501	0.501	0.501	0.501

*Multiple modes exist. The smallest value is shown.

ARI, Automated Readability Index; CLI, Coleman-Liau Index; FKGL, Flesch-Kincaid Reading Grade Level; FRES, Flesch Reading Ease Score; GFI, Gunning Fog Index; SMOG, Simple Measure of Gobbledygook.

The average FRES score was 52.5±15.4. The range was 15.1–75. An average FRES of 52.5 puts the data readability at about the 12th grade level and classifies it as ‘fairly difficult to read’. One-sample t-testing against the recommended score of 65 showed a significant difference (p<0.001, CI −19.4 to −5.4).

Only two websites achieved tallies above the recommended score of 65 (9.5%). Seven websites were classified at college level while 2 websites had scores less than 30 (33.3%), meaning that they were so difficult that a college graduate level education would be needed to read and understand them (9.5%). There was no difference in means based on whether the websites offered translations or did not (p=0.364).

A one-way ANOVA was conducted, showing a difference in FRES scores between groups (p=0.004). Post-hoc testing showed significant differences in scores between academic and commercial categories (p=0.017; CI −50.32 to −3.7) and between the physician and commercial categories (p=0.005; CI −67.37 to −9.5).

The CLI mean was 13.96±3. One-way ANOVA testing showed a statistically significant difference between groups (p=0.031). No significant difference between groups was noted on ANOVA testing for any of the other reading tests; the results are shown in table 6.

Table 6

ANOVA testing showing the differences between group scores for readability

ANOVA
		Sum of squares	df	Mean square	F	Sig.
FRES	Between groups	2861.769	4	715.442	6.034	0.004
	Within groups	1896.946	16	118.559
	Total	4758.715	20
FKGL	Between groups	31.314	4	7.828	2.424	0.091
	Within groups	51.668	16	3.229
	Total	82.981	20
SMOG	Between groups	5.432	4	1.358	0.349	0.841
	Within groups	62.297	16	3.894
	Total	67.730	20
GFI	Between groups	21.189	4	5.297	0.826	0.528
	Within groups	102.669	16	6.417
	Total	123.858	20
ARI	Between groups	13.831	4	3.458	0.379	0.821
	Within groups	146.081	16	9.130
	Total	159.912	20
CLI	Between groups	84.723	4	21.181	3.485	0.031
	Within groups	97.230	16	6.077
	Total	181.952	20
RGL	Between groups	13.233	4	3.308	0.991	0.441
	Within groups	53.433	16	3.340
	Total	66.667	20

ARI, Automated Readability Index; CLI, Coleman-Liau Index; FKGL, Flesch-Kincaid Reading Grade Level; FRES, The Flesch Reading Ease Score; GFI, Gunning Fog Index; SMOG, Simple Measure of Gobbledygook.

The RGL mean was 8.67±1.8; the range was 5–13. As shown in table 6, there was no difference between groups based on the ANOVA (p=0.441). There was also no difference between RGL scores based on whether translation services were offered or not (p=0.374). Only two websites met the RGL criteria of sixth grade or less (9.5%). When the mean was compared with the sixth grade standard using the one-sample t-test, it was found to have a significant difference (p<0.001; CI 1.83 to 3.49).

Discussion

The goal of healthcare websites is to ensure they provide reliable, timely information to health consumers. However, this high-quality information must be delivered at a level which is considered easily comprehensible by patients and caregivers.32 Considering that 93% of parents will have consulted the internet for the reason for their child’s symptoms before presenting to the emergency department, accessible information in the paediatric acute care setting is of the upmost importance.13 14

This study has found that the readability of health-related websites on the internet exceeds the comprehension level of the intended audience in the majority of cases with negative impact of the parent’s understanding and expectations of their child’s diagnosis and their ability to consent to treatment. Unfortunately, these findings are in keeping with the trends that have been observed across several other studies.23–28 32 The repercussions of these trends cannot be ignored when we consider the consequences—a lack of comprehension of the effected child’s condition will affect compliance with postoperative care, leading to missed appointments, increased complications and reduced satisfaction with overall treatment outcomes.17 18 It also potentiates the risk of parental cyberchondria developing, where a lack of credible information on the internet may cause a patient or caregiver undue anxiety or stress.33

When considering the ramifications of poor health literacy, it is thus disquieting that our study found that the RGL of the analysed SUFE sites was 8.67, an average which is well above the level recommended by the Amercian Medical Association (AMA), National Institute of Health (NIH) and USDHHS.20 21 34 35 Furthermore, 90.5% of the available material scored above the recommended sixth-grade reading level with roughly 28.5% of the materials evaluated were scored at readability levels advised for university textbooks. This observation was noted across the seven readability test metrics used, supporting a worrisome trend that some parents may be unable to comprehend the information regarding SUFE and its treatment.

Another disturbing trend noted during this research is the lack of translational services available for the material provided. Society has become increasingly multicultural and English is not always the first language of patients or their families; being unable to supply translated materials will result in increased inaccessibility. Furthermore, when we consider that children of Hispanic or African descent have a higher incidence of SUFE than the general paediatric population, it further highlights the need for reliable, accessible information which can be provided in any language.2 7 8

In a bid to combat the issues associated with health literacy, the Agency for Healthcare Research and Quality (AHRQ)36 has previously created several solutions which could be applied to the websites evaluated as part of this study. The AHRQ have advised that when preparing patient education materials, the physician or author should take an approach of assuming all patients and caregivers have difficulty understanding health information and should be communicated with in a manner that anyone can understand.36 Suggestions for the application of this method to education materials include the use of diagrams or videos to communicate meaning and writing simple words in a conversational style while avoiding jargon.37–40 Orthopaedic specialists working in paediatrics should be aware of the lack of high-quality healthcare websites regarding SUFE on the internet and instead, take responsibility for the education of the patients and their families based on these guidelines; this may mean extending the consultation period with the parents during busy clinic or on-call sessions to ensure they have a complete understanding of what SUFE entails as a conditions and its treatment. They should also ensure that parents and caregivers are given further opportunities to ask questions and clarify the information as needed.37–40 Paediatric orthopaedic specialists may wish to develop their own patient education materials to fulfil this niche but should ensure that all materials are assessed for readability using the widely available software before being issued to the general public. In a bid to ensure complete accessibility, these materials should be translated into a wide variety of languages.

This study is the first to consider the readability of the information parents will have access to on the internet should their child develop SUFE and need to undergo orthopaedic surgical fixation. It used seven different tests to determine the readability and is the first piece of literature we have found in this arc which considers the multiculturalism of modern society by assessing each website for translation services.

However, it is important to acknowledge that there are several limitations to this study. Although several readability formulae were used to enhance accuracy, it must be noted that the tests used have not had their validity previously tested for healthcare literature. Furthermore, each readability formula only considers the written information and does not consider any adjunct visual materials which can enhance comprehension.

Another limitation is that the readability formulae used determine the difficulty of the passage based on the letters per word, the syllables per word or the number of words per sentence. This means that everyday words such as ‘difficulty’ may generate a higher RGL than words with fewer syllables such as ‘physis’ which is a medical term and would be poorly understood by the general public. It must also be noted that the accuracy of the information provided on these websites and how this may affect the readability levels of websites was not assessed as part of the scope of this paper but would be an area of further research. Finally, the search for these websites was conducted over 1 day and only the first two pages of results were included; this may mean that additional websites with higher readability may have been missed.

Conclusions

In conclusion, the information on the internet regarding SUFE has been shown to be inaccessible to the majority of parents with readability scores well above recommended levels. Given the imperative role of health literacy to patient outcomes and the increasing usage of the internet among orthopaedic patients, a substantial amount needs to be done to improve the readability of these websites. Until this improves, physicians should err on the side of warning parents away from the internet.