The quality and readability of online molar incisor hypomineralization patient education materials: a systematic analysis.

BACKGROUND: Molar incisor hypomineralization (MIH) is estimated to affect 14% of children worldwide. It is crucial that patients and their families have access to easily comprehensible and reliable MIH-relevant information. This study aims to determine the quality, reliability and readability of online patient education materials about MIH.
METHODS: A systematic search strategy was adopted. Five validated tools were used to assess the content of the 21 websites that satisfied inclusion/exclusion criteria. Data analyses were applied via GraphPad Prism software version 9 (GraphPad Software, San Diego, CA, USA).
RESULTS: Five (23.8%) websites only satisfied the criteria for understandability and two (9.5%) websites satisfied the criteria for actionability using the Patient Education Materials Assessment Tool (PEMAT). No website contained the Health on the Net (HON)Code Seal and the mean (SD) Journal of the American Medical Association number of benchmarks per website was 1.33/4 (1.02). All websites failed to reach recommended minimum readability levels. Higher PEMAT scores were associated with 'easier' readability.
CONCLUSIONS: Online patient education materials related to MIH are lacking in quality and reliability, and are too difficult for most to read easily. The authors of MIH-related online content should consider reference to quality of information tools when developing patient education materials.

Molar incisor hypomineralization

Patient Education Materials Assessment Tool

Score Simple Measure of Gobbledygook

Flesch Reading Ease

Introduction

Molar incisor hypomineralization (MIH) is a development defect of enamel which is estimated to affect 14% of children worldwide. The presentation of clinical features varies widely and patients often suffer a significant deleterious impact on their dental health and quality of life due to the condition. In addition, MIH management options are often not straightforward and can require considered patient, clinician and parental/guardian input and planning for the short‐ and long‐term.

Consequently, information regarding MIH and potential management options can be challenging to effectively communicate in a dental clinic environment. Therefore, patients and their families might seek (or be encouraged by their dental care provider to seek) information from additional sources. A ready and convenient source for further health information is the Internet.

The public is increasingly accessing the Internet for health‐related information. This has become more pronounced since the onset of the COVID‐19 pandemic during which direct patient/clinician interaction has been adversely affected. , The Internet, however, is unregulated. Content that is inaccurate and potentially harmful, can be uploaded without discrimination. Reliable and accurate content, nonetheless, is of little use if the written material is difficult to read.

Health literacy is defined as the social and cognitive skillset required to seek, comprehend and apply information to advance health. Literacy or ease of readability is a crucial component of health literacy. Health literacy is low in most general populations, and in Australia, up to 60% are considered to lack sufficient health literacy to seek, understand and act on health information while 44% are considered to have poor literacy skills. Similar situations are found in the United States where the average literacy equates to the US school grade 8 level and the United Kingdom where the average reading age of adults is equivalent to that expected of a 9‐year‐old student. ,

It is important, therefore, to ensure that written health information on the Internet is easy to read in addition to being reliable. This is likely to result in patients and their families being able to understand and act upon any presented health information. The quality and readability of online information related to many oral conditions, appliances and treatments, has, however, been generally found to be deficient. , , , , ,

There appears to have been little research into the quality of online information in relation to MIH. In view of the prevalence of MIH and the significant health and financial burdens associated with the condition, it is essential that patients and their families obtain and easily read, high‐quality information materials to ensure informed decision‐making and effective management. The aim of the present study, therefore, was to determine the quality, reliability and readability of online information related to MIH targeted at affected patients and their families.

Materials and methods

Ethical approval was not required as the study only evaluated publicly available information.

Data collection

Web searches were carried out through three search engines on 1 July 2021 using the terms ‘molar incisor hypomineralisation’, MIH’ and ‘chalky teeth’. The searches were repeated 2 months later to identify relevant new added online content and note content which might be removed in the interim. The Internet browser search history was deleted prior to each search and the search engine’s geographical settings were also switched off before each search.

The links to the first 100 sites from each search were documented in a Microsoft Excel spreadsheet (Microsoft, Redmond, Seattle, WA, USA). The 900 links were screened to remove duplicated and/or non‐functioning links. Additional exclusion criteria comprised links to academic articles, content only aimed at health professionals and links to video footage and irrelevant content. The inclusion criteria consisted of English‐language web content aimed at patients, their families and the general public. Details regarding the websites’ country of origin and authorship were recorded following the application of the inclusion/exclusion criteria.

Quality analysis

The relevant website content was assessed for quality using three validated instruments. The Patient Education Materials Assessment Tool (PEMAT) was developed in 2014 and is used to determine the ‘understandability’ and ‘actionability’ of the presented information. There are two versions of the tool. The PEMAT‐P (used in the current study) assesses print material, and the PEMAT‐A/V assesses audiovisual information. The tool provides guidance on whether patients can understand and ‘take action’ on the presented health‐related information. The user of the tool (rater) is required to agree or disagree with up to a maximum number of 19 items or statements in order to determine the understandability. The items are related to content, word choice and style, the use of numbers, organization of material, the layout and design and the use of visual aids. The actionability of the material requires an assessment of the material via seven further statements. A score is assigned to each statement based on whether the rater agrees or disagrees with the statement as it pertains to the material being assessed. The scores are then converted into a percentage score for understandability and actionability. The developers of the tool recommend a score of ≥70% for each of the two components to ensure acceptable reader engagement and comprehension of the material.

The Journal of the American Medical Association (JAMA) benchmarks require that details regarding the content’s author(s) and their credentials must be clearly displayed (authorship), and the sources of the information must be clearly visible (attribution). In addition, the timepoint at which the content was written/posted or updated must be clear to the reader (currency) and information regarding content/publication ownership must also be on view (disclosure). The presence of at least three benchmarks is required for acceptable reliability.

The Health On the Net (HON) is a non‐profit organization founded to enable guidance for the health professional and the patient regarding the compliance of the website containing the health information with HON’s eight principles of ethical conduct. Websites awarded HON certification can be identified by the presence of the organization’s seal on the website.

Readability analysis

The readability of the information was determined via computation of the Score Simple Measure of Gobbledygook (SMOG) and the Flesch Reading Ease (FRE) of the written material. Both instruments are validated, reliable and are widely used in assessing the readability of health information. The SMOG score indicates the level of readability expected of an individual in a particular US school grade. The score is derived from the formula:1.043 × √ (number of polysyllables) × (30/sentences) + 3.1291. Several agencies have recommended that the ease of readability should be no higher than US school 6th grade levels. ,

The FRE Score assesses the ease of understanding of the assessed written material via the formula: 206.835 − (1.015 × average sentence length) − (84.6 × average number of syllables per word). Potential scores range from 1 to 100 with higher scores signifying greater ease in reading the material. The scores can be equated to the readability expected of individuals at different levels of education. A score ≥70 has been recommended as an appropriate readability level for health information.

The relevant text within each website to be evaluated was ‘copied and pasted’ to a Microsoft Word document (Microsoft). The text from each website was subsequently formatted to a standardized protocol. This included the removal of hyperlinks, images and bullet points to ensure accurate evaluation. Text up to the calculator’s limit of 3000 words was then entered into an online readability calculator (www.readabilityformulas.com), and SMOG and FRE scores were computed.

Statistical analysis

Statistical analyses, including descriptive statistics, were calculated using GraphPad Prism 9.0 (GraphPad Software, San Diego, CA, USA). Normality testing was carried out via the D'Agostino and Pearson test. The Shapiro–Wilks test was used when the number of variables contained within any assessed dataset was less than eight.

An unpaired t‐test with Welch’s correction was used to determine if there were any differences in PEMAT scores between websites based on their country of origin. In addition, a Pearson correlation coefficient testing was carried out to determine whether there was an association between the PEMAT scores and the presence of JAMA benchmarks and whether there was any association between the PEMAT scores and the SMOG readability scores.

Error study

Intraclass correlation coefficient (ICC) scores were carried out to determine the intra‐rater agreement for each PEMAT item 2 months after the initial analysis.

Results

A total of 21 websites satisfied inclusion and exclusion criteria (Fig. 1). No additional websites for assessment were added or removed as a result of the second search. Most websites were sourced from non‐specialist dental practices and from Australia (Fig. 2). Only one website originated in the United States.

Fig. 1

Flowchart demonstrating website selection.

Fig. 2

‘Author‐type’ and country of origin of included websites (n = 21).

Five websites recorded scores ≥70% for understandability and two websites scored ≥70% for actionability. The overall mean (SD) PEMAT percentage score for understandability was 46.28 (23.32) [95% CI: 35.67–56.9]. The overall mean (SD) PEMAT percentage score for actionability was 31.74 (27.58) [95% CI: 19.19–44.3].

Fig. 3 shows the distribution of websites according to PEMAT percentage scores for understandability and Fig. 4 illustrates the distribution of websites according to PEMAT percentage scores for actionability.

Fig. 3

Distribution of websites according to PEMAT percentage scores for understandability (n = 21) [≥70% indicates acceptability].

Fig. 4

Distribution of websites according to PEMAT percentage scores for actionability (n = 21) [≥70% indicates acceptability].

The PEMAT item related to the provision of a summary to the reader scored least in this study for understandability (Table 1). Table 2 illustrates the mean percentage score per item for actionability. The ICC scores for each PEMAT item were excellent (0.9–1).

Table 1

Mean percentage score per item for understandability

Item	Mean percentage score
The material makes its purposefully evident	62
The material does not contain distracting information	58
The material uses everyday language	19
Medical terms are defined and only used to familiarize readers with the terms	43
The material uses the active voice	19
Numbers used within the material are easy to understand and clear	43
The material does not expect the reader to carry out calculations	90
The material divides the information into short sections	89
Each section has an informative header	76
The material presents information in a logical sequence	67
The material provides a summary	11
The material uses visual cues (for example, larger font) to draw attention to key points	38
The material uses visual aids to improve understandability	38
Visual aids reinforce rather than distract from the content	35
The material's visual aids have clear titles or captions.	21
The material’s illustrations and photographs are clear and uncluttered.	21
Tables are simple and have short and clear headings.	0

Table 2

Mean percentage score per item for actionability

Item	Mean percentage score
The material identifies at least one action for the user to undertake	67
The material addresses the user directly	62
The material breaks down actions into explicit and manageable steps	29
The material provides a tangible tool to aid the reader to take action	5
The material provides instructions for calculations if required	–
The material explains how to use diagrams if diagrams are present	–
The material uses visual aids to improve actionability	14

The mean (SD) JAMA per website was 1.33 (1.02) [95% CI: 0.87–1.796; range: 0–3]. Fig. 5 shows that currency was the most common benchmark displayed on the assessed websites. No website displayed the HONCode seal.

Fig. 5

Number of websites containing each JAMA benchmark (n = 21).

The mean (SD) PEMAT percentage score for understandability of website content originating from the United Kingdom was 55.73 (27.85) [95% CI: 26.5–84.95] and the corresponding mean (SD) scores for Australian websites was 39.28 (21.84) [23.65–54.9]. An unpaired t‐test with Welch’s correction, however, showed that there was no significant difference based on the country of origin (P = 0.25; 95% CI: −13.79 to 46.7). In addition, there was no correlation between the PEMAT understandability scores and the presence of JAMA benchmarks (P = 0.16; 95% CI: −0.6659 to 0.1468) nor between PEMAT actionability scores and the presence of JAMA benchmarks (P = 0.12; 95% CI: −0.33 to 0.53).

The mean (standard deviation [SD]) SMOG score was 9.9 (1.44), (95% CI: 9.2–10.51). The mean (SD) FRE score was 50.45 (9.74), (95% CI, 46.02–54.89).

Correlation between quality and readability

Increased PEMAT understandability scores were associated with lower SMOG reading scores (r = −0.6; 95% CI: −0.82 to −0.23; P < 0.01). Increased PEMAT actionability scores were also associated with lower SMOG reading scores (r = −0.63; 95% CI: −0.84 to −0.29; P < 0.01).

Discussion

The present study appears to be the first to investigate the quality, reliability and readability of information related to MIH on the Internet. The condition’s high prevalence among children worldwide and the increasing use of the Internet for accessing health information highlight the importance of this study. The results indicated that the understandability and actionability of information regarding MIH aimed at patients and their families were lower than recommended values and that the readability was too difficult for many of those to whom the information is targeted.

The relatively small number of websites satisfying inclusion criteria in this study indicates that there is a scarcity of online sources of information regarding MIH. This contrasts with the large number of academic articles which reflects the importance the profession places on the relevance of the condition. A surprising finding, however, was that only one of the assessed websites originated in the United States. This differs with comparable studies in which the United States, the ‘home’ of the largest number of English‐language users of the Internet in the world, was the source of most health information websites. , , , , This might illustrate the view of some commentators that the United States involvement in acceptance and researching the condition has been sub‐optimal.

Several tools have been used to assess online health information. , , The PEMAT is the first, however, to attempt a specific analysis of the ease of public comprehension of the relevant material (‘understandability’) and the ability of patients/families to recognize what is advised based on the presented material (‘actionability’). It, therefore, aims to not only provide information in a comprehensible manner but to outline steps that patients and their families can adopt in managing the condition. Consequently, the tool attempts to ensure that those with low health literacy are not further marginalized by ‘difficult to understand’ information and guidance. Although the assumed increased health literacy of an evaluator in this study might be a source of unintentional bias, the tool has been shown to be easy to use and has shown a high degree of internal consistency and reliability.

The mean PEMAT score of 46.28% for understandability in this study compares with PEMAT scores of 55.9–75% recorded in studies investigating the quality of online patient education materials related to diabetes, laryngectomy, surgical site infection and dental treatment of patients with ischaemic heart disease (IHD). , , , The mean PEMAT score of 31.74% for actionability compared unfavourably with the actionability scores of 39–49% recorded in the studies regarding diabetes, laryngectomy, surgical site infection and dental treatment of patients with IHD. , , , The scores in the present study failed to meet the minimum of 70% recommended for each domain. This might mean that patients (and families) are unlikely to comprehend and describe key points from the information and, additionally, fail to recognize what they can do based on the presented material.

The display of the HONcode seal apprises the reader regarding the reliability of the information. Its presence indicates that the website presenting the information complies with minimum ethical standards related to the presentation of the information and to ensure that readers know the origin and the intention of the material being read. No website displayed the seal in the present study. The presence of the seal has been found to be limited in similar studies investigating a wide range of oral conditions and treatment modalities. , , , The lack of awareness regarding the seal’s existence among website publishers, healthcare professionals and the public might help to explain its absence in the assessed websites in the surveyed websites.

The presence of the JAMA benchmarks also aims to provide reassurance regarding the reliability of the presented information. They help the reader determine that the information has been written by a relevant authority and the information is credibly sourced. In addition, the disclosure of details such as the identity of the website funding source will allow the reader to decide whether there is any potential conflict of interest. The evaluated websites scored poorly with respect to the presence of the JAMA benchmarks in the current investigation. The presence of a mean of 1.33 JAMA benchmarks per website in this study is considerably less than the recommended minimum of three which indicates ‘trustworthiness’. Currency was the most common benchmark on display with 57.1% showing the date of initial upload and/or updating of the information. This is similar to surveys assessing the websites related to medication‐related osteonecrosis of the jaw, dental treatment for patients with IHD and treatment information provided by ‘direct‐to‐consumer’ aligner providers. , , Details regarding currency are important for the reader as it helps determine whether the information is ‘up‐to‐date’ with emerging relevant evidence.

The readability of the evaluated material was determined using the SMOG Index and the FRE score. The application of the two readability instruments increases the reliability of the results by a factor of 20%. The SMOG Index is considered a ‘gold standard’ instrument and is commonly used in similar surveys. The mean SMOG Index readability was higher than the recommended readability of scores (equivalent to the US grade level six) advocated by many public agencies. , In addition, the mean FRE score of 50.4 was considerably less than the minimum score of 70 recommended for materials targeted at the general public. , , This suggests that the presented information is written at a level that is too difficult for many readers to understand, irrespective of the quality of the presented information. Nevertheless, it must be acknowledged that authors of health‐related information face a challenge in conveying meaningful health information without the use of medical terms and jargon that increase reading difficulty.

Interestingly, however, the present study revealed increased PEMAT understandability and actionability scores were associated with lower SMOG reading scores. This is consistent with the finding of Wong et al. who investigated the quality of education materials related to dental treatment for patients with IHD. This suggests that authors can convey high‐quality understandability and actionability information that is relatively easy to read.

The limitations of this investigation must be recognized. The Internet is a dynamic medium, with content continually added and removed. Therefore, it has been suggested that studies of this type might rapidly become ‘out‐of‐date’. However, a second search using the same terms carried out 2 months after the initial search showed no change in websites included for evaluation. This indicates that online content available for patients and their families for some conditions might remain more ‘constant’. Nevertheless, the use of alternative search terms including the American English version of hypomineralisation (‘hypomineralization’) might have yielded different results. In addition, while the JAMA benchmarks and the HONCode Seal provide guidance related to the reliability and the ethical underpinning of a website and its content, they do not assess the scientific accuracy of the presented information. Furthermore, the search strategy aimed to identify websites only in the English language. The quality of MIH‐related online information might differ in other languages.

It is essential that general dental, paediatric dental and orthodontic professional bodies take the lead in developing high‐quality online patient education materials related to MIH that are easy for patients to read, understand and act upon. The incorporation of the tools described in the present study will aid authors in achieving these objectives. More specifically, authors should consider the adoption of unambiguous and clearly labelled visual aids, internal summaries, the use of the active voice, jargon‐free language and short sentences which will enable greater understanding of the presented information. The inclusion of details regarding author credentials, sources of the presented information and relevant financial interests of the website owners will improve the ‘trustworthiness’ of the website. Future research, however, is required to investigate the content for scientific accuracy. It will also necessitate the inclusion of patient/family feedback to ensure that the material is accessible to individuals with varying levels of health literacy.

Conclusions

The findings of this investigation indicate that online patient education materials related to MIH are lacking in quality and reliability and are too difficult for most (patients and their families) to read. The authors of MIH‐related online content should consider reference to the quality of health information tools when developing education materials targeted at patients and their families.