Readability of Orthopaedic Patient-reported Outcome Measures: Is There a Fundamental Failure to Communicate?

BACKGROUND: Patient-reported outcome measures (PROMs) are increasingly used to quantify patients' perceptions of functional ability. The American Medical Association and NIH suggest patient materials be written at or below 6th to 8th grade reading levels, respectively, yet one recent study asserts that few PROMs comply with these recommendations, and suggests that the majority of PROMs are written at too high of a reading level for self-administered patient use. Notably, this study was limited in its use of only one readability algorithm, although there is no commonly accepted, standard readability algorithm for healthcare-related materials. Our study, using multiple readability equations and heeding equal weight to each, hopes to yield a broader, all-encompassing estimate of readability, thereby offering a more accurate assessment of the readability of orthopaedic PROMS. QUESTIONS/PURPOSES: (1) What proportion of orthopaedic-related PROMs and orthopaedic-related portions of the NIH Patient Reported Outcomes Measurement Information System (PROMIS®) are written at or below the 6th and 8th grade levels? (2) Is there a correlation between the number of questions in the PROM and reading level? (3) Using systematic edits based on guidelines from the Centers for Medicare and Medicaid Services, what proportion of PROMs achieved American Medical Association and NIH-recommended reading levels?
METHODS: Eighty-six (86) independent, orthopaedic and general wellness PROMs, drawn from commonly referenced orthopaedic websites and prior studies, were chosen for analysis. Additionally, owing to their increasing use in orthopaedics, four relevant short forms, and 11 adult, physical health question banks from the PROMIS®, were included for analysis. All documents were analyzed for reading grade levels using 19 unique readability algorithms. Descriptive statistics were performed using SPSS Version 22.0.
RESULTS: The majority of the independent PROMs (64 of 86; 74%) were written at or below the 6th grade level, with 81 of 86 (94%) written at or below the 8th grade level. All item banks (11 of 11) and short forms (four of four) of the PROMIS® were written below the 6th grade reading level. The median reading grade level of the 86 independent PROMs was 5.0 (interquartile range [IQR], 4.6-6.1). The PROMIS® question banks had a median reading grade level of 4.1 (IQR, 3.5-4.8); the Adult Short Forms had a median reading grade level of 4.2 (IQR, 4.2-4.3) There was no correlation appreciated between the median reading grade level and the number of questions contained in a PROM (r = -0.081; p = 0.460). For PROMs above NIH-recommended levels, following edits, all (five of five) achieved NIH reading level goals and three (three of five) achieved American Medical Association goals. Editing of these PROMs improved readability by 4.3 median grade level (before, 8.9 [IQR, 8.4-9.1], after 4.6 [IQR, 4.6-6.4], difference of medians, 4.3; p = 0.008).
CONCLUSIONS: Patient literacy has great influence on healthcare outcomes, and readability is an important consideration in all patient-directed written materials. Our study found that more than 70% of PROMs commonly used in orthopaedics, and all orthopaedic-related portions of the PROMIS® are written at or below the most stringent recommendations (≤ 6th grade reading level), and more than 90% of independent PROMs and all PROMIS® materials are written at or below an 8th grade level. Additionally, the use of the Centers for Medicare and Medicaid Services guidelines for editing high reading level PROMs yields satisfactory results. CLINICAL RELEVANCE: Fears of widely incomprehensible PROMs may be unfounded. Future research to identify the most appropriate readability algorithm for use in the healthcare sector, and revalidation of PROMs after readability-improving edits is warranted.