An Evaluation of Health Numeracy among Radiation Therapists and Dosimetrists.

PURPOSE: Medical errors in radiation oncology sometimes involve tasks reliant on practitioners' grasp of numeracy. Numeracy has been shown to be suboptimal across various health care professionals. Herein, we assess health numeracy among American Society of Radiologic Technologists (ASRT) members. METHODS AND MATERIALS: The Numeracy Understanding for Medicine instrument (NUMi), an instrument to measure numeracy in the general population, was adapted to oncology for this study and distributed to ASRT members (n = 14,228) in 2017. Per NUMi scoring, health numeracy scores were categorized as low (0-7), low average (8-12), high average (13-17), or high (18-20). The impact of cGy versus Gy on numeracy performance was investigated. Spearman's rho and a Wilcox-Mann-Whitney test were used for comparisons between the different groups.
RESULTS: A total of 662 eligible participants completed the instrument and identified as radiation oncology professionals. In the cGy and Gy NUMi scores, approximately 2% of respondents scored low-average, approximately 40% scored high-average, and approximately 58% scored high, with a median score of 18.0. Although the optimum NUMi score for ASRT members is unknown, one might expect our cohort to have numeracy skills at least as high as college freshmen. Roughly one-sixth of our study group scored at or below the average score of college freshmen (NUMi = 15). In the subset analysis of NUMi questions pertaining to radiation dose unit (cGy vs Gy), respondents performed better with cGy (mean score: 2.94; range, 2-3) versus Gy (mean: 2.91; range, 0-3; P = .011).
CONCLUSIONS: In this study of limited sample size, overall numeracy is quite good compared with the general population. However, the range of scores is wide, and some respondents have lower scores that may be concerning, suggesting that numeracy may be an issue that requires improvement for a subset of the studied cohort. Performance was superior with the unit cGy; thus, the adoption of cGy as the standard unit is reasonable.

Introduction

Numerous societies in medicine and oncology have advocated for standardization as a means to reduce error and improve communication.1, 2, 3 Radiation oncology professional societies have followed suit, establishing guidelines for the structure and format of prescriptions, target and organ-at-risk nomenclature, terms to be included during big data efforts, content of treatment summaries, and inclusion of organs at risk by anatomic site.4, 5, 6, 7 Some decisions within these standardization efforts in radiation oncology were motivated by concerns about numeracy within radiation oncology professional groups.

Numeracy, defined as a person’s familiarity and ability to use and understand numbers and calculations, is a subset of health literacy and is further separated into objective (ie, performance-based measure of numeric skill) and subjective (ie, self-reported ease with numbers) numeracy. Inadequate objective numeracy has been identified as a source of medical error.8, 9, 10 The Radiation Oncology Incident Learning System aggregate reports describe numerous prescription incidents that might reflect suboptimal performance related to objective numeracy,, such as incidents where the number of fractions and dose per fraction were switched or calculation errors during emergency treatments. Such incidents support the rationale for standardization. Several primary studies and reviews have highlighted poor objective numeracy skills in pharmacists, nurses, physician trainees, and the general population.13, 14, 15

One particular aspect of numeracy is the potential impact of the units of radiation dose (Gy vs cGy), because this is used daily in clinical operations. The recommendation by the American Society for Radiation Oncology (ASTRO) to use cGy as the unit for dose prescription was met with considerable controversy (eg, during public comments and after publication on ASTRO’s social media platform). The recommendation endorsed by ASTRO made the choice for use of cGy for several reasons, including the avoidance of problematic decimals,16, 17, 18, 19 and concerns related to numeracy skills among radiation oncology professionals.

To our knowledge, objective numeracy has not been assessed in American Society for Radiologic Technologists (ASRT) members. Herein, we report on the results of a cross-sectional survey evaluation to assess objective numeracy in radiation therapist and dosimetrist ASRT members.

Methods and Materials

Study sample

Radiation therapist and dosimetrist members (n = 14,228) were identified as eligible for participation in the study, selected from the total ASRT membership (n = 153,102). A survey was sent to eligible ASRT members via the leadership committee through e-mail in May of 2017, and was open for participation for 2 weeks. To optimize the response rate, the introductory invitation e-mail described the salience of numeracy in patient safety, and a follow-up reminder e-mail was sent. No financial incentives were offered.

Study instrument

An existing survey of health numeracy, the Numeracy Understanding for Medicine instrument (NUMi), was adapted with the help of an expert in questionnaire development (MW). The NUMi was chosen for its ability to discriminate objective numeracy levels among participants with lower levels of numeracy and its cross-cultural equivalency. The previous data of inadequate numeracy levels among health care workers13, 14, 15, support the decision to choose a tool focused on discrimination among those with lower levels of numeracy for the initial assessment of this group.

NUMi is a measure of objective numeracy and consists of 20 numerical tasks within a health context with multiple-choice answers. The task of the assessment is to choose the correct result out of 4 possibilities offered for each question. The instrument measures objective health numeracy in 4 different domains: Number sense (5 questions), probability (5 questions), statistics (5 questions), and tables and graphs (5 questions). Performance on each of the domains of numeracy was evaluated. The final score is the sum of correct answers (theoretical range, 0-20). From the 20-question assessment, 17 questions did not mention radiation dose and had their original numerical units unaltered, and 8 of those 17 questions were adapted slightly to oncology (full assessment available in Appendix E1). The remaining 3 questions (within the categories of number sense and tables and graphs) were modified to each be asked both in the units of cGy and Gy, which increased the total number of questions asked to 23 for each respondent (ie, 3 questions asked in each of cGy and Gy units).

Each participant’s assessment was scored twice: Once for a cGy result (ie, 17 + 3 questions in cGy) and a second time for a Gy result (ie, 17 + 3 questions in Gy) with a maximum score of 20 each. Performance using cGy versus Gy is of interest to radiation oncology; thus, a subset comparison of the cGy and Gy questions was performed to mitigate the dilution effect of the other questions when looking for subtle differences in performance based on unit. Additional questions regarding basic demographics and preferences for units were added. As proposed by the developers of the NUMi, health numeracy scores were categorized as low (0-7), average (8-17), or high (18-20). Average was further divided as low average (8-12) and high average (13-17). For reference, the average NUMi score of the general population is 13.2. Those with no college education typically score 9.8, those with some college education 11.3, and those with ≥4 years of college 16.3. Medical students in their first year of a 6-year hybrid medical school/university program typically score 17.0 and those in their final year 19.0.

Statistical analysis

Descriptive and summary statistics were used to analyze the various examination domains (number sense, tables, probability, and statistics) and cohort demographics. Spearman’s rho was performed to identify whether respondent variables, such as age, sex, role (dosimetrist vs radiation therapist), and years in practice correlated with NUMi score. Wilcox-Mann-Whitney tests were used for comparisons of NUMi results between the different demographic groups. Significant variables (P < .05) were analyzed in a linear regression multivariable analysis. A paired t test was used to compare the scores on the 3 cGy versus Gy questions.

Results

A 6% response rate was obtained, with 992 individuals accessing and consenting to the survey. Of these respondents, 66.7% (n = 662; 4.7% of the total cohort invited) completed the instrument and identified as radiation oncology professionals. Only those completing the survey were included in the analysis. Respondents completing the survey were predominantly female, predominantly therapists, and the majority had been in practice at least 10 years (Table 1). Gender composition was consistent with ASRT membership. Participants who listed their role as other indicated that they were retired, served in dual roles, were in training, or were now in administration.

Table 1

Demographic characteristics of respondents (N = 662)

Sex, n (%)
Male	166 (25.1)
Female	483 (73.0)
Nonbinary	1 (0.2)
Unanswered	12 (1.8)
Age, n (%)
≤35 years	180 (27.2)
36-45 years	135 (20.4)
46-55 years	184 (27.8)
>55 years	154 (23.3)
Unanswered	9 (1.4)
Specialty, n (%)
Radiation therapy	436 (65.9)
Dosimetry	166 (25.1)
Other	55 (8.3)
Unanswered	5 (0.8)
Years in practice, n (%)
0-5 years	128 (19.3)
5-10 years	98 (14.8)
10-15 years	100 (15.1)
15-20 years	66 (10.0)
>20 years	264 (39.9)
Unanswered	6 (0.9)

As the data do not follow a normal distribution, median scores were reported. The median score in the overall study cohort was 18.0, which seems higher than the general population score previously reported at 13.2. Within the tested categories, descriptive statistics suggested that performance was the best for questions on number sense and tables and graphs (median percentage correct 100% within the category for both) and the poorest for probability and statistics (median percent correct within the category 80% for both).

On univariate analysis, health numeracy score was inversely correlated with both older age (Spearman’s rho: –0.17; P < .001) and length of time in practice (Spearman’s rho: –0.18; P < .001). When analyzed as a continuous variable, men had a statistically significantly higher score than women (median score: 18.0 for both; interquartile range for men 17-19 and women 16-19; P < .001). This difference was driven by lower scores in the tails of the curves by gender (Fig 1). Dosimetrists had a statistically significantly higher median health numeracy score compared with therapists (17.8 and 17.4, respectively; P = .03). On multivariable analysis, male sex and dosimetry profession remained statistically significant predictors for higher health numeracy score. This association held when examining assessments regardless of dose unit.

Figure 1

Distribution of Numeracy Understanding for Medicine instrument scores by sex.

For assessments where 3 of the 20 questions used cGy, 1.5% of respondents (n = 10) scored low average, 39.6% (n = 262) scored high average, and 58.9% (n = 390) scored high, with a median score of 18.0. For assessments where 3 of the 20 questions used Gy, 1.7% (n = 11) scored low average, 40.4% (n = 268) scored high average, and 57.9% (n = 383) scored high, with a median score of 18.0 (Fig 2). Although the optimum NUMi score for ASRT members is unknown, given the highly quantitative nature of their work, one might expect our cohort to have numeracy skills at least as high as a nonmedical group, such as college freshmen. Roughly one-sixth of our study group scored at or below the average score (NUMi: 15) in a group of first-year university sociology students. In the subset analysis of NUMi questions pertaining to radiation dose unit (cGy vs Gy), respondents performed better with cGy (mean score: 2.94; score range, 2-3) versus Gy (mean score: 2.91; score range, 0-3; P = .011).

Figure 2

Numeracy Understanding for Medicine instrument performance of respondents. Numeracy understanding for medicine instrument scores for cGy (gray) and Gy (black). ∗Conventional score bins (low average, high average, and high) are indicated on the x-axis of the graph. The median scores for other cohorts are given by vertical gray lines for reference. †Croatian 6-year medical school program.

Of those who responded, the majority of respondents (71.4%; n = 473) felt that cGy and Gy did not differ in error susceptibility, while 25.2% (n = 167) believed Gy was more susceptible to error and 2.7% (n = 18) felt cGy would be more error prone. Of those who responded, cGy was preferred by 64% (n = 421), Gy by 11% (n = 86), and a lack of preference or situation-dependent preference was stated by 23% (n = 155).

Discussion

This study is the first of its kind to measure health-related objective numeracy among ASRT radiation therapist and dosimetrist members. The assessment is geared for the general population (full assessment available in Appendix E1). NUMi performance measurements have not been made in other similar medical professional groups (eg, pharmacists, nurses); thus, limiting a direct comparison with these types of groups. However, some comparisons can be made based on available information. The mean NUMi score for the general population with some college education is 11.3, and when educational levels increase to at least 4 years of college, the mean score increases to 16.4. NUMi scores increase with progression through medical student education from 17.0 upon entry to 19.0 upon completion. Those with high levels of (nonmedical) education do not score as well as their medical colleagues, and those in later years of medical education score higher than their junior colleagues. Given this information, one would expect health care professionals who hold associate, bachelor, or even master degrees, such as dosimetrists and radiation therapists, to score more favorably than the general population, which is indeed the case (Fig 2). Although unclear what the ideal NUMi score is for this cohort, a large majority should score in the high range given their educational level and medical focus. Overall, only a slight majority (57.9%) of this sample scored in the high range for numeracy on this assessment geared for the general population, with a small number exhibiting low average numeracy. A comparison between this group and similarly educated health care professionals, such as licensed practical nurses or registered nurses, would be of interest but is not available. Thus, compared with the general population, overall numeracy is quite good in this cohort. However, the range of scores is wide. Because of the association of inadequate numeracy with medical error, the lower scores in this self-selected cohort are concerning.

Our study noted an inverse correlation between numeracy skills and years in practice. This finding is consistent with those of other studies noting associations with older age and poorer numeracy. This observation in our cohort may also reflect that the prerequisites for radiation therapy and dosimetry training have increased in recent years, leading the more recent, more highly educated graduates to perform better than their senior colleagues or a selection process in the training pathways that has changed over time.

Our study also noted a small but statistically different numeracy performance between genders, which was unexpected and in contrast to other studies. The collection of gender information can introduce stereotype threat,26, 27, 28 which can negatively influence the math performance of women. Demographic questions were asked at the end of the assessment to minimize the impact of stereotype threat, although nothing in the survey construction prevented individuals from viewing all questions before completion of the assessment. Given that women outnumbered men 3:1, the better sampling of women possibly allowed for the inclusion of individuals on the tails of the curves and this difference may simply represent a more complete sampling rather than an inherent difference. The gender difference observed is felt to be reflective of an expression of the stereotype threat phenomenon, better sampling of women, or other unseen bias. Regardless, the observed difference between genders was small.

In contrast to the small observed gender difference, the difference in performance between dosimetrists and therapists is likely valid, and felt to be related to the higher educational attainment of dosimetrists relative to therapists. The relationship between numeracy and education has been well established, and is congruous with the existing literature. However, interestingly, dosimetrist ASRT members could differ from dosimetrist members of the American Association of Medical Dosimetrists. Dosimetrist ASRT members could be more likely to have come to dosimetry through previous careers as therapists and receive training through their job rather than master and bachelor degree programs more common to those training directly as dosimetrists. As such, this dosimetrist cohort could be less highly performing in numeracy compared with the numeracy performance of the profession of dosimetry as sampled through American Association of Medical Dosimetrists membership.

An interesting finding in this study was that, upon direct comparison of the questions using cGy, respondents performed better with cGy than in questions using Gy. Although the difference was small, strikingly a performance difference was found in just 3 questions. Additionally, this difference was seen despite the fact that these questions were within the highest performing domains of number sense and tables and graphs, which is intriguing and supports the notion that cGy may be less error prone, as suggested by the ASTRO standard prescription paper. This is an important finding, because approximately 75% of our study cohort did not believe that one unit was more error prone than the other. Despite ASTRO’s proposal that cGy become the standard unit,, unit standardization in radiation oncology has not happened. A larger study with more questions involving both dose units may be warranted to see if this performance difference persists.

The descriptive statistics in this study also suggested that relative performance on the number sense and tables and graphs subcategories was superior to performance in the statistical and probability subcategories. Number sense tasks would seem to be the most safety critical tasks of this professional group (eg, involving the performance of emergency calculations or summing individual beam contributions to the total prescription dose). Similarly, tables and graphs are ubiquitous for these professionals in their daily work. However, mastery over other numeracy concepts (eg, statistics and probability) might also be helpful (eg, with regard to treatment plan evaluation/generation and associated tumor control and toxicity probabilities). Nevertheless, this wider understanding of health-related concepts are likely less critical to their daily task performance. The correlation of these measures of objective numeracy with certifying board-examination performance or performance reviews would add to our understanding of which measures of objective numeracy, if any, are best correlated with high-performing therapists and dosimetrists.

There are several limitations to our study. First, the response rate was low (6%), with 4.7% of the total cohort completing the entire instrument. The mean response rate for e-mail surveys is approximately 36%,, with more recent years having poorer response rates. Radiation oncology surveys, in particular, can have response rates as low as 5%. A shorter survey length, the absence of respondent prenotification, the presence of follow-up contacts, and issue-salience also seem to positively affect response rate., For this study, we chose a brief numeracy assessment, did not use respondent prenotification, did have follow-up contact, and discussed why numeracy was salient in the e-mail introduction to help address these issues. Second, only two-third of those starting actually completed the assessment, leading potentially to nonresponse bias. Considering how nonresponders may differ from responders is important. This survey was introduced as an assessment of numeracy, but those who hate numbers may have been less likely to electively complete a survey that requires something they do not enjoy or causes mathematics anxiety. Mathematics anxiety has been associated with inferior objective numeracy performance. Thus, the results from the subset who completed the survey might have a higher rate of objective numeracy than the broader cohort of potential participants, which suggests that this study may overestimate the actual numeracy in this cohort and underestimate the scope of the problem with numeracy. Third, performance on a test is more favorable than performance in clinical environments, which may have skewed the results to demonstrate a higher level of numeracy than practiced in a typical busy clinical environment. Fourth, the design of the NUMi only allowed for the adaptation of questions for expression in both cGy and Gy format in only 3 questions, which limits the magnitude of the difference to be observed. Finally, this work investigates only numeracy, and is not intended to be a comprehensive examination of all potential sources of error in radiation oncology.

Despite these limitations, these data are worthwhile as an initial objective numeracy assessment in this cohort. Available assessments of health numeracy are limited, and the NUMi was a good choice given that the instrument can discriminate among lower levels of health numeracy and is brief enough for use in voluntary subjects. The number of evaluable subjects is large, and provided an appropriate sample size for this initial assessment despite these limitations. These data suggest that numeracy issues may be present among dosimetrists and therapists, as seen in other health care professionals,, and a move to cGy as the standard unit may reduce the number of errors. Training and education are not particularly effective error prevention strategies; thus, we recommend system-based approaches to poor numeracy, such as the implementation of the standard prescription, adoption of cGy, and investigation of artificial intelligence means to detect prescription errors.

Conclusions

Within the limitations of this limited sample size, the majority of radiation therapists and dosimetrists have high levels of numeracy compared with the general population, although a subset of this ASRT cohort demonstrates concerning levels of numeracy for a health care worker group. Given the possible concerns related to numeracy (particularly in older subjects), initiatives to improve numeracy among radiation therapists and dosimetrists may be helpful. Better numeracy performance with the dose unit cGy was suggested in this cohort, which supports the utility of initiatives that may reduce numerical confusion, such as the ASTRO-endorsed standard prescription and unit standardization.