What Does Your PROMIS Score Mean? Improving the Utility of Patient-Reported Outcomes at the Point of Care.

STUDY
DESIGN: Prospective cohort.
OBJECTIVES: Patient-Reported Outcome Measurement Information System (PROMIS) has been validated for lumbar spine. Use of patient-reported outcome (PRO) measures can improve clinical decision making and health literacy at the point of care. Use of PROMIS, however, has been limited in part because clinicians and patients lack plain language understanding of the meaning of scores and it remains unclear how best to use them at the point of care. The purpose was to develop plain language descriptions to apply to PROMIS Physical Function (PF) and Pain Interference (PI) scores and to assess patient understanding and preferences in presentation of their individualized PRO information.
METHODS: Retrospective analysis of prospectively collected PROMIS PF v1.2 and PI v1.1 for patients presenting to a tertiary spine center for back/lower extremity complaints was performed. Patients with missing scores, standard error >0.32, and assessments with <4 or >12 questions were excluded. Scores were categorized into score groups, specifically PROMIS PF groups were: <18, 20 ± 2, 25 ± 2, 30 ± 2, 35 ± 2, 40 ± 2, 45 ± 2, 50 ± 2, 55 ± 2, 60 ± 2, and >62; and PROMIS PI groups were: <48, 50 ± 2, 55 ± 2, 60 ± 2, 65 ± 2, 70 ± 2, 75 ± 2, 80 ± 2, and >82. Representative questions and answers from the PROMIS PI and PROMIS PF were selected for each score group, where questions with <25 assessments or representing <15% of assessments were excluded. Two fellowship-trained spine surgeons further trimmed the questions to create a streamlined clinical tool using a consensus process. Plain language descriptions for PROMIS PF were then used in a prospective assessment of 100 consecutive patients. Patient preference for consuming the score data was recorded and analyzed.
RESULTS: In total, 12 712 assessments/5524 unique patients were included for PF and 14 823 assessments/6582 unique patients for PI. More than 90% of assessments were completed in 4 questions. The number of assessments and patients per scoring group were normally distributed. The mean PF score was 37.2 ± 8.2 and the mean PI was 63.3 ± 7.4. Plain language descriptions and compact clinical tool was were generated. Prospectively 100 consecutive patients were surveyed for their preference in receiving their T-score versus plain language description versus graphical presentation. A total of 78% of patients found receiving personalized PRO data helpful, while only 1% found this specifically not helpful. Overall, 80% of patients found either graphical or plain language more helpful than T-score alone, and half of these preferred plain language and graphical descriptions together. In total, 89% of patients found the plain language descriptions to be accurate.
CONCLUSIONS: Patients at the point of care are interested in receiving the results of their PRO measures. Plain language descriptions of PROMIS scores enhance patient understanding of PROMIS numerical scores. Patients preferred plain language and/or graphical representation rather than a numerical score alone. While PROs are commonly used for assessing outcomes in research, use at point of care is a growing interest and this study clarifies how they might be utilized in physician-patient communication.

Introduction

Patient-reported outcome measures (PROMs) have become important tools for assessing health status in a variety of patient populations. Legacy PROMs are narrow in scope and are limited by the burden associated with their administration, making them useful only for specific populations. The Patient-Reported Outcomes Measurement Information System (PROMIS) was developed to overcome these limitations. The system was developed utilizing item response theory and computerized adaptive testing (CAT) which allows for reliable and efficient estimation of underlying health traits using targeted item banks to assess multiple domains, including physical function and pain interference with the least number of questions. PROMIS has been validated in a variety of patient populations, including spine, and has demonstrated a marked improvement in measurement characteristics and reduced patient and administrative burden.[4-6]

Patients with spinal pathology often seek care due to loss of function or pain, making PROMIS Physical Function (PF) and Pain Interference (PI) domains particularly relevant as outcome measures for spine care. While the value of PROMIS has grown for research and health economics applications, its use for clinical applications has been more limited. In part, this limitation is intrinsic to the definition of a PROM, which is in essence any report of the status of a patient’s health condition that comes directly from the patient, without interpretation by a clinician or anyone else.[8,9] This explicit goal bypasses providers and has likely slowed clinical implementation.

While PROMs and PROMIS in particular, have great potential to improve and guide clinical decision making, currently their use is hindered by limited patient and provider understanding of how to interpret PROMIS scores. Improving the understanding of PROMIS scores facilitates application in the clinical setting for patient counseling, decision making, and outcome evaluation at the point of care. The purpose of this study was to develop plain language descriptions to apply to PROMIS PF and PI scores and to assess patient understanding and preferences in presentation of their individualized PRO information.

Methods

Prospectively collected PROMIS PF CAT v1.2 and PROMIS PI CAT v1.1 questionnaires from patients visiting a large tertiary, university-based spine center were retrospectively reviewed. The PF CAT was administered from the PROMIS Physical Function item bank v1.2, which consists of 121 items. The PI CAT was administered from the PROMIS Pain Interference item bank v 1.1, which consists of 56 items. For both PF CAT and PI CAT each question is individually validated and calibrated along the continuum of physical function or pain interference. The algorithm for the CAT, which assigns the next item to be answered by the patient based on the previous answers, was provided through an application program interface (API) connected to the PROMIS Assessment Center (PROMIS Group). Item category responses range from 1 to 5. The scores for the PF CAT and PI CAT were recorded in T scores, derived from the US population, which has a mean score of 50 and standard deviation of 10 points. Low scores in the PF CAT represent low physical function, while high scores represent high physical function. Low scores in the PI CAT represent less pain interference, while high scores represent greater interference.

Completed assessments for patients with back or leg pain were included in analysis. Assessments with missing total scores for PF CAT and PI CAT outcome measures, those missing individual question data, a standard error greater than 0.32, and without a designated injury location were excluded. Additionally, assessments with less than 4 or more than 12 questions, as well as with cervical or upper extremity complaint locations were also excluded.

Assessments were grouped into “score groups” to reflect a clinically significant scoring based on minimum clinically important difference (MCID) thresholds for PROMIS scores.[11,12] Specifically, PROMIS PF groups were: <18, 20 ± 2, 25 ± 2, 30 ± 2, 35 ± 2, 40 ± 2, 45 ± 2, 50 ± 2, 55 ± 2, 60 ± 2, and >62; and PROMIS PI groups were: <48, 50 ± 2, 55 ± 2, 60 ± 2, 65 ± 2, 70 ± 2, 75 ± 2, 80 ± 2, and >82. The number (%) of assessments within each score group and for each clinical question were calculated. Question frequency and difficulty were also recorded and analyzed. For question scores with a nonzero standard deviation, 95% confidence intervals (CIs) were estimated using generalized least squares mixed-effects models to account for correlation when patients contributed multiple assessments.

Due to the large number of clinical questions for each score group, the complete list of questions was prohibitive to report and would have limited utility in a clinical setting. Thus, within each scoring group, subsets of questions were selected to describe and represent the group’s physical ability or limitation. Questions were considered if they were asked at least 25 times and represented at least 15% of assessments in the scoring group. Two fellowship-trained spine surgeons further trimmed the questions to the 3 best clinical questions for each score group. Selection was based on uniqueness and nonoverlapping confidence intervals with other questions in the score group. Each surgeon independently reviewed and selected individual question and answer combinations for all questions in each score group. A consensus process was used to reconcile any disagreement according to published norms.

Once the 3 best questions for each group were determined, they were translated into representative statements for each group. PROMIS PF plain language descriptions were then used to create a unique 15 question survey to assess both patient comprehension of and preference for consuming PRO data. The survey may be viewed in the appendix. The study population for the survey was constituted by 100 consecutive adult spine patients at a tertiary academic referral center. Findings presented with descriptive statistics and graphical displays used R v.3.4.4.

Results

Data from 12 712 assessments and 5,524 unique patients was included in the analysis after application of exclusion criteria. The number of assessments and unique patients in each scoring group were normally distributed and the majority of assessments were completed in 4 questions to generate the PROMIS PF score (mean 4.10 ± 0.60, median 4.00, interquartile range [IQR] [4,4]). The mean PROMIS PF score for all patients presenting for a lumbar spine or associated lower extremity complaint was 37.2 ± 8.2 (Figure 1). Data from 14 823 assessments and 6582 unique patients was included in the analysis. The number of assessments and unique patients per scoring group were normally distributed and 94.3% of assessments were completed in 4 questions to generate the PROMIS PI score. The mean PROMIS PI score was 63.3 ± 7.4 (Figure 2). Question frequency with regard to scoring group were presented as heat maps for both PF and PI. Those with lower PROMIS scores were asked easier questions more often and those with higher scores were asked harder questions more often. Similarly, for any given question, lower scoring groups trended toward a lower mean score than a higher scoring group (Figure 3). The clinical tools are reported by scoring group for PROMIS PF and PROMIS PI (Tables 1 and 2). Due to the intrinsic design of the PROMIS question system there was a statistically significant difference for each statement (compared with mean score for other scoring groups) in the clinical tool.

Figure 1.

Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS PF) assessments per score group.

Figure 2.

Patient-Reported Outcomes Measurement Information System Pain Interference (PROMIS PI) assessments per score group.

Figure 3.

Heat map of question scores per PROMIS PF question (A) and PROMIS PI question (B). Questions are in increasing order of difficulty. Lighter color indicates lower frequency of the question being asked. Darker color indicates higher frequency of the questions be asked. PFC12 and PAININ9 are anchor questions asked in every assessment. PROMIS, Patient-Reported Outcomes Measurement Information System; PF, Physical Function; PI, Pain Interference.

Table 1.

Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS PF) Clinical Tool.

PROMIS PF group	Descriptive summary statement
<18	Unable to walk about the house. Unable to wash and dry their body.Unable to transfer to a bed and chair and back.
20 ± 2	Unable to carry a shopping bag or briefcase.Wash and dry their body with much difficulty.Transfer to a bed and chair and back with much difficulty.
25 ± 2	Unable to do chores such as vacuuming or yard work.Run errands and shop with much difficulty. Walk about the house with much difficulty.
30 ± 2	Unable to do 2 hours of physical labor.Unable to walk at a normal speed.
35 ± 2	Carry a laundry basket up a flight of stairs with much difficulty.Walk at a normal speed with some difficulty. Unable to walk more than a mile (1.6 km).
40 ± 2	Do 2 hours of physical labor with much difficulty. Walk more than a mile (1.6 km) with some difficulty.
45 ± 2	Walk more than a mile (1.6 km) with little difficulty.Do chores such as vacuuming or yard work with little difficulty.
50 ± 2	Do two hours of physical labor with little difficulty.Walk more than a mile (1.6 km) with no difficulty.
55 ± 2	Do strenuous activities such as backpacking, skiing, playing tennis, bicycling, or jogging with no difficulty.Do heavy work around the house like scrubbing floors, or lifting or moving heavy furniture with no difficulty.
60 ± 2	Do vigorous activities, such as running, lifting heavy objects, participating in strenuous sports with no difficulty.Run at a fast pace for 2 miles (3 km) with little difficulty.
>62	Exercise hard for half an hour with no difficulty. Do 8 hours of physical labor with no difficulty.Run 10 miles (16 km) with some difficulty.

Table 2.

Patient-Reported Outcomes Measurement Information System Pain Interference (PROMIS PI) Clinical Tool.

PROMIS PI group	Descriptive summary statement
<48	Pain did not interfere with day to day activitiesPain did not interfere with enjoyment of recreational activitiesPain did not interfere with enjoyment of life
50 ± 2	Did not interfere with work around the home Pain did not interfere with social participationPain did not interfere with my enjoyment of life or interfered only a little bit
55 ± 2	Pain interfered with day to day activities a little bitPain interfered with work at home a little bit or somewhatPain interfered with enjoyment of social activities a little bit or not at all
60 ± 2	Pain interfered with day to day activities somewhatPain interfered with social participation somewhat or a little bitPain interfered with enjoyment of social activities somewhat or a little bit
65 ± 2	Paint interfered with social participation quite a bit or somewhatPain interfered with enjoyment of social activities quite a bit or somewhatPain interfered with day to day activities quite a bit or somewhat
70 ± 2	Pain interfered with work around the home very much or quite a bitPain interfered with family life quite a bit or very muchPain interfered with socializing with others often or sometimes
75 ± 2	Pain interfered with day to day activities very muchPain interfered with interpersonal relationships very much or quite a bitPain interfered with ability to take in new information quite a bit or somewhat
80 ± 2	Pain interfered with ability to take in new information very muchPain was so always or often so severe that I could think of nothing elsePain interfered with ability to remember things quite a bit or very much
>82	Pain always made me feel anxiousPain always prevented me from sitting for more than 10 minutesPain interfered with ability to concentrate very much

A total of 100 consecutive patients participated in the survey portion of the study with a 100% completion rate. Ninety-two percent of patients could correctly interpret PRO information in graphical form. Seventy-nine percent of participants preferred their PRO date in graphical format (Figure 4) or in plain language format when compared with T-score (Figure 5).

Figure 4.

Graphical description for patient Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS PF) abstracted from the survey instrument.

Figure 5.

Patient preferences for receiving Patient-Reported Outcomes Measurement Information System (PROMIS) scores.

Seventy-eight percent of patients found receiving personalized PRO information helpful, while only 1% found this specifically not helpful (Figure 6). Sixty-seven percent felt knowing their expected outcome would help them make a treatment decision. Fifty-eight percent of patients found the plain language descriptions for their individual score to be mostly or completely accurate, with 37% finding an adjacent scoring group to be more accurate (Figure 7).

Figure 6.

Patients overwhelmingly found the idea of receiving personalized outcome information helpful.

Figure 7.

Accuracy of plain language descriptions for Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS PF) scores.

Discussion

PROMIS has produced an efficient and valid set of outcomes for physical function and pain interference that is particularly useful for spine care. In comparison with legacy measures such as the Brief Pain Inventory (BPI), Oswestry Disability Index (ODI), Neck Disability Index (NDI), or Short Form-36 Physical Function Domain (SF-36 PFD), use of the PI or PF domains can give a more accurate representation of health status with less patient burden.[5-7,14] While the value of PROMIS in research and health care economics is readily apparent, utility in a clinical setting remains more limited as clinicians and patients lack a clear understanding of what PROMIS PF and PROMIS PI scores mean.[8,9] Improving the understanding of PROMIS scores facilitates application in the clinical setting for patient counseling, decision making, and outcome evaluation at the point of care. Thus, the purpose of this study was to develop plain language descriptions and a clinical tool to apply to PROMIS PF and PI scores and to assess patient understanding and preferences in presentation of this PRO information.

Limitations of the present study are primarily those intrinsic to PROMIS. Additionally, while PROMIS is freely available to everyone, the use of computerized adaptive testing may not be readily available to all providers. Both PROMIS domains utilized in this study were administered via CAT thus, not every possible question is included for each assessment. This study looks at aggregated PROMIS PF and PROMIS PI scores to evaluate and define the status of a patient’s disease state at a discrete point in time. Taken in isolation the scores provide relatively useful information, however, they become an even more powerful clinical tool when patients are followed longitudinally.

In this study, we found PROMIS PF and PROMIS PI to be very efficient for rapid self-reported assessment of the lumbar spine population. Greater than 90% of PROMIS PF and PROMIS PI assessments were completed in 4 questions. Only patients at the extreme ends of the functional spectrum, being severely disabled or highly capable, routinely required more than 5 questions to generate a PROMIS score. Recent literature suggests that patients would on average require only 35 seconds to answer the 4 to 5 questions administered in this survey. This is particularly appealing as legacy instruments are narrow in scope when compared with PROMIS and the ODI, for example, routinely requires 10 or more questions.

To create meaningful summary statements for different levels of physical function or pain interference in the study population, scoring groups for both PF CAT and PI CAT were generated based on 5-point intervals across the range of PROMIS scores. This was done corresponding to reported minimally clinically important difference (MCID) thresholds in the PROMIS literature. MCID thresholds from the foot and ankle literature have been calculated as an increase of 4.2 or greater in PROMIS PF, a decrease in 3.7 or greater in PROMIS PI and are similar to those described using anchor-based methodology for cancer patients.[11,12] This suggests that patients in separate scoring groups are clinically distinct in terms of physical function and/or pain interference from adjacent scoring groups. The complete list of descriptive summary statement for PROMIS PF and PROMIS PI scoring groups remained unwieldy in a clinical setting thus the streamlined clinical tool was developed via a nominal group technique with 2 fellowship-trained spine surgeons achieving consensus regarding clinical summary statements. We believe using this as a clinical tool is a step toward effectively implementing PROMIS PF and/or PROMIS PI scores in a clinical setting.

To date no other studies have investigated spine patient’s understanding and preferences with individualized PROMIS outcome data. The survey portion of this study provides valuable information about patient preferences for the consumption of PRO in a point of care setting. The findings indicate that patients largely understand and find PRO data useful and would integrate it into their treatment decision-making if made available. There was not consensus among patients on a single preferred method of presentation for individualized PROMIS PF scores; however, patients appeared to prefer graphical and plain language descriptions, or both compared with a numerical score.

The survey portion of the study highlighted a potential limitation in our ability to translate a numeric score into written text with the result of 37% of patients believing an adjacent scoring group’s descriptors were more accurate. This may be due to potential overlap in the questions for these groups. Adjacent groups are more likely to get similar or same questions with simple key words changed such as “All” or “Most” or “Some,” which may provide confusion to the patient when seen written as a statement.

Conclusion

This study developed plain language descriptions of PROMIS PF and PROMIS PI scoring groups to enhance the usefulness of PROMIS for patients with lumbar spine issues. Patients at the point of care, are very interested in receiving the results of their PRO measures and prefer either plain language or graphical representation or both rather than the T-score alone. These plain language descriptions have utility to enhance shared decision making and are helpful for patient expectation management. While PROs are commonly used for assessing outcomes in research, use at point of care is a growing interest and this study clarifies how they might be utilized in physician-patient communication.