How Oncologists Perceive the Availability and Quality of Information Generated From Patient-Reported Outcomes (PROs).

BACKGROUND: Despite increased incorporation of patient-reported outcome (PRO) measures into clinical trials, information generated from PROs remains largely absent from drug labeling and electronic health records, giving rise to concerns that such information is not adequately informing clinical practice.
OBJECTIVE: To evaluate oncologists' perceptions concerning the availability and quality of information generated from PRO measures. Additionally, to identify whether an association exists between perceptions of availability and attitudes concerning quality.
METHOD: An online, 11-item questionnaire was developed to capture clinician perspectives on the availability and use of PRO data to inform practice. The survey also asked respondents to rate information on the basis of 4 quality metrics: "usefulness," "interpretability," "accessibility," and "scientific rigor."
RESULTS: Responses were received from 298 of 1301 invitations sent (22.9% response rate). Perceptions regarding the availability of PRO information differed widely among respondents and did not appear to be linked to practice setting. Ratings of PRO quality were generally consistent, with average ratings for the 4 quality metrics between "satisfactory" and "good." A relationship was observed between ratings of PRO data quality and perceptions of the availability.
CONCLUSION: Oncologists' attitudes toward the quality of information generated from PRO measures are favorable but not enthusiastic. These attitudes may improve as the availability of PRO data increases, given the association we observed between oncologists' ratings of the quality of PRO information and their perceptions of its availability.

Introduction

Cancer drugs often carry substantial treatment-related toxicities that may negatively impact patients’ physical functioning and overall health-related quality of life (HRQoL) (1). While measures of treatment activity have provided the primary support for drug approval and payment decisions in oncology, they do not necessarily reflect patient perceptions of treatment benefit. Patient-reported outcomes (PROs) and clinical outcome assessments more broadly are important for characterizing clinical benefit, or “the impact of a treatment on how a patient feels, functions or survives,” and can contribute meaningfully to efficacy and safety evaluations of a new treatment (2 –4).

Much of the recent excitement around PROs stems from the recognition that these tools can be meaningful and reproducible and in many cases more accurate than clinician assessments (5). Historically, PRO tools were used primarily in oncology as research tools or in the measurement of palliative care interventions. However, PROs are now also used to measure HRQoL, disease-related symptoms, functional impacts, treatment-related toxicities, treatment satisfaction, and in some cases the anticancer activity of drug interventions (6,7). Particular focus in recent years has centered around the measurement of adverse events reported by the patient (eg, PRO-CTCAE) (8).

A recent review of ClinicalTrials.gov found that between 2007 and 2013, the number of oncology trials that included at least one PRO measure has increased to approximately one-third of registered trials (9). Accordingly, regulatory agencies in the United States and Europe have taken steps to establish guidance for the use of PROs in clinical trials (10 –12).

Despite a growing consensus regarding the importance of PROs and their regular incorporation into trial designs, there are concerns that the information generated from PROs is not reaching clinicians and patients (13). Much of this concern centers around the limited inclusion of patient-reported information in US Food and Drug Administration (FDA) product labeling. A recent analysis found that out of 160 approved hematology and oncology drugs between 2010 and 2014, only 3 included information generated from PROs in labeling (14), although there have been additional label claims in the years since. More broadly, the literature on clinical trials that has included PROs has suffered from heterogeneity in the way data are analyzed, presented, and interpreted, hindering the incorporation of information into clinical guidelines and health policy (15).

We developed a survey to find out the degree to which clinicians felt that PRO information was available to them, where they typically find such information, and their opinion on the quality of that information. Insights from this survey are intended to help policymakers and others discover how to disseminate PRO data more effectively.

Methods

An 11-item, online physician survey was developed to collect anonymized information from physicians on their use of different sources of prescribing information (Table 1). Five items (items 7-11) specifically asked about physicians’ use of and attitudes about PRO information and are the subject of this analysis. The survey was piloted by 4 physicians prior to being distributed via e-mail to 1301 oncologists, who were recruited from a panel of medical professionals in the United States by a commercial research organization specializing in online physician surveys. Physicians were eligible if they reported being a board-certified oncologist or neurologist and had treated at least 10 patients in the past 12 months. The survey company, M3, verified the credentials of physicians opting in for survey research. Demographic and professional information was collected from each physician, including gender, type of practice (private, academic or community), and number of years in practice. Physicians were informed of the sponsors of the survey. The survey was open from December 2017 to February 2018. Each participating physician was given a small honorarium as compensation for their time.

Table 1.

Questionnaire Items.a

Item	Question	Response Choices
1-6	Items 1-6 of the questionnaire did not address the use of patient-reported outcomes
7	What is your level of agreement with the following statement? Patient-reported Outcome (PRO) data are widely available to prescribers in my field.	a. Strongly agreeb. Somewhat agreec. Neither agree or disagreed. Somewhat disagreee. Strongly disagree
8	To what extent have you considered patient-reported outcome (PRO) data when making prescribing decisions?	a. Alwaysb. Very Oftenc. Sometimesd. Rarelye. Neverf. Not applicable
9	Please rate the PRO information you have consulted in your practice on the following metrics:  Accessibility  Usefulness  Scientific rigor  Interpretability	a. Excellentb. Very goodc. Goodd. Satisfactorye. Poorf. Unsure
10	In the past, what sources have you used to access PRO data on a specific drug? (select all that apply)	a. Journal articlesb. Conference abstracts/postersc. Sponsor company resourcesd. Patient forumse. Product labelsf. Clinical guidelinesg. Otherh. None
11	What is your level of agreement with the following statement?  Adding a new section to the FDA product label that would contain PRO data would have a positive and meaningful impact on my prescribing decisions.	a. Strongly agreeb. Somewhat agreec. Neither agree or disagreed. Somewhat disagreee. Strongly disagree

aAn online, 11-item questionnaire was developed to collect anonymized information on oncologists’ attitudes toward different sources of prescribing information. The final 5 items of the questionnaire were the focus of this analysis. The questionnaire was comprised of Likert/Likert-type scale and multiresponse questions. Respondents were asked to indicate the extent to which they have consulted information from patient-reported outcomes and then to rate that information using Likert-type scales.

By electing to complete the survey, respondents provided consent to use their anonymous responses. This study qualified as market research, as it did not involve patients or data on patient characteristics. As such, institutional review board and ethics committee approval and informed consent were not required, per current US regulations.

The survey was comprised of Likert/Likert-type scale and multiresponse questions. Data were pooled across participants and analyzed at the item-level using the R software package. Respondents were excluded from the analysis who answered “never” or “not applicable” to item 8 (15 respondents) and “unsure” to item 9 (20 respondents for “interpretability,” 21 for “accessibility,” 23 each for “usefulness” and “scientific rigor”). The rational for excluding those who answered “unsure” at item 9 was that they represented a small fraction of the total number of respondents and it was unclear why they were not sure how to rate the PRO information on the suggested metrics. Hypothesis testing was performed to assess whether there is strong evidence that the majority of oncologists (more than 50%) report that they “somewhat agree” or “strongly agree” that PRO information are widely available to them (item 7); “always” or “very often” consider PRO information when making prescribing decisions (item 8); and “somewhat agree” or “strongly agree” with the utility of adding a new section to the FDA product label that would contain PRO information (item 11). Hypothesis testing was also conducted to determine which resources are used by the majority of oncologists to access PRO information.

An oncologist’s overall view of PRO data was quantified using a composite score based on their ratings of accessibility, interpretability, usefulness, and scientific rigor (item 9). The score was computed and validated using weights from a factor analysis (Supplemental Methods). Hypothesis testing compared the scores derived from the factor analysis across different populations of oncologists.

Results

Surveys were distributed to 1301 oncologists across the United States and responses were received from 298 (22.9% response rate). Of these respondents, 73% were male, 46% practiced in a private setting, and 41% had been in practice for 10 to 24 years (Supplemental Table 1). Information about the respondents’ main area of focus was captured. One hundred seventy-four (58%) respondents focus on general oncology, 142 (48%) on hematology, and 98 (33%) mentioned some specific areas of specialization, with breast, lung, and gastrointestinal cancers being named most often. Given the high proportion of respondents mentioning general oncology, as well as the high proportion of respondents who mentioned more than one area of specialization, no analysis at the specialty level was performed. Geographic information was captured for 59% of respondents.

Perceptions regarding the availability of PRO information and frequency of use in making prescribing decisions differed widely among respondents (Figure 1). For example, 43% of respondents agreed (either “strongly” or “somewhat”) that PROs are widely available and 34% disagreed (either “strongly” or “somewhat”). Additionally, 22% of respondents reported they “always” or “very often” consider PRO information when making prescribing decisions, whereas 27% reported they “rarely” or “never” consider PRO information. The most commonly cited sources of PRO information were “journal articles” (62%) and “clinical guidelines” (45%).

Figure 1.

Oncologists’ perceptions of availability of patient-reported outcome (PRO) data. Items 7 and 8 asked respondents to report their perceptions of the availability of PRO information as a prescribing resource. Item 10 asked respondents to select sources they have used to access PRO information in the past. Item 11 gauged respondents’ level of agreement with the utility of adding a new section to product labeling that would contain PRO data. Responses for 33 respondents were not considered for items 10 and 11 due to a response of “never” to item 8 or “unsure” to item 9.

Respondents rated the quality of PRO data between “satisfactory” and “good” on average (Figure 2). No major differences in ratings of the 4 quality metrics “usefulness,” “accessibility,” “interpretability,” and “Scientific rigor” were observed; however, respondents gave slightly higher scores to PRO data on the basis of the “usefulness,” with 54% of respondents providing a rating of “good,” “very good,” or “excellent.”

Figure 2.

Oncologists’ ratings of PRO information. Item 9 asked respondents to rate the PRO information they have consulted on the basis of 4 quality metrics: “usefulness,” “interpretability,” “accessibility,” and “scientific rigor.” Twenty-three respondents selected “unsure” when asked to provide ratings, and their responses were eliminated from the analysis.

Hypothesis testing was used to investigate the impact of specific criteria on ratings of PRO quality (Table 2). As theorized, there was evidence that oncologists who believe that PRO data are widely available and those who use PRO data to prescribe medications rated it higher on average. Results also showed that the majority (63%) of oncologists “somewhat to strongly agree” that adding a new section to the FDA product label with PRO data would have a meaningful impact on their prescribing decisions.

Table 2.

Hypothesis Test Results.a

Item(s)	Research Question	Hypothesis Test	Results, P [95% CI]
7	Do the majority of oncologists “somewhat agree” or “strongly agree” that PRO data are widely available to prescribers in their field?	One-sample proportionH0: P = .5HA: P > .5	.9946 [.37-.48]
8	Do the majority of oncologists consider PRO data when making prescribing decisions “always’ or “very often”?	One-sample proportionH0: P = .5HA: P > .5	1 [.18-.27]
9	Do the majority of oncologists consider PRO information “excellent” or “very good” on the basis of the following metrics?“Accessibility”“Interpretability”“Usefulness”“Scientific rigor”	One-sample proportionH0: P = .5HA: P > .5	1 [.21-.31]1 [.21-.32]1 [.27-.38]1 [.20-.30]
10	Do the majority of oncologists use the following sources to access PRO data on a specific drug?Journal articlesClinical guidelinesProduct labelsSponsor company resources	One-sample proportionH0: P = .5HA: P > .5	.0005 [.54-.65].98 [.38-.50]1 [.28-.41]1 [.16-.26]
11	Do the majority of oncologists “somewhat agree” or “strongly agree” with adding a new section to the FDA product label that would contain PRO data would have a positive and meaningful impact on their prescribing decisions?	One-sample proportionH0: P = .5HA: P > .5	<.0001 [.57-.69]
7, 9	Are “an oncologist’s opinion that PRO data is widely available” and “an oncologist’s rating of PRO data” related?	Chi-squareNull hypothesis: No relationship	<.0001 n/a
7, 9	Are oncologists who believe that PRO data are widely available more likely to rate it higher than those who do not believe it is widely available?	Two-sample t testH0: μ1 - μ2 = 0HA: μ1 - μ2 > 0	<.0001 [3.8-5.8]

Abbreviations: CI, confidence interval; FDA, Food and Drug Administration; N/A, not applicable.

aHypothesis testing was performed to assess whether there is strong evidence that the majority of oncologists (more than 50%) report that they: “somewhat agree” or “strongly agree” that PRO information are widely available to them (item 7); “always” or “very often” consider PRO information when making prescribing decisions (item 8); consider PRO information “excellent” or “very good” on the basis of 4 quality metrics; and “somewhat agree” or “strongly agree” with the utility of adding a new section to the FDA product label that would contain PRO information (item 11). Hypothesis testing was also conducted to determine which resources are used by the majority of oncologists to access PRO information.

Boldface values indicate statistically significant differences.

Discussion

We surveyed oncologists regarding their perceptions of available PRO information and the extent to which they use PRO data to inform treatment decision-making. Overall, we found that oncologists hold heterogeneous views on the extent to which PRO data are available and the quality of the information they have access to. Oncologists currently hold favorable but not enthusiastic opinions regarding the quality of PRO information they have considered. On average, respondents rated PRO information between “satisfactory” and “good” on the basis of 4 quality metrics: usefulness, interpretability, accessibility, and scientific rigor.

We also found that the majority of oncologists do not frequently use PRO data when making prescribing decisions. Given that PRO data have not traditionally been well represented in product information and the lack of standardization with regard to how such information is presented in the clinical trial literature (5), this is not entirely surprising. However, we found a clear link between those who consider PRO data as generally available and more positive attitudes about data quality. This suggests that familiarity with PRO data and scientific acceptance are associated with integration into practice. Although it is not possible to make statements about causality, increasing access to PRO data and improving the quality of the data may encourage integration into clinical practice and are worthy goals in the move toward patient-focused drug development.

Given the relationship between perceptions of PRO availability and ratings of PRO data quality, our research suggests that increased exposure to PRO information may improve physician regard for such data. Therefore, we lay out the following recommendations for how to increase utilization and uptake of PRO data for treatment decision-making by physicians.

First, continued efforts should be directed toward conveying PRO information through drug labeling. Information found on drug labels is used by a range of other prescribing resources and may thus increase prescriber exposure to such information in a range of venues. Moreover, some have suggested that market forces will encourage manufacturers to invest more in PRO labeling if they observe more success cases (16). However, given the many barriers to the inclusion of PRO data in labels, especially for cancer products, the FDA may need to consider additional opportunities for disseminating PRO data, such as through the development of a separate section of product labels specifically devoted to such information. As stated in a May 2017 public meeting, FDA officials are actively considering such an approach, either through the creation of a new section on printed package inserts or as online labeling appendices (17).

Second, in the absence of widespread access to PRO information on labels in the short term, clinical investigators will need to consider more digestible formats for the information in peer-reviewed publications. Peer-reviewed literature was identified in this study as the most relied upon source for accessing PRO information. As previously noted, the peer-reviewed literature has suffered from heterogeneity in the presentation of PRO data, hindering its accessibility.

Finally, utilization and uptake of PRO data will continue to increase if sustained support for patient-focused drug development continues. The 21st Century Cures Act and the most recent reauthorization of the Prescription Drug User Fee Act both contained important provisions related to the dissemination of PRO data and signaled policymakers’ support for a more patient-focused drug development process (18,19). Careful implementation of these statutes, as well as the timely development of new regulatory guidance, will further advance understanding of and support for patient-focused drug development.

Conclusion

This research summarizes the current acceptance and usage of PRO data for treatment decision-making among a sample of oncologists. Current attitudes toward PROs, though favorable, may improve as availability is increased, given the link between perceptions of PRO availability and oncologists’ rating of PRO information. Regulators should continue to evaluate new methods of conveying data from PROs to prescribers, such as through expansions of physician package inserts.

Click here for additional data file.

Supplemental_Method_Round_2 for How Oncologists Perceive the Availability and Quality of Information Generated From Patient-Reported Outcomes (PROs) by Michael Shea, Céline Audibert, Mark Stewart, Brittany Gentile, Diana Merino, Agnes Hong, Laura Lassiter, Alexis Caze, Jonathan Leff, Jeff Allen and Ellen Sigal in Journal of Patient Experience

Click here for additional data file.

Supplemental_Table_1_(1) for How Oncologists Perceive the Availability and Quality of Information Generated From Patient-Reported Outcomes (PROs) by Michael Shea, Céline Audibert, Mark Stewart, Brittany Gentile, Diana Merino, Agnes Hong, Laura Lassiter, Alexis Caze, Jonathan Leff, Jeff Allen and Ellen Sigal in Journal of Patient Experience