Mobile Clinical Trial Matching Technology in Medical Oncology Clinic: A Pilot Feasibility Study.

PURPOSE: The internet is a common source of health information for patients and can be leveraged to provide patient-facing clinical trial information. This pilot study integrated an online prostate cancer clinical trial matching technology, called Trial Library (TL), in an academic medical oncology clinic from February 2019 to April 2021. PATIENTS AND METHODS: This is a single-arm interventional pilot study among patients with a known prostate cancer diagnosis. Participants were given access to TL before seeing a provider. The primary and secondary study end points were the overall satisfaction with TL and the proportion of participant-initiated clinical trial discussion with providers after exposure to TL, respectively. The null hypothesis or true satisfaction rate (acceptability) was tested against a one-sided alternative and was rejected if 29 or more satisfactions were observed.
RESULTS: Among 272 patients approached, 66 provided informed consent to participate in the study. The mean age was 70.8 years (standard deviation = 7.9). The majority of participants were White (82%) and had metastases present at the time of enrollment (65%). The baseline clinical trial discussion rate ascertained via electronic medical record review was 28%. After accessing TL, a significantly larger proportion of participants (48.5%) discussed clinical trials during the clinic visit (P = .007), half of which were patient-initiated. The majority of participants indicated that TL increased their interest in clinical trials (68.2%); however, satisfaction/extreme satisfaction with the technology was 38%.
CONCLUSION: Access to TL resulted in a significant increase in patient-initiated discussions regarding clinical trials and an increase in interest in clinical trial participation although these data do not address if this resulted in increased accrual to clinical trials. The satisfaction rate did not meet the target to reject the null hypothesis, suggesting the need for iterative design of patient-facing health information.

INTRODUCTION

The internet is a common source of health information for patients with cancer and can be used to provide patient-centered clinical trial information.[1] Previous research evaluating YouTube content on prostate cancer (PCa) clinical trials, a commonly used and trusted source of online content in the United States,[1-3] was found to be of poor quality, containing commercial bias and lacking in racial/ethnic diversity in representation.[4] Clinician implicit bias has also been shown to contribute to inequities in which patients are offered clinical trial participation.[5,6] The underlying hypothesis of this pilot study is that access to reliable, online, and patient-facing health information regarding clinical trials will facilitate patient-initiated clinical trial discussions and contribute to promoting equity in access to clinical trials.

CONTEXT

Key Objective

To test the feasibility and efficacy of an integrated online prostate cancer clinical trial matching technology, called Trial Library (TL), in an academic medical oncology clinic on cancer clinical trial discussion.

Knowledge Generated

TL significantly influenced clinical trial discussion during the medical oncology clinic visit. The majority of participants indicated that TL increased their interest in clinical trials.

Relevance

The internet is a common source of health information for patients and can be leveraged to provide patient-facing clinical trial information.

This pilot study integrated a mobile PCa clinical trial matching technology, called Trial Library (TL), in medical oncology clinic to promote patient-initiated clinical trial conversations. This pilot study sought to measure acceptability and preliminary estimates of efficacy of TL in eliciting patient-initiated clinical trial discussion regarding clinical trials.

PATIENTS AND METHODS

Recruitment Procedures

Participants were recruited for a single-arm interventional pilot study at an academic medical center to gain access to TL.[7] The content of TL has been previously reported.[8] TL is a clinical trial matching technology developed using human-centered design and involving feedback from a diverse population of men with advanced PCa (Figs 1A-C). TL is a bilingual (English and Spanish) platform that allows users to answer simple clinical questions that allow matching to PCa (therapeutic and nontherapeutic) clinical trials available at the University of California, San Francisco. TL is not integrated in the electronic health record. Eligible study participants had a known diagnosis of PCa, were English-speaking or Spanish-speaking, and were scheduled for a new or follow-up clinical visit in genitourinary medical oncology clinic. Participants were recruited to the study from February 2019 to April 2021. Initially, all participants were recruited in-person by clinic research personnel at the time of check-in for a scheduled clinic visit. However, as a result of the COVID-19 pandemic, starting in June 2020, participants were approached and recruited virtually within 7 days before a telemedicine or an in-person clinic visit. All study procedures were approved by the University of California, San Francisco Institutional Review Board.

FIG 1.

TL wireframes: (A) Trial Library home page, (B) clinical trial matching questions, and (C) clinical trial report. TL, Trial Library.

Access to TL

Participants who provided electronic informed consent were given access to TL with an URL and unique access code before seeing a provider, whereas participants who provided in-person informed consent were given an iPad with a unique access code to access TL while in the waiting room. Participants using TL in a face-to-face visit were given an opportunity to generate a list of matching clinical trials and a printout of the list before seeing a provider. Participants completed a single-item literacy screener[9] to identify adults in need of help with printed health material.

Follow-Up Procedures

All participants were electronically mailed a postvisit survey to measure acceptability and efficacy of TL within one hour after the clinical visit with a provider. For participants who did not complete the postvisit survey, an electronic reminder was sent weekly up to three times. All patient participants and a subset of provider participants were invited for a follow-up semistructured 15-minute interview with a clinical researcher to assess feasibility and acceptability with TL and a study investigator (Appendix 1).

Electronic Medical Record Review

Patient demographic and clinical characteristics were collected through electronic medical record review. In addition, a review was undertaken of the electronic medical record of 100 randomly selected patients with PCa seen in the same clinic, who were not already on a clinical trial or participating in this study. In June 2021, to determine the baseline percentage of patients who discuss clinical trials with providers, all data were inputted by a research assistant into Research Electronic Data Capture software (Vanderbilt University).[10,11]

Google Analytics

Google Analytics software was used to capture passive measures of engagement such as the number of unique user sessions on website, average length of session (min:sec), average number of page views, and bounce rate, or percentage of single-page sessions in which there was no interaction.

Study End Points

The primary end point of the pilot study was to measure the overall participant satisfaction rate with TL. The secondary end point was to measure the proportion of patient participants who report initiating a clinical trial discussion. Semistructured interviews with a subset of patient and provider participants were undertaken to provide contextual information about the primary end points.

Statistical Analysis

Demographic and clinical characteristics were summarized using descriptive statistics. The sample size was 66 participants,[8] with a reference satisfaction rate of 35% that was previously defined in the protocol manuscript.[8] The overall satisfaction rate reflects the proportion of participants who report some degree of satisfaction on a five-item Likert scale from extremely dissatisfied to extremely satisfied with TL. The null hypothesis or true satisfaction rate (acceptability) was tested against a one-sided alternative. The null hypothesis was rejected if 29 or more satisfactions are observed. Feasibility and acceptability were also assessed using semistructured interviews. Qualitative methods applied were previously reported in the protocol manuscript.[8]

RESULTS

Study Enrollment

As seen in the CONSORT diagram (Fig 2), 272 patients were approached to participate (n = 39, in-person; n = 233, electronic), of whom 84 (31%) participants provided informed consent. According to Google Analytics, 71 participants entered an access code to log into the TL website, of whom 66 completed the postvisit survey.

FIG 2.

CONSORT diagram for TL in the in-person clinic pilot study. TL, Trial Library.

Patient Characteristics

As seen in Table 1, the mean age of participants was 70.8 (standard deviation [SD] = 7.9) years. A total of 54 (82%) participants were White, five (8%) were Asian/Pacific Islander, two (3.0%) were Black/African American, and five (8%) were others/unknown. The majority (n = 57, 86%) of participants had a bachelor's degree or higher and were Medicare-insured (n = 41, 62.1%), and only five (7.7%) had a positive single-item literacy screener. All study participants were English-speaking. The characteristics of patients included in the random baseline audit are reported in Appendix Table A1.

TABLE 1.

Participant Characteristics

TABLE A1.

Patient Characteristics of Randomly Audited Patient Charts (n = 100)

A substantial proportion (n = 24, 36.4%) of participants had multimorbidity (≥ 1 comorbidities). The mean time since PCa diagnosis was 3.86 years (SD = 4.87). The majority of participants had high-risk disease on the basis of the Gleason score ≥ 8 (n = 39, 59%) and metastatic disease (m = 43, 65%) at the time of study enrollment and were diagnosed in a community clinical setting (n = 47, 72%). A smaller subset of participants had metastatic androgen deprivation therapy–resistant PCa (n = 5, 7.6%). The mean prostate-specific antigen level at the time of enrollment was 14.2 ng/mL (SD = 48.4). Treatment history included radical prostatectomy (n = 30, 45.5%), definitive radiotherapy (n = 15, 22.7%), salvage/adjuvant radiation (n = 18, 27.3%), palliative radiation (n = 13, 19.7%), systemic chemotherapy (n = 3, 4.5%), radioligand therapy (n = 2, 3.0%), immunotherapy (n = 2, 3.0%), and hormone therapy (n = 49, 74.2).

TL Efficacy and Engagement

As shown in Table 2, a total of 32 (48.5%) participants discussed clinical trials during the visit with an oncologist, compared with 28% of patients in the random sample of audited medical records (P = .007). Among participants who discussed clinical trials, 50% initiated the discussion. In the medical record audit of 100 usual care patients, 28 had documentation of a clinical trial discussion. The majority of participants indicated that TL increased their interest in clinical trial participation (n = 45, 68.2%) and would recommend it to a friend or family member (n = 37, 56.1%). Among respondents, 25 (37.9%) reported being satisfied/extremely satisfied with TL. A subset (n = 11, 16.7%) reported being extremely dissatisfied/dissatisfied with the technology.

TABLE 2.

TL Efficacy and Engagement

TL Measures of Engagement

A total of 82 sessions on website were captured for the 66 participants. The average length of each session was 10 minutes and 7 seconds with an average number of 15.27 pages viewed per session and a 0% bounce rate or drop-off after viewing a single page.

Semistructured Interviews of Patients and Providers

A purposive sample of five patients (P) and three medical oncologists (MD) participated in semistructured interviews with a research coordinator to assess acceptability and feasibility of integration of TL in clinic. Four major themes arose from thematic analysis (Table 3): enriched patient-provider discussion, usefulness of platform, awareness of clinical trials, and suggestions for further refinement of TL. From the patient perspective, all participants reported that interacting TL introduced them to potential PCa clinical trials. One participant stated that it provided him “hope that there's other avenues, that there's other medicine or opportunities that will maybe prolong [his] life or help subside the side effects.” Another participant noted that it “sparked [him] to ask a couple of questions that [he] wouldn't have otherwise asked about what these trials were on and what the drugs were being tested or the procedures.” From the oncologist perspective, participants emphasized the usefulness of a patient-centered resource for clinical trials and how discussion of clinical trials may enhance the patient-provider experience. For example, one participant stated:

TABLE 3.

TL Semistructured Interview Themes and Example Quotes

“It helps me educate [patients] regarding the new trials that are available and why or why not that they could either consider the trial or not, depending on where they are in terms of their cancer therapy. It also enables the patients to have some kind of control on the cancer treatment that they're on.”—MD (2003)

Participants offered several suggestions for improvement to TL including development of a provider-facing platform, additional filtering criteria to further tailor the list of clinical trials, and reconsideration of the timing of introduction to TL. Participants suggested that complexity and length of language on the clinical trial description page contributed to dissatisfaction.

DISCUSSION

This study demonstrates the utility of online patient-facing intervention to promote clinical trial discussions in an academic oncology practice. We observed that TL had a significant impact on frequency of clinical trial discussion during medical oncology visits for patients with advanced PCa and increased interest in clinical trial participation. Given the rapidly changing clinical trials environment during the COVID-19 pandemic, an analysis of the impact of use of this patient-facing tool on the number of accruals to clinical trials was not undertaken. Satisfaction rates among study participants were lower than that expected driven by difficulty of reading, or low readability, of the clinical trial report. This satisfaction rate did not meet the target to reject the null hypothesis, suggesting the need for iterative design, simplification of language, cultural tailoring, and frequent user engagement.

Although patient satisfaction is a commonly used measure in health care,[12] it is seldom used to evaluate patient education materials or online health information available to the patient consumer. A previous study evaluating PCa YouTube content more generally observed high degrees of frank misinformation.[13] Another analysis examining PCa clinical trial YouTube content classified most as poor quality,[4] suggesting a need for consumer feedback on available online information. A large subset of participants (n = 25, 37.9%) indicated being satisfied/very satisfied with TL; however, this did not meet the target to reject the null hypothesis. A subset of some participants (n = 11, 16.7%) reported being extremely dissatisfied/dissatisfied with the technology. On the basis of the semistructured interviews and comments in the postvisit survey, dissatisfaction was primarily driven by limited usability of the matching filter for cancer staging (Fig 1B), which generated patient confusion and required troubleshooting by the user as well as difficulty in reading and understanding the generated clinical trial report. These observations highlight the need for iterative feedback that informs human-centered design of patient-facing online technologies.

Despite challenges with the user interface and low overall satisfaction rate, we observed that 56% of participants would recommend TL and 69% reported increased interest in clinical trial participation after exposure to TL. These findings suggest that the intervention was still acceptable by the study population. Consistent with this observation, participants appeared strongly engaged with the website with a long average length session, viewing almost all the available content (57 pages: 24 content pages and 33 clinical trial pages) on TL.

The observation that half of study participants discussed clinical trials with a provider after exposure to TL is consistent with observations on the effect of online content made on a national level. According to the Pew Research Center, approximately 59% of US adults use the internet to access health information in the past year,[1,14] of whom about 35% attempt to make conclusions about their medical condition from online information. Importantly, half of adults who use the internet to access health information discuss findings with a clinician. These observations on a population level validate this study's observation that about half of study participants discussed clinical trials with their provider. Given that the current COVID-19 pandemic has led to an increased use of the internet[15] and social media platforms to access health-related information, the effect of this content on clinical encounters may continue to grow.

A major finding in this study is that user satisfaction was informed by the level of complexity and length of the generated clinical trial report. With this observation in mind, we measured the readability of content across the TL using a Flesch Kincaid Grade Level Formula[16] and observed that the home page was a ninth grade reading level and matching tool was sixth grade; however, the clinical trial content was 12th grade. Given that the target reading level for content to be accessible to the general public is at or below a fifth grade level,[17] this demonstrates that user satisfaction with online health information is linked to accessibility of content. Moreover, it also demonstrates that clinical trial descriptions are the largest challenge to translate into more accessible language and this will need to be a key focus for further development.

Although this study is innovative in embedding a mobile matching technology in a clinical setting, it has a few limitations worth noting. The participant sample was primarily English-speaking White, with high health literacy and high socioeconomic status, and therefore, future studies will need to intentionally oversample minority groups to ensure satisfaction and efficacy across populations. Moreover, this study did not measure the impact of the intervention on participant accrual to therapeutic clinical trials. It is also possible that contacting participants electronically about clinical trials influenced clinical trial discussion rates, and future studies will need to control for this effect. Moreover, another significant limitation of this design was that the historical control patient characteristics differed from the study participants. Moreover, the control comparison data also relied on the assumption that physicians consistently document clinical trial discussion. Given that there are likely variable practice patterns around documentation of these discussions in the medical record, this is a major limitation of this study. Future studies will examine the impact of the technology on the end point of clinical trial enrollment in a multisite fashion and with a randomized trial design to address confounding. This study also encountered challenges with participant recruitment after the start of the current COVID-19 pandemic. The participation rate decreased from 92% during in-person recruitment to 21% for virtual recruitment, highlighting the limitations and costs associated with virtual means of recruitment.

In conclusion, this study observed a high rate of clinical trial discussions (48.5% intervention v 28% baseline, P = .007), half of which were patient-initiated discussions regarding clinical trials after receiving access to TL. We also observed an increased interest in clinical trial participation among study participants; however, the satisfaction rate was 38% because of user interface challenges and language accessibility. Future studies will adapt the user interface, simplify language, and evaluate the impact of TL on clinical trial accruals across more diverse populations.