Using tablet computers compared to interactive voice response to improve subject recruitment in osteoporosis pragmatic clinical trials: feasibility, satisfaction, and sample size.

INTRODUCTION: Pragmatic clinical trials (PCTs) provide large sample sizes and enhanced generalizability to assess therapeutic effectiveness, but efficient patient enrollment procedures are a challenge, especially for community physicians. Advances in technology may improve methods of patient recruitment and screening in PCTs. Our study looked at a tablet computer versus an integrated voice response system (IVRS) for patient recruitment and screening for an osteoporosis PCT in community physician offices.
MATERIALS AND METHODS: We recruited women ≥ 65 years of age from community physician offices to answer screening questions for a hypothetical osteoporosis active comparator PCT using a tablet computer or IVRS. We assessed the feasibility of these technologies for patient recruitment as well as for patient, physician, and office staff satisfaction with the process. We also evaluated the implications of these novel recruitment processes in determining the number of primary care practices and screened patients needed to conduct the proposed trial.
RESULTS: A total of 160 women (80% of those approached) agreed to complete the osteoporosis screening questions in ten family physicians' offices. Women using the tablet computer were able to complete all screening questions consistently and showed a nonsignificant trend towards greater ease of use and willingness to spend more time in their physician's office compared to those using IVRS. Using the proportion of women found to be eligible in this study (almost 20%) and other eligibility scenarios, we determined that between 240 and 670 community physician offices would be needed to recruit ample patients for our hypothetical study.
CONCLUSION: We found good satisfaction and feasibility with a tablet computer interface for the recruitment and screening of patients for a hypothetical osteoporosis PCT in community office settings. In addition, we used this experience to estimate the number of research sites needed for such a study.

Introduction

Pragmatic clinical trials (PCTs) are increasingly being used to assess the comparative effectiveness of therapies in real world settings. PCTs are characterized by broad inclusion criteria, large and diverse patient populations, and easily measurable and clinically-relevant endpoints (eg, mortality).1–3 Effectively engaging busy primary care physicians and their clinical and administrative staff, who typically have heavy clinical commitments and little research experience, is a key challenge in the design and implementation of optimal PCTs.4 Simple and efficient technological solutions that can facilitate PCT subject recruitment have the potential to enhance the feasibility of participation in community-based research by busy physician practices. We evaluated the feasibility and satisfaction of patients, physicians, and office staff with two methods to assess patient eligibility for a hypothetical osteoporosis active comparator PCT, namely a tablet computer and an interactive voice response system (IVRS) delivered via cell phone. We also evaluated the implications of these novel recruitment processes in determining the number of practices required to conduct the proposed trial, assuming a sample size of 10,000 consented participants. Our work, designed to set the stage for the future osteoporosis PCT, is generalizable to many types of future trials in older adults.

Materials and methods

A web-based tablet computer application and an IVRS technology were developed (Court Square Group, Inc, Springfield, MA, USA) to assess patient eligibility for a hypothetical PCT comparing medications used in the treatment of osteoporosis. We developed eligibility and screening questions targeted towards women 65 years of age and older from community primary care physician practices (Table 1). The patient eligibility questions were programmed into both technologies. We pilot tested the study questions and patient satisfaction surveys for comprehension and feasibility in a local university clinic (n = 21 patients) and two local physician offices (n = 26 patients). Additional pilot testing procedures included in-person/in-depth interviews (n = 8) to assess the cognitive burden and usefulness of the screening questions. Based on the result of the pilot testing process, we revised various aspects of the applications such as the font size, voice tone, and wording of questions prior to administering them in the larger sample of community-based physician offices. We obtained approval from the Institutional Review Board to conduct the study.

Table 1

Screening questions for a hypothetical osteoporosis therapy pragmatic clinical trial*

1. Are you 65 years of age or older?

2. Do you have Medicare?

3. Are your medicines covered in part or in full by Medicare?

4. Have you ever been told by a doctor or other health care provider that you have osteoporosis?

5. During the past year, have you taken a prescription medicine for thinning bones or osteoporosis?

6. Since age 50, have you broken a bone?

7. Have you had a DXA or bone mineral density test of your bones in the past 2 years?

Inclusion criteria: age, Medicare status, medications covered by Medicare, osteoporosis diagnosis or broken bone since age 50.

Exclusion criteria: use of osteoporosis medication in the past year.

Note:

These questions were used to determine subject eligibility, which is discussed in Table 3.

Abbreviation: DXA, dual X-ray absorptiometry.

We recruited a convenience sample of ten physician offices to participate in the study through the Alabama Practice Based Research Network (APBRN) and through the American Academy of Family Physicians National Research Network (AAFP NRN). In the first wave of office recruitment, we contacted 40 physician practices in the Birmingham, Alabama metropolitan area via email, phone, and fax, and we invited them to participate in this study. Our research staff conducted an in-person initiation visit with the participating physician office staff to demonstrate the proper use of the tablet computer and IVRS-programmed cell phone. We sent a second wave of recruitment emails and phone calls to over 200 nonlocal offices in the APBRN and AAFP NRN. For the “long-distance” locations that agreed to participate, we mailed all study materials to the study staff and conducted the site initiation procedures by telephone.

Each participating primary care practice received one tablet computer with a wireless Internet connection and one IVRS-programmed cell phone. During physician appointments scheduled over a 5-day period per office, physician office personnel pre-identified women ≥ 65 years and invited them to participate in the study. Brochures describing the study were placed in the office waiting rooms to inform patients about the study. Women were assigned to receive the tablet computer or cell phone based on their day of birth (even days receiving the tablet computer and odd days receiving the cell phone). Following their interaction with either study recruitment device, the office staff provided patients with a questionnaire to assess the feasibility and their satisfaction with each technology. On a 5-point Likert-type scale ranging from “1 – strongly agree” to “5 – strongly disagree,” participants rated six statements on the ease of use of devices, ease of seeing/hearing questions, difficulty of questions, duration of process, willingness to participate in an osteoporosis PCT, and willingness to spend extra time at the physician’s office. For the purposes of the analysis, the response categories were collapsed to “agree or strongly agree,” “neither agree or disagree,” and “disagree or strongly disagree,” and chi-square tests were used to compare responses between the tablet computer and IVRS groups. We also conducted debriefing interviews with the office staff to determine how the overall process affected the office workflow and routine patient care activities. We used a similar 5-point Likert-type scale for a short list of questions related to office workflow and satisfaction.

Osteoporosis PCT eligibility and sample size considerations

We used answers to the screening questions to determine potential eligibility for the hypothetical osteoporosis PCT (Table 1). Eligible patients were defined as women ≥ 65 years with Medicare drug coverage and no reported use of an osteoporosis medication during the past year. Additional eligibility criteria included either a fracture since age 50 or an osteoporosis diagnosis by a health care professional. These expanded criteria were used in our final sample size considerations. All criteria were based on patient self-report.

We used the proportion classified as eligible participants as well as other national data to estimate the number of women who would require screening to conduct the hypothetical osteoporosis PCT based on a study sample size of 10,000 women. Using a 5% sample of administrative claims made with US Medicare in 2009, we determined the number of office visits by physician specialty among women potentially eligible for an osteoporosis PCT. Consistent with the future target population, we restricted the screening size estimates to women who were eligible for Medicare based on age as of January 1, 2009, who had at least 12 consecutive months of full fee-for-services Medicare, and who were not enrolled in a Medicare Advantage plan, or those who had full coverage until the time of their death. Other variables, including the total number of patients needed in such a study (as determined by prior power calculations), the mean number of women ≥ 65 years at each practice, and initial eligibility from our feasibility study offices, were held constant in four varying osteoporosis recruitment scenarios. We varied sample size scenarios by: (1) the duration of recruitment in each community physician office (6 months or 130 working days versus 12 months or 260 working days), and (2) the number of patients meeting enrollment criteria who would ultimately agree to participate in such a study based on an estimated reduction from previous randomized controlled trials of osteoporosis drugs (25% to 35%).5,6 We also determined the number of patients that would be needed in each office and the total number of offices needed to conduct such a study.

We determined the sample size necessary for a three-arm comparative effectiveness study of the osteoporosis medications alendronate and two-dose regimens of zoledronic acid (5 mg yearly for 3 years, and 5 mg once over 3 years) based on a review of the relevant literature. This hypothetical osteoporosis trial sample size was based on the 3-year fracture rates of 12%, 8%, and 9% in the respective study arms.5–7 We incorporated a 14% inflation rate to adjust for losses to follow-up due to death and for patients switching into a Medicare Advantage plan who would therefore be lost to follow-up.

Results

Ten primary care physician offices, contacted through email and phone calls, agreed to participate in this feasibility study (seven from wave 1 and three from wave 2). Of those offices, nine were community physician offices in Alabama and one was located in New Jersey. The physician practices included offices with both solo and multiple practitioners who saw an average of 31 ± 15 patients per day. All participating physicians were family practitioners who accepted Medicare patients. Women aged 65 years and over comprised a range of 8% to 22% of the physicians’ daily appointment schedules.

Of 201 women deemed eligible for the future osteoporosis study by the office staff and who were subsequently offered participation in the feasibility study, a total of 160 women (80%) initiated the study and 139 (69%) completed the screening tool either by phone or tablet computer (Figure 1). Of the 93 women assigned to the tablet computer, all completed the screening questions. Of the 67 women assigned to the IVRS cell phone arm, 46 (69%) completed all screening questions. The women assigned to the tablet computer were slightly older (mean age ± standard deviation: 74.6 ± 7.1) compared to the women who used the IVRS (mean age ± standard deviation: 73.2 ± 6.7) (Table 2). The majority of women (68%) reported that their Medicare plan covered their prescription medications and 84% reported that they had not been on an osteoporosis medication within the past year (85% of those responding on tablet computer versus 80% of those using IVRS). About 35% of patients reported that they had been told by a health care provider that they had osteoporosis and over 20% reported that they had broken a bone since age 50 (18% of those using the tablet computer versus 33% of those using IVRS). Nearly 40% of all participants reported that they had received a bone scan by dual X-ray absorptiometry within the past 2 years, while 15% of participants were not sure.

Figure 1

Flow chart for the assessment of electronic technologies for osteoporosis pragmatic clinical trials.

Note: Study design and patient flow for a feasibility pilot comparing eligibility for a hypothetical pragmatic clinical trial of osteoporosis therapies.

Table 2

Patient distribution and characteristics by technology in hypothetical osteoporosis PCT

	Total	Tablet computer	IVRS
Number of patient participants, n (%)	160 (79.6%)	93 (58.1%)	67 (41.9%)
Patients per office, mean ± SD	16 ± 9.2	9 ± 5.6	7 ± 5.3
Patient age, mean ± SD	74 ± 6.9	75 ± 7.1	73 ± 6.7
Patients who completed screening questions, n (%)	139 (86.9%)	93 (100%)	46 (68.7%)
Patients with Medicare prescription coverage, n (%)	94 (67.6%)	65 (69.9%)	29 (63.0%)
Patients on prescription osteoporosis medication in the past year, n (%)	23 (16.5%)	14 (15.1%)	9 (19.6%)
Patients with a broken bone since age 50, n (%)	32 (23.0%)	17 (18.3%)	15 (32.6%)
Patients with an osteoporosis diagnosis, n (%)	49 (35.3%)	31 (33.3%)	18 (39.1%)
Patients with previous DXAs, n (%)	55 (39.6%)	36 (38.7%)	19 (41.3%)

Abbreviations: PCT, pragmatic clinical trial; IVRS, integrated voice response system; n, number; SD, standard deviation; DXA, dual X-ray absorptiometry.

Compared to women assigned to the IVRS, women assigned to the tablet computer technology were more likely to agree that the questions were easier to understand and answer (Table 3). They were also significantly more likely to express interest in participating in an osteoporosis clinical trial (35% of those assigned to the tablet agreed or strongly agreed versus 20% of those assigned to the IVRS; P = 0.045). They were also more likely to be willing to spend extra time in their physician’s office to enroll in such a study (41% versus 27%); however, this comparison was not statistically significant. Both patient groups reported that the technologies were easy to use and that the process was not unduly time-consuming during their physician visit.

Table 3

Patient satisfaction and feasibility survey comparing a tablet computer with IVRS

	Tablet computer			IVRS
	Strongly agree or agree	Neither agree nor disagree	Strongly disagree or disagree	Strongly agree or agree	Neither agree nor disagree	Strongly disagree or disagree
The iPAD/phone was easy to use	78 (85.7%)	4 (4.4%)	13 (9.9%)	58 (86.6%)	2 (3.0%)	7 (10.4%)
The questions were easy to see/hear	88 (96.7%)	1 (1.1%)	2 (2.2%)	61 (91.0%)	3 (4.5%)	3 (4.5%)
The questions were easy to answer	90 (97.8%)	1 (1.1%)	1 (1.1%)	64 (95.5%)	0 (0%)	3 (4.5%)
The process did not take too long	84 (91.3%)	1 (1.1%)	7 (7.6%)	61 (91.0%)	3 (4.5%)	3 (4.5%)
If eligible, I would be interested in participating in an osteoporosis drug trial*	32 (35.2%)	22 (24.2%)	37 (40.7%)	13 (20.3%)	18 (28.1%)	33 (51.6%)
I would not mind spending extra time at my doctor visit to enroll in the study	37 (41.1%)	17 (18.9%)	36 (40.0%)	17 (26.6%)	20 (31.3%)	27 (42.2%)

Note:

P = 0.045 by Pearson’s chi-square test comparing tablet computer to IVRS for those who answered “strongly agree or agree”, all other comparisons were not significant.

Abbreviation: IVRS, integrated voice response system.

Of the ten physician offices that participated in the study, all were debriefed by the study coordinator and eight provided answers to the short physician survey. The staff of these offices all found that the process was easy to follow and did not interfere significantly with their clinic workflow (Table 4). No sites required additional help or technical support from the study coordinator following the initial study implementation visit.

Table 4

Physician office staff satisfaction comparing a tablet computer (iPAD) with integrated voice response system

	Strongly agree	Agree	Neither agree nor disagree	Disagree	Strongly disagree
The iPAD/phone was easy to distribute and operate	7 (87.5%)	1 (12.5%)	0 (0%)	0 (0%)	0 (0%)
This process did not interfere with the clinic flow	5 (62.5%)	3 (37.5%)	0 (0%)	0 (0%)	0 (0%)
It was hard to decide which patients might be eligible to get the IVRS (cell phone) or iPAD	0 (0%)	0 (0%)	0 (0%)	2 (25%)	6 (75%)
Narrative comments received from office staff:
• “Did not like the cell phone (hard to hear for the elderly population – patients seemed to be getting frustrated); liked the iPAD (seemed easier for this patient population).”
• “Liked the way new technologies can be used to provide medical services.”
• “Preferred the phone over the iPAD due to problems with the connection and the touch requirement of the tablet.”

Abbreviation: IVRS, integrated voice response system.

Sample size considerations for the hypothetical osteoporosis PCT

Of the 139 patients who completed the screening questions, we determined that 76 (55%) would be eligible for the hypothetical osteoporosis PCT based on the basic criteria of age (65 years or older), having Medicare prescription medication coverage (Medicare Part D), and not having been on an osteoporosis medication in the past year (Table 5). Almost 20% (n = 27) of patients who met these basic criteria for the hypothetical osteoporosis PCT also reported having evidence of increased fracture risk, based on a fracture after age 50 or a known diagnosis of osteoporosis.

Table 5

Patients potentially eligible for osteoporosis pragmatic clinical trials stratified by the recruitment technology utilized

	Total (n = 139)	Tablet computer (n = 93)	IVRS (n = 46)
1. Eligible for OP PCT based on basic criteria only*	76 (54.7%)	55 (59.1%)	21 (45.7%)
2. Eligible for OP PCT based on basic criteria and fracture or osteoporosisdiagnosis†	27 (19.4%)	19 (20.4%)	8 (17.4%)

Notes:

Basic enrollment criteria (see Table 1 for questions): age ≥ 65 years, Medicare with medication coverage, no osteoporosis medications in the past year;

basic enrollment criteria + fracture since age 50 years, or an osteoporosis diagnosis by a health care professional.

Abbreviations: IVRS, integrated voice response system; OP, osteoporosis; PCT, pragmatic clinical trial.

Our analysis of the 2009 Medicare data showed that approximately 72% of women had traditional fee-for-services coverage and were not enrolled in a Medicare Advantage plan. Women seeing both internal medicine and family practice physicians made a median number of three visits during the calendar year, and women seeing physicians with geriatric medicine, endocrinology, or rheumatology specialties made a median number of two visits during the calendar year. The focus for our planned osteoporosis clinical trial would include primary care physicians and internal medicine physicians affiliated with a practice-based research network (PBRN). Our analysis showed that over 40% of noninstitutional physician visits by women with Medicare were made to these primary care specialties with ample numbers of physician visits per year.

Using the rates of available participants found in the feasibility study offices as well as Medicare data on physician specialty, we estimated the number of practices and women requiring screening for the future hypothetical osteoporosis PCT, based on practice and several participant scenarios (Table 6). We found that between 240 and 670 physician offices would be needed to recruit between nine and 30 patients in order to reach the predetermined sample size (n = 10,000), which was needed to show a difference in the 36-month nonvertebral fracture rates between two antiosteoporosis medications.

Table 6

Base case estimations of number of physician practices needed for a pragmatic clinical trial of osteoporosis treatments based on varying assumptions* for the osteoporosis feasibility study

	Recruitment scenarios assumptions
Appointments per day per practice (mean number)†	30
Women aged ≥ 65 years (%)†	14
Women with traditional Medicare coverage (%)‡	73
Rate of refusal (%)†	20
Initial eligibility rate (%)†	20
Estimated number of patients needed for study§	10,000
	Scenario 1	Scenario 2	Scenario 3	Scenario 4
Recruitment period (days)	130	130	260	260
Proportion of subjects that are eligible and agree to participate	25%	35%	25%	35%
Number of patients per physician practice who enroll in study	15	30	30	60
Number of physician practices needed for hypothetical study of 10,000 subjects	670	479	335	240

Notes:

Assumptions were varied based on the four scenarios listed in the table;

from data collected in the current Feasibility Pilot Study;

based on 2009 Medicare claims data: data from a 5% sample of Medicare data shows that in July 2009, 72.5% of women age 65 and over were not enrolled in a Medicare Advantage plan;

from preliminary power calculations of a study with three arms comparing an oral bisphosphonate with two dose regimens of intravenous bisphosphonate.

Discussion

Clinical trials can be very time consuming for patients and physicians, prohibiting some studies from being conducted in many primary care practices. As the US moves towards more comparative effectiveness investigations, it will be necessary to enroll large numbers of patients from primary care practices into PCTs. Improved technological infrastructure, which is effective and feasible for both patients and community physician office staff, may be able to ameliorate at least one barrier for busy clinicians. Improved technological infrastructure will also assist in achieving the needed level of patient recruitment for such large studies. In this feasibility study, we found that technologic innovations, particularly the use of a tablet computer, was highly accepted by older patients and proved to be an efficient and effective tool for the office staff and physicians in community based practices that are part of a PBRN. We also included more distant sites in our study in order to assess the future feasibility of using remote study initiation procedures.

We could find no published reports examining the comparative use of tablet computers or IVRS specifically to facilitate clinical trial research in community physician practice settings. Electronic questionnaires and IVRS have been used effectively to collect data for risk factor assessment and health interventions.8,9 Similar computer-assisted data collection tools, specifically audio–computer assisted “Talking Touchscreens,” have been shown to be effective among lower income, lower literacy women,10 and useful for health status and health literacy assessments in underserved populations.11 Primary care practices that have implemented a computer-based touch-screen tablet approach to patient data collection as part of routine patient care found that although some vulnerable subpopulations had difficulty with the data collection tool, the majority of primary care patients (84%) reported no difficulty.12 We found a similar rate of satisfaction (78%) and showed a preference for data collection using the web-based tablet computer among women over the age of 65 years.

The proportion of women who were 65 years and older (14%) among all patients visiting family physician offices in our study was consistent with a 2008 survey of 29,083 American Academy of Family Physicians showing that patients with Medicare as their principal health insurance coverage make up 22% of family physician practices.13 In addition, in a sample of 56 sites from the Distributed Ambulatory Research in Therapeutics Network, a national collaboration of PBRNs, women over the age of 65 years make up 9.4% of the patient populations of these family physician practices. Thus, our study sample appeared to be representative since the age and gender makeup of our population seem to be consistent with those from the national data sources. Although it is well known that men are also afflicted with osteoporosis, our study only focused on women due to the much higher likelihood that women with osteoporosis could be identified in our community physician practices.

Osteoporosis therapies have been proven in traditional placebo controlled randomized clinical trials to significantly lower fracture risk.5–7,14–22 There is little active comparator data by which to make treatment decisions and there has never been an adequately powered comparator trial with a fracture endpoint providing the basis for this decision making. Thus, very little is known about the comparative effectiveness of osteoporosis drugs due to the cost and limited support available to perform such challenging research. The increasing prevalence and cost burden of osteoporosis and the severe morbidity associated with osteoporotic fractures among the elderly make this a critically needed area of research. A PCT comparing the effectiveness of osteoporosis therapies conducted through community physicians with enhanced technological methodology and follow-up through Medicare data linkage would provide a cost-effective and highly generalizable way to conduct this important research. Based on our sample size estimates and scenarios for our hypothetical study, an estimated 240 to 670 participating community physician offices would be needed to recruit enough patients to conduct such a PCT. Implementation of novel technological approaches for recruitment and screening of these patients could be an integral part of this process.

In summary, we developed and preliminarily tested methods to simplify clinical trial recruitment for future osteoporosis PCTs. Our study also assessed the sample size considerations to conduct a PCT comparing osteoporosis drugs in the community. Further studies are needed to assess technologies and how they facilitate subject recruitment in this and other areas of PCT implementation.