When are clinical trials beneficial for study patients and future patients? A factorial vignette-based survey of institutional review board members.

OBJECTIVE: The ethicists believe that the goal of clinical research is to benefit future and not current (trial) patients. Many clinicians believe that the clinical trial enrolment offers best management for their patients. The objective of our study was to identify the situations when a clinical trial is beneficial for the patients enrolled in the trial and future patients.
DESIGN: Factorial vignette-based cross-sectional survey via the internet. PARTICIPANTS: Institutional review board (IRB) members of the US Medical Schools. MAIN OUTCOME MEASURES: Each participant was invited to review 9 clinical vignettes related to (1) study approval and (2) the assessment if the study is designed to help future or current patients more.
RESULTS: A total of 232 IRB members from 42 institutions participated. When considering approval of the trial, we found that uncertainty about treatment effects (OR=1.13; 95% CI 1.08 to 1.19) and requirement for continuation of standard therapy (OR=3.84; 95% CI 2.7 to 5.55) were the only statistically significant factors affecting IRB members' decisions to approve the study. The odds of IRB members approving a trial increased when a trial proposed to enrol patients with life-threatening versus chronic debilitating disease (OR=2.04; 95% CI 1.47 to 2.86). We also found that similar factors affected judgements related to the assessment whether the trial will benefit future or current patients more-(1) future patients: uncertainty (OR=1.27; 95% CI 1.18 to 1.37); continuation of standard treatment (OR=1.66; 95% CI 1.07 to 2.56); seriousness of condition (OR=1.78; 95% CI 1.15 to 2.28); (2) current patients: uncertainty (OR=1.54; 95% CI 1.4 to 1.7); continuation of standard therapy (OR=2.17; 95% CI 1.39 to 3.44).
CONCLUSIONS: IRB members view the proposed studies to be beneficial for current patients and future patients if there is uncertainty regarding treatment effects and if studies do not require stopping the current treatment. This finding supports the design and use of pragmatic trials which may reduce therapeutic misconception and improve trial enrolment, speeding up therapeutic discoveries. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This is the first study exploring not only the institutional review board (IRB) member's decision-making process regarding approval of a research study but also the factors that they consider most important for approval of the proposed study because it is deemed beneficial for study and future patients.

We conducted a study to find the common ground between these two diametrically opposite positions by identifying those clinical situations when enrolment into clinical trials serves trial and future patients equally well.

We pilot tested the vignettes among IRB members and conducted this factorial survey over the web.

A limitation of our study is that our target population did not include the members of commercial IRBs.

We did not investigate the impact of inclusion versus exclusion of vulnerable populations in trials on IRB members' approval in the current study as the number of vignettes would have (due to the factorial design) become unmanageable.

Introduction

Key ethical research documents, such as Belmont report and Declaration of Helsinki, state that clinical research is performed with the purpose of benefiting future patients and not study patients.1 However, healthcare professionals are duty bound not to subjugate their duties to patients' best interest to the utilitarian goals for the good of others.1 From this perspective, enrolment into clinical studies is justified only if it benefits study patients more than those outside of the trials.2 As a result, many clinicians believe that enrolling patients in well-designed studies benefit patients more than treating them outside of the research protocols. For example, the influential National Cancer Center Network (NCCN) states that ‘NCCN believes that the best management of any cancer patient is in a clinical trial’.3 This disagreement about the nature of clinical research among professionals has led to therapeutic misconception (TM) among study participants. TM is a state that ‘exists when individuals do not understand that the defining purpose of clinical research is to produce generalizable knowledge regardless of whether the subjects enrolled in the trial may potentially benefit from the intervention under study’.4

However, clinical research and clinical practice are not mutually exclusive categories, and the mixing of boundaries between research and practice further fuels TM. Researchers have identified key factors related to blurring of boundaries between treatment and research leading to TM.4 Which of these factors influences decisions regarding categorising a proposed research as helping patients in the trial proposed versus future patients is, however, not known.5 6 We hypothesise that it is possible to identify the situations where the interests of trial patients and future patients are aligned. In turn, this can help avoid TM and provide better ethical and scientific framework to foster faster and more reliable medical discoveries.

Methods

The target sample for this study consisted of active university institutional review board (IRB) members in the USA and was identified based on multiple strategies member's lists from the Association of American Medical Colleges and the Public Responsibility in Medicine and Research. We conducted a factorial vignette-based cross-sectional web-based survey in which clinical uncertainties and other factors were incorporated into several hypothetical clinical research studies.7 We used a factorial design whereby seven aspects of each scenario were randomly varied in 15 phrases (see online supplementary Table S1) to produce unique vignettes. We excluded the factors that were not relevant to a particular study design from vignettes for that specific study design. For example, phase I vignette did not include language regarding scientific purpose factors (explanatory trials vs pragmatic trials). Each participant reviewed a total of nine vignettes. That is, each participant reviewed four pairs of vignettes, with each pair representing one of the four types of clinical studies (phase I, phase II, randomised controlled trial (RCT) and a cohort study) and one vignette randomly selected, representing one of the four study designs. At the end of each vignette, the participants responded to the following three questions based on a 7-point Likert-type scale: (1) will the proposed study generate knowledge about medical treatment, benefitting future patients (ie, those not enrolled in the proposed trial); (2) will the proposed study help improve outcomes in the patients enrolled in the trial (‘current’ patients); and (3) will you approve the proposed study?

We pilot tested our vignettes (including language used for instructions) among IRB members before implementing the survey. Moreover, each vignette had the instructions clearly stated. It was specifically stated that ‘In your review of this vignette, please assume that the design and methods of the study are scientifically sound and appropriate, even though they are not described in detail. Do not focus on providing criticism of information that is lacking (i.e., sample size, inclusion criteria)’ (see online supplementary Table S1). Participants received a $5 gift card sent with advance letters. The study was approved by the University of South Florida IRB (No: 107911).

Statistical analyses

We evaluated the impact of the seven factors (see online supplementary Table S2) on IRB members' approval of trial and their perceived benefit for current and future patients, using a multilevel mixed-effects logistic regression with QR decomposition. The proportion of the total variation in outcome explained by clustering is expressed, using the intraclass correlation coefficient. All continuous variables were centred on the mean. Subgroup analyses according to the study design (for phases I–III, phase III RCT only and observational study only) were performed. Summary data are reported as OR along with 95% CIs. All analyses were performed using the Stata (V.13.1) software (STATA [program]. College Station, Texas, 2013).

Results

A total of 232 IRB members from 42 institutions with at least one person completing the survey participated in our study (table 1).

Table 1

Participant demographics

Baseline variables	Frequency (%) or median (range)
Gender (n=232)
Male	98 (42.2)
Female	134 (57.8)
Age (n=148)	57 (30–90)
Years on IRB (n=148)	6 (1–32)
Academic rank (n=148)
Instructor	5 (3.4)
Assistant Professor/Member	25 (16.9)
Associate Professor/Member	37 (25)
Professor/Senior Member	33 (22.3)
Professor Emeritus	4 (2.7)
NA/other	44 (29.6)
Highest level of education (n=148)
Bachelors/Masters*	46 (31.1)
Doctorate (PhD, PharmD, EdD)	61 (41.2)
Medical Doctor (MD)†	41 (27.7)
Consider yourself as scientist (n=148)
Yes	105 (70.9)
No	43 (29.1)
Type of review board (n=148)
Behavioural studies	28 (8.9)
Medical studies	120 (81.1)
Role in IRB‡
Board Chair	16
Committee Chair	6
Research compliance	4
Reviewer	118
Privacy officer/Attorney	2
Community representative	8
Content expert	17
Alternate member	10
Other	12

*Includes one member with an Associate's degree.

†Includes one member with a Doctor of Dental Surgery (DDS).

‡Variable with overlapping categories.

IRB, institutional review board.

Factors associated with the approval of a trial

When considering the approval of a trial, we found that uncertainty about treatment effect (OR=1.13; 95% CI 1.08 to 1.19) and no requirement for discontinuing standard therapy (OR=3.84; 95% CI 2.7 to 5.55) were the only statistically significant factors associated with IRB members' decisions to approve the study. That is, approval of the study was more likely with higher uncertainty. In addition, the odds of IRB members approving a trial increased when a trial proposed to enrol patients with life-threatening versus chronic debilitating disease (pancreatic cancer rather than rheumatoid arthritis (OR=2.04; 95% CI 1.47 to 2.86)). These findings were consistent across combined sets of phase I–III studies versus phase III RCT only. For observational studies alone, no requirement for stopping standard therapy (OR=5.26; 95% CI 1.28 to 20) and condition/disease severity (OR=4.54; 95% CI 1.14 to 20) were associated with trial approval.

Factors associated with benefit of current patients

Uncertainty about treatment effect (OR=1.54; 95% CI 1.4 to 1.7) and no requirement for stoppage of standard therapy, that is, continuation of the existing treatment (OR=2.17; 95% CI 1.39 to 3.44) were the only statistically associated factors affecting IRB members' judgement whether the proposed trials would benefit current patients (table 2). These findings were derived from the analysis of combined sets of phase I–III studies. For phase III RCTs only and observational study only, none of the variables were statistically significantly associated with IRB members’ judgements whether the proposed trial would benefit current patients.

Table 2

Factors influencing approval of a trial and IRB members’ judgements whether the proposed trial will benefit current versus future patients

Variable	OR (95% CI): trial approval	OR (95% CI): benefit of current patients	OR (95% CI): benefit of future patients
Competing interest of a clinician as investigator vs caregiver	1.1 (0.79 to 1.52)	1.08 (0.69 to 1.67)	1.14 (0.74 to 1.74)
No requirement for discontinuing of standard therapy (adherence to standard treatment)	3.84 (2.7 to 5.55)**	2.17 (1.39 to 3.44)**	1.66 (1.07 to 2.56)*
Testing and procedures that are intended to generate scientific knowledge vs necessary for patient care	1.06 (0.77 to 1.49)	0.86 (0.56 to 1.35)	1.04 (0.67 to 1.58)
Purpose of the trial (explanatory vs pragmatic)	1.18 (0.85 to 1.63)	1.43 (0.91 to 2.24)	1.16 (0.75 to 1.78)
Therapy type (gene therapy vs chemotherapy vs over counter medications)	1.16 (0.97 to 1.41)	1.02 (0.8 to 1.31)	1.04 (0.82 to 1.34)
Severity of condition/disease (life-threatening vs non-life-threatening)	2.04 (1.47 to 2.86)**	1.4 (0.9 to 2.17)	1.78 (1.15 to 2.28)**
Uncertainty about treatment effects (varied from 1% to 100%)	1.13 (1.08 to 1.19)**	1.54 (1.4 to 1.7)**	1.27 (1.18 to 1.37)**

For binary outcome: *significant at 0.05; **significant at 0.001; ORs are baseline adjusted for gender, age, review board status, education level, years on IRB, self-reporting as a scientist and quiz score.

IRB, institutional review board.

Factors associated with benefit of future patients

Uncertainty about treatment effects (OR=1.27; 95% CI 1.18 to 1.37), condition/disease severity (OR=1.78; 95% CI 1.15 to 2.28) and no requirement for discontinuing standard therapy (OR=1.66; 95% CI 1.07 to 2.56) were statistically associated with trial, benefitting future patients (table 2). These findings emerged from the analysis of combined sets of phase I–III studies. For phase III RCTs only , none of the variables were statistically significantly associated with members, suggesting that the proposed trial would benefit current patients. For observational studies alone, condition/disease severity was the only factor that affected IRB members' judgements whether the proposed study would benefit future patients (OR=14.28; 95% CI 1.39 to 125).

None of the baseline demographic factors (table 1) were significantly associated with the IRB members' approval of the trial, or whether they judged the study to benefit future or current patients.

Discussion

This is the first study exploring not only the IRB member's decision-making process regarding approval of a research study but also the factors that they consider most important for approval of the proposed study because it is deemed beneficial for study and future patients. We found that IRB members consider two factors (uncertainty about treatment effects and continuation of standard therapy) most important for approval of the proposed trial, which, if adhered to, can benefit future and study patients.

The finding that uncertainty about treatment effect is an important consideration for the trial approval is in line with the established normative precept that research is conducted to address existing uncertainties8 (figure 1). Our study also showed that IRB members are 74% more likely to approve a trial which allows the enrolled participants to continue use of standard therapy compared with a trial with the strict protocol adherence, requiring discontinuation of the standard treatment. Thus, IRB members, people who are charged with oversight of clinical research, appear to support the design and use of pragmatic trials as the ethically and scientifically most robust research design. According to our findings, pragmatic trials identify the situations where the interests of trial patients and future patients are aligned; they help avoid the problem of TM and, in turn, speed up therapeutic discoveries. The foundation of a pragmatic trial is the capability to appraise an intervention's effectiveness in real life and achieve maximum external validity, that is, to generalise results to many settings.9–11 Our findings provide additional ethical support to comparative effectiveness research, which typically relies on the use of pragmatic trials.12 13 It is expected from our ethical analysis that the use of pragmatic trials will bridge gaps in understanding between clinicians and researchers and ultimately lead to improved patient outcomes.9 12 13

Figure 1

Box plot of the probability of approving the trial as a function of uncertainty for all study designs combined.

In this study, we focused only on key factors which IRB members may consider while approving the studies. Obviously, there could be several other factors, such as inclusion of vulnerable populations and informed consent process, which might be considered by IRB members while approving a proposed study. We did not investigate the impact of inclusion versus exclusion of vulnerable populations in trials on IRB members' approval in the current study as the number of vignettes would have (due to the factorial design) become unmanageable. Hence, we have focused on the study designs in which majority of the population are recruited and studied.

Conclusions

We conclude that no requirement for discontinuing of standard therapy and considerations of the existing uncertainties about treatment effects are the key design attributes that should be taken into account when proposing new trials. We believe that before approving the trials, IRBs should introduce the policy to better assess the existing uncertainties in a given trial, employing methods, such as survey of experts, publication of the protocols and systematic reviews with a focus on identifying best standard therapy that should be retained in consideration of the proposed trial.14 Our ethical analysis supports the use of pragmatic trials as the mechanism to reconcile the differences between primary intentions of physicians (to make the best treatment available to their patients) and researchers (to test new treatments under ideal conditions to benefit future patients). This, we believe, will reduce ethical misgivings about trial enrolment potentially improving enrolment, a key prerequisite to accelerate the discovery process and help improve patient outcomes.