The impact of central IRB's on informed consent readability and trial adherence in SPRINT.

BACKGROUND: Federal agencies have encouraged the use of central institutional review boards (CIRBs) for multi-site clinical trials. There is limited evidence supporting the use of CIRBs. Our aim is to evaluate how SPRINT sites regulated by CIRBs performed regarding informed consent readability and participant trial adherence compared to those regulated by local IRBs.
METHODS: We conducted a cohort study using the SPRINT clinical trial. We collected the IRB of record from the stamped and approved 2012 informed consents from each of the sites. We defined CIRB as an IRB for more than one SPRINT site. Our outcomes were informed consent readability measured using the Flesch-Kincaid readability scale and trial adherence defined as a loss to follow-up, consent withdrawal, and missed last 3-month visit.
RESULTS: Sixty-one percent of all SPRINT sites used a CIRB as their IRB of record. The adjusted mean grade reading level for CIRB consents was 13.4 (95% CI 12.6-13.8) compared to 12.3 (95% CI 12.1-13.1) for non CIRB consents (p = 0.07). CIRB sites had similar rates of withdrawal of consent and loss to follow-up as non-CIRB sites; subjects missing the last appointment of the study were more likely to come from sites regulated by a CIRB. The Veterans Affairs CIRB had the lowest rate of withdrawal of consent (1.9%) and the lowest rate of missed appointments (1.9%) among CIRBs.
CONCLUSIONS: Niether CIRB-regulated sites nor IRB regulated sites enforce the recommended readability level of the informed consent documents. Sites regulated by both IRBs had similar participant trial adherence.

Introduction

Institutional review boards (IRBs) play a critical role in reviewing and approving research protocols, including evaluating the potential risk/benefit of research involving research participants [1]. At the same time, the process of conducting a multi-site clinical research trial is complex and often delayed by each site's IRBs [[2], [3], [4]].

Efforts to streamline an efficient and effective oversight system for multi-site clinical trials have led to development and mandate of centralized reviews [5,6]. July 2011, the Department of Health and Human Services proposed a change to the Common Rule requiring centralized review for multicenter clinical trials. Both the Department of Veterans Affairs and the National Institutes of Health now require centralized review for funded multicenter trials.

There is some evidence supporting the effectiveness of central IRBs (CIRBs) when reviewing multicenter clinical trials [7]. A recent systematic review of 8 empirical studies found that CIRBs improved the review process by reducing variability in review standards, reducing time to approval of initial reviews (by 33 days), and reducing investigator and staff hourly effort (by 6.1 h) [8]. A recent survey of investigators using the National Cancer Institute CIRB showed a shorter time to approval of major amendments (by 48 h) and greater investigator satisfaction with the process [7].

An important task of local and CIRBs is the assurance of readability of informed consent and participant consent comprehension [9]. Both tasks are related, since research subjects are less likely to read long forms and long consents can impair comprehension [10]. In the past, IRBs have failed to assure appropriate reading standards for clinical trials [11], and poor comprehension of informed consents has been reported [12], particularly in limited literacy populations [9,13].

The Systolic Blood Pressure Intervention Trial (SPRINT) was a multisite randomized controlled trial that was conducted at 102 clinical sites [14]. Because several IRBs had oversight over the SPRINT trial there is an opportunity to explore how they performed. Therefore, the objective of our study is to evaluate the overall effectiveness of CIRBs compared to the single local IRB review process, with informed consent readability and trial adherence within the SPRINT trial as measures of effectiveness.

Methods

Study design and setting

We conducted a prospective study using the SPRINT trial cohort. The SPRINT design, methods and principal results have been detailed elsewhere [14]. Briefly, SPRINT is a randomized, controlled, open-label trial that was conducted at 102 clinical sites, organized into 5 clinical center networks in the United States [14]. SPRINT aimed to test whether reducing systolic blood pressure to a goal of <120 mmHg would reduce cardiovascular disease (CVD) events relative to a goal of <140 mmHg. Participants were required to meet all the following criteria: an age of at least 50 years, a systolic blood pressure of 130–180 mmHg, and an increased risk of CVD. An increased risk was defined as the presence of one or more of the following: clinical or subclinical CVD; chronic kidney disease; a 10-year risk of CVD ≥15% estimated by the Framingham risk score; or an age ≥75 years. Patients with type 2 diabetes mellitus or prior stroke were excluded. The SPRINT intervention was stopped early (median 3.26 years of follow-up) because of a 25% reduction in the primary composite CVD end point and a 27% reduction in all-cause mortality in the intensive treatment group [15].

Definition of central institutional review board

We requested the 2012 english version approved and stamped informed consents from every active SPRINT site and collected the IRB of record from the stamped documents. We defined CIRB as an IRB regulating the SPRINT protocol for more than one SPRINT clinical site [4]. Those IRBs that had only one SPRINT site were labeled as non-CIRB sites. We grouped CIRBs into Veterans Health Administration (VA), university based, or commercial CIRBs.

Outcomes

Our primary outcome is informed consent grade level readability. We measured readability using the Flesch-Kincaid readability scale, as implemented in Microsoft Word, which has been validated and demonstrated to be reliable [11]. The Flesch-Kincaid scale assesses readability on the basis of the average number of syllables per word and the average number of words per sentence. We also measured the number of words, characters, paragraphs, sentences and the use of passive sentences in determining readability. Our secondary outcomes are a deviation from the original consent language and participant trial adherence to the SPRINT procedures. We defined a deviation from the original consent the difference in readability between the original consent and the approved consent by type of IRB.

We defined trial adherence using the loss to follow-up, consent withdrawal, and missing the last quarterly visit. We defined loss to follow-up as a subject who could not be reached by the end of the closeout visit. Additionally, we defined consent withdrawal as a research subject who did not wish to continue to participate in the study at any moment in the study. We defined missing last quarterly visit as a research subject who missed the last 3-month visit before the closeout visit as this was an important milestone in the study. These measures have been found to be surrogate metrics of informed consent comprehension, primarily in low-literacy populations, as research subjects who do not understand the study where they are enrolled are more likely to leave the study and be less adherent. These measures have also been established as a rationale for the use of ongoing consent [13,16,17].

Other variables

A detailed description of the SPRINT data collection methods has been provided elsewhere [14]. In addition to the measures collected about the consent document, we collected SPRINT participant level variables (gender, age, race/ethnicity, educational level) and site-level variables (number of participants randomized).

Statistical analysis

We report relevant baseline characteristics according to CIRB status. We summarized the data as frequencies and proportions for categorical variables and as means and standard deviations for continuous variables. We used chi-square tests for categorical variables and we used t-tests for continuous variables to compare baseline characteristics of CIRB and non CIRB sites. To compare the primary (readability) outcome between CIRB and non-CIRB sites, we used the Mann-Whitney test as the data were not normally distributed. To compare the secondary (trial adherence) outcomes we used chi-square.

We used generalized linear models to calculate adjusted means and the corresponding 95% confidence interval (CI) of the primary outcome and logistic regression to calculate the odds ratio (OR) and the corresponding 95% CI of the secondary outcome accounting for participant gender, age, minority status, education level, and number of participants enrolled per site. For the logistic regression multivariate model, we created a dummy variable for all types of CIRB. For all models we used non-CIRB as the reference unless comparisons were made between CIRB sites. All analyses were conducted using Stata 14.0 and all were two-tailed significance levels at p value of <0.05.

Results

Baseline characteristics

Table 1 shows the baseline characteristics of the CIRB and non-CIRB regulated sites. Ninety-seven out of 102 SPRINT sites participated in our study. Sixty-one percent of all SPRINT sites used a CIRB as their IRB of record. Compared to non-CIRB regulated sites, CIRB sites were less likely to be university sites (34% vs. 73%) and more likely to be VA sites (46% vs. 5%). The mean ± standard deviation number of participants recruited at CIRB-regulated sites was 84 ± 55 compared with 116 ± 55 at non-CIRB sites, p value < 0.01. Age and ethnicity of participants from CIRB sites were similar to non-CIRB sites (p value > 0.05) but participants at CIRB sites were more likely to be male and have a lower education (p value < 0.05).

Table 1

Baseline characteristics by CIRB site.

Characteristic	CIRB sites	Non-CIRB sites	p-value
Number	59	38
Total number of randomized participants	4918	4438
University site, %	34	73	<0.01
Veterans Affairs site, %	46	5	<0.01
Mean number of participants per site (sd)	84.7 ± 55.7	116.7 ± 55.5	<0.01
Mean age in years (sd)	68.8 ± 4.5	68.9 ± 3.6	0.84
Female, %	28	40	<0.01
Hispanic, %	14	6	0.10
African-American, %	27	31	0.44
High school education or less, %	30	23	0.01

CIRB: Central institutional review board.

Impact of CIRBs on grade level informed consent readability

In general, the informed consent documents of CIRB-regulated sites had more words and fewer paragraphs and sentences, although none of these differences were statistically significant (Table 2). The mean Flesch-Kincaid readability grade level of all the approved SPRINT informed consents was at 13.0 ± 2.7. The adjusted mean grade level for CIRB consents was 13.4 (95% CI 12.6–13.8) compared to 12.3 (95% CI 12.1–13.1) for non CIRB consents (p value = 0.07).

Table 2

Comparison of readability and adherence outcomes between CIRB and non-CIRB sites.

Mean outcome	CIRB sites	Non-CIRB sites	p-value
Number of words	6088.5 ± 4123.7	5888.7 ± 1418	0.77
Number of characters	28815 ± 6418	30016 ± 7199	0.39
Number of paragraphs	269.4 ± 121.3	294.7 ± 153.9	0.36
Number of sentences	239.7 ± 47.8	261.0 ± 61.5	0.05
Number of passive sentences	32.0 ± 6.8	32.4 ± 8.7	0.76
Flesch–Kincaid readability grade	13.4 ± 2.8	12.3 ± 2.5	0.07

Impact of CIRB on participant trial adherence

CIRB sites had a 2.9% withdrawal of consent rate compared to 3.4% in non-CIRB sites (p value = 0.16). There were similar lost to follow-ups between both types of sites. However, subjects missing the last non-closeout appointment of the study were more likely to come from sites regulated by a CIRB (Fig. 1) (OR 1.36; 95% CI 1.10–1.57 p value < 0.01) compared to non-CIRB sites.

Fig. 1

Participant trial adherence by CIRB site.

Impact of the type of CIRB on outcomes

Table 3 shows the characteristics and adjusted impact of the type of IRB on both primary and secondary outcomes. The VA had the most CIRB sites (n = 25) and only used one consent document for all sites. The university sites (n = 22) used 3 different consent documents for all sites and commercial CIRBs included 12 sites and only used one consent. Non-CIRB sites (n = 38) had the lowest adjusted mean informed consent grade level (12.2; 95% CI 11.5–12.6). VA and University sites regulated by CIRBs had similar informed consent readability grade levels, and those sites had lower adjusted mean readability grade levels than commercial CIRBs (14.8; 95% CI 14.3–15.3) (p value < 0.01). The VA CIRB had the lowest rate of withdrawal of consent (1.9%) (OR 0.32; 95% CI 0.11–0.89 p valew <0.01) and the lowest rate of missing the last appointment (1.9%) (1.87; 95%CI 0.70–5.00 p value = 0.32) compared to commercial or university CIRBs (Fig. 2).

Table 3

Adjusted * primary and secondary outcomes by type of CIRB.

Outcome	VA CIRB	University CIRB	Commercial CIRB	Non-CIRB
Number of sites	25	22	12	38
Number of consent documents	1	3	1	38
Adjusted mean Flesch–Kincaid readability grade	13.4 (12.9–13.8) ‡	13.4 (12.4–13.9)	14.8 (14.3–15.3) ‡	12.2 (11.5–12.6)
OR of consent withdrawal	0.32 (0.11–0.89)	1.22 (0.37–3.99)	1.69 (0.38–7.39)	Reference
OR of lost to follow-up	0.50 (0.18–1.40)	0.83 (0.24–2.79)	1.92 (0.35–10.36)
OR of missed last appointment	1.87 (0.70–5.00)	2.14 (0.68–6.70)	2.5 (0.63–9.82)

*Adjusted for participant gender, age, minority status, education level, and number of participants OR: odds ratio.

‡p < 0.05 when compared to non-CIRB sites.

Fig. 2

Participant trial adherence by type of IRB.

Changes to the original consent

The original consent provided by the SPRINT coordinating center had a mean grade level of 10.9. CIRBs increased the reading level by 2.5 grades, and non-CIRBs increased it by 1.4 grades. Commercial CIRBs increased the grade level by 4.5 grades, while VA CIRBs increased the grade level by 2.1.

Discussion

We found that CIRB and non-CIRB sites performed similarly with regards to the reading level of the informed consent, but both fell far short of the widely recommended 8th grade reading level. CIRB and non-CIRB sites had similar rates of withdrawal of consent and loss to follow-up, but sites regulated by CIRBs had a higher rate of patient failure to attend the last scheduled study visit. We also found that the VA CIRB had the lowest reading level and consent withdrawal of all CIRBs.

Our conclusions have to be weighed against several limitations. First, our definition of CIRB is not the accepted definition, which is a single IRB for the entire clinical trial. However, by having several IRB types, we are able to compare outcomes among them. Second, to assess informed consent readability, we used the Flesch-Kincaid scoring method. It is widely used due to its availability and integration within Microsoft Word. Third, we did not measure informed consent comprehension and relied on surrogates of consent comprehension in the form of adherence metrics. However, for these surrogates it has been shown that among people who reported to be well informed at initial consent, 39% of participants during the follow-up of the clinical trial were “not at all” informed, and 71% wanted more information [16]. Fourth, while we compared the consent readability between the different CIRBs, these results could be biased because the VA CIRB and commercial CIRB had a single consent compared to the multiple consents utilized by university CIRB sites. Fifth, we only reported missing the last 3-month visit instead of other missed visit as a measure of trial adherence.

There is currently little evidence regarding the effectiveness of CIRBs. Two studies have reported on CIRBs. The first, by Check et al. was a systematic review of the use of CIRBs for multicenter clinical trials in the United States [8]. They found that overall CIRB affiliation was associated with faster review and fewer hours of research and IRB staff effort. Additionally, using a net cost analysis, there was a research and IRB staff savings, as well as money being saved on a societal level defined as the difference of multiple IRBs reviewing a study and the costs of running the CIRB. However, there are barriers to its implementation that stem from institutional stakeholders’ opinions about CIRBs. Most institutions reported that they had never used a CIRB and that there was little interest in doing so due to concerns regarding institutional liability and loss of community representation in the review process. For this reason, the overall consensus amongst respondents was a lack of desire for CIRB adoption and an emphasis on using local IRB review processes instead. The second article is more recent and not included in the prior review. In this article, Massett et al. 8 reports the 16 year experience of the National Cancer Institute single IRB, and by conducting process evaluation and surveys, they found that there was a shorter approval time for major amendments and greater investigator satisfaction with the process.

Our study has several implications. First, neither CIRBs nor IRBs in the current study achieved readability standards of informed consents, which has been previously reported [11]. The recommended reading level of consents is an 8th grade level. This difference could be explained by the complexity of the SPRINT trial, leading to long consent forms containing words that are not part of the daily English language, particularly with adverse outcomes of hypertension treatment. Commonly, we found that non-CIRB sites, at the request of the IRBs, adjusted language to increase the consent reading level. IRB revisions of informed consent documents have previously been observed to lead to higher reading levels [10]. It has been suggested that this demonstrates a dilemma in choosing to meet regulatory standards by the IRB as well as meeting participant needs. CIRBs did not improve readability of informed consents; however, VA- and university-regulated CIRBs performed better than commercial CIRBs. The former two types of sites had participants with the lowest level of education, which has implications for comprehension. VA- and university-regulated CIRB sites also had the lower consent withdrawal and loss to follow-up than commercial CIRB sites. This finding could have several potential explanations, such as research participants from the VA and university sites being more receptive to participating in research, participants from those sites having a better understanding of the informed consent and having less frustration with the amount of time required to complete the research, participants being more satisfied with their care and/or experiencing fewer adverse events at those sites, or perhaps the sites having more experienced research coordinators working on-site.

There are many ways we can improve both consent readability and drop-out rates in research. We can assure that consents are written at the recommended grade level by using readiblity scales when drafting consents and when completing review at the IRBs. Also, we can have our community members be more involved in the IRB consent review to assure that it is understandable.

In conclusion, neither CIRBs nor IRBs achieve the recommended readability of informed consents. These findings are relevant to the mandated use of CIRBs by federal agencies and hopefully will prompt efforts to improve performance of CIRBs with respect to this important metric.