Avoiding Pitfalls With Implementation of Randomized Controlled Multicenter Trials: Strategies to Achieve Milestones.

You breathe a huge sigh of relief because you just received the official notice of award that your grant was funded by the American Heart Association (AHA), the National Heart, Lung, and Blood Institute (NHLBI), or the Patient‐Centered Outcomes Research Institute (PCORI). Your award will fund a study involving implementation of an intervention with patients at risk for cardiovascular events. Because your project was funded for 5 years and several million dollars, you obviously did an exceptional job designing the study. All you have to do now is implement your proposal. Not so fast!

The funding agency has granted you these funds to operate and complete a successful trial. This funding is not a gift. It is a payment for an expected service. Achieving funding from AHA, National Institutes of Health (NIH), or PCORI is an honor bestowed by one of the most important scientific funding sources in the world, and your study was recommended by experts on the study section review committee. You have an obligation to achieve the highest quality outputs possible in order to meet the expectations of these individuals. Your goal, moreover, should be to exceed these expectations. Ultimate success means timely publication of at least the main results from your study.

What many investigators do not realize is that every randomized controlled multicenter trial will encounter scores of operational problems, obstacles, and missteps that could lead to catastrophic failures for the study. Notice we did not say “might” occur. These problems are inevitable, and the best hope for the successful completion of a trial is to anticipate as many of these problems as possible and have a plan to address each issue long before the study begins.

One of us (BLC) has encountered a wide variety of problems and made plenty of mistakes conducting randomized controlled trials since 1980. Learning from early errors in planning, design and execution of clinical trials was critical to successfully achieving key milestones when our first two studies were funded by NHLBI in 2003. In the 11 years that followed, our research team received over $30 million for 6 RO1s or R18s. The first 4 studies were successfully completed; 2 are ongoing. However, we encountered numerous issues and problems that jeopardized our potential for success. Rapid actions by the research team were able to overcome most of the obstacles that we encountered.

We achieved 93.6% of our recruitment goals for the first 4 grants, a level typically considered to be very good. Understanding that any gap in meeting recruitment goals jeopardizes study power, we carefully examined our operations so that we could achieve our goals in subsequent grants. We achieved 100% of our enrollment goals 6 months ahead of schedule for 2 current NHLBI‐funded grants in 32 medical offices throughout the United States.1, 2 Early enrollment will provide more time for data analysis and timely submission of manuscripts. Our project officers have indicated that it is extremely rare to achieve enrollment targets on time, let alone early. The most important outcome from clinical trials is the publication of the main results. Our main papers from the first 4 grants were in print an average of 15 months (range 11–16 months) after trial completion.3, 4, 5, 6 Rapid publication is not typical as discussed below. These studies had numerous data elements, tools, and surveys that led to an additional 30 published ancillary papers with several more currently submitted or in development. These accomplishments would not have been possible without a talented, dedicated research team and support staff, early planning, and ongoing monitoring of progress towards achieving milestones. This paper highlights the pitfalls we encountered in multicenter clinical trials and the strategies our research team has used to achieve milestones critical to the expectations of NHLBI staff and the public. We will focus on health services research trials in primary care offices and not on drug or therapeutic trials funded by industry.

Research Team Composition

The research team includes the principal investigator (PI), coinvestigators, and support staff. Small trials implemented in a limited number of clinical offices that are geographically close might be conducted by a small core team including the PI, research assistant (or study nurse), project manager, data managers, and coinvestigators. We refer to this internal core group as the Clinical Coordinating Center (CCC). The members of the CCC maintain all the study databases, and a CCC biostatistician investigator conducts all the analyses.3, 4 The more aspects of the trial the PI can control, such as recruitment and data collection, the more likely key milestones will be achieved with complete and accurate data. However, this model is not practical for studies where offices are numerous and in distant locations.

For larger, geographically dispersed multicenter trials, the number of investigators and the size of the core working group must increase. The grant might fund research nurses, pharmacists, or other individuals within the local clinics. The funding agency might require a totally distinct Data Coordinating Center (DCC) for large multicenter trials. The DCC assumes responsibility for managing the data collected locally, monitoring study sites, ensuring data quality, and performing the statistical analyses. The DCC should have highly qualified data managers, information technology support, study monitors to do onsite visits, and biostatisticians. The world‐class Clinical Trials Statistical and Data Management Center affiliated with the University of Iowa's Department of Biostatistics has served as our DCC for our 2 largest trials,2, 5 conducting all the data management, site monitoring, and statistical analyses for these studies.

Large trials have hundreds of issues to address, products to develop, and milestones to achieve. The CCC and DCC must work together as seamlessly as possible. Our CCC and DCC meet twice a month to address the multitude of issues that arise in these trials.2, 7

The PIs for the CCC and DCC must ensure that all staff are highly capable and communicate effectively with external stakeholders such as site personnel and perhaps patients. The PI must maintain a performance‐oriented culture with a relentless pursuit of high performance. A balanced performance approach is critical to accomplish the goals of the study, benefit patients (public health), inspire the individual research team members and satisfy the goals of the funding agency.8

Research Clinic Selection

Staff employed within each of the clinical sites or affiliated research offices perform subject recruitment and data collection for large multicenter trials. Some trials utilize an organized practice‐based research network (PBRN) to engage a large group of research sites, but this approach requires addressing potential barriers to site participation. Clinic personnel might view distant researchers as outsiders who are simply using their site to obtain subjects and collect data. In addition, most research in PBRNs provides little financial support for participating clinical sites.

To minimize these barriers, our team creates collaborations between our CCC and our study offices. Our grant budgets allocate subcontract awards to the individual study sites to provide adequate funding. Table 1 displays typical subcontract funding for a pharmacist intervention to improve blood pressure or reduce cardiovascular risks.1, 2, 5 The major portion of the subcontract pays a study coordinator (SC) already employed either by the clinic office (usually a nurse or medical assistant) or by an affiliated research office to screen and enroll subjects and collect data. Some clinic physicians serve as local PIs and complete their local institutional review board (IRB) submissions. Lead physicians receive $2000 a year. Our studies include offices with clinical pharmacists employed in the office, and many of them manage the IRB applications, as well as provide the intervention. As an example, we provided the pharmacists with ≈$4000 a year for performing the intervention, submitting IRB materials, and other study duties. In rare cases, we have had 2 primary care offices affiliated with the same academic health sciences center. In these instances, we generated a unique subcontract for each office so each lead physician, SC, and office received these funds. These individuals become partners with the CCC and function as part of the extended research team. We have encouraged site personnel to use their data for presentations or publications, and some have taken the lead on ancillary publications (see below).

Table 1

Sample Site Subcontract

Intervention Arm	Year 1	Year 2	Year 3	Year 4	Year 5	Totals
Physician salary and benefits	$2000	$2000	$2000	$1500	$1000	$8500
Study coordinator salary and benefits	$4500	$6000	$5000	$3000	$2000	$20 500
Pharmacist salary and benefits	$4000	$6000	$5000	$3000	$2000	$20 000
Study coordinator patient visits @ $100/visit	$400	$2100	$2100	$400	$0	$5000
Laboratory testing @ $225 per set	$900	$4725	$4725	$900	$0	$11 250
4‐and 8‐month chart audited data @ $25 each	$0	$775	$475	$0	$0	$1250
Subject reimbursements @ $75/visit	$300	$1575	$1575	$300	$0	$3750
24‐month chart audit ($50 each)	$0	$0	$200	$500	$550	$1250
Total direct costs	$12 100	$23 175	$21 075	$9600	$5550	$71 500

One of the most important decisions for any PI is the identification of clinical sites that will be engaged, highly likely to meet recruitment milestones, and able to maintain quality control standards (Table 2). The chances of a good score on an application will be greatly improved if all the sites are confirmed and letters of support are included with the submission. The research team should collect critical site data such as providers, numbers of patients especially from minority groups, and other key statistics that will appear in the resources section of the application.

Table 2

Strategies to Achieve Milestones

Milestones	Strategies
Research Team	Recruit a highly competent and dedicated staff Frequent team meetings (twice monthly) Partner with clinic personnel, promote joint publications, offer clinic personnel use of their data for their own projects
Site Selection	Begin the process far in advance of the application submission Request work products prior to selection to determine how well they perform and communicate Make personal visits to develop relationships especially with the lead physician, SC, and other key personnel Build excess capacity for implementation when sites underperform
IRB Approvals	Begin your IRB submission as soon as funding is likely Work with your central IRB to negotiate reliance agreements with site IRBs Have research staff prepare template IRB materials for use when negotiating reliance agreements Carefully track IRB reliance agreements to identify delays so that steps can be taken to speed the process Register the clinical trial at clinicaltrials.gov or appropriate site.
Training Personnel	Take the training program to the site if possible to build relationships Cover all study policies and procedures in great detail Consider bringing site personnel to regional locations to improve efficiency of training Perform SC training at the PI's institution to improve efficiency, develop relationships, and provide SCs with peer interactions Provide refresher training as often as needed
Subject Recruitment	Assist SCs with obtaining diagnosis or billing records to identify subjects Create screening logs to track the progress of screening at every site Communicate frequently with the SC to determine how many and how often letters to potential subjects are mailed. Create an expected recruitment timeline and track recruitment relentlessly at every site Track SC's work to be sure they are dedicating appropriate time to the study Conduct weekly telephone calls with the SC if any recruitment steps are delayed PI must intervene with site PI and/or lead physician to address and resolve staffing issues Continue to express the need for ethical recruitment of subjects Have the PI's research team take over recruitment activities if appropriate
Subject Visits	Create tables of study visit windows for every subject so that SCs can easily determine when to schedule follow‐up visits Emphasize the need to schedule subjects early in the visit window so subjects can be rescheduled within the window if problems arise
Monitoring Data Quality	All studies should include site monitoring visits to evaluate subject recruitment, data quality, and the completeness of the data Provide remedial training if needed
Publication Timelines	Plan for publication of papers early in the course of the study Publish a methods paper Encourage students, postdoctoral trainees, coinvestigators, and site personnel to contribute to authoring ancillary publications The PI must keep other individuals on task to meet publication timelines and be prepared to take over the writing if authors fail to perform Develop a shell of the main results paper even before the results have been analyzed Have early and clear communication with the data managers and biostatisticians so the main results can be analyzed in a timely fashion Quickly make draft revisions and give coauthors a limited timeline to review and revise If the paper is rejected, make changes rapidly and resubmit to another journal rapidly If the paper is accepted provisionally, revise the paper in a few weeks unless new data analyses are required. Resubmit the paper as soon as possible

IRB indicates institutional review board; PI, principal investigator; SC, study coordinator.

We developed a National Interdisciplinary Primary Care PBRN prior to submitting the grant application for the CAPTION trial (Collaboration Among Pharmacists and physicians To Improve Outcomes Now).5, 7 This PBRN was certified by the Agency for Research and Quality (AHRQ) in 2008. We initially identified 48 primary care offices that all had clinical pharmacists on staff and were interested in participating in the trial.9 We eventually included 32 of the 48 medical offices.

The PI must identify the key decision‐makers in a clinic or medical office who will be the point person at that site. The research team must provide detailed descriptions of the responsibilities, timelines, and payments for the sites so that the clinic personnel clearly understand their obligations. Communication can initially be made by telephone, e‐mail, and/or Skype. Ideally, the PI should make a personal visit to the office to build trust and communication. If this personal visit cannot occur before the study is funded, it should occur early after funding.

Some site personnel will quickly agree to participate in a trial because they are well acquainted with the PI or welcome opportunities to do research. Despite their best intentions, though, most are busy practitioners and faculty with competing priorities. It is dangerous to assume that site personnel will perform well until they provide some evidence of performance. We have often required site leaders to complete surveys or other activities prior to making a final decision on their participation. IRB submission is another demanding step that can provide clues to future performance. It can take a year or 2 to develop strong relationships, become comfortable with site personnel, and collect the needed data. Difficult communication, delays in responding to requests, and failure to complete requested tasks constitute clear warning signs.

Building Excess Capacity

Some site personnel will not perform well, and organizational issues can arise, so it helps to identify alternate sites early. The CAPTION trial was originally designed for 27 medical offices in 15 states. When performance and recruitment lagged at some sites, we added 5 alternate sites late in the 2nd year of the grant funding period. We took advantage of our contacts at national meetings and within a PBRN developed by a national organization to identify and add these new medical offices. Unfortunately, our grant budget had included subcontract costs only for the first 27 sites. We reallocated portions of the funds budgeted for the SC payments from underperforming sites to the new sites, but the amounts that could be reallocated did not cover the fixed costs that had been budgeted for all sites. In addition, the process of identifying new offices was complex because our offices were stratified by the percentage of minorities in the office and the level of clinical pharmacy services at baseline.5 Therefore, new offices that were added had to fit the characteristics required for inclusion in the strata of the underperforming sites. Developing and negotiating new subcontracts and getting approval from local grant offices and IRBs required many months of deliberation and negotiation. Even though we acted quickly, adding new sites significantly challenged our ability to achieve our enrollment timelines.

This experience led us to change strategies in the MEDFOCUS trial (MEDication Focused Outpatient Care for Underutilization of Secondary prevention) for patients at high cardiovascular risk. MEDFOCUS is a cluster, randomized trial powered for 16 primary care offices. However, our original grant proposal and budget incorporated costs for 4 alternate sites that were stratified and randomized just like the 16 main sites.2 We developed IRB materials and subcontracts with these alternate sites as soon as we received the notice that our study was funded. This approach allowed us to quickly bring these alternate sites on board when recruitment obstacles emerged. This strategy, in part, allowed us to meet our enrollment targets 6 months early.

Obtaining IRB Approvals and Clinical Trial Registration

Delays in obtaining local IRB approvals can create a major obstacle to subject recruitment, especially when working with 5 to 30 different IRBs. Although some IRBs have approved our trials in an expedited fashion, others have required extensive review requiring over a year for approval.

Ideally, all IRB approvals would be in place as soon as the study is funded. But the IRBs for many institutions, including our own, refuse to consider a proposal before the project is funded. In addition, the University of Iowa IRB has required approval from at least one local IRB before we can submit a full application, requiring an additional process to speed approval by our own IRB. Historically, we have drafted a template for local IRB applications during the grant review process. We then identify one or more sites immediately after the project is funded where a local PI can quickly submit an application using our drafted materials and obtain approval from the local IRB.

A staff member on our Iowa team can customize the template for site IRB applications to meet local guidelines, relieving some burden on local investigators. Most site IRB reviews require revisions or responses to questions. Our staff can often prepare these revisions and turn these materials around for the local PI more quickly than can site personnel. However, the site PI remains responsible for verifying that the submission is complete. The CCC must also track and verify the status of continuing annual reviews to make sure that approval does not expire and that site SCs have the most recent approved consent materials. We frequently make minor changes to the study protocol or decide on additional survey instruments that require IRB approval. The office staff need to be prepared for these revisions and our CCC staff assist them with the process as much as possible.

IRB applications can still encounter barriers and delays even with our best efforts. Despite full review by a study section and funding by NHLBI, local IRBs in the CAPTION trial raised issues related to study design, study instruments, access to patient records, study procedures, the role of clinic physicians, and problems with modifications.10 These issues took considerable time for our CCC staff and the local PI to address and caused some local PIs to question future participation. The CCC PI must consider the costs associated with IRB problems and provide support staff to relieve the local PIs of as many burdens as possible when preparing the grant budget.

Fortunately, streamlining of IRB processes might ease the burden for both CCC investigators and members of the site team. The University of Iowa IRB is the IRB of record for 12 private practices throughout Iowa in the ICARE study (Improved Cardiovascular Risk Reduction to Enhance Rural Primary Care).1 Additionally, the NIH issued a policy on using a single IRB for multicenter trials on July 7, 2016. This policy, which takes effect on May 25, 2017, is designed to streamline the process, reduce redundant obstacles, and allow research to proceed more expeditiously. This development, though welcome, will probably generate questions and doubts on the part of some IRBs, so CCC staff must anticipate potential obstacles early.

All high quality journals will require that a clinical trial be properly registered. Registration must be completed before recruitment begins. Therefore, an excellent time to register the trial is during the time period when IRB applications are initiated. There are several public trial registry options available, but we have always registered our trials at ClinicalTrials.gov.

Training Site Personnel

Training site personnel is a critically important function that also helps solidify relationships. We attempt to provide training as close to subject recruitment timing as possible. For trials within the state of Iowa, several members of the research team took the training program to the sites and provided lunch during the sessions.1, 4 Onsite training was critical to build relationships but was challenging because some sites were a 5‐hour drive from our center. The most important points to be made during training sessions is the need for ethical treatment of subjects and the necessity to meet timelines, recruitment goals, and data quality. We spend the most time with the onsite SC who will be screening subjects, obtaining informed consent, and collecting data. In many cases, SCs were nurses or medical assistants who had not previously conducted research. Therefore, our training had to be extensive, clear, and very specific.

We held regional training sessions for the lead physician and pharmacist for the CAPTION trial.7 The most important training, for the on‐site clinical pharmacists delivering the intervention in their medical offices, used a toolkit developed from our other trials.3, 4 We requested that these providers perform a “train the trainer” session with other providers in their office within a few months of our training sessions.

The CAPTION and MEDFOCUS trials brought SCs to Iowa City for onsite 1½‐day training sessions. This strategy allowed us to meet the SCs and allowed them to socialize with their peers. We scheduled identical training sessions in 2 different months so SCs could attend one that was convenient for their work and travel schedule. The project manager from the CCC conducted the training on screening, recruitment, informed consent, and study procedures. Members of the DCC extensively reviewed the case report forms and the online data collection procedures for these trials. We have performed rigorous research blood pressure measurements as key primary end points in our studies. The SCs are carefully trained and certified on proper blood pressure measurements and they are recertified on a yearly basis.

We have needed to collect laboratory data to ensure subjects meet our inclusion criteria and to obtain our primary end point in studies evaluating the effect of the intervention on lipids or diabetic control. We utilize the usual laboratories used by each study clinic rather than central or core laboratories. This approach necessitates that we obtain information documenting each laboratory's Clinical Laboratory Improvement Amendments certifications. These certifications are stored by the DCC. It could be necessary to negotiate subcontracts with some laboratories. The SCs are trained to obtain the samples or have the proper technician obtain the samples, and the order is then sent to the laboratory. The study SCs must understand to track the results so that our case report forms can be populated in a timely manner, usually within 48 hours. We do not perform special tests such as cardiovascular imaging or angiography. However, for investigators who require these tests, timely communication and feedback with those core laboratories are essential.

Training must often be repeated for various reasons. For some sites, IRB approval is delayed and the SC needs a refresher before recruitment begins. In other cases, an SC may leave their office and a new SC must be hired. The project manager in the CCC must have very frequent communication with medical office staff to identify potential issues and rapidly provide retraining when necessary. Remote training via teleconference and webinar can be highly successful when retraining session agendas are highly detailed and exacting.

The Data and Safety Monitoring Board

NIH and most funding agencies will require a Data and Safety Monitoring Board (DSMB) to oversee any clinical trial. The PI should review the DSMB guidelines for the funding agency when applying for a grant. The DSMB is responsible for subject safety and data integrity. However, the DSMB may suggest that the study be stopped for ethical reasons such as: (1) the study fails to meet enrollment targets; (2) the intervention is so effective all subjects should be informed; (3) an interim futility analysis suggests no possible effect of the intervention; or (4) concern about subject safety. The DSMB generally meets twice a year with the investigators to review study progress and make recommendations.

Other areas of responsibility for the DSMB is the approval of the study protocol and tracking protocol deviations. The investigators need to have a process in place to identify and report protocol deviations. Investigators must also quickly retrain the SC or other staff as soon as a protocol deviation is identified.

In both the CAPTION and MEDFOCUS studies, the PI and members of the CCC were blinded, and all blind data were presented to the DSMB by the DCC in a closed session. We have used the same DSMB for both studies. The chair is an MD/PhD biostatistician with extensive cardiology trial experience. Other members include an MD cardiologist and a PharmD with NIH‐funded trials of pharmacy interventions. The NHLBI project officer is an ex officio member of the DSMB. The DSMB can be helpful to the investigators by suggesting strategies to improve recruitment or safety, and the investigators and members of the DSMB should be picked carefully.

Subject Recruitment, Tracking, and Communication

The recruitment clock ticks relentlessly as soon as the recruitment date arrives, and tracking recruitment must be planned well in advance of enrollment. The first step is screening subjects. Involving physicians, residents, nurses, or others can help supplement recruitment, but, in our experience, these providers have generally not provided sufficient referrals to meet study goals. Therefore, we do not rely solely on referral from providers. Our approach is to either hire our own staff or to pay a portion of an office staff member's salary to recruit subjects.

The SC that we pay is responsible for identification of subjects, contacting them initially by mail, contacting subjects by telephone, recruiting, and consenting subjects in our trials. Local staff typically run reports on the diagnoses of interest or billing reports to identify potential subjects. The SC then uses these lists to screen electronic medical records to determine whether subjects meet the inclusion and exclusion criteria. CCC staff create and distribute electronic screening logs for the SCs to complete as they screen subjects. We request that all SCs submit their screening logs monthly to ensure that they are actually screening subjects in a timely fashion.

For each potentially eligible subject, the SC prepares a personalized letter of invitation that is stamped by the IRB of record for the site. The letter is typically signed by the site lead physician or SC, explains the study, and provides both a return postcard and a telephone number for the subject to call. Most IRBs will allow an “opt out” approach whereby a postcard in the letter can be sent back if the individual does not want to participate. If a subject does not send the card back declining participation, most IRBs will allow the SC to call the subject in 10 to 14 days. Batches of invitation letters should be sent out every few weeks, so that the SC cycles through screening, sending out letters, and telephoning subjects to set up baseline appointments. The screening log includes information on the number of letters that are mailed and the outcomes of those mailings, providing evidence of an SC's progress and the number of potential study subjects identified.

If a local site has a goal of recruiting 24 subjects in 12 months, then a timeline of expectations should be developed for the site SC to enroll 2 subjects every month. The research team cannot wait even a few months to deal with recruitment issues. CCC staff must carefully track recruitments weekly for each site to identify delays or problems. One effective strategy we have used is to send out weekly enrollment reports to all SCs and site PIs. These graphic reports plot enrollments by the name of the site for all investigators to see, adding a degree of peer pressure for timely recruitments.

In our experience, 10% of site personnel do an excellent job with all of these steps and exceed our timelines. Approximately 40% of offices meet the timelines, 30% fall behind schedule but catch up with remedial efforts, and 20% fail to meet their targets. A plan for excess capacity has helped us overcome recruitment target failures.

We recommend that site personnel devote a consistent amount of time to the study. Although we fund ≈4 to 8 hours a week for the SC, staff shortages can prevent the supervisor from providing sufficient dedicated time for the study. Frequent communication can identify these staffing issues early. Funding for the lead physician provides some leverage when we request they work with the supervisor to resolve staffing problems.

As soon as the project manager determines that timelines for screening, sending letters, scheduling subjects, or enrollments are not being met, rapid remedial action is necessary. Weekly calls with the SC can keep them on task. If this approach is ineffective, we negotiate a conference call with all site research personnel, the project manager, and the PI to identify potential reasons timelines are not being met and develop solutions. Our studies include sites from the Eastern to the Pacific time zones, each site selected for important reasons. We modify our schedule to accommodate the site staff for conference calls when it is convenient for them. If problems persist, scheduling calls with the site team every month or 2 can keep pressure on the site. The PI, however, must appreciate the tension to obtain timely recruitment and the potential for quick, sloppy data collection or consent. The PI and other members of the research team must constantly emphasize the need for accurate data and the ethical processes for subject recruitment. The process for ensuring the ethical treatment of subjects, IRB procedures, and accurate data is discussed below in the site monitoring section.

Other approaches are sometimes needed. Our ICARE trial1 involving 12 private primary care offices throughout Iowa encountered significant staff shortage issues that impaired screening and recruitment at 2 sites within 20 miles of the CCC. Our research team negotiated with clinic personnel to permit CCC student research assistants to screen and recruit subjects. This approach required the students to complete human subjects training and obtain approval to utilize the electronic medical records in these offices. We shifted funds for screening and recruiting from the sites back to our central budget so we could pay our staff. Because we could control the process with dedicated students reporting to the research team, we were able to over‐enroll at these 2 offices to make up for other under‐enrolling sites. While this strategy was very effective, it would not have been possible for sites that are much farther from our location.

Recruiting Women and Minorities

Funding agencies, including AHA and NIH, desire to have subject recruitment that represents the population of the United States including women and minorities. We have never had a problem recruiting women into our health services research studies. In the 5 trials discussed in this paper and funded by NHLBI, the percentages of women were 49%,2 50%,1 56%,3 58%4 and 60%.5 Therefore, we have not needed to implement specific strategies to recruit adequate numbers of women.

The demographics of the state of Iowa have changed dramatically in the past 20 years. Nonetheless, the percentage of minorities that count as underrepresented by NIH is only about 5%. The first strategy to increase minority recruitment is to have the study coordinator screen their potential subject list and try to approach minority subjects first. This strategy only allowed us to recruit 5% to 9% of minorities.1, 3

Another strategy we have used in our cluster randomized trials with multiple medical offices is to recruit clinics that serve larger minority populations in Iowa. In one study, this allowed us to increase overall minority recruitment to 18%.4 The danger with this approach when randomizing by clinic is that imbalances can occur in study arms based on numbers of minorities, but also other sociodemographic factors associated with race and ethnicity. Therefore, large numbers of medical offices are required.

When we designed the CAPTION5 and MEDFOCUS2 trials, we had secondary aims to determine whether the intervention was as effective in minority populations as in nonminorities, so we had to power the study based on numbers of minorities.11 To accomplish these goals required recruiting a large number of medical offices, many of which had very high minority populations to balance those with few minority patients. Therefore, CAPTION included 32 medical offices and MEDFOCUS includes 20 offices throughout the United States. These offices included many located in the southeast United States or were clinics in the northern cities with large African American populations. Several other offices located in Texas served many Hispanic/Latino patients. This strategy allowed us to recruit 54% of minorities in CAPTION and 47% of minorities in MEDFOCUS.

Data Collection Windows

Most clinical trials have set dates when follow‐up data must be collected. In CAPTION, subjects were supposed to return at 6, 12, 18, and 24 months. In ICARE and MEDFOCUS, subjects have 1 follow‐up visit at 12 months. We typically allow a window of 1 month before to 1 month after the visit date to make it convenient and practical for subject scheduling. However, if a subject misses a window, the resulting missing data can be a significant problem. How can we plan to minimize missed visits?

First, we urge SCs to contact subjects before the window opens and schedule them early in the window. Then, if the subject does not show or needs to reschedule, there is time left in the window to accommodate this change. As an example: if we have a 12‐month visit to collect data, the window opens at 11 months and closes at 13 months. The SC is encouraged to contact the subject at 10 months to schedule them as soon as the 11‐month window opens. Then, if a problem arises, nearly 2 months remain to reschedule the visit.

Second, most SCs are far too busy to pay close attention to study windows. Think of the scenario where they have 18 subjects in the trial with windows opening and closing all on different dates. If the SC is still screening and recruiting subjects to fill their recruitment goal, scheduling both baseline and follow‐up visits requires extremely complex timing. Our research team tries to reduce the burden of rescheduling by providing lists when windows are opening and closing every few weeks. These tables allow the SC to see at a glance the subjects who require their immediate attention. Depending on the trial, monitoring completion of follow‐up study visits may be the responsibility of the CCC staff, DCC staff, or both. Effective, frequent communication between the CCC and DCC is essential. Achieving low rates of missing data requires critical attention to detail.

Monitoring Data Quality

The proper ethical recruitment of subjects and the integrity of all study data are absolutely essential. Every effort must be made to ensure that data are accurate and complete. CAPTION and MEDFOCUS required online, secure data entry into the DCC database. Including immediate automatic error messages in an online database is a very good way to quickly inform an SC of a data entry error, and a query system is used to correct inconsistencies between data fields.

The best way to evaluate quality is an onsite visit where the study monitor reviews all signed consent forms and compares study case report forms with the medical record. In addition, study monitors should examine all regulatory documents to be sure the protocol, IRB submissions and approvals, and approved consent forms are readily accessible and maintained in an orderly binder. The CCC and/or DCC should carefully evaluate the quality of the data from all the sites. For smaller studies, members of the CCC must visit the site to perform data monitoring. Members of the DCC will perform these functions for larger trials. It is important to perform the initial monitoring visits shortly after the first few subjects have been enrolled at a given site. Errors and misunderstandings are inevitable, and the monitoring visit can clear these up. Minor errors, including transcription errors, failure to record a data element that had been collected, and incorrectly understanding the intent of a data field, are generally easily remedied, often just by having the SC correct errors during the monitoring visit.

Remedial training might be necessary for more serious errors or in cases where numerous problems exist. Examples might include not following blood pressure measurement protocols or consenting subjects on a previously approved form. Data from these sites needs to be scrutinized carefully to be sure that subsequent data collection improves. Identified problems also might require reporting to the IRB of record.

Terminating the participation of a site might become necessary if quality concerns persist or more serious problems arise. Examples of such problems include multiple lapses in the ethical recruitment of subjects, inability of the study monitor to verify that certain data were collected or that the subject was actually seen, and suspected falsification of data. These serious concerns are fortunately very uncommon.

Meeting Publication Milestones

A recent analysis of 244 randomized clinical trials funded by NHLBI found that only 57% of the main results were published within 30 months after trial completion, and many grants never yielded a publication of the main results.12 These disturbing findings constitute a major concern for the funding agency. As noted above, we published our main results within 15 months of completion of trial funding, which required that plans for publication started early.

One approach to keep publishing high on the priority list is to quickly publish the methods of the study1, 2, 7, 13 and perhaps the baseline characteristics of the population.14 A methods paper should be fairly straightforward to write since much of it appears in the grant application and the study protocol. We have attempted to submit a methods paper in the first year of our trials.

The research team, including coinvestigators, should have periodic brainstorming sessions on potential publications. Most studies generate so much information that it would be hard not to have several publications, including results from theory surveys,15 guideline adherence,16, 17, 18, 19, 20 secondary end points,10, 21, 22, 23, 24 the mechanism of the intervention effect,25, 26, 27, 28, 29 the effect of the intervention on reducing sociodemographic health disparities11 24‐hour blood pressure results,30 adverse events,31 or cost analyses.32, 33 Many papers may be authored by research staff, students, postdoctoral trainees, or collaborating investigators from study clinics. Some ancillary papers may be published prior to the main results, others perhaps years after the study is completed.

By the time the methods paper and several ancillary studies are published, preparing the main paper should be straightforward. The author of the main paper should write much of the introduction, methods, and some discussion even before the data have been analyzed.

The PI needs to keep the research team on task to be sure that once data collection is complete, all data are reviewed and any errors corrected quickly. The data analysts and biostatistician must know well in advance when to expect complete data so they can prioritize the analyses. Since various data analyses can require months to prepare, it is important that analysis proceeds in a timely fashion. The author can then drop the initial results into the paper.

Drafts of the main paper should be quickly reviewed by the authors. It is common that other ideas emerge that require additional data analyses. These revisions and reviews by coauthors must be made quickly and turned around as rapidly as possible with a focus on accuracy of the findings.

High‐quality journals provide rapid review, often within a few weeks. It might be necessary to submit the paper to several journals before the paper is accepted for publication. The main author should address all comments from reviewers, including conducting additional statistical analyses. Rapid turnaround and resubmission is essential for timely publication and should occur in weeks, not months. Once the main results have been published, the entire research team can celebrate.

Conclusions

Research teams must ensure both successful completion of a trial and publication of the results. Meeting critical milestones in multicenter trials requires extensive planning, communication, and training. The best predictors of success are attention to detail, self‐imposed timelines, and a dedicated research team. Funding agencies will continue to require evidence of such planning before funding is released to investigators.

Sources of Funding

The National Heart, Lung, and Blood Institute, R01HL116311, and R18HL116259.

Disclosures

None.