A guide to research partnerships for pragmatic clinical trials.

Demonstration pragmatic clinical trials supported by the Health Care Systems Research Collaboratory of the US National Institutes of Health highlight the following lessons about building strong research partnerships between health researchers and healthcare systems:

• Participating in pragmatic clinical trials can provide healthcare systems with evidence and tools to improve healthcare, and researchers with opportunities to conduct high impact clinical studies

• Pragmatic clinical trials answer questions relevant to healthcare systems, so clinicians, healthcare managers, health information technology (IT) staff, and clinical operations staff need to be involved in the study design

• A successful pragmatic clinical trial starts with a strong partnership between researcher and healthcare system, goes through a rigorous objective evaluation of the ability of the partner healthcare system(s) to participate, and ends with evidence about sustainable ways to improve care, as well as a long term scientific relationship

Pragmatic clinical trials are comparative effectiveness studies conducted in real-world settings to answer questions relevant to patients, clinicians, and healthcare decision makers. In contrast, explanatory clinical trials study how treatments or interventions work in carefully controlled settings and study populations, often to investigate a biological hypothesis or test a drug or device to meet regulatory requirements. The pragmatic approach requires unique methods to achieve the goal of identifying sustainable, generalizable, evidence based ways to improve healthcare.1 2 3 4 5 While previous authors addressed statistical considerations,6 little guidance is available on the methods for establishing partnerships between researchers and the people and processes in usual care settings that are necessary to conduct a pragmatic clinical trial. Based on insights from an initiative to accelerate pragmatic research,7 this article summarizes best practices for researchers and partners in healthcare systems as they establish collaborative relationships, develop research questions, and implement sustainable pragmatic clinical trials.

Pragmatic trial example

A recent example of pragmatic research is the REDUCE MRSA trial by Huang et al, in partnership with Harvard Pilgrim Health Care (analytical center), Rush University (microbiology core), and Hospital Corporation of America (healthcare system).8 Previous studies identified screening, contact precautions, targeted decolonization, and universal decolonization of patients in intensive care units as candidate strategies to prevent healthcare associated infections, particularly by meticillin resistant Staphylococcus aureus (MRSA). However, the most effective usual care strategy was not known. The REDUCE MRSA project randomized more than 40 hospitals to address this practical issue and found that treating the entire intensive care population with daily antiseptic baths and a nasal topical antibiotic was more effective than either targeted decolonization or screening and contact precautions without decolonization.

The project developed because of shared interests among hospital management, the hospital’s infection prevention and control teams, and academic researchers. Because it involved changes to care processes, launching the trial required strong support from senior hospital management, as well as input from staff at each study site who were already designated as quality improvement champions. These champions used their knowledge and experience in implementing hospital quality improvement campaigns to introduce the study protocol and training materials in a way that was familiar and acceptable to frontline clinical staff. No researchers were on site: The study depended on usual quality improvement mechanisms with the aid of twice monthly coaching calls to train and support the local champions, introduce the training materials, and coordinate assessment of protocol compliance.8

Pragmatic trial design and methodological features

A defining characteristic of pragmatic trials is that they are designed to determine what interventions work for patients receiving typical care.9 The supplemental table provides additional examples of pragmatic trials and their implications for clinical practice. Clinical trials are seldom purely explanatory or pragmatic but fall on a continuum.4 None the less, trials that are more pragmatic have particular features. Pragmatic trials are more likely than explanatory trials to study interventions already used in practice than, for example, an experimental intervention such as a new drug versus a placebo. Studies must be designed accordingly:

Rather than randomizing individual patients to different treatments, pragmatic trials may randomize clinics, hospitals, or clusters of facilities to facilitate study implementation

Pragmatic trials often use existing resources such as electronic health records and population characteristics from censuses for study design, efficient participant recruitment, intervention implementation, and data collection.4 Using electronic health records requires careful attention to missing data

Pragmatic trials often measure factors with practical value for healthcare systems such as costs, the reach and sustainability of interventions, and variables that affect implementation.5 10

These features of pragmatic trials require statistical consultation to ensure appropriate sample size calculations, randomization procedures, and analysis methods.1 6 11

As the example above illustrates, designing and implementing a pragmatic trial requires close coordination between researchers and healthcare providers and staff.3 4 5 The specific considerations and methods involved in this coordination are not currently part of the literature or the training of most researchers and practitioners. Common questions for researchers and healthcare providers considering a pragmatic clinical trial include:

What would motivate a clinic, hospital, or entire healthcare organization (which we refer to collectively in this paper as healthcare system) to participate in a research study, given the existing demands on clinical staff?

How can a pragmatic intervention be designed to fit the workflow within which it will be administered?

What features do the intervention and trial need to ensure sustainable, translatable results?

Answering these questions requires planned interactions among researchers, healthcare system managers, and frontline providers, who usually work in different environments. As described by one of the authors (PJM), “research is highly structured, like classical music, while the nature of clinical practice is to be flexible, like improvisational jazz. Pragmatic clinical trials use rigorous scientific methods to answer research questions while making adjustments to create a study protocol that clinical settings are willing and able to conduct.”

Advancing pragmatic trial partnerships

In 2006, the US National Institutes of Health launched the Health Care Systems Research Collaboratory, an initiative to promote collaborative research with healthcare organizations.7 The Collaboratory funded seven demonstration projects in which researchers are partnering with healthcare organizations to conduct pragmatic clinical trials to answer important healthcare questions. To our knowledge, this is the first pragmatic trial program designed to address both clinical questions and methods for effectively designing and implementing research in everyday healthcare settings.

All trials had a pilot period. Structured workgroups provide investigators with opportunities to share their experiences in pragmatic trial design. Discussion topics include statistics, ethics, and interactions with healthcare organizations. In 2013, we (CT, ET, and EBL) conducted 30-minute semi-structured telephone interviews with the research leaders of the demonstration projects (table 1), three representatives from participating healthcare organizations, and four experts on collaborative medical research. We asked them what motivates clinical organizations to participate in research, what barriers they encountered in establishing pragmatic trials, and how they overcame them.

Table 1

 Demonstration pragmatic trial pilots of the NIH Health Care Systems Research Collaboratory funded in 2012

Project title	Research question	Principal investigator	Partner organizations
Strategies and Opportunities to Stop Colon Cancer in Priority Populations (STOP CRC)	Does an evidence based, culturally tailored approach increase colorectal cancer screening in minority and low income populations?	Gloria Coronado, Kaiser Permanente Northwest	Kaiser Foundation Hospitals; Oregon Community Health Information Network (OCHIN); Federally Qualified Health Center clinics
Collaborative Care for Chronic Pain in Primary Care (PPACT)	Does a team based program in primary care help patients manage chronic pain?	Lynn DeBar, Kaiser Permanente Northwest	Kaiser Permanente Georgia; Kaiser Permanente Northwest; Kaiser Permanente Hawaii
Time to Reduce Mortality in End-Stage Renal Disease (TiME)	Does systematically implementing a hemodialysis session duration of ≥4.25 hours improve survival, reduce hospitalizations, and improve quality of life for patients with kidney failure?	Laura Dember, University of Pennsylvania	Dialysis provider organizations Fresenius Medical Care-North America and DaVita
Bathing to Eliminate Infection (ABATE Infection)	Does routine bathing with antiseptic soap (compared with targeted use of a nasal antibiotic ointment) reduce infections and hospital readmissions in general medical, surgical, and oncology inpatient units?	Susan Huang, University of California Irvine School of Medicine	Hospital Corporation of America; Harvard Pilgrim Health Care; John Stroger Hospital of Cook County; Rush University
A Pragmatic Trial of Lumbar Image Reporting with Epidemiology (LIRE)	Can interpretation of diagnostic tests for lower back pain be improved by adding information to imaging reports on the prevalence of findings in patients without back pain?	Jeffrey Jarvik, University of Washington	Kaiser Permanente Northern California; Group Health Cooperative; Mayo Clinic; Henry Ford Health System
Pragmatic Trial of Population-Based Programs to Prevent Suicide Attempt	How well do two different intervention programs work to reduce suicide risk?	Gregory Simon, Group Health Research Institute	Group Health Cooperative; HealthPartners; Kaiser Permanente Colorado
Nighttime Dosing of Anti-Hypertensive Medications: A Pragmatic Clinical Trial (BPMedTime)	Does nighttime dosing of blood pressure medication reduce risk for cardiovascular events?	Gary Rosenthal, University of Iowa	University of Iowa, Duke University

Based on the responses, we developed research partnership guidelines summarized in the framework in the figure. The framework shows how researchers, delivery system representatives, and, increasingly, patients work together to design a pragmatic trial in several steps described in this paper: (1) build partnerships, (2) define clinically important questions, (3) assess feasibility, (4) involve stakeholders in study design, and (5) develop study workflows. For each step, we address limitations and provide guidance.

Framework for pragmatic clinical trial partnerships.

Build partnerships

Several of the demonstration trial partnerships began with prior studies, ranging from feasibility studies to full trials that needed to be translated to new settings. In the US, researchers and healthcare organization collaborators connect through varied mechanisms including consortia such as the HMO Research Network or the Clinical and Translational Science Awards Consortium supported by the National Institutes of Health. Other countries and regions may have similar structures for fostering collaboration, such as the Academic Health Science Networks or the Collaborations for Leadership in Applied Health Research and Care in the UK.

A challenge to collaboration is that researchers and healthcare system representatives must understand and respect each others’ needs, objectives, and work culture. Interviewees said that the healthcare organization should have a culture that values research, measurement, and evidence, while researchers must understand healthcare system priorities and processes.12 For example, in the REDUCE MRSA trial, the collaborators had shared interests in preventing healthcare associated infections but needed to develop mechanisms to work together to design an effective trial. These mechanisms included Hospital Corporation of America developing a team for research and academic affairs to work with other external partners. This team meets regularly to evaluate progress on current research but also to evaluate and design new studies of mutual interest. This relationship then fostered additional studies such as the ABATE Infection trial (table 1). Collaborators who are “boundary spanners” with both research and clinical experience are valuable because they facilitate the communication between researchers and clinical partners that is necessary for conducting a pragmatic trial and translating its results to usual care.13 14 The next step is defining the research questions that drive the study.

Select the research topic

Pragmatic trials answer questions that are relevant to patients, providers, and healthcare organizations.2 Questions might originate from academia, delivery systems, professional organizations, patient-clinician alliances, the public health community, or the general public. The National Institute for Health Research and the James Lind Alliance in the UK both offer mechanisms for prioritizing research that is most wanted by patients. The research topic for several of the Health Care Systems Collaboratory demonstration projects arose from discussions with healthcare system managers. For example, the PPACT study (table 1) emerged from what seemed like an intractable problem to patients, clinicians, and researchers about how to control chronic pain.

Assess feasibility

To be feasible, pragmatic trials must potentially add value to the healthcare system. Without this element, researchers are unlikely to receive the necessary clinical and operational commitment for the study. Therefore, research topics should fit a healthcare system’s need for (1) results that decision makers might use to improve care and patient outcomes, (2) integration with clinic workflow, (3) relatively low entry costs, and (4) positive, or at least little negative, impact on productivity and the healthcare system’s finances. Not all trials will meet all criteria. A limitation is that some trials must substantially disrupt clinical workflow; however, these trials can be feasible with effective joint planning and a potentially large clinical payoff. For example, in the REDUCE MRSA trial, the practical experience of the academic investigators as medical directors of infection prevention programs and the health system quality improvement experience of the Hospital Corporation of America co-investigators enabled the creation of functional and efficient changes to workflow to implement either targeted or universal decolonization that were not only acceptable by management and clinicians but were used for enduring implementation and dissemination after the trial results were known. Pragmatic trials must watch for conflicts with delivery system quality improvement projects. For example, for a demonstration trial embedded in a delivery system that was transitioning to a new electronic health record, researchers and delivery system representatives cooperated to limit changes in the electronic health record that might affect the study.

Pilot phases are particularly important to understand the best study process, resources, and management to move work into everyday workflow.15 All of the Collaboratory demonstration projects were funded as pilots in order to assess feasibility of full implementation. Even for previously successful research partnerships, an objective pre-assessment of the healthcare system’s capabilities for the proposed study is essential. Critical questions include:

Are sufficient patient numbers and data available for the analysis?

Can data be collected at all clinical sites?

How do the sites vary in services and capabilities?

Can the system’s regulatory and administrative infrastructure support approval and oversight by ethics committees and review boards?

Will the intervention add long-term value to the system?

If the pre-assessment process reveals a poor fit between a healthcare system and a planned study, the system cannot participate, although researchers can maintain relationships with its management for future studies and dissemination.

Involve stakeholders in design

A successful pragmatic clinical trial requires engagement and input from all levels of a healthcare system. Pragmatic trials give clinical staff a professional development opportunity to practice research skills and participate in quality improvements.16 17 However, clear communication between researchers and clinical personnel about processes, expectations, and roles and goals is essential (table 2), since the collaboration is often a new experience for all parties. All collaborators should acknowledge competing priorities and differences in work culture.18 For example, researchers may discount the ongoing quality improvement work in healthcare systems as not rigorous, while frontline clinicians may perceive research as slow, expensive, inflexible, or not clinically relevant. As described earlier, the REDUCE MRSA trial developed a successful trial by working closely with staff who were already designated as quality improvement champions about how to implement and monitor the protocol. Weekly steering committee and analytic calls were used to maintain communication across multiple stakeholders.

Table 2

 Typical roles and goals of healthcare system participants in pragmatic clinical trials

Participants	Roles	Goals	Other considerations
Frontline clinical staff	Help formulate and carry out study protocol	Add study to workflow while maintaining high-quality patient care. Produce evidence to improve patient care and clinical decision making	Staff should be engaged without interrupting their work and should receive tailored reports on study progress and findings
Leadership (senior management)	Promote and support study throughout the delivery system	Value: better patient outcomes, cost effectiveness, efficiency	Ideally, support is at all levels, but buy-in from top leaders is critical
Business operations	Ensure study integration with HCS billing	Compliance with regulations, avoid revenue loss	This factor is typically complex due to local variations
IT staff	Adapt EHR for study protocol and data collection; advise on feasibility of protocol; maintain functionality beyond the study	EHR and patient portal features that patients and clinicians use	IT staff, in particular, often have competing demands and resource limits
Operational managers	Translate study objectives into clinical workflow changes	Ensure study success with minimal clinical disruption	The research team must be flexible and realize that local considerations for this group include patient outcomes
Clinic champions	Liaison between HCS and researchers	Integration and sustainability of study intervention	Champions should have local credibility and be rewarded and recognized, especially for improving patient outcomes
Researcher	Propose, design, and adapt study for HCS. Translate clinical issues into researchable questions.	Answer research questions and positively impact public health	Expect the unexpected and be prepared to be flexible and to learn.

HCS=healthcare system. IT=information technology. EHR=electronic health record.

Getting the attention of busy managers is challenging, especially before funding is assured. Researchers approaching healthcare organization managers to propose research embedded in clinical practice should highlight advantages such as the potential for gains in patient outcomes, staff efficiency, or health information technology (IT) improvements, along with congruence with other organization-wide priorities. After getting leadership buy-in, interviewees recommended networking to find people throughout the organization with the knowledge, interest, and authority to contribute to the study, as well as the time to maintain regular contact with researchers.

Researchers we interviewed stated that clinical staff, healthcare system managers, and researchers need to work together to optimize design and implementation of the study protocol. This is especially important during design and piloting. Staff and managers know how to best use existing health IT, workflow, clinical procedures, and local champions to make study participation easier for clinical staff. For example, based on provider input, one demonstration project moved data collection from a study website to an existing clinical system, another changed an intervention from group to individual therapy, and a third shifted implementation responsibilities from doctors to pharmacists.

Research teams may also need to involve clinical managers and frontline staff to identify additional costs of the study. These might include wear and tear on medical devices, use of additional supplies and services, time and effort to collect additional data, or the need for additional staff or providers. In the UK, for example, clear demarcation and calculation of which costs are the responsibility of the researcher and which are the responsibility of the healthcare service are standard parts of grant applications to national funders. In countries with multiple payers, operational staff can prevent double billing and advise research teams about sustainability, such as who will pay for intervention costs when the trial ends.

Health IT staff are increasingly crucial to pragmatic trials because of the growing use of electronic health records in research. In our experience, many studies experienced challenges making additions or alterations to the electronic health record. These were resolved either by consulting with clinical IT staff or by changing study plans.

Implement the study into workflow

To facilitate study implementation by clinical staff, trial procedures should mimic normal clinical practices and use existing resources as much as possible. Because pragmatic clinical trials occur in the context of dynamic real-world care, researchers need to be prepared to adapt studies based on stakeholder input or changes in the care environment. Ideally this happens during a pilot phase but likely will continue even as the study is under way. Flexibility in adapting to specific clinical situations will improve study protocol compliance by clinicians and facilitate the long term sustainability of tested interventions.

Planning for sustainability from the beginning of a study lays a foundation for implementing and spreading successful changes, including establishing a sustainable funding model.14 A limitation of research embedded in practice is that researchers must consider how to remove or adapt an intervention if data indicate that it is not effective. This part of the study is facilitated by communicating tailored results to all groups involved in the study, including patients and their families.

Conclusion

Pragmatic medical research is receiving increasing attention and funding.12 19 20 Pragmatic trials are conducted in clinical settings for “a real-world test in a real-world population.”11 As our examples from the REDUCE MRSA trial and the NIH Health Care Systems Research Collaboratory illustrate, pragmatic trials require that researchers and healthcare system clinicians, senior management, and staff develop the attitudes, skills, resources, and shared vision for close collaboration. The insights presented above, together with the resources in the Linked information box, provide guidance for building engaged pragmatic clinical trial teams.

NIH Collaboratory. Health Care Systems Research Collaboratory. www.nihcollaboratory.org/Pages/Knowledge-Repository.aspx

NIH Collaboratory. Introduction to pragmatic clinical trials. http://sites.duke.edu/rethinkingclinicaltrials/introduction-to-pragmatic-clinical-trials/

Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Clin Epidemiol 2009;62:499-505.

NIH Collaboratory. Rethinking clinical trials: a living textbook of pragmatic clinical trials. http://sites.duke.edu/rethinkingclinicaltrials/

CTSA Consortium. CTSA: Clinical & Translational Science Awards. https://www.ctsacentral.org/

HMO Research Network Research Tools. www.hmoresearchnetwork.org/en/Tools%20&%20Materials/

Research Toolkit: A toolkit for health research in partnership with practices and communities. http://researchtoolkit.org/

James Lind Alliance. www.lindalliance.org/. Discusses stakeholder involvement in research priority setting.

NIHR: National Institute for Health Research. Identifying research questions. www.nets.nihr.ac.uk/identifying-research

NHS England. Academic health science networks. www.england.nhs.uk/ourwork/part-rel/ahsn/

NIHR: National Institute for Health Research. Collaborations for leadership in applied health research and care. www.nihr.ac.uk/about/collaborations-for-leadership-in-applied-health-research-and-care.htm