The REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) Study. Rationale and Design.

There is broad interest in improved methods to generate robust evidence regarding best practice, especially in settings where patient conditions are heterogenous and require multiple concomitant therapies. Here, we present the rationale and design of a large, international trial that combines features of adaptive platform trials with pragmatic point-of-care trials to determine best treatment strategies for patients admitted to an intensive care unit with severe community-acquired pneumonia. The trial uses a novel design, entitled "a randomized embedded multifactorial adaptive platform." The design has five key features: 1) randomization, allowing robust causal inference; 2) embedding of study procedures into routine care processes, facilitating enrollment, trial efficiency, and generalizability; 3) a multifactorial statistical model comparing multiple interventions across multiple patient subgroups; 4) response-adaptive randomization with preferential assignment to those interventions that appear most favorable; and 5) a platform structured to permit continuous, potentially perpetual enrollment beyond the evaluation of the initial treatments. The trial randomizes patients to multiple interventions within four treatment domains: antibiotics, antiviral therapy for influenza, host immunomodulation with extended macrolide therapy, and alternative corticosteroid regimens, representing 240 treatment regimens. The trial generates estimates of superiority, inferiority, and equivalence between regimens on the primary outcome of 90-day mortality, stratified by presence or absence of concomitant shock and proven or suspected influenza infection. The trial will also compare ventilatory and oxygenation strategies, and has capacity to address additional questions rapidly during pandemic respiratory infections. As of January 2020, REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) was approved and enrolling patients in 52 intensive care units in 13 countries on 3 continents. In February, it transitioned into pandemic mode with several design adaptations for coronavirus disease 2019. Lessons learned from the design and conduct of this trial should aid in dissemination of similar platform initiatives in other disease areas.Clinical trial registered with www.clinicaltrials.gov (NCT02735707).

RCT Entities:   Population Interventions Outcomes

For centuries, how physicians made treatment decisions was largely unmeasured. In the latter half of the 20th century, with greater auditing of healthcare delivery, it became apparent that clinical decisions were often made inconsistently and without strong scientific rationale (1). This observation led to the rise of evidence-based medicine, which rests on the randomized clinical trial (RCT) to generate reliable evidence of treatment effectiveness and the incorporation of that evidence into treatment guidelines. Today, policymakers use compliance with such guidelines as a measure of healthcare quality. However, experts criticize treatment guidelines, both because they frequently lack evidentiary support from RCTs and because evidence based on RCTs can often be too simplistic, failing to capture the nuance of individual patient circumstances (2). In other words, a physician may not follow a guideline because of concerns regarding best treatment options under conditions of uncertainty. These problems are particularly acute in pandemics (3).

Until recently, there was no easy resolution to this tension. However, the 21st century ushered in a digital revolution that is transforming our ability to understand biology, capture clinical data, and execute RCTs capable of nuanced estimates of treatment effects and rapid adaptation to pandemics. This article describes one such effort using a novel design known as a randomized embedded multifactorial adaptive platform (REMAP) (2) to test multiple therapies in patients admitted to the intensive care unit (ICU) with severe community-acquired pneumonia (CAP). We review the study’s rationale, design, and implementation.

The Decision to Study Severe CAP

We chose severe CAP because it is extremely common, case fatality is high, the strength of evidence guiding treatments is limited, and there is substantial variation in care. Worldwide, CAP remains one of the largest contributors to death and disability-adjusted life-years lost in rich and poor countries alike (4–6). Severe CAP, the subset at risk for acute hypoxemic respiratory failure and shock, is also the most common cause of sepsis, a frequent reason for ICU admission, with a mortality rate of 20–50% (7–9). Finally, viral pneumonia, especially influenza, is the most deadly recurring pandemic infection (4).

The treatment of severe CAP involves multiple therapies, including antimicrobial regimes, host immunomodulation, organ support, and interventions to prevent complications. Several guidelines address severe CAP treatment, but the specific recommendations frequently lack strong evidence. For example, high-quality evidence from RCTs supports only 4 of 44 recommendations in current European guidelines (10–12), 11 of 43 in U.S. guidelines (13), and 7 of 93 Surviving Sepsis Campaign guidelines (14). Furthermore, several statements are contradictory across guidelines. Not surprisingly, guideline compliance is poor and care is variable (15–18), with potentially adverse consequences (18, 19).

Challenges to the Generation of Robust and Useful Evidence for Severe CAP

Two issues hinder generation of high-quality evidence for care of patients with severe CAP. First, for endemic CAP, the effectiveness of interventions may vary by subgroups or use of concomitant treatments. For example, hydrocortisone effectiveness may vary by etiology of CAP (viral or bacterial), presence of shock, and choice of antimicrobial therapy. Traditional RCT designs are not well suited for assessing complex treatment–treatment and treatment–subgroup interactions. Second, RCTs launched in pandemics, such as the 2009 H1N1 influenza or coronavirus disease 2019 (COVID-19) pneumonia outbreaks, even when using “just-in-time” procedures, are often implemented too slowly to generate useful knowledge (3, 20).

A New Approach

Our solution for better evidence generation in severe CAP, the REMAP, combines two designs: a point-of-care RCT and an adaptive platform trial (2, 21, 22). Point-of-care RCTs boost capture of eligible patients via a clinical moment, or point of care, that triggers the trial apparatus (23, 24), ideally in the electronic health record (21). This approach is used for pragmatic comparative effectiveness studies (25, 26). Rather than testing individual interventions in a single homogeneous disease state and terminating when that task is complete, adaptive platform trials focus on a broader set of disease states and test multiple therapies simultaneously and sequentially (22, 27, 28). They are thus an experimental platform, rather than a series of experiments. They are adaptive in that they incorporate rules for changes in entry criteria, study arms, and the proportion randomized to each arm over time. There are several adaptive platform trials outside critical care (29, 30).

Description of the REMAP Design

REMAP combines a point-of-care RCT and an adaptive platform trial to create a design that, like the former, embeds the trigger for patient recruitment in routine clinical care, but, like the latter, then enrolls these patients into a platform capable of addressing complex study questions regarding multiple therapies in multiple subsets of patients (Figure 1) (2). Embedding the trial promotes capture of the greatest number of patients, which is key to generalizability, arguably essential for response to pandemics that “wave” rapidly through different regions (31), and efficient. Embedding also facilitates low operational complexity at the bedside, even though the internal clinical trial machinery may be complex. Thus, with REMAP-CAP, any patient admitted to the ICU with acute respiratory insufficiency due to suspected pneumonia is flagged for enrollment and randomization. Ideally, all eligible patients will be enrolled, generating an automatic custom order sheet relating to all the intervention assignments. Other aspects of the trial, such as ongoing monitoring and data collection, will also be embedded where possible in routine care. The trial design also coordinates with national ICU registries to permit comparison with unenrolled patients and avoid data collection redundancy (Appendix).

Figure 1.

Schematic of the REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) design. CAP = community-acquired pneumonia; DSMB = Data Safety Monitoring Board; EHR = electronic health record; ICU = intensive care unit; RCCs = regional coordinating centers; SAC = statistical analysis committee.

The trial is “multifactorial” in that it tests multiple interventions within multiple therapeutic domains and multiple patient strata (Table 1). In REMAP-CAP, the initial interventions are grouped under four domains (an antimicrobial domain consisting of four alternative antibiotic strategies and two host immunomodulation domains [one testing alternative hydrocortisone dosing regimens, one testing use of extended macrolide therapy] and one evaluating antiviral therapy). Domains relating to oxygen therapy and respiratory support strategies will be added, and pandemic COVID-19–specific domains are also now launched (described in the section Adaptation during a Pandemic). Each patient is randomly assigned a specific intervention within each domain; the set of assigned interventions defines the treatment regimen. The strata are patient characteristics identifiable at enrolment for which a differential effect on outcome by intervention is hypothesized. REMAP-CAP commenced with two strata: presence or absence of shock and presence or absence of suspected (or proven) influenza infection.

Table 1.

Summary of REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) features

Feature	Description
Patients
Entry criteria
Inclusion criteria	• Admitted to ICU within 48 h of hospital admission
	• Age ≥18 yr
	• CAP by clinical and radiologic criteria
	• Requiring respiratory (non-invasive or invasive ventilation) or cardiovascular (inotropes/vasopressors) support
Exclusion criteria	• Healthcare-associated pneumonia
	• Imminent death and no commitment to full active treatment
	• Prior enrollment in REMAP-CAP in the last 90 d
Stratum
Definition	A patient characteristic defined at enrollment used for the generation of specific treatment estimates
Starting strata	• Presence of shock or not (defined as hypotension or vasopressor requirement after volume resuscitation)
	• Presence of suspected or proven influenza infection or not
State
Definition	A clinical state that triggers a specific domain
Example	Mechanical ventilation
Operationalization	If a domain is only active for patients who enter a state (either at enrollment or later), the patient is randomized to an intervention within that domain but the intervention is only revealed when the patient enters the state
	Estimates of intervention effects within a state-specific domain are only generated for those who enter the state
Sites and regions
Starting conditions	The study launches at 50 hospitals in Europe, 35 sites in Australia and New Zealand, and 12 sites in Canada
Future additions	Expansion in the United States, Brazil, and Saudi Arabia is under discussion. Long-term planning includes other regions
Interventions
Nomenclature
Intervention	A treatment being tested in REMAP-CAP
Domain	A specific set of competing alternative interventions within a common clinical mode, which, for the purposes of the platform, are mutually exclusive and exhaustive
Regimen	The combination of assigned interventions across domains
Starting conditions
The trial launches with four domains
Antibiotics
	• Ceftriaxone plus macrolide
	• Piperacillin-tazocin plus macrolide
	• Amoxycillin-clavulanate plus macrolide
	• Respiratory quinolone
Immunomodulation with an extended macrolide
	• Standard course (3–5 d)
	• Extended macrolide (14 d)
Immunomodulation with hydrocortisone
	• No corticosteroid
	• Shock-dependent hydrocortisone
	• Hydrocortisone (7-d course)
Antiviral agents active against influenza
	• No antiviral agent
	• Oseltamavir (5 d)
	• Oseltamavir (10-d course)
	Patients can be ineligible for randomization within a domain (e.g., the antiviral domain is only active for those within the influenza stratum). Thus, the trial launches with 240 potential regimens (adding “not eligible” as an option in each domain, no. of regimens = 5 antibiotic x 3 extended macrolide x 4 steroid x 4 antiviral = 240)
Future additions
	Two additional domains (ventilator support and oxygen management) will be added shortly
	The ventilator support domain will be restricted to the state of mechanical ventilation. Interventions to be tested within this state-specific domain will be guideline-recommended ventilation and clinician-preferred ventilation
	The oxygen management will compare two interventions (usual oxygen titration vs. conservative oxygen titration). This domain will be eligible to all patients
	Once these domains launch, each with two options plus “not eligible,” the number of regimens becomes 240 × 3 × 3 = 2,160 regimens
Embedding
Description	To ensure capture of all possible patients, streamline integration with clinical care, and reduce study costs, the study has several features that embed it in clinical practice. Ideally, these embedded strategies are built through integration between REMAP-CAP trial machinery and usual clinical processes. Strategies include:
	• Triggering of patient identification and enrollment from a clinical “point-of-care”
	• Verification of eligibility, documentation of consent, and enrollment activation via software interface
	• Generation of stratum-specific randomly assigned REMAP-CAP regimen as “order set”
	• Intent to embed, where appropriate, within the electronic health record
Endpoints
Primary endpoint	• All-cause mortality at 90 d
Secondary endpoints	• ICU mortality
	• ICU length of stay
	• Ventilator-free days
	• Organ failure free days
	• Proportion of intubated patients receiving tracheostomy
	• Domain-specific end-points
Statistical methods
Overview	The trial is built on a Bayesian inference framework. After an initial run-in period, a prespecified Bayesian inference model is updated each month using the latest trial data to generate updated posterior probabilities of death for each patient regimen-by-stratum group, and hence the probability that any one intervention (or regimen) differs from any other. The model output is used both to update the randomization weights for ongoing random assignments and to trigger thresholds for superiority, equivalence, and inferiority
Multifactorial Bayesian inference model	The model predicts the primary endpoint rate for each patient regimen-by-stratum group, conditional upon patient age, trial site and region, and time era. Terms are included for intervention-by-intervention and intervention-by-stratum interactions, and for patients who are ineligible for either an intervention or a domain. The model is also configured in advance for the incorporation of state-specific domains (e.g., ventilator support)
Response-adaptive randomization	The posterior probabilities from the Bayesian inference model are incorporated into an algorithm that provides updated randomization proportions to each regimen by stratum. This algorithm adjusts for sample size to avoid large, potentially spurious changes. Consequently, interventions that are faring well will be randomly assigned more commonly, and those faring less well will be assigned less commonly
REMAP-CAP statistical conclusions	When an updated probability triggers a threshold, results are communicated to the DSMB and ITSC for public release and decisions regarding ongoing treatment assignment
Superiority	>99% probability that an intervention is superior to alternatives in a domain within one or more strata
Equivalence	>90% probability that odds of death for two interventions differ by <0.2
Inferiority	<1% probability that an intervention is superior in a domain
Operating characteristics	All trial parameters were tested through extensive Monte Carlo simulations of anticipated trial performance under different scenarios (Appendix)

Definition of abbreviations: ICU = intensive care unit; CAP = community-acquired pneumonia; DSMB = Data Safety Monitoring Board; ITSC = International Trial Steering Committee; REMAP-CAP = Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia.

This table describes REMAP-CAP in interpandemic mode, and excludes the coronavirus disease 2019 adaptations (described in the Adaptation during a Pandemic section of the text).

The trial estimates the effectiveness of one intervention over others within a domain, with the capacity to specify whether effects are affected by the choice of interventions within other domains or by strata. Which interactions are evaluated are prespecified. The trial uses response-adaptive randomization (RAR) (32), with the probability of randomization to any particular regimen adjusted over time to favor better-performing interventions, eventually triggering a stop when a predetermined threshold is attained (Figure 1). Colloquially, RAR allows the trial not to “play the winner,” but to “probably play what is probably the winner.” The RAR rules define separate randomization proportions for each stratum. For example, if one hydrocortisone dosing strategy appears beneficial for patients with shock, but neutral in patients without shock, then the RAR rule increasingly weights the odds for shock patients to receive that strategy, but maintains equal allocation for patients without shock.

Importantly, interventions may not be appropriate for a patient, either because the patient is eligible for a domain, but has a contraindication for a particular intervention within that domain, or because the patient is not in a clinical state that requires treatment within that domain. In the first situation, as long as at least two interventions remain available within the domain, the patient will be randomized. An example of the second situation would be a respiratory support domain restricted to patients requiring mechanical ventilation. If a patient is enrolled in the trial, but not intubated, she will be randomized, but the assignment will not be revealed until she enters the state (requiring mechanical ventilation) that triggers deployment of the intervention. In addition to “patient-level” exclusions, not all domains and interventions may be available at all sites, either because a participating site lacks equipoise or temporarily lacks availability of an intervention. In all these instances, the statistical inference model tracks and accommodates for these varying levels of participation.

Other adaptive trial features include the capacity to introduce new strata, domains, and interventions over time. The rules and operating characteristics of the platform are detailed in the REMAP-CAP core protocol and statistical analysis appendix with separate domain-specific and region-specific appendices to describe interventions and regional participating groups (Appendix; see also www.remapcap.org). The use of separate appendices permits an efficient, modular structure where any update to the design requires only that the relevant appendix or appendices be added or modified (Figure 2A).

Figure 2.

Overview of the REMAP-CAP (Randomized Embedded Multifactorial Adaptive Platform for Community-acquired Pneumonia) documentation and oversight. (A) Structure of the REMAP-CAP protocol and appendix documents. (B) Organogram of the REMAP-CAP oversight. ANZ = Australian and New Zealand; ANZ RMC = Australia and New Zealand Regional Management Committee; CRMC = Canadian Regional Management Committee; DSA = domain-specific appendix; DSMB = Data and Safety Monitoring Board; DSWG = domain-specific working group; EU = European; EU RMC = Europe Regional Management Committee; IIG = International Interest Group; IPWG = International Pandemic Working Group; ITSC = International Trial Steering Committee; RMC = Regional Management Committee; RSA = region specific appendix; SAC = Statistical Analysis Committee.

Study Sites, Patients, and Endpoints

Table 1 summarizes key trial features. REMAP-CAP is a global program intended to enroll critically ill patients with CAP worldwide (clinical trials registration no. NCT02735707; universal trial no. U1111-1189-1653). The trial was launched in Europe under the PREPARE (Platform for European Preparedness against (Re-)emerging Epidemics) consortium (https://www.prepare-europe.eu/about-us/workpackages/workpackage-5) with funding from the European Union. REMAP-CAP has also launched in Australia and New Zealand, supported by the Australian and New Zealand Intensive Care Society Clinical Trials Group and in Canada supported by the Canadian Critical Care Clinical Trials Group, with funding from the respective national governments. Together, these programs fund the first 4,000 patients, and are anticipated to recruit in 50 sites in Europe, 35 sites in Australia and New Zealand, and 15 sites in Canada. Other regions of the world will join as funding becomes available. Over 500 patients were enrolled as of March 2020. The trial is overseen by an international trial steering committee. An overview of trial structure is provided in Figure 2B.

To be included, participants must be admitted to the ICU within 48 hours of hospital admission, be ≥18 years of age, have CAP by clinical and radiologic criteria (33), and require respiratory or cardiovascular organ support. Exclusion criteria include healthcare-associated pneumonia, presumption that death is imminent with lack of commitment to full support, and participation in REMAP-CAP in the prior 90 days. There are also domain-specific exclusion criteria described in their appendices (Appendix; www.remapcap.org). The primary objective is to determine the effectiveness of different interventions, alone and in combination, for adult patients with severe CAP in decreasing 90-day mortality. Secondary objectives are to determine the effects on hospital and ICU length of stay, ventilator and organ failure–free days through 28 days, and functional outcomes at Day 180.

Initial Domains and Interventions

Antibiotic Domain

Empiric use of a β-lactam and macrolide combination or a respiratory quinolone alone are both recommended for severe CAP (10–12, 34). Patients will therefore be randomized (depending on availability and local equipoise) to one of three β-lactams (ceftriaxone, piperacillin-tazobactam, or amoxycillin-clavulanate) with a macrolide (azithromycin, clarithromycin, or roxithromycin), or to a respiratory quinolone (moxifloxacin or levofloxacin). Patients with known allergies are ineligible to receive an agent to which they are allergic, but will be allocated among remaining options.

Host Immunomodulation with Extended Macrolide Domain

Although macrolides are recommended for 3–5 days for CAP (13), an extended course may also be beneficial, in part because of macrolide antiinflammatory properties (35, 36). Therefore, patients randomized to any antibiotic arms containing a macrolide can also be randomized to a standard (3–5 d) or experimental 14-day course.

Host Immunomodulation with Corticosteroid Domain

Although severe CAP is associated with a potentially detrimental host immune response, successful immune modulation remains elusive. Benefit with corticosteroids was reported in vasopressor-dependent septic shock, severe Pneumocystis pneumonia, and late acute respiratory distress syndrome (37–41), but the evidence is inconclusive (42–49). Notably, two recent, large RCTs in septic shock reported conflicting results, though both suggested faster resolution of hemodynamic instability (50, 51). Patients will therefore be randomized to no steroid, hydrocortisone 50 mg intravenously every 6 hours for 7 days (the same strategy tested previously), or to hydrocortisone at the same dose, but prescribed only while in shock. Sites can choose any two (or all) of these options, depending on equipoise. The effect of corticosteroids will be evaluated separately in patients with or without baseline shock and with or without influenza infection.

Antiviral Domain

The effectiveness of oseltamivir, and other new anti-influenza agents, is not established in the critically ill. The modest impact of oseltamivir in uncomplicated seasonal influenza further raises uncertainty about its value in serious infection (52–54). There is also no consensus regarding duration of oseltamivir therapy (55). Patients with suspected or proven influenza will be randomized to no oseltamivir, oseltamivir 75 mg every 12 hours for 5 days, or oseltamivir 75 mg every 12 hours for 10 days. Only sites that do not use oseltamivir as standard care will participate in the no-oseltamivir intervention. We will add baloxavir, alone and in combination with oseltamivir, when it is more available (56).

Respiratory Support Domains

International guidelines support lung protection strategies that minimize excessive volume or pressure (14, 57, 58). The guidelines are based on patients with acute respiratory distress syndrome (ARDS), but whether this approach is optimal for patients with CAP without ARDS is unknown. Moreover, observational studies demonstrate poor uptake of guideline-recommended ventilatory strategy, with many clinicians personalizing ventilatory settings on a patient-by-patient basis (59). Optimal ventilatory strategy is also complex, involving tidal volume, mode (limiting breaths by pressure or volume), positive end-expiratory pressure (PEEP), and use of spontaneous ventilation.

To start determining optimal ventilatory strategy for patients with CAP, the ventilation domain will randomize patients to guideline-recommended care (set tidal volume of 6 ml/kg of ideal body weight and use of a PEEP:fraction of inspired oxygen table) or clinician-preferred ventilation. This phase has three goals: first, to determine whether adherence to guideline-recommended care can be achieved in trial patients; second, to identify testable strategies within the spectrum of observed care patterns in the clinician-preferred intervention arm; and third, to identify stratification variables, such as presence of ARDS, unilateral versus bilateral involvement, PEEP:fraction of inspired oxygen ratio, and lung compliance.

Oxygenation support is almost universal for patients with CAP. However, neither the optimal inspired concentration nor optimal hemoglobin saturation target is known, and the infected lung may be particularly sensitive to injury by reactive oxygen species. Observational studies and a small, single-center RCT suggest that use of a conservative oxygen strategy may be safe and beneficial in pneumonia (60–62). Some evidence points to improved outcomes with reduced oxygen exposure in several diseases, but recent RCT results are conflicting (63–67). We intend to implement an oxygenation strategy that compares conservative to liberal oxygenation support, and harmonize with a large scale trial of all-comer ICU patients.

Adaptation during a Pandemic

REMAP-CAP adapts to answer time-critical questions relevant to optimal care of patients with pneumonia due to a pandemic infection in several ways. The platform has a “sleeping” stratum for patients with proven or suspected pandemic infection that is triggered at each site. A pandemic-specific model tests the effect of different agents and regimens in the pandemic stratum. This model can use an alternative endpoint and be updated more frequently. The pandemic-specific model can incorporate data from nonpandemic patients with regard to all domains that are relevant in both pandemic and interpandemic periods, with consideration of potential interactions. In addition, other domains, such as novel antiviral therapies, immunoglobulins or convalescent sera, or other immunomodulation approaches, can be deployed.

In February 2020, REMAP-CAP entered pandemic mode in response to the COVID-19 pandemic with several adaptations, essentially as a subplatform, REMAP-COVID. These include a COVID-19 inference model for all confirmed and suspected cases that uses 21 ICU-based respiratory or cardiovascular support-free days (where death is assigned −1 d) as the primary outcome, with RAR as frequently as weekly. A specific REMAP-COVID core protocol was written to streamline onboarding of new sites that only enroll patients with COVID-19. Domains were implemented for COVID-19 antiviral therapy (including hydroxychloroquine and lopinavir-ritonavir) and immune modulation (including interferon [IFN]-β, IL1ra, and IL6ra agents), the corticosteroid domain was modified to include a higher dose, and other domains are under construction. The enrolment criteria were modified to allow entry at some sites of hospitalized patients who do not require ICU care for cardiovascular or respiratory support (defined as “moderate” COVID-19 disease state). The model tracks whether patients have moderate or severe disease at enrolment, includes interactions between domains (e.g., IFN-β and corticosteroids), and allows for nested analyses (e.g., comparing any antiviral therapy vs. none).

Statistical Considerations

Most RCTs are analyzed using frequentist statistics, which calculate the probability of observing patterns from a trial if a hypothesis is true (including patterns not observed). This approach relies on assumptions about frequency distributions of trial results that would arise if the same trial were repeated ad infinitum (hence the term “frequentist”). Thus, it requires specific sample sizes (the assumptions are for a specific trial of a specific size), which, in turn, require pre-experiment assumptions regarding plausible effect sizes and outcome rates (68). Although many clinicians are comfortable with this approach, the pretrial assumptions are frequently incorrect, and the design lacks flexibility to address the complex questions more reflective of clinical practice or to make midtrial corrections when pretrial assumptions are wrong.

To allow flexibility yet still generate robust statistical inferences, REMAP-CAP relies on a Bayesian, rather than frequentist, framework (69). A Bayesian approach calculates the probability a hypothesis is true, given observed data and prior information and beliefs. An advantage is that, as data accrue, the probability that a treatment is best can be updated (the updated probability is called the posterior probability). REMAP-CAP launches with no prior assumptions regarding which interventions are superior, akin to a typical RCT design. However, at regular intervals, newly accrued data are analyzed using a prespecified inference model to generate updated posterior probability distributions.

Although sample sizes are flexible, the trial nonetheless has rigorous prespecified elements that frame the design (Figure 1 and Table 1). The initial set of interventions within domains generates 240 regimens. The trial starts with a 2 × 2 structure based on two strata: presence or absence of shock (defined as receiving an infusion of vasoactive medication) and presence or absence of influenza infection, as assessed at the time of enrolment. The goal is to generate, for each domain, estimates of the difference in effect of any one intervention over another. Depending on the domain, this estimate may be conditional on stratum and intervention assignment within the other domains. The model estimates the probability of superiority for each treatment regimen for patients in one or more strata (which strata are applied in each domain varies, but is prespecified), conditional on allocation status in other domains (the domains for which intervention-by-intervention interaction is evaluated is prespecified), after adjustment for age, region and site, severity of illness, and 13-week time blocks (to adjust for drift). The model includes terms for the common effect of each intervention and selected interactions for all domains.

The model also accounts for patients who are ineligible for one or more interventions within a domain or for an entire domain. The starting conditions (assumptions set before data are accrued) for all terms in the model are specified in the Statistical Appendix. Noninformative prior probabilities are assigned to any direct intervention effects. Other terms (age, region, and interactions, etc.) are weakly assumed to potentially affect mortality such that they can be quickly overwhelmed by the data.

REMAP-CAP begins with randomization balanced across interventions. Thereafter, the Bayesian inference model is re-estimated at regular intervals with updated trial data. The updated posterior probabilities determine new randomization probabilities and can trigger a trial conclusion regarding an intervention’s effect. We set superiority as 0.99 or greater posterior probability that an intervention lowers mortality, equivalence as 0.90 or greater posterior probability that the odds ratio for mortality lies between 0.8 and 1.2, and inferiority as less than 0.01 posterior probability that the intervention is superior. These thresholds were selected before launch using Monte Carlo simulations to explore the trial’s operating characteristics (Appendix).

Advantages of the REMAP Design

The REMAP design offers four broad advantages: efficient use of data, improved participant safety, reduced downtime between trials, and enhanced knowledge translation (Table 2 and Figure 3). Four features improve efficiency. First, testing multiple interventions simultaneously allows more questions to be evaluated and avoids requiring a separate control group for every two-way comparison. Second, RAR and predetermined thresholds reduce or cease allocation of subjects to inferior arms, increasing power to differentiate between the remaining arms. Third, an overarching multifactorial model that drives RAR and stopping rules integrates information on treatment effects from all patient strata. Fourth, because randomization continues until superiority, equivalence, or inferiority thresholds are met, the platform avoids terminating a domain with indeterminate results.

Table 2.

Randomized embedded multifactorial adaptive platform design advantages

	Efficient Use of Information	Safety of Trial Participants	Avoiding Trial Downtime	Fusing Research with Care	Determining Optimal Disease Management	Learning Healthcare System
Multifactorial	✓	—	✓	✓	✓	—
Response adaptive randomization	✓	✓	—	✓	—	✓
Embedding	—	—	—	✓	—	✓
Frequent adaptive analyses	✓	✓	—	—	✓	✓
Analysis by stratum/subgroup	✓	✓	—	—	✓	—
Evaluation of interaction	—	✓	—	—	✓	—
Substitution of new interventions	✓	—	✓	—	✓	—

Figure 3.

Trial simulations comparing REMAP (Randomized Embedded Multifactorial Adaptive Platform) to traditional randomized clinical trial designs. The operating characteristics of alternative study designs are evaluated by running a Monte Carlo program, which randomly draws trial samples from simulated populations with predetermined characteristics (alternative “truths” about the true yet unknown effect of an intervention or regimen in a population). Each simulated trial accrues patients one at a time until a sample size of 2,000. The simulated trials are repeated 10,000-fold and the summary of all trials under each simulated scenario provides estimates of average trial performance. In all instances, the simulations are of trials testing eight regimens, consisting of three domains with two interventions in each domain (23 = 8 regimens). Results are presented for a comparison of a standard trial design, with equal allocation to each arm, versus a REMAP design, using response-adaptive randomization (RAR) to preferentially assign patients over time to better-performing arms. Sample size (primary y-axis) is 250 per arm for the standard design (represented by a solid horizontal line); gray bars indicate the REMAP design. Probability of superiority (a proxy for power, secondary y-axis) is represented as an open red circle for the standard design and a solid red circle for the REMAP design. The predetermined characteristics of the underlying simulated population are represented in the upper portion of each panel. (A) Result summary under a simulated truth where regimen 8 is superior, regimen 5 is second best, and all others are inferior, but equivalent. (B) Result summary where regimens 5 and 8 are equally good, but regimens 1, 3, 4, and 7 are harmful with respect to regimens 2 and 6. In both scenarios, power is similar or superior with the REMAP design, yet, because RAR minimizes exposure to arms performing less well, results are generated with fewer deaths.

The REMAP design enhances safety because the adaptive rules promote greater allocation to better-performing interventions and, by corollary, less exposure to poorly performing interventions, over time. As the trial learns, the benefits of reduced uncertainty are translated rapidly into improved odds of exposure to the optimal strategy for participants. Thus, although individuals may still be assigned to interventions that perform poorly, if the trial is testing therapies that affect outcome, but for which the conventional wisdom is equipoise (and exposure outside the trial is balanced), then the patient is, on average, safer in the trial than out of it.

There is considerable downtime between traditional one-at-a-time trials, which is costly and burdensome for clinical trial units and contributes to delay in the acquisition of medical knowledge, or even failure to accrue knowledge in situations like pandemics (3). Because REMAP is a single perpetual platform trial, this downtime is largely eliminated. Instead, new interventions or domains of interest are simply added to the on-going platform through protocol appendix amendments. When fully embedded in an entire healthcare system, a REMAP becomes a platform for continuous quality improvement (and instant knowledge translation), where all patients are flagged at admission, and assigned therapies proportional to the level of certainty that these therapies are optimal.

Ethical Approval and Trial Oversight

Human subject protection in REMAP-CAP falls under the same review process as any RCT. Local regulations govern consent requirements, with consideration that several comparisons are of alternative standard care options and most are deployed emergently. The current protocol, with the current suite of domains and interventions, is approved in 13 countries, all with deferred consent for domains that test only options within standard care. The rules for changing the odds of randomization and stopping portions of the trial are predetermined and executed automatically. However, they are overseen by a Data Safety Monitoring Board, which has the capacity to override algorithm decisions if the proposed rule is deemed no longer acceptable. When a threshold is passed and conclusions are drawn, that portion of the trial is reported via publication and usual routes of dissemination. New interventions and domains are introduced via protocol modifications, with approval of relevant ethics boards. Of note, REMAP-CAP operates under the International Conference on Harmonization Good Clinical Practice guidelines and has approval for the study of investigative medicinal compounds. It is therefore possible to evaluate experimental therapies with appropriate caveats regarding specific data that may be required for regulatory approval.

Logistical Considerations

Although the trial machinery is very complex, that complexity is made as invisible as possible to the clinical sites. The largest logistical challenges relate to embedding the trial into routine care, which requires identification of the clinical “point of care,” mechanisms for notification to the central coordinating center in as automated a fashion as possible, execution of consenting procedures, and the ability of the coordinating center to quickly provide the randomly assigned regimen. Key to this success includes web-based software designed to interface with local clinical and research-related processes. For example, the software is easily accessed by any clinician and, through efficient prompting of a short list of clinical questions, automatically determines eligibility for the platform, domains, and individual interventions.

Discussion

Although we outlined numerous potential advantages of the REMAP trial design, there are considerable barriers. First, the ability to embed the trial requires a new paradigm for engagement between clinicians and researchers in many ICU settings. Such close partnership exists in other fields, such as within oncology trial networks. Similarly, the large acute myocardial infarction trials in the 1980s and 1990s relied on extremely high capture rates. In critical care, the fluid resuscitation trials by the Australian and New Zealand Intensive Care Society Clinical Trials Group also achieved extremely high capture rates (70, 71), in part by generating a culture that any patient requiring resuscitation prompted the clinical team to enroll the patient. These efforts share a common commitment to education, engagement, and attention to practical details at participating sites.

One concern will be the use of Bayesian inference and flexible sample sizes. For example, Bassler and colleagues (72) argued that early stopping overinflates estimation of treatment effects. However, trials that stop early for superiority are trials that, on average, would overestimate treatment effect even if they run to term (just as trials that do not trigger early stopping underestimate the true effect) (73). Assuming that appropriate rules are in place, early stopping does not, in and of itself, significantly overestimate treatment effect (or inflate the chance of type 1 error). The best estimate of treatment effect is the summary of all trial results. If REMAP-CAP generates an early large superiority signal for an intervention, and no other trial data exist, it would be appropriate to consider the true effect size as somewhat smaller (74).

As with all Bayesian adaptive designs, traditional estimation methods for type 1 and type 2 errors are not possible. Rather, these error rates are explored through simulation of trial operating characteristics under different scenarios and assumptions. The U.S. Food and Drug Administration and others provide guidance, but there is little question that considerable expertise is required (75, 76). This expertise is currently limited, and concentrated particularly within a few companies. To broaden expertise, all the government grants for REMAP-CAP stipulate efforts to expand competency among local academic trials groups. As such, REMAP-CAP has several regional initiatives and runs an international statistics and reporting interest group (>40 statisticians and trialists from 16 universities; Figure 2). The statistical group for REMAP-CAP provides the design and simulation software for free to academic groups, and serves as a free National Institutes of Health–supported consultation service for prospective researchers.

There will be issues regarding the reporting of REMAP trials, and all adaptive platform trials (22). For example, REMAP conclusions are generated from a model that incorporates all the data from the entire trial. It is unclear whether the report should include information on all patients enrolled thus far, including those whose data are still contributing to ongoing questions, or to some portion of the patients most directly relevant to the portion of the trial that has stopped. Because most RCTs are frequentist, trial reports that use Bayesian statistics will be unfamiliar to many readers, impeding understanding and dissemination. However, Bayesian trials and analyses are becoming considerably more common, which should reduce this problem (30, 77–79).

In summary, we present a novel class of study design with an example tailored specifically to determine optimal therapies for severe interpandemic and pandemic pneumonia. The design generates information that is broad (reflecting real-world practice) and narrow (generating precision estimates for patients with particular clinical features). The platform can incorporate new study arms, making it ideal for pandemic situations. The design will nonetheless face challenges. However, with funding to launch REMAP-CAP on three continents, we expect that many lessons will be learned, aiding broader, more efficient use of REMAPs in critical care and elsewhere.