Design of Clinical Trials Evaluating Sedation in Critically Ill Adults Undergoing Mechanical Ventilation: Recommendations From Sedation Consortium on Endpoints and Procedures for Treatment, Education, and Research (SCEPTER) Recommendation III.

OBJECTIVES: Clinical trials evaluating the safety and effectiveness of sedative medication use in critically ill adults undergoing mechanical ventilation differ considerably in their methodological approach. This heterogeneity impedes the ability to compare results across studies. The Sedation Consortium on Endpoints and Procedures for Treatment, Education, and Research Recommendations convened a meeting of multidisciplinary experts to develop recommendations for key methodologic elements of sedation trials in the ICU to help guide academic and industry clinical investigators.
DESIGN: A 2-day in-person meeting was held in Washington, DC, on March 28-29, 2019, followed by a three-round, online modified Delphi consensus process. PARTICIPANTS: Thirty-six participants from academia, industry, and the Food and Drug Administration with expertise in relevant content areas, including two former ICU patients attended the in-person meeting, and the majority completed an online follow-up survey and participated in the modified Delphi process.
MEASUREMENTS AND MAIN RESULTS: The final recommendations were iteratively refined based on the survey results, participants' reactions to those results, summaries written by panel moderators, and a review of the meeting transcripts made from audio recordings. Fifteen recommendations were developed for study design and conduct, subject enrollment, outcomes, and measurement instruments. Consensus recommendations included obtaining input from ICU survivors and/or their families, ensuring adequate training for personnel using validated instruments for assessments of sedation, pain, and delirium in the ICU environment, and the need for methodological standardization.
CONCLUSIONS: These recommendations are intended to assist researchers in the design, conduct, selection of endpoints, and reporting of clinical trials involving sedative medications and/or sedation protocols for adult ICU patients who require mechanical ventilation. These recommendations should be viewed as a starting point to improve clinical trials and help reduce methodological heterogeneity in future clinical trials.

Clinical practice guidelines for managing mechanically ventilated adults in the ICU acknowledge the lack of high-quality evidence on which to base recommendations for sedation and analgesia (1, 2). High-quality evidence is sparse because numerous barriers make clinical research in this area complex (3). An absence of standardized approaches to study design and methods and a lack of consensus on the most important clinical outcomes and measures are notable barriers. For example, a sampling of clinical trials on ICU sedation from the “ClinicalTrials.gov” website and several recently published trials (4–8) revealed substantial heterogeneity in their inclusion and exclusion criteria, primary and secondary efficacy outcomes, safety outcomes, measurement instruments, and timing of outcome measures relative to the sedative intervention and ICU admission. Unfortunately, such heterogeneity hinders meaningful comparisons across trials and prevents the use of meta-analysis to synthesize evidence and provide recommendations regarding how to optimally provide sedation for mechanically ventilated adults in the ICU (8).

To address these gaps, the Sedation Consortium on Endpoints and Procedures for Treatment, Education and Research (SCEPTER) convened a meeting that focused on the design and conduct of clinical trials for sedation management in critically ill adults who require mechanical ventilation (SCEPTER III). SCEPTER is part of the Analgesic, Anesthetic, Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership with the Food and Drug Administration (FDA) (9). Previous recommendations regarding the design of clinical trials for procedural sedation have been developed by SCEPTER (10, 11). Briefly, ACTTION was conceived as part of the FDA’s Critical Path Initiative (12), with a mission “… to identify, prioritize, sponsor, coordinate, and promote innovative activities—with a special interest in optimizing clinical trials—that will expedite the discovery and development of improved analgesic, anesthetic, addiction, and peripheral neuropathy treatments for the benefit of the public health.”

The purpose of the SCEPTER III meeting was to develop pragmatic, evidence-based guidance to clinical investigators who are designing, conducting, and reporting clinical trials evaluating sedation in mechanically ventilated adults in the ICU. Recommendations for key elements of study design, conduct, and reporting of sedation-related clinical trials in the ICU are offered to help facilitate comparison of studies of new agents, combinations, or protocols.

METHODS

A 2-day, in-person meeting was held in Washington, DC, on March 28–29, 2019. This meeting was followed by a modified Delphi consensus process (conducted online from February through June 2020) that focused on discussion points from the in-person meeting. This article reports on results of both the meeting and the modified Delphi consensus process.

Meeting

The meeting agenda and participant list were developed by a seven-member ACTTION/SCEPTER III steering committee, supplemental data 1 (http://links.lww.com/CCM/G350) and supplemental data 2 (http://links.lww.com/CCM/G351) present the steering committee membership and the meeting agenda, respectively. Participants were an international, interprofessional group of experts who had either attended prior SCEPTER meetings, published research involving sedation in adult ICUs, and/or were experts in clinical trial design, short- and long-term ICU patient outcomes, pharmacology, and/or statistics. Attendees included clinical, academic, patient, FDA, and industry representatives.

Prior to the meeting, participants were asked to review the Society of Critical Care Medicine’s (SCCM) 2018 Pain, Agitation/Sedation, Delirium, Immobility, Sleep Disruption guidelines (1), two prior SCEPTER meeting publications (10, 11), and the SCEPTER III agenda. Thirty-six participants from academia, industry, and the FDA with expertise in relevant content areas and two former ICU patients attended the in-person meeting which included formal presentations, panel-led discussions, and informal discussion time. Particularly noteworthy was the session devoted to the patient and family perspective. The presentation from an ICU survivor was followed by a panel discussion led by another ICU survivor (supplemental data 2, (http://links.lww.com/CCM/G351). Both ICU survivors are anesthesiologists who had explicit memory of their time in the ICU, providing a unique perspective. The meeting was audio-recorded and professionally transcribed (13).

Postmeeting Modified Delphi Consensus Process

Following the in-person meeting, in July 2019, an online survey Supplemental Table 1 (http://links.lww.com/CCM/G352) was sent to all participants to assess their perspectives on key points discussed at the meeting. Survey results, participants’ reactions to those results, written summaries of the meeting panel discussions provided by the panel moderators, and a review of the meeting transcripts formed the basis for generating questions for the modified Delphi consensus process. These questions were written by two authors (D.W., D.E.), neither of whom participated in the survey. The questions were pilot tested and refined based on feedback with several intensivists. This modified Delphi protocol was reviewed by the University of Rochester Research Study Review Board (Institutional Review Board) and determined to be exempt (Study00003771). Web based software (Mesydel, Seraing, Belgium; https://mesydel.com/en) was used to conduct each round of the modified Delphi process.

Within the modified Delphi process, a nine-point Likert scale was used for most responses, anchors were “Not Important” (score: 1–3), “Important but Not Critical” (score: 4–6), and “Critical” (score: 7–9). A “No opinion” option was also provided (14). The remaining questions required selection of a specific time interval. As determined a priori, a recommendation was considered to have reached consensus when greater than or equal to 70% of respondents rated the recommendation as “Critical” (score ≥ 7) and less than or equal to 15% of respondents rated the recommendation as “Not Important.” For questions with a time scale response, consensus was defined as greater than or equal to 70% of respondents agreeing on a specific response option. Recommendations reaching this definition of consensus were not included in subsequent Delphi rounds. In the first and second Delphi rounds, participants could include comments, which were anonymously shared with all participants as part of the subsequent round. After three rounds, questions with greater than or equal to 70% of responses as “Important but Not Critical” or “Not Important” were also noted.

Final recommendations for this report, summarized in tabular form, were developed based on the meeting discussions, panel summaries, transcripts, and the modified Delphi consensus process, with iterative refinement of the draft recommendations by participants of the SCEPTER III meeting.

RESULTS

Table outlines the recommendations for key elements of a clinical trial of a new sedative, combination of sedatives, or sedation strategy in critically ill adults who require mechanical ventilation and represents the combined results of the 2-day meeting and the subsequent three-round modified Delphi consensus process. These recommendations aim to enhance the consistency and comparability of future sedation trials. Supplemental Table 2 (http://links.lww.com/CCM/G353) reports results for each Delphi recommendation.

Key Recommendations—Study Development

Critically ill adults represent a heterogenous study population even in specialty-focused ICUs. Study eligibility criteria should be defined to select patients most likely to benefit from the proposed intervention while balancing potential limitations of a restricted patient population for trial enrollment as well as study generalizability. Whenever possible, measurement instruments should have evidence of validity and reliability in the proposed study population and setting and should be used in a manner consistent with such evidence. The lack of validated assessment tools for alcohol and opioid withdrawal in critically ill adults represents a particularly pertinent gap for sedation research.

The perspectives of survivors of critical illness and their family/caregivers should also be considered during the clinical trial design to ensure a patient-centered focus. The impact of critical illness on the patient cannot be separated from the impact on loved ones and family members. The perspectives of survivors and family/caregivers are unique and panels assisting with study development should include both.

Key Recommendations—Study Enrollment

Patients eligible for a sedation clinical trial should be enrolled as early as possible within the constraints of urgent clinical care, availability of research staff, and the need for informed consent. Although enrollment prior to, or within 24 hours of, sedation initiation is ideal, later enrollment may be consistent with the goals of a clinical trial in some circumstances. Management of both control and intervention groups should be consistent with accepted clinical practice for the use of sedation medications and a target sedation level (e.g., no sedation vs light sedation vs deep sedation). Since heterogeneity in the patient population is expected, recording a validated ICU severity of illness score at the time of ICU admission or patient enrollment is recommended.

Key Recommendations—Study Conduct

Many measures (e.g., of level of sedation) are based on scoring systems with a subjective component. Ideally, all such assessments are performed by trained study personnel. However, this goal may not always be feasible on evenings, nights, and weekends. If measures obtained by clinical personnel are used for research purposes, personnel training and quality assurance monitoring of the completeness, accuracy, consistency, and reproducibility of such measures, over the duration of the study, are recommended.

Key Recommendations—Outcome Measures

Important outcomes include both those occurring during the ICU admission and after ICU and hospital discharge. During the ICU stay, measures of sedation, pain, and delirium should be evaluated using valid and reliable instruments for the ICU setting. This recommendation does not limit use of additional novel scales or techniques (e.g., processed electroencephalogram) in the study so long as they do not compromise the use of validated measures. The times to clinically important outcomes (e.g., extubation, ICU and hospital discharge, etc.) should be reported.

Consideration should be given to evaluating patient-centered outcomes after hospital discharge (e.g., post intensive care syndrome including mental health, cognition, and functional mobility as well as chronic pain, quality of life, etc.), while recognizing that proven associations between ICU sedation and these outcomes is evolving. An existing core outcome measurement set (COMS) (14), designed for research studies evaluating postdischarge outcomes of acute respiratory failure survivors, was presented at the meeting. This COMS is recommended for use by both a National Heart Lung and Blood Institute working group on clinical research in adult pulmonary and critical care (29) and the American Thoracic Society/European Respiratory Society task force as part of postdischarge follow-up of acute respiratory failure survivors with coronavirus disease 2019 (COVID-19) (30). Instruments from this COMS project also have also been recommended for clinical outpatient use by a SCCM international consensus conference (31). After the SCEPTER III meeting, a separate set of recommendations for measurement instruments for outpatient clinical use in critical illness survivors was published (32).

DISCUSSION

An international group of interprofessional experts met to develop recommendations for the design, conduct, and reporting of clinical trials evaluating sedation in adults requiring mechanical ventilation. The goals of the meeting were to improve the quality and consistency of data generated, reduce methodological heterogeneity, and provide practical guidance for these trials. Herein, we discuss three themes that merit further elaboration: 1) incorporating views of surviving patients and/or families, 2) data collection quality assurance, and 3) need for methodological standardization.

With the increased call for patient-centered focus in clinical research (31), trialists should formally incorporate the perspectives of patients and families/caregivers and patient comfort into the design of clinical trials of ICU sedation. Since posttraumatic stress disorder, depression, and other mental illnesses occur in the caregivers as well as ICU survivors, it is important to include both of these groups in a panel that is involved in all aspects of the clinical trial (33, 34). The involvement of patient and family advisors has been considered critical in designing trials that are patient centered and whose results change clinical care practice and social support may change outcomes (34). In the United Kingdom, it is not possible to obtain government funding without effective patient and public involvement in all stages of the project (36). The Patient-Centered Outcomes Research Institute (32, 37), among other funding bodies, has noted both the difficulty and importance of creating community partnerships that reflect the diversity of the population to be studied and eventually treated based on trial results. To achieve improved health equity and reduced health disparities, we must strengthen the capacity to create partnerships with individuals and families living in diverse and underserved communities; better involve them in their care; and engage them in the improvement of care processes, interprofessional education, and research (38).

The acute and severe nature of illness commonly experienced by patients in an ICU generally precludes a priori discussion of research participation. Furthermore, delirium is commonly experienced by adults in the ICU and limits patient participation. During his recovery, one of the authors reflected on delusions experienced during prolonged critical illness (15) and developed a framing tool for the ICU teams caring for him. Gaining patient and family perspectives during the design of a clinical trial may offer valuable insights to optimally serve future patients. Specifically, careful consideration of the nature of ICU patients’ experiences may improve the ability of clinical trials to help answer patient-centered research questions, optimize enrollment, and improve patient and family satisfaction with trial participation. In particular, family members are pivotal to patient enrollment because they usually serve as the legally authorized representatives for ICU patients who commonly lack capacity for informed consent. Also, patients and families are an essential part of community engagement exercises that are required for potentially obtaining exemption from informed consent (39).

Assessments of pain, sedation, delirium and quality of life, using valid and reliable instruments, can measure important trial outcomes when completed by personnel trained in use of these instruments (40–44). For data collection around the clock, reliance on clinical, rather than research, personnel may aid with feasibility of frequent measurements; however, this reliance on busy clinical personnel has the potential to introduce error. Hence, appropriate training and quality assurance is recommended for all personnel who perform such assessments required for sedation trials.

Clinical trials often use a “usual care” control group are sometimes unblinded and may have outcome measures that are subjective or can be influenced by actions of the clinical team. Hence, it is important to have a clear definition of “usual care” at the study site hospitals, including consideration of this issue as part of the study site selection process and standardization of the control group (e.g., management of pain, management of agitation, extubation protocols). Furthermore, designing study eligibility criteria to minimize the time interval between the start of sedation and study enrollment can help reduce exposure to medications that may confound assessment of the study intervention on the outcomes.

Several limitations of this work merit comment. The specific recommendations may have been influenced by the expert panel membership. The meeting attendees have collectively published extensively on sedation and related subjects but cannot exhaustively capture all potential viewpoints. In addition, dominant voices or opinions within the panel may introduce bias. However, the panel members were purposely selected to provide expertise in the field, along with a wide range of experience and opinions, and there was facilitation of robust, but respectful, discussion and debate during the 2-day in-person meeting. The in-person meeting was specifically designed to allow ample time for both formal discussion during panel-led questions as well as informal discussions during the group social gatherings.

Although the Delphi process spanned the early months of the COVID-19 pandemic, the panel members did not feel the pandemic affected our recommendations. Clinical trials for sedation should be consistent across COVID and non-COVID populations. The observation that COVID-19 patients may require increased depth of sedation does underscore our recommendation that a severity of illness score be recorded for all patients entered into the clinical trial (45–47).

Furthermore, recommendations were refined using a postmeeting survey to help identify topics that deserved further exploration or clarification via an anonymous, three-round, online modified Delphi consensus process followed by iterative refinement and debate. For example, there was discussion during the meeting on the problems of defining deep, moderate, and light sedation, which was deemphasized after the meeting in favor of recommendations concerning the sedation measurement instruments and frequency of evaluation.

In addition, these recommendations may not be suitable for all ICU trials, and adaptation of these recommendations may be appropriate for unique aspects of trial objectives and design. Furthermore, our focus was on pragmatic recommendations with the potential to reduce heterogeneity in clinical trial design, conduct, and reporting. However, we realize some ICU sedation trials will want to incorporate novel features not considered in these recommendations that may still help advance the field.

CONCLUSIONS

An international group of interprofessional multidisciplinary experts met, discussed, and agreed upon 15 recommendations to assist with the design, conduct, and reporting of clinical trials of sedation of mechanically ventilated adults in the ICU. We view these recommendations as the beginning of further developments and processes, with the goals of improving and reducing heterogeneity in research methods used in clinical trials and facilitating comparisons of studies of new sedation agents, combinations, or protocols.

ACKNOWLEDGMENTS

We thank Alla Bazini, MD; Allison Lin, PharmD, PhD; Rigoberto Roca, MD; and Martha Van Clief, MD, Food and Drug Administration, Division of Anesthesia, Addiction Medicine, and Pain Medicine, Office of New Drugs for their participation in the meeting. We also thank Valorie Thompson and Andrea Speckin, Innovations Consulting Group LLC, Washington DC, for their invaluable assistance organizing the meeting and facilitating distribution of the article among the authors. We also thank Anna Zhao-Wong, MD, PhD, of Medical Dictionary for Regulatory Activities Maintenance and Support Service Organization, Yoanna Skrobik, MD, MSc of Université de Montréal and McGill University, and Diane Martire, MD, MPH, and Wing Yu Tang, MPH, Pfizer, Inc. New York, NY, for their participation.

TABLE 1.

Key Elements in the Design and Conduct of Clinical Trials of Sedation in Adult Mechanically Ventilated ICU Patients

Study design

The specific clinical trial design will depend on the goals of the study, with adaptive, pragmatic, and/or noninferiority designs as potential options.

The number of study sites, type of ICUs eligible for the study, and patient eligibility criteria, along with the rationale for these choices, should be explicitly stated in the study protocol.

A panel of survivors of critical illness and their caregivers should be consulted throughout the design of the clinical trial (15).

Study enrollment

The specific indication(s) for use of sedation in an enrolled patient should be recorded (4).

Patient enrollment should occur as soon as possible, and preferably no later than 24 hr after initiation of sedation.

A validated ICU severity of illness score (e.g., Acute Physiology and Chronic Health Evaluation, Sequential Organ Failure Assessment, Simplified Acute Physiology Score) should be recorded, preferably at the time of ICU admission or study enrollment (16–18).

Study conduct

All pain, sedation, and delirium assessments should be performed by personnel who are trained in use of the assessment instrument (19). Ideally, these measurements are done by research (rather than clinical) personnel. Quality assurance monitoring of the completeness, accuracy, consistency, and reproducibility of the measures, over the duration of the study is recommended.

The use of “rescue” medications (e.g., for patient agitation and pain) should be standardized via the study protocol, recorded, and reported.

Outcomes and measurement instruments

Achieving the target level of sedation may be a primary or secondary outcome or a protocol adherence measure.

The sedation level should be assessed at least every 4 hr using a valid and reliable scale (e.g., Richmond Agitation and Sedation Scale [20] or Sedation-Agitation Scale [21]). The Ramsay Sedation Scale is not recommended (22).  Pain should be measured prior to study initiation and at least every 4 hr thereafter using a valid and reliable scale (e.g., numeric rating scale in patients who can self-report pain and the Critical Care Pain Observation Tool (23) or Behavioral Pain Scale in those who cannot [24]).

Consideration should be given to treating pain to a prespecified score prior to any sedation assessment or administration of a sedative.

Delirium should be assessed at least every 12 hr using a valid and reliable scale (e.g., Confusion Assessment Method for the ICU or Intensive Care Delirium Screening Checklist [25–28]).

ICU and hospital mortality, length of stay, mechanical ventilation duration, and mortality at 30 d (and possibly up to 180 d) should be measured and reported.

If outcomes beyond hospital discharge will be assessed, a core outcome measurement set for acute respiratory failure survivors should be used (14).