Protocol for a stepped wedge cluster randomized quality improvement project to evaluate the impact of medical safety huddles on patient safety.

Introduction: Physician engagement is crucial for furthering patient safety and quality improvement within healthcare organizations. Medical Safety Huddles, which are physician-specific huddles, is a novel way to engage physicians with patient safety and may reduce adverse events experienced by patients. We plan to conduct a multi-center quality improvement (QI) initiative to implement and evaluate Medical Safety Huddles. The primary objective is to determine the impact of the huddles on adverse events experienced by patients. Secondary objectives include assessing the impact of the huddles on patient safety culture and physician engagement, and a process evaluation to assess the fidelity of implementation.
Methods: This stepped wedge cluster randomized study will be conducted at four academic inpatient hospitals over 19 months. Each site will adapt Medical Safety Huddles to its own practice context to best engage physicians. We will review randomly selected patient charts for adverse events. Generalized linear mixed effects regression will be used to estimate the overall intervention effect on adverse events. Process measures such as physician attendance rates and number of safety issues raised per huddle will be tracked to monitor implementation adherence.
Conclusion: Medical Safety Huddles may help healthcare organizations and medical leaders to better engage physicians with patient safety. The project results will assess the fidelity of implementation and determine the impact of Medical Safety Huddles on patient safety.

Quality Improvement

World Health Organization

Coronavirus Disease 2019

Funding

This work was supported by the Mount Sinai Hospital-University Health Network Academic Medical Organization Innovation Fund.

Trial registration

Clinicaltrials.gov, NCT03062670. Registered on May 3rd, 2022.

Introduction

According to the World Health Organization (WHO), adverse events due to unsafe care are one of the ten leading causes of death and disability in the world. As such, the WHO has declared patient safety a “global health priority” and called for “urgent action” to improve patient safety in healthcare [1]. The introduction of safety huddles (brief team meetings) is one strategy that has been used in a variety of settings to try to improve patient safety. Safety huddles are theorized to improve safety by increasing healthcare providers' situational awareness, allowing for the real-time identification and mitigation of safety concerns [[2], [3], [4]]. Huddles described in literature are generally led by unit managers or charge nurses and focused mostly on the nursing and interprofessional teams. Physician participation in team safety huddles can be challenging due to factors such as scheduling and perceived low clinical relevance [3,4].

However, physician engagement is crucial for furthering patient safety and quality improvement within healthcare organizations [5]. A pilot study [6] showed that physician-specific Medical Safety Huddles can successfully engage physicians and reduce adverse events, which are “events or circumstances which result in harm to a patient” [7]. Medical Safety Huddles are led by a medical leader and conducted with a standardized agenda over teleconference.

To better understand how Medical Safety Huddles can improve patient safety, we plan to conduct a multi-center quality improvement (QI) initiative implementing and evaluating the Medical Safety Huddles in new settings using a stepped wedge cluster randomized design. The primary objective is to determine the impact of the Medical Safety Huddles on patient adverse events. A process evaluation to assess the fidelity of implementation is the secondary objective. Our hypothesis is that the Medical Safety Huddles will reduce adverse events and be successfully implemented at new sites.

Methods and analysis

Setting

This project will be conducted at four sites in Toronto, Canada.

Hennick Bridgepoint Hospital has 446 inpatient beds devoted to rehabilitation, palliative care, and complex continuing care. At any given time, eleven hospitalists and one palliative medicine specialist staff each of the inpatient areas as most responsible physicians. On-site consultants include six physiatrists and four psychiatrists.

St. John's Rehab is a rehabilitation hospital consisting of 5 wards with 174 beds. Five hospitalists serve as most responsible physicians. Five physiatrists, two psychiatrists and one geriatrician act as consultants.

The Princess Margaret Cancer Centre's Medical Oncology and Hematology inpatient service has a total of 120 beds managed by approximately 14 hospitalists plus nurse practitioners, physician assistants, oncologists and hematologists.

The Orthopedic Trauma Service at Mount Sinai Hospital is staffed by four orthopedic surgeons. Approximately 10 patients are admitted at any time to the service.

Study intervention

Recognizing the importance of context in QI, each site adapt the Medical Safety Huddles to their own practice context to best engage physicians with the huddles. Key elements of the huddles maintained across all sites will include: use of a standardized script over teleconference by a huddle leader and short duration (5–20 min) to minimize impact on physicians' workflows. During each huddle, physicians will be invited to identify potential patient safety issues they had experienced or can foresee. While all patient safety issues can be reported, focus will be placed on safety issues of relevance to physicians. The standardized script, which is based on the pilot study [7], will specifically ask participants about safety issues related to the following categories: handover and transitions in care, coverage, access to consultation and diagnostic tests, infection control, medication-related, patient navigation, technology and equipment issues. The patient safety issues raised will be addressed in a collaborative fashion by the physicians in conjunction with the huddle leader and actions may be taken in response by the huddle leader or some/all of the service's physicians. A minute taker will record the patient safety issues reported and actions in response to any issues decided during the huddles so that issues could be revisited at subsequent huddles to ensure resolution. The four sites will vary in terms of their huddle frequency (i.e., weekly or biweekly), involvement of medical learners and managers, and selection of huddle leaders. Either a medical leader or a frontline physician will lead the huddles at each site.

Study design

This is a multi-centered stepped wedge cluster randomized QI study. The Model for Improvement will be used as the underpinning QI framework for the project [8]. Please refer to Appendix 1 for a flow chart detailing the study for the stages of implementation, data collection and data analysis.

The nature of this project is fundamentally QI – our primary goal is to improve the safety of medical care at the four sites. However, we are choosing to use a randomized design to reduce bias and better assess the impact of the Medical Safety Huddles. We are specifically using a cluster randomized design because the QI intervention targets hospitals or hospital services rather than individual patients or providers. All sites plan to implement the intervention and it would be impractical for the intervention to be with-held from sites. Instead of having parallel arms of no intervention and intervention, the stepped wedge design has an initial period where no clusters are exposed to the intervention, and then at regular intervals, a cluster or groups of clusters would cross over to the intervention arm until all clusters have crossed over. The stepped wedge design allows all participating sites to implement the medical safety huddles as the “intervention”. Secondly, we have limited human resources to support the implementation of the huddles. The stepped wedge design helps us focus our resources on one site at a time to maximize implementation success.

Stakeholder engagement will occur via meetings with medical leads and service physicians to understand existing quality and safety structures and barriers for implementation. A huddle leader(s) will be designated for each site after consulting with interested frontline physicians and the medical leader (ie. the Department Head or the Administrative Medical Lead). The Co-PIs, the huddle leader(s) and any frontline physicians interested in the huddles will adapt the huddle for their site's context and create implementation plans.

The Medical Safety Huddles for the four services will be implemented in a randomized, sequential manner (see Fig. 1) over the 19-month study period. Microsoft Excel's random number generator will be used to generate the implementation order. After the Medical Safety Huddles start to be held at each site, the huddles will be refined to the needs of the site based on feedback from huddle participants.

Fig. 1

The implementation schedule for the multi-centre QI project.

Data collection and outcomes

Primary outcome

The primary outcome measure will be the frequency of adverse events as measured by adverse events/1000 patient days. We will assess whether there are changes in the frequency, severity, types or preventability of adverse events within each of the sites, and aggregated across all four sites. Additional variables include age, gender, admission diagnosis, and medical complexity as measured by the Charlson Comorbidity Index as covariates [9]. Data will be collected by manual chart reviews of 12 randomly selected patients per month admitted to each of the four sites. A research assistant will use a random number generator to select the charts from the master lists of all patients admitted each month. All patients admitted during the study period were eligible to be included, with no exclusion criteria.To ascertain adverse events, we will use the chart review method for detecting adverse events. Four board-certified physicians who are blinded to both the nature of the project and the implementation schedule will review patient charts for adverse events. The chart reviewers will undergo a training phase of 30–40 charts during which interrater reliability will be assessed prior to starting the chart reviews for the study. Reviewers will use a procedural manual with standard definitions. Adverse events will be confirmed using the WHO definition of adverse events: ‘An adverse event is an incident which results in harm to a patient”(7). The type and severity of adverse events were determined based on the WHO International Classification for Patient Safety [7]. The category “unknown” is being added for severity because it may not be possible to ascertain severity if the patient is transferred to a different institution following the adverse event. Preventability will be rated on a six-point Likert scale - ranging from virtually no evidence of preventability to virtually certain evidence of preventability based on the Canadian Adverse Event Study [10].

Secondary outcome

The secondary objective is to conduct a process evaluation to assess the fidelity of implementation. Process measures such as physician attendance rates, number of safety issues raised per huddle and number of action items taken in response to safety issues will be tracked to monitor implementation success. The project team had chosen targets for implementation success based on the results of the pilot: 1) huddles held as planned during the intervention period, 2) an average of 60% attendance at the huddles and 3) an average of one issue raised per huddle and one action item taken in response.

Sample size determination

To estimate the required sample size for the primary outcome we used a Monte Carlo Simulation. The baseline adverse event rate and anticipated intervention effect were borrowed from earlier publications [6]; we assume 93.6 AEs/1000 patient-days pre-intervention and a 26.7% reduction7 in AE rate post-intervention.3 Patient-days (lengths of stay) at monthly chart census were generated using a zero-truncated Poisson distribution with a mean of 10 days; the number of adverse events is offset by patient-days in the statistical model, so we require at least one patient-day at the time of census during which an adverse event could occur. The adverse events were simulated from a Poisson distribution in two ways; firstly assuming no site level heterogeneity (i.e. using the above baseline AE rate and % reduction for all four sites), and secondly assuming site level heterogeneity in both the baseline rate (1-sd set to 10 adverse events/1000 patient-days) and in the percent reduction in the post-intervention phase (1-sd set to 5%). The proposed stepped wedge design (Fig. 1) was used as a template to determine how many months of data to simulate for each site during each intervention phase. In total there are 4 sites × 19 months = 76 planned monthly data points, of which 28 fall in the pre-intervention period and 48 fall in the post-intervention period.

While varying the number of charts sampled per month at each site from 5 to 15 in increments of 1, we simulated 5000 data sets (2500 assuming no heterogeneity and 2500 assuming heterogeneity, as above) at each proposed sample size, and used each data set in turn to fit a Generalized Linear Mixed Effects Regression (GLMER) model with a Poisson likelihood (see supplemental material). The p-value corresponding to the difference in the log-rate for the post-implementation period compared to the pre-implementation period was used to assess statistical power, with a p-value < .05 considered to be a successful detection of the intervention effect. The models included a fixed effect for the post-implementation period (the parameter of interest in the power calculation), site level random intercepts for the pre-period (to model baseline heterogeneity), and site level random intercepts for the post-period (to model heterogeneity in the intervention effect). Assuming no heterogeneity, a monthly census of 7 charts per site (532 total charts) is sufficient for 80% power. Assuming heterogeneity as defined above, a monthly census of 8 charts per site (608 total charts) is sufficient for 80% power. We make the conservative recommendation of 12 charts per month per site (912 total charts) which yields at least 90% power for both of the data generating processes considered. The simulation was performed using R Version 4.2.1 and the GLMER implementation in package lme4. The code for the simulation can be found in Appendix 2.

Data analysis

Statistical process control charts will be used to compare the frequency of adverse events pre and post huddle implementation at each program/service. They will be used as an exploratory tool to visualize adverse events so as to identify any temporal or seasonal special-cause variations (particularly related to CoVID-19 pandemic waves) that should be accounted for in the primary outcome model.

The primary outcome will be analyzed with the a similar model to the one used in the simulations, i.e. GLMER with Poisson Likelihood with a fixed effect for intervention phase and random effects to reflect potential baseline heterogeneity among the sites as well as potential heterogeneity in the intervention effect among the sites. We will use AIC to assess seasonality by comparing a model with and without a random intercept for each calendar month, with a decrease in AIC of 2 or more points indicating a preferred model. We will assess autocorrelation using a Box-Ljung test on the residuals for each site; if any one site yields a p-value less than 0.05 we will examine the ACF plots and specify an appropriate AR correlation structure in the final model. We are not planning to adjust the primary outcome model for patient level covariates, but are prepared to do so if our descriptive analyses of the demographic variables indicate substantial differences in case mix among the sites and/or the intervention phases.

Chart reviewers’ inter-rater reliability on adverse events will be calculated using Krippendorf's alpha [11].

Statistical process control charts will be used to compare the frequency of adverse events pre and post huddle implementation at each program/service. They will be used as an exploratory tool to visualize adverse events so as to identify any temporal or seasonal special-cause variations (particularly related to CoVID-19 pandemic waves) that would be included in the primary outcome model. A generalized linear mixed effects model will be used to assess the overall intervention effect (reported as a rate ratio and as a % reduction) on adverse events while accounting for site level heterogeneity.

Chi-square test and Wilcoxon's test will be used to compare the distribution of the types and severities of adverse events pre and post huddle implementation.

For the secondary outcome, descriptive statistics and run charts will be used to characterize the process measures – the number of physicians attending each huddle, the number of huddles held, the number of action items per huddle and the number of safety issues raised per huddle.

Ethical considerations and dissemination

This project was reviewed by the Quality Improvement Review Committee of University Health Network. The nature of the project was deemed as quality assurance/quality improvement, as defined in Tri‐Council Policy Statement V.2, and the project was provided with a Research Ethics Board(REB) exemption. The Sinai Health REB approved the study. At St John's Rehab, research ethics review was not required because the project met criteria for exemption from such a review based on institutional process for confirming that the project was deemed improvement in quality and not human subject research. The project was registered in the organization's QI registry. A waiver of patient consent for the chart review portion of the project was granted by the sites' respective institutional research or quality improvement boards and processes.

No external data monitoring will be used, as this project does not study a drug, biologic or device, and there are minimal risks from huddles.

Results will be disseminated to physician and healthcare leaders via conferences, publications, and social media.

Current project status

This project has faced delays and hurdles to implementation related to the CoVID-19 pandemic. To date, all four sites have implemented the Medical Safety Huddles according to the planned schedule. Data collection is ongoing.

Discussion

This project seeks to engage physicians through Medical Safety Huddles to promote patient safety. The Medical Safety Huddles may improve safety by supplementing the existing hospital patient safety structures with the missing medical perspective, promoting teamwork and communication, and increasing physicians’ situational awareness. Issues identified at Safety Huddles will be taken to leadership with the intention that systems solutions be implemented. As a result, we anticipate that the adverse events will decrease at sites that implement the huddles successfully, particularly for adverse events related to clinical administration (ie. issues with handover, transfer of care, admission processes) and clinical processes (ie. incomplete/missing/incorrect investigations, errors in diagnosis/treatment) that would be prevented the most through better teamwork, improved situational awareness and more engagement from physicians for identifying safety issues in their work environment. The project results will have implications for healthcare organizations and medical leaders hoping to engage their physicians with the organizational patient safety agenda.

In addition, while huddles have been widely endorsed as a mechanism to promote patient safety in hospital-based settings, the evidence for the use of any type of huddles in hospital settings is still largely anecdotal [12]. Few studies on the use of hospital-based huddles in general are controlled, multi-centered and report on patient safety outcomes. Therefore, this project on Medical Safety Huddles will also add to our nascent understanding of whether huddles in general contribute to patient safety in the inpatient setting.

Conclusion

Medical Safety Huddles may help healthcare organizations and medical leaders to better engage physicians with patient safety. We will implement Medical Safety Huddles at four sites, assess the fidelity of implementation and determine the impact of Medical Safety Huddles on patient safety.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Drs. Mark Bayley, Christine Soong, Jordan Pelc, Christian Fortin, Lawrence Robinson, and Meiqi Guo all receive stipends for leadership positions in our respective government-funded, not-for-profit hospitals. The other authors do not have any conflicts of interests to disclose.