"P3": an adaptive modeling tool for post-COVID-19 restart of surgical services.

OBJECTIVE: To develop a predictive analytics tool that would help evaluate different scenarios and multiple variables for clearance of surgical patient backlog during the COVID-19 pandemic.
MATERIALS AND METHODS: Using data from 27 866 cases (May 1 2018-May 1 2020) stored in the Johns Hopkins All Children's data warehouse and inputs from 30 operations-based variables, we built mathematical models for (1) time to clear the case backlog (2), utilization of personal protective equipment (PPE), and (3) assessment of overtime needs.
RESULTS: The tool enabled us to predict desired variables, including number of days to clear the patient backlog, PPE needed, staff/overtime needed, and cost for different backlog reduction scenarios.
CONCLUSIONS: Predictive analytics, machine learning, and multiple variable inputs coupled with nimble scenario-creation and a user-friendly visualization helped us to determine the most effective deployment of operating room personnel. Operating rooms worldwide can use this tool to overcome patient backlog safely.

LAY SUMMARY

The COVID-19 pandemic forced hospitals to cancel non-life-threatening surgeries to try to stem the spread of the virus. This led to many people on a surgical wait list. We needed a tool that would help us decrease this wait list as safely, quickly, and affordably as possible.

Using data from 27 866 patients (May 1 2018–May 1 2020) stored in the Johns Hopkins All Children's data warehouse and taking into consideration 30 variables relating to the functions and flow of the operating room suite, we built mathematical models for (1) time to eliminate the patient wait list (2), utilization of personal protective equipment (PPE), and (3) assessment of staff overtime needs.

The tool enabled us to predict the number of days needed to eliminate the patient wait list, amount, and type of PPE needed and staff/overtime needed. We were also able to compare different ways of reducing the wait list.

The tool utilized predictive analytics, machine learning, multiple variables, and a user-friendly visualization, and helped to determine the most effective deployment of operating room personnel. Operating rooms worldwide can use this tool to safely eliminate patient wait lists.

INTRODUCTION

The COVID-19 pandemic made its clinical appearance in March 2020 and affected our healthcare services in significant and rapidly changing ways. As the highly contagious nature and clinical severity of the SARS-CoV-2 virus became evident, and a statewide stay-at-home order followed, access for nonemergent conditions was restricted to avoid unnecessary harm to patients, families, and staff. Clinicians at our freestanding academic children’s hospital and ambulatory center began rescheduling elective operative procedures and ambulatory visits of low acuity starting on March 16, 2020. The concern that children were not getting needed medical care led us to the rapid implementation of a clinical triage system and telemedicine.

As the number of COVID-19 cases dropped and the stay-at-home order was scheduled to be lifted in mid-May, we recognized that restricting access to all but the emergently ill was easier than a rational, controlled re-opening to those who had been waiting and potentially getting sicker. Therefore, we started planning for the expected resumption of elective surgery. Faced with a backlog of 2100 elective surgical cases by May 4, 2020, we found ourselves in a highly complex situation with many variables.

We needed answers to the following questions:

How long would it take to clear the backlog with optimal operating room (OR) utilization?

Given that we normally operate at 80% utilization of ORs, how would extended hours affect the time to backlog reduction?

How do we create an efficient schedule with cases of varying duration, by surgical specialty and procedure type?

How do we know in advance how much personal protective equipment (PPE) to procure?

How will the need to socially distance prolong OR utilization per case?

How could we clear the backlog in a fiscally responsible way?

Our original process for answering these questions involved the use of Microsoft Excel (Microsoft Excel 2019. https://office.microsoft.com/excel, last accessed March 1, 2021) spreadsheets. However, given the multitude of variables, the data originating from various sources and the rapidly changing COVID-19 environment, we were concerned about data integrity and security and did not feel this method allowed us the creation and use of a “unique source of truth.” [1] We wanted to develop a tool that would automatically draw on existing data from various sources and “push” the information to us. We felt this would allow us to more nimbly manipulate the multiple variables to evaluate which scenarios would clear the patient backlog most quickly, with the greatest safety and efficacy, and at the lowest cost. We are here providing the readership without access to a data team the key features as well as limitations of our tool, enabling them to create an application like ours. Given that People, Place, and PPE were some of the biggest bottlenecks, we called the tool “P3.”

METHODS

Study setting

The Johns Hopkins All Children’s Hospital is a leading pediatric acute care and academic research hospital in St. Petersburg, Florida. The hospital has 259 beds and is affiliated with the Johns Hopkins University School of Medicine and the University of South Florida Morsani College of Medicine. With more than 400 000 visits and 8200 surgical procedures annually, the hospital relies on an integrated electronic health record system (EHR) system (Epic Care Systems, Verona, WI) to collect, access, and populate its data warehouse for large-scale data analysis. We are fortunate to have on site a sophisticated data analytics team who were able to create a prototype within 1–2 weeks. For this preliminary study, we used a subset of the data warehouse to analyze relevant variables that affected backlog and projected case volumes. These variables were selected for relevance by subject-matter experts in the fields of finance, clinical operations, and supply chain.

While we did not formally study whether our tool provided a safer and more effective way to plan than the use of spreadsheets, the reliability and real-time speed of the data as well as the user-friendly user interface enabled us to implement accurate changes very quickly.

Data sample

We used a data set containing 27 866 in-patient surgical and anesthesia encounters at Johns Hopkins All Children’s Hospital between May 1, 2018 and May 1, 2020. During that time, 714 different surgical procedure types were performed. Table 1 lists the variables used in the development of our analytic solution along with their description after extracting the data for the analysis.

Table 1.

Descriptions of data variables included in the analytic data set

Variable	Data type	Descriptiona
Case start date/time	Date/time	The day and time the case is scheduled to take place
Surgical specialty	Text	The scheduled primary surgeon’s specialty
Primary surgeon	Text	The name of the surgeon that has scheduled the primary procedure (if more than one surgeon participating in a single case)
Patient type	Text	Patient encounter type (inpatient, clinical, observation, same-day surgery, etc.)
Admit type	Text	Roll-up of patient type as either inpatient or outpatient
Scheduled primary procedure	Text	The primary procedure scheduled to be performed during the surgical case
Primary procedure	Text	The actual primary procedure performed (typically the same as the scheduled procedure but is sometimes modified intraoperatively to provide a more detailed description)
OR case number	Text	Unique identifier for the case
Patient financial number	Integer	Billing account number for the patient encounter
Patient medical record number	Integer	Unique identifier for the patient
Operating room	Text	Actual room where the procedure was performed
Location	Text	Department within the hospital where the procedure was performed (Main OR, Cardiovascular OR, Special Procedures Unit, Cath Lab, etc.)
Preoperative diagnosis	Text	The preoperative diagnosis for the primary surgical procedure scheduled/performed
Anesthesia type	Text	The type of anesthetic administered for the surgical case/procedure
Case start time	Date/time	The time the patient is scheduled to be in the operating room
Patient in room date/time	Date/time	The actual day and time the patient arrived in the operating room
Patient out of room date/time	Date/time	The actual day and time the patient left the operating room
Secondary procedure(s)	Text	Additional procedures performed in the operating room, subsequent to the primary procedure
Add-on indicator	Text	Indicates (Y/N) whether the case was an added on after the elective schedule was published
Patient age string (at time of surgery)	Text	Patient age displayed in years, months, and days
ASA class	Text	The ASA physical status classification system is a system for assessing the fitness of patients before surgery
Primary anesthesiologist(s)	Text	The names of the anesthesiologists that administered the anesthetic to the patient during the surgical case
Total surgery minutes	Integer	The calculation of minutes between the start of the surgical procedure to the end of the surgical procedure
Patient in-room minutes	Integer	The calculation of minutes between the time the patient was transported into the procedure room to the time the patient was transported out of the procedure room

ASA, American Society of Anesthesiologists; OR, operating room.

The final data set is the product of combining multiple reports extracted from our data warehouse and prepared reports from our electronic medical record. We needed to perform data cleansing and preprocessing, and create multiple lookup/reference tables, to achieve a reliable and valid data set. As part of the validation phase, we looked for outliers and missing values, and corrected any errors as part of the data preprocessing phase.

Mathematical functions

A three-step approach was used to solve our problem. First, we calculated the time required to clear up the backlog along with already scheduled cases (1. Time case projection). Second, we accurately estimated the use of PPE (2. PPE estimates), and finally, we determined the right combination of rooms and hours to help minimize staff overtime (3. Overtime optimization).

Time case projection

We chose to analyze the data set grouped by specialty and location as the most useful format for the presentation to hospital staff. We performed a series of initial calculations based on the average room time (minutes) and booking duration (minutes), as seen in equation (1), for both inpatient and outpatient cases, along with turn-over time (minutes), total case count, backlog, and budgeted cases (equation 2), to standardize the unit of measurement (minutes). We developed this initial set of functions to calculate the time to clear up the backlog of cases. The functions can be presented in minutes, hours, days, weeks, and months [equations (3) and (4)].

Additionally, we developed a set of scenarios that varied the number of rooms, running hours, and weekdays that surgery cases could be scheduled for use in “What-If Scenario” analysis. What-If analysis is the process of changing the values in a given set of variables to see how those changes will affect the outcome of formulas on a mathematical model.

PPE estimates

Based on 2 years’ worth of historical data, PPE needs were extrapolated by type of surgery and surgical team composition .To create a precise PPE estimate by PPE type (gowns, N95 masks, surgical masks, and shields), we used the available weekly PPE inventory issue reports (detailed by day) and calculated the weekly averages for the number of minutes per case along with the average PPE use per minute in the same week [equation (5)]. The assumption was that the next day inventory issues from Central Distribution was to replenish the previous day’s use in the perioperative departments. This process allowed us to obtain a coefficient per PPE type that was used to calculate the overall distribution for any given number of backlog cases.

Overtime optimization

Optimization is the process of maximizing or minimizing some function relative to some set, often representing a range of choices available in a certain situation. The function allows comparison of the different choices for determining which might be “best.” In the following section, we generalize our problem and try to optimize the following objective function (6):

In equation (6), x is number of rooms, y is number of hours, wr is penalization weight for increased number of rooms, and wr is penalization weight for increasing overtime beyond 10 h per day. The weight values can be adjusted to simulate different scenarios. For example, if we are more concerned about the number of rooms then we will increase the wr and vice versa. The bounds for the number of rooms (equation 7) and hours (equation 8) are based on our hospital’s capacity as shown below:

We constrained the objective function for optimization based on an overtime limit (ot) that could be calculated as seen in equation (9):

The methods were implemented by developing a Tableau-based application (Tableau Desktop 2020. https://www.tableau.com, last accessed March 1, 2021) that allows loading, data preprocessing, data visualization, and descriptive analyses. As for the optimization algorithm, because we had a nonlinear constrained optimization problem, we used Python’s SciPy library for Sequential Least Squares Programming. We applied these methods and tools to a 2-year data set of the Johns Hopkins All Children’s Hospital data sample with a set of selected features to estimate how long it would take to clear the backlog with optimal OR utilization and PPE allocation, and optimized factors emerging as the most critical for affecting OR overtime scheduling.

RESULTS

The key features of our tool were an interactive application, the ability to directly input variables and an intuitive, user-friendly visual interface.

This provided the user with estimated recommendations for backlog clearance time, PPE procurement, and OR overtime in real time. Changing the variables in the visual interface enabled rapid comparison of various complex backlog reduction scenarios and selecting the one most appropriate, while also making adjustments as the facts on the ground changed.

These backlog reduction scenarios took into consideration OR space, staff availability, and financial restraints.

The interface updates the data in real time and presents an estimate of the total number of days or weeks needed to clear the backlog based on OR and staff availability. Additionally, it has a surgical specialty filter that can be used to obtain a more precise estimate of time based on each of the hospital’s divisions such as Heart Surgery and Ear–Nose–Throat surgery that have very different turnaround times (Figure 1).

Figure 1.

Johns Hopkins All Children’s Hospital Post-COVID-19 tool for estimating patient’s volume needed to clear backlog of cases.

The application also allows users to explore and identify baselines, trends, variability, and data distributions for a complete data analysis with contextual significance.

Moreover, the application allows the user to calculate the estimated PPE usage, given the initial set of variable values used for time estimation. Figure 2 shows the distribution (%) profile for surgical masks, gowns, N95 masks, and face shields for the main OR. Figure 3 shows a slider tool for entering the different variable combinations (rooms, hours) to evaluate the possible overtime (%), with a maximum threshold of 50%.

Figure 2.

Johns Hopkins All Children’s Hospital Post-COVID-19 tool for operating room overtime optimization.

Figure 3.

Johns Hopkins All Children’s Hospital Post-COVID-19 tool for estimating personal protective equipment (PPE) and staff overtime (%) needed to clear backlog of cases.

DISCUSSION

With rising labor costs and the potentially significant health and financial consequences of resuming elective surgical procedures, a robust tool is required to account for all of the variables that affect resumption of the complex perioperative process.

Although some literature has been published on reopening surgical services in the setting of COVID-19, it has exclusively focused on the adult population, for which the cancellation rate for benign conditions was 81.7% in an international review, compared to 70.9% at our institution [2]. The authors of that study anticipated that clearing the backlog would require 45 weeks. To clear the backlog in a structured manner, others have devised mathematical models to quantify cumulative surgical backlog and predict time to clear [3]. Salenger et al [3] concluded that if only prepandemic capacity is available, the backlog would never clear. Another study that used Monte Carlo stochastic simulation-based analysis determined that the time to clear the backlog might take 7–16 months and that deferment of elective surgical cases would have a profound effect on the American healthcare system as a whole [4].

Our approach is novel in that we used a machine-learning tool with multiple variables. Predictive analytics, machine learning, and multiple variable inputs coupled with nimble scenario-creation and a user-friendly visualization create a powerful, evidence-based tool to make the best real-life data-based decisions. Our machine-learning implementation used an optimization algorithm (Sequential Least-Squares Programming) to help us predict staff overtime. This implementation is an excellent example of the practical use of predictive analytics in the healthcare space.

Our goals of creating a tool to enable data-based safe, prompt and affordable backlog reduction was met. We utilized the tool to determine best deployment of anesthesiology staff and to decide whether fewer ORs but longer days, more ORs and shorter days, or operating on weekends with overtime cost differentials would optimize the time to clear the backlog. End-users, in our case the anesthesiologist in charge of OR scheduling and staffing, utilized the tool to schedule the required space, composition of operating teams, timing of procedures as well as supplies including PPE in advance, and felt that the tool allowed more rapid decision making using the most current variables, to the previous method of spreadsheets. The greatest benefit was the ability to toggle between various scenarios by modifying different parameters, helping to make decisions that optimized resource utilization and speed of safe backlog reduction. We were also able to evaluate the financial viability of different scenarios to reduce backlog (is extending operating hours into the evening financially acceptable, and by how much does that reduce the backlog?). It is this ability that sets the P3 tool apart and enabled us to do scenario planning of a sophistication and ease not previously possible on spreadsheets alone.

While we were able to decrease our backlog, it is difficult to assess the real impact, given the lower surgical volumes during the pandemic. The definitive amount of backlog reduction will have to be calculated once volumes return to pre-COVID baseline, which we look forward to doing. We do know that at times of high pressure with long patient backlogs, our tool enabled us to compare scenarios, make more rapid decisions, and based on more variables.

COVID-19 is not behind us, as the rising number of cases nationwide and internationally demonstrate. We have to anticipate a second surge, complicated by an already existing backlog, necessitating another shut-down followed by another reopening of clinical services. If not adequately addressed, every reopening will generate a larger backlog to the point of unmanageability.

We are not the only ones to see the need for a specific pandemic preparedness plan [5]. We presented our tool in draft form to the Hospital for Sick Children in Toronto, which then built its own tool around our core. We are planning to collaborate with clinicians there to demonstrate the tool’s utility in a multicenter study.

CONCLUSION

The rising complexity and cost of healthcare, particularly for large multihospital systems, coupled with the increasing likelihood of pandemics and natural disasters, underlines the critical importance of rapid and accurate decision making. We recommend adopting machine learning and predictive analytics tools as standard practices in the healthcare industry of the 21st century.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

AUTHOR CONTRIBUTION

Name: Divya Joshi, MD, MMM, FAAP, FACPE, CPE

Contribution: This author helped design the study, collect the data, analyze the statistical data, prepare the manuscript, and accept the final manuscript.

Name: Ali Jalali, PhD

Contribution: This author helped design the study, collect the data, analyze the statistical data, prepare the manuscript, and accept the final manuscript.

Name: Todd Whipple, MBA, RN

Contribution: This author helped design the study, collect the data, analyze the statistical data, prepare the manuscript, and accept the final manuscript.

Name: Mohamed Rehman, MD

Contribution: This author helped design the study, collect the data, analyze the statistical data, prepare the manuscript, and accept the final manuscript.

Name: Luis M. Ahumada, MSCS, PhD

Contribution: This author helped design the study, collect the data, analyze the statistical data, prepare the manuscript, and accept the final manuscript

CONFLICT OF INTEREST

The authors have no conflict of interests to declare.

DATA AVAILABILITY

The data underlying this article will be shared on reasonable request to the corresponding author.