Advanced practice providers versus medical residents as leaders of rapid response teams: A 12-month retrospective analysis.

PURPOSE: In a time of worldwide physician shortages, the advanced practice providers (APPs) might be a good alternative for physicians as the leaders of a rapid response team. This retrospective analysis aimed to establish whether the performance of APP-led rapid response teams is comparable to the performance of rapid response teams led by a medical resident of the ICU.
MATERIAL AND METHODS: In a retrospective single-center cohort study, the electronic medical record of a tertiary hospital was queried during a 12-months period to identify patients who had been visited by our rapid response team. Patient- and process-related outcomes of interventions of rapid response teams led by an APP were compared with those of teams led by a medical resident using various parameters, including the MAELOR tool, which measures the performance of a rapid response team.
RESULTS: In total, 179 responses of the APP-led teams were analyzed, versus 275 responses of the teams led by a resident. Per APP, twice as many calls were handled than per resident. Interventions of teams led by APPs, and residents did not differ in number of admissions (p = 0.87), mortality (p = 0.8), early warning scores (p = 0.2) or MAELOR tool triggering (p = 0.19). Both groups scored equally on time to admission (p = 0.67) or time until any performed intervention.
CONCLUSION: This retrospective analysis showed that the quality of APP-led rapid response teams was similar to the quality of teams led by a resident. These findings need to be confirmed by prospective studies with balanced outcome parameters.

Introduction

Hospital medicine is dealing with patients with increasingly complex disorders that require a highly efficient and high-quality healthcare organization [1, 2]. Rapid response systems with teams led by physicians have been shown to reduce in-hospital cardiopulmonary arrests and mortality [3, 4]. However, the organization of these rapid response systems is subject to the worldwide emerging shortages of physicians, especially in rural areas [5, 6].

These shortages force numerous hospitals to reorganize their rapid response systems and other teams in order to be able to continue to provide a 24/7 coverage.

One option that has been considered is that a rapid response team might be led by different health care professionals, ranging from attending physicians to nurses. Limited scientific evidence suggests that teams led by a physician perform better than teams led by non-physicians [7, 8]. In practice however, an increasing proportion of in-hospital acute and emergency care is delivered by junior clinicians in the first years of their training, including the responsibility of leading a rapid response team, which might reduce the efficacy and quality of these teams. One of the potential solutions is to reallocate this responsibility to physician assistants or nurse practitioners, also called advanced practice providers (APP). This profession is gaining recognition in critical care which is supported by clinicians who recognize their quality and continuity of their work [9, 10].

Very few experiences with APPs as leaders of a rapid response team have been reported [11, 12].

Two previous studies provided some guidance about the outcomes of patients visited by a team led by an advanced practice provider, but inter-comparability is hampered by differences between the considered health care systems and by the lack of validated outcome parameters. A third, retrospective single center study comparing outcome data of rapid response teams led by a nurse practitioner and by a registrar showed an improved hospital mortality in the nurse practitioner -led group after propensity matching. This study mainly reported patient outcomes [13].

The main objective of the present study was, to establish whether the performance of APP-led rapid response teams is comparable to the performance of teams led by a medical resident of the ICU, focusing on process- as well as patient-related outcomes.

Methods

Study design and setting

We performed a single-center retrospective cohort study over a period of 12 months. This time period was chosen to reduce the influence of confounders, such as changes within the organization of the hospital and the ward, for instance the implementation of a completely new operational Electronic Medical Record (EMR) in the hospital or the use of continuous monitoring devices within certain departments. The study was performed in the Catharina Hospital Eindhoven, a Dutch tertiary teaching hospital which houses all specialties except for neurosurgery and transplantation surgery. The hospital has a 33-bedded ICU, which facilitates a mix of post-operative cardiac and oncologic surgery and, on the other hand, specialties such as, neurology, pulmonology and gynecology and internal medicine, including dialysis. During the period of the study, clinical protocols regarding the rapid response system remained unchanged.

Patient selection

The EMR was queried to identify patients who had been assessed by the rapid response team. Patients and assessments were eligible if a bed-side assessment had been performed by the rapid response team in patients aged 18 years of older. Medical consultations by telephone were excluded.

Rapid response team triggering, modified early warning score and the MAELOR tool

On the wards, the modified early warning score is used as described by v Galen et al. [14] to identify deteriorating patients or patients in need of advanced care. In short, this tool assigns points to abnormal physiological parameters and in turn triggers a rapid response team call. It also provides the opportunity to call in support if there is a sense of unease about the condition of the patient.

The MAELOR tool is a validated tool to measure and quantify the performance of a rapid response team [15]. This tool consists of a flow chart which is triggered if the patient has a high modified early warning score and stops triggering if a patient is admitted within 4 hours after the initial call or has a resolution of critical clinical symptoms within 48 hours. The tool also stops triggering if there are treatment limitations and if ICU admittance is not warranted.

The MAELOR tool flow diagram is depicted in Fig 1. Clinical variables necessary for the MAELOR tool were only recorded until 48 hours after the initial call.

Fig 1

Flow chart MAELOR tool.

Rapid response team organization

The organization of the local rapid response system has been described in the COMET study, a multicenter study that evaluated the implementation of structured rapid response systems in the Netherlands [3]. Since implementation of the rapid response team in the Catharina Hospital Eindhoven in 2013, These teams have consisted of an ICU nurse and a team leader, the leader being either a medical resident working on the ICU or an advanced practice provider. One of the medical residents or advanced practice providers on the ICU manages the pager for the rapid response team during duty. A call to the rapid response team can be made if a patient on the ward scores a modified early warning score of ≥ 3 points or if a nurse experienced a substantiated sense of worry about a patient. This call can either be made by medical residents or by registered nurses on the general ward. Usually, when the modified early warning score indicates a critically ill patient, the nurse on the ward informs the resident, who in turn decides if a rapid response team assessment is necessary. The rapid response team call is postponed if the cause of the high early warning score is known and additional treatment, such as an operation, has been planned or if a sepsis can be treated on the normal ward.

The team carries a basic set of materials, which consists of resuscitation fluids, masks for supplemental oxygen and lifesaving medication, such as glucose or phenylephrine.

Following the first assessment, the leader of the rapid response team discusses the case with the intensivist on call for the ICU. This consultation results either in ICU admission or in recommendations for treatment on the general ward, which can include changing treatment limitations, such as the “do not resuscitate” (DNR) code or the “do not intubate” (DNI) code and other treatment limitations. If it was decided that the patient is to be admitted to the ICU, the rapid response team transports the patient and admits him or her to the ICU. After the admission to the ICU the leader of the rapid response team remains responsible for the care of the patient until the end of his duty, together with a dedicated ICU nurse. Shortly after ICU admission, the intensivist visits the patient in the ICU. The hospital has a separate team for non-ICU-related in-hospital cardiac arrests. In case of a cardiac arrest the rapid response team is involved if the patient experiences a return of spontaneous circulation and will be admitted to the ICU or the cardiac care unit (CCU).

Advanced practice providers and medical residents

The advanced practice providers who work in the ICU are qualified as physician assistants. They received a 2.5-year training in the medical domain, which grants them a master’s degree, and after graduation, they are qualified to perform all ICU tasks autonomously. The APPs all worked as ICU-nurse before their training to become an APP. They work in collaboration with intensivists. The medical residents originate from the following disciplines: internal medicine, cardiology, pulmonology, and surgery. The medical residents attend an internship during a period ranging from 3 months to 1 year.

Shifts

The ICU ward uses a system of rotating shifts with a minimum of four clinicians in each shift (advanced practice providers and medical residents). Usually, six of these clinicians are present during the day shift. During the evening shift, two or three of these clinicians are present, and during the night shift, two. The number of FTE in the entire group of advanced practice providers is 4.99, and in the MR group it is 10.69.

Ethics

Approval for the study was obtained from the national and regional Ethics committee in accordance with Dutch and European legislation (Medical Research Ethics Committees United (MEC-U); W17.095). A local applicability permission was obtained separately. This article was prepared using the Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines [16].

Data

Data of all the consecutive rapid response team visits over a period of 12 consecutive months (2017–2018) were retrospectively extracted from the hospital data base. The patient variables collected were age sex, diagnosis, hospital admission, discharge data, death, Apache IV score on admission and after 24 hours, all blood samples before and after the rapid response team consultations and treatment limitations. Additionally, the composition of the team was noted.

Since there are no mandatory rules for the composition and organization of a rapid response team, the performance and efficiency of different teams are difficult to measure and compare. We gathered several parameters to measure the efficiency of the process. The parameters were categorized in three groups. First, the patients’ outcome data: length of stay, mortality, and if applicable, treatment limitations. Second, parameters of team performance: to measure team performance, we used the time until change in early warning system score together with the time until various interventions: the time between the consultation of the rapid response team and the arrival on the ICU, the time between the consultation and interventions such as central or arterial catheterization or intubation. Arterial and central venous lines can be inserted both by residents and by APPs. Intubations by residents are performed under supervision. Third, the MAELOR tool, a validated instrument to assess the performance of rapid response teams was scored.

Statistics

The data were analyzed with SPSS statistical package version 25 (IBM corporation, Armonk, NY, USA). Means are reported with standard deviations for normal distributions and medians with interquartile range are reported for other distributions. Parameters which were recorded once every hour were considered continuous. Categorical independent variables were compared using the Chi-square test with Yates continuity correction. Categorical and continuous independent non-parametric variables were compared with the Mann Whitney U test, and for the independent parametric variables were analyzed with the independent samples t-test.

Categorical variables with two continuous variables at different points were compared using the mixed between-within subjects analysis of variance was used. A p-value of p< 0.05 was considered statistically significant.

Results

All 454 consecutive rapid response team calls during the assessed period were included in the analysis. Because not all patients received every treatment that was assessed in this study, data on antibiotic change, central venous access, arterial catheters and intubation were not available for all patients.

The team was led by an advanced practice provider in 179 cases and by a medical resident in 275 cases. Of the 454 rapid response team calls, 296 resulted in the patient being admitted to the ICU. This represents approximately 10% of the total yearly ICU admittances. The percentages of rapid response calls resulting in an admission to the ICU were comparable between teams led by an APP and those led by a medical resident (118 (65%) vs 178 (66%), p = 0.78). The level of experience of the APPs was a median of 6.25 years (3.33y-8.25y). In general, an APP handled twice as many calls as a MR.

Table 1 presents the baseline characteristics of patients assessed by APP-led teams and by teams led by a medical resident. Most patients were assessed in the emergency department and on the internal medicine ward. The APACHE IV score of the patients indicates a high severity of illness. No significant differences were found between these two groups except for diastolic blood pressure, which was significantly higher in patients assessed by MR-led teams.

Table 1

Patient characteristics on arrival of the rapid response team.

Data are given as numbers with percentages or as medians with IQR.

	Leader of rapid response team		p-value
	APP Median (IQR) n = 179	MR Median (IQR) n = 275
Age (years)	68 (56–76)	70 (58–78)	0.19
Sex (male)	99 (55%)	168 (61%)	0.26
Sex (female)	80 (45%)	107 (39%)	0.26
Apache IV predicted mortality	58 (42–86)	62 (37–76)	0.89
Temperature (degrees Celsius)	37.3 (36.9–38.5)	37.4 (37.0–38.4)	0.80
Systolic Blood Pressure (mmHg)	120 (99–140)	128 (109–151)	0.06
Diastolic Blood Pressure (mmHg)	69 (50–75)	70 (60–80)	0.03
Pulse (rate/min)	108 (86–124)	100 (85–119)	0.28
Respiratory Rate (rate/min)	25 (18–30)	20 (16–30)	0.37
Location of outreach for ICU-admitted patients	APP Number (%)	MR Number (%)
Surgery	18 (6.1%)	26 (8.8%)	1.00
Internal Medicine	21 (7.1%)	24 (8.1%)	0.40
Cardiology	4 (1.4%)	6 (2.0%)	NA
Pulmonology	14 (4.7%)	24 (8.1%)	0.82
Cardiothoracic surgery	7 (5.9%)	2 (1.1%)	NA
Neurology	2 (0.7%)	8 (2.7%)	NA
Gastroenterology	7 (2.4%)	7 (2.4%)	0.61
Emergency department	38 (12.8%)	73 (65.8%)	0.16
Other	7 (5.9%)	8 (4.5%)	NA

APP: advanced practice provider, MR: medical resident, IQR: interquartile range

Both the patient- and the process-related outcomes are described in Table 2.

Table 2

Outcome variables.

	Leader of rapid response team		p-value
	Response by APP: N(%), Med (IQR)	Response by MR: N(%), Med (IQR)
Number of calls	179	275
Admission ICU	118 (66%)	178 (65%)	0.87
Time to ICU (hours)	1.19 (0.56–1.75)	1.16 (0.59–1.75)	0.67
Within 24h:
Hospital mortality	13 (7%)	17 (6%)	0.80
ICU mortality	6 (5%)	10 (6%)	1.00
Time to insertion arterial line (hours)	1.68 (0.87–2.94)	1.54 (0.78–2.72)	0.50
Time from visit to insertion Central venous catheter(hours)	2.17 (1.24–5.78)	1.71 (0.92–3.30)	0.30
Time from visit to intubation (hours)	3 (1.5–16)	2 (1.07–10.50)	0.24
MAELOR not triggering anymore (good outcome)	165 (92%)	261 (96%)	0.19
MEWS admission	4.04 (2.03–6.29)	3.92 (1.98–6.56)	0.90
MEWS at 24 hours	2.13 (1.04–3.46)	1.63 (0.51–3.06)	0.12
Δ MEWS between leaders			0.20
Change in antibiotics	27 (30%)	40 (29%)	1.00
Time to change of antibiotics (hours)	1.33 (0.62–2.25)	1.40 (0.65–2.30)	0.77
ICU LOS (days)	1.00 (0.20–2.79)	1.10 (0.17–3.43)	0.80
Treatment limitation initiated (%)	54 (30%)	72 (26%)	0.44

MEWS: Modified Early Warning Score, APP: advanced practice provider, MR: medical resident, IQR: interquartile range

The baseline characteristics of the patients assessed by teams led by advanced practice providers and medical residents demonstrate that no statistically significant differences were encountered between the groups except for diastolic blood pressure. The differences in blood pressure were very small and are therefore considered clinically unimportant.

Concerning the validated MAELOR tool, we were able to retrieve the MAELOR tool outcome in 451 of the 454 cases. Three cases were excluded due to insufficient data. In the analysis of the MAELOR tool outcomes, no significant differences were found between patients treated by teams led by an advanced practice provider or by a medical resident.

Since the rapid response team leader in our ICU model remains responsible for the care provided to the admitted ICU patients, the efficiency of the team could be reflected in the time until arterial line insertion, the time until central venous catheter insertion and the time until intubation. These variables did not differ significantly between patients attended to by a team led by an APP or by a resident (Table 2).

To determine the efficiency of non-technical procedures, we also compared the times until change of administered antibiotics in the first 12 hours after admission to the ICU. In the 296 patients admitted to the ICU, the time until change of antibiotics after ICU admission was not significantly associated with the rapid response team leader being an APP or a resident (Table 2). In addition, no association was found between the leader of the rapid response team being an APP or a medical resident and the time until antibiotics administration after ICU admission.

The ICU LOS was determined for 283 of the 296 ICU admitted patients. No significant difference in ICU LOS was found between the patients attended to by APP-led teams and those attended to by rapid response teams led by a medical residents.

The early warning score was assessed at two different points in time (on admission and after 24h). There was no significant difference between the rapid response team leader and the early warning score, or the reduction in early warning score after 24 hours.

Of the 158 patients who remained on the general ward after the rapid response team visit, we were able to extract the early warning score after 24 hours for 123 patients. The warning scores after 24 hours and the reduction in warning scores did not differ between the APP-led teams and the resident-led teams.

In 126 out of 452 patients new treatment limitations were applied after the rapid response team visit. There was no significant association between the instigation of treatment limitations and the rapid response team leader (Table 2). In addition, no effect of rapid response team leadership on mortality was not found in the patients who were deceased on the ward or in the ICU within 24 hours after the RRT visit.

Discussion

The present study provides insight into the performance of APPs as leaders of rapid response teams in direct comparison with medical residents. In this retrospective study, we found no differences in either process-related or patient-related outcomes between rapid response teams led by APPs and teams led by medical residents. This comparability included the trend of the early warning score after the call and the triggering of the MAELOR tool 48 hours after the call, a tool validated for assessing the quality of rapid response team assessments [17].

To measure patient- and process-related outcomes, we used a wide variety of parameters, ranging from the standardized measurement tools that were validated for these rapid response team assessments to the times until interventions and general outcome data. Moreover, the environment and organization of the rapid response teams we assessed are in line with those in a multi-center trial that established a standard deployment of the rapid response teams which reduced in-hospital morality rates [3]. In our study, this organizational structure was considered efficient, based on the high number of calls that resulted in ICU admissions (60–70%) suggesting an effective afferent limb (detection), and on the relative fast reaction time as a parameter for the efferent limb (response) [18].

The absence of significant differences in outcomes between teams led by APPs and teams led by medical residents suggests that APPs are suitable alternatives for medical residents in leading rapid response teams. This finding is also supported by the higher number of calls handled per FTE by the APPs compared to the number of calls handled per FTE of medical residents of the ICU: one FTE APP handled approximately 2.5 times more calls than one FTE medical residents. An explanation for this substantial difference might be the continuity of care that APPs provide. This continuity is established by the APPs’ continuous coverage in most shifts together with their presence alongside the residents. This continuity of care probably also explains the observation that APPs more easily decide to respond to the calls than the rotating medical residents, who regard these calls as stressful events (personal communication).

The outcomes of this study are in line with the outcomes of the few previously published studies, although the settings of those studies were different, which makes a sound comparison is difficult. One study showed no differences in quality after the introduction of a nurse practitioner as leader of a rapid response team [12]. The study compared a situation with and without a rapid response team, but in the group of patients treated by a rapid response team, the nurse practitioner was not always available, and deployment restrictions for the rapid response team were in place. Additionally, another publication reported an shorter time until admission to the ICU in a APP-led rapid response team [11]. Despite this improvement, however, the time span was much longer than the 4 hours suggested to be adequate by the MAELOR tool. Also our results are in line with the findings of a recent retrospective single-center analysis. This study compared an APP-led rapid response team with a resident-led rapid response team. This study found no differences between the groups except for a shorter in-hospital stay of the patients visited by the APP-led group but after propensity matching [13]. The study mainly focused on patient outcomes and less on process outcomes.

In accordance with this study, there is growing evidence that APPs are a valuable substitute for a physicians as leaders of a rapid response team.

Although the implementation of an RRT in this study is the general method used in the Netherlands, comparability between countries and healthcare systems remains cumbersome because different RRT models that are used [3]. Apparently, so far, an optimal composition and implementation of a rapid response team has not yet been established [8, 19]. This fact and the lack of validated measurement tools except for the MAELOR tool probably explains why the literature reports different success rates and struggles with aligning outcome data when reporting on the performance of rapid response teams [3, 8, 13, 20, 21].

The main limitations of the present study are the retrospective design and the single-center setting. Before extrapolating our results to other hospitals, the local situation should be considered. Another limitation is that patients might have been missed if they did not fit our query, or were not registered in the database. However, we know that the database is used consistently and that registration is a central part of the workflow for all APPs and medical residents. It is therefore unlikely that selection is a major bias of this study. Another important limitation is that the patient- and process-related outcome measures were chosen arbitrarily, although they are clinically relevant. When focusing on time before intubation for example, there is a difference between the groups in time from admission to intubation without reaching significance. This is probably caused by the fact that there are a lot of oxygen therapies available which can be applied as initial treatment and if the included sample size is too small, significant differences in patient-related outcomes are difficult to detect. In addition, the outcomes are affected by many other variables related to the patient, the pathology, and the organization of the ward and the ICU. Especially, an ICU nurse may have a substantial influence on the rapid response team’s performance. Necessary critical steps such as oxygen administration or positioning of the patient to enable adequate breathing are steps often overlooked by junior clinicians. Even simple treatment recommendations provided by the nurse, can be very valuable for junior clinicians. These steps are often independently covered by the accompanying ICU nurse.

Regarding the use of the MAELOR tool, this tool was validated to evaluate the performance of a rapid response team. However, the acquaintance of researchers with this tool is limited and therefore its use to compare between studies is limited as well. In addition, the added value of APPs might have been underestimated because certain benefits of APPs might have been missed, such as patient satisfaction, communication skills, team guidance, situational awareness, and other non-technical skills. This same issue has been addressed in the literature before [22]. Prospective cohort studies are therefore needed to confirm the outcomes of the present study and to assess the potential additional benefits of APP-led rapid response teams.

Conclusion

In this observational retrospective single-center study on process- and patient-related outcome parameters of RRTs led by APPs and MRs, we have shown that APPs perform at least as well as MRs in leading a rapid response team. As the performance of rapid response teams is influenced by the organization of healthcare systems prospective studies in other institutions are needed to confirm our findings.

(XLSX)

Click here for additional data file.

19 May 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Hugh Cowley

Senior Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. "In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

"Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors present a retrospective observational study from a single tertiary centre in the Netherlands. They have two types of team leaders for rapid response calls, junior residents compared to physicians assistants or other trained staff. They report patient outcomes following these emergencies as well as some care access measures such as time to central line placement. Both groups of patients seem to have similar outcomes, although the patient numbers are very small for this type of study.

The study is largely well written. It is an important topic and it is good to see academic pilot exploration of the concept. It will be a good addition to the international literature on critical care delivery models in hospitals.

I would like to see data regarding the levels of training and experience of the two groups of providers (particularly the non-medical team leaders). When other sites consider rolling-out this staffing model, it will be important for them to understand that the response team is led by people who work full time in the ICU and have done for years. This is not likely to be the same resource available to rural centres, limiting the generalisability of the study, hence the importance of stating the setting of this study. (Also add a generalisability comment in your discussion).

I have a few minor suggestions for improvement of the manuscript:

# I recommend adding the time frame of your study to the abstract and methods.

# Data are pleural, this should be changed throughout the manuscript.

# The results should start with a participant flow diagram or statement. It is not clear if all rapid response team calls were included in your study at the moment.

# There is a sentence/section referring to the FTE of response by various healthcare workers to calls - this can be deleted, the gross numbers and percentages are enough. I think you are trying to convey levels of experience in these comments, which would be better dealt with another way.

# I recommend light editing throughout the manuscript by a native English speaker - the manuscript is well written but there are minor phrase and spelling issues throughout. eg "de" instead of "the"

# Table 1 should start with the numbers of patients in each column (n=xx)

# A lot of your narrative paragraphs in your results repeat data in Table 2 and could be deleted.

# Whilst you didn't reach statistical significance, your physicians assistants seem to be slower at initiating procedures in your ICU. This might be worth a comment. Some of the times are an hour or so longer - you seem not to have an adequate sample size if this doesn't reach statistical significance.

# if word counts are an issue, almost all sections could be more concise.

Reviewer #2: This paper compares rapid responses of teams led by a medical resident and by an advance practice provider. It is an interesting approach to broaden and valide skills of rapid response team leaders, based on the MAELOR tool. It does not however consider the role of the ICU nurse who is also part of the team, nor of the prior experiences of the leaders. These should be mentioned in the limitations. The accompanying nurse can play a major role in ensuring the quality of patient management: it is common for junior doctors to ask experienced nurses for their opinions and support, for example, and not considering them at all in the team's performance seems a bit of an oversight.

Overall, the presented analysis seems appropriate and the results are discussed with pertinence. It is interesting to consider making RRT leader specialists, since medical residents can have varying levels of expertise. It will be however important to ensure that medical residents continue to be exposed to and train for emergencies during their postgraduate training.

The are minor revisions needed for english (including : line 152, experienceS; line 161, no plural for advice; line 205, what is the word "en" ; line 216, correct continues for continuous; the paragraph 274-279 needs revision: the first sentence has no verb, the third is difficult to understand). The section in lines 263-266 are a repetition of the information ind lines 247-250.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Katherine Blondon

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

15 Jun 2022

Dear Editor,

We thank the editor for the opportunity to revise our manuscript. We would like to offer the revised manuscript: Advanced Practice Providers versus Medical Residents as Leaders of Rapid Response Teams: a 12-month retrospective analysis to the editor and the reviewers for reconsideration.

We changed the title and used the designation rapid response team instead of critical care outreach team because this is the designation used throughout the text.

To address the requirements of the Journal we consulted a scientific English writer for proper scientific writing. Regarding the other remarks:

1. We addressed the data Availability Statement by adding our revised database to the ‘supplementary materials’. We deleted columns which could make the APPs, residents or patients traceable to make the database privacy proof.

2. We addressed the remark about style by changing the headings to the required fonts and by changing the file naming of the figure.

3. We addressed the contributorship by adding the statement that all authors contributed equally to this work. There was no difference in effort. If different symbols in the author list are still required we will be happy to change this. We were not able to differ between the authors based on level of contribution because it was a team effort.

The several valuable comments of the reviewers were addressed in the uploaded file named: rebuttal.

For thoroughness we also copied the answers below.

Kind regards,

Herman Kreeftenberg

Rebuttal:

We thank de reviewers for their valuable feedback on the manuscript and we consider the requested changes as very helpful. To address the language and scientific writing we sent the manuscript to a native English scientific writer for improvement.

Comments reviewer #1:

1. I would like to see data regarding the levels of training and experience of the two groups of providers (particularly the non-medical team leaders). When other sites consider rolling-out this staffing model, it will be important for them to understand that the response team is led by people who work full time in the ICU and have done for years. This is not likely to be the same resource available to rural centres, limiting the generalisability of the study, hence the importance of stating the setting of this study. (Also add a generalisability comment in your discussion).

Answer: We added the experience of the APPs to the manuscript. We agree that experience is definitely a factor which is important to consider. What also has to be kept in mind that The APPs in this study also evolved from clinicians with very limited capabilities to valuable clinicians. We think that this reflects a dynamic environment which develops itself over years. The new APPs do not have the experience yet, but tend to learn very fast through guidance form the more experienced APPs. In this way this staffing model could also be applicable to healthcare organizations in rural areas. When deploying more APPs, clinical experience will develop faster and easier. It is probably the initial organizational setup which is most difficult. In addition, we experienced a transfer of nurses to become APPs. These nurses are often a selection which has an urge to develop and explore other professions. Because we offered them the opportunity to become an APP, these people continued to participate in our team.

2. I recommend adding the time frame of your study to the abstract and methods.

Answer: We added the timeframe: the pre-COVID period 2017-2018. We chose this period because we think this does better reflect the ordinary duties of the APP and MR than a period during the COVID pandemic.

3. Data are pleural, this should be changed throughout the manuscript.

Answer: this has been corrected together with the language and style by editing of the manuscript by a native scientific writer.

4.The results should start with a participant flow diagram or statement. It is not clear if all rapid response team calls were included in your study at the moment.

Answer: We added this statement at the beginning of the results section. Additionally, we explained why the number of intubations is less than the total number of patients: not all patients received this treatment. We therefore, did not delete all the text after the tables, because we considered it an explanation for the fact that we were not able to retrieve all data on every parameter.

5. There is a sentence/section referring to the FTE of response by various healthcare workers to calls - this can be deleted, the gross numbers and percentages are enough. I think you are trying to convey levels of experience in these comments, which would be better dealt with another way.

Answer: We deleted this section and made a general remark. We did considered this item worth mentioning because we experience some hesitation by the MR to confront critical situations on the normal wards, where the APP seems to consider these situations a challenge.

6. I recommend light editing throughout the manuscript by a native English speaker - the manuscript is well written but there are minor phrase and spelling issues throughout. eg "de" instead of "the"

Answer: We involved a native scientific writer for editing. We think the language and the manuscript has improved considerably.

7. Table 1 should start with the numbers of patients in each column (n=xx)

Answer: We added the number of patients to table 1.

8. A lot of your narrative paragraphs in your results repeat data in Table 2 and could be deleted.

Answer: we deleted a few repetitive parts of the text (as also mentioned by reviewer 2). We left some of the parts describing the results of table 2 unchanged because of the explanatory additional information in it.

9. Whilst you didn't reach statistical significance, your physicians assistants seem to be slower at initiating procedures in your ICU. This might be worth a comment. Some of the times are an hour or so longer - you seem not to have an adequate sample size if this doesn't reach statistical significance.

Answer: We agree with the reviewer, and described this weakness in the limitation section of the manuscript. The relative small number of calls and the additional smaller number of intubations indicate a too small sample size. We added these parameters however, to explore additional parameter besides the raw patient outcome data which are almost always in line with the quality standards in a adequately organized healthcare organization and they do not often point to the favorability of one profession.

10. if word counts are an issue, almost all sections could be more concise.

Answer: Parts of the text were deleted and the suggestions of the scientific writer made the manuscript more concise.

Comments reviewer #2:

This paper compares rapid responses of teams led by a medical resident and by an advance practice provider. It is an interesting approach to broaden and valide skills of rapid response team leaders, based on the MAELOR tool.

1. It does not however consider the role of the ICU nurse who is also part of the team, nor of the prior experiences of the leaders. These should be mentioned in the limitations. The accompanying nurse can play a major role in ensuring the quality of patient management: it is common for junior doctors to ask experienced nurses for their opinions and support, for example, and not considering them at all in the team's performance seems a bit of an oversight.

Answer: We thank the reviewer for addressing this topic. We also value the important additional support and recommendations of the nurse and therefore, added this to the limitations section. In addition, to the other limitations this is another reason why the measurement of separate professions in a team is difficult. Mostly the entire team performance is measured.

2. Overall, the presented analysis seems appropriate and the results are discussed with pertinence. It is interesting to consider making RRT leader specialists, since medical residents can have varying levels of expertise. It will be however important to ensure that medical residents continue to be exposed to and train for emergencies during their postgraduate training.

Answer: We agree with this comment. This is the reason why we have a mixed team with a limited number of APPs although they are considered experienced and valuable for quality and continuity of critical care medicine.

3. There are minor revisions needed for English (including : line 152, experienceS; line 161, no plural for advice; line 205, what is the word "en" ; line 216, correct continues for continuous; the paragraph 274-279 needs revision: the first sentence has no verb, the third is difficult to understand). The section in lines 263-266 are a repetition of the information in lines 247-250.

Answer: We addressed this remark by consulting a native English speaking scientific writer, who offered suggestions for the improvement of the language and style of the manuscript. We think this led to a considerable improvement and a better readable manuscript.

Submitted filename: Rebuttal.docx

Click here for additional data file.

25 Jul 2022

28 Jul 2022

Rebuttal second review:

We thank de reviewers for their latest valuable feedback on the manuscript and we consider all requested changes as very helpful.

Comments reviewer #2:

1. Page 10, line 281: “were” instead of “could be”

Answer: Changed

2. Revise figure 1.

Answer: We want to thank the reviewer for this suggestion. We revised the figure and have improved the contrast between the positive (black, white text) and the negative (grey, black text) outcomes of the MAELOR tool. We also improved the directions of the flows in the flowchart.

Submitted filename: Rebuttal second revision.docx

Click here for additional data file.

4 Aug 2022

Advanced Practice Providers versus Medical Residents as Leaders of Rapid Response Teams: a 12-month retrospective analysis.

PONE-D-21-35288R2

Dear Dr. Kreeftenberg,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

James Mockridge

Staff Editor

PLOS ONE

Additional Editor Comments:

In the Ethics section of the Methods, please amend your current ethics statement to include the full name of the ethics committee/institutional review board(s) that approved your specific study.

11 Aug 2022

PONE-D-21-35288R2

Advanced Practice Providers versus Medical Residents as Leaders of Rapid Response Teams: a 12-month retrospective analysis.

Dear Dr. Kreeftenberg:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr Joseph Donlan

Staff Editor

PLOS ONE