Examining the impact of ICU population interaction structure on modeled colonization dynamics of Staphylococcus aureus.

BACKGROUND: Complex transmission models of healthcare-associated infections provide insight for hospital epidemiology and infection control efforts, but they are difficult to implement and come at high computational costs. Structuring more simplified models to incorporate the heterogeneity of the intensive care unit (ICU) patient-provider interactions, we explore how methicillin-resistant Staphylococcus aureus (MRSA) dynamics and acquisitions may be better represented and approximated.
METHODS: Using a stochastic compartmental model of an 18-bed ICU, we compared the rates of MRSA acquisition across three ICU population interaction structures: a model with nurses and physicians as a single staff type (SST), a model with separate staff types for nurses and physicians (Nurse-MD model), and a Metapopulation model where each nurse was assigned a group of patients. The proportion of time spent with the assigned patient group (γ) within the Metapopulation model was also varied.
RESULTS: The SST, Nurse-MD, and Metapopulation models had a mean of 40.6, 32.2 and 19.6 annual MRSA acquisitions respectively. All models were sensitive to the same parameters in the same direction, although the Metapopulation model was less sensitive. The number of acquisitions varied non-linearly by values of γ, with values below 0.40 resembling the Nurse-MD model, while values above that converged toward the Metapopulation structure. DISCUSSION: Inclusion of complex population interactions within a modeled hospital ICU has considerable impact on model results, with the SST model having more than double the acquisition rate of the more structured metapopulation model. While the direction of parameter sensitivity remained the same, the magnitude of these differences varied, producing different colonization rates across relatively similar populations. The non-linearity of the model's response to differing values of a parameter gamma (γ) suggests simple model approximations are appropriate in only a narrow space of relatively dispersed nursing assignments.
CONCLUSION: Simplifying assumptions around how a hospital population is modeled, especially assuming random mixing, may overestimate infection rates and the impact of interventions. In many, if not most, cases more complex models that represent population mixing with higher granularity are justified.

Introduction

Dynamic transmission models have provided valuable insight toward controlling healthcare-associated infections (HAIs) for decades, particularly in addressing intervention effectiveness to limit colonization and spread of pathogens between hospitalized patients and healthcare workers [1-5]. The complexity and variation of HAI transmission models, as well as the advancement of methods for model fitting and sensitivity analysis, has increased over time. Methods used in HAI modeling studies vary widely, from adaptations of the classic Ross-McDonald model [6] that are simplified but analytically approachable [7-11] to complex network or agent-based models that include high fidelity representations of patient-to-staff interactions but are correspondingly more complex and vulnerable to results arising from subtle and unintuitive interactions between agents [12-17]. However, a 2013 systematic review by van Kleef et al. found that most HAI modeling studies used homogenous mixing compartmental models with limited hospital structure [18].

Concerns regarding the heterogeneous contact patterns that exist in hospitals suggest the need for network and agent-based models. The use of agent-based models for HAI modeling studies has increased greatly in recent years. A systematic review performed by Nguyen et al. in 2019 noted that while “systems dynamics models”–their term for compartmental models with homogeneous mixing–accounted for 38% of the models they reviewed, agent-based and discrete event models already accounted for the same volume of studies (38%), despite beginning to appear almost a decade later. While the authors note agent-based models can overcome the limitations of systems dynamics models, they also highlight the high computational costs, need for extensive data, and accompanying uncertainty analysis as major limitations [19]. These limitations do affect the generalizability and reproducibility of agent-based modeling studies, but efforts to address and mitigate this effect are underway [20].

Infectious disease research continues to grapple with the question of what level of complexity is required for robust results [21,22]. Complex network or agent-based models are computationally intensive and require a degree of software engineering expertise, whereas simpler models are more accessible. Rarely, however, are the sensitivity of the resulting models to structure decisions examined. If, and under what circumstances, simpler model formulations may be acceptable approximations of more complicated models, and what impacts these simplifications have on the model findings, remains an area that is underdeveloped within healthcare-associated infection modeling.

As an approach to understanding and quantifying these trade-offs, we consider the impact of structured contacts within a compartmental modeling framework using Methicillin-resistant Staphylococcus aureus (MRSA) as a motivating example. MRSA is a well-studied infection that is important in intensive care settings [2,3,10,19,23-28]. The vulnerable nature of the patients and the difficulty in treating severe infections makes preventing the spread of this antibiotic-resistant pathogen a priority among infection control efforts.

Patients admitted to the ICU have been found to have persistent colonization with MRSA 12–14 days after discharge from the hospital [29]. There is strong evidence that frequency and patterns of interaction between staff and patients play a critical role in transmission [30]. The ratio of nurses to patients has been found to contribute to the overall level of pathogen colonization and transmission within healthcare settings and ICUs [31-34]. Higher patient to nurse ratios have a positive correlation with increased transmission and poor health outcomes [34]. Hospital staffing levels, patient interactions, surveillance detection, and important parameters such as hand hygiene have been explored for several decades using mathematical models [2,3,25,26].

Using a stochastic compartmental model of an 18-bed ICU, we compared three potential population interaction structures: a single-staff-type model where all healthcare workers interact randomly with all patients, and there is no differentiation between nurses and physicians; a model that divides healthcare workers into nurses and physicians but continues to assume random mixing; and a highly structured model where each nurse is assigned a specific group of patients, i.e. a metapopulation-like structure. These models are hereafter referred to as “SST”, “Nurse-MD”, and “Metapopulation”, respectively.

We also developed hybrid model, which reflects the limited random interaction resulting from variance from patient assignments often seen in the ICU environment, such as cross-coverage during breaks, staff shortages, or complex procedures that require higher numbers of healthcare workers. For some portion of the work day, a nurse may to randomly interact with patients not originally under their direct care. The model accounts for this variation by utilizing a metapopulation that is primarily but not exclusively, organized into distinct subpopulations but still allows for some interaction between nurses and all patients.

These models represent moving from an extremely simple model structure to one that is of intermediate complexity, and finally to a model that approximates same interactions that might appear in a network or agent-based model while still adhering to the compartmental model framework (allowing it to use the same parameters and for differences in software implementation to be ignored in understanding the differences between models). In doing so, we examined the entire staff-patient interaction spectrum from random mixing to a highly structured model. We explored the sensitivity of these different models to changes in their underlying parameters, as reflected by the predicted number of MRSA acquisitions.

Methods

Model structure

MRSA transmission was simulated in an 18-bed medical ICU that included six nurses and a dedicated critical care physician based on a previously published model [27]. Three models were implemented with varying structures of provider-patient interactions. While ICU size and staffing levels can vary, we chose an 18-bed ICU with a patient-to-nurse ratio of 3:1 based on averages obtained from a large multicenter clinical trial [35]. A summary of all transmission model parameters can be found in Table 1.

Table 1

Parameters for modeling the acquisition of methicillin-resistant Staphylococcus aureus in an Intensive Care Unit.

Parameter	Parameter Description	Parameter Value	Source
ρ	Contact rate between patients and HCWs	4.154 (# of direct care tasks/hour)	[7,37]
ρ N	Contact rate between patients and nurses	3.973 (# of nurse direct care tasks/hour)	[7,37]
ρ D	Contact rate between patients and physician	0.181 (# of physician direct care tasks/hour)	[7,37]
σ	Probability that a HCW’s hands are contaminated from a single contact with a colonized patient	0.054	[38]
ψ SST	Probability of successful colonization of an uncolonized patient due to contact with a contaminated HCW when randomly mixed	0.1494	Fit to [35]
ψ Nurse-MD	Probability of successful colonization of an uncolonized patient due to contact with a contaminated HCW with physician separated	0.1660	Fit to [35]
ψ Metapopulation	Probability of successful colonization of an uncolonized patient due to contact with a contaminated HCW in metapopulation structure	0.4481	Fit to [35]
θ	Probability of discharge	4.39 days-1	[35]
ν u	Proportion of admissions uncolonized with MRSA	0.9221	[35]
ν c	Proportion of admissions colonized with MRSA	0.0779	[35]
ι	Effective hand-decontaminations/hour (direct care tasks × hand hygiene compliance × efficacy)	5.740 (10.682 direct care tasks/hour × 56.55% compliance × ~ 95% efficacy)	[7,35,37,39]
ι N	Effective nurse hand-decontaminations/hour	6.404 (11.92 direct care tasks/hour × 56.55% compliance × ~ 95% efficacy)	[7,35,37,39]
ι D	Effective physician hand-decontaminations/hour	1.748 (3.253 direct care tasks/hour × 56.55% compliance × ~ 95% efficacy)	[7,35,37,39]
τ	Effective gown or glove changes/hour (2 × # of visits × compliance)	2.445 (2.957 changes/hour × 82.66% compliance)	[35,38,40]
τ N	Effective nurse gown or glove changes/hour	2.728 (3.30 changes/hour × 82.66% compliance)	[35,38,40]
τ D	Effective physician gown or glove changes/hour	0.744 (0.90 changes/hour × 82.66% compliance)	[35,38,40]
μ	Natural decolonization rate	20.0 days-1	[41]
γ	Proportion of time nurses spend with assigned patients	Varied between 1/6 and 1

In the single staff type (SST) model, patients are assumed to mix randomly with healthcare workers (HCWs), with no distinction between the nurses or the physician (Fig 1A). Hospital staff are either uncontaminated (SU) or contaminated (SC), representing infectious material on their hands or person. Patients are either uncolonized (PU) or colonized (PC). This model, while unrealistic, was important to use as a baseline for comparison as it represents a homogenous mixing population often found in compartmental models. This simplified representation is also similar to how some agent-based models of larger-scale hospital networks represent single hospitals [13]. The SST model equations are available in S1 Table.

Fig 1

Compartment models of methicillin-resistant Staphylococcus aureus (MRSA) acquisitions.

Patients and hospital staff are classified as (un)colonized or (un)contaminated (U and C on diagrams), respectively. Solid arrows indicate transition states, while dashed arrows indicate routes of MRSA transmission (transition parameters and equations are in Tables 1 and S1, respectively. A) Single Staff Type model, B) Nurse-MD model, C) Metapopulation model, D) a hybrid model where nurses only spend a fraction of their time in their assigned patient groups and otherwise see patients at random.

The “Nurse-MD” model retained random mixing with all patients but separated the physician from the nursing staff as two distinct populations (Fig 1B). Separation of the physician also allowed the interactions between healthcare workers and the patients to be more realistic, using role-specific contact rates with patients. In this model, physicians have less direct care tasks (touching the patient or their immediate surrounding environment) when compared to either nurses or the generic healthcare workers in the SST model. This model had six compartments within it: the number of patients either colonized (PC) or uncolonized (PU), the number of nurses either contaminated (NC) or uncontaminated (NU), and the two additional compartments representing the physician as either contaminated (DC) or uncontaminated (DU). The equations for the Nurse-MD model may be found in S2 Table.

The “Metapopulation model” further segregated the healthcare workers by assigning each nurse a specific group of patients (one nurse for every three patients) and assuming the nurse cared exclusively for those patients. This practice is common in many ICUs for continuity of care, familiarity, and scheduling purposes. The model compartments thus become further divided into six subpopulations, with the physician acting as a bridge between them (Fig 1C). This model structure creates a metapopulation that better represents an actual ICU organizational staffing structure at the cost of increased complexity and reduced analytical tractability. This model only assumes that the nurse visits each patient in their assigned group randomly. The equations for the Metapopulation model may be found in S3 Table.

Finally, we created a Hybrid model to explore intermediate population interactions between purely random mixing and strict nurse-patient groupings (Fig 1D). In this model, nurses were assigned to a specific group of patients but also interact with patients outside their assigned group due to cross-coverage, staff breaks, or patient care tasks that require more than one nurse to perform. This model adds a parameter, gamma (γ), to represent the amount of time a nurse spends in their assigned group, with the remainder of the time spent moving randomly among patients outside their assignment. When γ = 1/6, this model replicates the Nurse-MD model, as a nurse is no more likely to spend time with their assigned patients as they are any other five patient groups. Similarly, when γ = 1, the model replicates the Metapopulation model, where nurses only treat their assigned patients.

Several assumptions underlie all four models. First, patients are assumed to have a single-occupancy room, do not leave their room at any time, and therefore do not interact with other patients. It is assumed that nurses and the physician only interact with the patients and do not interact with each other in ways relevant to pathogen transmission. The ICU is considered a “closed ICU”, meaning physicians or other hospital staff from outside the ICU do not interact with patients. The ICU is also considered to be at 100% capacity at all times, therefore if a patient is discharged it is assumed another patient is admitted to the bed immediately [36]. A hand hygiene opportunity occurs after every direct care task or any contact between a healthcare provider and a patient. Personal protective equipment (PPE) such as gowns and gloves are changed on entry and exit from the rooms of all colonized patients. Both hand hygiene and PPE are performed with imperfect compliance (Table 1). Lastly, we assumed that MRSA colonization is detected instantly and with perfect sensitivity and specificity to simplify the model, and that no treatments or interventions were performed for colonized patients other than the natural decolonization parameter, mu (μ).

Parameterization

Parameter values were obtained predominantly from a previously published model of MRSA transmission in an ICU [35] and are described in Table 1. The Nurse-MD and Metapopulation models introduce new interactions between the patient and their healthcare team, which required rederivation of some parameters from their original sources [40,42,43]. Specifically, the hand hygiene and gown/glove change rates incorporate nurse and physician specific contact rates, which were recalculated using the same methods as in the previous work.

Contact rates between patients and healthcare workers were represented by direct care tasks per hour for each healthcare worker type. Direct care tasks are defined as the physical interaction of the healthcare worker with the patient or their surrounding environment [40]. Effective hand-decontaminations per hour (ι) were calculated by the number of direct care tasks and taking into consideration the compliance rate and handwashing efficacy. Effective gown and glove changes per hour (τ) were calculated based on the number of visits to a patient per hour and a compliance rate–changing gowns and gloves was assumed to be 100% effective at removing contamination from a healthcare worker.

One additional parameter was added to the model differing from previously published work. A natural decolonization rate based on results from the STAR*ICU Trial was added based on evidence that colonization of MRSA is limited, and natural decolonization can occur without targeted treatment or decontamination efforts, moving patients from PC to PU at a low rate absent any direct intervention [29,41].

Model simulation

The SST, Nurse-MD and Metapopulation models were simulated to count the number of patients who transitioned to the colonized state (PC) in order to compare the average number of MRSA acquisitions. The models were stochastically simulated using Gillespie’s Direct Method [44] in Python 3.6 using the StochPy package [45] for 1,000 iterations per model. The initial conditions for each model were set to have no contaminated healthcare workers and no colonized patients, with initial MRSA infections being seeded from colonized members of the community being admitted to the ICU. Each iteration was run for a single year. The distribution of the acquisitions for each model’s 1,000 iterations was visualized in R v3.5.1 using the vioplot package [46], and the difference between them assessed using a Kruskal-Wallis test. The code for the model simulation and subsequent analysis may be found at github.com/epimodels/Metapopulation_MRSA.

Model recalibration

In addition to considering model outcomes using a single set of parameters (originally calibrated to the SST model), we also examined the difference in the estimated value of a single free parameter which could be fit within each model. The purpose of this recalibration is two-fold. First, it allows for a comparison of the models in a setting where their outcomes are equal. Second, it allows us to examine how each model form might influence the value of an estimated parameter–important information in a setting where models may be used to perform statistical inference and estimate intervention efficacy. The parameter chosen for this recalibration, ψ, is the probability of an effective colonization of an uncolonized patient from contact wtih a contaminated healthcare worker.

Approximate Bayesian Computation (ABC) [47] was used for the parameter fitting and to obtain an approximate Bayesian posterior of ψ for the SST, Nurse-MD, and Metapopulation models. This method samples a candidate value from a prior distribution, performs the model simulation using that candidate, and compares a summary statistic from that simulation to a target statistic. The candidate value is accepted if the simulation’s summary statistic equals the target statistic ± an error term episilon (ε). This is performed repeatedly, and the resulting distribution of accepted candidates approximates a Bayesian posterior distribution.

For this analysis, the target number of acquisitions was set to 5.94 acquisitions per 1,000 person-days with an episilon (ε) of 15%, matching the rate seen in the control arm of a large randomized clinical trial on MRSA prevention during the study period [35]. A uniform prior bounded by 0.0 and 1.0 was used, and 1,000,000 candidate parameters were drawn from this distribution to obtain the approximated Bayesian posterior of ψ for each model, using a simulation procedure similar to the one described above. For comparison between models, the median of this distribution was used as the value for ψ.

Parameter sensitivity analysis

In addition to assessing the difference in raw acquisitions in each model, we assessed the sensitivity of this outcome to changes in the model’s parameters. All parameters in the model were allowed to vary uniformly ±50% of their original values, and 100,000 parameter combinations were simulated for each model. For each model, the recalibrated value for ψ was used in order to ensure the models were compared against a consistent acquisition rate. The number of acquisitions in each simulation was then normalized as a percentage-change from the mean number of acquisitions. Linear regression was used on the normalized acquisition rate to determine the percentage change in acquisitions due to a single-percentage change in each parameter value.

The Hybrid model was used to explore a more structural sensitivity question within the Metapopulation model by varying the amount of time a nurse spends exclusively with their assigned group vs. other patients on the ward, γ. The Hybrid model was simulated 10,000 times, drawing a value of γ for each iteration from a uniform distribution bounded by 1/6 and 1. A segmented Poisson regression model was then fit to detect any thresholds in the value of γ where it’s relationship to the rate of MRSA acquisitions notably changed, or if the transition between the Nurse-MD model (γ = 1/6) and the Metapopulation model (γ = 1) was linear. This model incorporated linear and quadratic terms for γ and allowed the model to choose any number of break points.

Results

Model comparison

When using the same parameter set (calibrated to the SST model), the probability density and average number of MRSA acquisition were significantly different between the SST, Nurse-MD and Metapopulation models (χ2 = 1786.5, df = 2, p > 0.001) (Fig 2). Using the SST model as the baseline for comparison, a decrease in the average number of MRSA acquisitions were observed in both the separate Nurse-MD model and the Metapopulation model. By separating the physician from the nurses, the mean acquisitions decreased 20.6% from 40.7 acquisitions to 32.3 acquisitions, respectively. Limiting the nurses’ interaction to an assigned patient group yielded mean acquisitions of 19.8, a 51.4% decrease as compared to the original SST model.

Fig 2

Distribution of cumulative MRSA acquisitions in 3,000 simulated 18-bed intensive care units under three theoretical population structures.

The model parameter ψ is the probability of effective colonization of an uncolonized patient from contact with a contaminated healthcare worker and was used to calibrate each of the models. Calibration of the SST model resulted in the median value of the parameter of 0.024 (95% Credible Interval: 0.016, 0.034). The Nurse-MD model results were very similar to the SST model, with a median value of 0.029 (95% Credible Interval: 0.019,0.042). In contrast, the Metapopulation model had a median ψ value of 0.046 (95% Credible Interval: 0.032, 0.07), both a substantially higher estimate than the other models and one in which the bounds of the credible interval did not contain the other estimates. The altered contact patterns in the Metapopulation model require substantially higher per-contact colonization probabilities to sustain the same level of colonization.

Sensitivity analysis

While the Metapopulation model resulted in fewer acquisitions, certain parameters were found to affect the model outcomes to a larger magnitude when compared to the other models. The three parameters showing the largest proportional change (> 0.20) in cumulative acquisitions (Fig 3A) were contact rate (ρ), probability of patient colonization (ψ), and hand-decontamination (ι). We made similar findings for the Nurse-MD model, though generally only for the nurse-specific parameters (Fig 3B). The doctor-specific parameters had little effect on the model outcomes. Only one parameter of the Metapopulation model had a large change in cumulative acquisitions (> 0.20)–the nurse-specific contact rate (ρN). This is consistent with the previous two models (Fig 3C), although the effect was attenuated.

Fig 3

Global parameter sensitivity of three modeled ICU population structures.

Panel A depicts the change in proportional change in cumulative MRSA acquisitions per one-percent change in the value of a specific parameter, with light bars indicating increased acquisitions, and dark bars indicating decreased acquisitions for a model assuming random mixing and with a single staff type for both nurses and physicians. Pale grey vertical lines indicate a change greater than 0.2 in either direction, which was used as a boundary condition for major changes. Panel B depicts the same for a model that separates nurses and physicians into different staff types, while Panel C depicts the same for a metapopulation model where nurses were assigned to a strict subpopulation of patients. Panel D depicts the difference in proportional changes between the Metapopulation and Nurse-MD models.

The directionality of the overall change in cumulative acquisitions by parameter is an important measure of model stability and correct parameter estimates, as this reflects whether the models qualitatively give the same results as to whether or not a particular parameter value changing results in an increase or decrease in MRSA acquisitions, even if the models disagree as to the specific value of that change. All the parameters between the models are consistent in terms of directionality, with the Metapopulation model having a smaller change in magnitude of the cumulative acquisitions (Fig 3D).

Metapopulation interactions

Evaluating the relationship between gamma (γ), the proportion of time a nurse interacts with their originally assigned patient group, and MRSA acquisitions was identified as non-linear (Fig 4), with progressively higher values of γ resulting in drastically reduced rates of MRSA acquisitions. The segmented Poisson regression model identified a single change point, gamma* (γ*), at 0.40 (95% Confidence Interval: 0.37, 0.42). This value reflects a nurse spending a slim majority of their time (40%) with the fifteen patients not directly assigned to them, with the remainder focused on the three patients who are. Values below γ* were well approximated by the Nurse-MD model, and values above it rapidly approached the stricter assignment of the Metapopulation model.

Fig 4

Relationship between the proportion of time nurses spend treating patients outside their assigned group (γ) and cumulative MRSA acquisitions over 10,000 simulations, randomly sampling γ from a uniform distribution between 1/6 and 1.

Grey dots show an individual simulation, while the black line shows a segmented Poisson regression fit with linear and quadratic terms for γ. The vertical dashed line depicts the single segmentation point, γ*, to the left of which these more complicated models are adequately approximated by the Nurse-MD model where random mixing occurs. The shaded area shows the corresponding confidence interval.

Discussion

The reduction and control of healthcare-associated infections in recent decades has been a major accomplishment. In 2015, a point-prevalence survey found that HAIs affected roughly 3.2% of hospitalized patients in the United States, down from about 4% in 2011 [48]. Unfortunately, many drug-resistant or multidrug-resistant organisms (MDROs) remain an urgent or serious threat according to The Centers for Disease Control and Prevention (CDC), and certain pathogens, such as MRSA, continue to cause over 10,000 deaths annually [49]. The CDC continues to consider MRSA a serious threat to patient safety, and recent trends show that the incidence of hospital-onset bloodstream infections are no longer declining as observed for much of the last decade [50].

The evidence base for interventions to successfully address MRSA is mixed. For example, it has been difficult to quantify the effectiveness of MRSA screening and contact precautions, which has led to disagreement over their benefit [51-54]. However, more recent evidence clearly suggests that contact precautions among MRSA patients in Veterans Affairs acute care hospitals has a large reduction in transmission [55]. Other efforts, such as improved hand hygiene are often successful [56,57], but reducing MRSA acquisitions by means of improved transmission prevention continues to be a focus for hospital infection control efforts.

Even fairly subtle changes in model structure can greatly impact the estimated effectiveness of interventions. While remaining in the compartmental modeling framework, the more complex Metapopulation model was considerably more conservative when producing estimates of intervention effectiveness. While the models considered in this study had similar parameter sensitivity in terms of the direction of changes, the more highly structured models were relatively less sensitive. In all cases, the contact rate (ρ), probability of patient colonization (ψ), and hand-decontamination (ι) parameters had the largest impact, consistent with many of the known drivers of infection rates within hospitals. A model that assumes completely random mixing allows a higher degree of interaction with the patients and healthcare workers, resulting in an over-estimation of both the overall rate of MRSA acquisition and an over-estimation of the impact of interventions.

The sensitivity analyses also provided insight into the importance of accurately modeling different classes of healthcare worker. Our results suggest that the physician contact rate (ρD) had a very small effect on the change in cumulative MRSA acquisitions. Our model suggests that further exploration of nurse interaction and contact is most likely a more important focus for future infection control efforts.

The impact of changing the structure of a simulation ICU was, importantly, non-linear. The much simpler Nurse-MD model well-approximated the relatively more complex Metapopulation model when nurses spent between 16.7% (equivalent to random mixing) and 40% of their time with their assigned patients (γ). Informal estimates from two tertiary care academic medical centers in the southeastern United States estimated their ICUs at 80% and 90% (personal communication), an area of the parameter space where the impact of how one chooses to model the structure of an ICU has a pronounced impact.

Conclusion

When combined, these results suggest that while compartmental models assuming random mixing and those that have more specified population interactions may give qualitatively the same answer as to the benefit of an intervention, the magnitude of these estimates may vary considerably, which has implications for cost-effectiveness models and other studies that rely on these estimates. Additionally, if the interventions suggested by the model are implemented in practice, the performance of the intervention may differ from the model’s predictions due to the choice of how the population interacts. Finally, these results show that fitted parameters can vary considerably even for very similar models, suggesting even mild changes in model form necessitate refitting, and parameter estimates are not transportable from one model to another model with a differing population interaction structure.

The results of this study do have clinical implications. In an ICU setting, it is foreseeable that events like emergencies, breaks, or cross-coverage of nurses will occur with a reasonable degree of frequency. The COVID-19 pandemic’s impact on staffing, the creation of COVID-19 specific wards in spaces not intended for critical care, and the demands of treating COVID-19 patients is a particularly vivid example of this [58]. Our model suggests that even relatively small increases in the rate at which these interactions occur can have outsized impacts on MRSA acquisition rates, which is likely to be true for other healthcare-associated pathogens as well.

Methodologically, our study suggests that the appropriateness of using simplified model structures for healthcare-associated infections is highly context dependent. In circumstances where a simple qualitative answer or generalized ranking is needed, a simplified compartmental model that does not attempt to represent patient-provider mixing patterns may be sufficient. The COVID-19 pandemic, which has demanded rapid turn-around of modeling results in a largely distributed fashion is an example of a circumstance where the tradeoff between speed and accuracy in model development is likely important.

Our results also suggest that ICUs with highly interactive nurse-patient populations may not need more complex models–and while this is likely not the case in developed world urban tertiary care academic medical centers, it may be true in other environments such as rural hospitals or low and middle income country settings. Which model structure is most appropriate in these settings is empirically measureable, with an estimate of gamma informing both model structure and parameterization.

This study has several limitations. While the Metapopulation model is a more granular representation of a hospital population than the more-common SST model, it too is a simplification. Similarly, the parameter estimates used in the model are imperfect. It is likely that the hand hygiene rate is likely higher than the rates occurring in many hospitals, as reported rates are often substantially inflated. However, these estimates are drawn primarily from the established literature, and represent the field’s best understanding of the underlying processes.

Other limitations include the structure of the model–it focuses specifically on healthcare worker and patient interactions and does not account for interactions with individuals other than nurses and the physician. For example, interactions among patients, visitation by family and friends, medical or radiological technicians performing a specific procedure, etc. are not represented. Some of these individuals, such as technicians, arguably add an additional random element that would connect otherwise partially or wholly separated patients, and correspondingly increase infection rates. Others, such as visitors, primarily represent risk to a single patient. Similarly, transmission purely through environmental contamination is not represented. These simplifications were primarily chosen to make the illustration of the impact and potential necessity of moving to a more complex model structure as clear as possible.

This study shows that the random mixing assumption results in an over-estimation of both the overall rate of MRSA acquisition and an over-estimation of the impact of interventions as expressed as changes in model parameter values. In many–but not all–circumstances the use of more complex models is likely warranted, even for small-scale models of a single ICU. Importantly however, this necessity can be established not using a heuristic or qualitative assessment based on a modeler’s bias or preferences, but with a straightforward parameter estimate. The estimation of this parameter in several different healthcare contexts, including rural and LMIC settings, is ongoing. More broadly, the results of this analysis show the need for structural sensitivity analysis to accompany analysis of parameter uncertainty. While all the models explored here are adaptations of familiar compartmental forms, and the steps between them relatively simple, they can have dramatic impacts on the results of the models–impacts that can be amplified when these results are incorporated into larger models, cost-effectiveness analyses, guidelines and position papers, etc.

Transitions and equations for the Single Staff Type (SST) Model of MRSA Acquisition.

(DOCX)

Click here for additional data file.

Transitions and equations for the Nurse-MD Model of MRSA Acquisition.

(DOCX)

Click here for additional data file.

Transitions and equations for the Metapopulation Model of MRSA Acquisition.

(DOCX)

Click here for additional data file.

13 Sep 2019

Dear Dr Lofgren,

Thank you very much for submitting your manuscript 'Population Structure Drives Differential Methicillin-resistant Staphylococcus aureus Colonization Dynamics' for review by PLOS Computational Biology. Your manuscript has been fully evaluated by the PLOS Computational Biology editorial team and in this case also by independent peer reviewers. The reviewers appreciated the attention to an important topic, but they raised substantial concerns about the paper. Based on the reviews and editorial discussions, we regret that we will not be able to accept this manuscript for publication in the journal.

The reviews are copied below, and we hope they may help you should you decide to revise the manuscript for submission elsewhere. We are sorry that we cannot be more positive on this occasion, but hope that you appreciate the reasons for this decision and that you will consider PLOS Computational Biology for other submissions in the future.

Thank you again for your support of PLOS Computational Biology and open-access publishing. Please do not hesitate to get in touch (via ploscompbiol@plos.org) if we can provide any further assistance.

Sincerely,

Benjamin Althouse

Associate Editor

PLOS Computational Biology

Rob De Boer

Deputy Editor

PLOS Computational Biology

A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact ploscompbiol@plos.org immediately:

[LINK]

I agree with reviewer 1 on the lack of novelty of this manuscript and it missing mention to much previous work on this topic.

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: In the paper by Mietchen et al, the authors build upon a stochastic compartmental model to examine MRSA transmission in an ICU. They compare and contrast random vs. non-random mixing approaches in patient care and conclude that random mixing assumptions may be inappropriate. I congratulate the authors on developing a computational model examining an important topic (which I recognize takes many hours to do) yet I find their conclusion somewhat underwhelming as, in fact, this has been a known limitation in the epidemiology modeling field for some time. We have seen the rise of approaches that relax or omit this assumption over the past decade, but these other approaches are not mentioned in the manuscript. This ties in with my main concern: the background and discussion do not acknowledge the wealth of work done in modeling infectious diseases that do not assume a random mixing approach. I also have the following comments for the authors to consider to additionally improve this work.

Introduction:

• Please define consistent care teams: Are the authors referring to same providers, roles, or something else? What about shift changes?

• “grouping particular types of patients together”: Please use the term “cohorting”. This is the standard term in hospital infection prevention.

• The lit review seems to have missed some recent papers that examine how the built hospital environment and patient care networks impact MRSA transmission in the ICU. Some of these papers are based in the neonatal ICU, but have modeling implications for the adult ICU as well.

• Re: violating the independence assumption of many statistical techniques. I agree with the authors, and thus researchers use a variety of models with correlated errors to account for this occurrence. Would acknowledge that there are techniques that do not depend on the independence assumption.

• Re: limited or no control groups. But what about multicenter studies that either do a site-level randomization, or purely observational work? While differences between sites exist, the process of delivering care in an ICU is fairly standardized in the U.S.

• The authors state that “widely used models generally assume random mixing between healthcare workers and patients,” but there are no cites to back this claim up. This may have been true a decade or so ago, but use of network models relax the random mixing approach and in fact are now expected and move beyond compartmental modeling approaches. This is why in the epidemiology literature we see the increase in ABMs and network models, yet there is no acknowledgement of these in the paper. Examples exist in both the healthcare and non-healthcare environments.

Methods:

• Why include the SST model? This is not realistic. In fact, the majority of care in an ICU is provided by the nurse and not the physicians. Hand hygiene can also be differentially modeled by provider role.

• More detail regarding the setting is needed (brought in from the referenced publication). Is this meant to reflect an actual ICU or a hypothetical ICU? What type of ICU? How many admissions per year? Especially since the authors rely on parameter estimates from [17] that combines med/surg ICUs, these could be very different in terms of risk for organism colonization and infection.

• Re: MRSA colonization/detection assumption. Does this ICU carry out some kind of surveillance for MRSA colonization? Further, if an individual is colonized with MRSA, is a decolonization regimen performed (aside from the “natural” decolonization employed)? These are important considerations to accurately model MRSA transmission.

• Perhaps the authors can consider stratifying potential for contamination/transmission based on types of interactions with the patients. For example, there are a lot of routine, more mundane activities that confer lower risk, and fewer, invasive activities that confer higher risk. This would add an interesting and novel aspect to this work.

• While the model [appears to be] calibrated, was it also validated to ensure it reflects some kind of real ICU? This will make the findings more relevant for infection preventionists. Especially because in the Discussion the authors state that these models are useful “only if they can represent the population and transmission dynamics of a hospital.” This goes back to my earlier comment to better describe the setting.

• I applaud the authors for releasing their analytic code and data. More research groups should do this.

Results/Discussion:

• Figure 4/number of MRSA acquisitions: Please include a denominator, such as patient-days, when presenting incidence. This will make the findings comparable.

• Would be nice to include estimates of the proportion of MRSA transmissions due to the provider role under each scenario.

• Can the authors provide an intuitive interpretation of gamma=0.4? How does this finding potentially impact patient care dynamics in an ICU?

• The discussion needs to be expounded to compare and contrast to other work in this area, as well as provide recommendations for how the findings are useful for infection control in an ICU setting.

• The authors state that the “finding has broad implications for staffing levels and hospital policy." Such as ???

• The authors note the lack of non-nurse/MD personnel and visitors in their model. How does this omission potentially impact findings?

• How does the MRSA colonization/detection assumption impact findings? Especially because this is an important assumption and not reflective of how MRSA is actually detected. Suggest the authors bring in a surveillance aspect to this, and (possibly) decolonization if they use this in their ICU.

• Patient cohorting is mentioned in the introduction, but not returned to in the Discussion or elsewhere. Another opportunity to add an interesting angle to this work would be to compare and contrast cohorting and isolation of MRSA carriers.

Reviewer #2: This paper uses a modeling-based approach to compare several different structures of hospital ICU population interactions and how that affects MRSA acquisition rates. The goal seems to be to inform future models of MRSA acquisition within ICU settings, noting that simpler models overestimate both MRSA acquisition and the potential effects of interventions. This seems important and the importance of the findings could be highlighted more in the introduction.

The science in this work appears very well thought out, but a few improvements in the writing could make it easier for the reader to follow and understand earlier on in the paper.

1) short title includes “ICU”, main title should include “ICU” too

2) the term “Population structure” doesn’t seem quite right—perhaps “population interaction structure” is more accurate?

3) whenever feasible, it would be helpful for the reader to write out what the parameter is measuring, rather than just the parameter letter

4) it would be helpful to have a more general description of what the models measure and what they take into account earlier on (rather than just the population interaction structures)

5) it would also be helpful to label whatever parameters you could on the diagrams

6) at line 306, is the word contact correct? "The altered contact patterns in the Meta-population model thus needs substantially higher per-contact colonization probabilities to sustain the same level of contact.”

7) Figure 5: it would be easier to compare the 4 panels if the parameters were either in the same order or color-coded

--------------------

Have all data underlying the figures and results presented in the manuscript been provided?

Large-scale datasets should be made available via a public repository as described in the PLOS Computational Biology data availability policy, and numerical data that underlies graphs or summary statistics should be provided in spreadsheet form as supporting information.

Reviewer #1: Yes

Reviewer #2: Yes

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

14 Oct 2021

Submitted filename: Response.docx

Click here for additional data file.

9 Dec 2021

Dear Dr. Lofgren,

Thank you very much for submitting your manuscript "Examining the Impact of ICU Population Interaction Structure on Modeled Colonization Dynamics of Staphylococcus aureus" for consideration at PLOS Computational Biology.

As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Benjamin Althouse

Associate Editor

PLOS Computational Biology

Rob De Boer

Deputy Editor

PLOS Computational Biology

***********************

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #2: On re-review of this manuscript by Mietchen et al, I do find that I understand it better, but largely because of their responses to reviewers. More of these responses should make it into the paper.

This is good work that will help inform when certain models are appropriate to use, but the writing needs a lot of work. The introduction and discussion sections especially need to be more focused and set up the question better. And relevant models, such as from the 5 CDC sites, need to be cited. Much more citation of the modeling literature is needed—especially in the discussion section, which currently has zero citations.

General

• The main point of sentences should more often be at the beginning of the sentence so it’s easier to follow.

• For future versions, please use track changes correctly.

o It appears that you just cut everything and pasted—making it appear that everything was entirely re-written, which is not the case (the 1st sentence of the intro is the same.

• The writing throughout could be much more concise.

Abstract

• The “background” section isn’t actually background or justification for why you’re comparing these models—you need to set up your story. The author summary did a better job of this.

Introduction

• The introduction currently focuses too much and too early on the specifics of MRSA and infection prevention. It needs to highlight much more and earlier on, the importance of models in healthcare settings, the desire for simple models, and the question of whether simple models are appropriate.

• The specific details of MRSA aren’t as important and can be briefly summarized and cited.

Methods

• Model Structure section is too wordy. Can be shortened substantially.

• Cite Table 1 earlier on in your description of the model structures, and also cite the supplement with the equations.

Figures

• Note in all figure legends that arrows represent parameters found in table 1, and that equations are found in supplement.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at .

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

8 Feb 2022

Submitted filename: PLOS_CompBio Response Letter.docx

Click here for additional data file.

19 Apr 2022

Dear Dr. Lofgren,

Thank you very much for submitting your manuscript "Examining the Impact of ICU Population Interaction Structure on Modeled Colonization Dynamics of Staphylococcus aureus" for consideration at PLOS Computational Biology. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. The reviewers appreciated the attention to an important topic. Based on the reviews, we are likely to accept this manuscript for publication, providing that you modify the manuscript according to the review recommendations.

Please prepare and submit your revised manuscript within 30 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to all review comments, and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Thank you again for your submission to our journal. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Benjamin Althouse

Associate Editor

PLOS Computational Biology

Rob De Boer

Deputy Editor

PLOS Computational Biology

***********************

A link appears below if there are any accompanying review attachments. If you believe any reviews to be missing, please contact ploscompbiol@plos.org immediately:

[LINK]

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #2: Please see attached review.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example in PLOS Biology see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

References:

Review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Submitted filename: re-re-review.docx

Click here for additional data file.

19 May 2022

Submitted filename: PLOS_CompBio Response Letter.docx

Click here for additional data file.

3 Jul 2022

Dear Dr. Lofgren,

We are pleased to inform you that your manuscript 'Examining the Impact of ICU Population Interaction Structure on Modeled Colonization Dynamics of Staphylococcus aureus' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology.

Best regards,

Rob J. De Boer

Deputy Editor

PLOS Computational Biology

Rob De Boer

Deputy Editor

PLOS Computational Biology

***********************************************************

20 Jul 2022

PCOMPBIOL-D-19-01296R3

Examining the Impact of ICU Population Interaction Structure on Modeled Colonization Dynamics of Staphylococcus aureus

Dear Dr Lofgren,

I am pleased to inform you that your manuscript has been formally accepted for publication in PLOS Computational Biology. Your manuscript is now with our production department and you will be notified of the publication date in due course.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript.

Soon after your final files are uploaded, unless you have opted out, the early version of your manuscript will be published online. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting PLOS Computational Biology and open-access publishing. We are looking forward to publishing your work!

With kind regards,

Olena Szabo

PLOS Computational Biology | Carlyle House, Carlyle Road, Cambridge CB4 3DN | United Kingdom ploscompbiol@plos.org | Phone +44 (0) 1223-442824 | ploscompbiol.org | @PLOSCompBiol