Effects of Plasma Transfusion on Perioperative Bleeding Complications: A Machine Learning Approach.

Perioperative bleeding (PB) is associated with increased patient morbidity and mortality, and results in substantial health care resource utilization. To assess bleeding risk, a routine practice in most centers is to use indicators such as elevated values of the International Normalized Ratio (INR). For patients with elevated INR, the routine therapy option is plasma transfusion. However, the predictive accuracy of INR and the value of plasma transfusion still remains unclear. Accurate methods are therefore needed to identify early the patients with increased risk of bleeding. The goal of this work is to apply advanced machine learning methods to study the relationship between preoperative plasma transfusion (PPT) and PB in patients with elevated INR undergoing noncardiac surgery. The problem is cast under the framework of causal inference where robust meaningful measures to quantify the effect of PPT on PB are estimated. Results show that both machine learning and standard statistical methods generally agree that PPT negatively impacts PB and other important patient outcomes. However, machine learning methods show significant results, and machine learning boosting methods are found to make less errors in predicting PB.

Introduction

Bleeding in patients undergoing surgical procedures is a serious and relatively common complication that has been found to be associated with increased health care resource utilization, morbidity, and mortality. Although the origin of bleeding during surgery may be due to multiple factors, surgical factors and pre-existing abnormalities of the hemostatic system represent the principal causes of significant perioperative hemorrhage. [1] Excessive bleeding, reoperation for bleeding, and the need for transfusion of blood products are common both during and shortly after some types of surgical procedures. In spinal surgery, for example, between 30% and 60% of patients require allogeneic blood transfusion. [2] Reoperation for bleeding and administration of blood products are associated with postoperative complications including transfusion-associated lung injury (TRALI). [3] Despite the known negative effect on immunomodulation and increased risk of postoperative complications and mortality, various studies have shown large variability in the use of blood products among different centers, and even among individual anesthesiologists within the same center. [4, 5] For example, the number of red blood cell (RBC) units transfused annually in the US alone is about 14 million. Cost-wise, at an estimated cost of $761 per unit of RBC, this amounts to $10.5 billion in health care expenditures. [4] Therefore, strategies to minimize the need for allogeneic blood transfusions and perioperative bleeding complications are of great interest.

A key step to reduce the need for transfusion of allogeneic blood product is to assess patients who might bleed pre-transfusion. This might involve the identification of patients with impaired hemostasis and increase bleeding risk. However, our ability to predict these adverse events, typically estimated with a combination of patient- and procedure-related factors, [6] is incomplete. For patient-related factors, substantial emphasis is often placed on preoperative screening tests, such as the international normalized ratio (INR): a major driver for decisions about preoperative plasma transfusion. [7]

Fresh frozen plasma infusions are commonly used to improve coagulation or clotting and are the main therapy option for patients with elevated INR. A large proportion of the plasma components are transfused in the perioperative environment, [7,8] however, they are frequently administered prophylactically in the absence of significant active bleeding. This practice persists despite a growing body of literature questioning its efficacy. [8, 9] Moreover, plasma transfusions are increasingly recognized as important contributors to transfusion-related complications, including allergic reactions, TRALI, and transfusion-associated circulatory overload (TACO). [3, 10]

The goal of this study is to apply advanced machine learning methods to study the effect of preoperative plasma transfusion (PPT) on perioperative beeding (PB). The study is built on recent work described in Jia et al. [11] Based on propensity score matching estimated via standard statistical methods, Jia et al showed that PPT does not improve PB complications for patients with high INR scores. Generalized linear models such as logistic regression or linear regression are frequently used to estimate the propensity scores and models for the outcome. However, parametric models require assumptions regarding the functional form, distribution of the variables, and variable selection. If any of these assumptions are incorrect, the derived causal relationship may be misleading. Contrary to statistical approaches, machine learning methods can estimate complex relationships between the outcome and observed variables producing consistent estimates of the propensity scores and hence more reliable causal relationships.

Unlike the standard statistical analyses in Jia et al, [11] this work investigates the causal relationship between PPT and PB by making use of modern machine learning algorithms to obtain consistent and reliable relationships. The relationship between PPT and other secondary outcomes such as intraoperative RBC transfusion (Intra-RBC), blood loss, Re-operation for bleeding (Re-OP), need for ICU care, ICU length of stay (ICU-LOS), hospital length of stay (LOS), and mortality are also studied. Different from the approach in Jia et al, [11] the analyses in this study are geared towards designing advanced techniques to identify early patients who might bleed during or after a surgical operation. In particular, in the interesting case where potential non-bleeding patients are identified, a more conservative practice can be adopted in which PPT is administered not only based on elevated INR, but on accurate predicted individual need for PPT. The presented work is an initial step towards achieving this larger goal. In this light, the problem is studied under the framework of causal inference by estimating the causal effect of the treatment (in this case PPT). Thus, estimates of the average treatment effect for those who received PPT can be used to quantify the independent association of PPT with PB. The computed average treatment effect can be regarded as a form of attributable risk, comparing the overall risk of bleeding to the risk of bleeding for those who received PPT. [12] Large values can be interpreted as increased risk of bleeding.

Although machine learning methods have been applied extensively in healthcare, to the best knowledge of the authors, their use for estimating the causal treatment effect of blood product transfusion has not been investigated. Thus, besides being one of the first works on the application of machine learning for blood transfusion, this paper strives to study PPT on several important patient outcomes in the context of causal inference by computing robust estimates of the average effect of PPT that can be used to answer questions like “what and how big is the effect of PPT on patient outcomes?”

Data Structure and Parameter of Interest

This study is based on an observational comparative effectiveness research analysis using a cohort design described in Jia et al [11] that was approved before initialization and this study followed all guidelines for strengthening and reporting of observational studies.

The structure of the data used in this study has the typical structure of causal inference. The observations for each patient are given by (X, Y, Z) where Z ∈ {0,1} is the treatment indicator with Z=1 if patient was treated or Z=0 if patient was not treated. X is a vector of baseline covariates that records information specific to each patient prior to treatment. Y is the outcome variable such as perioperative bleeding, with Y = 1 if bleeding or Y = 0 if no bleeding. Y can also be continuous, such as in LOS.

In observational studies the vector of covariates X could be related to both the potential outcome of interest Y and the treatment administered Z. Since Z and Y are affected by X, Z is therefore not independent of Y. Since X can affect both the probability of treatment and the probability of the outcome it is referred to as confounders. Ignoring the potential confounding effects of patient characteristics may lead to biased estimation of the treatment effect. [13] Thus, it is vital in observational studies to address potential bias due to confounding. Unbiased estimates of the parameters of interest can be obtained after controlling for observed characteristics, for example with propensity scores (PS) estimated via parametric regression methods. [14] However, unbiased estimates can only be obtained if the regression models are correctly specified. To minimize bias due to model misspecification, machine learning methods have been employed recently to find the best-fitting model for the data, thus producing consistent estimates of the propensity scores. These methods have been shown in numerous simulation studies to be able to reduce bias due to model misspecification. [15, 16]

To define the parameter of interest, the problem is viewed under the potential outcome framework first introduced by Rubin [17] and the recent population intervention model proposed in Hubbard and Van Der Laan. [18] The potential outcome model defines the effect of the possible levels of treatment Z = 1 or Z = 0 for each patient and allows for the consideration and estimation of what would have happened if a patient receives a particular treatment, possibly contrary to what the patient actually received. This counterfactual procedure allows for the definition of summary measures that quantify the effect of the treatment. Figure 2 illustrates the potential outcome modeling procedure. The general procedure is to use a model to estimate the outcome of interest had the entire population been treated or not treated and compare the two estimates.

Figure 2

Potential Outcome (What If) Model

With the goal of reducing bleeding risk and optimization of plasma transfusion, this work is interested in studying for from those who received PPT, what would have happen if they did or did not receive PPT. Thus, measures like the average treatment effect for the treated (ATT) or the average treatment effect for the control (ATC) is of interest. ATC is defined as ε = E [E[Y | Z = 0, X] − E[Y], where E denotes the expectation with respect to X (see [18, 19] for more details). ATC can be regarded as a type of attributable risk, since it compares the overall mean of the outcome to the mean of the population of interest average over strata of X.

Estimation Methods

Assuming no other unobserved variables are present, the relationships between the observed variables are shown graphically in Figure 1. From the graphical representation, the factorized data likelihood can be written as:

Figure 1

Causal Graph and Confounding

Node Z in Figure 1 is of great interest in causal inference as the goal is to determine what happens to the outcome when some intervention is done on Z. The probability g(Z | X) = Pr (Z | X) represents the propensity or the causal disposition of the treatment to produce or create some outcome. Q(X) is the outcome model which represents the mean value of the outcome Y given Z and X. With the factorized data likelihood, different approaches can be used to answer the question “what and how big is the effect of PPT on PB ?”

For a meaningful measure of treatment effect, two robust estimation methods are considered for estimating the parameter ψ: Double Robust (DR) and Targeted Maximum Likelihood (TMLE) Estimation. More in-depth treatment of these estimators can be found in [18, 19].

Doubly Robust Estimator

The Doubly Robust (DR) estimator for ATC is derived in [18, 19] and given by

Where I is the indicator function, ĝ(Z = 0 | X) and Q̂0(X) are estimates of g(Z = 0 | X) and Q0(X) respectively.

Doubly robustness means that ψ is a consistent estimate of the causal treatment effect when either ĝ or Q̂0 is a consistent estimate of g and Q0 respectively.

Targeted Maximum Likelihood Estimator (TMLE)

From the definition of the DR estimator, the occurrence of g in the denominator shows that the estimator may blow up when ĝ is not well-bounded. For some patients where estimates of g is close to zero, ψ may be unstable. The lack of boundedness in ĝ is a violation of one of three identifying assumptions: namely, the positivity or experimental treatment assumption (ETA) that requires 0 < g(Z | X) < 1. See [19] for a more thorough discussion of these assumptions.

To solve this problem, the Targeted Maximum Likelihood Estimator (TMLE) was proposed in [20]. Like DR, TMLE is also double robust and locally efficient. The technical details of this estimator are beyond the scope of this paper and the interested reader is referred to the cited reference for more details.

Machine Learning Methods

Traditionally, the estimation of the functions g or Q0 is performed using generalized linear models such as logistic regression or ordinary least squares. However, these parametric approaches are prone to model misspecification. An alternative and more attractive approach is to employ machine learning methods. Machine learning aims to infer the true relationship between the outcomes and covariates through a learning procedure. Bias in the estimates resulting from model misspecification can thus be reduced.

This work makes aggressive use of six machine learning methods: Support Vector Machine (SVM), Neural Networks Using Model Averaging (NNET), AdaBoost (Stochastic gradient boosting), randomForest (RF), random k-nearest neighbor (rKNN), and Generalized Boosted Regression Models (GBM). A comprehensive review of these supervised learning techniques can be found in. [21] rKNN is a recent algorithm that builds multiple k-nearest neighbor classifiers or regression models and combines them using a similar strategy as in the RF algorithm.

Experiments

This section describes the experimental setup: the study population, covariate balancing, and variable importance.

Study Population

To be considered for this study, patients must meet the following criteria: age ≥ 18 years, noncardiac surgery and an INR ≥1.5 in the 30 days preceding surgery. See Jia et al [11] for more details on the selection criteria and data source.

Between January 1, 2008, and December 31, 2011, a total of 155,492 patients aged ≥ 18 years underwent noncardiac surgery at the participating institution. Of them, 14,743 had an INR measured within 30 days of the index surgical procedure, with 1,234 having an INR ≥ 1.5. This latter group comprised the study population.

Baseline patient demographics include age, weight, gender, and American Society of Anesthesiologists Physical Status I–V classifications (ASA). Disease conditions included myocardial infarction, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, connective tissue disease, diabetes mellitus, tumor, etc. Preoperative laboratory test results included INR, hemoglobin, creatinine, albumin, and activated partial thromboplastin time. Preoperative medications such as aspirin, clopidogrel, and heparin were also included in the models.

Covariate Balancing and Variable Importance

In causal inference, the models do not have to only fit the data well, but also represent (and hence balance) the features of patients who received PPT and patients who did not. Thus an initial investigation using RF was performed to determine if there are confounding covariates in the study population. See Jia et al [11] for the results from standard univariate and multivariate statistical approach in balancing the data.

First, RF variable importance measure was used to identify features that are statistically associated with PPT (results not shown). The method described in Luz Calle et al [22] was used to estimate p-values. Then a robust Covariate Balancing Propensity Score (CBPS) method [23] was applied to estimate propensity scores by maximizing the average treatment effect (ATE). The estimated propensity scores were then used in a non-parametric matching algorithm described in Ho [24] to pair patients who received PPT to patients who did not. Table 1 shows, under the covariate balancing columns, the RF permutation variable importance p-values based on the Gini impurity measure for the adjusted and unadjusted covariates. Similarly to the results in Jia et al, [11] the RF variable importance identified many between-group differences. However, after performing CBPS, the all non-significant p-values under the Adjusted column of Table 1 indicates that the propensity score matching was effective in addressing covariate imbalance.

Table 1

Random forest variable importance p-values for balanced and unbalanced data and risk factors for bleeding

	Covariate Balancing		Bleeding
Variables	Unadjusted	Adjusted	Risk Factors
Clopidogrel	0.050	0.950	0.030
Peptic Ulcer	0.673	0.459	0.033
Chronic Renal Failure	0.741	0.094	0.040
Creatinine	0.864	0.218	0.040
PLT	1.000	0.455	0.080
ASA Recoded	0.012	0.589	0.237
Hemiplegia	0.553	0.898	0.263
Coumadin	0.493	0.359	0.321
Peripheral Vascular	0.770	0.838	0.435
Dementia	0.553	0.697	0.465
MI	0.387	0.790	0.466
Age	0.365	0.218	0.474
Cancer	0.864	0.721	0.488
Lymphoma	0.447	0.357	0.512
DM organ damage	0.367	0.545	0.516
Heparin	0.072	0.403	0.559
Aspirin	0.473	0.242	0.565
Hemoglobin	0.679	0.253	0.576
Cancer meta	0.870	0.647	0.602
INR	0.946	0.453	0.616
Connective Tissue Disease	0.176	0.481	0.703
Gender	0.922	0.818	0.724
Leukemia	0.926	0.425	0.796
Cerebrovascular Disease	0.645	0.896	0.806
Pulmonary Disease	0.220	0.118	0.854
Emergency	0.002	0.561	0.872
DM	0.906	0.737	0.920
Procedure categories	0.008	0.541	0.976
Congestive Heart Failure	0.904	0.828	0.982
Liver Disease	0.445	0.224	1.000

Second, the same RF permutation variable importance measure was again applied to the CBPS balanced data to identify statistical relevant predictors of perioperative bleeding. The last column of Table 1 shows that Clopidogrel, Peptic Ulcer, Chronic Renal Failure, and Creatinine are significant risk factors for bleeding at the 5% significance level.

Results

This section presents the main results: Estimates of treatment effect of preoperative plasma transfusion on perioperative bleeding and other important patient outcomes. For each estimation method described in section 3, the g and Q functions are estimated using the six machine learning methods. For comparison with standard regression methods, g is also estimated with logistic regression (LR) while Q is estimated with LR or linear regression depending on whether the outcome is binary or continuous.

For each method, the parameter estimate, standard error, and significance represented by two sided p-value are reported. All computations were performed using a modified version of the R statistical package multiPIM, [25] a causal inference approach to variable importance analysis. The standard 5-fold cross-validation training procedure was followed for all experiments.

Table 2 presents estimates of ψ, the average treatment effect of PPT on PB and other outcomes for the untreated. Assuming that the ETA assumption hold from Young et al, [19] a value of ψ = −0.05 in the table for a binary outcome can be interpreted as: “The effect of preoperative plasma transfusion, given that all patients in the population are not transfused, is to reduce the risk of the outcome by 5 percentage points”.

Table 2

Impact of plasma transfusion on perioperative bleeding and other important patient outcomes

Estimator	Outcome	Statistics	LR	SVM	NNET	AdaBoost	RF	rKNN	GBM
DR	PB	ψ	−0.010	−0.016	−0.014	−0.011	−0.008	−0.015	−0.022
		SE	0.005	0.004	0.004	0.004	0.002	0.005	0.006
		p-values	0.186	0.000	0.007	0.032	0.008	0.006	0.001
	Intra-RBC	ψ	−0.010	−0.017	−0.016	−0.011	−0.008	−0.015	−0.022
		SE	0.005	0.004	0.004	0.004	0.003	0.004	0.006
		p-values	0.186	0.000	0.001	0.040	0.009	0.007	0.001
	Re-OP	ψ	−0.014	−0.027	−0.017	−0.014	−0.013	−0.022	−0.031
		SE	0.005	0.005	0.005	0.005	0.003	0.005	0.006
		p-values	0.049	0.000	0.002	0.019	0.001	0.000	0.000
	ICU-LOS	ψ	−0.058	−0.301	−0.244	−0.076	0.018	−0.200	−0.171
		SE	0.121	0.089	0.129	0.113	0.070	0.094	0.118
		p-values	1.000	0.001	0.119	1.000	1.000	0.066	0.299
	ICU-Care	ψ	−0.014	−0.022	−0.017	−0.011	−0.012	−0.021	−0.024
		SE	0.005	0.004	0.005	0.003	0.003	0.005	0.005
		p-values	0.057	0.000	0.002	0.003	0.001	0.000	0.000
TMLE	PB	ψ	−0.010	−0.018	−0.013	−0.011	−0.012	−0.008	−0.022
		SE	0.005	0.004	0.005	0.004	0.003	0.005	0.006
		p-values	0.185	0.000	0.031	0.041	0.000	0.459	0.001
	Intra-RBC	ψ	−0.010	−0.017	−0.012	−0.011	−0.013	−0.017	−0.022
		SE	0.005	0.004	0.005	0.004	0.003	0.005	0.006
		p-values	0.185	0.000	0.076	0.049	0.000	0.002	0.001
	Re-OP	ψ	−0.014	−0.029	−0.022	−0.013	−0.016	−0.007	−0.031
		SE	0.005	0.005	0.005	0.005	0.003	0.005	0.006
		p-values	0.047	0.000	0.000	0.050	0.000	1.000	0.000
	ICU-LOS	ψ	−0.058	−0.157	−0.171	−0.136	−0.085	−0.195	−0.170
		SE	0.121	0.105	0.113	0.109	0.071	0.094	0.119
		p-values	1.000	0.267	0.263	0.426	0.456	0.075	0.309
	ICU-Care	ψ	−0.014	−0.077	−0.019	−0.008	−0.015	−0.019	−0.025
		SE	0.005	0.008	0.005	0.003	0.003	0.004	0.005
		p-values	0.055	0.000	0.000	0.099	0.000	0.000	0.000

Overall, all algorithms confirmed that PPT increases the risk of PB and all other considered outcomes. However, machine learning methods turn to generate significant results as illustrated by the small p-values. The significant results at the 5% level are highlighted. Results from machine learning methods show that PPT significantly impacts PB and Intraoperative RBC transfusion by 1–2 %, and need for ICU care by 1–7%. While PPT negative impacts ICU length of stay, hospital length of stay, and mortality (results not shown), the effects are not statistically significant.

The performance measures of the classifiers in predicting PB, assuming the counterfactual that no patient was administered PPT, is shown in Table 3. PCC is the percent of cases correctly classified (accuracy), AUC is the area under the receiver operating characteristic curve, while sens and spec are the sensitivity and specificity, respectively. Standard errors are also shown. The overall performance of the classifiers was very good with boosting methods tending to show a slightly better performance.

Table 3

Accuracy measures for predicting perioperative bleeding if plasma transfusion was not administered

Classifier	PCC	PCC.se	AUC	AUC.se	sens	sens.se	spec	spec.se
AdaBoost	0.792	0.017	0.868	0.016	0.734	0.031	0.824	0.020
GBM	0.791	0.017	0.864	0.016	0.772	0.029	0.801	0.021
LR	0.786	0.017	0.860	0.016	0.734	0.031	0.816	0.020
NNET	0.767	0.018	0.852	0.016	0.717	0.031	0.796	0.020
RF	0.808	0.016	0.863	0.016	0.673	0.033	0.884	0.017
rKNN	0.765	0.018	0.842	0.017	0.709	0.032	0.796	0.021
SVM	0.739	0.018	0.823	0.018	0.734	0.030	0.743	0.022

The performance of the two estimators (DR and TMLE) appear to be quite comparable, especially for logistic and linear regression. However, as noted earlier, DR may be unstable when the estimated propensity scores are close to zero. No such instability was observed for the analysis in this paper.

Conclusion and Future Work

This work employed advanced machine learning methods to measure the effectiveness of plasma transfusion from observational data. The method applies a causal modeling framework that establishes a causal relationship between administering plasma transfusion and perioperative bleeding based on two robust estimation procedures. The paper provides meaningful interpretation of the estimation results which are more intuitive to understand than estimated coefficients from regression models.

Results from the analysis show that population wise, the action of not administering plasma product has the impact of reducing bleeding and positive effects on other patient important outcomes. These results are not new, as several authors have derived these results. However this paper takes a causal approach based on modern machine learning to quantify in a meaningful way the effect of not transfusing. Given that, on average, no plasma transfusion reduces the risk of bleeding, an interesting problem warranting further investigation is: “Can machine learning methods be used to identify a sub-population for which no plasma transfusion increases the risk of bleeding?” This paper is an initial step to answering this important question.