A Simple and Quick Screening Method for Intrapulmonary Vascular Dilation in Cirrhotic Patients Based on Machine Learning.

BACKGROUND AND AIMS: Screening for hepatopulmonary syndrome in cirrhotic patients is limited due to the need to perform contrast enhanced echocardiography (CEE) and arterial blood gas (ABG) analysis. We aimed to develop a simple and quick method to screen for the presence of intrapulmonary vascular dilation (IPVD) using noninvasive and easily available variables with machine learning (ML) algorithms.
METHODS: Cirrhotic patients were enrolled from our hospital. All eligible patients underwent CEE, ABG analysis and physical examination. We developed a two-step model based on three ML algorithms, namely, adaptive boosting (termed AdaBoost), gradient boosting decision tree (termed GBDT) and eXtreme gradient boosting (termed Xgboost). Noninvasive variables were input in the first step (the NI model), and for the second step (the NIBG model), a combination of noninvasive variables and ABG results were used. Model performance was determined by the area under the curve of receiver operating characteristics (AUCROCs), precision, recall, F1-score and accuracy.
RESULTS: A total of 193 cirrhotic patients were ultimately analyzed. The AUCROCs of the NI and NIBG models were 0.850 (0.738-0.962) and 0.867 (0.760-0.973), respectively, and both had an accuracy of 87.2%. For both negative and positive cases, the recall values of the NI and NIBG models were both 0.867 (0.760-0.973) and 0.875 (0.771-0.979), respectively, and the precisions were 0.813 (0.690-0.935) and 0.913 (0.825-1.000), respectively.
CONCLUSIONS: We developed a two-step model based on ML using noninvasive variables and ABG results to screen for the presence of IPVD in cirrhotic patients. This model may partly solve the problem of limited access to CEE and ABG by a large numbers of cirrhotic patients.

Introduction

Hepatopulmonary syndrome (HPS) is characterized by intrapulmonary vascular dilation (IPVD) and remodeling in the alveolar microcirculation resulting in impaired gas exchange.1 The presence of HPS increases the mortality risk and decreases the patients’ quality of life,2 and it can also award liver transplantation (LT) candidates exceptional points while awaiting a donor liver.3 The diagnosis of HPS is based on the presence of chronic liver disease, IPVD as evidenced by contrast-enhanced echocardiography (CEE) and abnormal arterial oxygenation.4 However, the screening and diagnosis of HPS are limited; only 0.45% of cirrhotic patients in a large cohort were diagnosed with HPS according to the International Classification of Diseases code, with an accuracy of only 22.5%.1 More importantly, only 143 out of 194 diagnosed HPS patients underwent CEE, and 61 out of 98 IPVD patients had arterial blood gas (ABG) analysis results,1 which indicated that although CEE and ABG are common, screening for IPVD or HPS is limited because most of the patients are asymptomatic. A simpler, CEE- or ABG-free method for screening IPVD or HPS would improve the present situation of HPS patient management.

In the past, researchers attempted to use single variables, such as spider angioma, acropachy and cyanosis, Child-Pugh score and bilirubin, prothrombin time, creatinine, the difference between the SpO2 (supine) and SpO2 (upright), to predict the presence of IPVD without CEE, but with unsatisfactory results.4–7 Since the underlying mechanism of IPVD is largely unknown, it would be helpful to use multiple variables to predict the presence of IPVD. This could be further improved by applying new algorithms for model construction to overcome the limitations of conventional statistical methods.

Machine learning (ML) methods, a subset of artificial intelligence, may offer an alternative approach for predicting IPVD to overcome existing limitations. Due to the complexity and heterogenicity of liver diseases, artificial intelligence- or ML-based methods would be quite promising in for identification, prediction, and assessment of multiple liver diseases, comparing over conventional methods.8 Recently, Kanwal et al.9 developed and validated a model based on gradient descent boosting algorithm to predict the 1-year mortality risk with better discrimination than the conventional scoring method. To date, there has been no investigation applying ML methods to predict the presence of IPVD in cirrhotic patients, largely based on routine clinical data. Therefore, we aimed to develop a simple and quick model based on routine clinical data to screen IPVD patients (with both normal and abnormal arterial oxygenation) without CEE from cirrhotic patients, especially hospitalized patients, with the goal of achieving better patient management.

Methods

Patients in the current study were retrospectively analysed from a prospective study approved by the institutional ethics committee of the First Affiliated Hospital of Third Military Medical University (No. 2017(35) on July 10, 2017), and written informed consent was obtained from each patient. We enrolled cirrhotic patients scheduled for elective surgery at our hospital between July 27, 2017, and March 14, 2018.

The inclusion criteria were as follows: a diagnosis of liver cirrhosis based on biopsy, typical clinical findings, imaging studies and characteristic laboratory results; age ranging from 18 to 80 years-old; American Society of Anaesthesiologists score of II–III; ability to comply with research programmes; and no primary cardiopulmonary disease (heart disease, emphysema, pneumonia, asthma).

Study procedure

Patients were screened for eligibility 1 day before elective surgery and were assessed by the preoperative interview, CEE, physical examination, and ABG sampling.

CEE

All patients underwent CEE to detect the presence of IPVD using Fujifilm sonosite Edge (Sonosite, Bothell, WA, USA) equipped with a 2.5 MHz transducer. A baseline two-dimensional apical four-chamber view was acquired and 20 mL of agitated 0.9% saline solution was injected via the peripheral vein. The presence of microbubbles in the left cardiac chambers, identified between 3 and 6 heartbeats after visualization in the right cardiac chambers, was considered a positive result for IPVD.10

Oxygen saturation

Pulse oxygen saturation (SpO2) was performed using a digital pulse oximeter after the participant maintained an upright posture (seated) for 5 minutes after, which he or she was repositioned supine for 5 minutes. One sample of arterial blood was obtained by percutaneous radial artery puncture while the patient was seated and breathed room air. The blood sample was analyzed by a standard blood gas analyzer (ABL800 FLEX; Radiometer, Copenhagen, Denmark). Arterial pH, O2 (PaO2) and CO2 (PaCO2) were documented and the alveolar-arterial gradient (A-a gradient) was determined according to the alveolar gas equation.11 Age-related threshold12 was defined as [10 + 0.43 × (age in years − 20)]. Abnormal arterial oxygenation was determined as A-a gradient greater than the age-related threshold.13

Clinical data collection

Clinical data collected from the electronic medical record included demographics, causes of liver cirrhosis, smoking and drinking history and morbidities such as hypertension and diabetes, as shown in Table 1. All the participants received a thorough physical examination and consultation to obtain details of spider angioma, acropachy, liver palm, ascites, encephalopathy and dyspnea. Serum albumin, total bilirubin, direct bilirubin, aspartate transaminase, alanine transaminase, creatinine, hemoglobin, prothrombin time, international normalized ratio (commonly known as INR) and total bile acid concentrations were measured for each patient using standard laboratory methods and via the instruments of Sysmex XE 2100, Sysmex CS 5100 (Wakinohama-kaigandori, Chuo-ku, Kobe, Japan) and Beckman-Coulter AU5800 (Brea, CA, USA). Laboratory results closest to the date of the ABG analysis were recorded. Model for end-stage liver disease (commonly referred to as MELD) scores were calculated using the following formula:14 10×[(0.378×ln [bilirubin])+(0.957×ln[creatinine])+(1.12×ln[INR])+6.43. The Child-Pugh score was calculated by hepatic encephalopathy grade, ascites, INR, albumin, and total bilirubin.14

Table 1

Comparison of patient characteristics according to the presence of IPVD

Variable	IPVD, n=117	non-IPVD, n=76	t/χ2/U	p
Age in years, mean (SD)	50.3 (12.3)	47.1 (13.4)	−1.70	0.090
Male, n (%)	90 (76.9%)	53 (69.7%)	1.24	0.266
BMI in kg/m2, mean (SD)	23.6 (3.6)	23.0 (3.6)	−1.07	0.287
Child-Pugh score, median (IQR)	9 (7–10)	8 (7–9)	4.21	<0.001
MELD score, median (IQR)	11.9 (5.8–16.3)	10.1 (5.9–14.3)	1.37	0.172
Cause of liver cirrhosis, n (%)			16.72	0.005
Hepatitis B	92 (78.6%)	59 (77.6%)
Alcohol	13 (11.1%)	2a (2.6%)
Hepatitis C	2 (1.7%)	1 (1.3%)
Primary biliary cholangitis	3 (2.6%)	0 (0%)
Drug-induced hepatitis	7 (6.0%)	8 (10.5%)
Autoimmune hepatitis	0 (0%)	4a (5.3%)
Nonalcoholic fatty liver disease	0 (0%)	2 (2.6%)
Hypertension, n (%)	15 (12.8%)	7 (9.2%)	0.59	0.441
Diabetes, n (%)	15 (12.8%)	10 (13.2%)	0.01	0.946
Drinking, n (%)	45 (38.5%)	21 (27.6%)	2.40	0.121
Smoking index, median (IQR)	0 (0–400)	0 (0–200)	1.56	0.119
Acropachy, n (%)	99 (84.6%)	38 (50.0%)	26.80	<0.001
Liver palm, n (%)	106 (90.6%)	44 (58.7%)	27.27	<0.001
Spider angioma, median (IQR)	2 (0–4)	0 (0–1.75)	5.96	<0.001
Dyspnea, n (%)	72 (61.5%)	17 (22.4%)	28.45	<0.001
Ascites, n (%)	87 (74.4%)	31 (40.8%)	21.85	<0.001
Encephalopathy, n (%)	16 (13.7%)	0 (0)	11.33	0.001
SpO2 seated, %, median (IQR)	97 (96–98)	98 (97–98)	−4.45	<0.001
SpO2 supine, %, median (IQR)	98 (97–98)	98 (98–98)	−2.89	0.004
pH median (IQR)	7.45 (7.42–7.48)	7.43 (7.41–7.45)	2.38	0.017
PaCO2 mmHg, mean (SD)	36.2 (4.5)	37.4 (3.9)	1.85	0.065
PaO2 mmHg, median (IQR)	79.4 (70.8–85.0)	94.6 (83.8–107.0)	−6.76	<0.001
A-a gradient mmHg, median (IQR)	25.8 (19.9–32.1)	8.3 (−3.6–17.5)	7.50	<0.001
Elevated A-a gradient, n (%)	71 (59.6%)	12 (15.8%)	37.89	<0.001
TBA µmol/L, median (IQR)	97.4 (31.0–210.6)	40.6 (15.6–189.6)	1.97	0.049
Hemoglobin g/L, mean (SD)	108.5 (22.1)	114.1 (23.8)	1.66	0.099
ALT U/L, median (IQR)	50. 6 (28.8–103.0)	65.1 (34.1–146.8)	−1.55	0.121
AST U/L, median (IQR)	67.1 (43.3–119.3)	57.2 (38.8–126.4)	0.30	0.767
Albumin g/L, median (IQR)	31.4 (28.4–35.3)	33.3 (29.9–37.6)	−2.40	0.017
Globin g/L, median (IQR)	30.5 (26.7–35.9)	29.6 (23.5–36.4)	0.72	0.469
TBIL µmol/L, median (IQR)	68.0 (25.6–177.8)	55.6 (19.8–156.5)	1.20	0.230
DBIL µmol/L, median (IQR)	45.6 (14.1–135.1)	32.8 (8.4–109.2)	1.22	0.222
IBIL µmol/L, median (IQR)	28.6 (13.1–54.3)	21.0 (11.1–38.2)	2.00	0.046
PT second, median (IQR)	16.5 (13.5–19.6)	13.7 (11.9–16.4)	4.19	<0.001
INR median (IQR)	1.4 (1.1–1.7)	1.2 (1.0–1.4)	4.40	<0.001
Creatinine µmol/L, median (IQR)	65.1 (52.3–74.8)	69.0 (56.6–81.0)	−1.94	0.052

Mean (standard deviation, SD) presented for normally distributed continuous variables, while median (interquartile range, IQR) was given to those with non-normally distributed continuous variable. Unless otherwise stated, n is as indicated in the column headings. Prevalence of liver disease etiology was statistically compared between IPVD and non-IPVD patients (ap<0.05). BMI, body mass index; DBIL, direct bilirubin; IBIL, indirect bilirubin; PT, prothrombin time; TBA, total bile acid; TBIL, total bilirubin.

ML methodology

The whole process of our model establishment was shown in Figure 1. First, the whole dataset was randomly divided into training (154) and testing (39) datasets. Data pre-processing was conducted separately on the training and testing datasets using binarization, categorization and Z-score normalization. For the features with only two candidate values, such as hypertension, diabetes, drinking and certain clinical symptoms, binarization was used. Age, body mass index, number of spider angiomas and SpO2 difference between seated and supine positions were categorized. For instance, the number of spider angiomas was classified into two categories, namely, less than two and more than three. For PCO2, PaO2, A-a gradient and pH, we performed Z-score normalization, which produces a mean and standard variance of 0 and 1, respectively. The formula of Z-score normalization is: where, x and σ are the mean and standard variance of the original data, respectively. The detailed valuations of noninvasive variables and ABG analysis results are shown in Supplementary Table 1.

Fig. 1

The whole process for establishing our two-step screening model with the training dataset.

When the predicted value of the INI model was less than 0.5, the result of the NI model was determined according to the results of the INI model; if the predicted value of the INI model was more than 0.5, the result of the NI model was determined by the results of the FNI model. When the result of the NI model was positive, we used the results of the NIBG model. The model fitting method for the INI, FNI, and NIBG model were AdaBoost, GBDT, and Xgboost, respectively.

Based on the Python packages, including “numpy” and “scikit-learn (sk learn)”, we analyzed the patients’ data with three algorithms: adaptive boosting (AdaBoost), gradient boosting decision tree (GBDT), and eXtreme gradient boosting (Xgboost). Based on these algorithms, we aimed to screen IPVD from cirrhotic patients by establishing a two-step model that could be used by different hospitals. This two-step model consists of a noninvasive (NI) model and a model that incorporates both noninvasive variables and the results of ABG analysis (the NIBG model). The model was constructed on the training dataset. The NI model was conducted by a two-phase method based on the noninvasive variables. The first phase of the NI model is called the important noninvasive (INI) model, and the second phase is the full noninvasive (FNI) model. In the INI model, the input of variables were selected by statistical methods (shown in Supplementary Table 2). When a single patient was predicted as positive, we called him or her an uncertain case. To further determine whether the uncertain cases had IPVD, they were re-evaluated by the FNI model. The input variables of the FNI model were all the noninvasive variables, and the trained cases were only of the uncertain cases. AdaBoost was used for model fitting for the INI model and GBDT for the FNI model. The parameter of the NI model is shown in Supplementary Table 7. When the predicted value of the INI model was less than 0.5, the result of the NI model was determined according to the results of INI model; on the other hand, if the predicted value of INI was more than 0.5, the result of the NI model was determined by the FNI model. As shown in Figure 1, when the result of the NI model was positive, we implemented the second-step model, the NIBG model. The input variables included all the noninvasive variables and the ABG analysis results. The training data for construction of the NIBG model were the patient data that yielded positive results in the NI model. The model fitting method for the NIBG model was Xgboost. The parameters of the Xgboost algorithm are shown in Supplementary Table 7.

Model performance was evaluated by precision, recall, F1 score and the area under curve of receiver operating characteristics (AUCROC). Precision, recall and F1 score were calculated by a confusion matrix as follows:where TP means true positive, FP means false positive, and FN means false negative. In our current study, we calculated the recall, precision, and F1-score for positive cases (recall (1), precision (1), F1 score (1)) and negative cases (recall (0), precision (0), F1 score (0)). Recall (1), recall (0), precision (1) and precision (0) are also called sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), respectively. The AUCROC is an important indicator for the model discrimination.

Statistical analysis

Descriptive statistics are summarized as the mean±standard deviation or median (interquartile range). Comparisons between groups or datasets were made with the Student’s t-test or Mann-Whitney U test for quantitative variables, and with the χ2 test or Fisher’s test for category variables. All statistical tests were two-sided, and p values less than 0.05 indicated statistical significance. The statistical analyses were performed using SPSS software for Windows, V.19.0 (IBM Corp., Armonk, NY, USA).

Ethics approval

The study protocol was approved by the institutional ethics committee of the First Affiliated Hospital of Third Military Medical University (No. 2017(35), approved as of July 10, 2017 for the study titled “Perioperative application of specific COX-2 receptor inhibitor in lung protection in cirrhotic patients”, of which this is the first part), and written informed consent was obtained from each patient.

Results

Baseline characteristics

As shown in Figure 2, 193 patients were finally included in the analysis. Between the training and testing datasets, except for smoking, there were no significant differences in all the other baseline characteristics, nor were there significant differences in the distribution of patients in the IPVD and non-IPVD patients (shown in Supplementary Table 2).

Fig. 2

Flow chart of the study population.

Patient characteristics according to findings on CEE

As shown in Table 1, 117 of the 193 patients (60.6%) had IPVD, as evidenced by CEE. There were no statistically significant differences in demographics, comorbidities, MELD score, drinking and smoking index between the IPVD and non-IPVD patients. Cirrhotic patients induced by various of causes would occur with IPVD, and in our study most of the cirrhotic patients were caused by hepatitis B. Patients with IPVD had significantly higher Child-Pugh scores (p<0.001), prothrombin time (p<0.001) and indirect bilirubin (p=0.046) than those without IPVD. Clinical features were significantly different between IPVD and non-IPVD patients, namely, the presence of acropachy (84.6% vs. 50.0%, p<0.001), liver palm (90.6% vs. 58.7%, p<0.001), spider angioma (p<0.001), dyspnea (61.5% vs. 22.4%, p<0.001), ascites (74.4% vs. 40.8%, p<0.001) and encephalopathy (13.7% vs 0.0%, p<0.001). Notably, 16 of the 193 patients suffered from mild encephalopathy. IPVD patients had significantly lower SpO2 at both positions compared to non-IPVD patients. IPVD patients had a lower level of PaO2 (79.4 mmHg vs.94.6 mmHg, p<0.001) and a higher A-a gradient (25.8 mmHg vs. 8.3 mmHg, p<0.001) than non-IPVD patients. Furthermore, according to the age-related threshold, the prevalence of elevated A-a gradient in IPVD and non-IPVD patients was 61% and 16%, respectively (p<0.001).

Model performance of NI model on training and testing dataset

The NI model was composed of the two-phase model, namely, the INI and FNI models. After data pre-processing, dyspnea, liver palm, number of spider angiomas, ascites, acropachy, encephalopathy, and SpO2 differences between seated and supine positions were found to be statistically significantly different between IPVD and non-IPVD patients in the training dataset (shown in Supplementary Table 3). Therefore, these seven features were used as the input noninvasive variables for the INI model. If the patient’s predictive value calculated by the INI model was higher than 0.5, the patient was called an uncertain case. Uncertain cases were re-evaluated by the FNI model, of which the input variables were all the noninvasive variables. The final model performance of the NI model is presented in Table 2. For the training dataset, the AUCROC, F1-score (0), F1-score (1) and accuracy of the NI model were 0.952 [95% confidence interval (CI): 0.918–0.986], 0.857 (95% CI: 0.802–0.912), 0.901 (95% CI:0.854–0.948) and 88.3%, respectively. The NPV, PPV, specificity and sensitivity of the NI model for the training dataset were 0.831 (95% CI: 0.772–0.890), 0.921 (95% CI: 0.879–0.964), 0.885 (95% CI: 0.835–0.936) and 0.882 (95% CI: 0.831–0.933), respectively. For the testing dataset, the AUCROC, F1-score (0), F1-score (1) and accuracy of the NI model were 0.850 (95% CI: 0.738–0.962), 0.839 (95% CI:0.723–0.954), 0.894 (95% CI: 0.797–0.990) and 87.2%, respectively. The NPV, PPV, specificity and sensitivity of the NI model for the testing dataset were 0.813 (95% CI: 0.690–0.935), 0.913 (95% CI: 0.825–1.000), 0.867 (95% CI: 0.760–0.973) and 0.875 (95% CI: 0.771–0.979), respectively.

Table 2

Model performances of the NI and NIBG model

	AUROC	Precision (0)	Precision (1)	Recall (0)	Recall (1)	F1-score (0)	F1-score (1)	Accuracy
NI model
training dataset	0.952 (0.918–0.986)	0.831 (0.772–0.890)	0.921 (0.879–0.964)	0.885 (0.835–0.936)	0.882 (0.831–0.933)	0.857 (0.802–0.912)	0.901 (0.854–0.948)	0.883
testing dataset	0.850 (0.738–0.962)	0.813 (0.690–0.935)	0.913 (0.825–1.000)	0.867 (0.760–0.973)	0.875 (0.771–0.979)	0.839 (0.723–0.954)	0.894 (0.797–0.990)	0.872
NIBG model
training dataset	0.966 (0.937– 0.995)	0.845 (0.788–0.902)	0.988 (0.971–1.005)	0.984 (0.964–1.004)	0.882 (0.831–0.933)	0.909 (0.864–0.954)	0.932 (0.892–0.972)	0.922
testing dataset	0.867 (0.760–0.973)	0.813 (0.690–0.935)	0.913 (0.825–1.001)	0.867 (0.760–0.973)	0.875 (0.771–0.979)	0.839 (0.723–0.954)	0.894 (0.797–0.990)	0.872

Statistical quantifications were demonstrated with 95% CI, when applicable.

Model performance of NIBG model on training and testing dataset

Patients with predicted values higher than 0.5 as calculated by the NI model were considered high-risk cases. When the hospital had access to CEE, high-risk patients underwent CEE and ABG analysis for the diagnosis of IPVD or HPS. However, if the hospital had limited access to CEE, high-risk patients only underwent ABG analysis and were re-evaluated by the NIBG model. The input variables of the NIBG model were the noninvasive variables and the ABG results. For the training dataset, the AUCROC, F1-score (0), F1-score (1) and accuracy of the NI model were 0.966 (95% CI: 0.937–0.995), 0.909 (95% CI: 0.864–0.954), 0.932 (95% CI: 0.892–0.972) and 92.2%, respectively. The NPV, PPV, specificity and sensitivity of the NI model for the training dataset were 0.845 (95% CI: 0.788–0.902), 0.988 (95% CI: 0.971–1.005), 0.984 (95% CI: 0.964–1.004) and 0.882 (95% CI: 0.831–0.933), respectively. For the testing dataset, the AUCROC, F1-score (0), F1-score (1) and accuracy of the NIBG model were 0.867 (95% CI: 0.760–0.973), 0.839 (95% CI: 0.723–0.954), 0.894 (95% CI: 0.797–0.990) and 87.2%, respectively. The NPV, PPV, specificity and sensitivity of the NI model for the testing dataset were 0.813 (95% CI: 0.690–0.935), 0.913 (95% CI: 0.825–1.001), 0.867 (95% CI: 0.760–0.973) and 0.875 (95% CI: 0.771–0.979), respectively.

Discussion

IPVD is one form of the extrahepatic vasculature changes induced by various chronic liver diseases, such as alcoholic cirrhosis, hepatitis B or C infection, and nonalcoholic fatty liver disease.7,15 IPVD is also deemed to be an essential criterion for the diagnosis of HPS, while most of the researchers considered that there was no need to screen IPVD patients with normal oxygenation from cirrhotic patients in the past. However, Manual et al.16 found that approximately 35% of IPVD patients with normal gas exchange developed HPS by serial ABG measurement. IPVD-only patients were found to be in a hyperdynamic state, presenting as a higher cardiac output, cardiac index, and left ventricular stroke volume, leading to higher risk of dyspnea.7 IPVD was also found to be associated with a higher prevalence of obstructed intrahepatic portal branches, of slowed or hepatofugal portal blood flow, and of large abdominal portosystemic shunts,15 which was in accordance with the intra- and extrahepatic vasculature changes in cirrhosis.17 Furthermore, Jin et al.18 observed that HPS reversed in 95.8% of the liver transplant patients at 6 months; however, the prevalence of IPVD was 69.2% at 6 months, suggesting a difference between HPS and IPVD reversibility. It was also reported that cirrhosis regression induced a significant reduction in portal pressure accompanied by a normalization of systemic hemodynamics; however, there was no change in extrahepatic vascular structures.19 These findings show that IPVD screening is important not only for indicating HPS but also for evaluating the changes in extrahepatic vasculature. Thus, IPVD patients should be treated with cirrhosis-related routine measures and measures for liver-derived hyperdynamic states, which may help to prevent IPVD patients with normal gas exchange from developing HPS.

As shown in Supplementary Table 4, many researchers have attempted to use single variables to predict the presence of IPVD but most of the results have been unsatisfactory. Spider angioma was thought to be a skin marker of HPS,20,21 while in other studies, spider angioma and liver palm were found to be ineffective for detecting HPS or IPVD.7,10,13 Dyspnea and acropachy were found to be good clinical features in HPS patients,5 while in IPVD patients with normal gas exchange, the two features lacked significance.7 As shown in Supplementary Table 5, spider angioma was a good indicator for IPVD, with an AUCROC, sensitivity and specificity of 0.74, 66.7% and 75%, respectively. When the cut-off value of the A-a gradient was 19.83 mmHg, the AUCROC, sensitivity and specificity were 0.82, 76.9% and 81.6%, respectively. However, approximately 40% of the IPVD patients in our study had a normal A-a gradient, so using the A-a gradient as a single predictor would cause loss of predictive information. Although commonly used, SpO2 <96% had low AUCROC (0.68) and sensitivity (36.8%), which is constant with Forde’s study (AUCROC: 0.59, sensitivity: 28%).22 Although some laboratory indicators showed significant differences between IPVD and non-IPVD patients, we found that routine laboratory indicators alone were not good indicators for IPVD, which is constant with previously reported findings.7,18 We also attempted to use noninvasive variables and ABG results to develop one-step models for screening for the presence of IPVD in cirrhotic patients, with unsatisfactory results (shown in Supplementary Table 6). Previous studies and our results suggest that it would be more practical to combine variables rather than using single variables for model construction. Meanwhile, two-step models should be considered and applied to patients with lower and higher risks.

The AUCROCs of the NI and NIBG model for the testing dataset were all larger than 0.85 and the accuracy of the two models were higher than 85%, which indicated that our two-step model had satisfactory discrimination. Considering that our aim was to screen IPVD patients from large numbers of asymptomatic cirrhotic patients, the ideal model should screen out as many patients as possible and miss as few IPVD patients as possible. Therefore, except for the discriminability, the PPV and NPV are of equal importance. If we assume that the prevalence of IPVD is similar to that observed in our current study, the PPV and NPV were 81.3% and 91.3%, respectively. For clinical use, the PPV and NPV should usually be adjusted to the prevalence of the positive cases. The prevalence of IPVD varies from 20.7% to 84.0% due to differences in the study populations.7,10,18,23,24 However, IPVD patients with abnormal gas exchange, with a prevalence of approximately 30%,25 should not be overlooked and need intervention. Thus, the adjusted PPV and NPV were 73.8% and 94.2% for both the NI and NIBG model, respectively, for the whole model, when the prevalence of positive cases was 30%. Our results suggest that regardless of whether the PPV and NPV are adjusted, the NPV was higher than 90%, indicating that missed diagnosis rate was less than 10%. As for the PPV of the NI model being higher than 70%, given that the prevalence of positive cases in the second step was higher, the PPV of the whole model would be higher than the calculated value. Moreover, the first step of our model used only noninvasive and economic variables to screen out high risk patients; then, according to the reality, patients could undergo CEE and ABG for final confirmation or undergo ABG and then be predicted by our second-step evaluation (shown in Fig. 3).

Fig. 3

Flow chart of the screening method for clinical use.

For an individual cirrhotic patient, we initially evaluated him/her with model I (the NI model) to determine if he/she was a high-risk patient for IPVD; if the patient was determined to be high-risk, according to the reality of clinics, he/she can undergo CEE and ABG for final confirmation or ABG and prediction by model II (the NIBG model).

Limitations

Our study had several limitations. This was a single-centre and small sample size study, for which bias is difficult to avoid. Four different researchers performed the electronic medical records data collection and physical examination, ABG analysis, CEE and final diagnosis to decrease the bias. In the near future, multicenter research should be performed to validate our model.

Conclusions

We developed a two-step model based on ML methods using noninvasive variables and ABG analysis to screen for the presence of IPVD in cirrhotic patients. This model may prove to be promising for improving the quality of management for cirrhotic patients with intra- and extrahepatic vascular complications.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.