Do We Need to Distinguish Thrombolysis and Nonthrombolysis Patients When Applying Stroke-Associated Pneumonia Predicting Scores? An External Validation from a 2-Center Database.

BACKGROUND Due to the lack of validation for predictive scoring of stroke-associated pneumonia in both thrombolysis- and nonthrombolysis-treated ischemic stroke (IS) patients, this study aimed to evaluate 4 scoring methods in the 2 subgroups. MATERIAL AND METHODS The CerebroVascular Database Project database included data from patients with cerebral IS that were admitted in 2 hospitals from February 2016 to January 2018. A total of 138 thrombolysis-treated and 138 nonthrombolysis-treated IS patients were enrolled. Area under receiver operating characteristic curves (AUROC) were performed to examine the discrimination of the 4 scores, and Hosmer-Lemeshow test was used to evaluate the goodness of fit. RESULTS The incidence of stroke-associated pneumonia was 24.8%. The thrombolysis and nonthrombolysis subgroups were not significantly different with regard to sex, present smoking, chronic obstructive pulmonary disease history, atrial fibrillation history, blood pressure, or glucose level on admission. However, significant differences were found in National Institutes of Health Stroke Scale scores (P<0.001), Glascow Coma Scale scores (P<0.001), Oxfordshire Community Stroke Project classification (P<0.001), dysphagia (P<0.001), and white blood cell counts (P=0.039). The AUROC for the Age, Atrial fibrillation, Dysphagia, male Sex, stroke Severity, National Institutes of Health Stroke Scale; Preventive ANtibacterial THERapy in acute Ischemic Stroke; Acute Ischemic Stroke-Associated Pneumonia Score (AIS-APS); and Independence, Sex, Age, National Institutes of Health Stroke Scale scores in total population were 0.80 (0.74-0.84), 0.75 (0.69-0.80), 0.80 (0.76-0.85), and 0.76 (0.71-0.81). The goodness of fit was 0.22, 0.22, 0.27, and 0.17, respectively. The AUROC of 4 scores between subgroups were not statistically significant. CONCLUSIONS The AIS-APS had the highest AUC and goodness of fit in our population. All 4 scores can be applied regardless of whether thrombolysis has been performed on patients.

Background

Stroke-associated pneumonia (SAP) is defined as the spectrum of pneumonia complicating the first 7 days after stroke onset in nonventilated patients [1]. The incidence varies between types of medical wards, ranging from 2.4% to 65% [2-7]. SAP is one of the most frequent complications after stroke, and it has been verified as an independent risk factor for poor functional outcomes and high mortality [6,8,9]. Therefore, early prediction and prevention of SAP is of great clinical significance. Many SAP risk factors have been found, including male sex, older age, dysphagia, high neurological defect scores, previous cardiac and pulmonary disease history, and use of antacids [10,11]. Eight SAP predicting scores have been developed since 2006 for ischemic stroke (IS). Four of them have more evidence for external validation with acceptable discrimination ability, including the Age, Atrial fibrillation, Dysphagia, male Sex, stroke Severity, National Institutes of Health Stroke Scale (A2DS2) [12], the Preventive ANtibacterial THERapy in acute Ischemic Stroke (PANTHERIS) score [13], the Acute Ischemic Stroke-Associated Pneumonia Score (AIS-APS) [14], and ISAN (The prestroke Independence, Sex, Age, National Institutes of Health Stroke Scale) score [15].

According to the outcomes from several randomized clinical trials, the time-window for thrombolysis using recombinant tissue plasminogen activator (rt-PA) acute IS patients with restricted indications can be extended to 4.5 h [16]. Intravenous rt-PA thrombolysis can help vascular recanalization, thus improving neurological function and reducing the stroke severity scores. A recent study shows that 24.4% IS patient received thrombolysis [17], which was a much higher proportion than in previous reports. However, existing SAP-predicting scores did not consider intravenous thrombolysis when they were established. Moreover, these scores have not been externally validated in thrombolysis-treated populations. Therefore, we aimed to examine the discrimination ability of 4 recent SAP scores in all IS patients, as well as in thrombolysis and nonthrombolysis subgroups. We hope that our study can provide evidence for the application of SAP scores in the growing number of thrombolysis patients.

Material and Methods

Population selection

The CerebroVascular Database Project (CVDP) is a 2-center, prospective, observational study set up in southeastern China. The centers include a tertiary-care hospital with the comprehensive stroke unit and a secondary-care hospital. Patients who had experienced cerebrovascular events within the previous 7 days were consecutively enrolled from February 2016 to January 2018. These events included transient ischemic attack, IS, intracerebral hemorrhage, subarachnoid hemorrhage and symptomatic lacunar infarction. The exclusion criteria were (1) age <18; (2) no signed informed consent; and (3) pregnancy. Internet-based and paper-based standard registry forms were used to collect information, including prehospital care, baseline severity, medical history, in-hospital managements, routine laboratory tests on admission, radiology data, discharge status, and 3-month follow-up. Physicians used neurological evaluation scales including the modified Rankin scale (mRS), National Institutes of Health Stroke Scale (NIHSS), Glascow Coma Scale (GCS), and Oxfordshire Community Stroke Project (OCSP) subtype in face-to-face interviews. All remaining data were obtained by medical records. The project was approved by the ZhongDa hospital central ethics committee and Wuxi Xishan hospital ethics committee. The informed written consent of all eligible patients or their legal representatives was obtained.

Equal numbers of age-matched nonthrombolysis patients and thrombolysis patients with IS were enrolled in our study. Patients with missing essential data were excluded. As there were more nonthrombolysis patients than thrombolysis patients, the nonthrombolysis patients were matched to the thrombolysis patients. Once multiple matches existed, we chose the patients treated by the same doctor or the closer admission date.

Data definition

The diagnosis of cerebrovascular disease was based on clinical presentations combined with assessments of brain computed tomography (CT) or magnetic resonance imaging (MRI) by physicians. SAP was diagnosed according to the recommendations from the pneumonia in stroke consensus group [1]. Swallowing function screening was operated by trained nurses using the water-swallow test (>1 defined as dysphagia). The A2DS2 is a 10-point score (age ≥75 years=1; atrial fibrillation=1; dysphagia=2; male sex=1; NIHSS score 0–4=0, 5–15=3, ≥16=5) [12]. The PANTHERIS score is a 20-point scoring system that was developed based on the following factors: GCS (<9=5, 9–12=2, >12=0), age (<60=0, 60–80=1, >80=2), systolic blood pressure (SBP) >200 mmHg on admission (no=0, yes=2), and white blood cell (WBC) count >11 000/μL (no=0, yes=3) [13]. Eleven variables were used to calculate the 34-point AIS-APS, including age (≤59=0, 60–69=2, 70–79=5, ≥80=7), medical history (atrial fibrillation=1, congestive heart failure=3, chronic obstructive pulmonary disease=3, current smoking=1), prestroke dependence (mRS ≥3=2), admission NIHSS score (0–4=0, 5–9=2, 10–14=5, ≥15=8), admission GCS (3–8=3, 9–15=0), symptom of dysphasia (no=0, yes=3), OSCP subtype (lacunar infarction or partial anterior circulation infarct=0, total anterior circulation infarct or posterior circulation infarct = 2), and admission glucose (≤11.0 mmol/L = 0, ≥11.1 mmol/L=2) [14]. The ISAN consists of 4 variables on admission: prestroke independence (mRS 0–1=0, 2–5=2); sex (female=0, male=1); age (<60=0, 60–69=3, 70–79=4, 80–89=6, ≥90=8); NIHSS (0–4=0, 5–15=4, 15–20=8, ≥21=10). The intravenous thrombolysis indications are referred to the guideline [18]. All centers used rt-PA for intravenous thrombolysis.

Statistical analysis

This study retrospectively analyzed the data recorded in the CVDP with SPSS 17 and MedCalc (version 11.4.2.0). The baseline information included sex, present smoking, chronic obstructive pulmonary disease history, congestive heart failure history, atrial fibrillation history, dysphasia, mRS, glucose at admission, SBP, WBC, GCS, OCSP subtype, and NIHSS score. The continuous variables are summarized as means with standard deviations, and the comparisons between subgroups are processed by Student t test and Mann-Whitney test. The categorized variables are listed as frequency and are analyzed with χ2 test or Fisher exact test. The AUC calculation was used to evaluate discrimination and the Hosmer-Lemeshow goodness-of-fit test was utilized to assess calibration of all scores. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Youden index were also obtained. The AUC for the subgroups was compared by Z-test. All P values were double-sided, and the level of statistical significance was set at 5% of the confidence level.

Results

Baseline characteristics

A total of 1471 patients were enrolled in the CVDP from February 2016 to January 2018. Among them, 141 IS patients received intravenous thrombolysis therapy. Three of these patients were excluded from the current study due to missing data. By age matching, we found another 138 IS patients who did not receive thrombolysis and had complete essential information in the CVDP. The incidence of SAP in all 276 patients, the nonthrombolysis subgroup, and the thrombolysis subgroup was 24.2% (67/276), 13.0% (18/138), and 35.5% (49/138), respectively. The descriptive data of the entire population are listed in Table 1. There were 176 men and 100 women in the enrolled population, with an average age of 70.35±11.68 years (range 31–94; Table 1). The nonthrombolysis and thrombolysis subgroups were not significantly different with regard to sex, present smoking, chronic obstructive pulmonary disease history, atrial fibrillation history, congestive heart failure history, SBP, and glucose level on admission. However, the 2 subgroups had significant differences in GCS (nonthrombolysis subgroup 14.09±1.76 vs. thrombolysis subgroup 12.46±2.41, P<0.001), NIHSS (nonthrombolysis subgroup 3.80±4.98 vs. thrombolysis subgroup 8.96±6.50, P<0.001), OCSP subtype (P<0.001), symptoms of dysphagia (P<0.001), and WBC (nonthrombolysis subgroup 7.44±2.29 vs. thrombolysis subgroup 8.14±2.8, P=0.039).

Table 1

Baseline characteristics of the entire group, non-thrombolysis subgroup and thrombolysis subgroup.

	Entire group (n=276)	Non-thrombolysis subgroup (n=138)		Thrombolysis subgroup (n=138)		Statistic		P value
Age
<50			7, 5.07%
50–60			21, 15.22%
61–70			41, 29.71%
71–80			40, 28.99%
81–90			27, 19.57%
>90			3, 2.17%
Gender							0.063		0.802
Male	176		89		87
Female	100		49		51
Present smoking							0.319		0.572
Yes	66		31		35
No	210		107		103
COPD							0.070
Yes	15		7		8				0.791
No	261		131		130
Congestive heart Failure							0.115		1.000
Yes	9		5		4
No	267		133		134
Atrial fibrillation							2.183		0.14
Yes	58		24		34
No	218		114		104
Dysphagia							32.504		<0.001**
Yes	95		25		70
No	181		113		68
mRS	0.39±0.89		0.44±0.92		0.33±0.87		−1.053		0.293
Systolic pressure	153.38±23.04		152.29±20.67		154.47±25.21		0.77		0.562
Glucose (mmol/L)	7.88±4.06		7.48±3.57		8.28±4.47		1.65		0.101
WBC (×109/L)	7.79±2.59		7.44±2.29		8.14±2.8		2.29		0.039*
GCS	13.28±2.26		14.09±1.76		12.46±2.41		−6.413		<0.001**
NIHSS	6.38±6.33		3.80±4.98		8.96±6.50		7.405		<0.001**
OCSP							6860.5		<0.001**
Lacunar infarction	12		3		9
TACI	32		1		31
PACI	185		106		79
POCI	46		28		18

p<0.05;

p<0.01.

The cutoff, sensitivity, specificity, PPV, NPV, and Youden Index

The cutoffs derived from our total population in the PANTHERIS, AIS-APS, A2DS2, and ISAN scores were 1, 11, 1, and 8, respectively. These values were different from the cutoffs derived from the original population, except the ISAN scores did not define a cutoff. In the nonthrombolysis subgroup, the AIS-APS score had the highest sensitivity and NPV (i.e., 0.94 and 0.98) but the lowest specificity and PPV (i.e., 0.52 and 0.23). In the thrombolysis subgroup, the specificity, PPV, and NPV of the AIS-APS scores (0.87, 0.73, and 0.82, respectively) were higher than those calculated from other scores. The A2DS2 score presented the highest sensitivity, 0.67, which was the same as that calculated using the ISAN scores in the thrombolysis subgroup. In the nonthrombolysis subgroup, the A2DS2 score presented the lowest sensitivity, 0.67, and the highest specificity, 0.80. The PANTHERIS score showed no extreme values for sensitivity, specificity, PPV, or NPV among all the scores, but it possessed the highest Youden index in the non-thrombolysis subgroup (Table 2).

Table 2

Diagnostic indexes in entire group and subgroups.

	Cut-off	Entire group
		Sensitivity	Specificity	PPV	NPV	Youden Index
PANTHERIS#	5	0.78	0.84	0.68	0.90	0.62
PANTHERIS*	1	0.85	0.52	0.36	0.92	0.37
AIS-APS#	8	0.79	0.77	0.31	0.97	0.56
AIS-APS*	11	0.61	0.84	0.55	0.87	0.45
A2DS2#	4/5	0.91	0.57	0.14	0.99	0.48
A2DS2*	1	0.51	0.80	0.90	0.30	0.31
ISAN#	-	–	–	–	–	–
ISAN*	8	0.63	0.80	0.51	0.87	0.43
	Cut-off	Non–thrombolysis group (n=138)
		Sensitivity	Specificity	PPV	NPV	Youden Index
PANTHERIS#	5	–	–	–	–	–
PANTHERIS*	1	0.83	0.66	0.27	0.96	0.49
AIS-APS#	8	–	–	–	–	–
AIS-APS*	11	0.94	0.52	0.23	0.98	0.46
A2DS2#	4/5	–	–	–	–	–
A2DS2*	1	0.67	0.80	0.33	0.94	0.47
ISAN#	-	–	–	–	–	–
ISAN*	8	0.83	0.58	0.23	0.96	0.41
	Cut-off	Thrombolysis group (n=138)
		Sensitivity	Specificity	PPV	NPV	Youden Index
PANTHERIS#	5	–	–	–	–	–
PANTHERIS*	1	0.43	0.82	0.57	0.72	0.25
AIS-APS#	8	–	–	–	–	–
AIS-APS*	11	0.65	0.87	0.73	0.82	0.52
A2DS2#	4/5	–	–	–	–	–
A2DS2*	1	0.67	0.75	0.60	0.81	0.42
ISAN#	–	–	–	–	–	–
ISAN*	8	0.67	0.73	0.58	0.80	0.40

Derived from original cohort;

derived from our study.

The comparison of AUC for 4 predictive scores

In the whole population, the AUC of the PANTHERIS, AIS-APS, A2DS2, and ISAN scores was 0.75 (0.69–0.80), 0.80 (0.76–0.85), 0.80 (0.74–0.84), and 0.76 (0.71–0.81), respectively. The AUC of the PANTHERIS score in the nonthrombolysis subgroup was 0.80 (0.72–0.86), which was the highest, with a corresponding R2 of 0.17. But in the thrombolysis subgroup, both AUC and R2 for the PANTHERIS scores were the lowest, 0.67 (0.59–0.75) and 0.19, respectively. The AUC and R2 for the AIS-APS score were 0.81 (0.73–0.87) and 0.34. These 2 values were the highest in the thrombolysis subgroup. Moreover, the R2 calculated from the AIS-APS score was also the highest in the nonthrombolysis subgroup and the entire population. However, the P value of AUC comparisons between the 2 subgroups for the PANTHERIS, AIS-APS, A2DS2, and ISAN scores were 0.0827, 0.4183, 0.7981, and 0.7404, showing no statistical differences (Table 3, Figure 1).

Table 3

AUC and goodness of fit for Four SAP scales.

	PHANTHERIS				AIS-APS
	Entire group	Non-T subgroup# (n=138)	T subgroup* (n=138)	P value	Entire group	Non-T subgroup# (n=138)	T subgroup* (n=138)	P value
AUC (95% CI)	0.75 (0.69–0.80)	0.80 (0.72–0.86)	0.67 (0.59–0.75)	0.08	0.80 (0.76–0.85)	0.75 (0.67–0.82)	0.81 (0.73–0.87)	0.42
Goodness of fit (Cox and Snell R2)	0.22	0.17	0.19		0.27	0.22	0.34
	A2DS2				ISAN
	Entire group	Non-T subgroup# (n=138)	T subgroup* (n=138)	P value	Entire group	Non-T subgroup# (n=138)	T subgroup* (n=138)	P value
AUC (95% CI)	0.80 (0.74–0.84)	0.74 (0.66–0.81)	0.76 (0.68–0.83)	0.80	0.76 (0.71–0.81)	0.77 (0.69–0.84)	0.74 (0.66–0.81)	0.74
Goodness of fit (Cox and Snell R2)	0.22	0.18	0.20		0.17	0.13	0.19

Non-T subgroup: Non-thrombolysis subgroup;

T subgroup: thrombolysis subgroup.

Figure 1

The AUC of the two subgroups for the Pantheris, AIS-APS, A2DS2 and ISAN scores.

Discussion

This study was the first comparison of the accuracy for 4 types of existing SAP predictive scores in nonthrombolysis and thrombolysis subgroups of patients with IS. Our main results demonstrated that the 4 SAP scores were not statistically different between the thrombolysis and nonthrombolysis subgroups. We suggest 2 main explanations for these findings.

First, the thrombolysis therapy may not be able to change the state of stroke-induced immunosuppression, which is considered the major pathogenesis mechanism underlying SAP. Usually, ischemic injury induced by an acute stroke triggers immune cells to secrete proinflammatory factors, such as interleukin-6 and tumor necrosis factor-α. If these cytokines are continuously present, the immune cell response may be exhausted, leading to immunosuppression. Three pathways have been reported to account for this immune process including the sympathetic nervous system [19], the parasympathetic nervous system [20], and the hypothalamus-pituitary-adrenal (HPA) axis [21,22]. In the sympathetic nervous system, catecholamines (epinephrine, norepinephrine, and dopamine) are released into the circulation and inhibit the activation of immune cells via downregulation of the level of nuclear factor (NF)-κB through cAMP-PKA-NF-κB and β-arrestin2-NF-κB pathways. In the parasympathetic nervous system, the afferent vagus nerve fibers can sense peripheral inflammatory processes. Then, the efferent fibers release acetylcholine, which exerts immune modulatory function by combining with the α7 nicotinic acetylcholine receptor on macrophages. In the hypothalamus-pituitary-adrenal axis, the hypothalamus produces the corticotropin-releasing factor in response to the inflammatory stress. Consequently, excessive glucocorticoids are secreted and induce lymphocytopenia. Meanwhile, the paraventricular nucleus of the hypothalamus can synchronize the neuroendocrine system with the visceral nervous system. Collectively, patients with acute stroke are prone to immunosuppression and vulnerable to bacterial infection. However, the thrombolysis therapy cannot reverse the pathological process for production of proinflammatory cytokines, as it increases the risk of ischemia-reperfusion damage [23]. Therefore, the risk of immunosuppression may not be reduced even if the vessels in some cases can be recanalized.

Furthermore, according to the previous epidemiological investigation, 58.1% of thrombolysis-treated patients with IS get satisfactory vascular recanalization and 5.8% patients have symptomatic intracranial hemorrhage [24]. In other words, neurological function improvement does not occur in approximately half of patients. Patients with no obvious functional improvement make up a large proportion of the thrombolysis subgroup whose SAP-predicting scores usually remain unchanged.

Our study is the first external validation for the PANTHERIS score in the Chinese population. The discrimination ability of the PANTHERIS score needs to be validated in additional Chinese cohorts. The other 3 SAP scores have all been externally validated and have achieved close AUC values in Chinese populations. Their AUC value ranges were 0.73–0.86 (A2DS2) [25,26] and 0.76–0.79 (AIS-APS) [14, 27] in Chinese cohorts. The AUC for the AIS-APS in our population was 0.80, which is similar to that in a previous validation. Our AUC for the ISAN score is equal to the value calculated from the China National Stroke Registry (AUC=0.76) [27]. It shows that the 3 SAP scores have relatively stable and replicable discrimination ability in Chinese cohorts. Notably, the AIS-APS had the highest goodness of fit and AUC in the total population and the thrombolysis subgroup among all 4 scores. This is possibly because the AIS-APS score is the only one established from a Chinese stroke registry cohort with ethnic similarity to our population. Its detailed grading items guarantee the accuracy as well.

Compared with the SAP incidence in previous studies, the SAP incidence in our study was at the middle level because the patients in our study are from both normal wards and intensive care units. We observed that the condition of patients in the thrombolysis subgroup was worse than in the nonthrombolysis subgroup with regard to stroke severity and WBC. This observation can be explained by the more severe the patients condition, the more likely they are to see doctors within the time window and receive thrombolysis. Although great progress has been made in public stroke education, it still needs continuous promotion and has a long way to go.

Our study has some drawbacks. It is a retrospective cohort study with a relatively small sample size. Our results need further validation in larger samples of thrombolysis-treated patients. Further, the thrombolysis subgroup should be divided into patients with satisfactory and unsatisfactory vascular recanalization in future clinical studies.

Conclusions

The overall performance of all 4 SAP scores was acceptable in our population. No statistically significant differences in AUC were observed between the thrombolysis and nonthrombolysis subgroups. The results preliminarily indicate that applying SAP scores to thrombolysis- and nonthrombolysis-treated patients is safe, although the subgroup analysis was not performed on the existing SAP scores when they were established. Our work offers some insights for the establishment of new SAP-predicting scores and results need to be validated with additional research.