Validating a novel index (SWAT-Bp) to predict mortality risk of community-acquired pneumonia in Malawi.

BACKGROUND: Community-acquired pneumonia is a major cause of mortality worldwide. Early assessment and initiation of management improves outcomes. In higher-income countries, scores assist in predicting mortality from pneumonia. These have not been validated for use in most lower-income countries. AIM: To validate a new score, the SWAT-Bp score, in predicting mortality risk of clinical community-acquired pneumonia amongst hospital admissions at Queen Elizabeth Central Hospital, Blantyre, Malawi.
METHODS: The five variables constituting the SWAT-Bp score (male [S]ex, muscle [W]asting, non-[A]mbulatory, [T]emperature (>38°C or <35°C) and [B]lood [p]ressure (systolic<100 and/or diastolic<60)) were recorded for all patients with clinical presentation of a lower respiratory tract infection, presumed to be pneumonia, over four months (N=216). The sensitivity and specificity of the score were calculated to determine accuracy of predicting mortality risk.
RESULTS: Median age was 35 years, HIV prevalence was 84.2% amongst known statuses, and mortality rate was 12.5%. Mortality for scores 0-5 was 0%, 8.5%, 12.7%, 19.0%, 28.6%, 100% respectively. Patients were stratified into three mortality risk groups dependent on their score. SWAT-Bp had moderate discriminatory power overall (AUROC 0.744). A SWAT-Bp score of ≥2 was 82% sensitive and 51% specific for predicting mortality, thereby assisting in identifying individuals with a lower mortality risk.
CONCLUSION: In this validation cohort, the SWAT-Bp score has not performed as well as in the derivation cohort. However, it could potentially assist clinicians identifying low-risk patients, enabling rapid prioritisation of treatment in a low-resource setting, as it helps contribute towards individual patient risk stratification.

Introduction

Lower respiratory tract infections (LRTIs) are the fourth most common cause of death globally, responsible for 3.1 million deaths annually, and remain the leading cause of death in low-income countries1. Pneumonia (LRTI with radiological evidence) accounts for 14% of deaths in Malawi2. At Queen Elizabeth Central Hospital (QECH), Blantyre, Malawi, there are approximately 1500 pneumonia admissions annually, with 20% in-patient mortality rate3, similar to other sub-Saharan countries4. Effective management relies upon rapid assessment, diagnosis and treatment. Accurate early assessment of disease severity is essential to patient triage5–7. Determining the need for hospital admission is medically and economically important, and initial assessment influences subsequent management decisions8,9. Clinical assessment of pneumonia frequently fails to accurately estimate the risk of clinical deterioration10–12. Severity scores provide a structured and objective severity assessment, and can improve the identification of high and low risk patients13,14.

Widely-used severity scores include the CURB-65 and CRB-65 scores10,15,16 and the pneumonia severity index (PSI)12. These have been validated in many countries including Canada, New Zealand, Spain and Pakistan5,10,17,18. However, their application in areas of high HIV prevalence, specifically sub-Saharan Africa, is not well studied. Indeed, the eligibility criteria for these scores exclude patients with HIV infection12,16. Birkhamshaw et al. showed that the sensitivity of the CRB-65 score in a Malawian hospital for a score of ≥2 was low (36.4%) compared to the UK (76.8%)16,19. This renders the CRB-65 score insensitive at predicting pneumonia mortality risk in this setting20. Birkhamshaw et al. proposed a novel score (SWAT-Bp) incorporating five variables independently associated with disease19. The constituent variables are: male [S]ex; muscle [W]asting; non-[A]mbulatory; [T]emperature (>38°C or <35°C); [B]lood [p]ressure (Systolic Pressure<100 and/or Diastolic Pressure<60). Mortality was positively correlated with overall score. We present prospective data from Malawi against which the SWAT-Bp score is validated. The aim was to describe the sensitivity and specificity of the score in a validation cohort.

Methods

The prospective cohort study was conducted on the medical wards of Queen Elizabeth Central Hospital. This 1,000-bed government hospital in Blantyre, Malawi, has typical staff ratios of 1 nurse to 50 patients. We approached all consecutive adults (≥16 years) admitted between 19th February and 1st April or 6th October and 27th November 2011, with a preliminary clinical diagnosis of community-acquired pneumonia. Patients were included if they had a clinical presentation of LRTI and a presumed primary diagnosis of CAP on admission in accordance with the British Thoracic Society primary care guidelines: (1) symptoms of acute lower respiratory tract illness, (2) new focal chest signs on examination, (3) evidence of systemic illness, (4) no other explanation for the illness and (5) a clinical decision that it should be treated with antibiotics.21Exclusion criteria were: (1) alternative primary admission diagnosis [not CAP], (2) hospitalisation within the preceding 14 days (excluding hospital-acquired pneumonia), (3) pneumonia as an expected terminal event in disseminated Kaposi's sarcoma, and (4) symptoms for more than 21 days. Ethical approval was obtained from the University of Birmingham BMedSc Population Sciences and Humanities Internal Ethics Committee, UK, and the College of Medicine Research and Ethics Committee, Blantyre, Malawi.

Data collection

Eligibility was confirmed from patient notes within 48 hours of admission. The first recorded temperature and blood pressure were used to derive the score. The other variables were measured on enrolment, within 48 hours of admission. Muscle wasting was defined as mid-upper arm circumference (MUAC) ≤190mm22 and non-ambulatory was defined as ‘unable to walk unaided’. The MUAC was measured for each patient during enrolment and the ambulatory status by verbal confirmation from the participant or relative (if not documented). Paper based data records were pseudo-anonymised using study numbers for confidentiality.

Patient outcomes were established by daily review and a triangulation of data from ward discharge books, electronic discharge system and paper records office. For analysis, patients were classified as ‘alive at discharge’ or ‘died in-hospital’. Those with indeterminable outcomes were excluded from the analysis and deemed lost to follow-up. Inpatients were followed up for a maximum of 30 days or discharge, whichever was sooner. Clinicians treating the patients were not involved in this research study. Hence, they were blinded to the scoring system, and this would not have impacted patient management.

Statistical analyses

We estimated that from a population of 200 patients, validation of sensitivity would be possible to within 7% and specificity to within 8.3%, assuming 80% sensitivity and 75% specificity16,23. Odds ratios of inpatient death were calculated for each variable in the SWAT-Bp score. Sensitivity, specificity, positive predictive values (PPV), negative predictive values (NPV) and area under the receiving operating characteristic (ROC) curve were calculated using PASW Statistics 17 manufactured by IBM.

Results

Between 19th February – 1st April and 6th October – 27th November 2011, 234 patients with a preliminary diagnosis of community-acquired pneumonia met the eligibility criteria (Figure 1). Of these, 18(7.7%) were excluded due to loss to follow-up. Of the 216 remaining participants included, 90 (41.7%) were male. Mean age was 37 years (median 35, range 16–79). HIV status was unknown for 45 patients (20.8%); 144 were HIV positive (84.2% of known HIV status). Overall inpatient mortality rate was 12.5% (27); 48% (13) of these deaths occurred within 48 hours and 70% (19) within one week of admission. Baseline characteristics were similar between included participants and those lost to follow-up (Table 1).

Figure 1

Flow-chart demonstrating the screening and follow-up of study participants

Table 1

Characteristics of participants included in the analysis and those lost to follow-up. Statistical comparisons were made

	Participants included in analysis		Participants lost to follow-up		Comparison
	N	%	N	%	Test	Value	P-value*
Total cohort	216	100	18	100	−	−	−
Male	90	41.7	10	55.6	χ2	1.3	0.25
In-hospital Mortalities	27	12.5	−	−	−	−	−
HIV positive (% of known)	144	84.2	10	90.9	χ2	0.36	0.55
HIV unknown	45	20.8	7	38.9	χ2	3.1	0.078
Median admission (days)	6	−	−	−	−	−	−
Median age (years)	35	−	32	−	Independent Samples Median	2.96	0.086
Age 16–24	35	16.2	2	11.1	-	-	-
Age 25–34	67	31.0	11	61.1
Age 35–44	66	30.5	3	16.7
Age 45–54	21	9.7	1	5.6
Age 55+	27	12.5	1	5.6
SWAT-Bp score
0	43	19.9	2	11.1	-	-	-
1	59	27.3	6	33.3
2	55	25.5	5	27.8
3	42	19.4	4	22.2
4	14	6.5	0	0
5	3	1.4	1	5.6

P-value of <0.05 is statistically significant,

The SWAT-Bp score of ≥2 demonstrates a sensitivity of 81.5% (95% CI 61.3–93.0) and specificity of 51.3% (95% CI 44.0–58.6). The SWAT-Bp score could further divide patients into three groups; 47% of participants score 0 or 1 with 4.9% mortality, 26% of participants score 2 with 12.7% mortality and 27% score 3 or above with 25.4% mortality. The moderate performance of the SWAT-Bp score in predicting mortality risk is demonstrated by the area under the ROC curve of 0.744. The most significant predictors for mortality were male sex (Odds Ratio 4.0) and being non-ambulatory (Odds Ratio 4.1) (Table 4). For these variables, sensitivity and specificity were 70.4% and 62.4% respectively for male sex and 66.7% and 67.2% for non-ambulatory status. Negative predictive value was 93.7% and 93.4% respectively (Table 3).

Table 4

Individual predictor variables in the SWAT-Bp score separated by number of patients scoring for that variable, number of mortalities and Odds Ratios

Individual SWAT-Bp Variables	Patients scoring N	Mortalities N (%)	Odds Ratio (OR)	95% Confidence Intervals
				Lower	Upper
Gender	90	19 (21.1)	4.0	1.6	9.5
Muscle wasting	39	9 (23.1)	2.7	1.1	6.5
Ambulatory status	80	18 (22.5)	4.1	1.7	9.6
Temperature	90	15 (16.7)	1.6	0.7	3.6
Blood Pressure	67	13 (19.4)	2.7	1.2	6.1

Table 3

Test characteristics for mortality prediction for the SWAT-Bp score

SWAT-Bp cut-off point	Sensitivity % (95% CI)	Specificity % (95% CI)	PPV % (95% CI)	NPV % (95% CI)
≥0	100 (84.5–100)	0 (0–2.5)	12.5 (8.5–17.8)	−
≥1	100 (84.5–100)	22.8 (17.1–30.0)	15.6 (10.7–22.1)	100 (89.8–100)
≥2	81.5 (61.3–93.0)	51.3 (44.0–58.6)	19.3 (12.7–28.0)	95.1 (88.4–98.2)
≥3	55.6 (35.6–74.0)	76.7 (70.0–82.4)	25.4 (15.4–38.7)	92.4 (86.7–95.8)
≥4	25.9 (11.9–46.6)	94.7 (90.2–97.3)	41.2 (19.4–66.5)	89.9 (84.7–93.6)
5	11.1 (2.9–30.3)	100 (97.5–100)	100 (31.0–100)	88.7 (83.5–92.5)
Gender	70.4 (49.7–85.5)	62.4 (55.1–69.3)	21.1 (13.5–31.2)	93.7 (87.5–97.0)
Non-ambulatory	66.7 (46.0–82.8)	67.2 (60.0–73.7)	22.5 (14.2–33.5)	93.4 (87.4–96.7)

PPV = Positive predictive value; NPV = Negative predictive value

Figure 3 demonstrates how the SWAT-Bp score could be used in clinical practice to help to stratify the mortality risk of patients attending hospital with clinical presentation of a lower respiratory tract infection and a presumed primary diagnosis of pneumonia.

Figure 3

The SWAT-Bp score is a severity assessment tool based on five adverse clinical features, for use in a hospital setting. This one-step strategy enables stratification of patients with community-acquired pneumonia into groups according to their risk of mortality.

Discussion

This study validates the novel SWAT-Bp score which could be used to stratify patients into three risk groups.19 A SWAT-Bp score of 0 or 1 indicates milder clinical pneumonia with 4.9% chance of mortality (2.2% in derivation cohort), likely suitable for outpatient management or early discharge from hospital. A score of 2 indicates moderate pneumonia with 12.7% mortality risk (7.4% in derivation), requiring hospital admission and management. Scoring ≥3 indicates severe pneumonia with one in four patients dying in-hospital (one in two in derivation), necessitating immediate hospitalisation and appropriate management. The objective variables of which the score consists are easily assessed on admission, enabling rapid triage in busy clinical settings and reducing bias, while requiring little training to administer. Differences in mortality risks from the derivation cohorts could be due to numerous factors including seasonal variation, variation in microbiological patterns and change in resources over time. Alternatives include patients lost to follow-up differing from the patients included, but these options are all speculative and therefore no definitive conclusions can be drawn. This study is conducted on in-patients and hence suggests that approximately half this population could be suitable for outpatient management or early discharge (those scoring 0 or 1). A reduction of this magnitude would be clinically and economically useful. Although the NPV for a score of 0 or 1 indicates that approximately 95% of these patients would survive, evidently, discharging all these patients with a one in 20 mortality risk would be perilous. It is important to highlight that this score is designed for pragmatic use during assessment in combination with clinical judgement. Reflecting on these statistics, one can further differentiate the management of the lower mortality risk scores. A score of 0 correlates to 0% mortality risk indicating outpatient management may be appropriate. A score of 1 indicates a relatively low risk of mortality, where caution should be heeded with discharge counselling used to facilitate early discharge. However, this score has not been validated for outpatient use and our data collection did not focus on individual management of patients whilst hospital inpatients. We can, therefore, not assume that these patients could all have been managed as outpatients as we do not know whether their survival was in part associated with provision of medical management that is unavailable in the community, such as oxygen therapy or intravenous antibiotics.

The sensitivity of a SWAT-Bp score of ≥2 (81.5%) indicates that this score aids in identification of patients at lower risk of deterioration. Practically speaking, if patients with a score of ≥2 deteriorate, they would already be in an appropriate setting in which higher level treatment may be available. The SWAT-Bp score stratifies patients with clinical pneumonia according to mortality risk, demonstrated by an AUROC of 0.744 in this study, and 0.867 in the derivation study.19 In the original validation cohort of the CURB-65 and CRB-65 scores, comparatively slightly better values were obtained for sensitivity (75% and 80% respectively) and specificity (75% and 61%), as well as the AUROC for the CURB-65 (0.79) and CRB-65 (0.76) scores.16 The CURB-65 and CRB-65 scores are widely used in higher-income countries, where there is much evidence supporting their accuracy.10,12,16,17 Despite recommendations that they should be revalidated for use in novel environments, few such studies have been reported. One study in Nigeria demonstrated strong sensitivity and specificity of the CURB-65 score, but the sample size was small (80) and HIV prevalence was low (14.3% amongst those with known status; a high proportion did not undergo HIV testing).24 Conflicting evidence exists regarding whether pneumonia in HIV-infected patients differs from HIV-negative individuals, in terms of clinical presentation and outcome.25–28 Therefore, it may be inappropriate to apply these scores to assess pneumonia severity in Malawi, where approximately 11% of adults in the community29 and 80% of those presenting to hospital with CAP19 are infected with HIV. The CRB-65 score was found to be less sensitive (sensitivity 36.4%, specificity 80.6%, AUC 0.649) in stratifying patients presenting with LRTI suggestive of pneumonia in Malawi than the SWAT-Bp score.19 Patients with pneumonia in Malawi are younger; the average life expectancy is 58 years compared to 80 in the UK.2,29 Simple, rapid pneumonia severity assessment tools are needed which perform well in regions of high HIV rates and few resources. A recent study in Uganda derived a new four-point score for stratification of mortality risk among HIV-infected individuals with pneumonia. It consisted of tachycardia, tachypnoea, hypoxia and CD4 count <50cells/mm3 and stratified patients with incremental mortality risk.30 This score required limited laboratory data but only included patients who were HIV positive which is unlikely to be pragmatic.

The most significant variables from the SWAT-Bp score in predicting mortality risk are simple objective measures: males (OR 4.0) and non-ambulatory status (OR 4.1). Ambulatory (NPV 93.4%) and female (NPV 93.7%) patients appear to be less likely to die from pneumonia in this study population. Sex disparity has been demonstrated amongst tuberculosis patients in South Africa; a smaller proportion of female subjects provided smear-positive sputum samples than males. This was attributed to a higher index of suspicion in females and delays in health-seeking behaviours in males.31 Additionally, male sex has previously been demonstrated as an independent risk factor for severe sepsis in Malawi.32 We cannot be certain if our study findings reflect increased female health-seeking behaviours or whether the most unwell females never attend hospital. However, it is also worth noting that the most significant variables in the validation cohort differed from the derivation cohort, making it difficult to draw precise conclusions about the importance of each individual variable in the composite score.

Strengths and limitations

Strengths of this study include the prospective design and hence inclusion of all patients with a primary diagnosis of pneumonia on admission, despite only 127 (63.2% of known discharge diagnoses) having a diagnosis of pneumonia at end-point. The inclusion of these patients is pragmatic and represents widespread practice in resource-limited settings. Evidently, it is clinically challenging to correctly diagnose pneumonia on admission, as demonstrated by a study in Uganda where only 9.6% of patients presenting with clinical pneumonia were confirmed bacterial pneumonia on discharge. Pulmonary tuberculosis accounted for over half of the incorrect pneumonia admission diagnoses.30 Additionally, by incorporating two separate data collection periods at different times throughout the year could help adjust for seasonality changes; it would be better to conduct data collection over an entire year to ensure more complete adjustment for this factor. To our knowledge, this is the only severity assessment score for pneumonia that has been derived and validated with two separate cohorts from a resource-limited setting with high HIV rates, rendering it a useful contribution to current research.

However, this study was primarily limited by the relatively small sample size (and hence small number of deaths) in addition to the limitation of being a single centre trial. This perhaps enables scope for revalidation in other centres in sub-Saharan Africa. The international definition of CAP is an acute infection of pulmonary parenchyma with acute infiltrate demonstrated on chest x-ray.16,33 In this setting, universal radiological confirmation was prevented by resource limitations, especially portable radiographs for the most unwell patients. Our loss to follow-up rate of 7.5% reflects limitations in clinical data collection, incomplete patient records systems and complicated patient flow. These factors feed into wider than envisaged confidence intervals (61–93% for sensitivity and 44–58% for specificity at a score of ≥2). Baseline characteristics were similar in those lost to follow-up and those included. The incorporation of community follow-up in future research would be beneficial to determine outcomes for discharged patients and enable validation of the score in an outpatient setting.

Conclusion

The SWAT-Bp score has been validated in a new population sample to predict mortality risk from clinical community-acquired pneumonia on admission to hospital in Malawi. It could be used as an adjunct to stratify patients with pneumonia into low, intermediate and high risk mortality groups, thereby aiding management decisions. It could be most useful in identification of lower-risk patients who may be suitable for early discharge or outpatient treatment. However, the score did not perform as well as in the derivation cohort.

Table 2

Relationship between the SWAT-Bp score and risk of mortality

SWAT-Bp Score	Patients N	Mortalities N (%)	95% Confidence Intervals (CI)
			Lower	Upper
0	43	0 (0)	0	8.2
1	59	5 (8.5)	3.7	18.3
2	55	7 (12.7)	6.3	24.0
3	42	8 (19.0)	10.0	33.3
4	14	4 (28.6)	11.7	54.6
5	3	3 (100)	43.8	100