Factors associated with in-hospital mortality of patients admitted to an intensive care unit in a tertiary hospital in Malawi.

OBJECTIVE: To determine factors associated with in-hospital death among patients admitted to ICU and to evaluate the predictive values of single severely deranged vital signs and several severity scoring systems.
METHODS: A combined retrospective and prospective cohort study of patients admitted to the adult ICU in a tertiary hospital in Malawi was conducted between January 2017 and July 2019. Predefined potential risk factors for in-hospital death were studied with univariable and multivariable logistic regression models, and the performance of severity scores was assessed.
RESULTS: The median age of the 822 participants was 31 years (IQR 21-43), and 50% were female. Several factors at admission were associated with in-hospital mortality: the presence of one or more severely deranged vital signs, adjusted odds ratio (aOR) 1.9 (1.4-2.6); treatment with vasopressor aOR 2.3 (1.6-3.4); received cardiopulmonary resuscitation aOR 1.7 (1.2-2.6) and treatment with mechanical ventilation aOR 1.5 (1.1-2.1). Having had surgery had a negative association with in-hospital mortality aOR 0.5 (0.4-0.7). The predictive accuracy of the severity scoring systems had varying sensitivities and specificities, but none were sufficiently accurate to be clinically useful.
CONCLUSIONS: In conclusion, the presence of one or more severely deranged vital sign in patients admitted to ICU may be useful as a simple marker of an increased risk of in-hospital death.

Introduction

Intensive care in low income countries (LICs) is at an early stage of development despite the high burden of critical illness [1-3]. Good quality critical care provided in intensive care units (ICUs) has the potential to reduce mortality and morbidity among critically ill patients [4].

Illness severity is a major determinant of patient outcomes in an ICU: the sicker the patient at admission the greater the risk of a poor outcome. A patient’s vital signs (heart rate, respiratory rate, blood pressure, conscious level, oxygen saturation) are commonly used as markers of illness severity in hospitals and are components of compound scoring systems [5-7]. Deranged vital signs have shown to correlate with negative outcomes such as cardiac arrest and mortality [8, 9].

Data about intensive care services in LICs are limited. In Malawi, there are gaps in knowledge about patient characteristics, illness severity on admission to ICU and the risk factors for poor outcomes. Understanding the factors associated with poor outcomes is a vital step for understanding which patients are most likely to benefit from the limited ICU capacity and to direct treatment protocols in the ICU. The primary aim of this study was to identify factors associated with in-hospital death among patients admitted to ICU. A secondary aim was to evaluate the ability of single severely deranged vital signs and other severity scoring systems to predict in-hospital mortality.

Materials and methods

A combined retrospective and prospective cohort study was conducted of patients admitted to the adult ICU in Queen Elizabeth Central Hospital (QECH) in Blantyre, Malawi between January 2017 and July 2019.

Setting

QECH is a state-run hospital in the southern region of Malawi with a bed capacity of 1200. It serves an immediate catchment population of 1 million and is a referral center receiving patients from across the country. The adult ICU has four beds and admits patients from all wards and all specialties in the hospital. The adult ICU provides mechanical ventilation, inotropic support and close monitoring and is staffed by an anesthesiologist, anesthetic clinical officers and nurses. On average there is a nurse-to-patient ratio of 1:1. QECH has an additional ICU for paediatric patients. Occasionally the adult ICU will admit paediatric patients when the paediatric ICU is full.

Data collection

All patients admitted to the adult ICU during the study period were included as study participants. In December 2017, the department created an electronic database of all admissions, including data entered from unit’s paper records for patients from 1st January to 30th November 2017 and prospectively directly from the patient from 1st December 2017. The prospective data were collected by the nurses immediately at admission or within 1 hour of admission if there were logistical delays in documentation, using a paper-based data collection tool. The data collection tool collected data on demographics, diagnosis, and ward from which they were admitted, vital signs, lab investigations and interventions received. The data extracted from the records for patients admitted between January and November included the first vital signs recorded during the first hour of admission to the ICU. Lab investigation results were obtained from the patients’ files. Data collection was supervised for quality in the ICU and double-data entered into the database. Follow-up of all the patients continued on the wards until hospital discharge or death. The primary endpoint for the study was in-hospital death. Patients lacking data on hospital outcome were excluded from analysis. The research team accessed the database at the end of July 2019.

Data management

The variables of interest for the primary aim were predefined by the researchers as those that were clinically plausible to be associated with in-hospital mortality. They were 1) any single severely deranged vital sign (Fig 1); 2) treatment with an inotrope or vasopressor; 3) received cardiopulmonary resuscitation; 4) treated with mechanical ventilation; 5) positive HIV status; 6) delayed capillary refill time (>3seconds); 7) sex; 8) undergone surgery before admission to ICU; 9) age (categorised as <50 or ≥ 50 years); 10) nature of admission (planned or emergency); and 11) deranged temperature (<35.5°C or > = 38.5 if aged <1 month and <35 or >40°C if aged > = 1 month). Cardiopulmonary resuscitation comprised advanced life support (ALS) to patients who suffered a cardiac arrest. [10] Cardiac arrest was defined as, “the cessation of cardiac mechanical activity confirmed by the absence of a detectable pulse, unresponsiveness, and apnea (or agonal respirations)” [11]. The definition of “any single severely deranged vital sign” for adults was based on previous studies conducted in Tanzania [12-14] (Fig 1). For children, the definition of “any single severely deranged vital sign” were obtained from a study done in Malawi identifying risk factors for mortality in severely ill children [15]. The blood pressure definitions in children were was based on the paediatric early warning score [16].

Fig 1

Cut–offs for severely deranged vital signs (13, 14).

Data analysis

Data analysis was done with STATA (Release 14, StataCorp, College Station, TX). Descriptive data were summarized using proportions, means, ranges, medians and interquartile ranges where appropriate. Diagnoses were categorised by the researchers into eight groups: 1) serious infection including endometritis, pneumonia, sepsis/septic shock, tuberculosis, malaria and meningitis; 2) non communicable diseases which included cancer, anaemia and unspecified tumours; 3) trauma which included head injury and burns; 4) bowel obstruction and perforation including typhoid perforation 5) acute respiratory disease including asthma and pulmonary oedema; 6) post-delivery and abortion care; 7) pre-eclampsia/eclampsia; 8) other/unknown. Univariable and multivariable logistic regression including all the defined variables were used to generate odds ratios with a 95% confidence interval and significance level of <0.05. Missing data was handled using imputation. The primary method was imputing normal values for missing data with the assumption that normality is more common, and danger-signs are more likely to be documented. To test this assumption, two sensitivity analyses were conducted, firstly using complete case deletion when data were missing and secondly imputing missing data as danger-signs (S1 and S8 Tables). As an additional a-priori planned analysis, analyses of the associations of each individually deranged vital sign with in-hospital death were also conducted.

For the secondary aim, we compared predictive values of binary critical/non-critical scores of the severity scoring systems: Universal Vital signs Assessment (UVA), quick Sequential Organ Failure Assessment (qSOFA), National Early Warning Score (NEWS), Malawi Intensive care Mortality risk Evaluation model (MIME), Tropical Intensive Care Score (TROPICS), TOTAL score (Tachypnoea, Oxygen saturation, Temperature, Alert and Loss of independence) plus the presence of any single severely deranged vital sign [5, 7, 14, 17–20]. These models were chosen for their greater potential feasibility in low resourced settings than other models such as the Acute Physiology and Chronic Health Evaluation (APACHE), the Simplified Acute Physiology Score (SAPS) and the Mortality Probability Model (MPM). A description of the models is provided in S2 Table.

Binary cut-offs of the scoring systems into critical/non-critical scores were used for reasons of simplicity—in our opinion in low-resourced environments with few staff, calculating and using complex compound scores is not feasible. The cutoffs chosen for the critical/non-critical scores for each scoring system were defined in the original papers except for the MIME score, for which we set the cut-off. For the secondary analyses, only patients 16 years of age and above were included and each patient’s qSOFA, UVA, NEWS, MIME and TROPICS scores were calculated from their component variables. Logistic regression analyses were done for each model with the outcome of in-hospital death. To assess the performance of the scoring systems for predicting death, sensitivity, specificity, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated. Additional analyses were conducted with the patients stratified as medical or surgical and whether they were included in the retrospective or prospective data collection periods. Further exploratory analyses were done using different cut-offs and the overall performance of the scores assessed using the Area Under the Receiver Operator Curve (AUROCs).

Ethical clearance was granted by the Malawi College of Medicine Research and Ethics Committee (P.07/18/2433).

Results and discussion

Demographics

A total of 824 patients were admitted in the ICU during the study period. Data on hospital outcome was not found for two patients and thus 822 patients were included in the analysis. The median age of the participants was 31 years, (Inter Quartile Range (IQR) 21–43 years), and 408 were female (50%). Fifty-three percent of the patients were admitted from the operating theatres, and the others were from the emergency departments, the hospital wards and from other hospitals. Patients were admitted from all specialties and were cared for in the ICU for a median of 2 days, (IQR 1–4 days) (Table 1). One hundred patients were under 16 years old—their median age was 5 years (IQR 1–10 years).

Table 1

Patient characteristics.

Variable	n (%) N = 822
Age Median (IQR)	31 (21–43)
Female Sex	408 (50)
Admitted to ICU from
Operating theatres	434(53)
Emergency department	187 (23)
Hospital ward	164(20)
Recovery room	3 (0.4)
Other hospitals	19(2)
Unknown	15 (3)
Admission type
Emergency	713 (87)
Planned	110(13)
Specialty
Surgery*	326 (40)
Medicine	180 (22)
Neurosurgery	132 (16)
Obstetrics and gynaecology	135 (16)
Paediatrics	49 (6)
Had surgery in hospital before admission to ICU	515 (63)
Positive HIV status n/N (%)**	32/114 (28)
Diagnosis
Serious infection***	182 (22)
Non-Communicable disease****	178 (22)
Trauma*****	135 (16)
Bowel perforation or obstruction******	121 (15)
Acute respiratory disease *******	47 (6)
Post-delivery or abortion care	49 (6)
Pre-eclampsia/eclampsia	20 (2)
Other/unknown	90 (11)

* includes Orthopedics, Plastics/Burns, ENT, ophthalmology

** N = Number of patients with known HIV status

***Including Meningitis, Malaria, Endometritis, Pneumonia, sepsis/septic shock and Tuberculosis

**** Cancer, Anaemia, unspecified tumours

*****Including Head injury and Burns

******including Typhoid perforation

******* includes Asthma and Pulmonary Oedema

Illness severity and outcomes

At admission, 525 patients (64%) had one or more severely deranged vital sign. The 24-hour ICU mortality for all the patients was 11% (87 patients), 284 patients died during their ICU stay (35%) and 368 patients died in-hospital (45%).

Factors associated with in-hospital mortality

Using univariable analysis, the following factors were associated with in-hospital mortality: the presence of one or more severely deranged vital sign (unadjusted Odds Ratio (uOR 2.5, 95% CI; 1.8–3.3)); treatment with inotrope or vasopressor (uOR 2.9, 95% CI; 2.0–4.0); received cardiopulmonary resuscitation (uOR 2.3, 95% CI; 1.6–3.3); treatment with mechanical ventilation (uOR 1.9, 95% CI; 1.4–2.5) and age >50 (uOR 1.5, 95% CI; 1.1–2.2). Having had surgery had a negative association with in-hospital mortality (uOR 0.6, 95% CI; 0.4–0.8) (Table 2). In the additional analysis of individual severely deranged vital signs, hypotension (uOR 2.0, 95% CI; 1.4–2.8), hypoxia (uOR 3.2, 95% CI 2.0–5.2) and low consciousness level (uOR 1.8, 95% CI; 1.3–2.4) were found to be associated with in-hospital mortality (S3 Table).

Table 2

Factors associated with in-hospital mortality.

Variable	Number of patients n (%) N = 822	Mortality n1/n2 (%) with the variable	Mortality n1/n2 (%) without the variable	Unadjusted odds ratio	95% CI	p-value	Adjusted odds ratio*	95% CI	p-value
Any severely deranged vital sign	525 (64)	276/525 (53)	92/297 (30)	2.5	1.8–3.3	<0.001	1.9	1.4–2.6	<0.001
Treatment with inotrope or vasopressor	189 (23)	122/189 (65)	246/633 (39)	2.9	2.0–4.0	<0.001	2.3	1.6–3.4	<0.001
Received cardiopulmonary resuscitation	161 (20)	99/161 (62)	269/661 (41)	2.3	1.6–3.3	<0.001	1.7	1.2–2.6	0.006
Treatment with mechanical ventilation	574 (70)	283/574 (49)	85/248 (34)	1.9	1.4–2.5	<0.001	1.5	1.1–2.1	0.024
Positive HIV status	32 (4)	15/32 (47)	353/790 (45)	1.1	0.5–2.2	0.807	0.8	0.4–1.6	0.464
Capillary refill time >3seconds	50(6)	23/50 (46)	345/772 (45)	1.1	0.6–1.9	0.857	0.8	0.4–1.5	0.467
Sex (Male)	414 (50)	194/414(47)	174/408 (43)	0.8	0.6–1.1	0.225	0.9	0.7–1.2	0.421
Had surgery during hospital admission	515 (63)	206/515 (40)	162/307 (53)	0.6	0.4–0.8	<0.001	0.5	0.4–0.7	<0.001
Age group >50	163 (20)	87/163 (53)	281/659 (43)	1.5	1.1–2.2	0.014	1.4	1.0–2.1	0.059
Emergency Admission	712 (87)	320/712 (45)	48/110 (44)	1.1	0.7–1.6	0.797	0.9	0.6–1.4	0.703
Severely deranged temperature	218 (27)	105/218 (48)	263/604 (44)	1.2	0.9–1.6	0.240	1.0	0.7–1.4	0.8609

* Covariates adjusted for were: HIV status, mechanical ventilation, capillary refill time, cardiopulmonary resuscitation, treatment with inotrope/vasopressor, had surgery during hospital admission, admission type, deranged temperature, sex and age

n1 = number of patients that died, n2 = number of patients

In multivariable analysis the same factors were associated with in-hospital mortality; the presence of one or more severely deranged vital sign (adjusted OR 1.9, 95% CI; 1.4–2.6); treatment with inotrope or vasopressor (aOR 2.3, 95% CI; 1.6–3.3); received cardiopulmonary resuscitation (aOR 1.7, 95% CI; 1.2–2.5)); and treatment with mechanical ventilation (aOR 1.5, 95% CI; 1.1–2.1). However the association of age > 50 (aOR 1.4, 95% CI; 1.0–2.1)) with in-hospital mortality was not statistically significant. Having had surgery had a negative association with in-hospital mortality (aOR 0.5, 95% CI; 0.4–0.7).

Predictive values of severity models

The predictive values for in-hospital death of the severity scoring systems using binary cut-offs for the 722 patients of age 16 years and above are shown in Table 3. Four hundred and eighty-one (68%) had one or more severely deranged vital sign. Sensitivity of any severely deranged vital sign for in-hospital death was 76% (95% C.I 72–81) and specificity 42% (95% C.I 37–47). The performance of the severity models did not change substantially following stratification of patients into medical and surgical, whether data were retrospectively or prospectively collected, or when excluding patients with missing data. This excluded TROPICS whose performance improved when only patients with complete data were analysed. (Tables 4 & 5, S5, S6 and S8 Tables).

Table 3

Predictive values of severity score models for patients over 16 years.

	Number with critical score (%) N = 722	Mortality n1/n2 (%) with critical score	Mortality n1/n2 (%) without critical score	Odds Ratio	p-value	95% C.I	Sensitivity % (95%C.I)	Specificity % (95% C.I)	PPV % (95% C.I)	NPV % (95% C.I)
Any severely deranged vital sign	481 (67)	256/481 (53)	79/241 (33)	2.3	<0.001	1.7–3.2	76 (72–81)	42 (37–47)	53 (49–58)	67 (61–73)
NEWS Score = >7	545 (76)	280/545 (51)	55/177 (31)	2.3	<0.001	1.6–3.4	84% (79–87)	32% (27–36)	51 (47–56)	69 (62–76)
qSofa = >2	267 (34)	147/267 (55)	188/455 (42)	1.7	<0.001	1.3–2.4	44 (39–49)	69 (64–74)	55 (49–61)	59 (54–63)
UVA Score > = 5	296 (41.).	154/296 (52)	181/426 (43)	1.5	0.012	1.1–2.0	46 (41–52)	63 (58–68)	52 (46–58)	58 (53–62)
TOTAL Score > = 2	517 (72)	258/517 (50)	77/205 (38)	1.7	0.003	1.2–2.3	77 (72–81)	33 (28–38)	50 (46–54)	62 (55–69)
TROPICSScore > = 8	58 (8)	32/58 (55)	303/664 (46)	1.5	0.164	0.9–2.5	10 (7–13)	93 (90–96)	55 (42–68)	54 (51–58)
MIME score > = 2	474 (66)	234/474 (49)	101/248 (41)	1.4	0.027	1.0–1.9	70 (65–75)	38 (33–43)	49 (44–54)	59 (53–65)

NEWS-. National Early Warning Score; qSOFA—quick Sequential Organ Failure Assessment UVA—Universal Vital signs Assessment (UVA); TOTAL score (Tachypnoea, Oxygen saturation, Temperature, Alert and Loss of independence); TROPICS—Tropical Intensive Care Score; MIME—Malawi Intensive care Mortality risk Evaluation model

Table 4

Predictive values of severity score models for medical patients over 16 years.

	Number with critical score (%) N = 170	Mortality N1/n2 (%) with critical score	Mortality n1/n2 (%) without critical score	Odds Ratio	p-value	95% C.I	Sensitivity % (95%C.I)	Specificity % (95% C.I)	PPV % (95% C.I)	NPV % (95% C.I)
Any severely deranged vital sign	125 (74)	72/125 (58)	20/45 (44)	1.7	0.131	0.9–3.4	78 (68–86)	32 (22–44)	58 (48–66)	56 (40–70)
NEWS Score = >7	132 (78)	73/132 (55)	19/38 (50)	1.2	0.567	0.6–2.5	79 (70–87)	24 (15–35)	55 (46–64)	50 (33–67)
qSofa = >2	71 (42)	42/71 (59)	50/99 (51)	1.4	0.265	0.8–2.6	46 (35–56)	63 (51–73)	59 (47–71)	49.5 (39–60)
UVA Score > = 5	64 (38)	35/64 (55)	57/106 (54)	1.03	0.908	0.6–1.9	38 (28–49)	63 (51–74)	55 (42–67)	46 (37–56)
TOTAL Score > = 2	128 (75)	72/128 (56)	20/42 (48)	1.4	0.331	0.7–2.8	78 (64–86)	28 (19–40)	56 (47–65)	52 (36–68)
TROPICS Score > = 8	7 (4)	4/7 (57)	88/163 (54)	1.1	0.870	0.2–5.2	4 (1–11)	96 (89–99)	57(18–90)	46 (38–54)
MIME score > = 2	129 (76)	73/129 (57)	19/41 (46)	1.5	0.253	0.7–3.1	79 (70–87)	28 (19–40)	57 (48–65)	54 (37–69)

Table 5

Predictive values of severity score models for surgical patients over 16 years.

	Number with critical score (%) N = 552	Mortality N1/n2 (%) with critical score	Mortality n1/n2 (%) without critical score	Odds Ratio	p-value	95% C.I	Sensitivity % (95%C.I)	Specificity % (95% C.I)	PPV % (95% C.I)	NPV % (95% C.I)
Any severely deranged vital sign	356 (64)	184/356(52)	59/196 (30)	2.5	0.000	1.7–3.6	76 (70–81)	44 (39–50)	52 (46–57)	70 (63–76)
NEWS Score = >7	413 (75)	207/413 (50)	36/139 (26)	2.9	0.000	1.9–4.4	85 (80–89)	33 (28–39)	50(45–55)	74 (66–81)
qSofa = >2	196 (36)	105/196 (54)	138/356 (39)	1.8	0.001	1.3–2.6	43 (37–50)	29(24–34)	33 (27–38)	40 (33–46)
UVA Score > = 5	232 (42)	119/232 (51)	124/320 (39)	1.7	0.003	1.2–2.3	49 (43–55)	63 (58–69)	51 (45–58)	61 (56–67)
TOTAL Score > = 2	389 (71)	186/389 (48)	57/163 (35)	1.7	0.006	1.2–2.5	77 (71–82)	34(29–40)	48(43–53)	65 (57–72)
TROPICS Score > = 8	51 (9)	28/51 (55)	215/501 (43)	1.6	0.103	0.9–2.9	12 (8–16)	93 (89–95)	55(40–69)	57(53–62)
MIME score > = 2	345 (63)	161/345 (47)	82/207 (40)	1.3	0.106	0.9–1.9	66(60–72)	41(35–46)	47(41–52)	60 (53–67)

The performances of the severity scoring systems when alternative cut-offs were used can be seen in S7 Table. The AUROCs for the severity scoring systems using all possible scores were 0.60 for any severely deranged vital sign, 0.59 for NEWS, 0.57 for qSOFA, 0.57 for UVA, 0.56 for TOTAL, 0.54 for TROPICS and 0.57 for MIME.

Discussion

We have found that the presence of one or more severely deranged vital sign at admission to ICU in a tertiary hospital in Malawi was associated with in-hospital mortality. In addition, treatment with inotrope or vasopressor, having received cardiopulmonary resuscitation, treatment with mechanical ventilation, and not having had surgery before admission were associated with in-hospital mortality.

The study population had an ICU mortality of 35% and in-hospital mortality rate of 45%. These figures are high but are in-keeping with ICU mortality rates of 27%-64% reported elsewhere in Africa [14, 21–25]. ICU mortality in high resource settings have been reported to range from 8 to 17% [26]. A worldwide survey of ICUs in 2014 found an in-hospital mortality of 22% among the 10,069 included patients [26]. The high mortality rates in our unit could be due to either the admission of severely ill patients with poor prognosis or challenges with provision of good quality of care on the unit. As the majority (65%) of patients had one or more severely deranged vital sign at admission, this may support the first explanation. This proportion of patients is similar to the 69% of patients in an ICU in Tanzania (12). Furthermore, 10.6% of the patients in our study died within 24 hours of admission—many of these patients may not have been salvageable.

Other factors that were associated with mortality were treatment with an inotrope or vasopressor, mechanical ventilation and the use of cardiopulmonary resuscitation in the first hour after admission to the ICU. These factors could be seen as markers of disease severity and failing body physiology. In this cohort, 65% who received vasopressors on arrival to the ICU died. Vasopressors are commonly used in patients in shock. While the underlying cause of the shock leading to vasopressor treatment varied, international guidelines on management of shock include early identification, fluid resuscitation and early usage of vasopressors if fluid unresponsive [27-29]. A meta-analysis of randomized clinical trials (28,280 patients from 177 trials) looking at the effect of inotropes and vasopressors on mortality reported a mortality of 32% in those receiving inotropes/vasopressors [30]. The higher mortality in our study could be due to patients being admitted to ICU at a more advanced stage of their disease process.

Seventy percent of the patients in our unit received mechanical ventilation at admission. This is higher than comparisons with a hospital in Uganda where 18.7% of intensive care patients were mechanically ventilated [31] and in a survey of 361 intensive care units in 20 countries, where 33% of patients received mechanical ventilation [32]. In our unit the main indication for admission is the requirement/need for mechanical ventilation. Critically ill patients requiring mechanical ventilation have been previously found to have higher mortality than those not requiring mechanical ventilation [33, 34].

Globally, reported success rates of cardio-pulmonary resuscitation (CPR) for cardiac arrests in hospital and ICU vary widely, from initial success rates of 16–44% and long-term survival to discharge from hospital of 3–16% [35-39]. Outcome of CPR among in-hospital patients has been reported to be dependent on the early recognition and early initiation of basic life support, early defibrillation as well as prompt institution of advanced cardiac life support [10, 40]. Patients with reversible clinical conditions tend to have better outcome following cardiac arrest and CPR. A high proportion of our patients (20%) received cardiopulmonary resuscitation. Majority were surgical patients admitted from theatre This may reflect the illness severity of the patients on admission to the ICU as well as the quality anaesthetic and surgical services. Thirty-nine percent of the patients in this study that underwent cardiopulmonary resuscitation survived hospital admission. A possible reason for the high success rate may be the young population of patients (median age 31) and few comorbidities such as diabetes mellitus, hypertension or cardiac disease. In the United States, the mean age of patients with in-hospital cardiac arrest is 66 years [39]. Alternatively, a large proportion of the cardiac arrests may have been due to reversible causes, such as a lack of timely identification of an acute deterioration, equipment failure, challenges transporting ill patients from admitting ward to the ICU or other potentially avoidable reasons.

Critical illness as defined by binary cut-offs in each of the severity models had an association with in-hospital mortality (statistically significant in all models except TROPICS). However, in our opinion the performances of the models were too low to be clinically useful as sole factors for individual patient decisions. The performance of models may be possible to increase if all possible values of the model were used, rather than binary scores, but even then the AUROCs in this study were low—between 0.54 and 0.60. The trade-off between performance and usefulness is a challenging one that we have discussed previously [41]. In this study, an a-priori decision based on perceived clinical feasibility and usefulness was made to convert compound scores into binary scores. Additionally, the patients in this study were heterogeneous and the majority were admitted from the operating theatres and emergency departments, with different characteristics than the populations used to derive the scores. Those from theatre may have residual effects of the anaesthesia affecting their vital signs, and patients may have undergone resuscitation and stabilisation of deranged vital signs before transfer to ICU. Other studies have found different performance of the models: in another population in Malawi, qSOFA was found to have a sensitivity of 88% and a specificity of 30.3% [42]. In South Asia, TROPICS had a sensitivity of 70% and a specificity of 69% [18]. One limitation of the TROPICS model in this study is that it requires laboratory values hemoglobin and urea which can be challenging to obtain in our unit. We had very limited data on hemoglobin and urea, and this might have affected performance. The models in this study can be useful for assessing the illness severity of patient cohorts for benchmarking and other comparisons. The key would be ensuring good quality data collection. The missing data in this study may have impacted the performance of the models. Though some of these models were developed utilizing non-ICU patient populations such as emergency departments and medical wards, they are still useful to identify severely ill patients. The component variables for the models can be easily collected during the clinical care of an ICU patient in a low-resource setting and do not require invasive monitors or laboratory measurements, as would APACHE, SAPS or SOFA [7, 43].

The simplest system—use of one or more severely deranged vital sign—performed as well as any of the compound scoring systems in our heterogeneous ICU population. While one study found that compound scores are better than single parameters for predicting poor outcome [44, 45], other studies have found conflicting results [14, 20, 22]. Compound scores have the negative attributes of requiring additional work for the health staff to add up the scores and a risk of incorrect addition [46]. Another challenge with compound scores is that they are difficult to use in settings where all the components are not used in routine clinical care and documentation quality is poor. Additionally, compound scores are unable to indicate potential appropriate treatments, in contrast to single severely deranged vital signs such as suggesting optimising oxygen therapy when hypoxia is identified [47].

Our study has several strengths. We included all patients admitted to the ICU during the study period, providing a large database of patients in a setting from which data are sparse. Our prospective data collection reduced the amount of missing data. The study’s limitations include that it was single-centered study from one teaching hospital and as such findings should be transferred to other settings with caution. We were unable to include data from the ongoing care of the patients in the ICU. Some of the data were collected retrospectively and there were some missing data, especially for GCS (42% of patients were lacking a GCS score) and for capillary refill time. The sensitivity analyses conducted showed no marked variations on estimates when missing data were handled in different ways, apart from for capillary refill time. Transferring the findings from this study to other patient populations should be done with caution given that they were a heterogeneous ICU cohort.

Conclusion

A mixed cohort of patients admitted to an ICU in Malawi with one or more severely deranged vital sign had a high risk of dying—this simple indicator could be a marker of an increased risk of death that could be useful for clinical decision making together with other clinical information. Several other simple parameters and treatments were also found to be associated with an increased risk of death.

Sensitivity analyses of the factors associated with in-hospital mortality when missing data are handled by i) only including patients with complete data and ii) imputed as deranged.

(DOCX)

Click here for additional data file.

Description of models used in the study.

(DOCX)

Click here for additional data file.

Univariable logistic regression of individual vital signs.

(DOCX)

Click here for additional data file.

Percentage of data missing per variable.

(DOCX)

Click here for additional data file.

Predictive values of severity score models for patients over 16 years included in the prospective data collection period.

(DOCX)

Click here for additional data file.

Predictive values of severity score models for patients over 16 years included in the retrospective data collection period.

(DOCX)

Click here for additional data file.

Predictive values of the scoring models for patients over 16 years with different cut offs for defining critical illness used.

(DOCX)

Click here for additional data file.

Predictive values of severity score models for patients over 16 years using cases with complete data only.

(DOCX)

Click here for additional data file.

8 Nov 2021

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Regan Marsh, MD, MPH

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for providing the date(s) when patient medical information was initially recorded. Please also include the date(s) on which your research team accessed the databases/records to obtain the retrospective data used in your study.

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

4. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

5. Please amend the manuscript submission data (via Edit Submission) to include author Singatiya Stella Chikumbanje.

6. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section.

7. Please upload a copy of Figure 2, to which you refer in your text on page 22. If the figure is no longer to be included as part of the submission please remove all reference to it within the text.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for asking me to review the manuscript, “Factors associated with poor outcomes in patients in an Intensive Care Unit in a tertiary hospital in Malawi.” The study covers an understudied and important topic of mortality prediction in critically ill patients from sub-Saharan Africa. The manuscript is well written and a nice contribution to the field. However, the authors may wish to consider the following:

Title

What is meant by “poor outcomes?” Since the study was designed to identify predictors of death in the ICU, would it not be better to explicitly state this in the title?

Methodology

What was the reason for the study design that included both retrospective and prospective data collection? This study design likely had an impact on the amount of missing data, which likely influenced the results of the analysis.

The authors state that data were collected from within one hour of admission to the ICU. Can they provide data regarding time from admission as this is time frame used to derive most of the severity scores tested.

Please provide more details about what imputation strategy was used and why the strategy was chosen.

A sensitivity analysis is described based on analysis of patients with complete data; however, I cannot find the results of the sensitivity analysis in the Results section of the manuscript. Was the sensitivity analysis done?

The greatest limitation of the study is found in the secondary analysis of severity scores. In this analysis, only sens/spec/ppv/npv was assessed using binary cutoffs from the scores. However, this analysis misses an opportunity to better assess the scores for their ability to discriminate for the outcome of death via the calculation of area under the receive operating characteristic curve. AUCs provide a better assessment of the overall discrimination ability of the tests. Using binary cutoffs reduces statistical power and provides a limited assessment particularly when the cutoffs used are essentially arbitrary and not validated in the population being analyzed, as in this case.

Furthermore, the argument that clinicians in LMICs are not capable of calculating simple scores does not carry much water, particularly in ICUs and when smart phones and smart phone apps, which could calculate scores, are so ubiquitous in LMICs including those in Africa. Cutoffs can be useful, but individual clinicians need to assess and validate their priorities for maximizing sens/spec before assessing specific binary cut-offs for clinical use. As well, ppv/npv are dependent upon prevalence of the outcome of the population. Accordingly, if binary cut-offs are to be included, they should not be limited to arbitrary cutoffs, but instead a table should be provided with columns for sens/spec/ppv/npv and rows for each cseverity risk score starting from 0 to the maximum calculated score. As well, AUCs should be calculated for each severity risk score and for the assessment of the aggregate of single vital sign abnormalities.

An additional significant limitation of this study is the application of severity risk scores to a heterogeneous clinical population that is different from the original derivation cohorts of the severity risk scores. In this study, the population was almost 50% post-surgical and clinically heterogeneous, which does not reflect the derivation population cohorts of the severity risk scores. Also, the scores were calculated at the time of admission to the ICU, not necessarily at the time of admission to hospital. Can the authors compare the performance of scores calculated at admission to hospital to those calculated at the time of admission to the ICU? Can the authors stratify the analyses based on medical vs surgical patients and whether data were retrospectively or prospectively collected? Also many patients were transferred from other hospitals so their physiology may already have been altered by preceding resuscitation which would impact the performance of severity risk scores. All of these issues should be detailed as limitations of the study in the Discussion

Results

Please provide a table of % missingness for each clinical variable/predictor used in the analyses. Please also provide data regarding how many patients were excluded from the analysis due to missing outcome data.

Please provide data for the pre-planned sensitivity analysis of patients with complete data

As above, please provide results of AUCs for severity risk scores and aggregate abnormal vital signs

As above, please provide results for sens/spec/ppv/npv for each calculated score of the different severity scores starting at ‘0.’

For the assessment of individual risk factors for death, why was age categorized as < or >50 years? This seems to be an arbitrary cutoff which will lead to decreased statistical power. Age should probably be analyzed as a continuous variable.

Consider creating a figure with a line graph of %mortality plotted against severity risk score results and a similar figure with OR 95%CI plotted against risk score results.

Reviewer #2: Using a combination of retrospective and prospective data, Mtisunge Kachingwe and colleagues aim to identify predictors of poor outcomes in patients admitted to an ICU in Malawi.

My primary comments are related to the data collection and definition of variables and the secondary aim (comparing predictive value of severity scoring systems within this cohort).

Major comments

Methodology

1. Additional details on data extraction would be helpful. What prospective data were extracted using the data sheet? What variables were abstracted from retrospective chart review? How was time of ICU admission determined? If multiple sets of vital signs were recorded within the first hour of ICU admission, which set of vital signs was included in analyses? How were laboratory values extracted? Were laboratory studies only included if they were drawn within the first hour of admission?

2. The authors state that data were extracted from the departmental electronic database of all ICU patients” and “the data were collected prospectively from December 2017 by the nurses at admission or within 1 hour of admission, using a paper-based data collection tool.”

a. In the Data Management section, it is not clear if the variables of interest are restricted to the first hour of admission. For example, for a patient to be considered to have received mechanical ventilation in data analyses, did they have to be ventilated during the first hour of admission or did they count ventilation at any point during ICU admission?

b. Was retrospective data collection restricted to the first hour of admission?

3. Lines 128 to 137: The section on comparison scoring systems (e.g. UVA, NEWS, TropICS) would benefit from additional clarification and explanation.

a. The primary outcome of this study was in-hospital mortality. However, many of the comparison scoring systems were derived and validated to predict different outcomes (e.g. 24 hour mortality, ICU mortality, 3 day mortality) among different populations (e.g. patients on medical wards) from the study population. This should be acknowledged and justified.

b. The authors state that “models were chosen for their greater potential feasibility in low resourced settings.” If this is the case, why did the authors select the NEWS instead of the MEWS which has been studied in Uganda? (PLoS ONE 11(3):e0151408. doi:10.1371/journal.pone.0151408)

c. The authors cite the Modified-MPM from Rwanda later in the manuscript, but do not include it as a reference scoring system. What was the reason for this?

Results

1. No information is provided about the number of records with missing data which were imputed as normal. Also no information provided about the results of the sensitivity analysis discussed in line 125.

2. The relatively low odds ratio for CPR is very interesting. Additionally, the proportion of patients receiving CPR within the first hour of ICU admission is quite high (20%). Additional information about how receipt of CPR was defined for the purposes of these analyses as well as additional data about the patients who received CPR would help interpret this finding.

Discussion

1. Line 261: I do not think the analyses support broad conclusions about “the predictive performance” of the models. Many of the models were evaluated for their ability to predict outcomes they were not designed to predict. This conclusion should be more qualified.

2. The limitation that some data were collected in a retrospective manner should be addressed.

Minor comments

Methodology

1. Line 72: The authors refer to the “main” ICU as the site of data collection. Is information available on the other ICUs in the facility and the types of patients they admit? This would help to understand any potential biases associated with limiting inclusion criteria to one of multiple ICUs at the facility.

2. Line 133: The authors include a citation for a commentary on the NEWS but not for the primary study and the TOTAL score manuscript should be cited here but is not cited until later in the manuscript.

3. Line 143: Authors state that “each patient’s qSOFA, UVA, NEWS, MIME and TROPICS scores were calculated from their vital signs.” However, UVA and TropICS include non-vital sign data.

Results

1. Line 157: “Other hospitals” is listed as a source of admissions to the ICU but this category is not included in Table 1.

2. Given the patient population is very heterogenous the authors may want to consider reporting odds ratios by the subgroups of surgical and non-surgical patients.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

11 Jan 2022

Reviewer #1:

Thank you for asking me to review the manuscript, “Factors associated with poor outcomes in patients in an Intensive Care Unit in a tertiary hospital in Malawi.” The study covers an understudied and important topic of mortality prediction in critically ill patients from sub-Saharan Africa. The manuscript is well written and a nice contribution to the field. However, the authors may wish to consider the following.

We thank the reviewer and have modified the manuscript to address the points made. We believe that the manuscript is now more readable, more informative, and its conclusions more useful to the public

1. What is meant by “poor outcomes?” Since the study was designed to identify predictors of death in the ICU, would it not be better to explicitly state this in the title?

Thank you. The title has been amended to – Factors associated with in-hospital mortality of patients admitted to an Intensive Care Unit in a tertiary hospital in Malawi

Methodology

2. What was the reason for the study design that included both retrospective and prospective data collection? This study design likely had an impact on the amount of missing data, which likely influenced the results of the analysis.

Thank you for your comment. We have included all the patients from the departmental database, which includes those entered retrospectively from January to November 2017. We believe the study is stronger for the inclusion of these additional patients. We have added text in the material and methods to clarify this.

3. The authors state that data were collected from within one hour of admission to the ICU. Can they provide data regarding time from admission as this is time frame used to derive most of the severity scores tested?

The vital signs documented were the first vital signs recorded on admission to the ICU, and the severity scores were based on these vital signs, not on admission to hospital. We do not have data on time of admission to hospital. Please let us know if this answers your query, or if we have misunderstood.

4. Please provide more details about what imputation strategy was used and why the strategy was chosen.

We have provided more details in the data analysis section in the material and methods.

5. A sensitivity analysis is described based on analysis of patients with complete data; however, I cannot find the results of the sensitivity analysis in the Results section of the manuscript. Was the sensitivity analysis done?

Apologies. We have now included the sensitivity analyses in supplementary table 1

6. The greatest limitation of the study is found in the secondary analysis of severity scores. In this analysis, only sens/spec/ppv/npv was assessed using binary cutoffs from the scores. However, this analysis misses an opportunity to better assess the scores for their ability to discriminate for the outcome of death via the calculation of area under the receive operating characteristic curve. AUCs provide a better assessment of the overall discrimination ability of the tests. Using binary cutoffs reduces statistical power and provides a limited assessment particularly when the cutoffs used are essentially arbitrary and not validated in the population being analyzed, as in this case. Furthermore, the argument that clinicians in LMICs are not capable of calculating simple scores does not carry much water, particularly in ICUs and when smart phones and smart phone apps, which could calculate scores, are so ubiquitous in LMICs including those in Africa. Cutoffs can be useful, but individual clinicians need to assess and validate their priorities for maximizing sens/spec before assessing specific binary cut-offs for clinical use. As well, ppv/npv are dependent upon prevalence of the outcome of the population. Accordingly, if binary cut-offs are to be included, they should not be limited to arbitrary cutoffs, but instead a table should be provided with columns for sens/spec/ppv/npv and rows for each severity risk score starting from 0 to the maximum calculated score. As well, AUCs should be calculated for each severity risk score and for the assessment of the aggregate of single vital sign abnormalities.

We agree that this is a really important issue and AUROCs can provide additional information. Indeed, in previous work we have studied AUROCs (for example Kayambankadzanja RK et al. The Prevalence and Outcomes of Sepsis in Adult Patients in Two Hospitals in Malawi. The American journal of tropical medicine and hygiene 2020, Baker T, et al. Single Deranged Physiologic Parameters Are Associated with Mortality in a Low-Income Country. Critical care medicine 2015; 43(10): 2171-9.)

However, for this study, we made the a-priori decision to use binary cut-offs for the scoring systems. This is due to our clinical and systems-based experience that in low-resourced environments with few staff, calculating and using compound scores is not feasible. In the settings where we have worked, including in the hospital in this study, when we have attempted to introduce compound scoring systems we have seen calculation errors, misclassifications, and after some time the scores are not used at all. Additionally, compound scores are unable to indicate potential appropriate treatments, in contrast to single severely deranged vital signs such as suggesting optimizing oxygen therapy when hypoxia is identified. Our aims in this study were to identify factors associated with in-hospital death and to study a binary score (single severely deranged vital signs) as we hypothesized that it could be useful and feasible to use, and compare it to other scores that have been converted to binary scores. The comparison of scoring systems was not the main aim of the study, and we are reluctant to add additional tables and performance measures as it could distract away from the main aim of the study.

We have added text about this in the discussion. We hope this is reasonable

7. An additional significant limitation of this study is the application of severity risk scores to a heterogeneous clinical population that is different from the original derivation cohorts of the severity risk scores. In this study, the population was almost 50% post-surgical and clinically heterogeneous, which does not reflect the derivation population cohorts of the severity risk scores. Also, the scores were calculated at the time of admission to the ICU, not necessarily at the time of admission to hospital. Can the authors compare the performance of scores calculated at admission to hospital to those calculated at the time of admission to the ICU? Can the authors stratify the analyses based on medical vs surgical patients and whether data were retrospectively or prospectively collected? Also many patients were transferred from other hospitals so their physiology may already have been altered by preceding resuscitation which would impact the performance of severity risk scores. All of these issues should be detailed as limitations of the study in the Discussion

Thank you for these good comments. We have added information to the discussion about these points.

We do not have data on the patients’ conditions on admission to hospital.

We have added tables 4 and 5 which show results of the models after stratification based on medical vs surgical patients

We have also added supplementary tables 5 & 6 showing the results after stratification based on prospective and retrospective data

The methods and results sections have been updated to report on these further analyses

Results

8. Please provide a table of % missingness for each clinical variable/predictor used in the analyses. Please also provide data regarding how many patients were excluded from the analysis due to missing outcome data.

Table with percentage of missing data has been added as a supplementary table 4

Two patients were excluded from analysis due to missing outcome data (start of results section)

9. Please provide data for the pre-planned sensitivity analysis of patients with complete data

Thank you. This is in supplementary table 1

10. As above, please provide results of AUCs for severity risk scores and aggregate abnormal vital signs

Thank you for your comment. We have responded in the above comment about AUROCs. We hope this is reasonable and acceptable.

11. As above, please provide results for sens/spec/ppv/npv for each calculated score of the different severity scores starting at ‘0.’

Thank you. We have responded to this in the above comment.

12. For the assessment of individual risk factors for death, why was age categorized as < or >50 years? This seems to be an arbitrary cutoff which will lead to decreased statistical power. Age should probably be analyzed as a continuous variable.

Thank you for the comment. A decision to convert age to binary was made a-priori for feasibility reasons and in keeping with the rest of the factors, even though we agree some statistical power is lost. Malawi has a young population hence the a-priori decision to categorize with a cut of 50.

13. Consider creating a figure with a line graph of %mortality plotted against severity risk score results and a similar figure with OR 95%CI plotted against risk score results.

Thank you for this recommendation. For this study we did not analyze for severity risk scores. We hope we understood the comment and that is acceptable.

Reviewer #2:

My primary comments are related to the data collection and definition of variables and the secondary aim (comparing predictive value of severity scoring systems within this cohort).

We thank the reviewer and have modified the manuscript to address the points made. We believe that the manuscript is now more readable, more informative, and its conclusions more useful to the public

Methodology

1. Additional details on data extraction would be helpful. What prospective data were extracted using the data sheet? What variables were abstracted from retrospective chart review? How was time of ICU admission determined? If multiple sets of vital signs were recorded within the first hour of ICU admission, which set of vital signs was included in analyses? How were laboratory values extracted? Were laboratory studies only included if they were drawn within the first hour of admission?

Thank you for your comments. We have added additional details to the materials and method section, data collection section. All patients admitted to the ICU during the study period were included as study participants. In December 2017, the department created an electronic database of all admissions, including data entered from the paper records for patients from 1st January 2017 and prospectively from December 2017. The prospective data were collected by the nurses immediately at admission or within 1 hour of admission if there were logistical delays in documentation, using a paper-based data collection tool. The data collection tool collected data on demographics, diagnosis, and ward from which they were admitted, vital signs, lab investigations and interventions received. The data extracted from the patients admitted between January and November included the first vital signs recorded during the first hour of admission to the ICU. Lab investigation results were obtained from the file. Data collection was supervised for quality in the ICU and double-data entered into the database. Follow-up of patients continued on the wards until hospital discharge or death. The primary endpoint for the study was in-hospital death. Patients lacking data on hospital outcome were excluded from analysis.

2. The authors state that data were extracted from the departmental electronic database of all ICU patients” and “the data were collected prospectively from December 2017 by the nurses at admission or within 1 hour of admission, using a paper-based data collection tool.”

a. In the Data Management section, it is not clear if the variables of interest are restricted to the first hour of admission. For example, for a patient to be considered to have received mechanical ventilation in data analyses, did they have to be ventilated during the first hour of admission or did they count ventilation at any point during ICU admission?

b. Was retrospective data collection restricted to the first hour of admission?

Thank you. This has been clarified in the data management section. The variable data was obtained from the data collection tool in which information was recorded within an hour of admission.

We have now clarified this in the data collection section. The retrospective data was also restricted to admission or within 1 hour of admission

3. Lines 128 to 137: The section on comparison scoring systems (e.g. UVA, NEWS, TropICS) would benefit from additional clarification and explanation.

A description of the models is now provided in supplementary table 2.

a. The primary outcome of this study was in-hospital mortality. However, many of the comparison scoring systems were derived and validated to predict different outcomes (e.g. 24 hour mortality, ICU mortality, 3 day mortality) among different populations (e.g. patients on medical wards) from the study population. This should be acknowledged and justified.

Thank you for this comment. We agree with the reviewers on this observation and have acknowledged this in the discussion. Assessments of the performances of the severity scoring systems has limitations as the population in this study had different case-mix characteristics and was exclusively an ICU population

b. The authors state that “models were chosen for their greater potential feasibility in low resourced settings.” If this is the case, why did the authors select the NEWS instead of the MEWS which has been studied in Uganda? (PLoS ONE 11(3):e0151408. doi:10.1371/journal.pone.0151408)

Thank you for your comment. We opted for NEWS as it more widely used. There are some studies that suggest that NEWS is better for example; Smith et al The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death, Resuscitation 84 (2013) 465–470

c. The authors cite the Modified-MPM from Rwanda later in the manuscript, but do not include it as a reference scoring system. What was the reason for this?

Thank you. We considered use of MPM score from Rwanda however we opted not to use it as we anticipated challenges to collect data on the variable of “confirmed or suspected infection”.

Results

4. No information is provided about the number of records with missing data which were imputed as normal. Also no information provided about the results of the sensitivity analysis discussed in line 125.

Table with amount of missing variable has been added as supplementary table 4

5. The relatively low odds ratio for CPR is very interesting. Additionally, the proportion of patients receiving CPR within the first hour of ICU admission is quite high (20%). Additional information about how receipt of CPR was defined for the purposes of these analyses as well as additional data about the patients who received CPR would help interpret this finding.

Thank you. We have amended the materials and method (data analysis) section to define CPR. Cardiopulmonary resuscitation consisted of advanced life support (ALS) to patients who suffered a cardiac arrest. (10) Cardiac arrest was defined in the Utstein style as “the cessation of cardiac mechanical activity confirmed by the absence of a detectable pulse, unresponsiveness, and apnea (or agonal respirations)”

The discussion section has been amended as follows

A high proportion of our patients (20%) received cardiopulmonary resuscitation. Majority were surgical patients admitted from theatre. This may reflect the illness severity of the patients on admission to the ICU as well as the quality anesthetic and surgical services. Thirty-nine percent of the patients in this study that underwent cardiopulmonary resuscitation survived hospital admission. A possible reason for the high success rate may be the young population of patients (median age 31) and few comorbidities such as diabetes mellitus, hypertension or cardiac disease. In the United States, the mean age of patients with in-hospital cardiac arrest is 66 years (39). Alternatively, a large proportion of the cardiac arrests may have been due to reversible causes, such as a lack of timely identification of an acute deterioration, equipment failure, challenges transporting ill patients from admitting ward to the ICU or other potentially avoidable reasons

Discussion

6. Line 261: I do not think the analyses support broad conclusions about “the predictive performance” of the models. Many of the models were evaluated for their ability to predict outcomes they were not designed to predict. This conclusion should be more qualified.

Thank you. The text and the conclusion have been modified.

7. The limitation that some data were collected in a retrospective manner should be addressed.

Thank you. This has been added to the discussion section.

Methodology

8. Line 72: The authors refer to the “main” ICU as the site of data collection. Is information available on the other ICUs in the facility and the types of patients they admit? This would help to understand any potential biases associated with limiting inclusion criteria to one of multiple ICUs at the facility.

The hospital has two ICUs. The second one recently established caters for pediatric patients and was not included in this study. Occasionally the adult ICU will admit pediatric patients when the pediatric ICU is full. We have changed from “main” to “adult” in the manuscript and clarified in the manuscript.

9. Line 133: The authors include a citation for a commentary on the NEWS but not for the primary study and the TOTAL score manuscript should be cited here but is not cited until later in the manuscript.

Thank you for this observation. The citing of TOTAL has been amended.

For NEWS the following article is cited as Royal College of Physicians. National Early Warning Score ( NEWS ) - Standardising the assessment of acute-illness severity in the NHS. Report of a working party. 2012. 47

We hope we understood this comment correctly.

10. Line 143: Authors state that “each patient’s qSOFA, UVA, NEWS, MIME and TROPICS scores were calculated from their vital signs.” However, UVA and TropICS include non-vital sign data.

Amended the terminology to variables

Results

11. Line 157: “Other hospitals” is listed as a source of admissions to the ICU but this category is not included in Table 1.

Table has been amended to say other hospitals

12. Given the patient population is very heterogenous the authors may want to consider reporting odds ratios by the subgroups of surgical and non-surgical patients.

Thank you. We have added tables 4 and 5 which report results based on surgical and medical patients.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

12 May 2022

Factors associated with in-hospital mortality of patients admitted to an Intensive Care Unit in a tertiary hospital in Malawi

PLOS ONE

Dear Dr. Mtisunge Kachingwe,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We appreciate your efforts for the study and the authors have made a careful revision to the manuscript. However, there are some important points that are required to define clearly. Please carefully respond to the reviewers’ comments and suggestions particularly the AUC for each score and some points in the discussion.

Please submit your revised manuscript by Jun 26 2022 11:59 PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Vipa Thanachartwet, M.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have made a careful evaluation of the reviewer comments and have responded with appropriate revisions to the manuscript. However, concern remains about the use of cut-offs for the severity risk scores. Given the use of scores in a heterogeneous and unvalidated ICU patient population, the authors should provide more information about the performance of the scores. Sens/spec/ppv/npv can go up or down depending on the cut-off used and depending on the priority of the clinician. For example, in the data provided, NEWS >/=7 has a high sens and low spec; whereas, qSOFA >/=2 has low sens and high spec. Presumably, these could be reversed by simply adjusting the cut-offs for each score. Since the cut-offs are arbitrary and we don't know the optimal cut-off for each score evaluated in this population, the authors should provide sens/spec/ppv/npv for each value within each severity risk score so they can be appropriately evaluated. To use the prior example again, perhaps 7 and 2 are simply not the correct cut-offs (if one exists) in this population for NEWS and qSOFA, respectively. The overall AUC for each score would also be useful information and should be reported. If the authors are unable or unwilling to provide these data, then they should consider not including the secondary aim of evaluating severity risk scores. Finally, the large amount of missing GCS data should be further emphasized as a limitation particularly for GCS evaluation alone and for scores which include GCS, i.e. UVA. Accordingly, each risk score should be added to Supplementary Table 1 to show the effect of missing data on the performance of the risk scores. Ideally, this would include how AUC changes.

Reviewer #2: Dr. Kachingwe and colleagues have carefully revised their manuscript.

Major comments

Discussion

1) The second part of the Discussion is dedicated to deranged vital signs and severity scores as predictors of in-hospital mortality. While I agree with the authors’ central argument—that the presence of one or more severely deranged vital signs on admission to an ICU in Malawi is a feasible prognostic marker for increased risk of in-hospital death—I think this conclusion needs further qualification due to the heterogeneity of the study population and the rate of missing data. Lines 331 to 339 provide an excellent summary as well as clear and helpful context. I think this is the core of the part of the discussion and would consider streamlining the rest of this section.

a. Study population heterogeneity: The authors state that “assessments of the performances of the severity scoring systems has limitations as the population in this study had different case-mix characteristics.” However, this also seems to be applicable to single deranged vital signs. I would consider acknowledging this and including discussion on how this limitation may have influenced the results.

b. Missing data: Please expand on the limitation that “there are some missing data”. There were significant rates of missing data, for the variables capillary refill time and Glasgow coma scale in particular. By providing the two sensitivity analyses in supplementary table 1 (i.e., case wise deletion of missing data and imputed abnormal data), the authors do an excellent job helping the reader to understand the potential impact of missing data on their estimates. I think the manuscript would benefit from a more detailed discussion in the text comparing estimates from the primary analysis with the two sensitivity analyses in table S1.

1) The NEWS score and any severely deranged vital sign had very similar point estimates in the primary analysis (Table 3). It therefore strikes me as inconsistent to say that “performances of the models were too low to be clinically useful by themselves for individual patient decisions” and to also suggest that severely deranged vital signs have clinical utility.

Minor comments

1. The authors state: “critical illness as defined by the binary cut-offs in each of the severity models showed a significant association with in-hospital mortality.” However, not all of the severity model binary cut-offs had a significant association with in-hospital mortality.

2. Tables 4 and 5: Percentages are missing from the first data column

3. Supplementary table 1: What is the total number of patients with complete data included in these analyses?

4. The manuscript needs additional copy editing.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

24 Jul 2022

Dear Editorial board members,

RE: Response to reviewers

We thank the two reviewers for their comments on our manuscript titled “Factors associated with in-hospital mortality of patients admitted to an Intensive Care Unit in a tertiary hospital in Malawi”. Below is our response to each point raised by the reviewers. We hope that we satisfyingly addressed them and that the manuscript will be now suited for publication.

Sincerely,

On behalf of all authors,

Dr Mtisunge Kachingwe

Reviewer #1:

The authors have made a careful evaluation of the reviewer comments and have responded with appropriate revisions to the manuscript. However, concern remains about the use of cut-offs for the severity risk scores. Given the use of scores in a heterogeneous and unvalidated ICU patient population, the authors should provide more information about the performance of the scores. Sens/spec/ppv/npv can go up or down depending on the cut-off used and depending on the priority of the clinician. For example, in the data provided, NEWS >/=7 has a high sens and low spec; whereas, qSOFA >/=2 has low sens and high spec. Presumably, these could be reversed by simply adjusting the cut-offs for each score. Since the cut-offs are arbitrary and we don't know the optimal cut-off for each score evaluated in this population, the authors should provide sens/spec/ppv/npv for each value within each severity risk score so they can be appropriately evaluated.

Thank you for this comment! We have further analysed the scoring systems using every possible cut-off for a critical score and present all the performance measures in Supplementary Table 7.

To use the prior example again, perhaps 7 and 2 are simply not the correct cut-offs (if one exists) in this population for NEWS and qSOFA, respectively. The overall AUC for each score would also be useful information and should be reported. If the authors are unable or unwilling to provide these data, then they should consider not including the secondary aim of evaluating severity risk scores.

Thank you for this. The AUCs for each scoring system has now been added to the results section.

Finally, the large amount of missing GCS data should be further emphasized as a limitation particularly for GCS evaluation alone and for scores which include GCS, i.e. UVA. Accordingly, each risk score should be added to Supplementary Table 1 to show the effect of missing data on the performance of the risk scores. Ideally, this would include how AUC changes.

We thank you for your comment. We have added text to the discussion to highlight the missing data, especially for GCS and capillary refill time. We have added a Supplementary Table 8 to show the performance of the severity scores with just complete data.

Reviewer #2: Dr. Kachingwe and colleagues have carefully revised their manuscript.

Major comments

Discussion

1) The second part of the Discussion is dedicated to deranged vital signs and severity scores as predictors of in-hospital mortality. While I agree with the authors’ central argument—that the presence of one or more severely deranged vital signs on admission to an ICU in Malawi is a feasible prognostic marker for increased risk of in-hospital death—I think this conclusion needs further qualification due to the heterogeneity of the study population and the rate of missing data. Lines 331 to 339 provide an excellent summary as well as clear and helpful context. I think this is the core of the part of the discussion and would consider streamlining the rest of this section.

Thank you for these comments. We have qualified our conclusions in the discussion. And we agree that the discussion is long, however it is hard to streamline further than has already been done, as several parts were requested to be added during the review process. I hope that is ok.

Study population heterogeneity: The authors state that “assessments of the performances of the severity scoring systems has limitations as the population in this study had different case-mix characteristics.” However, this also seems to be applicable to single deranged vital signs. I would consider acknowledging this and including discussion on how this limitation may have influenced the results.

Thanks, we have amended this to clarify that it concerns all results.

b. Missing data: Please expand on the limitation that “there are some missing data”. There were significant rates of missing data, for the variables capillary refill time and Glasgow coma scale in particular. By providing the two sensitivity analyses in supplementary table 1 (i.e., case wise deletion of missing data and imputed abnormal data), the authors do an excellent job helping the reader to understand the potential impact of missing data on their estimates. I think the manuscript would benefit from a more detailed discussion in the text comparing estimates from the primary analysis with the two sensitivity analyses in table S1.

Thank you, this has been added.

1) The NEWS score and any severely deranged vital sign had very similar point estimates in the primary analysis (Table 3). It therefore strikes me as inconsistent to say that “performances of the models were too low to be clinically useful by themselves for individual patient decisions” and to also suggest that severely deranged vital signs have clinical utility.

Thank you for this comment. We have clarified that we meant that all the scoring systems, including single severely deranged vital signs, do not have good enough performances to be used as sole markers for individual patient decisions, but that single severely deranged vital signs could be useful as a marker of an increased risk of death together with other clinical information.

Minor comments

1. The authors state: “critical illness as defined by the binary cut-offs in each of the severity models showed a significant association with in-hospital mortality.” However, not all of the severity model binary cut-offs had a significant association with in-hospital mortality.

Thank you. The sentence has been amended.

2. Tables 4 and 5: Percentages are missing from the first data column

Thank you, this has been added

3. Supplementary table 1: What is the total number of patients with complete data included in these analyses?

Thank you for this observation. The total number of patients with complete data per each variable/factor has been added to the supplementary table 1.

4. The manuscript needs additional copy editing.

The manuscript has been proof-read again by English language experts and changes made.

Submitted filename: Response to Reviewers 2.0.docx

Click here for additional data file.

12 Aug 2022

Factors associated with in-hospital mortality of patients admitted to an Intensive Care Unit in a tertiary hospital in Malawi

PONE-D-21-17302R2

Dear Dr. Mtisunge Kachingwe,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Vipa Thanachartwet, M.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

All issues were revised according to the reviewers' comments and suggestions.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: Dr. Kachingwe and colleagues have done an excellent job revising their manuscript--all comments are thoughtfully addressed.

The manuscript topic is of significant interest to the field, the data are well-presented, and the conclusions are well-reasoned and justified. I have no additional comments and congratulate the authors for their work.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Paul D Sonenthal, MD

**********

19 Aug 2022

PONE-D-21-17302R2

Factors associated with in-hospital mortality of patients admitted to an Intensive Care Unit in a tertiary hospital in Malawi

Dear Dr. Kachingwe:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Associate Professor Vipa Thanachartwet

Academic Editor

PLOS ONE