Evaluation of the foal survival score in a Danish-Swedish population of neonatal foals upon hospital admission.

BACKGROUND: It is highly desirable to assess the probability of survival in sick neonatal foals upon admission. The foal survival score (FSS) is a published scoring system used to estimate the probability of survival in hospitalized neonatal foals <4 days old. HYPOTHESIS/
OBJECTIVES: To evaluate the ability of the FSS to predict survival in older foals from a geographically different area compared to the original study. ANIMALS: Five-hundred ninety hospitalized neonatal foals ≤14 days of age.
METHODS: Retrospective Danish-Swedish multicenter study that included details of signalment, history, clinical examination, laboratory results, necropsy findings, and outcome. Scores and score variables were compared between survivors and nonsurvivors using logistic regression. The optimal cutoff and its test parameters were calculated using a receiver operator characteristic curve.
RESULTS: Prematurity, cold extremities, ≥2 infectious or inflammatory sites, blood glucose concentration, and total white blood cell counts were significantly associated with nonsurvival (P ≤ .02). The optimal cutoff to predict survival was ≥6, resulting in sensitivity 78%, specificity 58%, 92% positive predictive value, and 31% negative predictive value. The test performed equally well in foals <4 days old compared to those 4-14 days old. CONCLUSIONS AND CLINICAL IMPORTANCE: Using the suggested optimal cutoff of ≥6, the FSS performed moderately well and may aid in early determination of prognosis for survival. However, the FSS did perform differently in another population and therefore should be assessed under local conditions so that its diagnostic potential is not overestimated.

Acute Physiology and Chronic Health Evaluation

area under the curve

confidence interval

foal survival score

immunoglobulin G

negative predictive value

positive predictive value

receiver operator characteristics

serum amyloid A

systemic inflammatory response syndrome

white blood cell

INTRODUCTION

For sick equine neonates, early outcome prediction is desirable but difficult to determine. Treatment is expensive,1 and prognosis estimation on admission is helpful when communicating with owners. The use of survival scores in intensive care units for humans is now routine for predicting survival, estimating the severity of disease, allowing comparative audits, and facilitating evaluation of treatment interventions.2, 3 Scoring systems developed for general use in adult humans without disease specificity include the Acute Physiology and Chronic Health Evaluation (APACHE), the Simplified Acute Physiology Score and the Mortality Probability Model.2 Favorable characteristics of scoring systems are ease of use, the possibility of predicting outcomes early during hospitalization, reproducibility, cost predictability, and usefulness in different groups of patients.2, 4

For mixed populations of critically ill equine neonates, several models for predicting survival have been developed using historical, clinical, and laboratory variables.1, 5, 6, 7, 8 Two predictive equations for estimating survival1, 5 have been published previously. However, both of these studies have limitations because of small study size (<100 foals), resulting in few statistically significant variables in the final models. A more recent model6 was developed using 1065 foals, but it included variables that cannot be assessed on admission (eg, blood culture results). Another model7 was developed on a large population of foals (n = 577) ≤7 days of age and included variables obtained shortly after admission. However, for calculating the final survival score, a computer‐based model is needed for data analysis. The foal survival score (FSS) reported previously8 included variables that are easy to obtain during or shortly after admission and that are commonly generated as part of the standard diagnostic evaluation of sick foals, including prematurity, cold extremities, presence of ≥2 infectious or inflammatory sites, blood glucose concentration, total white blood cell (WBC) count, and immunoglobulin G (IgG) concentrations. For each of these variables, the foal is assigned a score of 0, 1, or 2, which simplifies calculations of the FSS. This scoring system was developed based on a population of both sick and healthy foals <4 days of age from the United States (n = 339) using a robust generalized boosted regression model. In a subsequent prospective evaluation, the score performed well, with reported sensitivity of 96% and specificity of 71% to predict survival. Compared to other models, the FSS had the highest accuracy (0.91).

In practice, a major difficulty is that scoring systems may be hospital dependent.9 Therefore, it is essential for the score to be tested in multiple foal populations.

Our objective was to assess performance of the FSS in hospitalized neonatal foals of a wider age range (≤14 days) and in different geographic locations (Denmark and Sweden). A secondary objective was to determine the optimal cutoff for predicting survival in the investigated population.

MATERIALS AND METHODS

Animals and data collection

A multicentric retrospective study involving 5 equine hospitals in Denmark and Sweden was performed. All foals ≤14 days old that had a blood sample obtained within the first 24 hours of admission were included. Data collection initially was performed for a retrospective study on serum amyloid A (SAA) concentrations in foals; therefore, it only included the years that participating hospitals measured SAA concentration (Table 1).

Table 1

Participating hospitals, year of data collection, and number of included foals

Hospital	Year	Number of foals
Large Animal Teaching Hospital, University of Copenhagen, Denmark	2007‐2017	109
Swedish University of Agricultural Sciences, Equine Hospital, Sweden	2007‐2017	117
Evidensia Equine Specialist Hospital, Strömsholm, Sweden	2010‐2017	57
Evidensia Equine Specialist Hospital, Helsingborg, Sweden	2012‐2017	282
Højgaard Equine Hospital, Denmark	2017	11

For foals included in the study, signalment (age, breed, sex), history (duration of gestation, presence of dystocia, induced foaling, placentitis, and mare's health status), clinical examination (rectal temperature, heart rate, respiratory rate, presence of cold extremities, presence of petechia or scleral injection, presence of abnormal mentation, and presence of infectious foci), laboratory findings (total leukocyte count, neutrophil count, band neutrophil count, toxic neutrophil changes, l‐lactate, fibrinogen, glucose and IgG concentration, arterial and venous blood gas analysis), blood culture results, diagnosis by treating clinician, necropsy findings, and outcome (survival or nonsurvival) were recorded. When a specific gestation duration was not documented, the included foals were considered to have normal gestation duration as long as they showed no signs of prematurity on clinical examination (small size, short silky hair coat, flexor tendon laxity,10 and domed forehead). Survival was defined as discharge from the hospital, and nonsurvival meant either euthanasia from poor prognosis or death (impending or natural). Foals euthanized because of either financial considerations or unknown outcomes were excluded from the study.

Sepsis was defined as ≥1 of the following: (1) positive blood culture; (2) evidence of localized infection and systemic inflammatory response syndrome (SIRS, Supporting Information 1);11 (3) sepsis score ≥12;12 or, (4) presence of multiple infectious sites on necropsy. The FSS was calculated as previously reported (Table 2).8

Table 2

Variables and scores in the foal survival score8

Variable	Survival		Nonsurvival
	Value	Score	Value	Score
Prematurity	≥320 days	1	<320 days	0
Cold extremities	No	2	Yes	0
≥2 Infection/inflammation sites	No	1	Yes	0
Blood glucose	≥80 mg/dL	1	<80 mg/dL	0
White blood cell count	>4.0 × 109 cells/L	1	≤4.0 × 109 cells/L	0
Immunoglobulin G	≥400 mg/dL	1	<400 mg/dL	0
Total		7		0

Statistical analysis

The association between variables included in the FSS and foal survival was investigated using univariate and multivariate logistic regression analyses (Stata/SE 14.2; StataCorp, College Station, Texas). A receiver operator characteristic (ROC) curve was used to assess the optimal cutoff for the FSS to predict survival in the investigated population (maximizing sensitivity and specificity). This assessment only was performed on foals with no missing data. The area under the ROC curve (AUC) was calculated to evaluate the accuracy of the model.

For the calculation of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), a web‐based statistical software for standard calculations was used (OpenEpi: Open Source Epidemiologic Statistics for Public Health, Version 3.01, www.OpenEpi.com). The population first was analyzed as a whole and then divided into 2 age groups: 0‐3 days (as in the original study) and 4‐14 days. A P‐value <.05 was used to indicate statistical significance.

RESULTS

Foal population

The medical records of 590 equine neonates were reviewed. Ten foals were excluded because of euthanasia for financial reasons and 4 foals because of unknown outcomes, resulting in 576 cases included in the study (Table 3).

Table 3

Sex, breed, and age of the foal population (576 foals)

	Number	%
Sex
Colt	332	58
Filly	227	39
Unknown	17	3
Breed
Warmblood	272	47
Standardbred	147	26
Pony	36	6
Thoroughbred	17	3
Other breed	92	16
Unknown	12	2
Age (days)
Median (IQR)	2 (1‐3)
Mean ± SD	3.0 ± 3.0
Range	1‐14

The median age of the included foals was 2 days (interquartile range, 1‐3), and most foals were presented at age 0‐3 days (n = 440) rather than 4‐14 days (n = 136).

Following the study's sepsis criteria, 156 foals were classified as having sepsis (27%), 416 foals were classified as sick nonseptic (72%), and 4 foals were classified as healthy (1%). The most common diagnoses recorded by the treating clinician were sepsis, meconium impaction, weakness, prematurity, or both, enteritis and umbilical disorders.

In the investigated population (n = 576), 477 foals survived until discharge, and 99 foals died or were euthanized, leading to an overall survival rate of 83% and a mortality rate of 17%. In the group of foals with no missing data used to calculate the FSS (n = 454), the survival rate was 86% and the mortality rate was 14%.

Association between variables of the FSS and nonsurvival

For the 6 score variables, <2% of the data was missing overall. The univariate analysis showed that all included variables (ie, prematurity, cold extremities, ≥2 infectious or inflammatory sites, blood glucose concentration, WBC count, and IgG concentration) were significantly associated with nonsurvival (Table 4, Univariate logistic regression analysis). When incorporated in a multivariate logistic regression model for nonsurvival, prematurity, cold extremities, ≥2 infectious or inflammatory sites, blood glucose concentration, and WBC count were retained in the final model (Table 4, Multivariate logistic regression analysis).

Table 4

Variables included in the foal survival score and their association with nonsurvival score

Variable (number)	Value (number)	Survival (%)	Nonsurvival (%)	OR (95% CI)	P‐value
Univariate logistic regression analysis
Prematurity (564)	<320 days (28)	15/469 (3.2)	13/95 (13.7)	0.21 (0.096‐0.45)	<.001
Cold extremities (572)	Yes (80)	45/474 (9.5)	35/98 (35.7)	0.19 (0.11‐0.32)	<.001
≥2 Infection/inflammation sites (575)	Yes (76)	49/477 (10.3)	27/98 (27.6)	0.30 (0.18‐0.51)	<.001
Blood glucose (513)	<80 mg/dL (80)	50/424 (11.8)	30/89 (33.7)	0.26 (0.15‐0.45)	<.001
White blood cell count (571)	≤4.0 × 109 cells/L (87)	49/475 (10.3)	38/475 (8.0)	0.18 (0.11‐0.29)	<.001
Immunoglobulin G (518)	<400 mg/dL (127)	94/442 (21.3)	33/76 (43.4)	0.35 (0.21‐0.58)	<.001
Multivariate logistic regression analysis
Prematurity (564)	<320 days (28)	15/469 (3.2)	13/95 (13.7)	0.31 (0.12‐0.84)	.02
Cold extremities (572)	Yes (80)	45/474 (9.5)	35/98 (35.7)	0.34 (0.18‐0.65)	.001
≥2 Infection/inflammation sites (575)	Yes (76)	49/477 (10.3)	27/98 (27.6)	0.28 (0.15‐0.54)	<.001
Blood glucose (513)	<80 mg/dL (80)	50/424 (11.8)	30/89 (33.7)	0.45 (0.01‐0.85)	.01
White blood cell count (571)	≤4.0 × 109 cells/L (87)	49/475 (10.3)	38/475 (8.0)	0.25 (0.14‐0.44)	<.001

Abbreviations: CI, confidence interval; OR, odds ratio.

Foal survival score cutoffs to predict survival

The survival rate for each FSS is reported in Table 5. With the original cutoff of ≥4 for survival, the FSS in the Danish‐Swedish population performed with sensitivity of 97%, specificity of 28%, PPV of 89%, and NPV of 64%. The results of the FSS correctly classified the outcome of 397 foals (87%).

Table 5

Survival for score 0‐7 of the foal survival Score (n = 454)

Score	Survival number (%)	Nonsurvival number (%)
0	0 (0)	0 (0)
1	1 (25)	3 (75)
2	1 (13)	7 (88)
3	8 (50)	8 (50)
4	22 (81)	5 (19)
5	52 (78)	15 (22)
6	101 (89)	13 (11)
7	204 (94)	14 (6)
Total	389 (86)	65 (14)

The ROC curve resulted in an optimal cutoff of ≥6 for maximizing sensitivity and specificity to predict survival in the investigated population, with sensitivity of 78%, specificity of 58%, PPV of 92%, and NPV of 31%. The AUC as determined by the ROC curve was 0.74, indicating fair accuracy of the score (Figure 1).

Figure 1

Receiver operating characteristic (ROC) curves of the global score of the 6 variables integrated to the foal survival score for neonatal foals on hospital admission. ─ fitted ROC curve with 95% confidence interval in grey; ● Observed values of the ROC curve; 1 to 7 are the score considered for the determination of the optimal cutoff

With the optimal cutoff for FSS, the calculations correctly classified 343 foals (76%). Of the 111 incorrectly classified foals, 84 (76%) were predicted to be nonsurvivors (total score, 0‐5) but survived. Most of these foals scored close to the cutoff, with a total score of 4 or 5 (n = 74, 88%). In the group of nonsurviving foals that were predicted to survive (scores of 6 or 7) by the FSS (n = 27, 24%), the most common diagnoses were congenital abnormalities (n = 6, 22%), sepsis (n = 6, 22%), meconium impaction (n = 4, 15%), and neonatal maladjustment syndrome (n = 4, 15%).

Foals with FSS of 6 and 7 were 3.5 (95% confidence interval [CI], 1.8‐7.2) and 6.6 (95% CI, 3.4‐13.1) times more likely to survive compared to foals with FSS of 0‐5.

Association between age and outcome in neonatal foals

In the population of foals used to calculate FSS, a significant association was found between age and outcome (P < .001), with survivors being younger on admission (mean, 1.5 days) compared to nonsurvivors (mean, 2.5 days). The mean age of foals classified as septic (3.0 days) or sick nonseptic (2.6 days) was not significantly different (P = .21).

Comparing the different age groups (0‐3 days and 4‐14 days), no significant differences in survival rate (P = .30) or proportion of septic foals (P = .66) were found. The FSS performed with similar sensitivity, specificity, PPV, and NPV in the different age groups (Table 6).

Table 6

Sensitivity, specificity, positive predictive value, and negative predictive value for the foal survival score, survival (%), and sepsis (%) in different age groups

	Cutoff value ≥4 (%)			Cutoff value ≥6 (%)
	0‐14 days	0‐3 days	4‐14 days	0‐14 days	0‐3 days	4‐14 days
Sensitivity	97	97	100	78	78	84
Specificity	28	31	19	58	61	50
Positive predictive value	89	90	85	92	93	89
Negative predictive value	64	60	100	31	30	40
Survival	86	87	82
Sepsis	27	27	29

DISCUSSION

The FSS used in our study showed sensitivity of 78%, specificity of 58%, PPV of 92%, and NPV of 31% with a cutoff of ≥6. In the original study,8 sensitivity of 96% and specificity of 71% were obtained when the FSS was used prospectively with a cutoff of ≥4. The statistically optimal cutoff for maximizing sensitivity and specificity to predict survival in our study was higher than that of the original study, but sensitivity was lower. The better performance of the FSS in the original study was reflected by the higher accuracy of the model (0.91) compared to that of our study (0.74). However, our results support the association between the variables included in the score and outcome, with all 6 variables being associated at statistically significant levels in the univariate regression model and 5 of 6 in the multivariate regression model. Immunoglobulin G concentration <400 mg/dL was the only variable with no statistical significance associated with nonsurvival in the multivariate analysis. A previous study13 investigated the relationship between IgG concentrations and nonsurvival and found an increased risk of mortality for foals with concentrations of <400 mg/dL and 400‐800 mg/dL compared to concentrations >800 mg/dL. It is possible that applying a cutoff of <800 mg/dL to predict nonsurvival in our study may have correlated better with mortality. It is also possible that prompt plasma transfusion received by most foals in our study made the association between this variable and nonsurvival weak. We chose to analyze the association between included FSS variables and nonsurvival in order to make comparisons between our study and the majority of previous studies that linked different variables to mortality. However, this approach is different than that of the original study, where the association between FSS variables and survival was investigated.

When used as a screening test to determine if a patient has the potential to survive, a test with high sensitivity is desirable to avoid unnecessary euthanasia. Applying the suggested cutoff of ≥6 in our study population resulted in moderate sensitivity (78%), suggesting that approximately 1 foal in 5 will be wrongly classified as a nonsurvivor. Applying the original cutoff (≥4) to the current population resulted in excellent sensitivity of 97%, but this change to the cutoff decreased specificity to an unacceptably low value of 28%. The high PPV of 92% in our study indicates that hospitalized foals with total scores of ≥6 have very high likelihood of survival. The specificity of 58% indicates that slightly <50% of the foals will be wrongly predicted to survive. This has less of an impact on the patient but potentially more of an impact on the financial and emotional commitment of the owner. Overall, our study supports the conclusion of the original study that the FSS is better at PPV than NPV. Clinicians therefore should not base their decision to euthanize solely on the FSS.

There were major differences between the population of foals in our study and in the previous study.8 Our study included older foals, a different geographical location and different referral hospitals. Additionally, the definition of sepsis was slightly different from that of the original study, because foals with evidence of localized infection and SIRS,11 presence of multiple infectious sites on necropsy, or both also were included in the our study. Despite broadening the sepsis criteria, our study classified fewer foals as septic (27% versus 38%), indicating that the 2 study populations were in fact different. Recently, the difficulty in identifying foals with sepsis using scoring systems was addressed,14 and the foals included in our study, therefore, potentially were misclassified. The overall survival rate was higher in our study than in the original study population (86% versus 76%). Moreover, surviving foals were younger than nonsurviving foals in our study. A possible explanation for this finding is older age at admission being linked to later referral. For serious conditions such as surgical colic in adult horses, longer referral time has been associated with mortality.15 In foals, this has been poorly investigated, but it is generally agreed upon by clinicians that early referral improves survival.

An important additional finding from our study was the fact that the FSS performed as well in the 4‐ to 14‐day‐old group of foals as in newborns (<4 days). In several other survival models, foals up to 10‐14 days have been included,1, 5, 6 and it is desirable for the scoring method to be useful on a wider age range and thereby on a larger population of foals.

A perfect scoring system to predict survival in equine neonates is thought to be unlikely. This is because some conditions such as congenital malformations may present with few systemic abnormalities and high survival scores but have poor long‐term prognosis, ultimately resulting in euthanasia. Our study confirmed that congenital malformations were indeed common in nonsurvivors despite high scores. Our study also showed that predicting outcome was most difficult in foals with scores of 4‐5. As stated in the original study,8 the FSS may be a valuable tool to predict survival, but it will never fully replace clinical experience in individual cases.

Comparing reported models for foal survival prediction, the FSS is the only model with a simple scoring system compared to less accessible computer‐based models. Only venous blood samples are required, and the blood variables (ie, total WBC count, glucose concentration, and IgG concentration) are accessible for most equine practitioners, potentially even in the field. The original study8 and our study, however, only tested the FSS under referral hospital conditions. Because PPV and NPV are linked to the population in which they are assessed, care should be taken not to extrapolate our findings to a field population without prior assessment of the FSS. In our retrospective evaluation of the medical records, <2% of the data was missing for included variables, indicating that they are already part of the standard evaluation of sick equine neonates and do not require additional work. We therefore believe that the FSS is a useful prognostic tool in clinical settings. Additionally, the ease of score calculation and possible use on a wider age range make the FSS suitable for clinical audits when measuring performance over time or between different intensive care units or when conducting interventional studies where illness severity needs to be defined in a heterogeneous foal population. However, our study highlights the importance of assessing the FSS under local conditions before broadly applying it. Significant differences in its performance can be present, including different optimal cutoffs.

A limitation of our study is its retrospective nature, with missing values and possibly inaccurate recordings. Applying a multicentric approach has the drawback of including many different clinicians and treatment protocols yet does provide data higher in the evidence‐based pyramid. The score itself is not associated with hospital treatment because it is obtained upon admission before initiating treatment. However, the outcome of the patient is expected to be highly dependent on treatment protocols and will subsequently affect the performance of the FSS. The decision to euthanize because of poor prognosis also may vary among clinicians. However, multicentric data collection was thought to be more representative for a geographic region compared to using only a single referral population, which strengthens the application of the FSS in various intensive care units. A general limitation of many studies of equine patients is a small sample size, which has ramifications for creating an accurate scoring system. For example, when the APACHE score was developed for humans, it was based on 100 000 patients16 compared to the 339 cases in the previous foal study.8 It also is important to understand that survival models are not static and must be updated as populations change over time and as diagnostic and treatment options improve.2

In conclusion, the FSS performed with moderate sensitivity (78%), specificity of 58%, NPV of 31%, and high PPV of 92% using a modified cutoff of ≥6 to predict survival. The score was relatively easy to use and included commonly recorded variables obtained during routine foal examinations at intensive care units in Scandinavia. Additionally, the score performed equally well with the inclusion of foals up to 14 days of age.

CONFLICT OF INTEREST DECLARATION

Authors declare no conflict of interest.

OFF‐LABEL ANTIMICROBIAL DECLARATION

Authors declare no off‐label use of antimicrobials.

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEE (IACUC) OR OTHER APPROVAL DECLARATION

Authors declare no IACUC or other approval was needed.

HUMAN ETHICS APPROVAL DECLARATION

Authors declare human ethics approval was not needed for this study.

Supporting Information 1. Definition of the Systemic Inflammatory Response Syndrome (SIRS) by Wong and Wilkins (2015).

Click here for additional data file.