Establishment and evaluation of a simplified evaluation system of acute respiratory distress syndrome.

PURPOSE: In recent years, a variety of acute respiratory distress syndrome (ARDS) evaluation systems have been developed worldwide; however, they are not so convenient for the doctors clinically, we decided to establish and evaluate a simplified evaluation system of ARDS (SESARDS).
MATERIALS AND METHODS: Data from 140 ARDS patients (derivation data set) were collected to screen for prognostic factors affecting outcomes in ARDS patients. By logistic regression analysis, scores were allocated to corresponding intervals of the variables, respectively, by means of analysis of the frequency distribution to establish a preliminary scoring system. Based on this primary scoring system, a definitive evaluation scheme was created through consultation with a panel of experts. The scores for the validation data set (92 cases) were assigned and calculated by their predictive mortality with the SESARDS and acute physiology and chronic health evaluation II (APACHE II). The performance of SESARDS was compared with that of APACHE II by means of statistical analysis.
RESULTS: The factors of age, pH, Glasgow coma scale (GCS), oxygenation index (OI), and the lobes of lung were associated with prognosis of ARDS respectively. The sensitivity and specificity of SESARDS for the validation data set were 96.43% and 58.33%, respectively. On the AUC, no significant difference between APACHE II and SESARDS was detected. There were no significant differences between the prediction and the actuality obtained by SESARDS for the validation data set the SESARDS scores were positively correlated with the actual mortality.
CONCLUSION: SESARDS was shown to be simple, accurate and effective in predicting ARDS progression.

INTRODUCTION

Presently, the mortality for acute respiratory distress syndrome (ARDS) is high.1,2 Jiang and Hu3 proposed that the currently available methods for evaluating ARDS disease severity are not reflective of the actual severity and can be inconsistent. However, these evaluation systems can be effective in indicating the degree of critical illness as well as in evaluating treatment and outcomes. In recent years, a variety of ARDS evaluation systems, such as acute physiology and chronic health evaluation (APACHE), simplified acute physiology score (SAPS), mortality probability models and multi-organ-dysfunction score, have been used worldwide; however, they are not so convenient for the doctors clinically, especially for those working in undeveloped and developing regions of the world. Recently, ARDS score was developed based on lung injury score at an American-European Consensus Conference in 1994; however, it has its limitations. On one hand, it is difficult to assess early lung injury in ARDS patients without mechanical ventilation, while on the other hand, positive end expiratory pressure (PEEP) for improvement of an oxygenated effect is dependent on time and the individual. Heffner, et al.4 reported that both lung injury score and ARDS score are accurate and affirmative in the diagnosis of ARDS, but they are no more effective than SAPS score in predicting mortality. Therefore, a more practical and simplified evaluation system for ARDS would be of great benefit.

MATERIALS AND METHODS

Subjects and grouping

From June 1999 to October 2008, data from the medical records of 232 patients from the Department of Respiratory Diseases of Nanfang Hospital, Shunde First People's Hospital, Guangzhou Chest Hospital; Department of Respiratory Diseases of Huizhou People's Central Hospital; Department of Respiratory Diseases, Pearl River Hospital, Southern Medical University; Xinhui People's Hospital, the Fifth Affiliated Hospital of Sun Yat-sen University; Xiangzhou People's Hospital of Zhuhai; Gaoming People's Hospital of Foshan; Guizhou Hospital of Foshan and Shenzhen Third People's Hospital were included in our study. Data were randomized into derivation data set (n=140) and validation data set (n=92). The derivation data set was subdivided into death group (n=90; male 67, female 23; averaged 41.85+14.51 years) and survival group (n=50; male 35, female 15; averaged 41.30+13.86 years); the validation data set was also divided into a death group (n=56; male 38, female 18; averaged 49.05+20.54 years) and survival group (n=36; male 26, female 10; averaged 46.56+18.36 years). Use patient records for our study was approved by Nanfang Hospital and patient confidentiality was maintained.

Diagnosis criteria for ARDS patients

Diagnosis of ARDS was made according to the criteria proposed at the American-European Consensus Conference on ARDS in 1994. In all 232 eligible patients, diagnostically, the ratio of the partial pressure of arterial oxygen (PaO2) to the fraction of inspired oxygen was less than 200 (adjusted if the altitude exceeded 1000 m), and bilateral infiltrations were evident on chest radiography, consistent with the presence of pulmonary edema without evidence of left atrial hypertension.5

Methods

Data collection

Physiological measurements, age and previous health status were used to calculate APACHE II score. Clinical parameters for ARDS were obtained within the first 24 hours after the diagnosis of ARDS was made, and included 1) age, sex, and comorbidities; 2) respiratory rate, blood pressure, body temperature, and heart rate; 3) white cell count and hematocrit; 4) serum level of sodium, potassium, creatinine and glucose; 5) arterial blood gas including pH, PaO2, PaCO2, and oxygenation index (OI); 6) Glasgow coma scale (GCS); 7) presence of ARDS (p); 8) number of risk factors for ARDS; 9) number of lobes involved; 10) amount of extra-pulmonary organ dysfunction or failure; and 11) vital status at discharge.

Assignment for indexes

In 1998, Gattinoni, et al.6 reported that pathological changes in pneumonia-induced ARDS were significantly different, in addition to the effects on PEEP from ARDS caused by abdominal disease. Accordingly, he classified ARDS into two categories: ARDS (p) and ARDS (exp), and then assigned scores of 1 to the former and 0 to the latter, when applying them to statistical treatment. For our statistical purpose, likewise, we handled the score assignments in this study for the four lobes of the lung (left-up, left-down, right-up and right-down) according to the standards for evaluating acute lung lesions. According to the scope of inflammation on chest imaging, we assigned scores of 1-4 for the four lobes, respectively. Then, we assigned scores of 1-5 for the total number of indicators of nonpulmonary organ dysfunction present, including alimentary tract hemorrhage, acute renal failure, liver function failure, nervous function failure, and function failure of the hematological system.

Statistical analysis

Statistical analysis was conducted based on the retrospective case-control study, and results of the descriptive statistics were presented as means±SD. The comparison between two means was analyzed by independent-sample t-test, and the frequency distributions in categorized variables were compared using Pearson's chi-square test. The risk factors affecting ARDS prognosis were screened by univariate logistic regression and multivariate logistic regression (forward conditional). Then, a probabilistic equation for a simplified evaluation system of acute respiratory distress syndrome (SESARDS) was established by full-factor logistic regression, and thereafter, the quartile frequency distribution method was used to establish a preliminary valuation system, from which a definitive evaluation scheme of SESARDS was developed and revised after consultation with a panel of experts. Finally, SESARDS and APACHE II were compared in regarding to validity, sensitivity, and specificity using receiver operating characteristics curves (ROC) curves. SPSS software version 13.0 (SPSS Inc., Chicago, IL, USA) was used for statistical analysis and plotting of graphs, scatter plots and ROC curves, as well as calculation of the AUC of ROCs.

RESULTS

Baseline demographics

Statistically, the differences between the two sets were insignificant in regards to age, sex, etiology of ARDS and mortality (Table 1).

Table 1

Baseline Demographics for Both Data Sets

ARDS, acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.

Factors influencing ARDS prognosis

After univariate logistic regression analysis, the variables of diastolic blood pressure, P(A-a)O2, pH, hematocrit, GCS, OI, the sum of lung field, amount of nonpulmonary organ dysfunction and MBP were used in a logistic regression model to screen for factors influencing the prognosis of ARDS, which included age, pH, GCS, OI and lobes of the lung (Table 3).

Table 3

Factors Influencing ARDS Prognosis on Multivariate Logistic Regression Analysis

ARDS, acute respiratory distress syndrome; GCS, Glasgow coma scale; OI, oxygenation index.

Development of SESARDS

Frequency analysis

The five factors for the derivation data set were treated with frequency analysis for their respective values when the percentiles were set at 25%, 50% and 75%.

The results of frequency analysis were used to establish the grades of each factor, taking into consideration the actual values of all five factors. For the actual age and lobes of the lung, we assigned to 0-25% a score of 0, 25-50% a score of 1, 50-75% a score of 2 and to 75-100% a score of 3, for both variables. For the two variables of pH and OI, we assigned to 0-25% a score of 3, 25-50% a score of 2, 50-75% a score of 1 and to 75-100% a score of 0. For the GCS variable we assigned to 0-75% a score of 1 and to 75-100% a score of 0.

The weights of the variables

Referring to Wang,7 based on the non-parametric statistics, we sent e-mails to 15 experienced experts asking them to sort these five variables according to their level of importance as indicators on the prognosis of patients with ARDS. The order of the sorted items was to be marked as serial numbers: from number 1 (as the most important) to number 5 (as the least important). To calculate the weight of each variable, we combined the serial numbers (1 to 5) with the Weight Calculation Sheet and used the following formula:

F formulation: F=∑[(n+1-i)fi], where n=5, i=serial number and fi=the frequency of rank i (Table 6).

Table 6

The Weights of the Variables

GCS, Glasgow coma scale; OI, oxygenation index.

Establishment of SESARDS

All the weights of the variables together with their grades were revised to establish the final SESARDS (Table 7).

Table 7

The SESARDS Severity of Disease Classification System

GCS, Glasgow coma scale; OI, oxygenation index.

The probability of death equation

Subsequently, the scores of SESARDS, as an independent variable, were applied to logistic regression using the possibility of death as a dependent variable, to work out the probability of death equation:

Comparisons between SESARDS and APACHE II

The Validity of SESARDS

The ROC curves of SESARDS and APACHE II

ROC was used to test the validity of SESARDS. The results showed no significant differences between SESARDS and APACHE II, statistically (Fig. 1).

Fig. 1

ROC curves of SESARDS and APACHE II. ROC, receiver operating characteristics curves; SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.

The sensitivities, specificities and accuracies of SESARDS and APACHE II

SESARDS and APACHE II were used on the validation data set for the comparisons of prediction of survival/death and actuality of survival/death (Table 8).

Table 8

Comparisons between SESARDS and APACHE II in Sensitivity, Specificity and Accuracy

SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.

The reliability of SESARDS

Comparisons between SESARDS and APACHE II were made in regards to their reliability by way of Lemshow-Hosmer test.

The relationship between SESARDS score and actual mortality

SESARDS was used to evaluate and score the cases studied from which actual mortalities were calculated. Finally, the SESARDS scores and the actual mortalities were compared, shown to be positively correlated (Table 10, Fig. 2).

Table 10

The Relationship between SESARDS Score and Actual Mortality

SESARDS, simplified evaluation system of acute respiratory distress syndrome.

Fig. 2

The co-relationship line between SESARDS score and actual mortality. SESARDS, simplified evaluation system of acute respiratory distress syndrome.

There were significant differences between the survival group and death group for both data sets, in view of SESARDS score (10.726±3.296 vs. 6.162±3.136)

DISCUSSION

An effective prediction system can accurately predict changes in patients, but should consist of relatively simple variables. This paper presents a newly developed SESARDS as well as the validation results thereof. As a severity of disease classification system, SESARDS uses points based upon the initial values of age, pH, GCS, OI, and lobes of the lung. In this study, an increasing score (range 0 to 18) was shown to be closely related with subsequent risk of hospital death in ARDS. When SESARDS scores were evaluated for the ability to determine ARDS prognoses, the results were strong and stable. Our data further indicated that classification could be made appropriately if done at an early point in time. This would allow for the severity classification to be more independent from treatment.

APACHE II, established by Knaus, et al.8 in 1985, is one of the most widely used evaluation systems and also the most authoritative method for scoring critical illness at present. It is characterized by excellent performance in predicting mortality in ARDS patients over lung injury score, focusing only on the severity of respiratory failure. APACHE II and SAPS II scores have been previously reported to be independent predictors of hospital mortality in ARDS patients.9-12 According to Oh, et al.,13 APACHE II score is closely related to mortality in patients in the Intensive Care Unit (r=0.81). However, APACHE II is obtained by dividing the aggregate sum by the number of components used, and the 14 components used in this system may make it more difficult for doctors to use it clinically.14

After development, we applied SESARDS to the validation data set for evaluation of its reliability and validity in comparison to APACHE II, and found that there were no significant differences in accuracy in predicting ARDS incidence, nor sensitivity or specificity. Importantly, the AUC of the ROC for SESARDS was not different from that of APACHE II statistically. Moreover, it was found that there was no significant difference between the predictive results and the actual results when SESARDS was used on the validation data set, indicating that SESARDS was satisfactory in regards to goodness-of-fit. At last, our study provided evidence that SESARDS score was positively correlated with the actual mortality ratio. Moreover, SESARDS is relatively simpler than APACHE II (5 variables compared to 14).

It should be emphasized that in the first 24 hours SESARDS did not perfectly predict death rates in individual patients. However, our data indicated that misclassification rates became smaller as the probability of death increased (Fig. 1). Nevertheless, the specificity of SESARDS (58.33%) was unsatisfactory, though it is based on a larger number of samples, compared to several other ARDS-specific studies.15,16

In conclusion, SESARDS was shown to be simple, accurate and effective in predicting ARDS progression.

Table 2

Results of Univariate Logistic Regression Analysis

GCS, Glasgow coma scale; OI, oxygenation index; DBP, diastolic blood pressure; MBP, mean blood pressure; Hct, hematocrit.

*Amount of nonpulmonary organ dysfunction.

Table 4

Results of Frequency Analysis

GCS, Glasgow coma scale; OI, oxygenation index.

Table 5

The Initial SESARDS

SESARDS, simplified evaluation system of acute respiratory distress syndrome; GCS, Glasgow coma scale; OI, oxygenation index.

Table 9

The Reliabilities of SESARDS and APACHE II Via Lemshow-Hosmer Test

SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.

Table 11

Comparison between SESARDS and APACHE II

SESARDS, simplified evaluation system of acute respiratory distress syndrome; APACHE, acute physiology and chronic health evaluation.