Comparison of different risk stratification systems in predicting short-term serious outcome of syncope patients.

BACKGROUND: Determining etiologic causes and prognosis can significantly improve management of syncope patients. The present study aimed to compare the values of San Francisco, Osservatorio Epidemiologico sulla Sincope nel Lazio (OESIL), Boston, and Risk Stratification of Syncope in the Emergency Department (ROSE) score clinical decision rules in predicting the short-term serious outcome of syncope patients.
MATERIALS AND METHODS: The present diagnostic accuracy study with 1-week follow-up was designed to evaluate the predictive values of the four mentioned clinical decision rules. Screening performance characteristics of each model in predicting mortality, myocardial infarction (MI), and cerebrovascular accidents (CVAs) were calculated and compared. To evaluate the value of each aforementioned model in predicting the outcome, sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio were calculated and receiver-operating curve (ROC) curve analysis was done.
RESULTS: A total of 187 patients (mean age: 64.2 ± 17.2 years) were enrolled in the study. Mortality, MI, and CVA were seen in 19 (10.2%), 12 (6.4%), and 36 (19.2%) patients, respectively. Area under the ROC curve for OESIL, San Francisco, Boston, and ROSE models in prediction the risk of 1-week mortality, MI, and CVA was in the 30-70% range, with no significant difference among models (P > 0.05). The pooled model did not show higher accuracy in prediction of mortality, MI, and CVA compared to others (P > 0.05).
CONCLUSION: This study revealed the weakness of all four evaluated models in predicting short-term serious outcome of syncope patients referred to the emergency department without any significant advantage for one among others.

INTRODUCTION

Syncope is a transient loss of consciousness and postural tone. About 35% (12–48%) of the population may experience syncope at least once in their lifetime. It is the chief complaint of more than 5% of the patients referred to emergency departments (EDs).[1] Annual incidence of syncope in patients over 75 years old has been reported to be about 6% and its therapeutic costs are estimated to be about 2 million dollars per year in the United States.[2] Management of syncope patients is a challenge for emergency physicians since its pathophysiologic processes and causative and stimulating factors have a wide range. Although some of the causative factors have good prognosis, about 11% are accompanied by serious outcomes such as myocardial infarction (MI), cardiac dysrhythmia, hemorrhagic brain insults, and death.[3456789] Therefore, rapid diagnosis of etiologic causes and determining prognosis can significantly improve management of these patients. Clinical decision rules can be helpful in this regard. These clinical models can aid in predicting the outcome and proper screening of patients regarding their need for complimentary diagnostic and therapeutic measures.[10111213141516] Risk Stratification of Syncope in the Emergency Department (ROSE), Boston, Osservatorio Epidemiologico sulla Sincope nel Lazio (OESIL), and San Francisco are among the specifically designed models for syncope.[51718] Since many of these models are newly developed, their value and probable limitations should be addressed before being widely used. There are contradicting findings in this regard;[61920] therefore, the present study aimed to evaluate and compare the values of San Francisco, OESIL, Boston, and ROSE clinical decision rules in predicting the short-term serious outcome of patients referred to the ED with syncope.

MATERIALS AND METHODS

Study design and setting

The present, prospective, diagnostic accuracy study with 1-week follow-up was designed aiming to evaluate the predictive values of San Francisco, OESIL, Boston, and ROSE clinical decision rules in predicting the outcome of syncope patients. The study was carried out in two teaching hospitals, Tehran, Iran, during 1 year from October 2013 to October 2014. Protocol of the study was approved by the Ethics Committee of Shahid Beheshti University of Medical Sciences (IR.SBMU.MSP.REC.1395.137), and informed written consent was obtained from all the participants. Over the course of the study, the researchers adhered to the principles of Helsinki Declaration.

PARTICIPANTS

In the present study, 187 patients who were referred to the ED with complaint of syncope were evaluated using convenience sampling. Age <18 years; pregnancy; not being able to give informed written or oral consent; being diagnosed with a cause other than syncope for loss of consciousness such as hypoglycemia, seizure, poisoning, and head trauma; and drug, substance, and alcohol abuse were considered as exclusion criteria. First impression of the patient was determined by a senior emergency medicine resident and approved by an emergency medicine specialist. In cases of challenge, in addition to accurate history taking and clinical examination, inclusion of the patient was decided based on neurologist consultation. A senior emergency medicine resident and a cardiologist blind to the aims of the study interpreted all the electrocardiograms (ECGs).

Definitions

Syncope

Syncope is a brief loss of consciousness along with loss of postural tone if: (1) Other causes such as migraine, hypoxia, hypoglycemia, seizure, transient ischemic attack, and catalepsy are not probable and (2) symptom initiation is rapid and recovery is complete.[6]

San Francisco model

In this clinical decision rule, the presence of any of the factors such as systolic blood pressure under 90 mmHg, history of cardiac failure, hematocrit <30%, presence of abnormal findings in ECG, and shortness of breath puts the patient in the high-risk group for serious outcome.[6]

Osservatorio Epidemiologico sulla Sincope nel Lazio model

In this rule, the presence of abnormal findings in ECG, history of cardiac failure, absence of warning signs, and age over 65 years put the patients in the moderate- to high-risk group.[3]

Boston model

In this risk stratification rule, the presence of signs of acute coronary syndrome, signs of cardiac conduction disturbances, history of heart failure, cardiac valve disease, history of sudden death in family, persistent abnormal vital signs in ED, decreased circulating volume, and brain lesions puts the patient in the high-risk group.[21]

Risk Stratification of Syncope in the Emergency Department model

In this model, bradycardia <50 times/min, lower gastrointestinal bleeding, hemoglobin under 90 mg/L, chest pain, presence of Q-wave in a lead other than lead III, and oxygen saturation under 94% put the patient in the high-risk group.[17]

Outcome

The evaluated short-term serious outcome in this study included mortality, MI, and cerebrovascular accidents (CVAs) (epidural, subdural, subarachnoid, and brain hemorrhage, brain edema, brain hernia, ischemic stroke) within 7 days of the first referral to the hospital with complaint of syncope.

Data collection

Factors needed for calculating scores for each mentioned model, including demographic data (age, weight, height, sex), paraclinical and ECG findings, vital signs, history of cardiovascular diseases, history of neurologic diseases in the patient or their family, hemorrhagic gastrointestinal disorders, and drug history, as well as brain computed tomography (CT) findings, and 1-week outcomes, were gathered using a predesigned checklist.

Statistical analyses

The required sample size for the study was estimated to be at least 84 cases considering 30% prevalence of short-term serious outcome and 0.1 desired precision (d = 0.1).[22] All the analyses were done using STATA 11.0. Quantitative data were reported as mean and standard deviation and qualitative ones were shown as frequency and percentage. To evaluate the value of each aforementioned model in predicting the outcome, sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio with 95% confidence interval (CI) were calculated and receiver-operating curve (ROC) analysis was done. For identifying the best clinical rule among the four mentioned models, their area under the ROC curve was compared using Cleves and Rock proposed method.[23] Finally, a pooled model was designed by combining all used variables in the 4 models and removing duplicated variables, and its predictive value was determined.

RESULTS

In the present study, 187 patients with the mean age of 64.2 ± 17.2 years (range: 10–98 years) were enrolled (64.0% males). Table 1 shows the baseline characteristics of the participants. The presence of inverted T-wave was the most common abnormal ECG finding with 30 (16.0%) cases. In the 1-week follow-up, mortality, MI, and CVA were seen in 19 (10.2%), 12 (6.4%), and 36 (19.2%) patients, respectively.

Table 1

Baseline characteristics of the patients

Comparison of the models

Table 2 and Figures 1–3 show the screening performance characteristics of the mentioned models in prediction of mortality, MI, and CVA. Area under the ROC curve was 0.57 (95% CI: 0.41–0.65) for San Francisco; 0.54 (95% CI: 0.38–0.70) for OESIL; 0.52 (95% CI: 0.38–0.65) for Boston; and 0.44 (95% CI: 0.31–0.56) for ROSE model. The analyses showed that none of the mentioned models have any advantage over the others in predicting mortality (P = 0.38). In addition, regarding prediction of MI, area under the ROC curve was 0.67 (95% CI: 0.55–0.78) for San Francisco model; 0.62 (95% CI: 0.43–0.80) for OESIL; 0.51 (95% CI: 0.40–0.62) for Boston; and 0.56 (95% CI: 0.40–0.72) for ROSE. Analyses showed that none of the models had any advantage over the others in predicting MI during the following week (P = 0.90). In prediction of CVA in the week after syncope, area under the ROC curve was 0.45 (95% CI: 0.36–0.54) for San Francisco model; 0.47 (95% CI: 0.37–0.57) for OESIL; 0.56 (95% CI: 0.49–0.62) for Boston; and 0.50 (95% CI: 0.39–0.61) for ROSE. No significant difference was seen between these models in risk prediction of CVA (P = 0.31).

Table 2

Screening performance characteristics with 95% confidence interval of the studied models for prediction of mortality, myocardial infarction, and cerebrovascular accidents

Figure 1

Comparing area under the curve of San Francisco, Osservatorio Epidemiologico sulla Sincope nel Lazio, Boston, and Risk Stratification of Syncope in the Emergency Department clinical decision rules in predicting the risk of 1-week mortality in syncope patients

Figure 3

Comparing area under the curve of San Francisco, Osservatorio Epidemiologico sulla Sincope nel Lazio, Boston, and Risk Stratification of Syncope in the Emergency Department clinical decision rules in predicting the risk of 1-week cerebrovascular accidents in syncope patients

Comparing area under the curve of San Francisco, Osservatorio Epidemiologico sulla Sincope nel Lazio, Boston, and Risk Stratification of Syncope in the Emergency Department clinical decision rules in predicting the risk of 1-week myocardial infarction in syncope patients

Pooled model

Screening performance characteristics of the pooled model in prediction of mortality, MI, and CVA were 0.51 (95% CI: 0.35–0.68), 0.65 (95% CI: 0.49–0.81), and 0.48 (95% CI: 0.37–0.59), respectively. The pooled model showed no significant difference in the ability to predict mortality (P = 0.43), MI (P = 0.38), and CVA (P = 0.46) during the week after syncope compared to other models.

DISCUSSION

Findings of the present study revealed the weakness of all four evaluated models in predicting mortality, as well as cardiac and neurologic outcomes, of the patients referred to the ED with syncope. In addition, none of the models showed any significant advantage over the others in the evaluated areas. Sensitivity of the models ranged 47.4–68.2% for prediction of mortality, 66.7–83.3% for MI, and 44.4–100% for CVA. In addition, specificity of the models ranged 33.9–48.8% for mortality, 34.9–50.3% for MI, and 35.1–46.4% for CVA. Only one case of 100% sensitivity was seen in the findings, which was related to the value of Boston model in prediction of CVA. We should note that the presence of brain injury on admission is one of the factors considered in calculation of the Boston model. Elimination of the brain injury factor from Boston model decreases its sensitivity in predicting risk of CVA to 44.4–63.9%. Even combining the studied models could not significantly improve their predictive value. A few studies exist that have aimed to compare the existing clinical decision rules in risk stratification of syncope patients. In these studies, contradicting results have been reported. For example, Puppala et al.[19] in their review that aimed to classify syncope patients expressed that still no optimum clinical decision rule exists for this purpose. They emphasized the value of expert opinion and believed that although predictive models can help, physician's opinion is most important in making the final decision. Costantino et al.[20] also showed that using clinical decision rules has no superiority to expert opinion regarding prediction of short-term outcome of syncope patients. This meta-analysis expressed that decision-making for syncope patients should not be based on clinical models solely. They showed that OESIL model has 78% sensitivity and 56% specificity in predicting 10-day outcome of the syncope patients. They also reported these values as 76% and 53%, respectively, for San Francisco model, and 63% and 61% for EGSYS. Saccilotto et al. showed 87% sensitivity and 52% specificity of San Francisco model in predicting short-term serious outcome.[6] Results of the three mentioned studies were in line with the findings of the present study. Their overall result is that a reliable scale for classifying syncope patients does not exist yet. On the other hand, Plasek et al. reported the sensitivity and specificity of OESIL model in predicting syncope patient outcome as 93% and 54.6%, respectively.[24] In addition, Ebell estimated the sensitivity and specificity of San Francisco model to be 98% and 56%, respectively.[25] As can be seen, the findings of the studies are contradicting in this regard. This might be due to variation of the physicians’ expertise and method of data gathering. For example, in a study by Quinn et al. that led to extraction of San Francisco model, serial ECG evaluations were done over the course of the patient's presence in ED and any abnormal finding was included as a factor.[26] While in most studies including the present one, ECG findings were only evaluated at the time of admission.[27282930]

This problem was not relieved even when the models were combined (pooled model). There are numerous questions and doubts regarding accuracy and reliability of the models for using them widely and independently in daily practice. Further studies in validation of the existing models or making more accurate and powerful new ones are largely helpful and needed.

Limitations

The present study was conducted with 1-week follow-up. Although the findings of this study showed that the mentioned models have low accuracy in prediction of the patients’ short-term outcome, the findings cannot be generalized to long-term ones. In addition, in ROSE model, one of the considered factors was the presence of bradycardia or brain natriuretic peptide over 300 pg. In the present study, this peptide could not be measured in the patients due to limitations. Therefore, only the bradycardia part was taken into account, which might have somehow affected the findings of this model.

CONCLUSIONS

Findings of the present study revealed the weakness of ROSE, Boston, OESIL, and San Francisco models in predicting mortality, MI, and CVA in syncope patients referred to the ED. In addition, none of the models showed any significant advantage over the others in these regards.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

AUTHORS’ CONTRIBUTION

SS, AB, and BH designed the study. LM, SS, and FR were participated in data collection. SS, MMF, and MM participated in analysis and interpretation of the result. SS and LM wrote the first draft of the work. All authors revised the manuscript critically for important intellectual content. All authors had provided final approval of the version to be published and had agreed to be accountable for all aspects of the work.