Association is not the same as accuracy.

Dear Editor,

In a recent article, dos Santos et al. have concluded that, for patients with acute coronary syndromes, risk scores correlated with coronary artery disease severity[1]. That apparently positive conclusion hides the poor performance of those scores to predict obstructive coronary artery disease due to two reasons. Firstly, in face of a diagnostic situation (presence of coronary artery disease ≥ 50%), the clinical focus should be on accuracy. Secondly, association (described in the article as correlation) does not ensure accuracy.

The major focus of that study should have been accuracy measures, such as area under the ROC curve, ranging from 0.56 to 0.70. Although statistically significant, those values indicate low accuracy from the diagnostic viewpoint. Corroborating those findings, in a recently published article in this same journal, our group has concluded that the degree of association (between scores and coronary anatomy) is not sufficient for risk scores to accurately predict the results of angiography[2].

The different conclusions from two studies of similar findings is due to the perception that statistical significance (true association) and clinical relevance (accuracy) do not exactly represent the same phenomenon.

When Santos et al[1] have concluded that risk scores correlate with coronary anatomy, they considered all analyses performed: statistical correlation analysis with the non parametric Spearman test[2] and the predictive ability of those scores to discriminate individuals who might and might not have a coronary artery lesion ≥ 50%, which was initially determined by using the non-parametric Mann-Whitney test[2], and was later assessed by using C statistics (area under the ROC curve)[3]. As shown in the results, that study emphasizes both analyses: assessment of the existence of a relationship between risk scores and coronary anatomy (Table 3)[1] and the predictive ability of the scores to discriminate who might have coronary lesion ≥ 50% (Chart 1)[1]. Thus, the word "correlation" cited in the manuscript[1] was used as the "relationship between risk scores and coronary anatomy".

The TIMI[4] and GRACE[5,6] risk scores have not been primarily developed to predict coronary lesion, but adverse clinical events. Thus, they are not supposed to have a strong discriminatory power to assess coronary lesion ≥ 50% or any other variable different from the specific clinical events of the original model. Nevertheless, they showed an ability that cannot be overlooked[7] to discriminate who will or will not have coronary lesion ≥ 50% as follows: TIMI risk score, area under the ROC curve = 0.704; hospital GRACE score, area under the ROC curve = 0.623; 6-month GRACE score, area under the ROC curve = 0.562.

It is worth noting that, for the TIMI risk score, the area under the ROC curve in the study by Santos et als.[1] was greater than that for the specific events of the original model of development (area under the ROC curve = 0.65)[4]. Similarly, the TIMI risk score, despite its limited predictive ability[7] for adverse clinical events, due to its clinical relevance and practicality, is one of the most used models worldwide, recommended by national and international guidelines.

Sincerely,

Elizabete Silva dos Santos

Luciano de Figueiredo Aguiar Filho

Luiz Minuzzo

Roberta de Souza

Ari Timerman