Letter to the editor regarding "To what extent are patients with migraine able to predict attacks?"

Dear editor

With interest, we read the article by Gago-Veiga et al published in Journal of Pain Research in October 2018.1 Migraine attacks, encompassing a wide range of symptoms, greatly undermine the quality of life for patients. Premonitory symptoms usually precede and alert the patients of the attack. The objectives of this prospective study1 were to illuminate if any good predictor or specific combination of premonitory symptoms exists for prediction of migraine attacks. A total of 34 patients recording 229 attacks were analyzed: 67.6% were able to predict at least one attack, while only 35.3% were able to predict >50% of attacks. The positive predictive value was 85.1%. The authors concluded some specific symptoms were predictive, even though only a few were good predictors (predicting >50% of attacks).

However, certain issues regarding the methodology must be addressed. First, the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio are the most suitable estimates to assess the validity of a test compared to the gold standard. Nevertheless, also reporting the diagnostic added value using receiver-operating characteristic curves should be considered, since all these validity estimates can be acceptable, whereas the diagnostic added value might be clinically negligible.2 Second, correlation, even with statistical significance (P<0.01), cannot guarantee prediction of a certain outcome. Furthermore, for prediction study, we usually need data from two distinct cohorts, or at least from one single cohort but divided into two, first to build a prediction model and then to validate it. Misleading results are generally the major outcome of research without validation of prediction models.2,3 Finally, in prediction study, we should evaluate the interactions between important variables.3 Final results could be affected dramatically if qualitative interactions are present, which means without evaluating interaction terms, prediction studies will mainly convey misleading messages.

We value Fang et al’s comments, and thank them for the insights provided on our recent work. We understand some of the comments, but we would like to let you know why we decided to present the results in the most suitable way from the perspective of the analysis performed using machine-learning techniques.

With regard to the issue of, eg, sensitivity and specificity values, we also understand that these are important metrics, and most of Fang et al’s requirements were already satisfied in the text. Let us notice that:

sensitivity = TPR = recall

PPV = precision

ROC = ROC

Additionally, all these values, and much more, can be calculated from the confusion matrix. That was the reason we provided this matrix in the text.

Regarding the comment about prediction and the need for two distinct cohorts, let us clarify that our work was not on prediction from the computational point of view (the “time” variable is not present in the model), but on classification (AdaBoost is a classification algorithm). Also, our work uses cross-validation for the validation of the classification model as a computationally valid technique for the estimation of performance of a predictive model. In any case, we have repeated our experiments with two cohorts (70% for building the classification model, 30% for validation), and obtained better results than those presented with our safer cross-validation:

Finally, regarding the comment about the interaction between “important variables”, let us say that we already analyzed the effect of sociodemographic variables, and this was found negligible.

Once again, we thank Fang et al for taking the time to read our work and provide comments.

	μ	σ
Accuracy	86.3%	1.2%
Sensitivity = recall = TPR	80.3%	1.7%
Specificity = TNR	89.7%	0.9%
Precision = PPV	80.4%	2.0%
NPV	89.9%	1.0%
LR+	9.6	1.0
LR−	0.2	0
DOR corrected	8.5	1.4