Timothy T Houle1, Hao Deng1, Charles H Tegeler2, Dana P Turner1. 1. Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA. 2. Department of Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA.
Abstract
OBJECTIVE: To illustrate the benefits of deploying individual headache forecasting models using continuous updating with Bayesian methods. BACKGROUND: The ability to reliably forecast headache attacks within an individual over time would enhance the study of attacks and allow preemptive treatment. However, deploying a suitable forecasting model in a clinical setting will likely involve several unique challenges related to heterogeneity in the predictor weights, limited or sparse data, and the need for a quick "warm-up." The use of Bayesian methods offers solutions to each of these specific challenges. METHODS: This was a post hoc analysis of a cohort study of individuals with episodic migraine attacks. Individuals completed daily diaries that allowed the estimation of several forecasting models, each using different types of ancillary information incorporated into formal prior probability distributions. An in silico analysis was conducted that mimicked the deployment of these models in a clinical-like setting where the parameters of the models were continuously updated and evaluated each day using root mean square error (RMSE). RESULTS: Individuals (N = 95) were followed for 50 days and contributed 3359 days of nonmissing diary data. During the observation period, there were 1293/3359 (38.5%) days with a headache attack. Self-reported baseline headache frequency was associated with the corresponding predicted probability of future attacks, r = 0.15-0.39. At Day 25, the correlation between baseline information and predicted attack likelihood was r = 0.29 (95% CI: 0.09-0.47). Additionally, the use of prior probability distributions for model parameters improved the model fit, especially early in the deployment of the models (e.g., Day 5 RMSE 0.45 vs. 0.43). Models using informative prior probability distributions outperformed the models estimated without this information during the first 42 days of observation, although performance became more similar as more data were collected. CONCLUSIONS: This analysis demonstrates the value of Bayesian methods in using additional available information to improve forecasting model performance, especially early in the deployment of a forecasting model. To obtain the full value of such models or to apply any model in clinical settings, a model with adequate discrimination and calibration will be needed.
OBJECTIVE: To illustrate the benefits of deploying individual headache forecasting models using continuous updating with Bayesian methods. BACKGROUND: The ability to reliably forecast headache attacks within an individual over time would enhance the study of attacks and allow preemptive treatment. However, deploying a suitable forecasting model in a clinical setting will likely involve several unique challenges related to heterogeneity in the predictor weights, limited or sparse data, and the need for a quick "warm-up." The use of Bayesian methods offers solutions to each of these specific challenges. METHODS: This was a post hoc analysis of a cohort study of individuals with episodic migraine attacks. Individuals completed daily diaries that allowed the estimation of several forecasting models, each using different types of ancillary information incorporated into formal prior probability distributions. An in silico analysis was conducted that mimicked the deployment of these models in a clinical-like setting where the parameters of the models were continuously updated and evaluated each day using root mean square error (RMSE). RESULTS: Individuals (N = 95) were followed for 50 days and contributed 3359 days of nonmissing diary data. During the observation period, there were 1293/3359 (38.5%) days with a headache attack. Self-reported baseline headache frequency was associated with the corresponding predicted probability of future attacks, r = 0.15-0.39. At Day 25, the correlation between baseline information and predicted attack likelihood was r = 0.29 (95% CI: 0.09-0.47). Additionally, the use of prior probability distributions for model parameters improved the model fit, especially early in the deployment of the models (e.g., Day 5 RMSE 0.45 vs. 0.43). Models using informative prior probability distributions outperformed the models estimated without this information during the first 42 days of observation, although performance became more similar as more data were collected. CONCLUSIONS: This analysis demonstrates the value of Bayesian methods in using additional available information to improve forecasting model performance, especially early in the deployment of a forecasting model. To obtain the full value of such models or to apply any model in clinical settings, a model with adequate discrimination and calibration will be needed.
Authors: Lillian Sung; Jill Hayden; Mark L Greenberg; Gideon Koren; Brian M Feldman; George A Tomlinson Journal: J Clin Epidemiol Date: 2005-03 Impact factor: 6.437
Authors: Timothy T Houle; Dana P Turner; Adrienne N Golding; John A H Porter; Vincent T Martin; Donald B Penzien; Charles H Tegeler Journal: Headache Date: 2017-07 Impact factor: 5.887