Computerized migraine diagnostic tools: a systematic review.

BACKGROUND: Computerized migraine diagnostic tools have been developed and validated since 1960. We conducted a systematic review to summarize and critically appraise the quality of all published studies involving computerized migraine diagnostic tools.
METHODS: We performed a systematic literature search using PubMed, Web of Science, Scopus, snowballing, and citation searching. Cutoff date for search was 1 June 2021. Published articles in English that evaluated a computerized/automated migraine diagnostic tool were included. The following summarized each study: publication year, digital tool name, development basis, sample size, sensitivity, specificity, reference diagnosis, strength, and limitations. The Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool was applied to evaluate the quality of included studies in terms of risk of bias and concern of applicability.
RESULTS: A total of 41 studies (median sample size: 288 participants, median age = 43 years; 77% women) were included. Most (60%) tools were developed based on International Classification of Headache Disorders criteria, half were self-administered, and 82% were evaluated using face-to-face interviews as reference diagnosis. Some of the automated algorithms and machine learning programs involved case-based reasoning, deep learning, classifier ensemble, ant-colony, artificial immune, random forest, white and black box combinations, and hybrid fuzzy expert systems. The median diagnostic accuracy was concordance = 89% [interquartile range (IQR) = 76-93%; range = 45-100%], sensitivity = 87% (IQR = 80-95%; range = 14-100%), and specificity = 90% (IQR = 77-96%; range = 65-100%). Lack of random patient sampling was observed in 95% of studies. Case-control designs were avoided in all studies. Most (76%) reference tests exhibited low risk of bias and low concern of applicability. Patient flow and timing showed low risk of bias in 83%.
CONCLUSION: Different computerized and automated migraine diagnostic tools are available with varying accuracies. Random patient sampling, head-to-head comparison among tools, and generalizability to other headache diagnoses may improve their utility.

Introduction

The rise of disruptive digital health technologies is revolutionizing healthcare.[1,2] The increasing prevalence of chronic health burden such as headache disorders and the growing digitally native population are drivers for innovative modes of disease management.[3,4] The application of digital health tools (e.g. computerized expert systems/software, mobile health, wearables, health information technology, Internet of Medical Things, telehealth and telemedicine, robotic surgery) is known to reduce diagnostic delays and inaccuracies, improve inefficiencies, improve access, and facilitate remote management and continuity of care – all of which reduce healthcare-related costs.[1,2,5] In addition, digital health platforms empower patients’ self-efficacy by increasing patient’s shared decision-making within the clinician–patient relationship as well as by facilitating behavioral activation and internal locus of control.[3,4] Patient-centered digital care can be enhanced by promoting patient engagement and redefining patient–doctor relationship.[3,4] In the United States, healthcare delivery is suboptimal and inefficient as compared with economically similar countries that spend half annual per capita cost.[6,7] Digital health tools can promote value-based care to improve the health of populations (scalability), improve individual patient outcomes, and lower per capita costs.[1,8,9] The ongoing COVID-19 pandemic demonstrates the utility of digital health platforms such as telehealth.

That the diagnosis of headache disorders is primarily based on patient history provides a valuable opportunity for developing and evaluating digital tools to diagnose and manage headache types. Headache disorders remain largely underdiagnosed and misdiagnosed, and thus are undertreated and mistreated.[11,12] Delays in accurate diagnosis lasting more than a decade lead to a higher risk for chronicification and increased complexity in headache treatment.[13-16] Digital tools have the potential to close the gap between accurate diagnosis and increasing headache burden. Daily monitoring of headache incidence and related attributes using e-diary reduces diagnostic errors due to recall bias.[17,18] Automated prediction of headache attacks and medication overuse can be made by using data-driven machine learning approaches.[19,20] Wearables can identify real-time digital biomarkers for early detection of headache attacks (e.g. sleep time, galvanic skin response,[21-24] heart rate variability[21,22]) and can also be used to improve adherence and deliver treatment such as smartphone-based biofeedback,[22-24] behavioral therapy, and smartphone-controlled electronic pulses.

The strong relationship between psychiatric comorbidities, traumatic brain injury, and headache disorders such as migraine is an important aspect of the growing headache burden.[27,28] Comprehensive evaluation of psychiatric disorders (e.g. depression, anxiety, post-traumatic stress disorder) is a critical component of proper headache management which can be optimized using digital health tools.[27-29] Digital health tools can solve unmet needs in these complex migraine comorbidities by providing early and accurate real-time identification of clinical symptoms or signs (e.g. quantitative electroencephalography/ EEG wearable headsets, transcranial Doppler, photoplethysmography for temporal artery pulse wave), as well as by aiding in monitoring symptom management. Currently, there is active research involving development and validation of digital tools for traumatic brain injury and headaches by utilizing physiological biomarkers such as nystagmus, vestibular-ocular function, saccades, intracranial pressure, optic nerve sheath diameter, pupillary reflex, and reaction time.

The diagnostic classification of headache disorders (International Classification of Headache Disorders, 3rd edition or ICHD-3 ) is well suited for building artificial intelligence automated diagnoses. Patient-centered care and patient–physician shared decision-making are important features that are characteristic of digital headache tools.[31,32] Headache clinical trials and population research can benefit by utilizing digital health tools for screening, improved oversight, and real-time analysis of patient outcomes.[33,34] Personalized headache care, behavioral, and lifestyle-based management approaches can be enhanced using digital health applications.[35,36] The number of headache-trained healthcare providers is relatively low and poorly accessible to most headache sufferers. With nearly a billion people estimated to suffer from migraine worldwide, digital health technologies have the scalability potential to expand and improve access and outcomes for patients. Migraine stigmatization, prevalent in headache care, and migraine-specific disabilities present an opportunity for digital tools to serve patient populations not comfortable in seeking care through traditional face-to-face interviews or unable to access care due to disease-specific limitations.

Although multiple computerized headache diagnostic tools have been developed and evaluated since the 1960s, migraine ranks last among the eight most prevalent conditions globally in terms of available mobile health applications. Migraine causes the highest disability in the productive age group. Despite the development of many computerized migraine diagnostic and management digital tools, there is no systematic review that specifically summarized the accuracy, performance, and quality of published computerized migraine diagnostic digital tools. In this study, we conducted a systematic review to summarize and critically appraise the quality of published studies that developed and evaluated computerized migraine diagnostic tools. We hypothesized that there will be a significant variation in the accuracy performance of the published migraine diagnostic tools.

Methods

Search methodology

The methods of systematic review followed the standard guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) . Literature search was performed using PubMed Advanced Search Builder, Web of Science Advanced Search, and Scopus Advanced Search using text-word searching. For PubMed Advanced Search Builder, the following search terms were combined using the Boolean operator ‘AND’: ((computerized) AND (diagnosis)) AND (migraine); ((algorithm) AND migraine)) AND (diagnosis); ((automated) AND (migraine)) AND (diagnosis). For Web of Science Advanced Search, the Web of Science Core Collection database was employed using the following search terms, Boolean operators, and field tags: TS=(computerized* AND migraine diagnosis); TS=(computerized* AND migraine); TS= (automated* AND migraine); TS=(algorithm* AND migraine). The field tag TS stands for topics. The asterisk * represents a wildcard in Web of Science for any group of characters, including no character. For Scopus Advanced Search, the following search terms were used: ‘migraine AND computerized AND diagnosis’.

We employed PubMed, Web of Science, and Scopus because these three databases are the most comprehensive and most optimum resources for conducting systematic review of medical literature.[46,47] We did not include ScienceDirect as it covers publications by Elsevier only – which are all available in Web of Science/PubMed/Scopus. Embase was not employed as its coverage is similar to PubMed’s MEDLINE. PsycInfo is for specialized articles on behavioral studies; we attempted PsycInfo search but could not find relevant articles. Additional search was done by snowballing and citation searching. Cutoff date for search was 1 June 2021.

The inclusion criterion was published articles and translated abstracts in English that evaluated a computerized/automated diagnostic tool for migraine. Some of the included studies involved evaluation of computerized digital tools for other headache types in addition to migraine – in such studies, only the migraine data were extracted for our systematic review. The study protocol was registered on PROSPERO (International Prospective Register of Systematic Reviews) [PROSPERO CRD42021033196]. The search results were uploaded to Covidence (http://covidence.org) for deduplication and screening. Two authors (YWW, RPC) screened references and reviewed full-text articles independently.

Data extraction

After reviewing the identified articles and deciding on inclusion, data were extracted and entered into a summary table by first author and publication year, digital tool name, development basis, sample size, concordance rate, sensitivity, specificity, reference diagnosis, strength, and limitations.

Summary analysis

Descriptive statistics [median, interquartile range (IQR), range, percentages] were used to summarize the following parameters: age, sex ratio, proportion of tools developed based on ICHD criteria, proportion of self-administered and physician/provider-administered, study settings, that is, percentage of tools validated in headache clinical centers versus nonclinical settings, percentage of reference diagnosis conducted using face-to-face interviews, and median diagnostic accuracy (concordance, sensitivity, specificity). Studies were categorized by decades to assess whether there was progressive increment in the number of digital tools developed. Median group differences (where group size imbalance is not larger than 1:3) were compared using the Mann–Whitney test. Spearman’s nonparametric correlation between year of publication and accuracy performance measurands was examined to assess whether there was improvement in diagnostic accuracy over the successive years. A value of p < 0.05 was considered to be statistically significant.

Quality Assessment of Diagnostic Accuracy Studies

The Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 tool was applied to critically appraise the quality of the included studies in terms of risk of bias and concern of applicability over four key domains: patient selection, index test, reference standard, flow, and timing. Risk levels are classified as high, low, and unclear. See Supplemental File 1 for details on QUADAS assessment.

Results

Search results

The number of published studies identified by PubMed Advanced Search Builder on ((computerized) AND (diagnosis)) AND (migraine); ((algorithm) AND (migraine)) AND (diagnosis); ((automated) AND (migraine)) AND (diagnosis) was 155, 218, and 97, respectively. The number of published studies identified through Web of Science Core Collection Advanced Search database using the following search terms TS=(computerized* AND migraine diagnosis); TS=(computerized* AND migraine); TS=(automated* AND migraine); TS=(algorithm* AND migraine) was 58, 199, 138, and 294, respectively. The number of published studies identified through Scopus Advanced Search database using the terms ‘migraine AND computerized AND diagnosis’ was 1813. There were 6 studies identified through snowballing and 15 studies through citation searching on ResearchGate and Google Scholar (Figure 1 PRISMA flowchart).

Figure 1.

PRISMA flowchart.

PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Included studies and summary results

After screening records, removing duplicates, and reviewing full-length articles, 41 studies (median age = 43 years; 77% women) were included in the systematic review (Figure 1 PRISMA flowchart; Supplemental File 2 and 3) as per the inclusion criteria. The median sample size of the studies was 288. Sixty percent of the digital tools were developed based on ICHD criteria. The remaining tools were developed based on local or national criteria (8), Wolff 1962 criteria (2), 1962 Ad Hoc criteria (1), case-based reasoning following ICHD-3-beta (1), artificial immune algorithm following ICHD-2 (1), deep learning framework (1), and ID-CM or Identify Chronic Migraine (1). Half (50%) of the tools were self-administered. Eighty-eight percent of tools were evaluated in headache clinical centers. The remaining were validated in nonclinical or community care settings, that is, employees (1), primary care (2), and students (1).

Face-to-face interviews were used in 82% of the reference diagnosis, phone interviews in two studies, and questionnaire-based diagnosis in four studies. The majority (70%) of the digital tools were available in English. Five were available in Italian, four in Turkish, three in Chinese, one in Polish, one in Persian, two in Dutch, one in Serbian, one in Spanish, and one in Japanese. Some of the automated algorithms and machine learning programs involved case-based reasoning, deep learning, classifier ensemble, ant-colony, artificial immune, random forest, white and black box combinations, and hybrid fuzzy expert systems. Ten (25%) studies compared multiple machine learning programs to identify the most accurate tool in diagnosing migraine. Two of the digital tools are available as open-source software, and two other tools contained a daily electronic diary (e-diary). Management options were embedded in one of the digital tools.

There was a 4.5-time increase in the number of digital tools after 2005 compared with before (Figure 2). There were six new digital tools since the COVID-19 pandemic started in the beginning of 2020. The median diagnostic accuracy was concordance = 89% (IQR = 76–93%; range = 45–100%), sensitivity = 87% (IQR = 80–95%; range = 14–100%), and specificity = 90% (IQR = 77–96%; range = 65–100%). Five (12%) of the digital tools achieved diagnostic accuracy of 100% in one of the accuracy performance measurands, that is, sensitivity, specificity, and concordance. Lack of random patient sampling was observed; only 2 (5%) studies applied random sampling, while the majority 23 (56%) of the studies used convenience or consecutive sampling. Patient sampling method was unclear in 16 (39%) studies. There was no description of age or sex ratio of participants in 25 (61%) studies. There was no statistically significant correlation between year of publication and accuracy performance of the digital tools.

Figure 2.

A decade-by-decade timeline of the number of computerized migraine diagnostic tools developed and validated since 1965. There was a 4.5-time increment in the number of computerized migraine diagnostic tools since 2005 compared with before 2005. There were six new tools developed and evaluated since the beginning of 2020.

Twelve (30%) of the digital tools were developed specifically for migraine diagnosis (i.e. binary classification into ‘migraine’ or ‘no migraine’), while the remainder 29 (70%) were developed to diagnose migraine as well as other primary headache disorders [i.e. multiclass classification into ‘migraine’ or ‘tension-type headache (TTH)’ or ‘trigeminal autonomic cephalalgias (TAC)’ or ‘other headache’]. Although the accuracy performance of the migraine-only diagnostic tools (binary) was observably lower than the tools that diagnose migraine plus other headache disorders (multiclass), the difference was not statistically significant (Figure 3). We are currently conducting a separate systematic review appraising accuracy performance of digital diagnostic tools for other headache disorders (e.g. TTH, TACs) in addition to comparing accuracy performance of migraine diagnosis versus diagnosis of other headache disorders (e.g. TTH, TACs).

Figure 3.

Comparison of diagnostic accuracy performance between binary and multiclass classification. The digital tools that were specifically designed for diagnosis of migraine only (‘migraine’ versus ‘no migraine’, i.e. binary classification) performed observably lower than the digital tools that were developed for multiclass classification of migraine and other headache disorders. The median values of sensitivity, specificity, and accuracy for the migraine-only classification digital tools (black bars) were 75%, 85%, and 61%, while for tools that diagnosed migraine and other headache disorders were 89%, 88%, and 88%. The differences did not reach statistical significance on the Mann–Whitney test.

D/O, disorders.

Brief description of performance of selected migraine diagnostic digital tools – a timeline

Since the 1960s, there have been efforts to create computer-generated diagnostic tools in headache.[42,50] One of the early computer-based self-administered diagnostic tools developed in 1968 correctly identified all nine headache patients as vascular headache, as compared with diagnoses made by neurologists with interest in headache. A self-administered computer-based diagnostic tool developed in 1972 performed with sensitivity and specificity of 74% and 75% of diagnosing migraine, respectively, while correctly differentiating 36 of 50 patients to common migraine, classical migraine, cluster migraine, muscle contraction, and other headache types, as compared with diagnoses made by neurologists with interest in headache. However, in 1974, another computerized diagnostic tool performed considerably lower with sensitivity and specificity of 14% and 98%, respectively; the reason for its lower performance was considered to be due to lesser power of algorithms involving 19 headache types in its diagnostic capacity. In 1980, a voice-based, self-administered, computer diagnostic study involving 40 headache patients showed that 38 (95%) patients found the computer interview easy to use, while 22 (55%) patients thought that the computer took a complete headache history; 55% of the headache clinicians found the computer interview to be useful. While these early studies on computer-based tools laid the foundation for self-administered headache diagnostic tools, the lack of standard headache classification and absence of widely available computers made it challenging to adopt their use for daily practice. Headache classification has evolved greatly through the decades; phrases such as vascular headache, common migraine, cluster migraine, and muscle contraction are currently outdated.

In the 2000s, the Italian Neurological Association for Headache Research developed and validated the Archivio Informatico con funzioni di Diagnosi Assistita per Cefalee – Informatic Register with Assisted Diagnosis for Headaches (AIDA CEFALEE) – computerized ICHD-2-based expert software that is filled by physicians and automatically generates a textual patient history, diagnosis, and treatment recommendations.[53,54] Its diagnostic performance was evaluated in 200 headache patients from five headache centers against an ICHD-2-based structured clinical interview conducted by medical staff.[53,54] The AIDA CEFALEE[34,35] and the structured clinical interview showed 100% sensitivity and 76% specificity in diagnosing migraine.[53,54] In 2008, US researchers developed the self-administered Computerized Headache Assessment Tool (CHAT) based on the 1988 ICHD-1 and evaluated its diagnostic accuracy in 135 headache patients using a reference of phone interview diagnoses by headache specialist nurse. The reference diagnosis was based on the 1994 UCSD (University of California San Diego) Migraine Questionnaire and Silberstein–Lipton criteria. The CHAT had an overall accuracy of 92.8% (91/98), that is, 91.6% (77/84) in migraine [100% (35/35) in episodic migraine, 85.7% (42/49) in transformed migraine] and 83.3% (5/6) in probable migraine. The reference used to validate the CHAT, that is, the Silberstein–Lipton criteria are currently outdated.

In 2014, Chinese researchers developed and validated the first ICHD-3-based, physician-administered, computerized clinical decision support systems (CDSS) in 543 patients against a reference diagnosis by 2 headache specialists. The CDSS was designed to assist general practitioners based on the ICHD-3. Its diagnostic sensitivity and specificity were 94% and 95%, respectively. That the AIDA and CDSS were only available in Italian and Chinese, respectively, and had to be physician administered reduce their utility for direct patient use and self-management in anglophone populations. In 2015, a four-item migraine diagnostic algorithm, that is, the Migraine-4, was found to have a sensitivity and specificity of 94% and 92%, respectively, compared with a reference diagnosis made by structured self-administered questions. The performance assessment of the Migraine-4 was conducted in a nonclinical population and its accuracy was not compared with diagnosis made by live interviews.

In the last 5 years, Turkish, Serbian, Iranian,[62,63] German, US, and Dutch researchers have evaluated different algorithms, machine learning expert systems in diagnosing migraine and other headache types. These algorithms involved ant-colony, artificial immune, classifier ensembles, white and black box combinations, and hybrid fuzzy expert systems. Their accuracy ranged from 65% to 100%. Some tools are available as open source, include electronic headache diary, and provide treatment options. Limitations of these newly developed tools include the fact that they were based on non-ICHD criteria, retrospective analysis, and that they were not tested against live interviews. Some of these tools contained contamination where the reference diagnostician was exposed to diary data before interviews. The ongoing COVID-19 pandemic may have contributed to the rise in digital tools and telemedicine research in headache including mobile phone applications. In 2020, the M-sense app was developed based on ICHD-3 criteria and 3-monthly, patient-administered electronic headache diary. The diagnostic accuracy of M-sense was validated in 102 patients against a headache specialist diagnosis using the same data collected by M-sense. The M-sense showed 96.9% (62/64) sensitivity and 68.4% (26/38) specificity for migraine.

QUADAS results

Consecutive or convenience sampling was used in 56% of the studies (Table 1, Supplemental File 4). The sampling method was unclear in 39% of the studies. Only 5% of the studies applied random sampling. Most studies (61%) mentioned consecutive or random sampling resulting in low risk of bias in patient selection (domain 1) according to QUADAS-2. Case–control design was avoided in all studies. The index tests (domain 2) showed low risk of bias (interpreted without knowledge of the reference test result) and low concern of applicability (the index test, its conduct, or interpretation did not differ from the review questions) in all the studies. Similarly, the majority (76%) of reference standard tests (domain 3) exhibited low risk of bias (the reference standard, its conduct, or its interpretation did not introduce bias) and low concern of applicability (the target condition as defined by the reference standard, i.e. migraine matches the review question). The majority, 83% of the studies, featured low risk of bias in the flow and timing (domain 4) of participating patients, that is, most patients received both the index and reference tests within appropriate interval. Percentage results are displayed in Supplemental File 4. Sample size estimation was lacking in all included studies.

Table 1.

QUADAS assessment of included studies.

Study	Risk of bias				Applicability concerns
	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
Freemon 42	Low	Low	Unclear	Low	Low	Low	Unclear
Stead 50	Unclear	Low	Low	Low	Low	Low	Low
Toole et al. 51	Unclear	Low	Unclear	Low	Low	Low	Unclear
Penzien et al. 67	Unclear	Low	Low	Low	Low	Low	Low
Andrew et al. 68	Unclear	Low	Low	Low	Low	Low	Low
Gallai et al. 69	Low	Low	Low	Low	Low	Low	Low
Pryse-Phillips et al. 70	Unclear	Low	Low	Low	Low	Low	Low
Kopec et al. 71	Unclear	Low	High	High	Low	Low	High
Sarchielli et al. 72	Low	Low	Low	Low	Low	Low	Low
Sarchielli et al. 73	Low	Low	Low	Low	Low	Low	Low
De Simone et al. 54	Low	Low	Low	Low	Low	Low	Low
Maizels and Wolfe 55	Low	Low	Low	High	Low	Low	Low
Mendes et al. 74	Low	Low	Low	High	Low	Low	Low
Porta-Etessam et al. 75	Low	Low	Low	Low	Low	Low	Low
van Oosterhout et al. 76	Low	Low	Low	Low	Low	Low	Low
Tezel and Köse 77	Unclear	Low	Unclear	High	Low	Low	Unclear
Yurtay et al. 78	Low	Low	Low	High	Low	Low	Low
Krawczyk et al. 79	Unclear	Low	Low	Low	Low	Low	Low
Eslami et al. 80	Low	Low	Low	Low	Low	Low	Low
Dong et al. 58	Low	Low	Low	Low	Low	Low	Low
Yanping and Huilong 81	Unclear	Low	High	Low	Low	Low	High
Yin et al. 82	Low	Low	Low	Low	Low	Low	Low
Jackowski et al. 83	Unclear	Low	High	Low	Low	Low	High
Çelik et al. 60	Unclear	Low	Low	Low	Low	Low	Low
Walters and Smitherman 59	Low	Low	Low	Low	Low	Low	Low
Lipton et al. 65	Low	Low	Low	Low	Low	Low	Low
Çelik et al. 84	Low	Low	Low	Low	Low	Low	Low
Çelik and Yurtay 85	Low	Low	Low	Low	Low	Low	Low
Keight et al. 86	Unclear	Low	High	Low	Low	Low	High
Vandewiele et al. 87	Unclear	Low	Low	Low	Low	Low	Low
Kaiser et al. 88	Low	Low	Low	High	Low	Low	Low
Khayamnia et al. 62	Low	Low	Unclear	Low	Low	Low	Unclear
Sacco et al. 89	Low	Low	Low	Low	Low	Low	Low
Qawasmeh et al. 63	Unclear	Low	Low	Low	Low	Low	Low
Roesch et al. 64	Low	Low	Low	Low	Low	Low	Low
Kwon et al. 90	Low	Low	Unclear	Low	Low	Low	Unclear
Katsuki et al. 91	Low	Low	Low	Low	Low	Low	Low
van Casteren et al. 66	Low	Low	Low	High	Low	Low	Low
Simić et al. 61	Unclear	Low	Unclear	Low	Low	Low	Unclear
Simić et al. 92	Low	Low	Low	Low	Low	Low	Low
Groccia et al. 93	Unclear	Low	Low	Low	Low	Low	Low

QUADAS, Quality Assessment of Diagnostic Accuracy Studies.

The risk of bias in the domain of patient selection was low in 56% (23) of the studies that used consecutive/convenience sampling and 5% (2) of the studies that applied random sampling. The sampling method was unclear in 39% (16) of the studies. The index test results were all interpreted without knowledge of the results of the reference standard. The conduct or interpretation of the index test did not introduce bias – hence resulting in ‘low risk’ and ‘low concern’ for ‘Risk of Bias’ and ‘Applicability’. Likewise, most studies (76%) featured ‘low risk’ and ‘low concern’ in the conduct and interpretation of the reference tests. Similarly, most (83%) studies showed ‘low risk’ of bias in the flow and timing of patients, that is, all patients received both the index and reference tests at appropriate interval.

For each included study, Risk of bias and Applicability concerns are tabulated as “Low” (green shade), “Unclear” (blue shade), and “High” (orange shade).

Discussion

Our systematic review shows the progressive increment in the quality and quantity of computerized migraine diagnostic tools. The rise of digital health in headache medicine deserves special attention because of shortage and/or inaccessibility of headache specialists for a growing headache burden worldwide. As artificial intelligence and machine learning tools are advancing rapidly, we anticipate a continued research in and adoption of computerized headache diagnostic and management tools. This is evidenced by the 4.5-time increase in the number of digital tools developed and validated in the last decade. The high median values of 87–90% for concordance, sensitivity, and specificity reflect the high level of accuracy and precision of computerized migraine diagnostic tools. That the ICHD criteria were used by most of the tools to build the automation algorithm is a worthy feature in terms of applying standardized migraine diagnosis.

That half of the computerized diagnostic tools were self-administered was a useful indicator that the tools can be completed at patient’s convenience, saving time and empowering patient self-efficacy. The majority of the computerized tools were evaluated in a headache clinical center – a barrier for the generalizability of the results to community or primary care settings. Future studies may develop and conduct field testing of such tools designed for community level or nonclinical populations and for primary care settings.

Our findings show that the majority of the validation studies did not apply probability patient sampling (e.g. random patient selection). Nonrandom patient recruitment (e.g. consecutive or convenience sampling) leads to a type of selection bias known as spectrum bias. Spectrum bias reflects the varying accuracy of diagnostic tools in different patients. That case–control study design was avoided by all included studies and reflects the strong quality of the results.[49,95] By including extreme populations, that is, cases of chronic migraine versus controls of fully healthy participants, the performance of diagnostic accuracies can be more inflated in case–control designs than when including the average migraine patient versus typically healthy individuals. The majority of the included studies measured the migraine diagnostic accuracy by enrolling patients with migraine and control participants that included patients with other headache types (e.g. TTH) as well as healthy individuals. Future studies can improve presentation of their results by providing sensitivity (e.g. subgroup) analysis to address heterogeneous patient and control populations.

The application of face-to-face interviews by the majority of the reference diagnosis was a strong feature, as a direct comparison between machine automation and human can be allowed. Some diagnostician interviewers possess naturally engaging personality and utilize memory jogging techniques to reduce recall bias during interviews. Embedding e-diary in the digital tools can reduce patients’ recall bias, as was seen in two of the included studies.[64,66] One of the digital tools includes a management option – indicating the potential of the tools for headache care in terms of both diagnosis and management. Digital tools have been validated in several clinical trials as self-management therapies (e.g. neurofeedback, behavioral therapy) for migraine and other headache disorders.[22,23] Overall, the QUADAS quality assessments showed low risk of bias and low concern of applicability in all categories, which indicated the quality of the validation studies.

The expert systems utilized in the computerized migraine diagnostic tools involved purely data-driven approaches and hybrid expert diagnostic systems. Hybrid systems involved combination of different automation and algorithm approaches as well as combination of data- and knowledge-driven modalities. This indicates the versatility of the digital platform in incorporating multiple approaches. The marginal improvement we found in the accuracy performance of multiclass over binary classification digital tools may indicate that multiclass classification may capture multiple response variables of clinical features, thereby matching with refined headache phenotypes. However, this speculation involving potential differences in machine learning migraine diagnostic digital tools needs to be appropriately tested and validated.

Our systematic review was focused on migraine because migraine is the most commonly studied headache diagnosis. Some of the computerized diagnostic tools included (Table 1) are capable to simultaneously perform diagnosis of multiple headache disorders. It will be interesting to compare computerized migraine diagnostic accuracy with computerized diagnosis of other headache types.

Limitations

Because the studies included in this systematic review evaluated different digital tools, we were not able to conduct a weighted meta-analysis and derive a combined performance for diagnostic accuracy. Similarly, because of the interstudy heterogeneity of diagnostic automation software, it was not possible to undertake detailed sensitivity analysis such as publication bias, confounding or moderation, small-study bias, outliers, and cumulative meta-analysis among the different computerized migraine diagnostic tools. That patient selection method was missing from 39% of the included studies is another limitation. Studies that involved inappropriate exclusion criteria (e.g. overexclusion of difficult cases) may inflate the accuracy performance of a digital migraine diagnostic tool – only because confirmed or known migraine cases were enrolled, whereas ‘difficult to diagnose’ patients were excluded.[97,98] Vice versa, studies that involved overinclusion of ‘difficult to diagnose’ migraine cases may underestimate the accuracy performance of a digital migraine diagnostic tool – only because ‘easy to diagnose’ migraine patients were excluded.[97,98]

By virtue of incorporating the same ICHD question items in the index test (digital tools) and reference tests (interviews), validation of computerized migraine diagnostic tools is inherently faced with incorporation bias which can overestimate accuracy performance of the digital tools tested.[99-101] This circularity problem can only be solved by testing digital/computerized migraine diagnostic tools that rely on digital biomarkers (e.g. heart rate variability, photoplethysmography) instead of self-report symptoms and signs, that is, when the index and reference tests contain parameters that are completely independent to each other (e.g. physiological/objective versus patient-reported parameters).[99-101] Our review was limited to studies published in English, although we have included translated abstracts. As such, we may have missed computerized migraine diagnostic tools published in journals with other languages.

Conclusion

The emergence of digital health in headache medicine is evident. Computerized migraine diagnostic tools have been shown to perform with diagnostic accuracy as high as 100% compared with traditional interview-based diagnosis by headache experts. Increased testing in different clinical settings and patient populations as well as random patient sampling and a priori sample size estimations will enhance generalizability and robustness of computerized migraine diagnostic tools for implementation toward daily use. By virtue of being a chronic condition whose episodic attacks can be triggered by lifestyle-related changes (e.g. sleep disruption, skipped mealtime) and featuring several lifestyle-related multimorbidities (e.g. depression, anxiety),[14,17,38] migraine care can benefit from digital health applications helping sufferers self-monitor and better manage their headache and related problems by reinforcing healthy lifestyle behavior.

Compared with traditional headache care model that is not capable of reaching the growing burden of a billion of migraine sufferers worldwide, computerized migraine diagnostic tools have the potential to provide efficient, patient-centered, and improved headache care by delivering early diagnosis and management, enhancing diagnostic accuracy, saving time, boosting accessibility, enabling remote care with reduced costs, and decreasing travel to hospital/clinic care setting thereby reducing the exposure to communicable diseases in healthcare settings. People with migraine can increase their self-efficacy and get a greater control of their health by using digital tools. Personalized migraine care can be facilitated by digital tools. Headache care providers can obtain a comprehensive objective overview of the patient including real-time monitoring via sensors and diary recording. Virtual or augmented reality treatment modalities can be tested using migraine digital health tools, for example, to reduce head pain perception or to provide biofeedback sessions for associated psychological comorbidities. Scalability as well as data security of digital migraine tools can be increased by using blockchain technology. Ingestible sensors can be tested to measure gastroparesis, gut metabolomics, or gut–brain axis involvement in migraine.

Click here for additional data file.

Supplemental material, sj-docx-1-taj-10.1177_20406223211065235 for Computerized migraine diagnostic tools: a systematic review by Yohannes W. Woldeamanuel and Robert P. Cowan in Therapeutic Advances in Chronic Disease

Click here for additional data file.

Supplemental material, sj-docx-2-taj-10.1177_20406223211065235 for Computerized migraine diagnostic tools: a systematic review by Yohannes W. Woldeamanuel and Robert P. Cowan in Therapeutic Advances in Chronic Disease

Click here for additional data file.

Supplemental material, sj-docx-3-taj-10.1177_20406223211065235 for Computerized migraine diagnostic tools: a systematic review by Yohannes W. Woldeamanuel and Robert P. Cowan in Therapeutic Advances in Chronic Disease

Click here for additional data file.

Supplemental material, sj-docx-4-taj-10.1177_20406223211065235 for Computerized migraine diagnostic tools: a systematic review by Yohannes W. Woldeamanuel and Robert P. Cowan in Therapeutic Advances in Chronic Disease