Non-invasive neuromodulation for migraine and cluster headache: a systematic review of clinical trials.

Non-invasive neuromodulation therapies for migraine and cluster headache are a practical and safe alternative to pharmacologics. Comparisons of these therapies are difficult because of the heterogeneity in study designs. In this systematic review of clinical trials, the scientific rigour and clinical relevance of the available data were assessed to inform clinical decisions about non-invasive neuromodulation. PubMed, Cochrane Library and ClinicalTrials.gov databases and the WHO's International Clinical Trials Registry Platform were searched for relevant clinical studies of non-invasive neuromodulation devices for migraine and cluster headache (1 January 1990 to 31 January 2018), and 71 were identified. This analysis compared study designs using recommendations of the International Headache Society for pharmacological clinical trials, the only available guidelines for migraine and cluster headache. Non-invasive vagus nerve stimulation (nVNS), single-transcranial magnetic stimulation and external trigeminal nerve stimulation (all with regulatory clearance) were well studied compared with the other devices, for which studies frequently lacked proper blinding, sham controls and sufficient population sizes. nVNS studies demonstrated the most consistent adherence to available guidelines. Studies of all neuromodulation devices should strive to achieve the same high level of scientific rigour to allow for proper comparison across devices. Device-specific guidelines for migraine and cluster headache will be soon available, but adherence to current guidelines for pharmacological trials will remain a key consideration for investigators and clinicians. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Non-invasive neuromodulation alters neural activity through the stimulation of nerves or neural tissue. This technology is emerging as a practical and safe alternative to conventional pharmacological interventions for the treatment of migraine and cluster headache.1 The demonstrated efficacy and safety of several non-invasive neuromodulation therapies, including non-invasive vagus nerve stimulation (nVNS), external trigeminal nerve stimulation (e-TNS) and single-pulse transcranial magnetic stimulation (sTMS), has generated interest among clinicians and patients.2 3 The clearance of these therapies by CE mark in the European Union and the US Food and Drug Administration (FDA) provides an alternative for the treatment of primary headache.4–6 These devices could present a unique flexible treatment option because they can be used as an early therapy, either independently or concomitantly with pharmacological treatment. Non-invasive neuromodulation also offers a novel alternative for those in sensitive patient populations (eg, pregnant women or adolescents), or for those affected by poor tolerability or lack of efficacy with pharmacologics.

Comparisons among the expanding number of non-invasive neuromodulation device trials are challenging because of the heterogeneity of clinical study designs and range of predefined efficacy outcomes. These inconsistencies among the non-invasive neuromodulation trials may be due in part to the absence of guidelines for conducting trials in primary headache. While such guidelines are in development, clinicians should currently base their treatment decisions on the best available evidence from rigorous clinical trials. Until non-invasive neuromodulation-specific study recommendations become available, neuromodulation trials designed and conducted in accordance with existing guidelines of the International Headache Society (IHS) for controlled studies of pharmacological therapies continue to provide the greatest scientific rigour and consistency.7 Adherence to future guidelines currently under development will allow for greater uniformity among studies to help clinicians make informed decisions on appropriate treatment options for their patients.

Objective

The objective of this article was to systematically review the clinical study designs of non-invasive neuromodulation trials for the treatment of migraine and cluster headache in order to assess the scientific rigour and clinical relevance of these devices and their associated clinical data. Clinicians and investigators can use this assessment to make informed decisions on treatment options and design more rigorous studies for patients with migraine and cluster headache.

Methods/literature search strategy

Four databases were systematically searched for clinical trials in the non-invasive neuromodulation of migraine and cluster headaches: PubMed.gov, the Cochrane Library, ClinicalTrials.gov and the WHO’s International Clinical Trials Registry Platform (WHO-ICTRP). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology was followed in this review.8

Identification of research studies

Medline through PubMed.gov was systematically searched for all clinical trials assessing neuromodulation treatments in migraine and cluster headache that were published between 1 January 1990 and 31 January 2018. Search terms were ‘neuromodulation OR neurostimulation OR stimulation’ and ‘migraine OR headache’; publication type ‘clinical trial NOT review’; and language ‘English’. The Cochrane Library was searched for ‘neuromodulation OR neurostimulation OR stimulation’ in Title, Abstract, Keywords; ‘migraine OR headache’ in Title, Abstract, Keywords; ‘NOT review’ in Publication Type; and ‘1990’ to ‘2018’ in publication year. A search limit was placed on ‘trials’ only.

ClinicalTrials.gov was systematically searched for all clinical trials (ie, all types of recruitment status) assessing neuromodulation OR neurostimulation OR stimulation treatments in migraine OR headache (ie, Condition or disease) between 1 January 1990 and 31 January 2018. The WHO-ICTRP was also searched for all clinical trials (ie, all types of recruitment status) assessing neuromodulation OR neurostimulation OR stimulation (ie,Intervention) treatments in migraine OR headache (ie, Condition) between 1 January 1990 and 31 January 2018. Since the neurologic effect can be excitatory or inhibitory, the preferred term is neuromodulation rather than neurostimulation, which is used in this review. Clinical trial registration is currently required by the US FDA for approval and clearance of devices and by the International Committee of Medical Journal Editors as a condition of consideration for publication.9–11

The broad search term ‘headache’ was used instead of ‘cluster headache’ to provide comprehensive search results and minimise exclusion of relevant studies. ‘Non-invasive’ was not specified in the search because studies often do not explicitly categorise treatments in this manner and instead use implied terminology (eg, transcutaneous, transcranial, external).

Screening

Trials from WHO-ICTRP were cross-referenced with those from ClinicalTrials.gov, and duplicates were removed to create a master list of unique clinical trials. Research articles from PubMed and the Cochrane Library were matched to a registered clinical trial12; matched data were combined as a single record for analysis.

Inclusion and exclusion criteria

A list of a priori inclusion and exclusion criteria was applied to ensure that only relevant studies were evaluated (table 1). Only clinical trials of neuromodulation devices in patients with migraine or cluster headache, as defined by the International Classification of Headache Disorders Third Edition (or earlier editions when applicable), were eligible.13–15 Studies with patient populations that comprised comorbidities were excluded. Only trials evaluating a non-invasive treatment of the central or peripheral nervous system in migraine and cluster headache were included for analysis. Invasive neuromodulation therapies were excluded from this review because these treatments are reserved for intractable diseases and are considered separate from standard of care options. Studies that were observational or diagnostic only or that used any invasive device including acupuncture were excluded, as were studies that had only healthy patient populations or patients with non-specific pain. Only studies reported in peer-reviewed publications and/or properly registered clinical trials were included; posters and congress abstracts were excluded from formal analysis.

Table 1

Clinical trial neuromodulation devices analysis inclusion and exclusion criteria

Inclusion	Exclusion
Non-invasive devices Transcutaneous Transcranial External	Invasive devices Implanted Percutaneous Requiring surgical procedures
Neuromodulation device Neurostimulation Stimulation Electrical or magnetic	Non−peer review or registry publications News articles Abstracts and posters Conference proceedings
Patients with migraine or cluster headache as defined by International Classification of Headache Disorders	Secondary headache-related indications Traumatic brain injury Cervical injury Substance abuse/withdrawal Non-specific or facial pain Infection
Published/registered between 1 January 1990 and 31 January 2018
Human clinical trials only	Observational or diagnostic only
English-language articles or registries	Review articles

Two reviewers independently reviewed the inclusion of studies according to a priori inclusion criteria. Data were independently categorised by the same reviewers and confirmed by a third reviewer. The same two reviewers evaluated the data and results of the clinical trials included in this assessment.

Data collection and analysis

The data collected in this systematic review were categorised by type of headache treatment and indication. The elements evaluated were the total patient population among all studies and whether each study was blinded, was sham controlled and used predefined IHS-recommended endpoints. For acute and preventive migraine treatments, the number of studies conforming to the IHS recommendation to include both patients with aura and without aura was also examined.16 IHS recommendations for controlled trials of drugs in migraine and cluster headache were used to evaluate whether the recommended primary and secondary endpoints were evaluated in the identified studies for each device.16–18

All devices that are both CE marked and cleared by the FDA for migraine and cluster headache as well as devices used in more than two studies for either the acute or preventive treatment of migraine are reported in the Results section for all applicable study indications. The remaining devices that had two studies or fewer are presented only in tables 2 and 3. All devices in cluster headache were reported in the Results section.

Results

Search results

Of the 647 studies identified in the systematic search, 509 were screened for eligibility, and 71 fulfilled the specified inclusion criteria (figure 1). Fifteen published studies were not registered with ClinicalTrials.gov or another WHO primary registry. Sixteen published studies had a corresponding registration and thus fulfilled the ethical recommendations for reporting results of clinical trials.19 The remaining 40 studies were registered with ClinicalTrials.gov (34 studies) or another WHO primary registry (six studies) without a corresponding publication.

Figure 1

Flow diagram of trial selection. *Trials were excluded because study objectives were mechanistic only. ICTRP, International Clinical Trials Registry Platform.*Trials were excluded because study objectives were mechanistic only.

The search identified non-invasive neuromodulation clinical trials in the acute treatment of migraine (20 studies), migraine prevention (48 studies), acute treatment of cluster headache (2 studies) and cluster headache prevention (2 studies). One study evaluated both acute and preventive interventions in migraine. The complete list of studies is available as online supplementary references.

This analysis compares clinical trial designs of non-invasive neuromodulation devices with the available guidelines from the IHS for acute and preventive pharmacological treatment of migraine and cluster headache.16–18 Developed as guidelines for the design of pharmacological clinical trials, the current IHS recommendations do not specifically address therapeutic devices. These guidelines were determined as the most rigorous standard against which to assess the quality of non-invasive neuromodulation studies in the treatment of migraine and cluster headache.

Devices with more than two studies for an indication are reported. Six devices each had two or fewer studies in the acute treatment of migraine (table 2), and eight devices each had two or fewer studies in migraine prevention (table 3). All devices with studies evaluating the treatment of cluster headache are presented in table 4. The devices reported based on these criteria are as follows: nVNS, e-TNS, repetitive transcranial magnetic stimulation (rTMS), transcranial direct current stimulation (tDCS), occipital and supraorbital transcutaneous nerve stimulation (OS-TNS), conditioned pain modulation (CPM) and caloric vestibular stimulation (CVS). sTMS was evaluated in only three studies (ie, two in acute treatment; one in preventive treatment) but is included because of its CE mark and FDA clearance as both an acute and preventive treatment for migraine.

Table 2

Non-invasive neuromodulation clinical trials in acute migraine treatments

Device type*	Studies (n)	Enrolledpopulation (total n)	Both aura and no aura patients included (%)	Blinded studies(n)†	Sham-controlled studies (n)	Percentage of studies evaluating IHS-recommended endpoints
						Primary (%)‡	Secondary (%)‡
Electrical nerve stimulation
Non-invasive vagus nerve stimulation (nVNS)	4	396	100	2	2	100	100
External trigeminal nerve stimulation (e-TNS)	3	196	100	1	1	33	100
Occipital and supraorbital transcutaneous nerve stimulation (OS-TNS)	3	106	100	2	3	0	100
Transcranial direct current stimulation (tDCS)	1	32	100	0	0	0	0
Transcranial alternating current stimulation	1	40	100	1	1	100	0
Auricular transcutaneous vagus nerve stimulation	1	50	100	1	1	0	0
Magnetic stimulation
Single-pulse transcranial magnetic stimulation (sTMS)	2	309	50	2	1	50	50
Repetitive transcranial magnetic stimulation (rTMS)	1	201	0	1	1	100	100
Other
Conditioned pain modulation (CPM)	3	476	100	3	3	0	33
Intranasal kinetic oscillation stimulation	1	36	100	1	1	0	0

*All references for studies in this table are listed in the online supplementary references.

†Double-blind or better.

‡Percentage of efficacy studies in which either a primary or a secondary endpoint recommended by the International Headache Society was used. Safety studies were excluded from this calculation.

IHS, International Headache Society.

Table 3

Non-invasive neuromodulation clinical trials in preventive migraine treatments

Device type*	Studies (n)	Enrolledpopulation (total n)	Both aura and no aura patients included (%)	Blinded studies(n)†	Sham-controlled studies (n)	Percentage of studies evaluating IHS-recommended endpoints
						Primary (%)‡	Secondary (%)‡
Electrical nerve stimulation
Transcranial direct current stimulation (tDCS)	18	855	94	15	15	50	72
Non-invasive vagus nerve stimulation (nVNS)	4	626	100	3	3	100	33
External trigeminal nerve stimulation (e-TNS)	3	194	100	1	1	33	100
Transcutaneous electrical nerve stimulation	2	76	100	0	0	0	50
Transcutaneous supraorbital nerve stimulation	1	80	100	1	1	100	100
Auricular transcutaneous vagus nerve stimulation	1	58	100	1	1	0	100
Cranial electrotherapy stimulation	1	68	100	1	1	100	100
Magnetic stimulation
Repetitive transcranial magnetic stimulation (rTMS)	10	416	90	5	8	30	80
Single-pulse transcranial magnetic stimulation (sTMS)	1	263	100	0	0	0	100
Deep transcranial magnetic stimulation	1	14	100	0	0	0	100
Other
Caloric vestibular stimulation (CVS)	3	291	67	2	2	100	67
Intranasal kinetic oscillation stimulation	2	220	100	2	2	100	100
Auditory biofeedback	1	40	100	0	0	0	0

*All references for studies in this table are listed in the online supplementary references.

†Double-blind or better.

‡Percentage of efficacy studies in which either a primary or a secondary endpoint recommended by the International Headache Society was used. Safety studies were excluded from this calculation.

IHS, International Headache Society.

Table 4

Non-invasive neuromodulation clinical trials in acute and preventive cluster headache treatments

Device type*	Studies (n)	Enrolled population (total n)	Blinded studies (n)†	Sham-controlled studies (n)	Percentage of studies evaluating IHS-recommended endpoints
					Primary endpoint (%)‡	Secondary endpoint (%)‡
Acute cluster headache treatments
Non-invasive vagus nerve stimulation (nVNS)	2	253	2	2	100	100
Preventive cluster headache treatments
Non-invasive vagus nerve stimulation (nVNS)	1§	97	0	0	100	100
Transcranial direct current stimulation (tDCS)	1	32	0	0	0	100

*All references for studies in this table are listed in the online supplementary references.

†Double-blind or better.

‡Percentage of efficacy studies in which either a primary or a secondary endpoint recommended by the International Headache Society was used. Safety studies were excluded from this calculation.

§Marin et al’s study (used for recent Food and Drug Administration clearance) not included here as it was published outside of predefined date range.

IHS, International Headache Society.

Non-invasive vagus nerve stimulation

nVNS was examined in 10 studies for the treatment of primary headache: four studies of the acute treatment of migraine, four studies in migraine prevention, two studies in acute cluster headache treatment and one study in cluster headache prevention. One nVNS study included acute and preventive migraine treatment regimens and was assessed in both interventions. Four studies were registered on ClinicalTrials.gov or another WHO primary registry but were not yet published, whereas six studies were both published and registered.

Migraine

In the acute treatment of migraine (table 2), the four nVNS studies had a total population of 396 patients who had migraine with and without aura. Two of the acute nVNS trials used blinding and sham controls along with the IHS-recommended primary and secondary efficacy endpoints; the remaining two acute nVNS trials were exclusively safety studies and were not included in the assessment of efficacy endpoints. One nVNS study (ClinicalTrials.gov identifier: NCT02378844) evaluated both acute and preventive treatment regimens and included patients with either episodic or chronic migraine; the IHS recommends excluding patients with chronic migraine from acute treatment studies, but the primary and secondary endpoints of this study were focused on evaluating migraine prevention.20 In migraine prevention (table 3), the four nVNS studies had a total population of 626 patients who had migraine with and without aura. Three of the preventive nVNS trials used blinding and sham controls along with at least one of the IHS-recommended primary efficacy endpoints; one trial used the pertinent IHS-recommended secondary efficacy endpoints. One preventive nVNS trial was designed as a safety study and was not included in the assessment of efficacy endpoints.

Cluster headache

nVNS is the only identified non-invasive neuromodulation device that has been evaluated in the acute treatment of cluster headache (table 4). The two nVNS studies had a total population of 253 patients, were blinded and sham controlled and adhered to the IHS-recommended primary and secondary efficacy endpoints. In cluster headache prevention (table 4), one study with a population sample size of 97 patients was randomised and controlled, but not blinded or sham controlled. This study adhered to the IHS-recommended primary and secondary efficacy endpoints.

External trigeminal nerve stimulation

e-TNS was examined in six studies for the acute or preventive treatment of migraine. Four studies were registered on ClinicalTrials.gov, one study was published and registered and one study was published without registration.

In the acute treatment of migraine (table 2), the three e-TNS studies had a total population of 196 patients with and without aura; one study was blinded and sham controlled. One acute e-TNS study used a primary endpoint recommended by the IHS. All three studies used at least one IHS-recommended secondary efficacy endpoint. Contrary to IHS recommendations for studies in acute migraine treatment, two of these studies included patients with chronic migraine. In preventive migraine (table 3), the three e-TNS studies had a total population of 194 patients with and without aura; one study was blinded and sham controlled. One preventive e-TNS study used a primary efficacy endpoint recommended by the IHS; all three studies used at least one IHS-recommended secondary efficacy endpoint.

No e-TNS studies for the treatment of cluster headache were identified.

Single-pulse transcranial magnetic stimulation

sTMS was evaluated in three studies for the acute and preventive treatment of migraine. Two studies were published and registered; one study was published without registration.

In the acute treatment of migraine (table 2), the two sTMS studies had a total population of 309 patients. One study used patients with and without aura. Both acute studies were blinded, and one used a sham control. One acute sTMS study used the IHS-recommended primary and secondary efficacy endpoints. One sTMS study in the acute treatment of migraine did not adhere to the IHS recommendation of excluding patients with chronic migraine. The preventive sTMS study evaluated 263 patients (table 3) with either episodic or chronic migraine, both with or without aura. The study was not blinded or sham controlled. A reduction in headache days rather than the IHS-recommended reduction in migraine days was used as the primary efficacy endpoint. Three IHS-recommended secondary efficacy endpoints for the prevention of either episodic or chronic migraine were used.

No sTMS studies for the treatment of cluster headache were identified.

Repetitive transcranial magnetic stimulation

rTMS was evaluated in 11 studies for the acute and preventive treatment of migraine. Four studies were registered on ClinicalTrials.gov or another WHO primary registry but were not published; one study was published and registered; six studies were published without registration.

In the acute treatment of migraine (table 2), one rTMS study of 201 patients was identified. The rTMS study was blinded and sham controlled and included only patients with aura. The rTMS study used IHS-recommended primary and secondary efficacy endpoints. In migraine prevention (table 3), 10 rTMS studies with a total population of 416 patients were identified. Nine studies on rTMS for preventive treatment evaluated patients with migraine with and without aura, five studies were blinded and eight studies were sham controlled. Thirty per cent of the preventive rTMS studies used IHS-recommended primary efficacy endpoints, and 80% used IHS-recommended secondary efficacy endpoints.

No studies of the rTMS device for the treatment of cluster headache were identified.

Transcranial direct current stimulation

tDCS was evaluated in 20 studies for the acute and preventive treatment of migraine and for chronic cluster headache prevention. Fifteen studies were registered on ClinicalTrials.gov or another WHO primary registry but were not published; one study was published and registered; four studies were published without registration.

In the acute treatment of migraine (table 2), the single tDCS study had a population of 32 patients with and without aura; the study was neither blinded nor sham controlled and did not evaluate IHS-recommended primary and secondary endpoints. In migraine prevention (table 3), 18 studies were evaluated with a total population of 855 patients. Seventeen studies had patients with and without aura, 15 studies were blinded and 15 studies were sham controlled. Fifty per cent of the studies used an IHS-recommended primary efficacy endpoint, and 72% used an IHS-recommended secondary efficacy endpoint.

In cluster headache prevention (table 4), one tDCS study was identified. The study included 32 patients and was not blinded or sham controlled. An IHS-recommended primary efficacy endpoint was not used, but three IHS-recommended secondary efficacy endpoints were used.

Occipital and supraorbital transcutaneous nerve stimulation

OS-TNS was evaluated in three studies of the acute treatment of migraine (table 2), which were registered on ClinicalTrials.gov but not published. These studies had a total population of 106 patients who had migraine with and without aura. All were sham controlled, but only two of the three were blinded. None of the OS-TNS studies used the IHS-recommended primary endpoint, but all used at least one IHS-recommended secondary endpoint. Two of the acute studies did not comply with the IHS recommendation to exclude patients with chronic migraine.

No studies of the OS-TNS device for the treatment of cluster headache were identified.

Conditioned pain modulation

CPM was evaluated in three studies of the acute treatment of migraine (table 2): two studies were only registered on ClinicalTrials.gov, and one study was both published and registered. The three CPM studies had a total population of 476 patients, and all three consisted of patients with migraine with and without aura and were blinded and sham controlled. None of the CPM studies used the IHS-recommended primary endpoint, and only one used at least one of the IHS-recommended secondary endpoints.

No studies of the CPM device for the treatment of cluster headache were identified.

Caloric vestibular stimulation

CVS was evaluated in three studies of migraine prevention (table 3); two studies were registered on ClinicalTrials.gov but not published; one study was both published and registered. The three CVS studies had a total population of 291 patients; two studies enrolled patients with migraine with and without aura. Two studies were blinded and sham controlled. All three studies used at least one of the IHS-recommended primary efficacy endpoints; two of the CVS studies used at least one of the IHS-recommended secondary efficacy endpoints.

No studies of the CVS device for the treatment of cluster headache were identified.

Discussion

We conducted a comprehensive systematic review of non-invasive neuromodulation clinical trials in migraine and cluster headache using the PRISMA criteria to ensure an unbiased analysis of all relevant studies. This analysis used the current IHS recommendations for drug studies as a benchmark for scientific rigour in the absence of guidelines specific to non-invasive neuromodulation16–18 and further demonstrated the heterogeneity of study designs and endpoints in clinical trials with non-invasive neuromodulation. The IHS guidelines for acute treatment studies of migraine recommend the assessment of migraine pain at a consistent time point (ie, pain freedom at 2 hours) after the intervention, and guidelines for studies of preventive migraine treatments recommend one of three clinically relevant outcomes (ie, change in migraine days, change in moderate to severe headache days, or responder rate), with the remaining two recommended as secondary endpoints.

In our systematic search, we identified pivotal studies for the three non-invasive neuromodulation therapies with regulatory clearance for migraine and/or cluster headache (ie, nVNS, e-TNS, sTMS), which are the focus of this discussion. Pivotal studies are clinical trials intended to provide evidence for regulatory approval. Publications of these studies for the available devices were assessed using the criteria provided by the European Federation of Neurological Societies (EFNS; table 5).21–30 Studies with a published classification of evidence by the American Academy of Neurology were also assessed alongside the EFNS criteria.21 27 31 32 Two of the pivotal studies in the acute treatment of migraine, Tassorelli et al’s21 study of nVNS (n=285) and Lipton et al’s28 study of sTMS (n=267), had comparable population sample sizes, with the e-TNS pivotal trial having enrolled a smaller number of patients (n=106).26 Of all identified clinical studies of migraine prevention, the PREMIUM study of nVNS had the largest population sample (n=479).20 33 The pivotal trials by Starling et al 29 for sTMS and by Schoenen et al 27 for e-TNS used population sample sizes of 263 and 84 patients, respectively. The nVNS and e-TNS pivotal trials in the acute treatment of migraine and all three key migraine prevention studies followed the IHS recommendation to enrol patients both with and without aura, whereas the acute sTMS study evaluated only patients with aura. All three pivotal studies of the acute treatment of migraine as well as the PREMIUM preventive nVNS trial and the pivotal study of preventive e-TNS were blinded and sham controlled. The pivotal study of sTMS in migraine prevention was an open-label trial and used an estimate of the placebo effect that was statistically derived according to historical controls (ie, performance goal) rather than a sham control group.

Table 5

Clinical evidence of available devices

Device	Study	Authors	CE marked	FDA cleared	AAN Classification of Evidence	GRADE
nVNS	Acute migraine treatment*	Tassorelli et al 21	Yes	Yes	I	High
	Migraine prevention*†	Silberstein et al (EVENT)31	Yes	No	II	Low
	Acute CH treatment	Silberstein et al (ACT1)22	Yes	Yes¶	NA	High
		Goadsby et al 23			NA	High
	CH prevention‡	Gaul et al 24	Yes	Yes	NA	High
		Marin et al 25			NA	Moderate
e-TNS	Acute migraine treatment	Chou et al 26	Yes	Yes	NA	Moderate
	Migraine prevention*	Schoenen et al 27	Yes	Yes	III	Moderate
	Acute CH treatment§	NA	No	No	NA	NA
	CH prevention§	NA	No	No	NA	NA
sTMS	Acute migraine treatment	Lipton et al [28]	Yes	Yes	NA	Moderate
	Migraine prevention	Starling et al 29	Yes	Yes	NA	Low
	Acute CH treatment§	NA	No	No	NA	NA
	CH prevention§	NA	No	No	NA	NA

*AAN Classification of Evidence is published for this study.

†The EVENT study was not a pivotal study.

‡Marin et al’s study published outside of predefined date range; included here based on recent FDA clearance.

§No studies identified.

¶FDA cleared for episodic CH only.

AAN, American Academy of Neurology;ACT, acute CH treatment; CH, cluster headache;FDA, Food and Drug Administration; GRADE, Grading of Recommendations Assessment, Development and Evaluation;NA, not available;e-TNS, external trigeminal nerve stimulation;nVNS, non-invasive vagus nerve stimulation;sTMS, single-pulse transcranial magnetic stimulation.

The pivotal studies of acute nVNS and sTMS evaluated the IHS-recommended primary endpoint of pain freedom at 2 hours. The acute e-TNS pivotal study predefined its primary endpoint as the change in visual analogue scale pain score at 1 hour and used pain freedom at 2 hours as a secondary endpoint. Therapeutic effects on the pain-free response rate at 2 hours were comparable among the three pivotal studies of acute treatment, with significance (vs sham) demonstrated for sTMS (active, 39%; sham, 22%; p=0.0179)28 but not for nVNS (active, 30.4%; sham, 19.7%; p=0.067)21 or e-TNS (active, 19%; sham, 8%; p=0.136).26 The IHS-recommended primary endpoint of the reduction in migraine days (from run-in to the third month of therapy) was used in the key migraine prevention trials for both nVNS (active, –2.26 days; sham, –1.80 days; p=0.146)33 and e-TNS (active, –2.06 days; sham, –0.32 days; p=0.054),27 whereas the pivotal preventive sTMS study used the reduction in headache days as the primary endpoint and did not evaluate the reduction in migraine-specific pain.29

Interpretation of the acute and preventive findings across the three devices is challenging because of differences in patient characteristics, rescue medication use and sham devices used in the trials. The acute sTMS study results represent those of an aura population, whereas the acute nVNS and e-TNS results are from a population composed of patients both with and without aura. The acute e-TNS study allowed rescue medication use within 24 hours after treatment, whereas the acute nVNS and sTMS studies defined requirements to delay rescue medication use for 2 hours.21 26 28 The therapeutic difference between nVNS and sham in both the acute and preventive nVNS studies was likely affected by the elevated effects in the sham groups. The sham device that was used in the nVNS trials was shown in a separate study to be partially active according to its observed inhibitory effects on the trigeminal autonomic reflex.34 This heterogeneity in the pivotal studies of nVNS, e-TNS and sTMS makes the comparison of these devices and their efficacy outcomes difficult.

The nVNS device is the only non-invasive neuromodulation therapy currently cleared for acute and preventive use in cluster headache. Only one other device (ie, tDCS for preventive use) has been evaluated in a study of cluster headache. The two pivotal studies of nVNS for acute cluster headache treatment (ACT1 and ACT2) evaluated primary endpoints consistent with IHS recommendations: pain relief at 15 min for the first attack without rescue medication use through 60 min and pain freedom at 15 min among all attacks without rescue medication use through 15 min.22 23 Significant therapeutic benefits were observed for the episodic cluster headache cohorts of both ACT1 (active, 34.2%; sham, 10.6%; p=0.008)22 and ACT2 (active, 48.0%; sham, 6.0%; p<0.01)23 but not for the total populations or chronic cluster headache cohorts. The two pivotal studies of nVNS for preventive use in cluster headache evaluated the frequency of attacks per week as the primary endpoint,24 25 which is consistent with IHS recommendations.18 The PREVA24 study indicated a significantly greater reduction in the number of attacks per week versus controls (−5.9 vs −2.1, respectively) for a mean therapeutic gain of 3.9 fewer attacks per week (p=0.02). Marin et al’s25 study demonstrated a decrease in mean attack frequency from 26.6 attacks per week before initiation of nVNS treatment to 9.5 attacks per week (p<0.01) afterward.

The scope of this systematic review was limited by the heterogeneity among the clinical trials analysed and the unavailability of many of the study results, which precluded a formal systematic meta-analysis of all identified studies. Thirty-seven of the 41 studies that were registered with ClinicalTrials.gov or another registry but not published did not have results available, and some did not comprehensively report all study design components examined. Studies evaluated in this review are heterogeneous in both study design and statistical power (ie, patient population sample size), which affects the ability to interpret and compare the true effects.35 36 Uniform use of prespecified outcome measures and other study design components is essential for conducting meta-analyses and cost-benefit analyses to compare different treatment interventions. It was not the scope of this review to compare the results of these clinical trials as they are heterogeneous.

The results of this review suggest several considerations for the ongoing development of clinical trial guidelines for non-invasive neuromodulation devices in primary headache. First, in an acute setting, the IHS emphasis on the first attack may not be optimal for non-invasive neuromodulation devices. These devices require patient training to ensure proper administration, which may not be complete at the first attack. To allow for complete patient training with non-invasive neuromodulation devices, it may be advisable to assess the efficacy of multiple attacks or after proper administration is demonstrated. Second, the mechanisms of preventive neuromodulation (eg, electrical or magnetic) treatments in primary headache are indirect, and efficacy may be multidimensional when considering the different pathophysiologies of migraine and cluster headache. The full preventive potential of a neuromodulation regimen may be best captured with an observational time period equal to that of the IHS-recommended period for pharmacological therapies (ie, a minimum 12 weeks). Third, reproducibility is needed to support the validity of efficacy outcomes in all therapeutic trials. Many neuromodulation devices have not yet been evaluated in multiple randomised controlled studies. The field of neuromodulation would benefit if more devices were evaluated in more than one rigorous study to support efficacy outcomes. Lastly, appropriate blinding is a particular challenge in non-invasive neuromodulation trials. Ideally, the sham device should mimic the active device as closely as possible while avoiding any inadvertent nerve activity. Sham devices with an active signal risk producing an active therapeutic effect, which could diminish the ability to achieve significant treatment differences. The suggested modifications to clinical design elements recommended in this review could support scientific rigour and inform the development of recommendations for non-invasive neuromodulation studies in migraine and cluster headache.

The efficacy and low side effect burden of non-invasive neuromodulation make it a practical, tolerable and safe therapeutic option for patients and clinicians, but efficacy may vary across the different devices. Greater rigour and consistency among non-invasive neuromodulation studies are needed to improve clinicians’ ability to select the appropriate option for their patients. Development of guidelines for evaluating the efficacy of non-invasive neuromodulation therapies is under way. Well-designed studies, such as those for nVNS that consistently adhere to stringent IHS recommendations for pharmacological trials, may help in the design of future clinical trials until neuromodulation-specific guidelines are established.

Conclusions

In this systematic review of non-invasive neuromodulation device studies in the acute and preventive treatment of migraine and cluster headache, nVNS studies demonstrated the most consistent adherence to IHS recommendations. Future trials should be rigorously designed to facilitate comparisons across devices. Guidelines specific to evaluating non-invasive neuromodulation therapies for the treatment of migraine and cluster headache will soon be available, but adherence to current recommendations for pharmacological trials will minimise heterogeneity among neuromodulation studies.