Research design considerations for randomized controlled trials of spinal cord stimulation for pain: Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials/Institute of Neuromodulation/International Neuromodulation Society recommendations.

ABSTRACT: Spinal cord stimulation (SCS) is an interventional nonpharmacologic treatment used for chronic pain and other indications. Methods for evaluating the safety and efficacy of SCS have evolved from uncontrolled and retrospective studies to prospective randomized controlled trials (RCTs). Although randomization overcomes certain types of bias, additional challenges to the validity of RCTs of SCS include blinding, choice of control groups, nonspecific effects of treatment variables (eg, paresthesia, device programming and recharging, psychological support, and rehabilitative techniques), and safety considerations. To address these challenges, 3 professional societies (Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials, Institute of Neuromodulation, and International Neuromodulation Society) convened a meeting to develop consensus recommendations on the design, conduct, analysis, and interpretation of RCTs of SCS for chronic pain. This article summarizes the results of this meeting. Highlights of our recommendations include disclosing all funding source and potential conflicts; incorporating mechanistic objectives when possible; avoiding noninferiority designs without internal demonstration of assay sensitivity; achieving and documenting double-blinding whenever possible; documenting investigator and site experience; keeping all information provided to patients balanced with respect to expectation of benefit; disclosing all information provided to patients, including verbal scripts; using placebo/sham controls when possible; capturing a complete set of outcome assessments; accounting for ancillary pharmacologic and nonpharmacologic treatments in a clear manner; providing a complete description of intended and actual programming interactions; making a prospective ascertainment of SCS-specific safety outcomes; training patients and researchers on appropriate expectations, outcome assessments, and other key aspects of study performance; and providing transparent and complete reporting of results according to applicable reporting guidelines.

1. Introduction

Chronic pain that is refractory to conventional medical management is common and is associated with high costs and significant consequences to individuals and society.[72] Spinal cord stimulation (SCS) has been used since the 1960s to treat refractory chronic pain and other conditions, including intractable angina and limb ischemia.[127] Spinal cord stimulation is a type of “neuromodulation,” which encompasses methods to stimulate the nervous system as well as intrathecal drug delivery. Other neuromodulation techniques include peripheral nerve stimulation for pain and deep brain stimulation for Parkinson disease and other indications. [17,67] The key goal of neuromodulation is relief of an otherwise refractory condition without pharmacologic side effects. SCS requires invasive procedures, including the implantation of devices, and therefore engenders risks, some of which might lead to reoperation, including infection, mechanical failure, and neurologic injury.

As with any treatment, a thorough benefit-risk assessment is required for SCS to allow patients, clinicians, researchers, regulatory authorities, and reimbursement agencies to determine its place in therapy. Methods for designing and conducting clinical trials to generate such evidence have evolved over the past half century; it is widely accepted that various levels of evidence are generated using various trial methods that provide different levels of confidence in the study results. Although real-world evidence generated from registries and case series can provide valuable insights, international consensus has accepted the randomized controlled trial (RCT) (or meta-analyses of such trials) as the highest level of evidence of the efficacy of a treatment.[12] The quality of evidence provided by RCTs, however, can vary widely depending on the quality of trial design, conduct, and analysis, especially when blinding is challenging. Therefore, determining the factors that generate credible clinical trial results must go beyond simply advocating for an RCT.

As with other invasive procedures, clinical trials of SCS are associated with special challenges compared with studies of pharmaceuticals. Examples include difficulties with blinding, choice of control groups, the fact that permanent implants may occur only in patients who have had successful trial periods, nonspecific accompaniments to treatment (such as paresthesia or lack thereof, device programming and recharging, psychological support, and rehabilitative techniques), and special safety considerations. Simply randomizing study subjects to an active or control condition does not fully address these issues.

Authorities of various types have various roles regarding market access and reimbursement. Standards for medical device studies of these organizations differ among jurisdictions and do not mandate RCT evidence; however, expectations about the design and conduct of trials to ensure the safety and effectiveness of devices have been increasing. Payers expect robust trial data to inform reimbursement decisions, which may include health-related quality of life, return to work and work productivity, long-term safety, and the total cost of care, in addition to fundamental design principles. Several important research questions, such as long-term safety, health care use, and costs, are generally not answered by RCTs; consequently, consideration of real-world data or real-world evidence is required to fill these gaps.[135]

Based on the issues outlined above, achieving consensus on standards for the design, conduct, analysis, and reporting of RCTs of SCS for pain is an urgent priority. Such standards can drive the generation of high-quality information that will inform stakeholders in their efforts to improve the treatment of chronic pain with SCS and related techniques. After extensive discussion, we chose to exclude dorsal root ganglion and other forms of neurostimulation from this report, since they do not strictly speaking involve stimulation of the spinal cord; however, we invite readers interested in the design of those studies to consider our recommendations, since similar principles apply.

2. Methods

In November 2018, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT), the Institute of Neuromodulation (IoN), and the International Neuromodulation Society (INS) convened a meeting with the aim of developing recommendations regarding the design and conduct of clinical trials of SCS for the treatment of pain {IMMPACT is a consortium of individuals from academia, government agencies (eg, Food and Drug Administration [FDA] and National Institutes of Health), pharmaceutical and device companies, and patient advocacy and research organizations; ION is a consortium of multidisciplinary experts in the field of neuromodulation with an aim to promote research and innovation to advance the field of neuromodulation to improve health and quality of life for patients; and INS is a global professional membership society dedicated to be a forum and disseminator of information pertinent to the education, scientific, and clinical standards of matters to do with neuromodulation}. Although the focus of the meeting was SCS for pain, many considerations apply to other medical devices for the treatment for pain and other conditions, particularly where subjective endpoints are evaluated. Meeting participants were selected for their expertise in preclinical and clinical research, administration, policy, and clinical care related to SCS or in conducting and interpreting clinical trials. The meeting was intended to generate general recommendations that would address a broad set of issues related to SCS clinical trials; thus, the composition of the meeting reflected a broad representation of relevant disciplines and perspectives (eg, anesthesiologists, neurologists, psychologists, basic scientists, pain experts, clinical trialists, health economists, and manufacturers) from a number of countries, while limiting the overall meeting size to promote fruitful and efficient discussion. All companies identified as manufacturers of spinal cord stimulators were invited to participate to ensure that their insights and perspectives were represented. The content of this article represents the consensus of all authors, and no editorial control was vested in any specific authors or groups.

A set of background articles was circulated before the meeting, so that participants would be familiar with all relevant issues. In addition, background lectures presented by several of the authors of this article (S.E., J.G., S.H., B.K., N.K., E.M., J.M., R.N., C.P., A.R., R.S, R.T., and S.T.) covered a broad range of relevant clinical research design issues (see http://immpact.org/meetings/Immpact22/participants22.html). After the meeting, additional literature searches were conducted, reviewed, and incorporated into the summary of the discussions and recommendations.

3. The history of research on the mechanisms, efficacy, and safety of spinal cord stimulation

Early work by Bishop and Landau had shown that when large diameter fibers in peripheral nerves were blocked, pain resulting from activation of smaller diameter fibers that mediated pain sensation was enhanced.[5] These observations gave rise to the Gate Control Theory of Pain, which posited that large diameter nerve fibers dampened the input from small diameter pain fibers at a “gate” in the dorsal horn of the spinal cord, thus increasing interest in methods to selectively stimulate large diameter nerve fibers to enhance this inhibitory effect. [103] The first attempt to accomplish this applied electrical stimulation to the infraorbital nerve.[160] Unfortunately, peripheral nerve stimulation of other mixed peripheral nerves is sometimes limited by the fact that thresholds for motor efferents are similar to those for large sensory afferents, producing uncomfortable motor effects. In the human dorsal columns, however, primary afferents are conveniently segregated from motor fibers. The first report of an implantable SCS device was from Shealy in 1967.[137] When physiologic studies suggested that SCS stimulates spinal cord structures beyond the dorsal column, terminology transitioned from “dorsal column stimulation” to “spinal cord stimulation.” Critics of the gate control theory[23,106] also suggested that the mechanism of action of SCS might be more complex than initially believed; thus, many studies have been conducted on the mechanism of pain relief produced by SCS.[94,95]

Initial reports on SCS indicated that about 50% success rate in implanted patients[126]; later reports with longer follow-up and new assessors reporting success rates as low as 15%.[39] Trial stimulation was introduced in 1975[38] as a method for assessing initial responsiveness before implantation, and independent third-party follow-up was introduced in 1977 to reduce observer bias.[110] Researchers also began to improve the comprehensiveness of outcome assessment and reporting. Clinical research progressed from mechanistic studies and uncontrolled clinical studies to controlled trials comparing devices and stimulation paradigms. Studies that focused on safety began documenting technical complications, including electrode migration and lead failures.[110] Safety has been given increased attention with the publication of studies on specific complications and of comprehensive reviews.[19,34,62,91] Studies designed to evaluate clinical outcomes as well as mechanisms of action, including recent studies of SCS-induced pain relief with functional magnetic resonance imaging, demonstrated the usefulness of combining these approaches.[90,92,101,128,162] To date, more than 2500 citations of studies of SCS that report primary data appear on The Neuromodulation Foundation's searchable database (www.wikistim.org)[115]; yet, RCTs are rare, and blinded RCTs rarer yet.

4. Types of spinal cord stimulation

Spinal cord stimulation electrodes are implanted in the epidural space over the dorsal aspect of the spinal cord. Manipulation of stimulation parameters is believed to allow for preferential targeting of specific fibers or cells to produce different effects.[11,95] Spinal cord stimulation is categorized based on stimulation attributes such as frequency, tonicity, induction of paresthesia, and use of feedback to adjust stimulation[11,123] (Table 1). The traditional form of SCS uses relatively low frequencies, generally induces paresthesia, and is commonly called “tonic”; however, because the term “tonic” strictly speaking applies to any waveform with a constant amplitude and evenly spaced pulses, in this article, the term “low frequency” will be used. More recently introduced types of SCS use higher frequencies (eg, 10 kHz or “high-density” stimulation) or deliver multiple pulses in quick succession in each stimulation phase (“burst stimulation”). These other SCS categories are programmed at amplitudes generally below the paresthesia threshold. All SCS therapies are challenged by the ever-changing distance between the electrode and the spinal cord with patient movement, the cardiorespiratory cycle, and coughing,[68] which might contribute to variability in clinical outcomes.[164] For this reason, investigators developed “closed loop,” which measures the evoked compound action potential from the spinal cord after each pulse and automatically adjusts the strength of the next pulse to maintain a specified evoked compound action potential size. [101,128,129] In principle, stimulation parameters also may be adjusted based on other inputs, such as accelerometers and time-of-day, and they may be interleaved, “shuffled,” or otherwise varied in innumerable ways.[89,112] New approaches to SCS are under continual development, including efforts to preferentially stimulate the dorsal horn by delivering a broad field shape by using multiple independent contacts, and multiplexed stimulation with targets that include glial cells. The main implication of various types of SCS from a study design perspective is that some are programmed to produce perceptible paresthesia and others are not, which creates challenges in blinding and measurement of paresthesia. In addition to the multiple approaches outlined above, SCS advances have also resulted in improved types of SCS electrodes (Fig. 1A), pulse generators (Fig. 1B), and power sources.

Table 1

Types of spinal cord stimulation.

Name	Frequency	Pulse width	Amplitude	Waveform	Comment
Low frequency	10-100 Hz	100-1000 µs	1-10 mA		Traditionally paresthesia-based, manually adjustable output. Also called “tonic,” although this technically describes any waveform with constant evenly spaced pulses.
Burst	Passive charge recovery: 40-Hz intrabursts of 5 pulses at 500-Hz interburstsActive charge recovery: Up to 8 0-Hz intrabursts of 3-7 pulses at 2- to 1200-Hz interbursts	500 µs20-1000 µs	1-5 mA1-5 mA		Several types of burst stimulation, some with passive, others active charge recoveryFixed output, typically below paresthesia threshold
High frequency	1-10 kHz	30 to 150 µs	1-5 mA		Fixed output, typically below paresthesia threshold.
High charge	300-1200 Hz	150-800 µs	1-5 mA		Fixed output, typically below paresthesia threshold.Minimal time between pulses (high duty cycle).
ECAP-controlled closed loop	10-100 Hz	100-450 µs	Automatically adjusted for every pulse (usually within 1-10 mA).		Stimulation amplitude adjusted based on physiologic response to stimulation, eg, evoked compound action potentials (ECAPs), to maintain a target physiologic response amplitude.
Multiple contact calibrated field shape	10-1000 Hz	100-350 µs	1-5 mA		Stimulation amplitude on each contact adjusted to preferentially modulate different areas of the spinal cord

Figure 1.

(A) Types of generators. All spinal cord stimulation systems include an external, portable (and therefore battery powered) transmitter which emits (and in some cases receives) a wireless or radiofrequency signal providing telemetry and/or power. The implanted generator may contain a battery (which may be rechargeable), allowing autonomous operation, or it may operate on external power alone. (B) Types of electrodes.

5. Systematic review of methods in randomized controlled trials of spinal cord stimulation for pain

A systematic review of research methods of SCS RCTs for pain was performed to inform the discussion of clinical trial standards; results of this review, to be published elsewhere, are summarized here, excluding studies on angina. The dates of study manuscripts ranged from 1994 to 2018. Most of the 34 RCTs studying pain focused on back or leg pain, including failed back surgery syndrome. The majority of studies (n = 20) used a crossover design, although a substantial minority (n = 14) used a parallel design. Most studies reported on subjects who received low-frequency SCS (n = 22), with fewer studies covering high-frequency (n = 8), burst (n = 6), or other waveforms (n = 1). The most common control group was low-frequency SCS (n = 13), followed by placebo on/off (the device could be turned off) (n = 10), usual care (n = 8), physical therapy (n = 2), and surgery (n = 1). Trial stimulation was included in about half of the studies, and most studies (60%) allowed programming adjustments after randomization. Coadministration of noninvasive pain treatments was specifically allowed in most studies (65%), unspecified in about one-third, and prohibited in one. Most studies did not specify among the eligibility criteria the willingness of patients to discontinue or keep concomitant medications stable, failure of more conservative treatments, minimum pain intensity, or pain duration. The median duration of intervention (and assessment of the primary outcome) was 12 weeks (ranging 0-208 weeks). The primary endpoint was specified in nearly all studies; pain intensity was the most frequent. About a third of studies had multiple primary outcomes; few addressed adjustment for multiplicity. Reporting of primary and secondary endpoints was often unclear.

Most studies did not specify how adverse events (AEs) were collected. Adverse events of special interest, which can significantly affect the outcome of SCS and are not normally subject to specific data collection and reporting in clinical trials (eg, fractured electrodes and infection), were reported in a minority of studies; among these, a clear method for prospectively capturing them was seldom prespecified. The primary hypothesis was superiority in about half of the studies (n = 18), with fewer studies designed as noninferiority (n = 4) or equivalence studies (n = 1), and the remainder not specifying (n = 11). The primary analysis cohort was intention-to-treat (ITT) or modified ITT in about a third, with a few reporting both a per-protocol and an ITT analysis. Few studies reported a method for accounting for missing data; the most common method reported was last observation carried forward. Sample size calculations were reported in about two-thirds. The median number of participants completing the primary analysis was 33. The method of randomization was assessed as having a low risk of bias in most studies (eg, computer-generated randomization) but was not specified (higher risk of bias) in about a third. The majority was unblinded (68%). Half were single-center studies and half multicenter; only a minority noted registration. Most studies reported funding sources; of those, all were industry-funded.

In summary, many studies failed to adhere to basic elements of clinical research methodology,[55,56,65,99] including clear eligibility criteria, clear process of randomization, blinding, standardized patient management, prespecification of analysis methods, comprehensive reporting of efficacy and safety outcomes, reporting of treatments and interactions beyond the study interventions, and transparency (registration of the study and disclosure of funding sources and conflicts). Thus, although these studies were all ostensibly RCTs, inadequacies in study design, analysis, conduct, and reporting undermine the credibility of the conclusions.

6. Objectives, comparators, and research designs

A well-designed RCT begins with a study objective, or hypothesis, framed in terms of the comparison of a test treatment with 1 or more comparators. There are 3 basic approaches: (1) Superiority studies evaluate whether one treatment is better than another. (2) Noninferiority [NI] studies evaluate whether the test treatment is no worse than the reference (by an acceptable amount, called the NI margin) (Fig. 2). The presumption is that the new treatment has some other advantage (eg, in terms of availability, cost, invasiveness, safety, etc.). (3) Equivalence studies aim to show that a new treatment is neither better nor worse than a reference treatment by a specified difference; this approach is common in pharmacokinetic studies and is infrequently performed in drug or device studies.

Figure 2.

Interpretation of noninferiority studiesa. The letters indicate the point estimate of efficacy, and the error bars the 95% confidence intervals in these hypothetical trials. The vertical line labeled “0” indicates the point of zero difference between “new treatment” and comparator treatment. Δ indicates the prespecified noninferiority margin. Adapted from Ref. 61, with permission.

The advantages of superiority designs are clarity of interpretation (although statistical superiority and clinical superiority must be considered separately) and smaller sample size requirements (compared with NI and equivalence designs). In pharmaceutical studies, demonstration of superiority over placebo has been considered synonymous with efficacy, largely because randomization, double-blinding, and equal management of patients in both study arms mitigate biases that inflate the observed efficacy of the study treatment. The fundamental question of whether SCS is superior to sham SCS has been addressed, and superiority claimed, in a few RCTs[1,145]; however, because measures to control these biases generally have not been implemented, conclusions of superiority are difficult to substantiate.

Noninferiority is achieved when the lower limit of the 95% confidence interval (CI) around the observed effect of treatment excludes the NI margin; this is interpreted to mean that the possibility of the true effect of the test treatment being worse than the reference treatment beyond the specified threshold has been excluded. Various approaches are available for selecting NI margins.[36,122] The rationale for NI designs must include evidence for the efficacy of the comparator treatment (eg, a superiority study vs placebo).[122] Because of the high and variable placebo response rate in pain trials,[37] demonstration of superiority of the comparator to placebo in a previous clinical trial does not guarantee that the comparator would have been superior to placebo in the current trial. Therefore, NI studies of treatments for pain are not scientifically valid in the absence of an internal demonstration of assay sensitivity (and consideration of biases, see below).[47] Internal demonstration of assay sensitivity can be achieved by demonstration of superiority of any active treatment over sham treatment or by superiority of one treatment group over another. Failing to implement measures to control sources of measurement error (eg, accuracy of pain reporting, intersite variability, and concomitant pain treatments) increases the likelihood of a finding of NI, since measurement error makes it virtually impossible to differentiate treatments that are in fact different. he only practical way to ensure that such sources of measurement error have been controlled is an internal demonstration of superiority of one group over another (assay sensitivity). (A similar issue applies to a superiority study that fails to demonstrate a between-group difference: It can be challenging to discern whether the lack of a difference was due to a failure of the study to control measurement error or to true lack of difference between the treatments.). An additional disadvantage of NI designs is the requirement for a large sample size, potentially 4 times the size of a superiority study, depending upon the choice of NI margins and other assumptions.

The choice of comparator in SCS studies is driven by the study objectives/hypotheses. Options include (1) Comparison of a test stimulation paradigm with sham (for the purpose of this article, sham and placebo will be used interchangeably) using the same device. For example, implanting a single type of device then entering patients into a crossover study comparing 1 stimulation paradigm with sham stimulation using that device.[1,146] (2) Comparison of a test stimulation paradigm to a comparator stimulation paradigm using the same device.[109] (3) Comparison of SCS treatment with an alternative form of care, such as a surgical procedure or medical management.[88,111] (4) Comparison of one stimulation paradigm vs another using different devices.[15,78] (5) Comparison of the same stimulation paradigm using different devices, which might involve alternative implantation techniques.[2,84,113]

The most common study design frameworks are parallel studies or crossover studies. Superiority and NI hypotheses can be tested in either design. In a parallel study, patients are randomized to one treatment or another and followed for a sufficient length of time to support the desired clinical interpretation. The main advantage of a parallel study is simplicity of execution and interpretation. Longer observation periods are feasible, compared with a crossover study, where, eg, a 3-month observation of each treatment means the patient is scheduled to be observed for 6 months. Parallel studies require larger sample sizes than crossover studies; in addition, parallel studies do not allow for within-subject comparisons of treatments. In parallel studies of SCS, care must be given to defining the primary analysis cohort: If patients are randomized to different SCS treatments, then undergo trial stimulation, and only the implanted patients are included in the primary analysis cohort, the populations might no longer be equivalent, which violates the intent-to-treat principle (and might invalidate conclusions). Comparing 1 waveform administered by 1 device with another waveform administered by another device will leave uncertain whether any observed advantages were due to waveform, device, both, or potential confounders, such as differences in ancillary care or programming. An optional crossover at the end of the originally assigned treatment in a parallel study phase might yield further information, although careful attention to biases and statistical power are needed to support interpretation of such observations.

In the crossover design, patients are randomized to sequence, where they get 2 or more treatments in a prespecified and balanced order; in incomplete block designs, more than 2 treatments are studied, but each patient does not receive all treatments. An advantage of a crossover designs is the ability to obtain patients' direct observations on the differences between treatments (eg, preference). The main disadvantages are the potential for interpretation problems due to carry-over effects (the effect of a treatment is influenced by the previous treatment), period effects (the effect of treatments are different in different periods), sequence effects (efficacy depends on sequence), and most importantly, treatment-by-period interactions, where the relative efficacy of the treatment is different in one period compared with another.[121,136] In SCS studies, implanting a single device then performing a crossover experiment might be the most efficient (least sample size) and most unbiased method for comparing one waveform with another. Selecting patients for implantation based on their screening stimulation response to 1 waveform, however, creates a bias in favor of that waveform. Comparing one device with another requires a parallel study to avoid having to implant 2 different devices in the same patient.

Since each study is unique, it is impossible to recommend specific study designs for every scenario. In general, for studies seeking to compare one waveform with another (where comparing devices is not relevant), the most efficient approach is implanting a single device and comparing waveforms in a crossover design—with care taken to avoid biases during the screening phase, as noted above. If long observation periods are desired, a parallel study might be needed so patients are not in the study for too long. For studies seeking to compare different devices, or SCS vs another form of treatment, parallel studies generally make the most sense to avoid having to implant 2 different devices in the same patient and to allow long observation periods. Attention to bias and expectation is, as always, critical in such designs to avoid uninterpretable results. When rigorous double-blinding cannot be achieved (and documented), such as when comparing a paresthesia-based to a non–paresthesia-based form of stimulation, attention to documenting similarity of patient management and expectation management in all study groups, as discussed below, becomes critical. Finally, a study must be feasible: Attempting the perfect study—eg, sham controlled, when clinicians already accept efficacy—might be impossible.[93]

7. Sources of bias in randomized controlled trials of spinal cord stimulation and their mitigation

Bias refers to a type of measurement error where the effect of 1 treatment is exaggerated or diminished in comparison with another treatment due to the way the study was designed or conducted.[57,159] The consequence of bias is that the observed difference between treatments is different from the true difference between treatments (thus, study results are by definition inaccurate). Although no study result is perfectly accurate, when biases are significant, the trial might lead to a false conclusion: a finding that the treatments are different when they are not (false-positive) or a finding the treatments are not different when they are (false-negative). The results of bias are not binary: Biases can increase or decrease observed differences between treatments whether the primary hypothesis test is statistically significant or not. Put another way, a study might claim superiority because of P < 0.05, but if the study was biased, the observed superiority might not be due not to the study treatment but to how the study was designed or conducted. Bias can produce as large an effect, if not larger, than actual treatment effects.[76]

Sources of bias and measurement error, emphasizing those relevant to trials on SCS, are listed in Table 2. Some examples of biases include allocation bias, observer/expectation bias, and asymmetric interactions between treatment groups.[116,134] Expectation and observer biases can be conscious or unconscious,[77] and the expectation of the investigator can be transmitted to the subject consciously, unconsciously, verbally, or nonverbally.[13,58] For that reason, “single-blind” studies, where the investigator knows the treatment assignment and the patient in theory does not, must be regarded as unblinded. Unfortunately, blinding certain types of SCS studies can be challenging or impossible. In such cases, options to mitigate expectation/observer bias include training of patients and research staff to a balanced and neutral level of expectation. [165] Reporting on the effectiveness of blinding, or magnitude of expectation of benefit, might help in the interpretation of study results.[15,101,145] Furthermore, patients in different arms of SCS studies must be treated equally, and this equal treatment must be documented and reported.[90,101] Studies comparing one device to another deserve particular scrutiny with respect to bias and in addition raise the issue of whether the latest versions of the respective technologies are being compared.

Table 2

Sources of bias and measurement error in randomized controlled trials and their mitigation.

Source of error	Description	Mitigation options
Allocation bias	Investigators choose which subjects go in which groups	Randomization[133]Concealment of assignments before patient selection
Expectation bias	Subjects report the response they expect (eg, pain relief)Research staff expectation is transmitted to patients consciously or unconsciously	Double-blinding[133]Placebo controlsNeutral expectation training of researchers and subjects[165]Balanced information in all groups[76]
Unbalanced randomization*	Treatment groups differ by prognostic factors or treatment effect modifiers	Stratified randomization[82]Adjust analyses for potential confounding factors
Observer bias	Those who observe the treatment effect report their desired outcome	Double-blindingNeutral expectation training of observers[58]
Unbalanced ancillary treatment	Patients in 1 group get more attention, supplemental treatments, visits, psychological support, etc.	Double-blindingStandardized and documented ancillary treatment and interactions (eg, programming)
Patient selection or characterization
Inaccurate diagnosis	Patient does not have the disease being studied	Central review of diagnostic assessment[85,97]Investigator training[97]
Inaccurate pain reporters	Patients might not be able to report pain accuratelyInaccurate pain reporters are also placebo responders	Accurate pain reporting training[149,150]Exclude patients with excess variability of clinical or experimental pain[31,32,42,61,140,148]
Placebo responders	Preferential placebo responders have higher than average responses to placebo but not to active treatment	Select internally focused patients[31,32,148]Neutralize expectation[32,150,165]
Baseline score inflation	Subjects/investigators might inflate baseline scores to meet enrollment criteria	Mask entry requirementsUse different measures for the primary endpoint and for inclusionStatistical surveillance[31,86]
Unstable or resolving pain conditions	Pain that is highly variable or destined to resolve during the study decreases assay sensitivity	Enroll patients with a history of at least 12 months of moderate to severe chronic painMinimum pain intensity of 4-5/10Prerandomization period long enough to establish stable baseline
Psychological comorbidities and substance abuse	Patients with psychological comorbidities or substance abuse report pain less reliably and might be less compliant with study procedures	Exclude such patients based on established validated assessments (including urine drug screens) unless specifically studying these populations[31,32]
Studying heterogeneous phenotypes	Studying mixed phenotypes might result in failed studies when the treatment is effective in a specific phenotype	Phenotype all subjects at baseline and evaluate efficacy by phenotype[33,119]
Duplicate subjects	Patients often deceitfully enroll in the same study at multiple sites or in multiple studies, putting themselves and the study at risk	Use a duplicate subject detection service in every study[21,22,138]
Medical and treatment history	Patients are often unable to supply all relevant information about past or current medical history and pharmacologic and nonpharmacologic treatments	Consider methods to import prescription monitoring data and electronic medical records data for enrolled subjectsTry to obtain medical records
Outcome assessment
Insensitive outcome measures	Measures must not only be valid and reliable, but also responsive to treatment differences	Choose the most responsive measure that is valid for the target conceptPrioritize disease-specific over generic measuresConsider developing a new measure if no suitable measures are available or if there is reason to believe that a new measure would be substantially more responsive than available measures[29,49,125]
Noncompliance with outcome assessments, eg, e-diaries	E-diary compliance is poor in many studiesMissing data in general must be minimized	Automated reminders; alerts to coordinators for missed entries; calls from coordinators to subjects after missed entries; and real-time central monitoringBack-up in-clinic assessments of the primary endpointAvoid paper diaries[142]
Adherence to study treatments
Failing to measure adherence (to study or rescue treatment) accurately or to achieve adherence	Variable and poorly documented adherence to SCS regimen or rescue medications	Document prescribed SCS regimen and adherence to itStandardize and provide rescue meds; measure adherence electronicallyReal-time central monitoring of adherenceAdherence promotion strategy[6,8]
Confounding by subject
Subjects failing to follow protocol	Subjects need to follow the protocol, particularly medication adherence, diary compliance, accurate symptom reporting, and stable regimens of nonstudy treatment (eg, physical therapy)	Perform a training needs assessment based on risks to data quality[48]Follow principles of validated training (Katz N, unpublished, 2020)
Physical and psychological treatments	No new physical or psychological interventions should begin during studies. Patients should maintain unchanged physical and psychological regimens	Provide structured guidance to subjects about physical and psychological therapeutics; consider structured supportCapture changes in physical and psychological regimens using questionnaires; consider objective measures such as actigraphy[153]
Site selection and management
Overly heterogeneous sites or regions	Heterogeneity in health care systems, language, culture, availability of treatment, etc. introduces error	Minimize the number of sites; invest in prestudy recruitment activities to maximize the number of patients/site[104]Minimize heterogeneity in sites and regions
Variability in study conduct by sites	Sites implement protocols in varying ways that might be difficult to predict, describe, or understand	Perform a training needs assessment based on risks to data quality[48]Follow principles of validated training (Katz N, unpublished, 2020)Central statistical monitoring and intervention[48,70] (and Katz N, unpublished, 2020)

Can produce a positive or negative bias.

SCS, spinal cord stimulation.

8. Study population

Selecting patients for clinical trials requires a balance between internal validity (the results of the experiment can be attributed to the study treatments rather than to extraneous factors), which requires patient homogeneity, and external validity (being able to extrapolate the study results to a definable population of interest), which requires that the study population resemble the external population of interest. Since an invalid trial does not generalize to anybody, internal validity must be considered before external validity; for this reason, stringent inclusion/exclusion criteria, and sufficiently detailed subject characterization to determine who was studied and which subject characteristics might predict response, are required in clinical trials.[44] Subject eligibility for SCS trials progresses in stages: Initial screening for general and disease-specific criteria determines whether patients are eligible to proceed to randomization and a temporary trial of SCS (in either order), and the screening trial itself, when used, determines eligibility for implantation and continued observation.

General eligibility criteria typically include a clear and confirmed diagnosis and pain type, stable underlying pain pattern, minimum and maximum pain intensity and duration, minimal psychosocial vulnerabilities, ability to cope with technology, failure of more conservative treatments, and stable or absent concomitant analgesics.[32] Patients can be characterized using body maps,[9] where areas of pain and its intensity can be captured; bedside sensory testing for sensory phenotyping,[33,117] quantitative sensory testing, and other neurophysiologic techniques can be used for mechanistically oriented studies. Condition-specific inclusion criteria must be carefully considered, including comorbidities and factors that influence SCS outcomes. For example, in failed back surgery syndrome, patients might have variable contributions of neuropathic pain vs nociceptive pain. Certain types of neuropathic pain, especially those with predominant large fiber damage and relatively preserved nociceptor function, might be more responsive to SCS than nociceptive pain.[88] Using tools to determine the extent to which the patients' pain is neuropathic might be helpful[97]; however, the predictive validity of these tools for response to treatment is not yet established.[51,98] Importantly, evaluating whether patients meet eligibility criteria should not be left to a simple checklist but should instead depend upon collection of primary data, so the investigator's determination that a patient met eligibility criteria can be verified.

The rate of conversion of trial stimulation to permanent implants varies widely among centers for unclear reasons—potential explanations include variability in characterizing patients, the duration of trials, a lack of blinding and other controls for nonspecific treatment effects, and measurement error in capturing clinical data.[69] Thus, trial stimulation procedures should be standardized and well-documented. Alternatively, since the evidence for the predictive value of SCS trials is scant and evolving, consideration may be given to skipping trial stimulation.[35]

The concept of enrichment[44] applies in several different ways to SCS RCTs and can be considered in terms of prerandomization vs postrandomization enrichment. Prerandomization enrichment can be accomplished by performing a screening trial then randomizing responders to different treatments, eg, waveforms or devices. Enrichment procedures, however, might bias the study in favor of one treatment arm over another. For example, if enrichment is performed with waveform A, and responders are then randomized to waveform A vs waveform B, the study is biased in favor of waveform A. Multiple stage enrichment has been performed to address this issue. For example, the PROCO study first enriched based on response to low-frequency stimulation that evoked paresthesia, then, after implantation, required a response to paresthesia-free stimulation[146]; this approach raises the issue of prerandomization vs postrandomization enrichment.

Postrandomization enrichment introduces analysis and interpretation issues and does not meet criteria for a valid enriched enrollment design.[44] For example, patients can be randomized to device A or device B, undergo a trial of the device to which they were assigned, and if they experience an initial response, continue onto implantation and follow-up. Although reflective of clinical reasoning, such studies require careful analysis and interpretation, since the analysis cohorts are no longer the originally randomized groups, violating the ITT principle and potentially introducing bias.

9. Outcome assessment

Planning to assess clinical outcomes requires reviewing the purpose of the study, considering information different stakeholders might be interested in (patients, families, health care providers, regulators, payers, and employers), and crafting an “endpoint model.” [124,161] The endpoint model begins with establishing the study objectives (primary, secondary, and exploratory), which are clinical concepts (or domains) to be evaluated in the study. For example, an objective could be to evaluate the effectiveness on pain intensity of device A vs device B. A clinical outcome assessment (also called instrument or measure) refers to a tool that will be used to measure this clinical concept (eg, a 0-10 numerical rating scale for average pain over the past 24 hours, with 0 = no pain and 10 = worst pain imaginable). The endpoint refers to how the assessments will be captured to address the objective. For example, an endpoint could be the average of 7 consecutive scores during the baseline week subtracted from those during week 12, captured with an electronic diary. The endpoint model is a tabular array of each study objective, assessment, and endpoint, ensuring that each objective is supported by appropriate assessments and endpoints and, conversely, that each assessment and endpoint is mapped onto an objective. Assessments that do not map onto a specific study objective, such as baseline demographic assessments, should be categorized as such.

Outcome assessments can be clinical (COAs) or nonclinical (biomarkers).[161] COAs include any assessment that can be influenced by human choices, judgment, or motivation. Biomarkers, such as imaging, heart rate variability, and other physiological assessments, are less likely to be linked to patient influences, but their clinical relevance requires demonstration of a link to a COA. COAs must be well-defined and possess adequate measurement properties to demonstrate the benefits of treatment. Types of COAs include patient-reported, clinician-reported, observer-reported, and performance outcomes.[120,161] Performance outcomes can be clinician-rated or supervised (eg, staircase maneuver in osteoarthritis of the knee)[151] or unsupervised “objective” measures (eg, actigraphy). [147] Measuring only physical activity parameters, such as walking distance, speed of walking, or time spent resting, might be misleading. Long-term recordings of detailed physical activity measurement using multiple body sensors suggest that the temporal dynamics of activity patterns are important, and their complexity decreased in chronic pain.[118,119]

Table 3 presents a list of recommended domains and measures originally recommended by IMMPACT for chronic pain studies,[28,154] later suggested by patients in an IMMPACT survey,[155] additional ones commonly used in chronic pain studies, and several specifically recommended for SCS studies. Different painful disorders for which SCS may be studied will require additional domains and measures specific to those disorders, such as blood flow or ulcers for studies in critical limb ischemia or allodynia in studies of neuropathic pain. In addition to the listed measures, Patient-Reported Outcomes Measurement Information System (PROMIS) is a comprehensive set of measures of physical, mental, and social health developed with funding by the US National Institutes of Health that may be used in clinical trials.[131]

Table 3

Recommendations for randomized controlled trials of spinal cord stimulation for chronic pain: outcome measures and reporting.

Outcome evaluation
Endpoint model	Describe the endpoint model in the protocol; have clear and aligned objectives, assessments, and endpoints	161
Primary endpoint	Must be prespecifiedAdjust for multiplicitySpecify missing data imputation method	25,56,155
Secondary endpoints	Key secondary endpoints will more likely show differentiation between treatments if they also meet minimum baseline severity criteria
Adverse events (AEs)	Prespecify the method for ascertaining AEs (eg, open-ended, spontaneous, checklists, scripts)	80,141
Outcome measures
Core domains/measures*	Pain intensity 0-10 numerical rating scale†	27,31
	Physical function Brief Pain Inventory Interference Items Oswestry Disability Inventory Roland–Morris Disability Questionnaire WOMAC function subscale	28,156
	Emotional functioning Beck depression inventory Hospital anxiety and depression Scale Profile of mood states	28,156
	Global improvement or satisfaction Patient global impression of change Patient satisfaction scale Would you do this again?	28
	Concomitant and rescue medications Careful capture in-clinic of concomitant analgesic medication including dose, frequency, and reason for use (ie, index or nonindex pain) Rescue medication use by electronic methods Opioid consumption	6,28
	Patient Disposition Adherence to treatment regimen: SCS and any additional treatments Reason for early termination	154
	Sleep and fatigue MOS Sleep Scale Pittsburgh Sleep Quality Index Multidimensional Fatigue Inventory	28,156
	Health-related quality of life EQ5-D SF-36 Quality-adjusted life years (QALYs)	28,156
	Cost-effectiveness Dollars per QALY	126
	Health care costs Country-specific health and social care costs	126
	Work status Work Productivity and Activity Impairment Questionnaire Workplace Activity Limitations Scale Work Limitations Questionnaire	156
	Patient preference (for crossover studies) Preference scale	63
	Abuse-related events MADDERS	152
	Opioid side effects Opioid Side Effects Scale	40
	SCS-specific measures Device survival/revision-free survival Durability of analgesia Adherence to SCS regimen (eg, hours use/day) Recharging burden (recharge interval, time required for recharge) Programming parameters (see below)	102
Safety and complications‡	Prospective monitoring for Infection (superficial, deep) Cerebrospinal fluid leak Hematoma Stimulator pocket fluid collection Wound dehiscence Skin erosion Allergic reaction Lead migration/breakage Hardware malfunction Battery failure Loose connection Implantable pulse generator migration/discomfort Dysesthesia Device explantation (and reason) New device-related pain syndrome	62
Reporting§
Methods	Patient characteristics and eligibility Neuropathic pain assessment Sensory phenotyping Literacy and numeracy Ability to report pain accurately Level of expectation
	Blinding How it is maintained and documented Patient access to controllers and recharging
	Expectation How balanced expectations are created and monitored Information provided to subjects and staff
	Adherence Prescribed SCS regimen, allowed concomitant and rescue medication How adherence will be measured and documented
	Analysis Primary estimand Complete primary endpoint including primary analysis cohort, clinical outcome assessment, analysis method, and handling of missing data All secondary endpoints (secondary endpoints can be considered for claims if appropriate procedures for handling multiple endpoints are implemented) Confidence intervals with interpretation Handling of multiplicity for studies with multiple primary endpoints Sample size calculation and supporting assumptions	56
Results	Follow applicable reporting guidelines CONSORT statement Pain-specific supplement to CONSORT Recommendations for reporting crossover studies Recommendations for describing complex interventions Recommendations for reporting cost-effectiveness studies	106525466132
	SCS-specific reporting recommendations Programming details Position of cathode, method of placement (eg, paresthesia mapping), and details of trial stimulation
	Description of risk-based quality management activity

Recommendations are not meant to be prescriptive; measures are provided as examples and should be evaluated based on the study context. Not all domains or measures will be appropriate for all studies. Review of the psychometric properties of all measures should be performed for each study.

Location should be specified, eg, index vs nonindex pain location.

Note that some complications are associated with AEs and others are not. Ascertainment of specified complications should be prospective whether associated with AEs or not.

Reporting recommendations are not exhaustive but highlight areas of special importance in reporting SCS studies.

SCS, spinal cord stimulation.

Responder analyses (proportion of patients achieving a predefined outcome) may also be considered for clinical interpretability, statistical, or regulatory reasons. The US FDA, eg, evaluates responder analyses and composite endpoints to help reveal the overall effect of an intervention on an individual basis as well as overall clinical benefit. The FDA recommends additional discussion of the pros and cons of a responder analysis vs mean analysis for assessing populations vs individuals, across a variety of trial designs. Single outcome responder indices refer to prespecified cutoffs on a single measure (eg, decreased pain intensity by 30% or 50%). Composite measures combine different patient assessment measures into a single determinant of whether the patient has “responded” to treatment; eg, a patient may be considered a responder if their pain decreases by at least 50% and they do not increase use of concomitant analgesics. Composite responder indices might be more or less statistically responsive than continuous or single outcome responder indices, so caution must be exercised and decisions preferably based on analyses of previous clinical trials. Depending on the goals of the study and the needs of consumers of the data, various elements can be considered for composite measures, such as pain, analgesic consumption, function, tolerability, completer status, and global satisfaction. Specific disorders might suggest specific elements of a composite, such as with angina or limb ischemia. The composite measure should be a measure of a specific clinical concept and not a complex of different clinical concepts, particularly where regulatory approval for a specific indication is sought. Generally, the most statistically efficient endpoint should be selected for the primary endpoint to experiment on the fewest subjects possible to achieve the study objective. Less efficient endpoints needed for clinical interpretation can be relegated to secondary endpoints, recognizing that the study will be underpowered for those results.

10. Adverse events

Spinal cord stimulation complications occur in 23% to 55% of patients, must be addressed early, and can be reduced.[4,62,102,157] The rate of reported AEs depends on the type of study (retrospective, prospective), duration of follow-up, method of capturing and reporting AEs, and other factors. Adverse events can be divided into biological and technical categories. Biological complications refer to adverse consequences to the body, such as infection, hematoma, or wound dehiscence; technical complications refer to device hardware problems, including lead migration, lead fracture, and implanted pulse generator (IPG) malfunction, which might also have medical consequences, and uncomfortable paresthesia[62,86] (Table 3). Infections are reported in 3% to 10% of studies[20,62,157] and most commonly occur in the IPG pocket; nearly all infected devices are partly or completely removed.[50] Reporting should distinguish between superficial and deep surgical site infections.[20] Surgical device explantation due to an AE is another key safety outcome and can be presented as a survival curve (median time to explantation) or as the proportion explanted over a fixed period.[4] Explantation can be performed for other reasons as well (eg, the patient no longer has pain, and the device is no longer effective), which should also be reported. Complications can be characterized by whether they require surgery, whether the device was partially or completely explanted, time to occurrence (which differs among different complications), and severity.[62]

Study reports should include AEs collected by a standard and explicit method (eg, a nonleading question, eg, “have you had any issues since your last visit”), and should identify serious AEs, AEs related to the device, AEs of special interest (which may be agreed between sponsor and health authority and may be assessed by a prospective clinician questionnaire), and unanticipated AEs related to the device. Adverse events must be collected in a similar fashion for all participants in the study according to postmarket surveillance guidelines within their respective country/state. If a novel SCS is being trialed, additional measures may be required for monitoring AEs by the competent authority at the time of clinical investigation approval. Definitions of AEs and related terms are available in the ISO/DIS 14,155 guidance on “Clinical investigation of medical devices for human subjects—Good clinical practice.” [73] Adverse events and risks should also be evaluated in the context of benefit in any given study. This risk-benefit ratio is also a factor in the review of a device for marketing authorization in several countries and regions.

11. Study execution

Any clinical trial protocol can be ruined by poor study conduct. Study conduct begins with crafting a simple and doable protocol, then proceeds to site selection. For SCS studies, capabilities required beyond general clinical research capabilities include skill in the investigational procedure,[64] skill in the comparator procedure(s), skill in providing clinical care for patients undergoing SCS, which might include psychological or rehabilitative support and the management of complications, skill in performing diagnostic and outcome assessment, and a quality system focused on SCS. Capturing and potentially controlling for the level of expertise of the center should be considered[64]; a checklist of investigator characteristics is provided in supplemental Table 1 (available at http://links.lww.com/PAIN/B272). In multicenter trials, standardization of implementation across centers can be a significant challenge, and seemingly minor differences can introduce variability of outcomes, reducing the opportunity to establish (or falsify) efficacy of any treatment approach.

Patient selection is a challenging issue in any therapeutic area; in SCS studies, additional considerations apply. Clinical research ethics require equipoise, which can be achieved by providing balanced information in all available sources, including company websites, internet postings about the clinical trial, informed consent documents, patient education materials, and verbal descriptions and body language.[76,81] Interactions should be scripted to the extent possible to minimize the risk of bias. E-consent might provide a better method for standardizing consent procedures compared with traditional informed consent interactions.[59]

Ethical issues include making implantation or earlier access to treatment conditional upon research participation, which is a form of coercion. All patient recruitment materials should be made available in clinical study reports and, to the extent possible, in publications. Consideration should be given to measuring and reporting subject expectation of benefit at the time of randomization. Once the job of recruitment is finished, the job of retention begins. Attention should be paid to assisting patients in overcoming the burdens of clinical trial participation, including compensation, transportation support, reminders and communication, and positive feedback (but not by inflating expectation of benefit).

Training participants in clinical trials to perform their roles to specified standards has become “best practice” in clinical trials and is specifically called out in regulatory guidelines addressing quality control issues. [48,70] Participants are rarely told that the success of any clinical trial depends entirely on their performing their duties to a certain quality standard, although ample research demonstrates that this is the case.[31,32,130] At a minimum, participants must be educated about their responsibilities under the protocol, such as adherence to treatment (since adherence to the SCS regimen is necessary for assessment of efficacy), entering data, blinding procedures, use of concomitant and rescue medications, avoiding prohibited treatments, and reporting AEs. In addition, participants should be trained on reporting their symptoms accurately and on appropriate expectations of benefit (often called “placebo response reduction training”). [150]

In addition to standard training (Good Clinical Practice, protocol), investigators should be thoroughly trained on all key study procedures where variability might be problematic and have competence documented; performance should be monitored quantitatively throughout the study. Patient behavior that has an impact on the integrity of study results must be documented. Accurate documentation of adherence to the prescribed SCS regimen might be obtainable from the devices themselves. Reliable documentation of rescue medication adherence requires electronic methods (eg, smart packaging, although this does not document “pill to mouth” but rather “pill to hand”). Pill counts and in-clinic self-report of recalled rescue medication consumption are not accurate measures.[6,8] Daily electronic capture of rescue medication consumption has been increasingly used to capture rescue medication consumption and might be accurate; however, we are unaware of any publications documenting the accuracy of this method.

Patients frequently take pain medications for nonindex pain syndromes, such as headache or sprains. These may be classified as “concomitant analgesics taken for nonindex pain,” and since patients often fail to disclose these (because of poor recall, inconsistent querying methods, or reluctance to report taking prohibited medications), it is important to systematically query patients in a nonjudgmental manner at each clinic visit about any pain medication taken since the last visit and to capture the reason for taking each one. Pain medication taken for a new pain syndrome, or worsening of a stable pain syndrome, would generally be reportable as an AE. Home paper diaries for capturing clinical outcomes assessments (eg, pain) are not reliable [142]; electronic data capture methods are required for assessments that are not completed in clinic.[74,75]

Patients should receive accurate and complete communication about what happens after the completion of the study. Factors to consider are the disposition of the device after the study is complete, coverage of costs during and after the study, and documentation of communication of these issues and other issues of concern to patients about events after study completion.

12. Data analysis, interpretation, and reporting

Although standards for data analysis, interpretation, and reporting have been much discussed in the literature, the review of SCS studies summarized above suggests that several of these standards are frequently not followed. Several salient points will be discussed herein without an attempt to cover all potential issues. One concern is controlling type I error (the probability of a false-positive result). Simply performing multiple comparisons can cause a type 1 error: The more statistical tests are performed, the more likely one will reach the significance threshold by chance and the study be interpreted as “positive.” Sponsors must therefore prespecify a primary endpoint, label other endpoints as secondary, and specify an endpoint model, including details of analysis methods. If multiple primary endpoints are chosen, then methods for handling multiplicity, such as alpha sharing or hierarchical testing procedures, must be prespecified. [155]

Basic data integrity—that the data provided by the patient are the same as that being analyzed in the clinical data set—is a fundamental requirement of research sponsors. Missing data, ie, data not provided by the subject or site, undermine the interpretability of clinical trials, and methods for handling it must be prespecified. The method of imputation depends on choice of estimand—or the population and endpoint that reflects the scientific question of interest,[70] which might differ for different analyses of the same clinical trial data set. For example, if the estimand is the efficacy of a treatment assuming that participants are able to use the treatment, missing data could be imputed using a multiple imputation method that assigns missing outcome values using the characteristics of the participants whose data are missing, the outcomes of the other participants in the group to which the participant was randomized, and the participant's outcome score at the time of discontinuation. If, however, the estimand is the efficacy of the treatment in the overall population, including those who have an adverse reaction to the treatment, missing data could be imputed using a multiple imputation method similar to the one described above if a participant withdrew for any reason other than an AE; but if a participant withdrew due to an AE, then the missing data could be imputed using the data from participants in the control group. The rationale for this method, which is called control-based imputation, is that if a participant cannot use the treatment, their outcomes would still be affected by the nonintervention aspects of the trial (eg, attention, expectation), but they should not be counted as having received benefit of the treatment, even if they did experience some efficacy benefit before they had to discontinue due to an AE. Different methods to handle missing data make different assumptions regarding the patterns of missing data. These assumptions, including missing completely at random, missing at random, or missing not at random, and their implications, are reviewed in depth elsewhere.[107]

Another source of widespread confusion is that of clinical meaningfulness. It is critically important to differentiate WITHIN-PATIENT clinical meaningfulness from BETWEEN-GROUP meaningfulness.[25,30] Within-patient clinical meaningfulness refers to the degree of change in a measure reported by individual patients that will be rated by patients, on average, as meaningful to them. This is useful for defining “responders” to treatment in a clinical trial. The most commonly used cutoff is 30% pain reduction, although arguments can be made for cutoffs as low as 20% or as high as 80%. [41,83] Reporting a cumulative distribution function of response is useful in all studies, since readers can examine the between-group difference at any response cutoff.[41] Note that this is an average—some patients are satisfied with smaller improvements, and others require larger improvements. Within-patient clinical meaningfulness is an entirely different matter than between-group clinical meaningfulness, which refers to the degree of difference in an endpoint between the treatment and control groups that indicates that the treatment provided a clinically meaningful benefit in that study sample.[26,30] There is no consensus on between-group differences that are clinically meaningful for all treatments, nor can there be, since this depends on many factors including risk of treatment, inconvenience, availability of alternatives, etc., and will differ for different audiences. For example, a modest improvement in pain intensity reduction over placebo might be clinically meaningful for a new treatment that is associated with trivial risks; alternatively, a large benefit in pain reduction will be needed for the benefits of a riskier treatment to be considered meaningful.

Interpretation of a clinical study is rarely as straightforward as “positive” or “negative” despite the inordinate attention paid to the P = 0.05 boundary. As famously noted by Jacob Cohen, “the primary product of a research inquiry is 1 or more measures of effect size, not p-values.” [14,143] Study interpretation begins with the effect size of the primary endpoint, then the pattern of effect sizes on secondary endpoints, then their P-values, without losing sight of safety and how it was assessed, with the ultimate goal of understanding the overall risk-benefit balance of treatment compared with control. Confidence intervals can inform interpretation of statistically significant or nonsignificant superiority trials.[53] In a case where the primary endpoint P-value is greater than 0.05, CIs can further determine whether the trial failed to demonstrate superiority (ie, superiority is still possible, but the trial was inconclusive) or the data are consistent with true lack of superiority.

Reporting of clinical trials in general, and trials of SCS in particular, has been problematic, due to inadequate reporting of primary data, failure to adhere to reporting standards, and discrepancies between documents submitted to regulatory authorities, published articles, and marketing materials. General reporting standards are available from the CONSORT group.[105] A pain-specific supplement to CONSORT is also available from Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION),[52] as well as a discussion of issues that arise in the reporting of crossover trials.[54] A guideline for describing complex interventions in clinical trial reports has been published.[66] Additional recommendations for the reporting of SCS trials for pain are provided in Tables 3 and 4.

Table 4

Recommendations for randomized controlled trials of spinal cord stimulation for chronic pain: study design and conduct.*

Issue	Recommendation	References
Overall design
Mechanistic studies	Study mechanisms of action and biomarkers in concert with clinical outcomes whenever possible	11,93,96
Noninferiority designs	Noninferiority designs, if used at all, should incorporate internal demonstration of assay sensitivity (eg, a successful superiority test to placebo control)	47
Blinding	Double-blind whenever possible; claims that blinding was not possible require explanation	43,133
	For partially blinded studies (eg, unblinded implanter and blinded assessor), document site-specific blinding plans, and compliance
	Assess patients for success of blinding; consider assessing research staff
Trial stimulation	Trial methods should be prespecified, standardized, and reported	35
	Account for potential biases based on type of trial stimulation
	Be clear on whether trials are performed prerandomization or postrandomization and account for it in the choice of estimand and analysis plan
Minimum duration of treatment	12 weeks to draw conclusions about efficacy for long-term use; 12-24 months is preferable	31
	At least 1 year to draw conclusions about long-term safety
	Shorter studies might be appropriate for initial comparison of waveforms or for conclusions about temporary use
Recruitment and retention
Burden of participation	Assist patients in overcoming the burden of participation by providing adequate compensation; transportation, food, and lodging; rescue medication; and communication with family and external caregivers and by minimizing unnecessary procedures	7
Site selection
No. of sites	Keep to a minimum	105
	Invest in recruitment methods to maximize the number of patients per site
Investigator experience	Document investigator expertise with the investigational procedure, comparator procedures, managing patients with chronic pain, and performing any special outcome measurement procedures; specify standards for investigator and site selection	46,64
Comparators
Placebo/sham	Use placebo/sham comparator whenever possible; otherwise, a finding of similar effectiveness of 2 active treatments cannot be readily interpreted	43,47
Study population
Diagnosis	Use clear diagnostic criteria; train investigators on diagnostic assessment; and consider central verification	85,97
Minimum pain intensity	Minimum baseline pain intensity of 4 or 5 (on a 0-10 rating scale) documented over a sufficient period to ensure stability (minimum 1 week)	31,32
	Implement procedures to address baseline score inflation
Failure of previous treatments	Failure of previous treatments before SCS should be consistent with clinical practice guidelines and the study objectives and documented	18
Exclude substance abuse	Use validated screening questionnaires and urine drug screen	32
Neuropathic pain component	Document the extent of neuropathic pain as appropriate using validated tools	98
Psychological comorbidity	Exclude patients with significant levels of psychological comorbidities using validated tools (eg, PHQ-9, GAD-7, Beck Depression Inventory, and Hospital Anxiety and Depression Scale)	31,32,156
Minimum pain duration	One year of moderate to severe pain	31,32
Patient management
Concomitant treatments	Keep concomitant treatments constant or, if flexibility is required, indicate how changes are recorded and report them	32
Ancillary procedures and interactions	Specify handling of all procedures and interactions, how they will be balanced by treatment arm, measured, and reported	15
Programming
Programming	Programming prescription (usage, programming parameters), adherence to programming prescription (usage, programming parameters, in alignment with subject reported outcomes), programming session (frequency, duration), and programming personnel and oversight	6,15,49
	Programming training and policy on qualifications of study personnel performing programming
	Industry should provide “best practice” algorithms as available
	Industry personnel should not generally provide programming; justify exceptions and how oversight is provided
	Report number of programming sessions, mean (range) time, personnel involved, and SCS parameters (eg, amplitude, pulse width, frequency [mean/ranges], and outcome of each session). Device usage and programming parameters used by patients that directly link to data collection interval.
	Compliance/adherence and persistence measure are required to evaluate SCS therapy.
Bias control
Expectation bias	Patients should receive written, verbal, and online information that supports equipoise; all information should be made available	76,163 (and Erpelding N, unpublished, 2020)
	Neutral expectation training for staff and patients
Training
AE reporting	Train investigators and coordinators on a standardized method for capturing AEs; monitor consistency and completeness of AE reporting using risk-based monitoring techniques	31,48
Expectation	Document efforts to achieve neutral expectation on the part of researchers and subjectsMeasure expectation at baseline and endpoint	3 (and Erpelding N, unpublished, 2020)
Procedures	Document training and competency on test and comparator procedures and ancillary care	46
Accurate pain reporting	Train staff and subjects on accurate symptom reporting including pain intensity and location	32,140,150
Subject responsibilities	Train subjects on their key responsibilities, including use of the SCS device, medication adherence, physical and psychological modalities, prohibited treatments, adverse event reporting, retention, and address the burden of research participation	(Erpelding N, unpublished, 2020)
Quality control
Risk-based monitoring	Comprehensive risk-based quality management	48 (and Katz N, unpublished, 2020)
Administrative
Funding/conflicts	Disclose funding source and conflicts of interest	45,71

Issues of heightened importance for SCS studies, beyond clinical trials in general, are bolded.

SCS, spinal cord stimulation.

A final consideration is what claims should be possible from a given study; although the term “claims” is usually invoked in the regulatory context with respect to product labeling, the same concept applies in the clinical context with respect to what recommendations for use of the product are justified based on the study results. For example, a specific study might justify a “claim” for temporary use in the treatment of chronic pain or long-term use depending on study design features. Studies should be designed to anticipate an end result that would indicate what treatment recommendations would be appropriate based on potential study results.

13. Economic outcomes and cost-effectiveness analyses

Many SCS trials are designed to characterize primary efficacy and safety, frequently for regulatory approval. If payers do not pay for the treatment, however, the studies designed for regulatory approval become pointless, which exposes human subjects to risks of no social benefit.

Types of data of interest to payers vary by region. For example, in the United States, the average patient changes health coverage every 3 years; thus, health economics measures that involve break-even points beyond 3 years might not be attractive to third-party payers. Depending on the economic question of interest, endpoints can be added to primary efficacy/safety studies to provide further insight. For questions of interest requiring long-term observation or access to claims data, standard RCTs might need to be augmented by studies designed specifically to address those issues.

Most studies primarily focused on establishing safety and efficacy are relatively short term, with a median duration of postrandomization follow-up of 12 weeks, as noted above. Health economics endpoints that can be incorporated into such trials include work-related endpoints (eg, absenteeism, presenteeism, and productivity), health care use and associated costs, and health-related quality of life endpoints (using, eg, the EQ5-D or the SF-6D), which can be converted into quality-adjusted life years (QALYs). The most common health economic construct of interest is cost-effectiveness, which is typically measured by difference in costs and outcomes between the intervention and comparator. Cost-utility studies have also been conducted in SCS and report results in terms of QALYs. Endpoints with economic implications that might require dedicated, often postmarketing studies, include long-term safety, incidence of rare AEs, long-term economic impacts, and those that can be illuminated with claims data, such as costs of care not readily captured in an efficacy/safety RCT (emergency department visits, comprehensive medication costs, behavioral health costs, hospitalizations, etc.).

Despite the studies demonstrating cost-effectiveness,[10,87,114,139,144] with the general rise in health care costs, the reimbursement climate for SCS has become increasingly challenging. Payers have observed that most studies designed to support regulatory approval are NI in design, without placebo controls, and without cost-effectiveness data, and accordingly question the value of such studies. Available cost-effectiveness analyses, although performed rigorously and yielding supportive results, are dependent on data from RCTs of SCS that, as described above, are typically prone to bias; however, the National Institute of Health and Care Excellence in 2008 recommended SCS for patients with neuropathic pain based on modelling of an acceptable cost per QALY (<£20,000/QALY).[108] Going forward, to be persuasive to payers, cost-effectiveness studies need to use data based on well-designed RCTs that have appropriately controlled for bias, with relevant outcome assessments and sufficient duration of follow-up. Thus, sponsors should consider performing trials that are not only sufficiently rigorous to convince regulators for approval but also can robustly demonstrate to payers the value of SCS, using the recommendations provided in this article.

14. Special issues in randomized controlled trials of spinal cord stimulation, including sham stimulation and programming comparisons

The need for a trained individual to perform skillful programming of the device on a repeated basis to maximize the benefit of treatment creates unique challenges in SCS studies. Programming in trials is usually delegated to industry representatives operating in isolation from clinical staff, which creates the opportunity for inconsistent interactions and bias, since attention, number of visits, time spent during visits, and enthusiasm of health care providers are all likely to increase placebo responses.[158] Thus, if an RCT of SCS in which the number, length, and content of programming interactions is not standardized and balanced between groups demonstrates superiority of one group over another, the superiority cannot necessarily be attributed to the SCS treatment. Information about programming in SCS study reports is often absent, scant, or confusing, making it difficult for the reader to evaluate potential bias.

We recommend that a standardized approach to programming be devised and presented in the Methods section of reports and publications.[15,90] Standardized approaches will need to allow sufficient flexibility to meet individual patients' needs. Interactions should be scripted and monitored. Industry should provide best practice programming approaches to inform the protocol. Ideally, industry personnel should not perform programming; exceptions should be justified and monitored. Standards for the qualifications of individuals at the sites who perform programming should be established and documented. Outcomes of programming, including programming results (eg, mean, ranges, and mode values for the programming parameters), whether scheduled or nonscheduled, duration of sessions, who was involved, any deviations from the initial planned programming algorithm, and costs (for studies including economic data) should be included in the results. Reprogramming as a treatment for an AE should be captured formally as an AE.

Double-blinding can be challenging in SCS studies, especially when comparing low-frequency stimulation with either placebo or non–paresthesia-based stimulation. Investigators have attempted to improve the integrity of blinding by delivering ultra-short-duration paresthesia or subthreshold paresthesia stimulation to patients in the control group, but it has been argued that these tactics might have a neuromodulatory effect and, therefore, might not act as pure placebo but rather a low dose of treatment[166] (It may equally be argued, however, that if ultra-short stimulation (minutes/day) has a neuromodulatory effect equivalent to traditional SCS, then it should be used in preference to traditional stimulation given the potential savings in battery life.). Another potential source of unblinding is the patients' interaction with their handheld SCS controller, due to subjects accessing information about their stimulus intensity, program parameters, response from their SCS IPG, or the charge status of their IPG, especially in crossover designs: A sudden drop (or increase) in recharging requirements might lead subjects to guess that they have entered the placebo (or active) treatment period. Some investigators have programmed IPGs to leak current so as to require recharging at similar intervals to the active study arm.[1,121] Investigators, therefore, need to specify the type of IPG used in a study (rechargeable vs nonrechargeable) as well as any steps taken to control-related sources of unblinding.

The complexity of these blinding-related issues makes it impossible, and undesirable, to prescribe one-size-fits-all approaches. However, investigators should perform double-blind studies whenever possible (exceptions should be strenuously justified and the lower level of evidence of such studies acknowledged); use blinded assessors in any studies that are not fully blinded; explain in the Methods section how potential threats to blinding were handled; and perform and report assessments of patient expectation, effectiveness of blinding, or the equivalent. Claims in study reports that double-blinding was not possible are offset by reports of success with double-blinding in single-center and multicenter studies of SCS and similar devices.[15,24] A statistically significant effect demonstrated in an unblinded study failed to be replicated when the same treatments were compared under blinded conditions,[15,78] further supporting the importance of maximizing blinding in SCS trials.

A critical research question is whether one waveform is more effective than another. Since in theory any device can deliver any waveform, studies designed to compare waveforms should be possible with a single device. Yet, with a few recent exceptions,[146] some manufacturers have been reluctant to allow their devices to be programmed for this purpose for proprietary reasons. Using different devices to compare different waveforms might introduce confounding and increase patient risk. Similar issues apply to programming current leaks to maintain blinding. The ethics of manufacturers' positions on features of their devices requires careful consideration to enhance technical programming sharing with investigators and achieve full transparency.

15. Conclusions and recommendations

The last half century has been characterized by important advances in device technology and the clinical research methods used to study them. Yet, the evidence base for the efficacy and safety of SCS for pain still consists largely of clinical trials that, even when RCTs, are sufficiently lacking in rigor that conclusions about the efficacy of SCS for chronic pain, or the superiority of one waveform or device over another, while firmly believed by many clinicians, remains below current evidentiary standards. The main inadequacies of SCS RCTs conducted to date include the use of NI designs without internal demonstrations of assay sensitivity; failure to control and transparently report how important sources of bias, such as double-blinding, imbalances between groups in interactions with patients, programming, and the sources of information that lead to unbalanced expectation were addressed; and unclear or inconsistent safety reporting. Our hope is that the recommendations contained herein will lead to the generation of higher quality data to support decisions of patients, clinicians, regulators, and reimbursement authorities on the utility of SCS in treatment. Based on the review presented, we recommend that the features listed in Tables 3 and 4 and the considerations for management of bias and measurement error presented in Table 2 be considered in the design of all future studies evaluating the efficacy and effectiveness of SCS for chronic pain.

Conflict of interest statement

The views expressed in this article are those of the authors, none of whom have financial conflicts of interest specifically related to the issues discussed in this article. At the time of the meeting on which this article is based, several authors were employed by medical device companies and others had received consulting fees or honoraria from 1 or more pharmaceutical or device companies. The authors of this article who were not employed by industry or government at the time of the meeting received travel stipends, hotel accommodations, and meals during the meeting provided by funds from the Analgesic, Anesthetic, and ACTTION public–private partnership with the US FDA, the Institute of Neuromodulation (IoN), and the INS, which have received research contracts, grants, or other revenue from the FDA, multiple pharmaceutical and device companies, philanthropy, and other sources. Preparation of background literature reviews and draft manuscripts was also supported by ACTTION. No official endorsement by the FDA, US National Institutes of Health, or the pharmaceutical and device companies that have provided unrestricted grants to support the activities of ACTTION, IoN, and INS should be inferred. E. Buchser has received consulting fees from Medtronic, and his department has received research funding from Medtronic. R.H. Dworkin has received in the past 5 years' research grants and contracts from the US Food and Drug Administration and the US National Institutes of Health, and compensation for serving on advisory boards or consulting on clinical trial methods from Abide, Acadia, Adynxx, Analgesic Solutions, Aptinyx, Aquinox, Asahi Kasei, Astellas, AstraZeneca, Biogen, Biohaven, Boston Scientific, Braeburn, Celgene, Centrexion, Chromocell, Clexio, Concert, Coronado, Daiichi Sankyo, Decibel, Dong-A, Editas, Eli Lilly, Eupraxia, Glenmark, Grace, Hope, Hydra, Immune, Johnson & Johnson, Lotus Clinical Research, Mainstay, Medavante, Merck, Neumentum, Neurana, NeuroBo, Novaremed, Novartis, NSGene, Olatec, Periphagen, Pfizer, Phosphagenics, Quark, Reckitt Benckiser, Regenacy (also equity), Relmada, Sanifit, Scilex, Semnur, Sollis, Spinifex, Syntrix, Teva, Thar, Theranexus, Trevena, Vertex, and Vizuri. S. Eldabe has received consulting fees from Medtronic, Saluda Medical, and Mainstay Medical. His department has received research funding from Medtronic and Nevro. G. Fiore is an employee of Fiore Healthcare Advisors which provides services to sponsors of clinical trials and manufacturers of commercial products. Fiore Healthcare Advisors receives consulting fees from Sollis Therapeutics. J. Gewandter has received consulting income in the past 36 months from MundiPharma, Disarm Therapeutics, Asahi Kasei Pharma, SK Life Science, Orthogonal, and Science Branding. S.M. Hayek has received in the past 36 months' honoraria for consulting from Biotronik and Boston Scientific. N. Katz is an employee of WCG Analgesic Solutions, which provides services to sponsors of clinical trials. WCG Analgesic Solutions has received consulting fees from Mainstay Medical and Boston Scientific. A. Leitner is an employee of Saluda Medical. E. McNicol has nothing to declare. R.B. North serves as an unpaid officer of (1) the charitable nonprofit Institute of Neuromodulation, which in turn receives support from the nonprofit North American Neuromodulation Society, and (2) the charitable nonprofit Neuromodulation Foundation, Inc, to which (like his former employer Johns Hopkins University) grants and support have been provided by Abbott, Boston Scientific Corp, Medtronic, Inc, Nevro Corp, Nuvectra, and Stimwave, Inc. He receives royalties from Abbott, royalties and consulting fees from Nuvectra, and consulting fees from Stimwave, Inc, in which his wife holds shares. T. Nurmikko has nothing to declare. A.R. Rezai: Board of Directors and potential equity position in Autonomic Technologies (ATI), Neurotechnology Innovation Management (NIM), and Sollis Therapeutics. R. Taylor is a paid consultant for Medtronic, Nevro, and Saluda. S. Thomson has received honoraria for work as a consultant for British Standards of Industry, Boston Scientific Neuromodulation, and Mainstay Medical. D. Turk has received research grants and contracts from the US Food and Drug Administration and US National Institutes of Health and compensation for consulting on clinical trial and patient preferences from AcelRx, Eli Lilly, Flexion, GlaxoSmithKline, Pfizer, and Swing Therapeutics. J. Shipley's employer, The Neuromodulation Foundation, a charitable, nonprofit corporation, has received grants and/or consulting income from Abbott (formerly St. Jude), Boston Scientific, Greatbatch, Medtronic, Nevro, Nuvectra, and Stimwave.

Supplemental digital content associated with this article can be found online at http://links.lww.com/PAIN/B272.