Literature DB >> 32187319

Precision non-implantable neuromodulation therapies: a perspective for the depressed brain.

Lucas Borrione1, Helena Bellini1, Lais Boralli Razza1, Ana G Avila2, Chris Baeken3,4,5,6, Anna-Katharine Brem7,8, Geraldo Busatto9, Andre F Carvalho10,11, Adam Chekroud12,13, Zafiris J Daskalakis10,11, Zhi-De Deng14,15, Jonathan Downar16,17, Wagner Gattaz18,19, Colleen Loo20, Paulo A Lotufo21, Maria da Graça M Martin22, Shawn M McClintock23, Jacinta O'Shea24, Frank Padberg25, Ives C Passos26, Giovanni A Salum27, Marie-Anne Vanderhasselt3,5,28, Renerio Fraguas9,29, Isabela Benseñor21, Leandro Valiengo1, Andre R Brunoni1,18,19,29.   

Abstract

Current first-line treatments for major depressive disorder (MDD) include pharmacotherapy and cognitive-behavioral therapy. However, one-third of depressed patients do not achieve remission after multiple medication trials, and psychotherapy can be costly and time-consuming. Although non-implantable neuromodulation (NIN) techniques such as transcranial magnetic stimulation, transcranial direct current stimulation, electroconvulsive therapy, and magnetic seizure therapy are gaining momentum for treating MDD, the efficacy of non-convulsive techniques is still modest, whereas use of convulsive modalities is limited by their cognitive side effects. In this context, we propose that NIN techniques could benefit from a precision-oriented approach. In this review, we discuss the challenges and opportunities in implementing such a framework, focusing on enhancing NIN effects via a combination of individualized cognitive interventions, using closed-loop approaches, identifying multimodal biomarkers, using computer electric field modeling to guide targeting and quantify dosage, and using machine learning algorithms to integrate data collected at multiple biological levels and identify clinical responders. Though promising, this framework is currently limited, as previous studies have employed small samples and did not sufficiently explore pathophysiological mechanisms associated with NIN response and side effects. Moreover, cost-effectiveness analyses have not been performed. Nevertheless, further advancements in clinical trials of NIN could shift the field toward a more "precision-oriented" practice.

Entities:  

Mesh:

Year:  2020        PMID: 32187319      PMCID: PMC7430385          DOI: 10.1590/1516-4446-2019-0741

Source DB:  PubMed          Journal:  Braz J Psychiatry        ISSN: 1516-4446            Impact factor:   2.697


Introduction

Current psychiatric guidelines recommend several antidepressants and cognitive-behavioral psychotherapy (CBT) as first-line treatments for major depressive disorder (MDD).1,2 However, more than one-third of depressed patients will not achieve remission even after four adequate medical prescriptions of antidepressant drugs.3 Moreover, despite advances in psychopharmacology, even new medications can still produce several adverse effects that reduce tolerability and increase risk.4 Psychotherapy, in turn, is costly, time-consuming, does not suit all patients, and is not readily available in remote areas of the world.5 One possible explanation for the limited efficacy of mainstream antidepressant treatments is that they are typically applied in a “one-size-fits-all” and trial-and-error paradigm, with little biological guidance – i.e., information is mostly observational, with almost complete disregard for the specific neurobiological mechanisms underpinning the corresponding depressive symptomatology. To address this significant limitation in personalizing antidepressant treatments, a new field of “precision psychiatry” has been proposed, which aims to tailor medical treatment to the characteristics of each patient.6 Although this concept is not necessarily new (e.g., blood transfusion is “guided” by blood type examination), three new emerging tools6-8 are involved in the precision psychiatry framework: 1) incorporating the biological pathways of disease – in psychiatry, this is represented by the National Institute of Mental Health (NIMH) Research Domain Criteria (RDoc), a framework that evaluates mental illness at multiple clinical, endophenotypic, and neurobiological levels9; 2) multimodal big data collection, i.e., acquisition of clinical and biological data at scale, as exemplified by the opportunities presented by international consortiums such as the Enhancing NeuroImaging Genetics through Meta-Analysis (ENIGMA)10 and mega-cohorts such as the UK Biobank11; and 3) artificial intelligence for analysis of multidimensional and complex patterns in manifold data collected at multiple biological levels.12,13 Although precision psychiatry is still in its infancy, the continuous, rapid development of these tools will reshape clinical and research practice, enhancing treatment and minimizing adverse effects.6 Non-implantable neuromodulation (NIN) interventions, such as transcranial magnetic stimulation (TMS), transcranial direct current stimulation (tDCS), electroconvulsive therapy (ECT), and magnetic seizure therapy (MST), are non-pharmacological, non-psychotherapeutic interventions with distinct efficacy, safety, tolerability, and availability profiles.14-16 These techniques have been developed over multiple decades to bridge the efficacy and safety gaps of traditional antidepressant treatments, with concrete results.17 Nevertheless, major caveats remain, such as limited efficacy and significant adverse effects. In this context, the development of a “precision NIN” approach could both enhance clinical usability of NIN techniques (by improving efficacy and/or maximizing tolerability) and unveil their neurobiological mechanisms of action, which to date remain poorly understood. Additional challenges for precision NIN are the ability to combine them with other interventions18 and their spatiotemporal resolution, as the effects of NIN can be enhanced or decreased according to the site of application of the coils or electrodes and their synchronization with ongoing neuronal activity.19 Knowledge acquired from computer modeling and functional neuroimaging can be directed toward this purpose (Figure 1).
Figure 1

Precision non-implantable neuromodulation (NIN). In a precision NIN framework, advancements in related areas of research and knowledge directly influence treatment protocols (parameters such as stimulation site, timing, and dose, as well as combined behavioral/pharmacological interventions), aiming to increase individual antidepressant response.

In this review, we present the concept of precision NIN as applied to antidepressant treatment. This framework would also be useful for other neuropsychiatric disorders. We first provide an overview of the state of the art and of the main NIN antidepressant modalities, and then present challenges and recent developments and opportunities of using NIN in the framework of precision psychiatry.

Methods

We convened a group of national and global leaders on the topics addressed in this review, such as MDD, neuroimaging, noninvasive brain stimulation, machine learning, neuropsychology, and precision psychiatry. These authors were invited to address specific parts of the manuscript, as well as to review its content as a whole. The PubMed, Google Scholar, and Web of Knowledge databases were searched from inception up to August 2019. Preference was given to recent comprehensive reviews, meta-analyses, pivotal randomized clinical trials that concerned NIN in the treatment of MDD, and highly cited articles in the field, with a view to offering an up-to-date and comprehensive perspective from experts. We focused our review on clinical articles that investigated MDD.

Non-implantable neuromodulation

Introduction and mechanisms of action

NIN techniques use electrical or magnetic energy targeted at the brain20 (Figure 2). They do not require surgery, are less invasive, and involve less risk than implantable techniques, such as deep brain stimulation and vagus nerve stimulation.22 They can be categorized into subconvulsive and convulsive modalities, the former also often described as “non-invasive brain stimulation” (NIBS), which include TMS and tDCS,15 while the convulsive modalities include ECT and MST.23 Compared to the convulsive modalities, NIBS does not require sedation or anesthesia, and presents excellent safety and tolerability profiles.24-28
Figure 2

Examples of NIN techniques (top panel) and the corresponding electric field distribution in the brain (bottom panel): A) tDCS using 5 × 5 cm electrodes placed over the bilateral DLPFC; electrodes are colored red and blue to distinguish anode (red) vs cathode (blue). B) TMS using the MagVenture B70 coil over the left DLPFC. C) Right unilateral ECT; conventional ECT uses a bipolar waveform and therefore does not distinguish between anodal vs. cathodal electrodes. Electric field strengths are normalized to their respective maximum value (Emax); absolute field strengths are very different across the modalities (< 1 V/m for tDCS to > 100 V/m for TMS and ECT). Figure produced using SimNIBS software.21 DLPFC = dorsolateral prefrontal cortex; ECT = electroconvulsive therapy; NIN = non-implantable neuromodulation; tDCS = transcranial direct current stimulation; TMS = transcranial magnetic stimulation.

The neurobiological rationale supporting NIBS in depression is based on specific alterations of neurocircuitry function, which can be normalized by targeted stimulation of areas such as the dorsolateral prefrontal cortex (DLPFC) – the cortical area most commonly targeted, from early pilot studies to more recent, pivotal NIBS trials.29-33 Besides the practical convenience of safely targeting this area, the DLPFC is a key hub of the frontoparietal network (FPN), which has been implicated in the regulation of several processes, including decision-making, working memory, and attention, and is impaired in depression,34 particularly the left DLPFC.35 Hypoactivity of the FPN is associated with hyperactivity of the default mode network (DMN), which may promote depressive behaviors and cognitions such as negative bias, self-referential processing, and depressive rumination.36 The DLPFC is also a key node of the salience network (SN), which plays a key role in cognitive control (i.e., the self-regulation of thought, emotion, and behavior).37 Deficiencies in cognitive control and SN function and structure appear not only in depression, but transdiagnostically across a variety of Axis I disorders,38 suggesting that the mechanism of action of DLPFC-NIBS may be pertinent not only to depression, but to other forms of mental illness in which cognitive control is impaired. NIBS to the DLPFC is thought to modulate the activity of this brain area, thus promoting an increase in FPN activity and a concomitant downregulation of DMN activity, leading to the improvement of depressive symptoms.39 Notably, anodal tDCS and high-frequency rTMS usually increase cortical excitability, although the net effect is also influenced by the underlying cortical activity.15 This rationale has been supported to some extent by neuroimaging studies in depressed patients receiving rTMS21,40 and by recent validation studies.41 In turn, ECT and MST – both of which are performed in a controlled environment, under general anesthesia – induce seizures via depolarization of neuronal networks.42 The mechanisms underlying the striking antidepressant effects associated with convulsive therapies remain poorly elucidated,43 and may include increased brain-derived neurotrophic factor (BDNF) levels,44 hippocampus and amygdala volumes,45,46 and hippocampal functional connectivity.47 Nonetheless, it is unclear whether the increase in hippocampal volume, a well-documented effect,48 is an epiphenomenon or a necessary mechanism underpinning therapeutic ECT effects, as depression improvement is unrelated49 or even negatively associated48 with this outcome. The inflammatory theory associated with the convulsive NIN modalities is promising, as inflammatory cytokines decrease after ECT in depressed patients.50,51 As inflammation triggers the kynurenine pathway, leading to oxidative stress and serotonin depletion,52 rapid reduction of inflammation could mitigate depressive symptoms.53

Transcranial magnetic stimulation

TMS is based on the principle of electromagnetic induction via an electric alternating current passing through a coil. The magnetic field, which varies over time (1.0-2.5 Tesla), induces a secondary electric field, eliciting action potentials when targeting the underlying cortex.54 The original magnetic field passes through several layers (scalp, bone, meninges, etc.), with no resistance and deflection, inducing a relatively focal field.54 TMS can be delivered in various modalities – namely, high-frequency (HF, 5-20 Hz), low-frequency (LF, ≤ 1 Hz), theta-burst (TBS), and deep TMS (dTMS).21,55 Since the seminal study by Pascual-Leone et al.32 which showed efficacy of HF-rTMS over the left DLPFC, many trials have been performed. Of note, O’Reardon et al. randomized 301 patients with MDD without concomitant antidepressant use to receive either sham or active HF-rTMS over the left DLPFC, and showed superior improvement in depression in the active group.30,56 This study provided pivotal data for FDA clearance of rTMS as antidepressant therapy. The effectiveness of HF-rTMS was further confirmed in two subsequent meta-analyses, with positive results both in accelerating clinical response to antidepressants57 and as monotherapy in unipolar and bipolar depression.58 Low-frequency rTMS (LF-rTMS) over the right DLPFC has demonstrated effectiveness in the treatment of MDD,59 with both HF and LF being regarded as first-line protocols.21 LF-rTMS can be especially advantageous when there is a high risk of seizures, poor tolerability to pain, or when the patient does not respond to HF-rTMS.55 Intermittent TBS, on the other hand, provides a new avenue for busy clinical services through exploration of more time-efficient rTMS protocols. A recent multicenter non-inferiority randomized clinical trial showed that intermittent theta-burst stimulation (iTBS), which lasts only 3 minutes, was non-inferior to 37.5 minutes of treatment with HF-rTMS. Both protocols were applied over the left DLPFC.29 Finally, deep rTMS with an H-shaped coil has also demonstrated clinical effectiveness in MDD; this type of coil allows for deeper penetration of the magnetic field into the brain.55 Network meta-analyses have further demonstrated active rTMS to be superior to sham, albeit inferior to ECT.17,60 In these studies, it was found that priming, HF-rTMS, LF-rTMS, bilateral rTMS, and TBS were more effective than placebo, although no active intervention was superior to any other. Based on these results, rTMS is considered a first-line treatment for patients who have failed at least one trial with an antidepressant medication.21,61 On the other hand, the level of treatment resistance is known to be an important negative predictor of response to rTMS.62 Less is known regarding rTMS efficacy as a maintenance treatment in depression. In patients who have responded favorably to an acute course of rTMS, naturalistic studies have found that relapse is more common without any maintenance antidepressant strategy,63,64 while a meta-analysis evaluating durability of HF-rTMS showed that the antidepressant effects are small following a shorter acute treatment.65 Current recommendations have not reached a clear consensus regarding the most effective antidepressant maintenance protocol.21,61 Side effects of rTMS include the possible occurrence of mild headache and pain at the site of stimulation, although these symptoms typically resolve spontaneously and the treatment is well tolerated overall.55 The most serious side effect of rTMS is triggering of epileptic seizures, although this phenomenon is rare in clinical practice. In fact, seizures have been found to be extremely rare, and mainly occurred when rTMS protocols exceeded safety guidelines.24 Animal studies suggest that even at high intensities and prolonged exposure durations, there is little likelihood of damage to brain structures.66,67 Finally, the only absolute contraindication to rTMS is the presence of metallic and electronic material, such as cochlear implants, in close contact with the coil.55

Transcranial direct current stimulation

For tDCS, an electric current of low intensity (usually 1.0-2.5 mA) is applied to the brain, via two electrodes placed over the scalp (anode and cathode), which is the most common protocol.68 The current passes through the skin, subcutaneous tissue, skull, and cerebrospinal fluid (CSF), and into the gray and white matter. Due to the impedance of the skull, only a fraction of the injected current reaches the brain.69 In addition, as the conventional sponge-electrode set is large (25 to 35 cm2) and the electrodes are placed relatively far apart on the head, the induced field is non-focal as the current flows from the anode to the cathode.28 The injected electrical current does not generate action potentials per se, but rather facilitates or inhibits synaptic transmission, respectively, via an increase or decrease in the frequency of action potentials in endogenous neuronal firing.69 For depression, tDCS montages employ anodal stimulation over the left DLPFC (with contralateral, variable cathode sites), thus aiming to counterbalance the hypoactivity of this brain area and subsequent hyperactivity of the DMN.70 tDCS is considered a safe and well tolerated technique, especially since the standard range of current intensities used does not induce brain injury.25,26,28 The most common side effects include itching and tingling at the scalp application sites.15 Skin burns are uncommon, and the risk can be further reduced with proper soaking of the electrodes, customized sponges, and adequate use of saline solution.71 Due to its portability and ease of use, tDCS has been investigated as an augmentative and substitute treatment for antidepressant medications. In a factorial study design, Brunoni et al.72 randomized 120 antidepressant-free depressed patients to receive placebo, sertraline only, tDCS only, or combined treatment with the two.72 The main study finding was that the combined treatment led to a faster and greater response compared to the other treatments. Subsequently, the same group31 designed a non-inferiority, sham-controlled design to compare tDCS vs. full-dose escitalopram. The study failed to show non-inferiority of tDCS vs. escitalopram, although superiority analyses revealed that tDCS was more effective than placebo. Accordingly, recent meta-analyses have shown that tDCS is superior to placebo for response, remission, and depression improvement outcomes.73,74 Only three studies have investigated continuation of tDCS sessions after the acute treatment phase.75-77 All showed 6-month relapse rates varying from 25-50%. Interestingly, the study that reported the lowest relapse rate had tDCS performed twice a week,76 whereas the one reporting higher relapse rates performed tDCS every other week.75 Taken together, this suggests an intensive tDCS treatment regimen is associated with lower relapse rates, although these studies were limited by small sample sizes and short follow-up periods. While future randomized clinical trials involving tDCS in the treatment of depression should continue to investigate maintenance phase protocols, it would also be interesting to individualize the delivered dose using computer models, while evaluating the feasibility and safety of home-based sessions.70,78 Furthermore, electrical stimulation with different wave formats, such as transcranial alternating current stimulation (tACS)79 or transcranial random noise stimulation (tRNS),80 could be used to target MDD-related oscillatory brain activity in the DLPFC, possibly in combination with individualized neurofeedback strategies.81

Convulsive modalities

ECT delivers a stimulus of alternating polarity pulses, with an amplitude of 800-900 mA, via two electrodes placed on the scalp.82 Although the procedure is considered a second-line treatment for MDD due to the risk of cognitive side effects, it is regarded as a first-line treatment in some cases (e.g., MDD with acute suicidal or psychotic features).21,83 ECT is more effective than sham, antidepressant medications and psychotherapy, and rTMS,84 achieving very high response and remission rates.21 Historically, bitemporal (or bifrontotemporal) electrode positioning has been used, although right unilateral (RUL) placement has gained currency as a modality with relatively fewer cognitive side effects.82 In fact, there is a complex relationship between ECT “dose” (total charge delivered – a composite measure of current pulse amplitude, pulse width, frequency, and number of pulses – indexed by the seizure threshold, ST) and electrode placement as a determinant of cognitive and antidepressant outcomes, which can be partly explained by electric field distribution.82 In the past, there was controversy as to the comparative efficacy of RUL vs. bitemporal ECT, the latter being considered more effective. However, recent meta-analyses have found that high-dose RUL and bitemporal ECT are equally effective,17,85 and considered that previous RUL ECT trials that used lower doses (e.g., less than six times the seizure threshold) might have underestimated its treatment effects. More recently, bifrontal ECT has been introduced as a form of ECT with efficacy comparable to that of bitemporal ECT, but fewer side effects.86 Maintenance-phase ECT (weekly, biweekly, or monthly sessions) is also considered effective and tolerable, with increased efficacy in association with medications, especially the combinations of nortriptyline and lithium87 and venlafaxine and lithium.88 Despite its effectiveness, ECT is limited due to the need for infrastructure (general anesthesia, trained personnel, and clinical evaluation),83 social stigma,89 and potential cognitive side effects, including postictal disorientation, anterograde amnesia, retrograde amnesia, and impairments in multiple other cognitive domains, including verbal fluency and executive function.21 The persistence, severity, and characterization of cognitive impairment remain topics of great debate in the current ECT literature, even after decades of research.90 Nevertheless, most acute ECT-related adverse events – whether cognitive or non-cognitive – are mild, transient, and self-limited; more severe cardiovascular and neurological complications are rare, and can be managed through prophylactic and therapeutic measures.91 MST is a TMS variant that passes through the skull unimpeded and results in a more focused superficial electric field, concentrated in the cerebral cortex; hence, there is minimal stimulation of inner brain structures, such as the hippocampus.92 It delivers 25-100 Hz pulses for up to 10 seconds to trigger generalized seizures.23 Potentially, the more focal and limited electric field induced by MST could be associated with a lower incidence of cognitive side effects compared to ECT.93 Nevertheless, MST seizures show less robust ictal expression, postictal suppression, and generalization to the hippocampus compared to ECT.94 To date, studies that have compared MST and ECT found promising results, with MST having antidepressant effects comparable to those of RUL ECT and no cognitive side effects.23,95,96

Challenges and opportunities for precision non-implantable neuromodulation

NIBS methods are not yet mainstream treatments for depression. On the one hand, these techniques excel in safety and tolerability; on the other, they have modest antidepressant effects, are associated with variable costs, and not widely available.15,17,97 ECT, although highly effective, is limited by cognitive side effects and social stigma, while MST is currently experimental. To promote NIN applicability, we discuss the challenges and opportunities for increasing NIBS clinical effectiveness and decreasing ECT- and MST-related side effects, in a precision-oriented framework and in light of different forms of “target engagement” (a target being either a mechanism related to the disease or to the mode of action of the intervention itself).98

The “how”: combining non-invasive brain stimulation with cognitive interventions

Evidence has demonstrated progressively that the neurobiological, behavioral, and antidepressant effects of NIBS are dependent on the “state” of the targeted neural area at the time of stimulation.18 For example, in depression, patients exhibiting higher rostral anterior cingulate cortex (ACC) activity prior to stimulation showed a better antidepressant response to subsequent rTMS.99 This raises the prospect of experimentally controlled “pre-shaping” of brain states, induced by cognitive tasks and/or NIBS techniques, to more effectively target stimulation to redress neurobiological imbalances in depression. For instance, hyperconnectivity between the ACC and the medial prefrontal cortex (mPFC) has been linked to maladaptive depressive ruminations, and both CBT and rTMS, each alone, have been shown to downregulate this dysfunctional brain activity.18 Could pairing these interventions yield synergistic effects? In an interesting pilot study, concurrent rTMS and self-system therapy (SST, a modality similar to CBT) were performed in (albeit only five) depressed subjects, with positive results.100 Functional MRI was used to assess brain change in the left DLPFC (which was previously shown to be activated by SST-like tasks), and rTMS was then targeted at this individual area while an actual SST session was delivered.100 In a naturalistic, open-label, multicenter study, 196 depressed patients (most of whom were treatment-resistant) were assigned to receive CBT sessions with simultaneous HF (10 Hz) or LF (1 Hz) rTMS. Response rates reached 66%, with no difference between rTMS modalities.101 Regarding tDCS, a study in a rodent slice model showed that excitatory direct-current stimulation can strengthen cellular mechanisms thought to underlie learning and memory formation (long-term potentiation, LTP102). Critically, this enhancement occurred only when stimulation was applied “online,” i.e., during LTP – there was no effect when the identical stimulation was applied “offline,” i.e., prior to LTP. Behaviorally, parallel findings in rodents and humans showed that excitatory tDCS during learning enhanced memory for what was learned.103 Such enhancement has been shown to depend critically on stimulation during learning – the same stimulation applied prior to learning can have null or even antagonistic effects.104 Such basic neuroscience work suggests that the clinical efficacy of tDCS could potentially be enhanced if it is applied during learning that is designed to promote positive mood change, e.g., CBT (so-called functional targeting).105 The combination of tDCS with psychotherapy is particularly appealing. As both interventions target the prefrontal cortex, their combination might result in a positive synergy, with tDCS potentially enhancing a range of cognitive processes recruited during psychotherapy.106 Various forms of psychotherapy have been combined with tDCS. In an early, open-label study, Martin et al. combined tDCS with a task designed to improve identification of emotional states, in treatment-resistant depressed participants, with positive results (41% of study completers displayed treatment response).107 In another recent pilot study,108 patients with treatment-resistant depression (TRD) received active tDCS (20 min, 2 mA, applied to the left DLPFC) on 8 consecutive days and were randomly assigned to receive either 2 hours of mindfulness-based cognitive therapy (MBCT) or a 30-minute relaxation session immediately after each tDCS session. Results indicate a longer lasting reduction of depressive symptoms and enhanced cognitive processes in patients receiving the tDCS/MBCT combination. An ongoing, multicenter study is evaluating the efficacy of tDCS combined with group CBT.109 In this study, 192 depressed patients are being randomized to 12 sessions of either: 1) CBT + active tDCS (2 mA, 30 minutes); 2) CBT+sham tDCS; or 3) CBT alone.109 The combination of tDCS with DLPFC activation techniques, such as working memory training or combined cognitive training, has also shown promising results in healthy subjects,110 schizophrenia,111 mild cognitive impairment,112 and cognitive impairment in Parkinson’s disease.113 For combined treatment, tDCS has advantages over rTMS: it does not produce noise, which can be a distracting factor in rTMS sessions, and is portable. Cognitive remediation, such as the training of specific cognitive control processes, can also be performed following an intervention with TMS or iTBS in a sequential fashion, where cognitive remediation would be timed to take advantage of the enhanced cognitive capacities provided by the NIBS intervention. In this direction, an ongoing trial (PACt-MD) is comparing the efficacy of tDCS combined with cognitive remediation vs. double placebo in slowing cognitive decline and preventing Alzheimer’s disease in older persons with mild cognitive impairment, or MDD with or without mild cognitive impairment (PACt-MD, ClinicalTrials.gov, number NCT02386670). Furthermore, given that other forms of psychotherapy, such as interpersonal therapy, have been found to be effective in the treatment of MDD,114 it would also be interesting to study the combination of these techniques with NIN.

The “when”: combining NIBS with real-time neuroimaging and electrophysiology

The neurobiological effects of NIBS can also be assessed during or after application sessions,115-117 in what have been termed “online” and “offline” approaches, respectively.115 The “online” approach allows use of imaging techniques to quantify local neural network properties and application of NIBS so as to interfere with ongoing neuronal processing, visualizing how NIBS modulates the level or timing of neuronal activity with imaging and electrophysiology.115 For instance, a few studies have used fMRI to evaluate the online propagation of TMS-induced effects targeted to the prefrontal cortex.118-120 Further studies are required to explore this propagation as a potential biomarker for rTMS efficacy in the treatment of depressive patients. In the “offline” approach, one can increase or decrease the excitability of a brain region and measure the consequences thereof (i.e., with tDCS).115 For instance, a recent study of volunteers with high trait anxiety showed, through fMRI observation, that a single session of the typical tDCS protocol used in depression suppressed hyperactive fear signaling in the amygdala and increased activity in frontoparietal attentional top-down control regions.121 By these approaches, different combinations of NIBS modalities can be synchronized with neural oscillatory network activities, through real-time EEG or fMRI readouts which are further analyzed, thus closing the loop between stimulation and neurobiological response.116

The “where”: positioning and dose quantification

Optimal coil/electrode positioning is important to decrease within- and between-subject heterogeneity in the induced electric field and enhance clinical efficacy. Methods for standardizing coil/electrode positioning are commonly used, such as scalp landmark or hotspot-based coil placement for rTMS,122 or headgear that secures the electrodes in the desired location.123 Studies have shown that even small changes in coil/electrode positioning can change the induced neurobiological effects and negatively affect clinical outcomes.124,125 More sophisticated targeting approaches use individual or group-level anatomical and/or functional imaging to define the stimulation site, including the possible use of multiple electrodes to stimulate wider brain networks in a multifocal approach.126 Targeting based on fMRI guidance has been shown to produce stronger online rTMS effects compared to other targeting strategies.127 The use of neuronavigation systems can greatly improve the spatial precision of TMS. Furthermore, robotic coil-holder systems can provide millimeter accuracy and continuous tracking of the TMS coil. One such robotic system recently received FDA 510(k) clearance. “Dose” quantification is key to determine the dose-response gradient and to titrate the intervention parameters accordingly. Nonetheless, determining dose is challenging in NIN, as various stimulation parameters are employed (e.g., current intensity, waveform, and duration for electrical stimulation; number, frequency, pattern, and intensity for magnetic stimulation; frequency and duration of treatment course), which influence one another in complex interactions.128 The net result (excitation, inhibition, or no effect) is influenced by other concomitant interventions (pharmacological agents, cognitive tasks, other NIBS interventions, and psychological interventions),26,129 as well as by network activity (i.e., brain state). For convulsive therapies, there might be a trade-off between using higher doses to produce greater clinical benefits but with additional side effects.85 More recently, computational models have been used to quantify electric fields (EFs) in brain regions of interest (ROIs). In fact, there are freely available software packages that perform electric field simulations using high-quality MRI images (templates or individualized) via a series of steps: 1) automated tissue segmentation of structural MRI; 2) meshing of the different tissue compartments to form a 3D model of the head and brain; 3) processing DTI data to extract white-matter anisotropic conductivity values; 4) incorporating electrodes and the TMS coil on the head model; 5) assigning appropriate electrical properties for the tissues and electrode/coil; 6) solving for the electric field and current density, via numerical methods such as finite element or boundary element methods; and 7) exporting, visualizing, and/or transforming electric field distribution to standard space for group analysis.130 The resulting EFs can be graphically assessed; the results of EFs per ROI can be printed out and mapped into standard spaces, such as MNI and FreeSurfer, for analysis of correlations between current distribution, clinical outcomes, and structural and functional neuroimaging findings. These computational models allow researchers to 1) understand the biophysics and mechanism of action for NIN modalities; 2) benchmark and compare different technologies; and 3) quantify interindividual variation in the induced dose as it relates to clinical outcome. Further work is needed to investigate the effects of EFs on brain tissue, and whether models can be used for treatment planning and optimization.

The “who”: identifying responders

The predictors of NIN response are mostly unknown. Although treatment resistance in MDD is a robust clinical predictor of poor response for most antidepressant strategies, including NIN interventions,131-133 this might be related more to the depressive episode per se than to the intervention. In addition, the absence of improvement in response to rTMS early during treatment predicts continued non-improvement with further rTMS treatment,134 whereas acute improvement with ECT predicts final remission.135 Other predictors have shown mixed results. For instance, in tDCS, higher “dose” was associated with better outcomes in one meta-analysis,131 but a further sham-controlled trial using a higher dose than previous ones yielded nonsignificant findings.136 Another study that applied tDCS over the motor cortex also concluded that enhancement of tDCS “dose” does not necessarily increase the neurobiological effects of stimulation, but might shift the direction of excitability alterations.137 Furthermore, tDCS responders have been found to display greater improvements in the MADRS dysphoria and retardation factors compared to nonresponders.138 For rTMS, although most research to date has focused more on testing the efficacy of different interventions rather than on identifying subgroups of patients who would respond better to a particular intervention,139 some recent work suggests that the treatment may be most effective for certain particular “biotypes” of depression, detectable from whole-brain network connectivity on functional MRI,140 and that the optimal rTMS parameters to achieve antidepressant effect might vary depending on treatment resistance, age, and sex.141 Moreover, greater response to a LF-rTMS protocol has been associated with lower MADRS retardation scores at baseline.142 Regarding ECT, even first-line recommendations such as older age, psychosis, and melancholic features21 were not consistently identified in a meta-analysis.133 One study reported that ECT responders displayed higher scores on a MADRS dysphoria factor compared to nonresponders, while the procedure had only a small effect on a MADRS vegetative factor.143 Although relatively large trials have been conducted in the NIN field,29,31,144 to the best of our knowledge, there is no published research using machine-learning algorithms to predict NIN response based on a clinical dataset – as has already been done in pharmacological trials.145 For example, using a gradient boosting model, Chekroud et al.145 identified in STAR*D a dataset of 25 variables that predicted depression response significantly above chance. In another STAR*D analysis, Chekroud et al.146 identified which pharmacotherapies would be associated with greater depression improvement for patients grouped according to a cluster of depressive symptoms. These approaches are important and fundamentally different from statistical approaches to identification of predictors, which are based on groups, not individuals.13,147 In fact, statistical methods focus on inference – creating a mathematical model that tests a hypothesis about how a system behaves, whereas machine learning focuses on prediction – i.e., finding generalizable predictive patterns that aim to forecast future behaviors regardless of their mechanistic basis12 (Figure 3). Additionally, through employing almost no pre-assumptions and a nonlinear function canvas, machine learning techniques can model complex patterns that can identify relationships between large amounts of data and data of diverse types,148,149 increasing the processing speed and output of predictive models. For instance, a machine-learning modality known as “deep learning” provides a promising approach for analysis of the relationship between electromagnetic fields and biological tissues (i.e., a head model is automatically generated through MRI, with correspondence between voxels to specific tissue types with given electrical conductivity values).150
Figure 3

Example of a machine learning pipeline. Analysis pipeline. A) Treatment outcomes of group are predicted according to the feature dataset. B) Models are trained to classify responders and non-responders at the study endpoint. Performance is evaluated in a repeated nested cross-validation paradigm. C) Features with the highest contribution to the model can be identified. RCT = randomized clinical trial.

One concern about such approaches is the lack of interpretability that the resulting models usually possess. There is no clear way of interpreting complex nonlinear models. In many clinical applications, including selection of treatment or prediction of side effects, the clinician does not need to fully comprehend how the machine is processing information. In that case, the main concern is how effectively the model can predict a specific outcome. Preliminary work from our group (under review) used data from the ELECT-TDCS31 to estimate single-subject prediction of treatment response to tDCS, escitalopram, or placebo. A total of 245 subjects were included, of whom 55% were women (n=166) and 29% had TRD (n=91).31 The feature dataset included baseline clinical, sociodemographic, somatic, treatment-related, and depression-related variables, as well as mood and anxiety scales. Using a XGBoost tree boosting algorithm, we could predict response to placebo, escitalopram and tDCS with 45% (95%CI 39-52), 56% (50-61%), and 67% (62-71%) balanced accuracy, respectively. This preliminary work reveals that ML-based approaches can predict NIBS response above chance, facilitating further investigation of this approach in upcoming studies using clinical and biological data. The identification of responders could potentially be greatly enhanced by using biomarkers.151 For precision NIN, biomarkers of treatment response would be especially useful, not only due to their predictive value but also – particularly – to shed light on the mechanisms of action of NIN. Importantly, biomarkers of disease might not be necessarily related to treatment response, and a biomarker for a pharmacological treatment might not necessarily apply to NIN. Among several potential biomarkers, we propose that cognitive, neuroimaging, and neurophysiological markers would be particularly useful for precision NIN, since they are directly targeted by these techniques. Structural and functional neuroimaging biomarkers have indicated that the volume and thickness of certain structures (e.g., portions of the prefrontal cortex and anterior cingulate cortex), the resting-state connectivity of certain networks, and the connectivity between ROIs predict and are modified by the antidepressant effects of NIBS.152-155 For instance, a recent study showed that patients respond bimodally to rTMS: at the beginning of treatment, nonresponders exhibited higher anhedonia and lower connectivity in a brain network classically associated with reward, consisting of the ventral tegmental area, striatum, and part of the VMPFC. This study indicated that a subtype of depressive patients, identified on the basis of syndromic and neuroimaging characteristics, may respond better to rTMS.156 Also looking at treatment-response biomarkers, researchers found early response to rTMS treatment to be predictable by the integrity of an extended salience-executive system, indexed by fronto-insular connectivity and SN connectivity with visual processing regions, although this was not true for sustained response at 3-month follow-up.157 In an exploratory analysis, researchers found that higher functional connectivity between the DLPFC and striatum predicted better treatment response to TMS in a group of depressed patients.158 In another study140 that used functional magnetic resonance imaging (fMRI) in a large, multisite sample of 1,188 depressive patients, four distinct neurophysiological subtypes (“biotypes”) were identified on the basis of distinct patterns of dysfunctional connectivity in frontostriatal and limbic networks. Patients in “biotype 1” were approximately three times more likely to benefit from rTMS over the DMPFC than those in “biotypes” 2 or 4,140 although these findings need to be interpreted with caution, given questions about their replicability.159 For tDCS, a recent study showed that larger gray-matter volumes in the left DLPFC at baseline were further associated with antidepressant response to tDCS, but not to escitalopram or placebo.160 Although the effect sizes were small and had no individual-level predictive value, these findings contribute to our understanding of the antidepressant effects of tDCS by showing a specific association between the stimulated area and further antidepressant response. So far, there does not seem to be a consistent unique pattern of functional or structural abnormality to predict the effect of noninvasive neuromodulatory MDD interventions. In future, combined use of biomarkers may help guide treatment selection.161 Cognitive functions are another set of biomarkers worth exploring in NIN trials. In depression, these are often impaired independently of mood162-164 and are associated with network dysfunctions.165,166 With the exception of ECT, NIBS appears to be cognitively safe in healthy adults,25,167 with most single-session studies indicating no cognitive decline, although exceptions have been noted.168,169 In depressed patients, recent meta-analyses found that NIBS techniques are not associated with cognitive side effects.170,171 In contrast, cognitive adverse effects are common in convulsive therapies.91 NIBS interventions could be designed to not only improve mood, but to concomitantly induce cognitive enhancement. The implementation of such a combined approach would depend upon multiple factors, including NIBS protocols, specific neurocircuitry and physiological premises, online and offline stimulation, pre-existing cognitive difficulties, and other clinical and demographic factors. For instance, NIBS administered over the PFC induced improvement in a working memory task.172 Thus, it can be supposed that, in MDD, PFC stimulation could exert pro-cognitive effects, particularly in complex attention and working-memory domains.173 Nevertheless, although some studies suggested cognitive improvement in some tasks after rTMS in depressive patients, the majority of studies showed no cognitive benefits after NIBS.174 Possible reasons for null findings are limitations of specific NIBS paradigms (e.g., poor spatial targeting, inadequate dose), practice effects, reduced sensitivity and specificity of the tests (e.g., paper-and-pen instead of computerized tests), ceiling effects,174,175 and the lack of concomitant cognitive activity. Cognitive functions could also be leveraged to individualize treatment approaches and predict treatment outcome. For example, baseline cognitive performance or acute cognitive effects after the first NIN session can predict antidepressant response to NIN,176,177 and could therefore be used as a potentially straightforward method for prediction in combination with machine-learning approaches. Cognitive functions have also been shown to be useful predictors for outcomes of other treatment approaches, such as psychotherapy,178 and more long-term outcomes, such as return to work.179 Moreover, evaluating cognitive changes can provide mechanistic insights into the antidepressant mechanisms of action of NIN – e.g., by exploring whether they moderate and/or mediate depression improvement – and into NIN-induced changes in specific brain structures.180 In this case, cognitive changes have been operationalized as fundamental mechanisms of action, but also as more translational processes, such as self-referential thoughts and emotions (e.g., negative affect, rumination, regret, cognitive bias). To date, most of this research is being performed in healthy volunteers, but the transition to clinical samples – also based on the idea of functional targeting of similar functional and neuroanatomical circuits using multimodal interventions – is slowly moving forward. Motor cortical excitability (MCE) measures were the first neurophysiological biomarkers investigated in MDD, as the motor cortex can be easily probed using single- and paired-pulse TMS, which are associated with GABA and glutamate activity in this structure.181,182 Studies have shown that baseline measures of cortical inhibition and facilitation were associated with antidepressant response to tDCS, rTMS, and ECT,31,183,184 although effect sizes were small. In addition, EEG-based neurophysiological parameters are associated with antidepressant response to rTMS.185,186 More recently, it has been advocated that prefrontal excitability indexed using TMS-evoked potentials and TMS-EEG systems might be more a specific marker of NIN effects compared to MCE, as these methods can probe the cortical excitability of frontal brain areas implicated in depressive pathophysiology, with high temporal resolution.187 However, these techniques are still novel and technically challenging, and it is still unclear which indexes better represent GABA activity.116 These limitations notwithstanding, promising findings have been observed using this biomarker modality to predict NIN response.188-190 Other biological markers have also been explored in NIN, including genetic and non-genetic peripheral biomarkers and heart rate variability.44,191-199 Although some positive findings were found, results have been inconsistent, and mostly derived from open studies. In addition, the identification of candidate genes has been challenged by more recent studies showing that most previous findings are likely to be false positives.200 Likewise, the literature on peripheral depression biomarkers is fraught with bias.201,202

Limitations and perspectives

Although promising, the precision NIN framework should be pursued and expanded with a view to improved clinical applicability. For instance, despite the investigation of several biomarkers, most positive findings have emerged from poorly controlled exploratory studies using small sample sizes, thus requiring further validation. In this regard, properly controlled studies of biomarkers could further expand our knowledge of their role in the pathophysiological processes related to MDD and help predict response to treatment. Strategies that would enhance biomarker validity include adequately powered sample sizes and a priori hypotheses for the role of the markers of interest. Novel clinical trials investigating NIN interventions should embed the investigation of biomarkers in their design. For certain NIN modalities in which clinical efficacy is already proven, such as most variants of rTMS and ECT, sham-controlled trials are not necessary and, in fact, not feasible from an ethical perspective, as equipoise to placebo cannot be assumed. Therefore, academic centers that perform rTMS and ECT should incorporate systematic data collection of clinical and demographic characteristics – as well as questionnaires, inventories and scales measuring depression and cognitive changes during treatment – into their clinical routine. Ideally, molecular and neuroimaging biomarkers should be collected as well, and data could be shared by different data centers. Such an approach is exemplified by the Global ECT-MRI Research Collaboration, which already includes more than 22 centers collecting ECT and MRI data worldwide.48 On the other hand, for other NIN interventions, such as tDCS and MST, phase-3 controlled studies are still necessary, as clinical efficacy remains unproven. For such trials, we recommend that investigators incorporate a comprehensive set of biomarkers to be investigated within the context of the primary study hypothesis. In this context, the framework of “target engagement” and “target validation” is useful – i.e., those biomarkers deemed most promising on the basis of preclinical and early clinical findings should be investigated as predictors and moderators of clinical response. Likewise, even though the logic of using approaches such as machine-learning in individual patient data and meta-analytic datasets is sound and necessary, any findings would still be retrospective, requiring further validation in novel datasets distinct from those on which the classifiers were trained, to determine generalizability. This necessitates a global effort in scientific collaboration and data sharing, with particular focus on overcoming difficulties in access to the literature and primary data held beyond paywalls. In this sense, open-access initiatives are welcome, as are promising changes in business models of academic publishing. In this context, one of the leading journals of the field (Brain Stimulation) has become fully open-access as of January 1, 2020.203 Moreover, policies ensuring that properly anonymized data from clinical trials (whether sponsored by public or private institutions) can be shared under request should be endorsed by regulatory agencies to further promote data-sharing initiatives. Machine learning-guided intervention trials in NIN are still a necessary second step to further validate predictive algorithms.204 Cost-effectiveness analyses should also be performed to verify whether a precision-oriented approach is economically advantageous: on the one hand, enhancing efficacy and decreasing side effects can increase individual benefits and reduce treatment cost; on the other, the additional costs of using precision techniques should be considered. For instance, the advantages of tDCS include its low cost and portability, but the neuroimaging scans and individualized electrode positioning required before treatment could make these advantages moot if additional gains in efficacy and efficiency are not achieved. Although not discussed in the present review, computational and preclinical studies would be useful to deepen our understanding of NIN techniques,205 while studies in healthy volunteers are needed to narrow the parameter spaces of these techniques (for instance, by using closed-loop bayesian, adaptive optimization).206 Finally, it should be noted that the selection of studies for this special article was unsystematic, i.e., publications were deemed relevant and selected according to the perspective of the authors, with the inherent subjective limitations that such a narrative entails.

Conclusions

Depressive disorders are prevalent, disabling conditions. Conventional antidepressant treatments fail to induce remission in approximately one-third of patients with MDD, may result in intolerable side effects (first-line medications), or may be expensive and time-consuming (psychotherapy). Moreover, such therapies are still prescribed on a “trial-and-error” basis, in which achievement of a satisfactory response can take several months. In this context, NIN techniques are increasingly considered safe, tolerable, and effective, whether as monotherapy or augmented with other interventions, such as medications and psychotherapy. Furthermore, as stimulation parameters can be directed to specifically affected brain areas, NIN is also undergoing a paradigm shift towards a precision-oriented framework that takes into consideration “knowledge about brain circuits that underlie complex cognitive, emotional and self-reflective functions” in order to guide individualized patient-oriented treatments.36 Ultimately, this new framework does not rely solely on observable clinical outcome information, but also on data from multiple biological levels, from cells to circuits. As open-access initiatives across the globe give space to merging and analyzing large datasets and subjects in clinical trials are increasingly assessed via multimodal approaches, a greater understanding of methods for handling big data will be mandatory for specialists in the field. In this context, future NIN-related research would benefit from a focus on optimization of its parameters, discovery of remission- and response-related biomarkers, elucidation of cognitive safety and enhancement mechanisms, and advancement of scientific knowledge related to mechanisms of NIN action.

Disclosure

Z-DD is listed as inventor on patents/patent applications related to brain stimulation technology, assigned to Columbia University and NEVA Electromagnetics, not licensed, and with no remuneration. Although he is an employee of the U.S. government, the views expressed are his own and do not necessarily represent the views of the National Institutes of Health, the Department of Health and Human Services, or the U.S. government. The other authors report no conflicts of interest.
  201 in total

1.  Big data analytics and machine learning: 2015 and beyond.

Authors:  Ives Cavalcante Passos; Benson Mwangi; Flávio Kapczinski
Journal:  Lancet Psychiatry       Date:  2016-01       Impact factor: 27.083

Review 2.  Working memory improvement with non-invasive brain stimulation of the dorsolateral prefrontal cortex: a systematic review and meta-analysis.

Authors:  André Russowsky Brunoni; Marie-Anne Vanderhasselt
Journal:  Brain Cogn       Date:  2014-02-08       Impact factor: 2.310

Review 3.  Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 Clinical Guidelines for the Management of Adults with Major Depressive Disorder: Section 2. Psychological Treatments.

Authors:  Sagar V Parikh; Lena C Quilty; Paula Ravitz; Michael Rosenbluth; Barbara Pavlova; Sophie Grigoriadis; Vytas Velyvis; Sidney H Kennedy; Raymond W Lam; Glenda M MacQueen; Roumen V Milev; Arun V Ravindran; Rudolf Uher
Journal:  Can J Psychiatry       Date:  2016-08-02       Impact factor: 4.356

4.  Computational Psychiatry: Embracing Uncertainty and Focusing on Individuals, Not Averages.

Authors:  Adam M Chekroud; Chadrick E Lane; David A Ross
Journal:  Biol Psychiatry       Date:  2017-09-15       Impact factor: 13.382

5.  International randomized-controlled trial of transcranial Direct Current Stimulation in depression.

Authors:  Colleen K Loo; Mustafa M Husain; William M McDonald; Scott Aaronson; John P O'Reardon; Angelo Alonzo; Cynthia Shannon Weickert; Donel M Martin; Shawn M McClintock; Adith Mohan; Sarah H Lisanby
Journal:  Brain Stimul       Date:  2017-10-27       Impact factor: 8.955

6.  Heart rate variability is a trait marker of major depressive disorder: evidence from the sertraline vs. electric current therapy to treat depression clinical study.

Authors:  Andre Russowsky Brunoni; Andrew H Kemp; Eduardo M Dantas; Alessandra C Goulart; Maria Angélica Nunes; Paulo S Boggio; José Geraldo Mill; Paulo A Lotufo; Felipe Fregni; Isabela M Benseñor
Journal:  Int J Neuropsychopharmacol       Date:  2013-06-12       Impact factor: 5.176

7.  Plasma biomarkers in a placebo-controlled trial comparing tDCS and escitalopram efficacy in major depression.

Authors:  André R Brunoni; Frank Padberg; Erica Leandro Marciano Vieira; Antônio Lucio Teixeira; André F Carvalho; Paulo Andrade Lotufo; Wagner F Gattaz; Isabela Martins Benseñor
Journal:  Prog Neuropsychopharmacol Biol Psychiatry       Date:  2018-06-09       Impact factor: 5.067

8.  Pharmacological strategies in the prevention of relapse after electroconvulsive therapy.

Authors:  Joan Prudic; Roger F Haskett; W Vaughn McCall; Keith Isenberg; Thomas Cooper; Peter B Rosenquist; Benoit H Mulsant; Harold A Sackeim
Journal:  J ECT       Date:  2013-03       Impact factor: 3.635

Review 9.  Repetitive Transcranial Magnetic Stimulation for the Acute Treatment of Major Depressive Episodes: A Systematic Review With Network Meta-analysis.

Authors:  Andre R Brunoni; Anna Chaimani; Adriano H Moffa; Lais B Razza; Wagner F Gattaz; Zafiris J Daskalakis; Andre F Carvalho
Journal:  JAMA Psychiatry       Date:  2017-02-01       Impact factor: 21.596

10.  A novel concurrent TMS-fMRI method to reveal propagation patterns of prefrontal magnetic brain stimulation.

Authors:  Jord J T Vink; Stefano Mandija; Petar I Petrov; Cornells A T van den Berg; Iris E C Sommer; Sebastiaan F W Neggers
Journal:  Hum Brain Mapp       Date:  2018-08-29       Impact factor: 5.038

View more
  4 in total

Review 1.  Transcranial Electrical Stimulation for Psychiatric Disorders in Adults: A Primer.

Authors:  Hyein Cho; Lais B Razza; Lucas Borrione; Marom Bikson; Leigh Charvet; Tracy A Dennis-Tiwary; Andre R Brunoni; Pedro Sudbrack-Oliveira
Journal:  Focus (Am Psychiatr Publ)       Date:  2022-01-25

Review 2.  A narrative review on invasive brain stimulation for treatment-resistant depression.

Authors:  Manoj P Dandekar; Alexandre P Diaz; Ziaur Rahman; Ritele H Silva; Ziad Nahas; Scott Aaronson; Sudhakar Selvaraj; Albert J Fenoy; Marsal Sanches; Jair C Soares; Patricio Riva-Posse; Joao Quevedo
Journal:  Braz J Psychiatry       Date:  2022 May-Jun

3.  Combined effects of theta-burst stimulation with transcranial direct current stimulation of the prefrontal cortex: study protocol of a randomized, double-blinded, sham-controlled trial using 99mTc-ECD SPECT.

Authors:  Laís B Razza; Carlos A Buchpiguel; Stefanie De Smet; Izio Klein; Chris Baeken; Ricardo Galhardoni; Marie-Anne Vanderhasselt; André R Brunoni
Journal:  Trends Psychiatry Psychother       Date:  2021-02-24

4.  High-frequency repetitive transcranial magnetic stimulation alleviates the cognitive side effects of electroconvulsive therapy in major depression.

Authors:  Xing Chen; Tongtong Zhang; Xiaoyan Shan; Qun Yang; Peiyun Zhang; Haijiao Zhu; Fei Jiang; Chao Liu; Yanzhong Li; Weijun Li; Jian Xu; Hongmei Shen
Journal:  Front Psychiatry       Date:  2022-10-03       Impact factor: 5.435

  4 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.