Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning.

We aimed to replicate a published effect of transcranial direct-current stimulation (tDCS)-induced recognition enhancement over the human ventrolateral prefrontal cortex (VLPFC) and analyse the data with machine learning. We investigated effects over an adjacent region, the dorsolateral prefrontal cortex (DLPFC). In total, we analyzed data from 97 participants after exclusions. We found weak or absent effects over the VLPFC and DLPFC. We conducted machine learning studies to examine the effects of semantic and phonetic features on memorization, which revealed no effect of VLPFC tDCS on the original dataset or the current data. The highest contributing factor to memory performance was individual differences in memory not explained by word features, tDCS group, or sample size, while semantic, phonetic, and orthographic word characteristics did not contribute significantly. To our knowledge, this is the first tDCS study to investigate cognitive effects with machine learning, and future studies may benefit from studying physiological as well as cognitive effects with data-driven approaches and computational models.

1 Introduction

Transcranial direct-current stimulation (tDCS), a method of safely and non-invasively delivering a weak electric current through the cortex, has been gaining increasing attention [1] as a tool for studying and possibly enhancing episodic memory [2]. Anodal tDCS (atDCS) involves increasing cortical excitability in a target region such as the dorsolateral prefrontal cortex (DLPFC), which may in turn facilitate or enhance memory performance by decreasing reaction time and/or increasing memory accuracy [3, 4]. However, there remains well-founded skepticism about widespread applications of tDCS, partly because of the lack of knowledge about mechanisms of action [5] and difficulty in replicating results [6].

Most tDCS studies in episodic memory have targeted the DLPFC (see [7] for a review) because of its role in selective attention [8], strategic retrieval [9], and other executive functions involved in episodic memory [10]. In line with the hemispheric encoding/retrieval asymmetry model [11], previous studies with tDCS show functional asymmetry in the DLPFC, with stimulation over the left DLPFC affecting encoding and stimulation over the right DLPFC affecting retrieval [12-14]. However, memory enhancements due to atDCS have been inconsistent over the DLPFC, in both individual studies [4, 10, 12, 13, 15] and meta-analyses [16-18]. Most studies over the DLPFC examined effects when atDCS was delivered predominantly during encoding. At least two studies examined purely offline encoding effects: [19] delivered atDCS over F3 for 15 minutes before encoding and found no effects on accuracy or reaction time, although atDCS over the contralateral hemisphere (F4) led to faster reaction times. Lu and colleagues [20] delivered atDCS over a more lateral site (FC5) for 20 minutes before encoding and found an increase in correct memory responses for previously-presented items.

To the authors’ knowledge, no study to date has systematically examined effects of timing of administration on the DLPFC, comparing online vs offline effects at encoding or retrieval. However, a previous study by the authors showed significant differences in online and offline effects over an adjacent site, the left VLPFC [20]. The authors administered atDCS before the study phase (offline encoding) or during the study task (online encoding) that involved intentionally memorizing each presented word (Experiment 1). As expected given support from fMRI and TMS studies, the left VLPFC seemed to be strongly modulated by atDCS at encoding, specifically online but not offline encoding.

The VLPFC and DLPFC are thought to play functionally distinct roles in long-term verbal memory, and the left VLPFC appears selectively engaged in verbal but not non-verbal material. Specifically, the VLPFC may be more involved in encoding for individual items, while the DLPFC is more engaged in associative or relational encoding [21-24]. Moreover, activation in the DLPFC may predict long-term memory success through DLPFC involvement in domain-general working-memory or executive processes such as mental manipulation of information (e.g. visualizing rotating objects; [25]) and applying a specific strategy (e.g. for retrieval of information; [26]). In a tDCS study conducted by [27], tDCS was applied over the left DLPFC in a cued-word-completion task to modulate memory performance. The results showed that cathodal stimulation inhibited memory accuracy after error learning in comparison with sham: the errors were evoked through guessing, or the incorrect completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect). They concluded that the DLPFC was only modulated when its processing demands were higher for conflicting information. In contrast, the VLPFC appears to be engaged in maintaining information in working memory [28] and processing semantic information including accessing lexical identity and connecting words to other words in the semantic network [29].

The current study involved administering atDCS over the left DLPFC during offline or online encoding to identify the effective time of administration and confirm the causal role of the DLPFC in encoding processes. Participants were assigned to a Sham group (over the DLPFC) or one of three tDCS groups: DLPFC Online, DLPFC Offline, and VLPFC Online. Previous research suggests that atDCS could potentially enhance memory performance when delivered online or offline encoding over the DLPFC [30-34], so we predicted higher memory performance for either condition (DLPFC Online or DLPFC Offline). In addition, the study provides a novel comparison of atDCS effects during encoding over the VLPFC vs DLPFC. We predicted a successful replication of Experiment 1 from [20] with higher memory accuracy for the VLPFC Online group compared to Sham, but we also predicted a larger effect size for VLPFC Online than DLPFC Online or DLPFC Offline based on previous rTMS studies comparing memory disruption in the VLPFC vs DLPFC. For example, [35] found that stimulation over the VLPFC during encoding led to a greater disruption in memory performance than over the DLPFC, suggesting that the VLPFC may play a more important role in encoding processes. Finally, there remains a lack of knowledge about the mechanisms of atDCS on verbal memory and whether atDCS effects can be influenced by word characteristics, specifically semantic (meaning), orthographic (letters), and phonological (sounds). We also aimed to examine how the tDCS effects interacted with language and linguistic factors, since the replication study was conducted in Russian and the original study was conducted in English. Thus, data were applied from a previously-published experiment ([20]; Experiment 1) and the current experiment from the Online Encoding (VLPFC) group to an AUROC analysis. After comparisons of group means in reaction time and accuracy on the recognition test, a regression algorithm was applied to infer the quality of recognition using semantic and phonological features of words. Semantic and phonological features were added to the model and predictive capability was assessed. We predicted that the model would be able to distinguish between words that are hard and easy to remember in Russian and English speakers. Thus, machine learning algorithms were applied in order to reveal the impact of factors including semantics, phonetics and individual variance to episodic memory performance as well as the interaction with tDCS.

2 Methods and materials

In line with a replication study, we followed the same procedure as in Experiment 1 of [20] with few exceptions (translation of materials to Russian and comparison of DLPFC and VLPFC as stimulation sites). Broadly, participants memorized words presented individually on a screen while undergoing tDCS (before or during this phase), and 24 hours later they performed a recognition test. The study and test stimuli were translated based on the first word meaning in the vocabulary entry, and the translation achieved relatively matched frequency (M = 24.47; M = 40.38; [36]) and number of letters (M = 6.17; M = 6.19). Specifically, we applied the same tDCS settings (2 mA active tDCS and 30 s Sham tDCS) over the DLPFC (all conditions: offline, sham, and online) in addition to the VLPFC (online encoding only). The electrodes were placed at both sessions (study and test 24 hours later), but stimulation was only delivered before or during the encoding task in the study phase. The anode was placed over the left VLPFC (F7) or the left DLPFC (F3), and the cathode was placed over the ipsilateral (left) shoulder. See Section 2.1 below for the experimental conditions. Stimulation was delivered via a battery-powered, constant-current stimulator (Brainstim, EMS, Bologna, Italy) through 5 × 7cm2 electrodes. In addition, participants were asked to describe the sensations that they felt during the stimulation and indicate whether they believed they received real or placebo stimulation. Participants were single-blinded, and the questionnaire indicated that blinding was successful: 80% believed that they received real stimulation. Data were analyzed in SPSS (version 24; IBM, Armonk, New York) and machine learning studies were conducted in Python (version 3.7).

2.1 Participants

Participants were randomly assigned to one out of four groups: VLPFC Online, DLPFC Online, DLPFC Offline and Sham. Based on a power analysis to detect a large effect size [20], d = 1.29;α = 0.05, 1 − β = .95), we aimed to recruit 31 participants per group (124 total), but we analyzed data from 97 participants because of exclusions (see below). Participants (female = 63, male = 49, M = 20.51, SD = 2.89) were native Russian speakers with normal or corrected-to-normal vision and no history of neurological or psychiatric illness. Bonferroni-corrected pairwise comparisons of ae between pairs of groups showed a significant difference in age between VLPFC Online (M = 21.76, SD = 3.54) and DLPFC Online (M = 19.21, SD = 1.10), p = .014. However, there were no significant differences in age between the other groups, p > .062.

Data from 12 participants were excluded from analysis because 1) the participant was feeling unwell (2 participants: DLPFC Sham and VLPFC Online) 2) there were technical issues (6 participants: two in DLPFC Online, two in VLPFC Online, and two in DLPFC Sham) or 3) the participant was left-handed (one participant: DLPFC Online) and 4) there was an experimenter error (3 participants: one in DLPFC Sham and two in VLPFC Online). The exclusions resulted in 26 in the VLPFC Online group, 31 in the DLPFC Offline group, 28 in the DLPFC Online group, and 27 in the Sham group. Moreover, fifteen outliers were excluded (see data analysis for criteria), leaving a final sample size of 25 for VLPFC Online (one outlier), 25 for DLPFC Offline (six outliers), 23 for DLPFC Online (five outliers), and 23 for DLPFC Sham (four outliers). For all Machine Learning analysis all DLPFC Offline participants except one were used (30 participants).

The study was approved by the ethics committee of National Research University Higher School of Economics (Moscow, Russia) and followed the corresponding ethical guidelines. All participants provided written informed consent and were given monetary compensation (500 rubles) for their time.

2.2 Procedure and experimental design

Each participant, regardless of group assignment, came to the laboratory twice within a 24 hour-interval: on the first day participants memorized the verbal stimuli one word at a time through pleasantness judgements (pleasant or unpleasant), and on the second day they performed an old/new recognition memory task. For DLPFC Sham, stimulation was delivered for 30 seconds before the stimulator was turned off, and the start of the stimulation corresponded to the start of the reading task for half the Sham participants and the start of the memory task (study phase) for the other half. Stimulation was applied for the entire duration (10 minutes) of the reading task for the DLPFC offline group and for the entire duration (10 minutes) of the study memory task that immediately followed for the VLPFC Online and DLPFC Online groups. In summary, the stimulation was delivered for approximately 10 minutes to cover the entire duration of the reading and memory tasks.

2.3 Data analysis

One-way ANOVAs were conducted for each measure of recognition accuracy [37]: discrimination index (Pr: combined index involving hits and false alarms), proportion of hits (correctly identifying an “old” item as studied), and proportion of false alarms (incorrectly identifying a new item as studied). In addition, one-way ANOVAs were conducted for average reaction times and reaction times for hits and false alarms separately. Finally, a one-way ANOVA was conducted on response bias, an index involving hits and false alarms (higher values indicate a more conservative pattern of responding to old items). Significant effects were followed up with Bonferroni-corrected planned contrasts between each stimulation group and Sham (one-tailed). Outliers were excluded based on two standard deviations from the mean on any of the following dependent measures: discrimination index Pr, proportion of hits, proportion of false alarms, and reaction times for hits or false alarms.

2.4 Area under the receiver operating characteristic curve as a measure of predictive performance for episodic memory changes

For the second analysis including data from the [20], we included all 17 participants from the Sham group and 17 from the Online tDCS group, resulting in the same mean age with no significant differences between the groups. From the current study, we only included data from the DLPFC Sham (N = 23) and VLPFC Online groups (N = 25) with no significant differences in age between groups (Bonferroni-corrected paired comparisons), p > .852. We implemented a 2 × 2 design with the first factor as language (English or Russian) and the second factor as tDCS group (VLPFC or Sham).

Applying Area Under the Receiver Operating Characteristic Curve (AUROC) was appropriate given that the task involved binary classification (coded as 1 if the word was presented at study and 0 if unpresented), which fits any ML approach to measuring predictive performance of classification models [38]. In ML Classification models, true positives represent the model predicting the positive class correctly and false positives represent incorrect prediction of positive class.

AUROC 1) provides information about true and false positives in a single measure 2) shows the source of model error, with larger values indicating that the model predicts better than chance (greater true positives) and 0.5 indicating the opposite and 3) does not assume a normal distribution and is robust to unequal sample sizes (whereas average accuracy measures would overestimate performance). Thus, it is the most suitable measure for the current data.

We included a trial-based approach, computing AUROC for each word based on the entire sample’s responses for that word, and a participant-based approach, computing AUROC for each participant based on responses (old or new) for all words presented to that participant.

AUROC values can show whether tDCS is effective in enhancing memory performance, and the subsequent ML analyses using AUROC can show the effects of various factors (semantic, phonetic, orthographic word characteristics) on recognition of individual words. We attributed any differences in memory performance not explained by differences in sample size, linguistic characteristics, or group (tDCS vs Sham) to individual differences in memory function.

If stimulation enhances episodic memory performance, we expect a significant increase or decrease in AUROC (both over words and over participants in both VLPFC groups compared to Sham). If tDCS impairs memory performance, we expect values close to 0.5, which reflects that participant response decisions (old or new) approached chance.

If tDCS has no effect, there would be no difference in AUROC values across words or AUROC distributions between VLPFC and Sham. Since tDCS enhanced episodic memory in the original experiment and we expect a successful replication, we expected AUROC values for VLPFC to be significantly different from chance, with significantly different AUROC values and distributions compared to Sham. Moreover, tDCS significantly increased reaction time in [20], so we predicted a significant difference between VLPFC and Sham in reaction time. Since we did not assume a normal distribution, we applied the two-sample Kolmogorov-Smirnov test for accuracy and reaction time. We have also computed median test for differences in median AUROC.

2.5 Extraction of semantic, phonological and orthographic determinants of episodic memory performance

To investigate the effect of semantic and phonetic features on memorization and the interaction with tDCS, we extracted words with significantly more true positive rates (reflecting hits) and false positive rates (reflecting false alarms) in each group (VLPFC and Sham). We examined the top ten words from each category (see S1 Appendix). A preliminary descriptive analysis of the highest and lowest AUROC-words showed no association between individual variance and memory accuracy in VLPFC and Sham groups. Therefore, we do not report these results further.

Moreover, we used word embeddings, which approximated the semantic similarities between words. In line with the trial-based and participant-based approaches, we conducted two kinds of ML studies: 1) Participant-independent (trial-based) analysis tests the success of a model that predicts the AUROC for each word using either word embeddings or letter-based one-hot encoding (each letter i is replaced by a vector of N components with the i-th component being equal to 1 while every other component is zero.) and 2) Participant-specific (participant-based) analysis tests the success of a model that predicts whether a participant was able to recall the word given either word embeddings or letter-based one-hot encoding for each word for individual participants. Word embeddings were applied from FastText [39], a library developed in Facebook that incorporates semantic information from each word as well as subwords contained within to embed vectors. The pretrained embedding was aligned following [40] so that English and Russian vectors could lie in the same vector space and we could use it to train a model that uses both English and Russian vectors to check whether there are any language-independent semantic determinants. To artificially enlarge the sample size for our models, we use the augmentation approach described in [41] (see S2 Appendix for details).

For prediction, we applied TPOT [42], an automated Machine Learning library that enables searching for the best classification and regression model using Evolutionary Algorithms, and AutoPyTorch [43], an automating Machine Learning library for the PyTorch [44] deep learning framework. Pipelines found by TPOT or AutoPyTorch are usually better than those found by grid search or manual construction. TPOT was used for experiments with word vectors, and AutoPyTorch was used for experiments with one-hot encoded word images.

We applied the following parameters: for TPOT, default ones for evolutionary algorithm; 20 for number of generations and 5 for population size; 10 folds for cross-validation; mean absolute error as the scoring function, and “TPOT light” as the configuration dictionary, for AutoPyTorch, default validation setup, “tiny-cs” config for participant-independent trials and “medium-cs” config for participant-dependent trials.

Prior to training, for each experiment, we have randomly extracted 10% of the dataset and used it as a hold-out set for test. We trained our AutoML on the rest 90% with the respective cross-validation schemes and we examine the generalization ability of a model by assessing the difference between the quality metrics on training and hold-out sets. The model could perform very well on training set, but show lack of predictive ability on the hold-out. Such an outcome would be a sign of poor generalization ability which would show that based on this data we can not infer the existence of learnable connection between features and a label.

3 Results

3.1 ANOVA

Accuracy was significantly above chance for all groups, t > 3.88, p < .005. Levene’s test (F > 2.84, p < .042) and frequency distributions suggested that most data did not follow a normal distribution. Although non-parametric statistics may be more appropriate than an ANOVA, to enable comparison with the original effect sizes, corrected parametric statistics (Brown-Forsythe’s F statistic) are reported in text for corresponding dependent measures. There were no significant differences between groups in recognition accuracy , response bias , or average reaction time . To enable comparison with the original paper, we report the results of multiple ANOVAs. However, to control for multiple tests and the difference in age between groups, we analyzed all dependent variables including AUROC with a MANOVA, Pillai’s trace (12, 294) = .166, p = .148, and recognition accuracy and average reaction time with age as a covariate, F(3, 98) = .097 and F(3, 98) = .557, respectively. There was no significant difference between groups with hits, false alarms, recognition accuracy, and average reaction time as dependent variables. There was also no significant difference in the proportion of hits , or associated reaction times . There was a significant difference in proportion of false alarms F(3, 93) = 2.85, p = .042 but not associated reaction times . Planned contrasts for false alarms revealed significant differences betzween VLPFC tDCS and Sham t(93) = 2.78 and between Offline DLPFC tDCS and Sham t(93) = 2.14, p = .018, with higher false alarm rates for VLPFC tDCS and Offline DLPFC tDCS. There were no significant differences between Online tDCS and Sham t(93) = 1.36, p = .178. See Table 1 for means and standard deviation for the mean accuracy for each group and Table 2 for means and standard deviation for reaction times.

Table 1

Means and standard deviations for memory accuracy across groups.

	N	Discrimination Pr	Br response bias	Pr hits	Pr false alarms
Online DLPFC mean	24	0.15	0.61	0.68	0.53
Online DLPFC SD	24	0.16	0.14	0.10	0.18
Offline DLPFC mean	25	0.07	0.60	0.63	0.57
Offline DLPFC SD	25	0.12	0.12	0.12	0.14
VLPFC mean	23	0.16	0.56	0.63	0.47
VLPFC SD	23	0.18	0.17	0.16	0.18
Sham mean	25	0.09	0.66	0.69	0.59
Sham SD	25	0.11	0.12	0.11	0.12

Table 2

Means and standard deviations for reaction time accuracy across groups.

	N	Average RT	RT hits	RT false alarms
Online DLPFC mean	24	501.49	504.82	498.17
Online DLPFC SD	24	144.91	151.20	139.84
Offline DLPFC mean	25	503.00	505.85	500.16
Offline DLPFC SD	25	103.62	104.71	103.66
VLPFC mean	23	535.72	532.52	538.91
VLPFC SD	23	156.16	142.72	171.43
Sham mean	25	484.00	477.82	484.00
Sham SD	25	82.58	86.10	82.58

3.2 AUROC analysis and ML models

The range of performance as measured by AUROC (0.38-0.62; see Fig 1) indicates that participant decisions were close to random choice, since an AUC coefficient of 0.75 or higher reliably reflects accurate performance. The VLPFC group shows a similar distribution of individual AUROC coefficients to the Sham group, indicating that there was no effect of tDCS on memorization and perhaps other factors (word characteristics, individual differences, error) contributed more highly.

Fig 1

Violin plots of AUC distributions for Russian vs English participants in each group.

Panel A shows the Russian sample (p-value of median test is 0.267) and Panel B shows the English sample (p-value of median test is 0.17).

Although the distributions of AUROC coefficients for individual words appeared to vary depending on the sample (English or Russian) and group (VLPFC or Sham; see Fig 2), with the English VLPFC group showing more variance than either of the Sham groups, there was no significant difference in AUROC distributions between English or Russian words or experimental groups, suggesting that the greater variance in the English group reflected smaller sample size.

Fig 2

Violin plots showing AUROC distributions for Russian vs English words in each group.

A—all groups (p-value of median test is 0.004), B—sham (p-value of median test is 0.274), C—VLPFC online (p-value of median test is 0.001).

We found significant differences in distributions for reaction time between Russian and English words as well as significant differences in distributions between sham and VLPFC online across both samples, p < .001, but no significant difference in median AUROC (p-values are either way larger than 0.001 or borderline—about 0.001-0.004), suggesting that as for the accuracy data, differences in distribution reflected differences in sample size (see Fig 3).

Fig 3

Reaction time distribution for English and Russian words.

Finally, we found no significant contributions of semantic, orthographic, or phonological characteristics of words in terms of predicting the success of participant in recalling the word for participant-independent (see S3 Appendix) or participant-dependent (see S4 Appendix) models. The Spearman correlations for predictions and real labels for hold-out set in prediction of word AUROC were close to zero, the AUROCs of participant-independent models for hold-out set were close to 0.5 with small standard deviation, suggesting that the model did not learn anything useful for prediction of experiment outcome.

4 Discussion

The aim of the current study was to replicate and extend the findings of [20](Experiment 1) with the VLPFC and DLPFC. However, the effect of tDCS over the left VLPFC (decrease in false recognition) was not replicated, and there were weak, negative effects over the DLPFC in the Offline group. In fact, there was a trend toward increased false alarms after online tDCS over the VLPFC and offline tDCS over the DLPFC compared to Sham. The trend suggests that an increase in false alarms obscured group differences, and the increase in false alarms could be driven by an increase in semantic elaboration that leads to better memory for features common to multiple items [45, 46]. While the results do not support effects over the DLPFC and VLPFC, this speaks to the lack of tDCS effect rather than the lack of involvement of these regions in episodic memory. Although it is clear from TMS studies that the VLPFC is necessary for episodic memory, TMS disruption of the DLPFC does not consistently impair episodic memory and the specific roles of both regions remain to be clarified. While meta-analyses and studies aimed at replication of tDCS cognitive effects do not support the effectiveness of tDCS generally [47, 48], the majority of tDCS studies lack deeper explorations of non-linear effects and individual differences through modelling and ML [49]. In our case, a limitation was not estimating whether maximum current flow was underneath the anode electrode and at the region of interest (VLPFC or DLPFC). However, the aim of the current study was to clarify the consistency of the effect found in the original study using an identical method, including montage. Future studies could test a montage with greatest likelihood of current flow at the region of interest by modeling current density maxima. More insight is needed into the relationship between biological and cognitive effects. For example, future studies could attempt to classify groups by using concurrent EEG activity during tDCS administration. However, even tDCS-EEG studies alone may not be beneficial because EEG reflects the synchronized firing of large populations of neurons across the brain (not just the stimulated region), and the neural signature is not always modulated by changes in cognitive performance. Thus, more sensitive measures of tDCS cognitive and neural effects can be implemented, such as changes in resting-state connectivity [50], GABA and glutamate concentrations [51], and cerebral blood flow [52]. It remains unclear whether the significant tDCS modulation of long-term verbal encoding in the original study occurred through modulation of related processes such as lexical retrieval, working memory, or attention. For example, tDCS has been shown to modulate the speed of vocabulary learning but not final vocabulary recall over multiple sessions [53]. Moreover, atDCS over the VLPFC led to improved semantic processing in language tasks [32] such as naming common objects [54] and working memory tasks [55]. Thus, tDCS could modulate a semantic process or working memory maintenance that contributes to long-term memory rather than the formation of the memory trace, which is thought to rely more on the hippocampus [56, 57]. Individual differences in baseline memory performance, correlated cognitive functions, and neuroanatomy could obscure the effect of tDCS on measures of memory performance, especially if the effect is subtle. It is clear that effects of tDCS on physiology may not correspond with cognitive effects, and future studies can better explore this relationship with physiological as well as cognitive measures and by estimating the current density maxima. It is notable that using AUROC we could not discriminate hard and easy to remember words in Experiment 1 of [20]: the AUC curve revealed no significant effect of VLPFC tDCS on English or Russian speakers. Although average memory accuracy was above chance, AUC suggested that when considering individual trials, participants were guessing. Future studies can test the reliability of the recognition test as an assessment for tDCS-induced cognitive effects and try to increase participant motivation and concentration to attain optimal trial-by-trial performance for examining tDCS-induced changes. The ML analysis showed that phonological, semantic, and orthographic features did not influence the episodic memory. Although it is likely that these features contributed to memory formation, their influence was minimal and overshadowed by the large interaction between atDCS and individual differences. This individual variability has been examined in other studies that found differences between high and low performers (e.g. [10]). Individual differences in baseline memory performance and encoding ability appear to be an influential factor, more than language or word characteristics. The results indicate that individual differences may be one of the most important parameters not only in the current study and tDCS field but also in replication studies. A limitation was that we did not conduct a within-subjects study in which we examined differences in individual performance. A replication of a within-subjects study could reveal that the effect of a between-subjects study was due to differences between participants and when participants are compared to their own performance, there is little effect of tDCS. Indeed, studies including [20] have found varying effects when comparing between-subjects to within-subjects tDCS effects over the same location, with potentially smaller or absent effects for within-subjects studies (e.g. compare between-subjects results in [30, 32, 58, 59] and Experiment 1 in [20] to within-subjects results in Experiment 3 in [20]; Experiment 2 in [60] and [14]. Furthermore, it is important to identify the reliability of tDCS effects with direct replications within the same lab as well as other labs [61] with sufficiently large sample sizes. However, continued future examinations of tDCS cognitive effects may have less value if not supplemented with a measure of biological effects. Future data-driven studies should aim to predict when physiological effects such as increases in BOLD activation lead to cognitive outcomes such as higher performance and whether duration and frequency of tDCS are involved. Although the sample size of the current study should have had sufficient statistical power to detect the original effect size ([20], Experiment 1), the true effect size could still be eluded because of a statistical phenomenon known as the “winner’s curse”. The winner’s curse posits that the first studies to find a significant and novel effect will be published, and the reported effect will be exaggerated because these studies tend to be exploratory and include smaller sample sizes. If several small-sample studies are investigating the same effect, random error and sampling variation may lead to one but not all of the studies finding an effect that crosses significance threshold because under-powered studies can only find large effects. The true effect is likely to be smaller, so it would not otherwise emerge in under-powered studies. This phenomenon is illustrated well in the meta-analysis by [2], in which most included articles reported at least one significant effect, but the average effect size was close to zero when all studies were included in the analysis. For example, [62] found an effect of atDCS in their first experiment but not in a subsequent replication in the same paper. The conclusion of [63] is in line with the “winner’s curse” effect, while the authors suggest a smaller sample size (between d = .40 and d = .50 according to Cohen, 1988) as more appropriate for tDCS studies. We did not successfully replicate the results of the previous work, although we used a larger sample size and nearly identical method. It is worth mentioning that the results of the replicated study are not significant based on the ANOVA (and this statistical model is appropriate for the original study) or the AUROC analysis applied to both datasets. It does not seem that there was a cultural or linguistic component involved, in line with previous tDCS experiments that found similar effects on verbal memory performance between countries (Italy: [32]; England: [20]; USA: [59]). However, culture-dependent tDCS should be tested directly by comparing individuals of different cultures in the same language. We would expect culture-dependent effects in social cognition (e.g. [53]) but not in processes such as memory encoding that are thought to rely on the same neural architecture across people. In the current study, tDCS did not lead to significant differences in memory performance or reaction time. While it seems unlikely that the significant effect in the original study was due to sample size alone, we conclude based on the lack of replication that the effect of tDCS on long-term verbal memory when the anode is placed over the VLPFC or DLPFC may be subtle. Moreover, the effect may be completely obscured during interactions with other factors that may be difficult to measure or quantify, such as the state of the network. Regression approaches could be successful for future studies that model interactions between biological and cognitive effects. In spite of the significant tDCS effects found in previous memory studies, we suggest caution in interpreting these effects and applying tDCS as a neuromodulator until replications are conducted with biomarkers.

Significantly different words.

Russian and English words that differ significantly in the number of hits, false alarms, AUROC and reaction time.

(PDF)

Click here for additional data file.

Data augmentations.

The procedures used to augment the training set according to the method proused in [41].

(PDF)

Click here for additional data file.

Participant-independent ML trials.

The performance of participant-independent TPOT and AutoPytorch models.

(PDF)

Click here for additional data file.

Participant-dependent ML trials.

The performance of participant-independent TPOT and AutoPytorch models.

(PDF)

Click here for additional data file.

20 Jul 2020

PONE-D-20-16976

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

PLOS ONE

Dear Dr. Kirillov,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration by 2 Reviewers and an Academic Editor, all of the critiques of both Reviewers must be addressed in detail in a revision to determine publication status. If you are prepared to undertake the work required, I would be pleased to reconsider my decision, but revision of the original submission without directly addressing the critiques of the two Reviewers does not guarantee acceptance for publication in PLOS ONE. If the authors do not feel that the queries can be addressed, please consider submitting to another publication medium. A revised submission will be sent out for re-review. The authors are urged to have the manuscript given a hard copyedit for syntax and grammar.

==============================

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The aim of the this study conducted by Petrovskaya et al. was to use tDCS to replicate and extend the findings of Medvedeva et al. (2019)](Experiment 1) with the anode over VLPFC and DLPFC. However, the effect of tDCS with the anode over the left VLPFC (decrease in false recognition) was not replicated, and there were weak, negative effects with the anode over the DLPFC in the Offline group. In fact, there was a trend toward increased false 309 alarms after online active tDCS over the VLPFC and offline active tDCS over the DLPFC compared to Sham. The trend suggests that an increase in false alarms obscured group differences, and the increase in false alarms could be driven by an increase in semantic elaboration that leads to better memory for features common to multiple items. While the results do not support effects of tDCS with the anode over the DLPFC and VLPFC, this speaks to the lack of tDCS effect rather than the lack of involvement of these regions in episodic memory.

The manuscript is well written and I like the approach to investigate cognitive effects with machine learning, but there are some issues that need to be addressed.

The authors stated that tDCS was applied over VLPFC and DLPFC but only the anode was placed over these brain areas. I do not see any evidence that this electrode montage targeted the VLPFC and DLPFC. The current density maxima seems to be between the electrodes. Please provide evidence that the montage targets VLPFC and DLPFC. For instance, HD-target software (Soterix medical) use to determine the optimal electrode configuration for your chosen brain target shows that in order to target the Left DLPFC the anode should be over C3 and the cathode over AF3 according to 10-20 EEG electrode positioning system. If you do not have evidence you should mention this issue as a limitation.

Please state that the anode was applied over VLPFC and DLPFC instead of anodal tDCS.

Reviewer #2: This is a well-written replication study of the effect of tDCS on memory encoding. Such studies are important to verify the claims in the brain stimulation literature, particularly as there have been some criticisms of the field in recent years (e.g. Horvath et al. meta-analysis cited by the authors of the present study). My only major concern is regarding the interpretation of results:

Page 10, Line 330 – “Nonetheless, the results of the current study support the conclusion that tDCS does not modulate memory in the VLPFC and DLPFC directly but rather a subprocess leading to successful memory formation and retrieval.”

It's not clear that this can be claimed on the basis of the present findings. There was no direct examination of subprocesses leading to successful memory formation and retrieval. Rather, results appear to largely conform with the null hypothesis. Indeed, the significant ANOVA findings for false alarms could be somewhat questioned, given multiple ANOVAs were conducted (7 in total), which can inflate the false positive rate in much the same way that conducting multiple pairwise comparisons can. Correcting for multiple tests, by using a MANOVA for example, would likely leave the false alarm results non-significant. If this is indeed the case, one would have to conclude the the current study failed to replicate the previous findings, that the stimulation conditions did not significantly differ on any of the outcome variables, and that tDCS to the DLPFC or VLPFC was not observed to improve memory encoding.

Page 11, Line 400 – “We conclude that tDCS may exert a subtle modulation that also interacts strongly with individual differences, particularly in baseline activation and neuroanatomy.”

Similar to previous point, it’s not clear what empirical data obtained from the current study leads to the conclusion that individual differences, including baseline activity and neuroanatomy, may alter findings for tDCS and memory encoding. While this can be presented speculatively as an important variable/s to consider and investigate in future research, it’s not clear that these factors are at all relevant to the current findings. It may well be possible, at least theoretically, that they are irrelevant and tDCS exerts little to no reliable effect on memory encoding in healthy individuals.

Minor issues

Abstract – sample size would be useful information to include in the Abstract.

Abstract – Acronyms for VLPFC and DLPFC not explicitly defined in the abstract.

Page 1, Line 6 – a citation is needed to support the claim that tDCS can decrease reaction time or increase memory accuracy.

Page 2, Line 35 – “…and the left hemisphere in the VLPFC appears selectively engaged…” Somewhat oddly worded, could be rephrased to the left VLPFC

Page 3, Line 42 – “In one tDCS study, [27] found that cathodal stimulation over the left DLPFC (F3) in a cued-word-completion learning task where errors were evoked through guessing the wrong completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect).”

This sentence is difficult to parse, and may need to be presented in two sentences to better convey the meaning.

Page 3, Line 51 – “The current study involved administering atDCS over the left DLPFC offline or online encoding”. I believe there is a missing word, ‘during’, between DLPFC and offline.

Methods – A few additional details should be provided in the Methods. What was the specific timing of the stimulation relative to the task in the offline condition (e.g. did one immediately follow the other)? What was the duration of stimulation for active tDCS? This is described in citation [1], but could be repeated in the present manuscript.

Page 4, Line 113 – Minor typo of ‘ae’ instead of ‘age in “Bonferroni-corrected pairwise comparisons of ae…”

Considering the significant difference in age between two of the groups (VLPFC online and DLPFC online), the authors could run one-way ANCOVAs, incorporating Age as a covariate, to ensure that this variable doesn’t confound the results.

There is some repetition in the Methods that could be minimised:

Page 5, Line 96 – “However, stimulation was delivered only on the first day and at different sites (DLPFC or VLPFC) and times (offline or online encoding) depending on the group.”

Page 4, Line 137 – “Stimulation was only delivered on the first day, but electrodes were placed on the head on the second day without stimulation (to replicate the method of [1] exactly).”

Page 4, Line 139 – “For all groups, stimulation was delivered on the first day only.”

Page 6, Line 218 – A brief description of what is meant by ‘letter-based one-hot encoding’ would be useful.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Stevan Nikolin

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

==============================

Please submit your revised manuscript by January, 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Stephen D. Ginsberg, Ph.D.

Section Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Your abstract cannot contain citations. Please only include citations in the body text of the manuscript, and ensure that they remain in ascending numerical order on first mention.

3. Please ensure that you refer to Figure 3 in your text as, if accepted, production will need this reference to link the reader to the figure.

4. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 in your text; if accepted, production will need this reference to link the reader to the Table.

18 Nov 2020

Responses to reviewer #1:

Reviewer #1: The aim of the this study conducted by Petrovskaya et al. was to use tDCS to replicate and extend the findings of Medvedeva et al. (2019)](Experiment 1) with the anode over VLPFC and DLPFC. However, the effect of tDCS with the anode over the left VLPFC (decrease in false recognition) was not replicated, and there were weak, negative effects with the anode over the DLPFC in the Offline group. In fact, there was a trend toward increased false 309 alarms after online active tDCS over the VLPFC and offline active tDCS over the DLPFC compared to Sham. The trend suggests that an increase in false alarms obscured group differences, and the increase in false alarms could be driven by an increase in semantic elaboration that leads to better memory for features common to multiple items. While the results do not support effects of tDCS with the anode over the DLPFC and VLPFC, this speaks to the lack of tDCS effect rather than the lack of involvement of these regions in episodic memory.

The manuscript is well written and I like the approach to investigate cognitive effects with machine learning, but there are some issues that need to be addressed.

The authors stated that tDCS was applied over VLPFC and DLPFC but only the anode was placed over these brain areas. I do not see any evidence that this electrode montage targeted the VLPFC and DLPFC. The current density maxima seems to be between the electrodes. Please provide evidence that the montage targets VLPFC and DLPFC. For instance, HD-target software (Soterix medical) use to determine the optimal electrode configuration for your chosen brain target shows that in order to target the Left DLPFC the anode should be over C3 and the cathode over AF3 according to 10-20 EEG electrode positioning system. If you do not have evidence you should mention this issue as a limitation.

Please state that the anode was applied over VLPFC and DLPFC instead of anodal tDCS.

Response: We are glad the reviewer finds the approach and manuscript suitable and thank the reviewer for pointing this out as an important consideration. Modelling approaches are valuable for confirming that the stimulation corresponds to the expected montage. We have stated a lack of evidence for our montage targeting the VLPFC and DLPFC as a limitation. We have also corrected the description of the montage to state “the anode was applied” rather than “anodal tDCS.”

Throughout the manuscript, atDCS was replaced with tDCS in reference to the current study.

Page 4 lines 95-97

Specifically, we applied the same tDCS settings (2 mA anodal active tDCS and 30 s Sham tDCS) over the DLPFC (all conditions: offline, sham, and online) in addition to the VLPFC (online encoding only).

Page 10 lines 342-348

In our case, a limitation was not estimating whether maximum current flow was underneath the anode electrode and at the region of interest (VLPFC or DLPFC). However, the aim of the current study was to clarify the consistency of the effect found in the original study using an identical method, including montage. Future studies could test a montage with greatest likelihood of current flow at the region of interest by modeling current density maxima.

Reviewer #2: This is a well-written replication study of the effect of tDCS on memory encoding. Such studies are important to verify the claims in the brain stimulation literature, particularly as there have been some criticisms of the field in recent years (e.g. Horvath et al. meta-analysis cited by the authors of the present study). My only major concern is regarding the interpretation of results:

Page 10, Line 330 – “Nonetheless, the results of the current study support the conclusion that tDCS does not modulate memory in the VLPFC and DLPFC directly but rather a subprocess leading to successful memory formation and retrieval.”

It's not clear that this can be claimed on the basis of the present findings. There was no direct examination of subprocesses leading to successful memory formation and retrieval. Rather, results appear to largely conform with the null hypothesis. Indeed, the significant ANOVA findings for false alarms could be somewhat questioned, given multiple ANOVAs were conducted (7 in total), which can inflate the false positive rate in much the same way that conducting multiple pairwise comparisons can. Correcting for multiple tests, by using a MANOVA for example, would likely leave the false alarm results non-significant. If this is indeed the case, one would have to conclude the the current study failed to replicate the previous findings, that the stimulation conditions did not significantly differ on any of the outcome variables, and that tDCS to the DLPFC or VLPFC was not observed to improve memory encoding.

Page 11, Line 400 – “We conclude that tDCS may exert a subtle modulation that also interacts strongly with individual differences, particularly in baseline activation and neuroanatomy.”

Similar to previous point, it’s not clear what empirical data obtained from the current study leads to the conclusion that individual differences, including baseline activity and neuroanatomy, may alter findings for tDCS and memory encoding. While this can be presented speculatively as an important variable/s to consider and investigate in future research, it’s not clear that these factors are at all relevant to the current findings. It may well be possible, at least theoretically, that they are irrelevant and tDCS exerts little to no reliable effect on memory encoding in healthy individuals.

Response: We thank the reviewer for the thoughtful consideration of the manuscript and helpful suggestions for clarifying and resolving certain issues. We added a correction for multiple ANOVAs using a MANOVA and tested group differences in recognition accuracy and reaction time with age as a covariate. In line with these results that show no differences between groups, we clarify in the discussion that there may be no significant effects of tDCS on memory performance in the current study. We clarify that while individual differences may interact with tDCS to influence cognitive effects, further work is needed to explore this relationship.

Page 7 Lines 270-277

To enable comparison with the original paper, we report the results of multiple ANOVAs. However, to control for multiple tests and the difference in age between groups, we analyzed all dependent variables including AUROC with a MANOVA, Pillai’s trace (12, 294) = .166, p = .148, and recognition accuracy and average reaction time with age as a covariate, F(3, 98) = .097 and F(3, 98) = .557, respectively. There was no significant difference between groups with hits, false alarms, discrimination ability, and average reaction time as dependent variables.

Page 11

Lines 362-364

It remains unclear whether the significant tDCS modulation of long-term verbal encoding in the original study occurred through modulation of related processes such as lexical retrieval, working memory, or attention.

Lines 370-376

Individual differences in baseline memory performance, correlated cognitive functions, and neuroanatomy could obscure the effect of tDCS on measures of performance, especially if the effect is subtle. It is clear that effects of tDCS on physiology may not correspond with cognitive effects, and future studies can better explore this relationship with physiological as well as cognitive measures and by estimating the current density maxima.

Page 12

Lines 437-443

In the current study, tDCS did not lead to significant differences in memory performance or reaction time. While it seems unlikely that the significant effect in the original study was due to sample size alone, we conclude based on the lack of replication that the effect of tDCS on long-term verbal memory when the anode is placed over the VLPFC or DLPFC may be subtle. Moreover, the effect may be completely obscured during interactions with other factors that may be difficult to measure or quantify, such as the state of the network.

Minor issues

We present parts of the manuscript that address each comment.

Abstract – sample size would be useful information to include in the Abstract.

Abstract – Acronyms for VLPFC and DLPFC not explicitly defined in the abstract.

We aimed to replicate a published effect of transcranial direct-current stimulation (tDCS)-induced recognition enhancement over the human ventrolateral prefrontal cortex (VLPFC) [1] and analyse the data with machine learning. We investigated effects over an adjacent region, the dorsolateral prefrontal cortex (DLPFC). In total, we analyzed data from 97 participants after exclusions. We found weak or absent effects over the VLPFC and DLPFC. We conducted machine learning studies to examine the effects of semantic and phonetic features on memorization, which revealed no effect of VLPFC tDCS on the original dataset or the current data. The highest contributing factor to memory performance was individual differences in memory not explained by word features, tDCS group, or sample size, while semantic, phonetic, and orthographic word characteristics did not contribute significantly. To our knowledge, this is the first tDCS study to investigate cognitive effects with machine learning, and future studies may benefit from studying physiological as well as cognitive effects with data-driven approaches and computational models.

Page 2, Line 6 – a citation is needed to support the claim that tDCS can decrease reaction time or increase memory accuracy.

Transcranial direct-current stimulation (tDCS), a method of safely and non-invasively delivering a weak electric current through the cortex, has been gaining increasing attention [2] as a tool for studying and possibly enhancing episodic memory [3]. Anodal tDCS (atDCS) involves increasing cortical excitability in a target region such as the dorsolateral prefrontal cortex (DLPFC), which may in turn facilitate or enhance memory performance by decreasing reaction time and/or increasing memory accuracy. (3,4)

3. Investigating the Role of Current Strength in tDCS Modulation of Working Memory Performance in Healthy Controls

Florence Teo,1 Kate E. Hoy,1,* Zafiris J. Daskalakis,2 and Paul B. Fitzgerald1

Front Psychiatry. 2011; 2: 45. Published online 2011 Jul 18. doi: 10.3389/fpsyt.2011.00045

PMCID: PMC3141358

PMID: 21811474

4. Shaping Memory Accuracy by Left Prefrontal Transcranial Direct Current Stimulation

Bastian Zwissler,1 Christoph Sperber,1 Sina Aigeldinger,1 Sebastian Schindler,2 Johanna Kissler,2 and Christian Plewnia,3

J Neurosci. 2014 Mar 12; 34(11): 4022–4026.

doi: 10.1523/JNEUROSCI.5407-13.2014

Page 2, Line 35 – “…and the left hemisphere in the VLPFC appears selectively engaged…” Somewhat oddly worded, could be rephrased to the left VLPFC

The VLPFC and DLPFC are thought to play functionally distinct roles in long-term verbal memory, and the left VLPFC appears selectively engaged in verbal but not non-verbal material.

Page 3, Line 42 – “In one tDCS study, [27] found that cathodal stimulation over the left DLPFC (F3) in a cued-word-completion learning task where errors were evoked through guessing the wrong completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect).”

This sentence is difficult to parse, and may need to be presented in two sentences to better convey the meaning.

In a tDCS study conducted by [27], tDCS was applied over the left DLPFC in a cued-word-completion task to modulate memory performance. The results showed that cathodal stimulation inhibited memory accuracy after error learning in comparison with sham. The errors were evoked through guessing, or incorrect completion lowered memory accuracy in contrast to a non-error condition or anodal stimulation in either condition (no effect).

Page 3, Line 51 – “The current study involved administering atDCS over the left DLPFC offline or online encoding”. I believe there is a missing word, ‘during’, between DLPFC and offline.

The current study involved administering atDCS over the left DLPFC during offline or online encoding

Methods – A few additional details should be provided in the Methods. What was the specific timing of the stimulation relative to the task in the offline condition (e.g. did one immediately follow the other)? What was the duration of stimulation for active tDCS? This is described in citation [1], but could be repeated in the present manuscript.

Page 5

Lines 149-154

Stimulation was applied for the entire duration (10 minutes) of the reading task for the DLPFC offline group and for the entire duration (10 minutes) of the study memory task that immediately followed for the VLPFC Online and DLPFC Online groups. In summary, the stimulation was delivered for approximately 10 minutes to cover the entire duration of the reading and memory tasks.

Page 4, Line 113 – Minor typo of ‘ae’ instead of ‘age in “Bonferroni-corrected pairwise comparisons of ae…”

Bonferroni-corrected pairwise comparisons of age

Considering the significant difference in age between two of the groups (VLPFC online and DLPFC online), the authors could run one-way ANCOVAs, incorporating Age as a covariate, to ensure that this variable doesn’t confound the results.

As cited in an earlier response, we conducted an ANCOVA in addition to a MANOVA:

Page 7 Lines 270-277

To enable comparison with the original paper, we report the results of multiple ANOVAs. However, to control for multiple tests and the difference in age between groups, we analyzed all dependent variables including AUROC with a MANOVA, Pillai’s trace (12, 294) = .166, p = .148, and recognition accuracy and average reaction time with age as a covariate (ANCOVA), F(3, 98) = .097 and F(3, 98) = .557, respectively. There was no significant difference between groups with hits, false alarms, discrimination ability, and average reaction time as dependent variables.

There is some repetition in the Methods that could be minimised:

Page 5, Line 96 – “However, stimulation was delivered only on the first day and at different sites (DLPFC or VLPFC) and times (offline or online encoding) depending on the group.”

Page 4, Line 137 – “Stimulation was only delivered on the first day, but electrodes were placed on the head on the second day without stimulation (to replicate the method of [1] exactly).”

Page 4, Line 139 – “For all groups, stimulation was delivered on the first day only.”

We minimized repetition and increased conciseness in the method:

Page 4 Lines 97-101

The electrodes were placed at both sessions (study and test 24 hours later), but stimulation was only delivered before or during the encoding task in the study phase. The anode was placed over the left VLPFC (F7) or the left DLPFC (F3), and the cathode was placed over the ipsilateral (left) shoulder. See section 2.2 below for the experimental conditions.

Page 6, Line 218 – A brief description of what is meant by ‘letter-based one-hot encoding’ would be useful.

Lines 227-229

In line with the trial-based and participant-based approaches, we conducted two kinds of ML studies: 1) Participant-independent (trial-based) analysis tests the success of a model that predicts the AUROC for each word using either word embeddings or letter-based one-hot encoding (each letter i is replaced by a vector of Nalphabet components with the i-th component being equal to 1 while every other component is zero) and 2) Participant-specific (participant-based) analysis tests the success of a model that predicts whether a participant was able to recall the word given either word embeddings or letter-based one-hot encoding for each word for individual participants.

Submitted filename: Petrovskaya_et_al_rebuttal_17_11_20.pdf

Click here for additional data file.

23 Nov 2020

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

PONE-D-20-16976R1

Dear Dr. Kirillov,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Stephen D. Ginsberg, Ph.D.

Section Editor

PLOS ONE

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors addressed all my concerns. Their response yo my questions was hood. I think the paper is ready for publication

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

26 Nov 2020

PONE-D-20-16976R1

Examining the effects of transcranial direct current stimulation on human episodic memory with machine learning

Dear Dr. Kirillov:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Stephen D. Ginsberg

Section Editor

PLOS ONE