Impact of operator experience on transcranial magnetic stimulation.

OBJECTIVE: To determine the impact of an operator's experience on transcranial magnetic stimulation (TMS) measurement.
METHODS: Operator B (beginner), operator E (expert), and 30 healthy participants joined the study consisting of two experiments. In each experiment, each operator performed a TMS protocol on each participant in a random order.
RESULTS: Compared with operator E, operator B exhibited higher resting motor threshold (RMT) in experiment I (60.1 ± 13.0 vs. 57.4 ± 10.9% maximal stimulation output, p = 0.017) and the difference disappeared in experiment II (p = 0.816). In 1-mV motor evoked potential (MEP) measurement, operator B exhibited higher standard deviation indicating lower consistency in experiment I compared with experiment II (1.05 ± 0.40 vs. 1.05 ± 0.16 mV with unequal variances, p = 0.001) and had poor intrarater reliability between the experiments (intraclass correlation coefficient = -0.130). There was no difference in the results of active motor threshold, silent period, paired-pulse stimulation, or continuous theta burst stimulation between the operators.
CONCLUSIONS: An operator's experience in TMS may affect the results of RMT measurement. With practice, a beginner may choose a more precise stimulation location and have higher consistency in 1-mV MEP measurement. SIGNIFICANCE: We recommend that a beginner needs to practice for precise stimulation locations before conducting a trial or clinical practice.

Introduction

Transcranial magnetic stimulation (TMS) has been widely used in clinical practice and research in recent years. TMS is a noninvasive and safe procedure to stimulate the brain. The U.S. Food and Drug Administration has approved the use of repetitive TMS (rTMS) for treating drug-refractory major depression and obsessive–compulsive disorder. In addition, rTMS can be used to treat neurological diseases, such as migraine and dystonia, and increase motor function in patients with chronic stroke (Chen et al., 2019, Erro et al., 2017, Feng et al., 2019). Theta burst stimulation (TBS) is one of the most efficient and widely used stimulation patterns of rTMS (Huang et al., 2005).

Some parameters have been frequently used in TMS studies. Under an adequate stimulation intensity, the use of single-pulse TMS on the motor cortex can generate motor evoked potentials (MEPs) in a target muscle. MEPs of 1 mV are often used in TMS studies as baseline cortical excitability. Motor threshold is the minimum stimulating intensity that can elicit MEPs of a given amplitude when the target muscle is in the resting state (resting motor threshold, RMT) or during contraction (active motor threshold, AMT). Motor threshold can be used to evaluate motor cortical excitability and as a reference for the stimulation intensity in paired-pulse stimulation (PPS) and TBS (Fried et al., 2017). During voluntary contraction of the target muscle, TMS elicits an MEP followed by a pause in the electromyogram, which is called silent period. The silent period can be used to examine intracortical inhibition (Hupfeld et al., 2020). PPS can be used to measure intracortical facilitation or inhibition. The duration of interstimulus intervals (ISIs) determine the outcome of PPS. Short-interval intracortical inhibition (SICI) inhibits MEPs at an ISI of 1–6 ms, whereas intracortical facilitation (ICF) facilitates MEPs at an ISI of 10–15 ms (Kujirai et al., 1993).

Because TMS is widely used, intersubject and intrasubject variabilities in response to TMS are matters of concern. Many studies have reported that intersubject and intrasubject variability in response to rTMS may reduce or even lose the effect of rTMS (Corp et al., 2021, Corp et al., 2020, Ellaway et al., 1998, Hamada et al., 2013). Up to 50% of participants can fail to have an “expected” response to the rTMS protocol (Hamada et al., 2013). In addition to rTMS, single- and paired-pulse TMS exhibit intersubject and intrasubject variability in the response, thus possibly affecting the sensitivity and reproducibility of TMS measurements (Corp et al., 2021, Ellaway et al., 1998, Goldsworthy et al., 2016, Wassermann, 2002). Previous studies reported that multiple factors are associated with the variability in response to rTMS including physiological states of participants and methodological factors. Intake of substances, medications, sleep deprivation, exercise, and arousal state are modifiable factors that may interfere in intersubject or intrasubject variability in the response to rTMS (Guerra et al., 2020, Huang et al., 2017, Mang et al., 2014). Oscillations in the electroencephalogram, motor threshold, and baseline MEP amplitude may be associated with intersubject or intrasubject variability in the response to single- or paired-pulse TMS (Corp et al., 2021, Iscan et al., 2016).

The use of neuronavigation was reported as a predict factor for individual single-pulse TMS amplitude (Corp et al., 2021). Therefore, technical factors should be considered crucial for the variability of results. Allowing the coil to steadily stimulate the target spot at a fixed angle during the entire course is the most ideal condition. However, unlike a robot, a human operator may experience difficulty in maintaining the stability and consistency. To the best of our knowledge, no study has focused on the effects of operators on TMS results. Therefore, we conducted the study and hypothesized that an operator with little experience in TMS may reduce the effect of TMS or increase variability in response to TMS. The results of this study can provide insights into research on TMS and the reference experience level required for operators in clinical TMS applications.

Material and methods

Participants

We recruited 31 healthy adult volunteers aged ≥20 years. Exclusion criteria were pregnancy or any possibility of pregnancy, presence of metals in any part of the body, history of seizure, or family history of epilepsy. All participants provided informed consent before participating in the study. This study was approved by the Institutional Review Board of Chang Gung Medical Foundation. Of the 31 participants, one dropped out of the study due to personal reasons.

Operators

Two operators, operator E and operator B, participated in this study. Operator E was experienced in performing TMS for more than 15 years. Operator B was a well-trained beginner who could independently operate TMS but had no experience in executing a trial of TMS.

TMS facility

We used a figure-of-eight coil with a loop diameter of 70 mm (Magstim Co., UK) for magnetic stimulation. Single-pulse stimulation and PPS were delivered by a Magstim 2002 stimulator, whereas TBS was delivered by a Magstim Rapid2 stimulator. Surface electromyography (EMG) was performed to monitor muscle contraction and record MEPs that responded to TMS on the motor cortex.

Study designs

Experiment I

Both operators performed the same TMS protocol on one participant in a random order in the same period of time. The TMS protocol is shown in Fig. 1 and described as follows. The participant was asked to sit and relax on a chair with a backrest. The lead of the surface EMG was placed on the participant’s right first dorsal interosseous (FDI) muscle. Single-pulse TMS was applied to find the “hot spot,” which is the ideal spot to generate the largest MEP amplitude in the FDI muscle at the left motor cortex. The lowest stimulation intensity required to generate an MEP amplitude not less than 50 μV at least five times was determined when stimulation was applied on the hot spot 10 times. The stimulation intensity was defined as RMT. The participant was asked to voluntarily contract the right FDI muscle and maintain the contraction. Under this condition, the lowest stimulation intensity required to generate an MEP amplitude not less than 200 μV at least five times was determined when stimulation was applied on the hot spot 10 times. The stimulation intensity was defined as AMT. The stimulation intensity required to generate an MEP amplitude of approximately 1 mV in a relaxed state was determined and defined as 1-mV MEP. This stimulation intensity was used to generate and record MEPs 20 times at an interval of 5 s. If the participant did not relax during stimulation, data were rejected until all twenty 1-mV MEPs were recorded. PPS was applied to achieve SICI or ICF. Two Magstim 2002 stimulators were connected to a Bistim module to apply PPS. A 1-mV MEP stimulation intensity was used as the test pulse and 80% AMT as the conditioning pulse. The stimulation program consisted of four patterns that appeared in a random order: a single test pulse and PPS with an ISI of 3, 7, and 10 ms. The stimulation was performed eight times in total for each pattern. An interval of 4.5–5.5 s was maintained between stimulations. Both operators were in the same room but blind to the localization of hot spots marked by each other.

Fig. 1

Protocols of experiment I and experiment II. (A) In experiment I, two operators performed the same TMS protocol on one participant in a random order in the same period of time. (B) experiment II, two operators performed the same TMS protocol for each participant at the same time on the same day of the week in 2 consecutive weeks. RMT = resting motor threshold; AMT = active motor threshold; MEP = motor evoked potential; PPS: paired-pulse stimulation; SP = silent period; cTBS = continuous theta burst stimulation.

Experiment II

TBS was included in this part of the measurement. Because the effect of TBS may last for days, participants did not receive TBS twice in a day. To minimize intrasubject variability, operators performed the same TMS protocol for each participant at the same time on the same day of the week in 2 consecutive weeks. Each TMS measurement was performed by one operator in a random order. One operator was not in the room when the other one was performing TMS. And both operators were blind to the results performed by each other. All participants were asked to perform their regular daily activities in these 2 weeks. The protocol applied is shown in Fig. 1 and described as follows. The intensities of RMT, AMT, and 1-mV MEP were sequentially determined; these were also determined in experiment I. The participants were asked to voluntarily contract the right FDI muscle and maintain the contraction. Then, a 1-mV MEP stimulation intensity was used to stimulate the hot spot during right FDI muscle contraction. The silent period was defined as the interval from the start of the MEP to the end of the pause of EMG activities following the MEP. The silent period was recorded 10 times with an interval of 5 s. Then, 1-mV MEPs were recorded 20 times at an interval of 5 s, and the average of data was used as the baseline MEP. If the participants did not relax during stimulation, data were rejected until all 20 MEPs were recorded. Then, 80% AMT was used to generate continuous TBS (cTBS) on the hot spot for a total of 300 pulses (Huang et al., 2005). Under the stimulation intensity of 1-mV MEP, the MEP after cTBS was recorded 12 times immediately and then every 5 min until 30 min.

Measurements before TMS

Participants recorded their daily activities from a week before every TMS measurement. The records included the amount and date of the intake of caffeine, alcohol, nicotine, and medication in the past 7 days; the duration and intensity of exercise in the past 3 days; the duration and quality of sleep the night before TMS measurement; and the working hours, pain, and physiological condition before undergoing TMS on the day of TMS measurement. Sleep quality was measured using a self-reported scale ranging from 1 (poorest) to 5 (best). In experiment II, in addition to the aforementioned records, we used the visual analogue scale to evaluate the level of pain, the EVEA scale to assess mood, and the Karolinska sleepiness scale (KSS) to evaluate the level of alertness. The 16-item EVEA scale is used to assess transitory moods. Each item is rated from 0 (not at all) to 10 (high). All the items are classified into four subscales (each subscale including four items): anxiety, happiness, anger–hostility, and sadness–depression (Sanz, 2001). The KSS is a nine-point scale used to evaluate participants’ alertness from 1 (extremely alert) to 9 (very sleepy, great effort required to keep awake, or fighting sleep) (Kaida et al., 2006).

Statistical analysis

All statistical analyses were performed using SPSS. The results for the silent period, 1-mV MEP amplitude, and MEPs after PPS and cTBS are shown as the grand average of the individual average data of repetitive measurements (e.g., grand average of 30 individual averages of twenty 1-mV MEP amplitudes). The paired t-test was used to compare results between operator B and operator E, the results of PPS and cTBS with controlled MEPs, and the pre-TMS condition of participants between the measurements. The McNemar test was performed to compare the categorical variables of the participants. Two-way analysis of variance (ANOVA) was used to compare the results of PPS and cTBS between the operators. Statistical significance was set at p < 0.05. A responder to cTBS was operationally defined as a participant with a grand average of post-cTBS MEP of <1 (Hamada et al., 2013).

Variability and reliability analyses

We calculated the standard deviation (SD) and intraclass correlation coefficient (ICC) to analyze variability and reliability in response to TMS, respectively. The ICC is widely used to analyze intrarater and interrater reliability (McGraw and Wong, 1996). We calculated the ICC of RMT and AMT (two-way mixed model, absolute agreement type, and single measurements), silent period, and the amplitude of MEPs after PPS and cTBS (two-way mixed model, absolute agreement type, and average measurements) between the operators to analyze interrater reliability. For intrarater reliability, we calculated the ICC of RMT and AMT (two-way mixed model, absolute agreement type, and single measurements) and 1-mV MEP amplitude (two-way mixed model, absolute agreement type, and average measurements) between experiment I and II for each operator (Koo and Li, 2016, Shrout and Fleiss, 1979). An ICC value of <0.5, between 0.5 and 0.75, and >0.75 indicates poor, moderate, and good reliability, respectively (Portney and Watkins, 2009). Levene’s test was used to assess if the results of the TMS measurement have equal variances.

Results

Comparison of TMS measurements between two operators

A total of 30 right-handed participants completed both experiments I and II. As shown in Table 1, in experiment I, the mean RMT performed by operator B was significantly higher than that performed by operator E (p = 0.017); however, no difference in the mean AMT was evident between the two operators (p = 0.076). Because of the limitation of the stimulator, the MEP amplitude could not reach 1 mV under the maximal stimulation intensity in four participants. Three of them could not reach 1 mV MEP amplitude in both operators’ performance and one of them could not reach 1 mV only in operator B’s performance while operator E used 99% maximal stimulation output (MSO). We excluded the data of these four participants when analyzing the average amplitude of 1-mV MEP. The average amplitude of MEP of the remaining 26 participants had a significant difference between the two operators (p = 0.018). Regarding PPS, after normalization to the MEP amplitude of the single-test pulse, the MEP amplitude of ISI at 3 ms (SICI) performed by operator B and operator E significantly decreased compared with the control MEP (p = 0.008 and p < 0.001, respectively). Furthermore, the MEP amplitude of ISI at 7 and 10 ms (ICF) performed by both operators significantly increased compared with the control MEP (p = 0.002 and p = 0.002, respectively, for operator B; p = 0.003 and p = 0.001, respectively, for operator E). No difference in the comparison of the normalized MEP amplitude of SICI and ISI at 7 ms and ICF was observed between operator B and operator E (p = 0.121, p = 0.1, and p = 0.811, respectively). The results of two-way ANOVA revealed no differences in PPS results between the two operators [OPERATOR × ISI interaction, F (2, 58) = 0.741, p = 0.481; OPERATOR effect, F (1, 29) = 2.155, p = 0.153].

Table 1

Results of TMS measurements between operator B (beginner) and operator E (expert).

	Operator B	Operator E	P value
Experiment I
RMT (%MSO, mean ± SD)	60.1 ± 13.0	57.4 ± 10.9	0.017
AMT (%MSO, mean ± SD)	40.6 ± 7.9	42.1 ± 8.4	0.076
1-mV MEP amplitude (mV, mean ± SD) a	1.05 ± 0.40	1.22 ± 0.28	0.018
MEP in PPS (normalized to single test pulse)
SICI (mV, mean ± SD)	0.81 ± 0.37	0.70 ± 0.38	0.121
ISI at 7 ms (mV, mean ± SD)	1.34 ± 0.56	1.18 ± 0.30	0.1
ICF (mV, mean ± SD)	1.33 ± 0.54	1.31 ± 0.45	0.811
Experiment II
RMT (%MSO, mean ± SD)	48.9 ± 8.7	49.4 ± 8.7	0.816
AMT (%MSO, mean ± SD)	53.2 ± 7.7	55.6 ± 9.3	0.241
1-mV MEP amplitude (mV, mean ± SD) b	1.05 ± 0.16	1.10 ± 0.21	0.355
Silent period (ms)	137.9 ± 28.64	142.4 ± 31.88	0.443
MEP amplitude post cTBS (normalized to baseline MEP)
0 min (mV, mean ± SD)	0.94 ± 0.32	0.91 ± 0.30	0.697
5 min (mV, mean ± SD)	0.89 ± 0.41	0.94 ± 0.43	0.687
10 min (mV, mean ± SD)	0.87 ± 0.31	0.86 ± 0.29	0.883
15 min (mV, mean ± SD)	0.91 ± 0.44	0.88 ± 0.37	0.78
20 min (mV, mean ± SD)	1.01 ± 0.65	0.79 ± 0.24	0.097
25 min (mV, mean ± SD)	0.84 ± 0.33	0.73 ± 0.26	0.157
30 min (mV, mean ± SD)	0.82 ± 0.48	0.72 ± 0.27	0.381

a: Exclude 4 participants whose MEP amplitude could not reach 1 mV under maximal stimulation intensity. b: Exclude 2 participants whose MEP amplitude could not reach 1 mV under maximal stimulation intensity. Abbreviations: RMT = resting motor threshold; AMT = active motor threshold; MSO = maximal stimulation output; MEP = motor evoked potential; PPS: paired-pulse stimulation; SICI = short interval intracortical inhibition; ISI = interstimulus intervals; ICF = intracortical facilitation; cTBS = continuous theta burst stimulation.

We planned to start experiment II after all the 30 participants finished experiment I. However, due to COVID-19 pandemic, we had difficulty in enrolling volunteers. Both operators started experiment II after 28 participants completed experiment I. In experiment II, we compared the conditions of participants before receiving TMS between the two visits. The results are shown in Table 2. Participants were slightly more alert before TMS performed by operator E than operator B (KSS score = 5.37 vs. 5.53, p = 0.023). When individual data were reviewed, the difference in the KSS score was not more than 1 for every participant between the two visits. Furthermore, no differences in participants’ pain, mood, sleep duration and quality the night before TMS, working hours in the day, exercise, and substance intake (caffeine and nicotine) on the day were observed between the two visits.

Table 2

Pre-TMS conditions of participants in experiment II.

	Operator B	Operator E	P value
Pain	0.60 ± 1.39	0.34 ± 0.88	0.397
Mood (EVEA scale)
Anxiety	4.87 ± 6.11	4.00 ± 5.02	0.160
Happiness	18.43 ± 8.91	18.67 ± 8.45	0.779
Anger-hostility	1.90 ± 3.61	2.70 ± 5.04	0.219
Sadness-depression	2.40 ± 3.90	2.27 ± 4.74	0.876
Arousal state (KSS)	5.53 ± 0.78	5.37 ± 0.81	0.023
Duration of sleep the night before (hours)	6.63 ± 0.88	6.28 ± 1.68	0.205
Quality of sleep the night before (1–5)	3.33 ± 0.76	3.57 ± 0.86	0.229
Working hours on the day (hours)	3.01 ± 2.03	3.13 ± 2.02	0.491
Exercise on the day (% participants)	6.67	6.67	1
Caffeine on the day (% participants)	50	60	0.375
Nicotine on the day (% participants)	3.33	3.33	1

KSS = the Karolinska sleepiness scale.

In experiment II, the mean RMT and mean AMT did not differ between operator B and operator E (Table 1). The amplitude of MEP could not reach 1 mV under the maximal stimulation intensity in two participants. One could not reach 1 mV MEP amplitude in both operators’ performance and the other one could not reach 1 mV only in operator B’s performance while operator E used 96% MSO. After removing the data of these two participants, the average amplitude of 1-mV MEP for the remaining 28 participants did not differ between the operators (p = 0.355). In addition, no difference in the silent period was observed between operator B and operator E (p = 0.443). The results of cTBS are shown in Table 1 and Fig. 2. After being normalized to baseline MEP, the amplitude of MEPs at 0, 5, 10, 15, 20, 25, and 30 min post cTBS showed no differences between operator B and operator E. The results of two-way ANOVA revealed no differences in cTBS results between the two operators [OPERATOR × TIME interaction, F (5.192, 150.574) = 1.481, p = 0.197; OPERATOR effect, F (1, 29) = 0.632, p = 0.433]. The number of cTBS responders was nonsignificantly higher for the performance of operator E than that of operator B (25 vs. 21, p = 0.157).

Fig. 2

Results of continuous theta burst stimulation. Compared with baseline MEP, the normalized MEP exhibited a significant decrease at 10, 25, and 30 min post cTBS in the performance of operator B and at 10, 20, 25, and 30 min post cTBS in the performance of operator E. *: in Operator B’s performance, MEP at the timepoint had significant decrease compared with baseline MEP, p value < 0.05. †: in Operator E’s performance, MEP at the timepoint had significant decrease compared with baseline MEP, p value < 0.05. MEP = motor evoked potential; cTBS = continuous theta burst stimulation.

Variability and reliability of operators

The variability and reliability results of operator B and operator E are listed in Table 1 and Table 3. In the RMT measurement, operator B and operator E had good interrater reliability in both experiment I and II. However, the intrarater reliabilities of both operators were moderate. In the AMT measurement, operator B and operator E had good and moderate interrater reliability in experiment I and experiment II, respectively. However, both operator B and operator E exhibited low intrarater reliability in the AMT measurement between experiment I and experiment II. In the 1-mV MEP measurement, experiment I performed by operator B exhibited the highest SD, indicating the lowest consistency between the two experiments performed by both operators. Operator B had unequal variances in 1-mV MEP measurement between experiment I and II (p = 0.001) whereas other comparisons of 1-mV MEP amplitude had equal variances, such as experiment I vs. II performed by operator E (p = 0.091), operator B vs. E in experiment I (p = 0.154), and operator B vs E in experiment II (p = 0.613). This finding was also reflected in the intrarater reliability of 1-mV MEP that was considerably lower for operator B than operator E (ICC = −0.130 vs. 0.570). For the silent period, operator B and operator E had good interrater reliability. In the PPS measurement, the interrater reliability was moderate in SICI but low in ICF. After cTBS, the interrater reliability was low in the post-cTBS MEP measurement.

Table 3

Reliability of operator B (beginner) and operator E (expert).

	Interrater reliability (95% CI)	Intrarater reliability (95% CI)
		Operator B	Operator E
RMT
Experiment I	0.863 (0.704, 0.936)
Experiment II	0.805 (0.630, 0.902)
Experiment I vs. II		0.526 (−0.217, 0.903)	0.622 (−0.069, 0.868)
AMT
Experiment I	0.849 (0.702, 0.925)
Experiment II	0.668 (0.410, 0.827)
Experiment I vs. II		0.325 (−0.076, 0.698)	0.345 (−0.082, 0.714)
1-mV MEP amplitude
Experiment I vs. II a		−0.130 (−1.632, 0.503)	0.570 (0.088, 0.803)
Silent period	0.837 (0.661, 0.922)
PPS
SICI	0.635 (0.252, 0.824)
ISI at 7 ms	0.486 (−0.038, 0.750)
ICF	0.414 (−0.258, 0.724)
MEP amplitude post cTBS	0.389 (0.201, 0.533)

Reliability is presented by the intraclass correlation coefficient. a: Exclude 4 participants whose MEP amplitude could not reach 1 mV under maximal stimulation intensity in experiment I or/and II. Abbreviations: CI = confidence interval; RMT = resting motor threshold; AMT = active motor threshold; MEP = motor evoked potential; PPS: paired-pulse stimulation; SICI = short interval intracortical inhibition; ISI = interstimulus intervals; ICF = intracortical facilitation; cTBS = continuous theta burst stimulation.

Discussion

The results of this study indicated that the RMT performed by operator B in experiment I was higher than that performed by operator E; however, the difference was not observed in experiment II. In 1-mV MEP measurement, the average amplitude of MEP performed by operator E was higher than that performed by operator B in experiment I. And operator B had poor intrarater reliability in 1-mV MEP measurement with lower consistency in experiment I compared with experiment II. We discuss the results and variabilities of TMS measurements performed by operators as below.

RMT

RMT performed by operator B in experiment I was higher than that performed by operator E. Braack et al. found that individual RMT showed no significant change during the day (Ter Braack et al., 2019). Thus, the higher RMT measured by operator B may be attributed to the operator per se. A more precise location on the motor cortex can require a lower stimulation intensity to generate the same MEP amplitude (Corp et al., 2021). Therefore, these findings suggest that operator E might have chosen a more precise hot spot than operator B in experiment I. In experiment II, no difference in RMT was observed between both operators. This finding indicates that operator B found the hot spot as precisely as operator E. RMT plays an important role in TMS studies. For example, compared with normal controls, patients with Alzheimer disease and juvenile myoclonic epilepsy have lower RMT (Brigo et al., 2012, Freitas et al., 2011). RMT reflects membrane excitability of corticospinal neurons and interneurons projecting onto the neurons in the motor cortex (Kobayashi and Pascual-Leone, 2003). Less precise RMT measurement could further mislead the results of studies.

1-mV MEP

While analyzing the mean amplitude of 1-mV MEP, we excluded the data of four and two participants whose MEP amplitudes could not reach 1 mV under the maximal stimulation intensity in experiment I and II, respectively. Tracing the preliminary data, both two excluded participants in experiment II were also excluded in experiment I. One participant in each experiment could not reach 1 mV under 100% MSO only in operator B’s performance but not in operator E’s (99% and 96% MSO, respectively). This finding suggests that all the excluded participants had high threshold. However, operator E still could perform 1-mV MEP successfully in one third of participants that operator B failed to perform. Two participants were excluded in experiment I but not in experiment II. We thought it was because they had higher motor cortical excitability in experiment II. They both had higher motor threshold in experiment I performed by both operators (RMT performed by operator B vs. E in experiment I = 94% MSO vs. 76% MSO and 85% MSO vs. 88% MSO, respectively; RMT performed by operator B vs. E in experiment II = 75% MSO vs. 73% MSO and 54% MSO vs. 60% MSO, respectively).

In experiment I, the mean amplitude of 1-mV MEP performed by operator E was higher than that performed by operator B. We speculated that operator E was used to generate a slightly higher 1-mV MEP amplitude to make good consistency in the measurement. 1-mV MEP is widely used in TMS studies, such as the test pulse in PPS or the baseline MEP. Choosing an optimal stimulation intensity to elicit an MEP amplitude of approximately 1 mV with good consistency is more important than the exact amplitude of 1 mV. In this study, the consistency of 1-mV MEP did not differ between the operators in both experiments. However, we found that operator B had higher consistency in experiment II compared with experiment I. We believed that the improvement of consistency was through practice.

Pre-TMS measurements

In experiment II, we found that participants were slightly more alert before TMS performed by operator E than operator B although we had already controlled some factors that might affect alertness. We suppose this might result from some unrecorded factors that would affect alertness, such as composition of the meal, missing a meal, or loading of work on the day (Neely et al., 2004, Paz and Berry, 1997). It needs further studies to proof this.

Reliability

Operator B and operator E had a relatively high interrater reliability in the measurements of RMT, AMT, and silent period but moderate interrater reliability in SICI and low interrater reliability in ICF and post-cTBS MEP amplitudes. It seems that operators had lower interrater reliability in paired-pulse stimulation and TBS compared with single pulse stimulation. Individual variability in response to TMS could be the reason. Some studies reported that there is higher reproducibility in RMT, AMT, and silent period compared with cTBS in healthy adults (Campana et al., 2019, Corp et al., 2020, Fried et al., 2017, Fritz et al., 1997, Jannati et al., 2019). SICI had higher interrater reliability than ICF. Biological factors may be considered, such as asynchrony and phase cancellation of descending volleys or additional variable changes in cortical excitability in response to shorter ISIs (Boroojerdi et al., 2000). Technical problems related to an operator might be the least likely reason because different ISIs appeared in a random order during PPS in this study.

In this study, each operator performed RMT and AMT twice for each participant. Interrater reliability in both experiments was higher than either operator’s intrarater reliability in both AMT and RMT measurements, meaning that the individual motor threshold was closer between visits in experiment II than between the experiments. The only difference is that we controlled for modifiable factors individually in experiment II. Therefore, these findings indicate that modifiable factors that we controlled for this study, namely the time and day of stimulation; caffeine, alcohol, and nicotine intake; medication; exercise; working hours in the day; and sleep duration the night before TMS measurement, are crucial factors for cortical excitability.

Limitation of the study

In this study, the number of participants and operators were small. Future studies including a higher number of sessions, participants, and operators can provide more detailed information. However, enrolling more operators to perform more sessions is challenging. Although we controlled for the modifiable factors of participants, we could not control some factors such as the arousal state or muscle relaxation during the course, which may affect TMS results.

Conclusions

An operator’s experience in TMS may affect the results of RMT measurement. Compared with an experienced operator, a beginner performed higher RMT which may be attributed to stimulating on less precise locations and could further mislead the results of studies. With practice, a beginner may choose a more precise stimulation location and have higher consistency in 1-mV MEP measurement.

Funding

This work was supported by Ministry of Science and Technology project (MOST 109-2314-B-182A-159-). The funders had no role in study design, collection, analysis, and interpretation of data, writing of the report, or decision to submit the article for publication.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.