On the One Hand or on the Other: Trade-Off in Timing Precision in Bimanual Musical Scale Playing.

Music performance requires simultaneously producing challenging movement sequences with the left and right hand. A key question in bimanual motor control research is whether bimanual movements are produced by combining unimanual controllers or through a dedicated bimanual controller. Here, 34 expert pianists performed musical scale playing movements with the left or right hand alone and with both hands simultaneously. We found that for the left hand, scale playing was more variable when playing with both hands simultaneously rather than with one hand at a time, but for the right hand, performance was identical. This indicates that when task constraints are high, musicians prioritize timing accuracy in the right hand at the cost of detriment of performance in the left hand. We also found that individual differences in timing substantially overlap between the unimanual and bimanual condition, suggesting control policies are similar but not identical when playing with two hands or one. In the bimanual condition, the left-hand keystrokes tended to occur before right-hand ones, and more so when the hands were further apart. Performance of the two hands was furthermore coupled so that they tended to be early and late together, especially in the beginning and end of each scale. This suggests that experts are able to achieve tightly coupled timing of scale playing movements between the hands. Taken together, these findings show evidence for partially overlapping and partially separate controllers for bimanual and unimanual movements in piano playing. Copyright:

Introduction

Human motor performance often requires the use of left and right limbs simultaneously, such as controlling the legs to walk or using the hands to tie shoelaces (MacKenzie & Marteniuk, 1985; Oliveira & Irvy, 2008). In such cases, the movements produced by the limbs must be tightly coupled in order for the movement to be effective, and even small misalignment of the movements in time can have disastrous consequences. Music, and in particular piano playing, compound this challenge because they involve producing a great deal of keystrokes in a short amount of time (Globerson & Nelken, 2013; Münte, Altenmüller, & Jäncke, 2002). The timing of notes is critical in music because notes that occur early or late convey important expressive cues contributing to the emotional effect of music (Bhatara, Tirovolas, Duan, Levy, & Levitin, 2011). During bimanual piano playing, the two hands are often required to produce simultaneous sounds, which is challenging because work in auditory perception has shown that millisecond differences in note timing can be detected by the ear (Exner, 1875). A great deal of research investigates musicians’ control of timing (Jabusch, Alpers, Kopiez, Vauth, & Altenmüller, 2009; MacKenzie & Van Eerd, 1990; van Vugt, Furuya, Vauth, Jabusch, & Altenmüller, 2014; van Vugt, Jabusch, & Altenmüller, 2012; van Vugt, Jabusch & Altenmüller, 2013; Wagner, 1971) but less is known about bimanual control.

Research on bimanual motor control has focused on how the two hands can achieve independence. It is famously difficult to pat your head and rub your stomach at the same time (Oliveira & Irvy, 2008). This is presumably because of cross talk between the control signals sent to the two hands. Instead of controlling the bimanual movement as a combination of two unimanual motor programs, it may therefore be more beneficial for the brain to consider the bimanual movement as a separate movement pattern. This idea is the basis of ongoing debate about whether bimanual movements are separate, atomic motor entities or combinations of unimanual control strategies (Marteniuk, MacKenzie, & Baba, 1984; Schmidt, 1975; Yokoi, Bai, & Diedrichsen, 2017). Previous work showed that force field dynamics learned in a unimanual reaching condition transferred partially to a bimanual condition (Nozaki, Kurtzer, & Scott, 2006), indicating that the control processes for bimanual reaching movements are partially overlapping and partially separate from those for the unimanual condition. Expert pianists have amassed many thousands of hours at their instrument and, therefore, it remains unclear whether at that stage the unimanual and bimanual representations are integrated or not.

Here, we monitored expert pianists performing the demanding task of playing musical scales at a fast rate, either with one hand at a time or with both hands separately. We approached the question whether unimanual and bimanual controllers are separated or integrated in three ways. First, if bimanual playing relies on a dedicated, separate controller, this should enable independent modulation of the precision of the two hands, that is, timing accuracy in one hand could be prioritized over another. We hypothesized that if one hand is prioritized, that hand may show the same variability during bimanual and unimanual playing, at the expense of an increase in variability during bimanual playing in the other hand. The second way to approach integration or separation of controllers was to investigate individual differences in scale playing timing. Previous work showed the existence of robust interindividual differences in scale playing timing between players (van Vugt et al., 2013) and here, we hypothesized that if bimanual control relies on a combination of unimanual controllers, the individual timing differences should be maintained between unimanual and bimanual control. The third way to approach the question of overlap or separation of uni- versus bimanual control was to assess the amount of temporal coupling between the hands. We hypothesized that if bimanual playing is achieved by executing two unimanual controllers in parallel the timing deviations of corresponding keystrokes of the two hands should be independent, but if bimanual control relies on an integrated controller, the hands may be early or late together. In order to assess this coupling, we investigated the correlation between timing deviation of corresponding keystrokes of the left and right hand.

Methods

Participants

Thirty-four right-handed piano players participated in the experiment (18 female, 16 male). They were enrolled in the prestigious piano programme at the Hannover Music School (HMTMH), Germany. Participants were 24.71 (SD = 4.40) years of age at the time of the experiment, and they had accumulated 13.82 (SD = 9.0) × 10^3 hours of training at the instrument over the course of 17.7 (SD = 4.02) years.

Procedure

Participants were seated at the Kawai MP9000 digital piano (Kawai, Krefeld, Germany) and were instructed to play C major scales using one hand at a time (unimanual) or both hands simultaneously (bimanual; see Figure 1, Panel A). Participants played the scales over two octaves in the inward or outward directions alternatingly with a short break in between each run. Playing was paced using a metronome producing clicks at 120 beats per minute and the instruction was to play four notes per metronome click. Participants were asked to use the conventional fingering (see Figure 1, Panel A) and play mezzo-forte in a smooth legato-style. The task was to play as evenly as possible, keeping in time with the metronome. In each block, approximately 30 cycles of the inward and outward scale were recorded without interruption in one condition (unimanual or bimanual, see Figure 1, Panel B) and all subjects performed three blocks: left hand unimanual, right hand unimanual, and bimanual scales. The order of these three blocks was counterbalanced (e.g., a possible ordering was left hand unimanual, bimanual, right hand unimanual). The keystrokes were captured through the MIDI interface and recorded on a PC through a custom-made C program for offline analysis.

Figure 1

Panel A: Participants played C-major scale with two hands simultaneously (bimanual) or one hand at the time (unimanual), alternatingly outward and inward playing directions. Panel B: In each block, inward and outward scales were recorded alternatingly (see example right hand unimanual block) and participants performed three blocks (left hand unimanual, right hand unimanual, and bimanual) in counterbalanced order.

Data Analysis

Scale runs with incorrect notes or omissions were discarded from further analysis. In each correct scale run, the intervals between the onset of subsequent keystrokes were calculated (inter-onset-interval, IOI, in milliseconds). In order to assess the timing accuracy, we computed the SD of the IOI, which is referred to as unevenness. The larger this value, the more variable the intervals are and hence, more irregular the timing of the keystrokes. We performed a repeated-measures analysis of variance (ANOVA) and report generalised effect sizes (Bakeman, 2005). In order to quantify the unique individual timing pattern of each pianist, we computed a timing fingerprint by averaging the IOIs for each pair of adjacent keystrokes in the scale, yielding a vector of 14 elements (corresponding to the 15 keystrokes in the two-octave scale).

Results

Timing was more variable in bimanual playing relative to unimanual playing, but only in the left hand (see Figure 2). A repeated-measures ANOVA with actors of condition (unimanual, bimanual), hand (left, right), and playing direction (inward, outward) revealed a significant main effect of hand, F(1, 33) = 55.36, p < .0001) indicating that the left hand is more variable overall, as previously found. A two-way interaction between hand and direction was also fonud, F(1, 33) = 6.11, p = .02, η²= .007. Because of the interaction, we analysed the two hands separately. For the left hand, we found that bimanual playing was more variable than unimanual playing, F(1, 33) = 21.39, p < .0001, η² = .05, and outward scales were more variable than inward scales, F (1,33) = 11.50, p = .002, η² = .03. There was no interaction between direction and bi-/unimanual condition. In the right hand, there was no significant difference between unimanual or bimanual playing, F(1, 33) = .89, p = .35, there was also no effect of direction or interaction between those two, F(1, 33) = .01, p = .91.

Figure 2

The left hand showed greater timing variability when playing with both hands simultaneously than with the left hand alone, and this was not true for the right hand. Violin plots indicate density estimates, and inside each is indicated the mean and SEM.

In order to assess whether unimanual and bimanual control is governed by the same or different motor control processes, we assessed the pattern of individual differences for each pianist and investigated whether this was maintained from unimanual to bimanual playing.

Does the individual timing pattern in the unimanual condition predict the bimanual timing pattern? For each participant, the average timing pattern was computed as the vector of the average IOI for each keystroke (the timing fingerprint, a vector of 14 elements corresponding to the intervals between 15 keystrokes, see Figure 3, Panel A for an example). We then computed the Euclidean distance between the bimanual and unimanual timing vector (within; one value) for each pianist and the average distance between the pianists bimanual timing vector and the unimanual timing vectors of all other pianists (between; one averaged value). We found that distances between the timing vectors were smaller within the same pianist than across pianists (see Figure 3, Panel B). A repeated-measures ANOVA with distance as the dependent variable and factors of hand (left, right), direction (inward, outward), and comparison (within, across) revealed an interaction between hand and comparison, F(1, 33) = 6.98, p = .02, η² = 0.19. For the two hands separately, the distances were smaller within than across pianists, F(1, 33) > 197.88, p < .0001, η² > .17 in both cases. Auxiliary findings were that distances were generally larger in the left hand, F(1, 33) = 69.54, p < .001, η² = .11, presumably because the left hand shows more variability between trials. This analysis revealed that at least a portion of individual timing deviations was preserved between the unimanual and bimanual condition.

Figure 3

Individual timing patterns in the unimanual conditions predict the bimanual timing pattern. Panel A: Timing patterns between the unimanual and bimanual condition are similar within pianists but more different between different pianists. Black: example timing vector (fingerprint) of one pianist in the bimanual (dashed line) and unimanual (solid line) condition, and the timing vector of another pianist (orange). Panel B: Group data for the Euclidean distance (presented here as an average absolute distance for each keystroke interval), indicating that timing patterns are more similar between the unimanual and bimanual condition within pianists than across different pianists. Panel C: The timing patterns of individual scale runs were compared for each pianist within or across the unimanual and bimanual conditions. When comparing pairs of unimanual or pairs of bimanual scale runs, the timing patterns were more similar than when comparing a unimanual to a bimanual scale run and this was true for both hands and playing directions.

A second analysis was performed where we extracted the scale runs within each individual pianist. Within each individual pianist, the timing of all possible pairs of individual scale runs (both unimanual and bimanual) was compared using the Euclidean distance measure. Then, these distances were grouped according to the kind of runs that were compared: unimanual versus unimanual, bimanual versus Bimanual, or unimanual versus bimanual (see Figure 3, Panel C). The idea was that if unimanual and bimanual playing rely on separate motor representations, the distances between pairs of unimanual or pairs of bimanual scales should be smaller than between unimanual and bimanual scales. We ran an ANOVA with the factors of hand, playing direction, and comparison (three levels: unimanual vs. unimanual, bimanual vs. bimanual, unimanual vs. bimanual). It was found that the distance between the fingerprints of the left hand were greater overall than those of the right hand, F(1, 33) = 72.722, p < .0001, in line with greater timing variability in the left hand reported above. There was also an interaction between hand and comparison, F(2, 66) = 8.86, p < .001. Crucially, planned contrasts revealed that the cross-condition comparison (unimanual vs. bimanual) was significantly greater than the within-condition comparisons (unimanual vs. unimanual or bimanual vs. bimanual) for both hands and playing directions, F(1, 33) > 27.21, p < .0001, η² > .11 for all comparisons. This analysis thus reveals that there are differences in individual timing patterns of scales between the bimanual and unimanual conditions. Taken together with the earlier analysis (between pianists), this reveals a nuanced picture where some, but not all individual variation is retained between the bimanual and unimanual condition.

During bimanual playing, subjects were instructed to play corresponding keystrokes of the left and right hand exactly simultaneously. In reality, we observed small timing differences between the onset of the keystrokes of the two hands in the order of a dozen milliseconds. In general, left hand keystrokes occurred earlier in time than the corresponding keystrokes of right hand (see Figure 4, Panel A). Interestingly, when the hands were closer together in space, the keystrokes also occurred more close together in time and the timing offset between the hands increased as the hands moved further apart in space. For outward scales, the timing offset increased, F(1, 33) = 30.53, p < .0001, and for inward scales it decreased, F(1, 33) = 39.10, p < .0001 (Figure 4, Panel A).

Figure 4

Although the left keystrokes preceded the right, the timing of the two hands was tightly coupled, especially in the beginning and end of the movement and when the hands were close together in space. Panel A: Left hand keystrokes occurred before the corresponding right hand keystrokes (left hand leads, right hand lags) when the hands were further apart. For outward scales, the left-right timing offset was close to 0 initially, when the hands were close together, and increased in the course of the scale run; the opposite pattern was found for the inward scales where hands were far apart initially and became closer together. Panel B: Left and right hand scales are coupled in time: The interval played by one hand correlates with the interval simultaneously played by the other hand. Panel C: The slope of the relationship between the lateness of the left and right hand is close to 1, implying that the hands are early or late together by the same amount. Panel D: The timing of the left and right hand is correlated across the entire scale, but more strongly in the beginning and end of the scale.

The timing of the two hands was tightly coupled on a trial-by-trial basis. There was a correlation between the length of the interval played by the left hand in a particular trial and the interval played simultaneously by the right hand (see Figure 4, Panel B). In other words, when the left hand played a longer interval than the target interval (125 ms), the right hand also played a longer interval on average, F(1, 33) = 30.53, p < .0001, η² =.48 for outward scales and F(1 ,33) = 39.10, p < .0001, η² =.54 for inward scales. The slope of this relationship between the intervals of the two hands was close to 1, indicating that the left and right hand tended to be early and late by the same amount of time (see Figure 4, Panel C). The correlation between the timing of the left and right hand was present across the entire scale, but more pronounced at the beginning and end of the scale (see Figure 4, Panel D). To test whether this was statistically significant, we performed an analysis of covariance (ANCOVA) with the individual correlation estimate (Pearson's r) as the dependent variable and distance from the center of the scale (covariate, values from 0 to 7) and playing direction (inward, outward, which was not significant, F(1, 33) = .34, p = .56) as factors. The distance from the center of the scale was significant, F(1, 33) = 7.14, p = .01, η² =.07, which shows that the correlation between the hands was higher at the beginning and end of the scale run.

Discussion

The present study investigated timing control of bimanual scale playing in expert musicians. A great deal of research in bimanual motor control has investigated the difficulty experienced in decoupling the movements of the two hands when the task requires it. Here, pianists played mirrored scale playing movements which, on the contrary, required tight coupling in time between the movements of the two hands in order to achieve simultaneity of the resulting sounds.

Key findings are that the left, but not the right hand is more variable during bimanual control than unimanual control. Individual timing patterns are partially, but not completely maintained between unimanual and bimanual conditions. The left-hand keystrokes tended to precede those of the right hand, especially when the hands were further separated in space, and both hands tended to be early and late together, especially in the beginning and end of each scale run. These findings suggest a great deal of integration between the hands and a priority given to temporal precision in the right hand.

The Right Hand is Prioritized During Bimanual Playing

The present data show that the left, but not the right hand is more variable during bimanual as opposed to unimanual playing. This suggests that the right hand is prioritized during bimanual control. In relation to the main question of this study, this suggests that bimanual control relies on a distinct controller in which timing precision can be different from that of the individual unimanual controllers. The present study only tested right-handed participants. As a result, it is possible, at least in principle, that the deterioration in timing accuracy in the left hand is because this is the nondominant hand. However, previous work suggested that timing asymmetries are related to musical practice, not handedness. Indeed, it was found that the left hand is generally less precise during musical scale playing (van Vugt et al., 2014), and this is true for both left- and right-handed pianists (Kopiez et al., 2011) and asymmetries in tapping performance between the hands are reduced with musical training (Jäncke, Schlaug, & Steinmetz, 1997). Kopiez et al. also provide an intriguing argument that why the left-right asymmetry in unimanual playing may occur: They show that Western classical music tends to put higher demands on the right hand, as shown by a greater number of notes played by the right hand in typical repertoire. The current work extends the previous unimanual findings by showing that not only is the right hand more precise overall, its superior temporal precision is maintained during bimanual playing, at the expense of further deterioration in the left hand. Possibly the prioritization of the right hand observed in the present study is similarly driven by the fact that this hand tends to play more notes.

Two Equals One plus One?

Does the brain treat bimanual control as a combination of the two unimanual control processes or is the bimanual controller a unit of its own, more or less independent of the two unimanual processes? The generalized motor program theory proposes the latter idea of a separate bimanual control process (Schmidt, 1975). In support of this theory, a recent sequence learning study found that there was no transfer from unimanual learning to bimanual performance (Yokoi et al., 2017). On a neural level, work on rhesus monkeys indicates that most cells in primary motor cortex show activity specific to bimanual movements (Donchin, Gribova, Steinberg, Bergman, & Vaadia, 1998). Furthermore, a study with macaque monkeys shows that transcallosal connections reduce the correlation between the representations of the two unimanual movements, providing evidence for a separate bimanual representation (Rokni, Steinberg, Vaadia, & Sompolinsky, 2003). Alternatively, the brain may create bimanual control by simultaneously activating the two unimanual control processes (Marteniuk et al., 1984). If bimanual control builds on the unimanual controllers, learning in the unimanual condition should transfer to bimanual control. Indeed, previous studies have observed partial transfer between effectors in force field learning (Nozaki et al., 2006).

The present data show, for the first time, that individual differences in the form of a timing fingerprint are partially, but not completely, preserved across unimanual and bimanual conditions. Individual differences were previously shown in deviations away from regularity in the order of milliseconds and these were consistent within pianists, enabling a machine learning algorithm to identify the players (van Vugt et al., 2013). The (partial) preservation of these timing patterns provides evidence for overlap between the unimanual and bimanual controllers, and the lack of complete preservation provides evidence that there is also a level on which these controllers are separate. Thus, rather than either complete separation or complete integration of the bimanual and unimanual controllers, the present study suggests that there may be partial overlap, which is in line with previous mixed results from the literature, where some studies find evidence for integration and others for separation of the bimanual and unimanual controllers (Marteniuk et al., 1984; Nozaki et al., 2006; Yokoi et al., 2017).

Early Together, Late Together

Timing of the two hands showed a great deal of coupling. When one hands keystroke came early, there was a tendency for the other hand to also make an early keystroke. This correlation between the two hands occurred within individuals and for each of the various keystrokes separately, indicating that the coupling occurred on a trial-by-trial basis. This coupling between the keystrokes of the two hands supports the idea that bimanual playing is controlled using a distinct bimanual controller. If instead there were two independent unimanual controllers, it would be expected that timing deviations would be independent between the two hands. The finding is similar to observations in bimanual reaching movements, where short movements became longer when the other hand made a simultaneous long movement and vice versa (Marteniuk et al., 1984). These effects are generally thought to be due to cross-talk in the control signals sent to the two limbs (Oliveira & Irvy, 2008). The present data extend the previous work by showing that this phenomenon is still present in a selective sample of pianists from a world-class piano program (Hannover Music University), suggesting that even massed practice fails to achieve independence in the control between the two hands.

The present study found that timing in the two hands during bimanual control was tightly coupled. In the present study, the two hands produced mirror movements, leading to activation of homologous muscles. Because of the coupling between the hands, bimanual control tends to favour symmetric movements. For example, bimanual movements are more synchronous and less variable when they are produced by homologous muscles (Cohen, 1971). In a similar way, when humans produce cyclic antisymmetric movement patterns at high speeds, often they shift to the symmetric pattern in which homologous muscles are activated simultaneously (Haken, Kelso, & Bunz, 1985; Kelso, 1981; Ryu & Buchanan, 2004, again suggesting a preference for symmetric (in-phase) movement patterns. On a neural level, during unimanual movements, there is a tendency to activate homologous muscles of the two limbs. For example, during unimanual movements, excitability is modulated for the motor pathways for the unused hand (Carson, Welsh, & Pamblanco-Valero, 2005), presumably due to interhemispheric synchronisation (Serrien & Brown, 2002). Performance of asymmetric (antiphase) movements tends to elicit greater activity than symmetric movements (Sadato, Okada, Honda & Yonekura, 2002; Stephan et al., 1999), presumably reflecting the increased complexity of control of nonhomologous movements (Globerson & Nelken, 2013). With extensive practice, skilled pianists can achieve greater degrees of independence in the control of intensity and rhythm (Shaffer, 1981), and reduce the typically seen increased cortical activation in motor association areas during asymmetric relative to symmetric movement patterns (Haslinger et al., 2004). Similarly, long-term musical practice is associated with increased corpus callosum volume (Schlaug, Jäncke, Huang, Staiger, & Steinmetz, 1995) and changes in transcallosal inhibition (Ridding, Brouwer, & Nordstrom, 2000). The corpus callosum has been implicated in bimanual coordination (Gooijers & Swinnen, 2014) and, therefore, these music-induced changes are thought to underlie the improved capacity for pianists to perform separate movements with the two hands. At first glance, the present data may seem to contradict previous work showing greater segregation of control of the two hands in musicians. However, note that in the present case, coupling between the two hands was adaptive to the task because it would increase synchrony of the keystrokes which is what participants were asked to do. Future research could build on the present study by investigating movements that are not mirrored, for example, by asking pianists to simultaneously produce an inward scale with one hand and an outward scale with the other hand, in which case achieving coupling between the hands is more challenging.

The present data also showed that the left-hand keystrokes tended to occur earlier in time than the corresponding keystroke of the right hand, especially when the hands are further apart. This left-lead effect has been observed before and appears to occur especially when lower notes are played by another hand than the higher notes (Hartmann, 1932; Repp, 1996; Vernon, 1937), as is the case here. The left-lead phenomenon, by which lower notes played with the left hand appear early, is thought to reflect individual aesthetic choice (Repp, 1996) and, therefore, it is surprising that it occurs in the present study as well, where the participants were instructed to aim for regularity and synchrony rather than aesthetic playing. Other studies have reported an opposite pattern, known as the melody-lead effect, by which the keystrokes playing the melody may occur several dozens of milliseconds earlier than those playing the lower accompaniment notes (Palmer, 1989). However, the melody-lead effect has been shown to occur as an artifact of greater keystroke velocities for melody notes (Goebl, 2001).

The current work investigated timing control of challenging bimanual sequences in expert pianists and explored the question whether bimanual and unimanual controllers are integrated or separate. It was found that bimanual control was more variable than unimanual control, but only in the left hand, indicating a priority given to the right hand, which maintained its high level of timing accuracy. This provides support to the idea of a dedicated bimanual controller, in which the timing precision of the two hands can be modulated differently than during unimanual playing. Individual timing patterns were partially, but not completely maintained between bimanual and unimanual control, suggesting overlap between the motor representations guiding playing with two hands or with one. During bimanual control, the two hands were tightly coupled, tending to be early or late together, suggesting that the two hands are not driven by separate (unimanual) controllers, but rather using a single bimanual controller. In sum, the present study found evidence for partial but not complete overlap between unimanual and bimanual controllers.

The present data show, for the first time, that individual differences in the form of a timing fingerprint are partially, but not completely, preserved across unimanual and bimanual conditions. Individual differences were previously shown in deviations away from regularity in the order of milliseconds and these were consistent within pianists, enabling a machine learning algorithm to identify the players (van Vugt et al., 2013). The (partial) preservation of these timing patterns provides evidence for overlap between the unimanual and bimanual controllers, and the lack of complete preservation provides evidence that there is also a level on which these controllers are separate. Thus, rather than either complete separation or complete integration of the bimanual and unimanual controllers, the present study suggests that there may be partial overlap, which is in line with previous mixed results from the literature, where some studies find evidence for integration and others for separation of the bimanual and unimanual controllers (Marteniuk et al., 1984; Nozaki et al., 2006; Yokoi et al., 2017).