A tradeoff between musical tension perception and declarative memory.

Successful formation of long-term declarative memory is influenced, among other things, by attention, emotion, and deviation from expectations. A unique form of expectation can be elicited through musical tension, evoked by the prolongation of certain musical progressions. We examined the effect that musical tension exerts on the formation of declarative memory, by composing three original music pieces that contained tension segments, achieved by delays in release from dominant to tonic harmonies. Music-evoked tension was validated using music information retrieval (MIR) analysis, as well as skin conductance response (SCR) measures. Indeed, tension-evoking musical excerpts were associated with heightened SCR, corroborated by increased subjective ratings of tension, as compared to neutral excerpts. In the main experiment, 50 participants listened to the three musical pieces while they were presented with unique images that were randomly assigned to four conditions: tension, tension-release, neutral music, and silence. One day later, their memory for the images was examined using a recognition test. We found that memory performance was enhanced for images presented during both neutral and tense music compared to silence. Moreover, we observed a tradeoff effect between post-experiment tension perception and memory, such that individuals who perceived musical tension as such displayed reduced memory performance for images encoded during musical tension, whereas tense music benefited memory for those with lower musical tension perception. Understanding the interrelations between musical components, which exert powerful and fundamental responses in humans, and cognitive faculties, may provide insights as to the basic features of memory formation.

Introduction

The ability of musical stimuli to induce strong emotional responses is well established (Judde & Rickard, 2010; Krumhansl, 1997; Rickard, 2004). The effects of musical stimuli on cognitive functions, however, are more complex and controversial. Musical stimuli provide a unique opportunity to study cognitive functions and their neuronal underpinnings, by examining the effects that musical properties exert on listeners while carrying out diverse cognitive tasks (Echaide et al., 2019; Lehne et al., 2014). Musical experiences are unique in that depending on context, they may evoke a sense of abstract expectation, which awaits to be resolved into a more stable state (Leonard B. Meyer, 1961). In the current study, we set out to examine how the perception of musically induced tension, which is based on unresolved expectations, may affect the formation of declarative memories of visual stimuli.

Previous findings on the effects of music on cognition in general and specifically on memory are inconclusive (de la Mora Velasco & Hirumi ,2020). Rauscher et al. (1993) claimed that listening to Mozart's music is directly correlated with the activation of neurons involved in spatial tasks, thus leading to noticeable improvements in visuospatial task performance. Conversely, other studies were not able to establish significant influences of background music on cognitive performance (Anderson & Fuller, 2010; Freeburne & Fleischer, 1952; Thompson et al., 2011). While empirical evidence on the relationship between music and visual memory is scarce, several studies have been conducted on the relationship between listening to music and memory with regards to verbal items (i.e.Ferreri et al., 2015; Iwanaga & Ito, 2002; Jäncke & Sandmann, 2010; Kang & Williamson, 2014; Woo & Kanachi, 2005). Findings from these studies are indecisive as well; some results point to a positive effect of listening to music on source memory (Ferreri et al., 2015; Ludke et al., 2014; Simmons-stern et al., 2010), whereas others (e.g., Jäncke & Sandmann, 2010) did not find such an effect. Thus, the exact mechanisms underlying the relationship between background music and learning, if any, are still largely unknown.

Two plausible models regarding the effect that background music may have on learning were previously suggested. The first, adopted from Kahneman’s “cognitive capacity model” (Kahneman, 1975) maintains that incoming cues compete for attention, such that the processing of any part of the incoming stimuli will come at the expense of performance relating to other stimuli. Thus, when multiple stimuli are simultaneously present, individuals will automatically divide their attentional resources, which may consume more cognitive resources that could be otherwise allocated to the main task. The usage of background music on top of a learning task can thus grab the performer’s attention, and consequently distract them from learning (Proverbio et al., 2015). In this sense, background music can be described as a “seductive detail” (Lehmann & Seufert, 2017; Rey, 2012). This suggestion was supported by studies showing that background music impeded learning of a new motor task (Miskovic et al., 2008) and driving performance (North & Hargreaves, 1999). Although previous research on the interaction between background music and memory in general is limited (Lehmann & Seufert, 2017), according to this view, background music is likely to interfere with the primary cognitive task, thereby diminishing overall behavioral/cognitive performance.

The second model – the “arousal mood hypothesis” – pertains to the emotions that music induces and their effect on concomitant cognitive performance (Husain et al., 2002). It is well established that music is a powerful tool for inducing emotions (Carr & Rickard, 2016; Gabrielsson, 2001), affecting physiological responses, including skin conductivity, heart rate, and other autonomic signatures (Baumgartner et al., 2006; Ellis & Simons, 2005; Grewe et al., 2009; Salimpoor et al., 2009). Given that memory formation is dramatically affected by emotional states (Cahill & McGaugh, 1998; Labar, 2007), it can be hypothesized that music may affect learning and memory via emotional responses. Indeed, emotionally arousing background music has been shown to enhance visual memory (Carr & Rickard, 2016; Proverbio et al., 2015). The effects of music on memory are, however, highly influenced by the type of music employed. For instance, music that is enjoyable for the listener is likely to improve performance of cognitive tasks (Husain et al., 2002; Thompson et al., 2011). This hypothesis has been supported by several studies showing that background music can improve both verbal memory encoding (Ferreri et al., 2013) and autobiographical memory in patients suffering from Alzheimer’s disease (El Haj et al., 2015), and that verbal and visual processing speed are improved when listening to background music (Angel et al., 2010). Nevertheless, arousal is not always beneficial for learning, as excessive arousal is associated with anxiety, which is harmful to learning (Lehmann & Seufert, 2017) and is likely to result in memory impairment under certain circumstances (Kenya & Vuyiya, 2020; Kizilbash et al., 2002).

A ubiquitous component in the experience of music is that of tension, which is often evoked by chords that within a particular context instill an unresolved sensation, which creates an expectation towards resolution to a more stable chord (Bigand et al., 1996; Leonard B. Meyer, 1961; Portabella & Toro, 2020). Since musical stimuli typically lack actual negative real-life implications, these “tense” experiences, which may otherwise be considered negative, can be experienced as positive and rewarding, as part of the aesthetic experience (Lehne & Koelsch, 2015). Among the most common and strongest harmonic progressions is the movement of the dominant chord (V) to its resolution in the tonic chord (I). Within the paradigm of classical tonal music, this harmonic progression (from V to I) is considered a stereotypical component in many familiar musical passages (Bigand & Parncutt, 1999). Upon delays in release, tension will likely reach its peak, and the listener will wait for the outcome to take place. Thus, prolonged delays create longer and more intense periods of tension (Huron, 2006, p.314). Examples of delayed tension release can be found in diverse musical genres and eras, from classical musicians (e.g., Chopin’s Nocturne in C sharp minor, first two bars) to popular composers such as Freddie Mercury. In Bohemian Rhapsody (Queen), for example (in secs 0:45 and 5:27), the composer sustains the fifth dominant chord, which causes tension that is released by resolving on the tonic chord. It is noteworthy that the delay effect increases the more a specific outcome is expected. Hence, any intentional prolonged delays in these types of “predictable” musical passages are expected to be correlated with increasing states of tension in comparison to passages with minimal or no delays. The means by which musical tension may affect declarative memory formation are, however, not well established.

According to the “cognitive capacity model,” we would expect that memory performance would be negatively affected by perception of musical tension, due to the simultaneous processing of competing stimuli. Alternatively, the “arousal mood hypotheses” would predict that arousing music, induced in this case by tension, would increase memory performance (Proverbio et al., 2015). In order to test these competing hypotheses, three original piano pieces were composed (by the author - N.K.), containing both tense and neutral segments, as well as events of release from tension and periods of silence. Tension was achieved by playing a musical phrase twice, first devoid of delays in tension release, and once again with delaying tension release. In this manner, when the delayed resolution is played, the participants may compare the novel stimulus with the previous presentation. We validated the musical tension manipulation by testing measurements of skin conductance responses (SCR), subjective ratings of suspense, and applying music information retrieval (MIR) analyses. During the experiment, participants were presented with unique images while listening to the musical pieces. One day later, their memory was examined using a memory recognition test. We found a tradeoff between musical tension perception and memory performance, such that the perception of tense versus neutral music was associated with lower subsequent memory for concurrently presented visual images.

Methods

Participants

The current study comprised 50 healthy participants aged between 20 and 45 years (M = 26.30, SD = 4.81). Thirty-eight females with a mean age of 26.39 years (SD = 5.42) and 12 males with a mean age of 26 years (SD = 2.08) participated in the memory experiment. An additional 22 healthy participants took part in a pilot study, designed to validate the musical tension stimuli by measuring skin conductance response (SCR). Three participants were excluded from analysis due to undetectable changes in SCR, leaving 19 participants (13 females and six males; age range: 20–35 years, M = 25.08, SD = 4.03). All participants were recruited through university posters and using the University of Haifa SONA system. Participants remained completely naïve about the aims and purposes of the study. The experiment was approved by the ethics committee of the Psychology Department of the University of Haifa, and participants were remunerated for their participation. Due to the Covid-19 pandemic, 20 participants carried out the experiment in an online version, by receiving a video link to the experiment, monitored by the experimenter using the Zoom meeting software for the learning session, and the Pavlovia platform (https://pavlovia.org/) for the memory test session (the same procedures were applied for both online and laboratory versions).

Musical material

Three different piano pieces were composed specifically for the purposes of this research. These pieces were recorded using a digital piano in the Music Studio of the Department of Music at the University of Haifa. Previous research has shown that the effect of musical familiarity could interfere with participants' performance (Büdenbender & Kreutz, 2016; Hilliard & Tolin, 1979). Therefore, all pieces were novel compositions (and not adapted from already existing musical works) to preclude familiarity with any of the musical stimuli (see Online Supplementary Material (OSM) for links to the musical pieces).

The musical pieces (of lengths 2.37, 3.49, and 3.08 min) were composed so as to contain a sufficient number of distinct events of tension and tension releases, with intentional delays of resolutions (see OSM for links to musical pieces). The musical pieces contained both harmonic tension with immediate release (no delay, neutral condition) and harmonic tension with delayed resolution (tension condition). Harmonic tension was evoked by using dominant seventh chords, which typically evoke a sense of prediction by eliciting expectations regarding future (musical) events (Bigand et al., 1996). The pieces were composed in such a manner that non-experienced musicians were also likely to experience this “anticipatory” sensation; this is achieved by playing a musical phrase twice, first devoid of resolution delays, and once again with delaying the release from tension (see Fig. S2, OSM). In the first presentation of the musical phrase, a representation of the expected resolution is acquired, such that when the delayed resolution is played, the participants may compare the novel stimulus with the previous presentation. The tension chords were followed by a Root Chord, which served to resolve, or release, the previously elicited tension (Huron, 2006; Meyer, 1956). Importantly, the neutral and the suspenseful musical conditions did not differ in terms of rhythm, tempo, or harmony. These elements were kept similar, and the only substantial difference was in the temporal delays of the harmonic resolution in each condition.

The music’s waveforms, spectrograms, and novelty information were extracted using the Music Information Retrieval (MIR) toolbox for Matlab (Lartillot et al., 2008). Using the MIR toolbox, the musical waveforms were segmented according to “attack” onsets, followed by the extraction of the peaks and subsequent troughs of amplitudes. This enabled us to quantify the decay time (in seconds), within which each pre-defined event type was presented (“tension,” “neutral,” and “tension release”). We also computed mean amplitudes (the music’s volume) for each condition by creating an envelope from the waveform and extracting the relevant positive values. In addition, we used the mirnovelty function to characterize the transitions between successive states in the music throughout. This feature is computed by using a self-similarity analysis approach, comparing local configurations along the diagonal of a cross-correlation matrix in an unsupervised manner (Foote & Cooper, 2003). Applying a kernel size of 400 frames, the algorithm created a novelty curve, representing transitions between successive changes in music properties. Following this, we averaged the novelty values across events of each type, and plotted them.

Visual stimuli

The visual stimuli we used for examining incidental declarative memory formation consisted of 160 images selected from an open access database – the Bank of standardized Stimuli (BOSS; Brodeur et al., 2014). These images were all emotionally neutral, and previously validated for various properties, including familiarity and visual complexity (ibid).

Physiological validation of musical tension

In order to examine the physiological response to the musical tension manipulation, we carried out a pilot study on 22 participants using both subjective and objective measurements. This experiment did not test memory, and included only the musical stimuli. The experimental procedure consisted of three stages: In Stage1, participants were instructed to sit on a comfortable chair, while skin conductance was recorded from electrodes attached to the distal phalanx of the second and third fingers of their non-dominant hands. They listened to the three abovementioned musical pieces interleaved with short silent intervals. In Stage2, participants were asked to rate the degree to which they felt musical tension for short musical excerpts that were selected from the three musical pieces conveyed during Stage1, each either containing musical tension or not. Participants were asked to rate their felt-tension for each excerpt on a scale from 1 to 5. Following this, Stage3 included a questionnaire, in which participants were asked to answer questions that gauged their musical background. Indeed, musical excerpts with tension, characterized by prolonged delays in melodic resolution, were associated with increased subjective ratings of suspense as compared to neutral excerpts. Three out of the 22 participants were excluded from the analysis since they did not show any significant SCR measurements.1 The analysis of SCR data consisted of down-sampling the data from 500 Hz to 10 Hz (in order to reduce computational load). Statistical analysis was performed using Ledalab (a Matlab-based toolbox; Benedek & Kaernbach, 2010).

Main experiment procedure

The main experiment consisted of two sessions (Fig. 1), conducted over two consecutive days. During the learning session, participants sat comfortably in front of a computer screen in an acoustically isolated room and under dimly-lit conditions. Participants were presented with 80 neutral pictures that were randomly selected from The Bank of Standardized Stimuli (BOSS; Brodeur et al., 2014). Each picture was presented on a computer screen for 2 s, and were equally divided (in a within-subject fashion) across four conditions (20 pictures in each condition) related to the musical components within the composed pieces. Specifically, 60 pictures were presented within the musical timeline, equally divided across musical tension, neutral, and tension release periods. As a memory formation baseline, 20 pictures were displayed in the absence of music (between musical pieces; silence condition). In order to counterbalance the stimuli-music associations across participants, four versions of the encoding phase were designed, each consisting of a different picture-musical condition assignment. Each participant was randomly presented with one of the versions of musical condition-visual stimuli combinations in the learning session.

Fig. 1

Schematic illustration of the experimental design. (A) A schematic timeline and examples of the stimuli presented during learning (left) and memory test (right) experimental stages. (B) Musical notation of an excerpt that contains the different musical conditions, along with a depiction of example images that were presented during the learning session

In the memory test session, participants underwent a recognition memory test that included 160 pictures (80 that were presented during the learning session and 80 new pictures) devoid of auditory input. The participants were instructed to determine whether or not each picture had appeared in the previous stage, and to rate their confidence concerning their answer on a scale from 1 (guess) to 9 (sure).

Subjective report of felt-tension and musical background

Following the memory test, participants were instructed to provide subjective ratings of six short excerpts taken from the original musical pieces, which either contained or did not contain musical tension (three of each kind). Participants were requested to rate their perceived tension for each excerpt on a scale from 1 (no tension perceived at all) to 5 (high perceived tension). Importantly, the tension rating task was performed only after the learning and test stages of the experiment had ended, since we did not wish participants to be explicitly aware of the research question regarding the effects of tension on memory, which might lead to biases in learning and memory performance. For each participant, we calculated a subjective tension perception score, defined as the difference between average perceived-tension for the tension excerpts minus average perceived tension for the neutral excerpts. In a subsequent analysis, this score was correlated with memory performance indices, as described below. In addition, a post-experiment questionnaire was administered in order to assess participants’ musical background. This questionnaire was adapted from the music background questionnaire (Zhao et al., 2012). Although none of the participants were professional musicians, this questionnaire provided information regarding the individuals’ experience with music through factors such as engagement with music, hours spent listening to music, etc.

Memory performance assessment

Memory performance was assessed using a Discriminability index (d’), calculated for each participant and condition as the difference between normalized hit rates and false alarm rates. In addition, confidence data was accounted for by performing a dual-process signal-detection (DPSD) analysis (Yonelinas, 2002), using the ROC Toolbox for MATLAB (Koen et al., 2017). Memory performance was binned by confidence ratings to create an 18-point scale (9 for “old” responses and 9 for “new” responses). The cumulative proportion of hits was plotted against the cumulative proportion of false alarms starting from the most stringent criterion (i.e., the proportion of hits and false alarms at the highest level of confidence) to the most liberal criterion. A receiver operator characteristic (ROC) curve was then fit to these points using maximum likelihood estimation, and the area under the curve was measured for each participant and condition.

Results

Music analysis

Three musical pieces, containing periods of neutral music, tension, and release from tension, were composed for the purpose of this study. To validate the musical conditions and their relation to memory performance, we applied a music information retrieval (MIR) approach to extract meaningful features from the composed musical pieces, with an emphasis on decay times and musical novelty (Lartillot et al., 2008). Decay times refer to the time-periods in seconds between successive peaks and troughs of amplitudes as detected by the MIR software, and novelty denotes transitions between successive musical states (see ). Figure 2 depicts the results of the MIR analysis for one of the three musical pieces (the results for all the music pieces are shown in Fig. S1, OSM). The amplitude waveform of musical piece #1 is shown along with vertical lines that correspond to the times and musical condition in which the visual images were displayed. Musical tension was designed here to contain prolonged dominant harmonies, and indeed, mean decay lengths (as detected using the MIR toolbox) were longest surrounding images of the predefined tension condition (mean ± SE = 8.26 ± 2.37 s), followed by tension release (1.89 ± 0.41 s) and neutral music (0.89 ± 0.18 s) (Fig. 2D). A one-way ANOVA revealed a significant difference in the decay times of the different conditions (F(2,19) = 5.58, P = 0.012), stemming from longer decays in the tension versus neutral and tension versus tension release conditions (P < 0.05, Bonferroni corrected for multiple comparisons). As shown in Fig. 2B, amplitude values did not differ among the conditions.

Fig. 2

Music waveform analysis. (A) Spectrogram of the music waveform of piece #1, indicating frequency power and volume across time. (B) Average music amplitude at the time of image presentation from each condition. (C) The musical waveform along vertical lines, representing the temporal location of images presented during learning on the background of tense music (red), neutral music (blue), and release from tension (green). Normalized frequencies are shown on the y-axis, and their power is indicated by the color intensity. (D) Decay lengths (in seconds) surrounding each musical condition, averaged across the music waveform. Images of the predefined tense music condition were shown on the background of epochs with longer decay lengths as compared to neutral and tension release conditions. (E) Novelty curve throughout the musical piece, indicating variations in musical transitions (see ), along with vertical lines corresponding to the temporal locations of images from the different conditions. (F) Mean novelty during the presentation of images from the different musical conditions. Similar analyses for the entire musical stimuli are shown in Fig. S1 (Online Supplementary Material)

In contrast, novelty values (indicating meaningful transitions in musical elements) were similar during times of tension and neutral music, but were significantly higher during release from tension (0.31 ± 0.04, 0.28 ± 0.03, and 0.53 ± 0.06, for tension, neutral, and tension release, respectively, F(2,21) = 3.47, P = 0.05). Figure 2(E and F) shows the novelty curve for musical piece 1 and corresponding mean novelty scores for each condition separately. Similar results were found for the entire musical stimuli (see Fig. S1, OSM).

Twenty-two subjects participated in a pilot study that examined skin conductance responses to the musical pieces. Three of them were excluded from the analysis since they did not show any significant SCR measurements.2 Peaks of SCRs were detected using the Ledalab toolbox, and averaged for each participant separately across periods of predefined tense and neutral music. The mean of SCR peaks was significantly higher when listening to tense music (1.533 ± 0.13 μS) as compared to listening to neutral music (1.079 ± 0.11 μS, P < 0.05, paired-sample t-test). This indicates that the suspenseful musical excerpts employed in the experiments were successful in inducing an arousal response (see Fig. S3, ,OSM). No correlation was found between participants’ musical background and their reactions to tense versus neutral music.

Perception of tension

In the preliminary pilot experiment, participants (n = 19) were asked to rate their felt-tension for selected musical excerpts on a scale from 1 to 5 after the experiment was concluded. The results, shown in Table 1, indicate that neutral musical excerpts (no delays in harmonic resolutions) were associated with relatively low tension-ratings (mean ± SE 1.89 ± 0.2), whereas tension-evoking musical excerpts (with delays in harmonic resolutions) were associated with higher felt tension (2.91 ± 0.21). A Wilcoxon nonparametric test on felt-tension ratings for tense versus neutral excerpts revealed a significant difference between these conditions (Z(19) = 3.15, P < 0.005). This is in line with the hypothesis that delays in harmonic resolutions relate to higher levels of felt tension. This effect was repeated in the main memory experiment, where tension-evoking excerpts were rated as being more tense than neutral ones (mean ± SE 2.73 ± 0.13 for excerpts containing suspended delays) versus neutral music (2.23 ± 0.11, P < 0.05), using a Wilcoxon matched pairs test. The musical background of participants was also assessed, as shown in Table 2.

Table 1

Subjective ratings of felt-tension for the pilot and main experiments

	Pilot Experiment(n = 19)	Main Experiment(n = 50)
Tension-evoking musical excerpts	2.91 ± 0.21	2.73 ± 0.13
Neutral musical excerpts	1.89 ± 0.2	2.23 ± 0.11

Table 2

Post-experiment responses to the musical background questionnaire

Percentage of participants who play a musical instrument/sing (mean years of playing/singing)	58% (5.03 years ± 0.9)(n = 29/50)
Percentage of participants with formal music training (+average duration of formal musical training)	50% (5.03 ± 0.8) (n = 25/50)
Estimated average time spent listening to music (hours per week)	7.34 ± 0.9
Estimated average time spent on musical activity (hours per week)	0.91 ± 0.26
Overall musical interest (1–no interest, 5–high interest)	4 ± 0.12
Overall musical ability (1–no ability, 5–high ability)	2.78 ± 0.16
Engagement with music (1–no engagement, 4–high engagement)	2.68 ± 0.17

Means and standard errors of subjective tension reports are detailed for all participants (in the pilot and main experiments), on a scale from 1 to 5: 1 indicating no felt tension and 5 indicating highly felt tension

The relationship between musical tension and declarative memory

One day after the learning session, participants carried out the memory recognition test, where they were instructed to judge whether each of the presented pictures was presented the day before or not. One participant was excluded from the analysis due to incomplete data, thus the remaining sample for analysis consisted of 49 participants. Recognition rates varied among the four different conditions, yielding the following hits percentages: 75.2% ± 2.35 for images presented during musical tension, 72.34% ± 2.58 for neutral, 67.44% ± 2.45 for tension resolution, and 64.38% ± 3.01 for silence. Mean false alarm rate was 21% ± 2.13. Mean sensitivity measures of d-prime (d’) for the tension, neutral, release, and silence conditions, respectively, were 1.72 ± 0.13, 1.59 ± 0.12, 1.43 ± 0.11, and 1.34 ± 0.13 (see Fig. 3A). A repeated-measures ANOVA test was carried out on the sensitivity measure (d’) across the musical conditions, yielding a significant memory effect for music conditions on memory performance (F(3,144) = 9.88, P < 0.00001). Bonferroni-corrected post hoc comparisons showed that images presented during the tension condition were better recognized than those presented during both the tension-release condition (P < 0.005) and silence (P < 0.001). Furthermore, mean hits for the neutral music condition were higher than those of the silent conditions (P < 0.005). Using the ROC-toolbox (Koen et al., 2017), we computed ROC curves for each participant and each condition (see Fig. S4, OSM), and extracted the AUC as an index of memory performance for different confidence criteria. A repeated-measures ANOVA of AUC across musical conditions yielded a significant effect (F(3,144) = 11.53, P < 0.0001, see Fig. 3B). As for d-prime, Bonferroni-corrected post hoc comparisons showed that AUC for tension music significantly differed from tension release (P < 0.001) and silence (P < 0.001), and neutral differed from silence (P < 0.005). Note that the memory indices for tension and neutral music were not significantly different than one another.

Fig. 3

Perceived tension vs. memory performance (d’) and area under the curve (AUC) measurements. (A) Mean memory performance (d-prime) for images presented in each musical condition. (B) Mean area under the receiver operating characteristics curve (AUC) for each musical condition. (C) Differential d’ for tense vs. neutral conditions (y-axis) plotted against differential perception reports for tense vs. neutral excerpts (the neutral condition was treated as a baseline for memory performance comparisons). (D) Differential AUC for tense vs. neutral conditions (y-axis) plotted against differential perception reports for tense vs. neutral excerpts. (E) Differential d-prime for tense release vs. neutral conditions plotted against perception of tension. (F) Differential AUC for tense release vs. neutral conditions plotted against perception of tension. (G) Differential d-prime for musical tension vs. silence conditions plotted against perception of tension. (H). Differential AUC for musical tension vs. silence conditions plotted against perception of tension. ** P < 0.005, *** P < 0.001

Reaction times and confidence ratings were compared between trials on which the subjects correctly identified a previously presented picture (hits) and those on which they did not remember the pictures (misses). A paired-samples t-test carried out on hits versus misses for all conditions showed that RTs were faster (P < 0.001) for hit responses (M = 1.36 ± 0.03 s) than for misses (M = 1.54 ± 0.04 s). Furthermore, a paired-samples t-test performed on confidence ratings showed that mean confidence ratings for hits (7.89 ± 0.07) were significantly higher than mean confidence ratings for misses (6.11 ± 0.14, P < 0.001).

In order to examine subject-by-subject relationships between tension perception and memory, we examined memory performance for images presented during tension and tension-resolution periods (as compared to baseline memory for neutral music background) against tension perception indices. Figure 3C depicts a negative correlation between tension perception and the tension memory index, indicating that the higher the tension ratings were to tense-evoking excerpts, the lower was their memory for images presented during tense-evoking compared to neutral music (Spearman’s rho = -0.39, P < 0.01 for d-prime, and rho = -0.49, P < 0.0005 for AUC, Figs. 3C and D). However, there were no significant correlations between memory indices of d-prime and AUC for the tension release condition versus tension perception (Figs. 3E and F, respectively), as well as for tension versus silence (Figs. 3G and H, respectively). This implies that the tradeoff effect between tension perception and memory performance was not a general trait, as it only took place in relation to the tension music condition.

We next examined the possibility that musical background and engagement may relate to tension perception and memory (see Table 2 for musical background data). Formal musical training was not correlated with tension perception (independent-samples t-test: t(47) = 0.99, P = 0.32). Similarly, musical activity (as assessed by the music background questionnaire) did not correlate with tension-perception, and memory performance (d-prime and AUC) did not differ between individuals with and without musical training (t(47) = 0.97, and t(47) = 1.61, respectively, NS). Additionally, d-prime and AUC measures for the musical tension condition were not correlated with subjective reports of musical interest, ability, hours of listening to music, or subjective engagement with music (as tested by Spearman’s rho tests). These results are presented in Table S1 (OSM). In order to assess for possible influences of musical training on the demonstrated relationship between perceived tension and memory performance, we carried out two regression analyses, with either tension-related d’ or AUC as dependent variables, and differential tension perception and musical background (categorical yes/no) as independent variables. As expected, for both d’ and AUC, the slopes of the perceived tension variable were significant (beta values for perceived tension were significant at P < 0.05 for d’ and P < 0.01 for AUC). In contrast, the slopes of the musical background did not reach significance (P = 0.34 and P = 0.118, respectively). Thus, musical background did not contribute to the explanation of the variability in memory performance.

Discussion

The aim of the current study was to investigate the effect of musical tension and its perception on the formation of declarative memory. We found that, overall, memory performance for images presented during musical tension was superior compared to tension release epochs and silence. Moreover, perception of memory tension was negatively correlated with memory performance, such that the higher the perception of tension, the lower the memory performance (as indexed by d’ and AUC measures) for images presented during musical tension, as compared to the baseline of neutral music. We applied music information retrieval (MIR) analysis to validate the predefined tension condition, and confirmed that this condition differed from tension release and neutral periods only in decay times (indicative of the prolongation of dominant V chords), and not in amplitude (volume) or musical novelty. We also confirmed that musical tension yielded enhanced skin-conductance responses as compared to neutral music. Overall, tension perception seems to act as a mediator between musical tension and its interference with memory performance. Participants who do not detect musical tension are more successful in forming memories for images while listening to tense music, compared to listening to neutral music or silence. These findings are discussed in light of the cognitive capacity model and the arousal-mood hypothesis, as they relate to musical perception and cognition.

Previous studies have found that cognitive performance may be enhanced (Hallam et al., 2002), impaired (Moreno & Mayer, 2000; Ransdell & Gilroy, 2001; Thompson et al., 2011; Woo & Kanachi, 2005), or remain unaffected by background music (Jäncke & Sandmann, 2010; see Lehmann & Seufert, 2017, for a review on this matter). The results of the current study show that memory performance for visual stimuli is enhanced when the stimuli are encoded in the presence of music, whether the music expresses tension or neutral features, in comparison to silence. All participants were unfamiliar with the music they heard during learning, a feature that enabled us to control for any familiarity effects, which were previously shown to interfere with participants’ performance (Büdenbender & Kreutz, 2016; Chew et al., 2016; Parente, 1976). It should be noted that only a handful of previous studies, which assessed the effects of background music on learning, made use of music that was specifically composed for the experimental intervention (de la Mora Velasco & Hirumi, 2020a). To the best of our knowledge, this is the first study that uses original background music to investigate its effects on declarative memory for images. Pertaining to musical training, participants varied in their musical background, but none of them were professional musicians. This fact might have contributed to the general positive effect that the background music exerted, as previous studies have pointed to a negative impact of background music on musicians in comparison to non-musicians, such that professional musicians seem to consume more cognitive resources in analyzing musical properties, as they are more sensitive to tonal structures (Morrison et al., 2003; Patston & Tippett, 2011).

It is noteworthy that the tension condition (and to some extent tension resolution as well) is characterized by minimal auditory input, yet the context of this minimal input still influences cognitive performance. In the tension condition, one chord was prolonged (the dominant fifth), such that in effect, the listener heard a sustained prolongation of the previous chord until tension was released. According to the classical music paradigm, harmonic progressions are interpreted according to their local context, within which the tonal function of particular chords can be determined as creating tension (Bigand & Parncutt, 1999). On the perceptual level, the different contexts of these conditions lead to different experiences. Based on the law of good continuation (Gestalt Psychology), as it applies to musical stimuli, expectations that arise from delays in musical phrases will yield an affective response, unless the process is rationalized on a conscious level (Meyer, 1956, p.88). Therefore, in contrast to the silence condition, the tension condition is accompanied by an expectation, which is resolved only upon the release from tension. In our study, the musical tension condition, which was characterized by minimal musical stimuli, yielded overall superior memory formation in comparison to silence, stressing the importance of musical context on cognitive performance.

The observation that the presence of music was associated with increased visual memory as compared to silence is in line with the arousal-mood hypothesis (Husain et al., 2002). According to this hypothesis, listening to music can affect both physiological arousal and emotional feelings, which may in turn influence cognitive performance in diverse ways (Thompson et al., 2001). In a related study, Bezdek et al. (2017) dissociated between suspense related to the content of selected movie segments and suspense related to background music. They found that the former, but not the latter, increased accuracy for memory recall of movie details. Here we show that musical stimuli, both with and without delays in tension-release, benefit memory formation, as compared to silence. On top of this relationship, we demonstrate a weakening in memory formation for participants who perceive tension music as such, compared to the neutral music condition, and a memory facilitation effect when tension is present in the music but not necessarily perceived as such. This negative correlation between tension perception and memory formation implies that beyond the overall arousing effect that music exerts on listeners’ performance, the perception of specific musical features mediates memory performance. According to the seductive detail effect (Rey, 2012), interesting but otherwise irrelevant information to a given task, such as background music, can grab attention resources and consequently impede learning (Lehmann & Seufert, 2017). This is also in line with the cognitive capacity model, as learners have to invest more cognitive resources to process background music on top of the primary cognitive task (Deutsch, 2013). Mirroring this effect, individuals who do not perceive tension show a facilitating influence of tension on encoding. This raises the possibility that tension conveyed by music elicits an automatic emotional response, thus aiding encoding.

The present study accounts for the complex relationships among musical features, physiological responses to music, and perception of musical elements, to provide insights as to the relation of musical tension and memory performance. Musical tension in the current experiment was elicited by prolonging delays in harmonic resolutions (postponing the root chord that is expected to succeed the dominant chord). Future studies may consider using other musical characteristics known to induce tension, such as changes in tempo, dissonance, and rhythm. The effect that musical expertise may have on cognitive performance while listening to music remains unresolved. In the current study, musical training or activity (as assessed by the music background questionnaire) did not correlate with tension-perception. Similar findings were previously reported, providing evidence that musicians and non-musicians do not differ in their ratings of musical tension for single chords (Lahdelma & Eerola, 2020), sequences of chords (Bigand & Parncutt, 1999), or prolonged musical pieces (Fredrickson, 2000). Since none of our participants were professional musicians, and given the above, it is not surprising that the degree of musicality or training did not correlate with tension perception. Although little is known regarding the interaction between musical expertise and the effects of music on cognition (de la Mora Velasco & Hirumi, 2020b), our results are in line with previous research showing that participants with musical background do not necessarily perform better in memory tasks related to visual stimuli (Talamini et al., 2017).

Finally, since we wished for the tension perception manipulation to be implicit, we measured participants’ felt-tension only after the encoding and memory stages were completed. Thus, we report correlations between memory performance based on information encoded on day 1, and tension perception from the rating task on day 2. Although we also validated the musical tension manipulation in a separate pilot by measuring SCR during the encoding stage, future research could use continuous measures of objective and subjective arousal during the main experiment, enabling direct correlations between arousal, tension, and memory. Additionally, future studies are necessary for examining whether this interfering effect of music on memory formation can be found in musical features other than tension. For instance, if participants were asked to rate changes in tempo or dissonance, would subjective ratings be negatively correlated with memory performance.

In summary, we provide evidence for a trade-off between perception of musical tension and declarative memory, paving the way to a better understanding of the interplay between musical features, emotion, and memory.

(DOCX 1438 kb)