Overtone focusing in biphonic tuvan throat singing.

Khoomei is a unique singing style originating from the republic of Tuva in central Asia. Singers produce two pitches simultaneously: a booming low-frequency rumble alongside a hovering high-pitched whistle-like tone. The biomechanics of this biphonation are not well-understood. Here, we use sound analysis, dynamic magnetic resonance imaging, and vocal tract modeling to demonstrate how biphonation is achieved by modulating vocal tract morphology. Tuvan singers show remarkable control in shaping their vocal tract to narrowly focus the harmonics (or overtones) emanating from their vocal cords. The biphonic sound is a combination of the fundamental pitch and a focused filter state, which is at the higher pitch (1-2 kHz) and formed by merging two formants, thereby greatly enhancing sound-production in a very narrow frequency range. Most importantly, we demonstrate that this biphonation is a phenomenon arising from linear filtering rather than from a nonlinear source.

Introduction

In the years preceding his death, Richard Feynman had been attempting to visit the small republic of Tuva located in geographic center of Asia (Leighton, 2000). A key catalyst came from Kip Thorne, who had gifted him a record called Melody tuvy, featuring a Tuvan singing in a style known as Khoomei, or Xöömij. Although he was never successful in visiting Tuva, Feynman was nonetheless captivated by Khoomei, which can be best described as a high-pitched tone, similar to a whistle carrying a melody, hovering above a constant booming low-frequency rumble. This is a form of biphonation, or in Feynman’s own words, "a man with two voices". Khoomei, now a part of the UNESCO Intangible Cultural Heritage of Humanity, is characterized as "the simultaneous performance by one singer of a held pitch in the lower register and a melody … in the higher register" (Aksenov, 1973). How, indeed, does one singer produce two pitches at one time? Even today, the biophysical underpinnings of this biphonic human vocal style are not fully understood.

Normally, when a singer voices a song or speech, their vocal folds vibrate at a fundamental frequency (), generating oscillating airflow, forming the so-called source. This vibration is not, however, simply sinusoidal, as it also produces a series of harmonics tones (i.e., integer multiples of ) (Figure 1). Harmonic frequencies in this sound above are called overtones. Upon emanating from the vocal folds, they are then sculpted by the vocal tract, which acts as a spectral filter. The vocal-tract filter has multiple resonances that accentuate certain clusters of overtones, creating formants. When speaking, we change the shape of our vocal tract to shift formants in systematic ways characteristic of vowel and consonant sounds. Indeed, singing largely uses vowel-like sounds (Story, 2016). In most singing, the listener perceives only a single pitch associated with the of the vocal production, with the formant resonances determining the timbre. Khoomei has two strongly emphasized pitches: a low-pitch drone associated with the , plus a melody carried by variation in the higher frequency formant that can change independently (Kob, 2004). Two possible loci for this biphonic property are the source and/or the filter.

Figure 1.

Frequency spectra for three different singers transitioning from normal to biphonic singing.

Vertical white lines in the spectrograms (left column) indicate the time point for the associated spectrum in the right column. Transition points from normal to biphonic singing state are denoted by the red triangle. The fundamental frequency () of the song is indicated by a peak in the spectrum marked by a green square. Overtones, which represent integral multiples of this frequency, are also indicated (black circles). Estimates of the formant structure are shown by overlaying a red dashed line and each formant peak is marked by an x. Note that the vertical scale is in decibels (e.g., a 120 dB difference is a million-fold difference in pressure amplitude). See also Appendix 1—figure 1 and Appendix 1—figure 2 for further quantification of these waveforms. The associated waveforms can be accessed in the Appendix [T1_3short.wav, T2_5short.wav, T3_2shortA.wav].

Appendix 1—figure 1.

Same as Figure 1 (middle left panel; subject T2, same sound file as shown in the middle panel of Figure 1), except with overtones and estimated formant structure tracked across time.

Appendix 1—figure 2.

Same data/layout as in Figure 1 but now showing as defined in the 'Materials and methods'.

These plots show the energy ratio focused between 1–2 kHz. Vertical red dashed lines indicate approximate time of transition into the focused state. An expanded timescale is also shown for singer T2 (middle panel) in Appendix 1—figure 3.

A source-based explanation could involve different mechanisms, such as two vibrating nonlinear sound sources in the syrinx of birds, which produce multiple notes that are harmonically unrelated (Fee et al., 1998; Zollinger et al., 2008). Humans however are generally considered to have only a single source, the vocal folds. But there are an alternative possibilities: for instance, the source could be nonlinear and produce harmonically-unrelated sounds. For example, aerodynamic instabilities are known to produce biphonation (Mahrt et al., 2016). Further, Khoomei often involves dramatic and sudden transitions from simple tonal singing to biophonation (see Figure 1 and the Appendix for associated audio samples). Such abrupt changes are often considered hallmarks of physiological nonlinearity (Goldberger et al., 2002), and vocal production can generally be nonlinear in nature (Herzel and Reuter, 1996; Mergell and Herzel, 1997; Fitch et al., 2002; Suthers et al., 2006). Therefore it remains possible that biphonation arises from nonlinear source considerations.

Vocal tract shaping, a filter-based framework, provides an alternative explanation for biphonation. In one seminal study of Tuvan throat singing, Levin and Edgerton examined a wide variety of song types and suggested that there were three components at play. The first two (‘tuning a harmonic’ relative to the filter and lengthening the closed phase of the vocal fold vibration) represented a coupling between source and filter. But it was the third, narrowing of the formant, that appeared crucial. Yet, the authors offered little empirical justification for how these effects are produced by the vocal tract shape in the presented radiographs. Thus it remains unclear how the high-pitched formant in Khoomei was formed (Grawunder, 2009). Another study (Adachi and Yamada, 1999) examined a throat singer using magnetic resonance imaging (MRI) and captured static images of the vocal tract shape during singing. These images were then used in a computational model to produce synthesized song. Adachi and Yamada argued that a "rear cavity" was formed in the vocal tract and its resonance was essential to biphonation. However, their MRI data reveal limited detail since they were static images of singers already in the biphonation state. Small variations in vocal tract geometry can have pronounced effects on produced song (Story et al., 1996) and data from static MRI would reveal little about how and which parts of the vocal tract change shape as the singers transition from simple tonal song to biphonation. To understand which features of vocal tract morphology are crucial to biophonation, a dynamic description of vocal tract morphology would be required.

Here we study the dynamic changes in the vocal tracts of multiple expert practitioners from Tuva as they produce Khoomei. We use MRI to acquire volumetric 3D shape of the vocal tract of a singer during biphonation. Then, we capture the dynamic changes in a midsagittal slice of the vocal tract as singers transition from tonal to biphonic singing while making simultaneous audio recordings of the song. We use these empirical data to guide our use of a computational model, which allows us to gain insight into which features of vocal tract morphology are responsible for the singing phonetics observed during biophonic Khoomei song (e.g., Story, 2016). We focus specifically on the Sygyt (or Sigit) style of Khoomei (Aksenov, 1973).

Results

Audio recordings

We made measurements from three Tuvan singers performing Khoomei in the Sygyt style (designated as T1–T3) and one (T4) in a non-Sygyt style. Songs were analyzed using short-time Fourier transforms (STFT), which provide detailed information in both temporal and spectral domains. We recorded the singers transitioning from normal singing into biphonation, Figure 1 showing this transition for three singers. The of their song is marked in the figure (approximately 140 Hz for subject T2, 164 Hz for both T1 and T3) and the overtone structure appears as horizontal bands. Varying degrees of vibrato can be observed, dependent upon the singer (Figure 1; see also longer spectrograms in Appendix 1—figure 6 and Appendix 1—figure 7). Most of the energy in their song is concentrated in the overtones and no subharmonics (i.e., peaks at half-integer multiples of ) are observed. In contrast to these three singers, singer T4 performing in a non-Sygyt style exhibited a fundamental frequency of approximately 130 Hz, although significant energy additionally appears around 50–55 Hz, well below an expected subharmonic (Appendix 1—figure 5).

Appendix 1—figure 6.

Spectrogram of the entire T2_5.wav sound file.

The sample rate was 96 kHz. The analysis parameters used were the same as those used for Figure 5.

Appendix 1—figure 7.

Spectrogram of the first song segment of the T1_3.wav sound file.

The analysis parameters used were the same as those for Figure 5.

Appendix 1—figure 5.

Spectrogram for singer T4 singing in non-Sygyt style (first song segment of T2_4shortA.wav sound file).

For the spectrogram, 4096 point windows were used for the fast Fourier transform (FFT) with 95% fractional overlap and a Hamming window.

If we take a slice, that is a time-point from the spectrogram and plot the spectrum, we can observe the peaks to infer the formant structure from this representation of the sound (red-dashed lines in Figure 1 and Appendix 1—figure 4). As the singers transition from normal singing to biphonation, we see that the formant structure changes significantly and the positions of formant peaks shift dramatically and rapidly. Note that considering time points before and after the transitions also provides an internal control for both normal and focused song types (Appendix 1—figure 4). Once in the biphonation mode, all three singers demonstrate overtones in a narrow spectral band around 1.5–2 kHz; we refer to this as the focused state. Specifically, Figure 1 shows that not only is just a single or small group of overtones accentuated, but also that nearby ones are greatly attenuated: ±1 overtones are as much 15–35 dB and ±2 overtones are 35–65 dB below the central overtone. Whereas the energy in the low-frequency region associated with the first formant (below 500 Hz) is roughly constant between the normal-singing and focused states, there is a dramatic change in the spectrum for the higher formants above 500 Hz. In normal singing (i.e., prior to the focused state), spectral energy is distributed across several formants between 500 and 4000 Hz. In the focused state after the transition, the energy above 500 Hz becomes narrowly focused in the 1.5–2 kHz region, generating a whistle-like pitch that carries the song melody.

Appendix 1—figure 4.

Stemming directly from Figure 1, the right-hand column now shows a spectrum from a time point prior to transition into the focused state (as denoted by the vertical black lines in the left column).

The shape of the spectra from Figure 1 is also included for reference.

To assess the degree of focus objectively and quantitatively, we computed an energy ratio that characterizes the relative degree of energy brought into a narrow band against the energy spread over the full spectrum occupied by human speech (see Materials and methods). In normal speech and singing, for , typically is small (i.e., energy is spread across the spectrum, not focused into that narrow region between 1 and 2 kHz). For the Tuvan singers, prior to a transition into a focused state, is similarly small. However once the transition occurs (red triangle in Figure 1), those values are large (upwards of 0.5 and higher) and sustained across time (Appendix 1—figure 2 and Appendix 1—figure 3). For one of the singers (T2) the situation was more complex, as he created multiple focused formants (Figure 1 middle panels and Appendix 1—figure 6, Appendix 1—figure 8). The second focused state was not explicitly dependent upon the first: The first focused state clearly moves and transitions between approximately 1.5–2 kHz (by 30%) while the second focused state remains constant at approximately 3–3.5 kHz (changing less than 1%). Thus the focused states are not harmonically related. Unlike the other singers, T2 not only has a second focused state, but also had more energy in the higher overtones (Figure 1). As such, singer T2 also exhibited a different time course, which took on values that could be relatively large even prior to the transition. This may be because he took multiple ways to approach the transition into a focused state (e.g., Appendix 1—figure 9).

Appendix 1—figure 3.

Similar to Figure 2 for singer T2 (middle panel), except an expanded time scale is shown to demonstrate the earlier dynamics as this singer approaches the focused state (see T2_5longer.wav).

Appendix 1—figure 8.

Singer T2's transition into a focused state.

Note that while the first focused state transitions from approximately 1.36 to 1.78 kHz, the second state remains nearly constant, decreasing only slightly from 3.32 to 3.17 kHz (T2_1shortB.wav).

Appendix 1—figure 9.

Spectrogram of singer T2 exhibiting pressed voicing heading into transition to focused state (T2_2short.wav).

Plotting spectra around the transition from normal to biphonation singing in a waterfall plot indicates that the sharp focused filter is achieved by merging two broader formants together ( and in Figure 2; Kob, 2004). This transition into the focused state is fast (∼40–60 ms), as are the shorter transitions within the focused state where the singer melodically changes the filter that forms the whistle-like component of their song (Figure 1, Appendix 1—figure 8).

Figure 2.

A waterfall plot representing the spectra at different time points as singer T2 transitions from normal singing into biphonation (T2_3short.wav).

The superimposed arrows are color-coded to help visualize how the formants change about the transition, chiefly with F3 shifting to merge with F2. This plot also indicates the second focused state centered just above 3 kHz is a sharpened F4 formant.

Vocal tract MRI

While we can infer the shape of the formants in Khoomei by examining audio recordings, such analysis is not conclusive in explaining the mechanism used to achieve these formants. The working hypothesis was that vocal tract shape determines these formants. Therefore, it was crucial to examine the shape and dynamics of the vocal tract to determine whether the acoustic measurements are consistent with this hypothesis. To accomplish this, we obtained MRI data from one of the singers (T2) that are unique in two regards. First, there are two types of MRI data reported here: steady-state volumetric data Figure 3 and Appendix 1—figure 18) and dynamic midsagittal images at several frames per second that capture changes in vocal tract position (Figure 4A–B and Appendix 1—figure 20). Second is that the dynamic data allow us to examine vocal tract changes as song transitions into a focused state (e.g., Appendix 1—figure 20).

Figure 3.

3-D reconstruction of volumetric MRI data taken from singer T2 (Run3; see Appendix, including Appendix 1—figure 18).

(A) Example of MRI data sliced through three different planes, including a pseudo-3D plot. Airspaces were determined manually (green areas behind tongue tip, red for beyond). Basic labels are included: L – lips, J – jaw, To– tongue, AR – alveolar ridge, V – velum, E – epiglottis, Lx – larynx, and Tr – trachea. The shadow from the dental post is visible in the axial view on the left hand side and stops near the midline leaving that view relatively unaffected. (B) Reconstructed airspace of the vocal tract from four different perspectives. The red circle highlights the presence of the piriform sinuses (Dang and Honda, 1997).

Appendix 1—figure 18.

Mosaic of single slices from the volumetric MRI scan (Run3) of subject T2 during focused overtone state.

Spectrogram of corresponding audio shown in Appendix 1—figure 19.

Figure 4.

Analysis of vocal tract configuration during singing.

(A) 2D measurement of tract shape. The inner and outer profiles were manually traced, whereas the centerline (white dots) was found with an iterative bisection technique. The distance from the inner to outer profile was measured along a line perpendicular to each point on the centerline (thin white lines). (B) Collection of cross-distance measurements plotted as a function of distance from the glottis. Area function can be computed directly from these values and is derived by assuming the cross-distances to be equivalent diameters of circular cross-sections (see Materials and methods). (C) Schematic indicating associated modeling assumptions, including vocal tract configuration as in panel B (adapted from Bunton et al. (2013), under a Creative Commons CC-BY license, https://creativecommons.org/licenses/by/4.0/). (D) Model frequency response calculated from the associated area function stemming from panels B and C. Each labeled peak can be considered a formant frequency and the dashed circle indicates merging of formants F2 and F3.

Appendix 1—figure 20.

Representative movie frames and their corresponding spectra for singer T2, as input into modeling parameters (e.g., Figure 5).

The corresponding Appendix data files are DynamicRun2S.mov (MRI images) and DynamicRun2sound.wav (spectra; see also DynamicRun2SGrid.pdf). The top row shows a ‘low pitch’ (first) focused state at about 1.3 kHz whereas the bottom row shows a ‘high’ pitch at approximately 1.9 kHz. Note a key change is that the back of the tongue moves forward to shift from the low to the high pitch. Thin gray bars are added to the spectra to help to highlight the frequency difference. The legend is the same as that shown in Figure 1.

The human vocal tract begins at the vocal folds and ends at the lips. Airflow produced by the vocal cords sets the air-column in the tract into vibration, and its acoustics determine the sound that emanates from the mouth. The vocal tract is effectively a tube-like cavity whose shape can be altered by several articulators: the jaw, lips, tongue, velum, epiglottis, larynx and trachea (Figure 4C). Producing speech or song requires that the shape of the vocal tract, and hence its acoustics, are precisely controlled (Story, 2016).

Several salient aspects of the vocal tract during the production of Khoomei can be observed in the volumetric MRI data. The most important feature however, is that there are two distinct and relevant constrictions when in the focused state, corresponding roughly to the uvula and alveolar ridge. Additionally, the vocal tract is expanded in the region just anterior to the alveolar ridge (Figure 4A). The retroflex position of the tongue tip and blade produces a constriction at 14 cm, and also results in the opening of this sublingual space. It is the degree of constriction at these two locations that is hypothesized to be the primary mechanism for creating and controlling the frequency at which the formant is focused.

Modeling

Having established that the shape of vocal tract during Khoomei does indeed have two constrictions, consistent with observations from other groups, the primary goals of our modeling efforts were to use the dynamic MRI data as morphological benchmarks and capture the merging of formants to create the focused states as well as the dynamic transitions into them. Our approach was to use a well-established linear "source/filter" model (e.g., Stevens, 2000) that includes known energy losses (Sondhi and Schroeter, 1987; Story et al., 2000; Story, 2013). Here, the vibrating vocals folds act as the broadband sound source (with the and associated overtone cascade), while resonances of the vocal tract, considered as a series of 1-D concatenated tubes of variable uniform radius, act as a primary filter. We begin with a first order assumption that the system behaves linearly, which allows us for a simple multiplicative relationship between the source and filter in the spectral domain (e.g., Appendix 1—figure 10).

Appendix 1—figure 10.

Overview of source/filter theory, as advanced by Stevens (2000).

The left column shows normal phonation, whereas the right indicates one example of a focused state.

Acoustic characteristics of the vocal tract can be captured by transforming the three-dimensional configuration (Figure 3) into a tube with variation in its cross-sectional area from the glottis to the lips (Figure 4 and Figure 5). This representation of the vocal tract shape is called an area function, and allows for calculation of the corresponding frequency response function (from which the formant frequencies can be determined) with a one-dimensional wave propagation algorithm. Although the area function can be obtained directly from a 3D vocal tract reconstruction (e.g., Story et al., 1996), the 3D reconstructions of the Tuvan singer’s vocal tract were affected by a large shadow from a dental post (e.g., see Figure 4) and were not amenable to detailed measurements of cross-sectional area. Instead, a cross-sectional area function was measured from the midsagittal slice of the 3D image set (see Materials and methods and Appendix for details). Thus, the MRI data provided crucial bounds for model parameters: the locations of primary constrictions and thereby the associated area functions.

Figure 5.

Results of changing vocal tract morphology in the model by perturbing the baseline area function to demonstrate the merging of formants and , atop two separate overtones as apparent in the two columns of panels A and B.

(A) The frames from dynamic MRI with red and blue dashed circles highlighting the location of the key vocal tract constrictions. (B) Model-based vocal tract shapes stemming from the MRI data, including both the associated area functions (top inset) and frequency response functions (bottom inset). indicates the constriction near the alveolar ridge while the constriction near the uvula in the upper pharynx. (C) Waveform and corresponding spectrogram of audio from singer T2 (a spectrogram from the model is shown in Appendix 1—figure 14). Note that the merged formants lie atop either the 7th overtone (i.e., ) or the 11th (i.e., ).

Appendix 1—figure 14.

Similar to Figure 5, but additional manipulations were considered to create a second focused state by merging F4 and F5, as exhibited by singer T2 (see middle row in Figure 1).

In addition, the spectrogram shown here is from the model (not the singer’s audio). See also Appendix 1—figure 20 for connections back to dynamic MRI data.

The frequency response functions derived from the above static volumetric MRI data (e.g., Figure 4D) indicate that two formants and cluster together, thus enhancing both their amplitudes. Clearly, if and could be driven closer together in frequency, they would merge and form a single formant with unusually high amplitude. We hypothesize that this mechanism could be useful for effectively amplifying a specific overtone, such that it becomes a prominent acoustic feature in the sound produced by a singer, specifically the high frequency component of Khoomei.

Next, we used the model in conjunction with time-resolved MRI data to investigate how the degree of constriction and expansion at different locations along the vocal tract axis could be a mechanism for controlling the transition from normal to overtone singing and the pitch while in the focused state. These results are summarized in Figure 5 (further details are in the Appendix). While the singers are in the normal song mode, there are no obvious strong constrictions in their vocal tracts (e.g., Appendix 1—figure 11). After they transition, in each MRI from the focused state, we observe a strong constriction near the alveolar ridge. We also observe a constriction near the uvula in the upper pharynx, but the degree of constriction here varies. If we examine the simultaneous audio recordings, we find that variations in this constriction are co-variant with the frequency of the focused formant. From this, we surmise that the mechanism for controlling the enhancement of voice harmonics is the degree of constriction near the alveolar ridge in the oral cavity (labeled in Figure 5), which affects the proximity of and to each other (Appendix 1—figure 12). Additionally, the degree of constriction near the uvula in the upper pharynx () controls the actual frequency at which and converge (Appendix 1—figure 13). Other parts of the vocal tract, specifically the expansion anterior to , may also contribute since they also show small co-variations with the focused formant frequency (Appendix 1—figure 14). Further, a dynamic implementation of the model, as shown in Appendix 1—figure 14, reasonably captures the rapid transition into/out of the focused state as shown in Figure 1. Taken together, the model confirms and explains how these articulatory changes give rise to the observed acoustic effects.

Appendix 1—figure 11.

Setup of the baseline vocal tract configuration used in the modeling study.

(a) The area function () is in the lower panel and its frequency response is in the upper panel. (b) The area function from (a) is shown as a pseudo-midsagittal plot (see text).

Appendix 1—figure 12.

Results of perturbing the baseline area function so that and converge on 1800 Hz.

(a) Perturbed area function (thick black line) and the corresponding frequency response; for comparison, the baseline area function is also shown (thin gray line). The frequency response shows the convergence of and into one high amplitude peak centered around 1800 Hz. (b) Pseudo-midsagittal plot of the perturbed area function (thick black line) and the baseline area function (thin gray line).

Appendix 1—figure 13.

Results of perturbing the baseline area function so that and converge on 1200, 1350, 1500, 1650, and 1800 Hz.

(A) Perturbed area functions and corresponding frequency responses; line thicknesses and gray scale are matched in the upper and lower panels. (B) Pseudo-midsagittal plot of the perturbed area functions. The circled regions (dotted) denote constrictions that control the proximity of and to each other and the frequency at which they converge.

To summarize, an overtone singer could potentially ‘play’ (i.e., select) various harmonics of the voice source by first generating a tight constriction in the oral cavity near the alveolar ridge, and then modulating the degree of constriction in the uvular region of the upper pharynx to vary the position of the focused formant, thereby generating a basis for melodic structure.

Discussion

This study has shown that Tuvan singers performing Sygyt-style Khoomei exercise precise control of the vocal tract to effectively merge multiple formants together. They morph their vocal tracts so to create a sustained focused state that effectively filters an underlying stable array of overtones. This focused filter greatly accentuates energy of a small subset of higher order overtones primarily in the octave-band spanning 1–2 kHz, as quantified by an energy ratio . Some singers are even capable of producing additional foci at higher frequencies. Below, we argue that a linear framework (i.e., source/filter model, Stevens, 2000) appears sufficient to capture this behavior including the sudden transitions into a focused state, demonstrating that nonlinearities are not a priori essential. That is, since the filter characteristics are highly sensitive to vocal tract geometry, precise biomechanical motor control of the singers is sufficient to achieve a focused state without invoking nonlinearities or a second source as found in other vocalization types (e.g., Herzel and Reuter, 1996; Fee et al., 1998). Lastly, we describe several considerations associated with how focused overtone song produces such a salient percept by virtue of a pitch decoherence.

Source or filter?

The notion of a focused state is mostly consistent with vocal tract filter-based explanations for biphonation in previous studies (e.g., Bloothooft et al., 1992; Edgerton et al., 1999; Adachi and Yamada, 1999; Grawunder, 2009), where terms such as an ‘interaction of closely spaced formants’, ‘reinforced harmonics’, and ‘formant melting’ were used. In addition, the merging of multiple formants is closely related to the ‘singer’s formant’, which is proposed to arise around 3 kHz due to formants – combining (Story, 2016), though this is typically broader and less prominent than the focused states exhibited by the Tuvans. Our results explain how this occurs and are also broadly consistent with Adachi and Yamada (1999) in that a constricted ‘rear cavity’ is crucial. However, we find that this rear constriction determines the pitch of the focused formant, whereas it is the ‘front cavity’ constriction near the alveolar ridge that produces the focusing effect (i.e., merging of formants and ).

Further, the present data appear in several ways inconsistent with conclusions from previous studies of Khoomei, especially those that center on effects that arise from changes in the source. Three salient examples are highlighted. First, we observed overtone structure to be highly stable, though some vibrato may be present. This contrasts the claim by Levin and Edgerton (1999) that “(t)o tune a harmonic, the vocalist adjusts the fundamental frequency of the buzzing sound produced by the vocal folds, so as to bring the harmonic into alignment with a formant’. That is, we see no evidence for the overtone ‘ladder’ being lowered or lifted as they suggested (note in Figure 1, is held nearly constant). Further, this stability argues against a transition into a different mode of glottal pulse generation, which could allow for a ‘second source’ (Mergell and Herzel, 1997). Second, a single sharply defined harmonic alone is not sufficient to get the salient perception of a focused state, as had been suggested by Levin and Edgerton (1999). Consider Appendix 1—figure 9, especially at the 4 s mark, where the voicing is ‘pressed'. Pressed phonation, also referred to as ventricular voice, occurs when glottal flow is affected by virtue of tightening the laryngeal muscles such that the ventricular folds are brought into vibration. This has the perceptual effect of adding a degree of roughness to the voice sound (Lindestad et al., 2001; Edmondson and Esling, 2006). There, a harmonic at 1.51 kHz dominates (i.e., the two flanking overtones are approximately 40 dB down), yet the song has not yet perceptibly transitioned. It is not until the cluster of overtones at 3–3.5 kHz is brought into focus that the perceptual effect becomes salient, perhaps because prior to the 5.3 s mark the broadband nature of those frequencies effectively masks the first focused state. Third, we do not observe subharmonics, which contrasts a prior claim (Lindestad et al., 2001) that ”(t)his combined voice source produces a very dense spectrum of overtones suitable for overtone enhancement’. However, that study was focused on a different style of song called ‘Kargyraa’, which does not exhibit as clearly a focused state as in Sygyt.

Linear versus nonlinear mechanisms

An underlying biophysical question is whether focused overtone song arises from inherently linear or nonlinear processes. Given that Khoomei consists of the voicing of two or more pitches at once and exhibits dramatic and fast transitions from normal singing to biphonation, nonlinear phenomena may seem like an obvious candidate (Herzel and Reuter, 1996). It should be noted that Herzel and Reuter (1996) go so far to define biphonation explicitly through the lens of nonlinearity. We relax such a definition and argue for a perceptual basis for delineating the boundaries of biphonation. Certain frog species exhibit biphonation, and it has been suggested that their vocalizations can arise from complex nonlinear oscillatory regimes of separate elastically coupled masses (Suthers et al., 2006). Further, the appearance of abrupt changes in physiological systems (as seen in Figure 1) has been argued to be a flag for nonlinear mechanisms (Goldberger et al., 2002); for example, by virtue of progression through a bifurcation.

Our results present two lines of evidence that argue against Sygyt-style Khoomei arising primarily from a nonlinear process. First, the underlying harmonic structure of the vocal fold source appears highly stable through the transition into the focused state (Figure 1). There is little evidence of subharmonics. A source spectral structure that is comprised of an and integral harmonics would suggest a primarily linear source mechanism. Second is that our modeling efforts, which are chiefly linear in nature, reasonably account for the sudden and salient transition. That is, the model is readily sufficient to capture the characteristic that small changes in the vocal tract can produce large changes in the filter. Thereby, precise and fast motor control of the articulators in a linear framework accounts for the transitions into and out of the focused state. Thus, in essence, Sygyt-style Khoomei could be considered a linear means to achieve biphonation. Connecting back to nonlinear phonation mechanisms in non-mammals, our results provide further context for how human song production and perception may be similar and/or different relative to that of non-humans (e.g., Doolittle et al., 2014; Kingsley et al., 2018).

Nevertheless, features that appear transiently in spectrograms do provide hints of source nonlinearity, such as the brief appearance of subharmonics in some instances (Appendix 1—figure 15B). This provides an opportunity to address the limitations of the current modeling efforts and to highlight future considerations. We suggest that further analysis (e.g., Theiler et al., 1992; Tokuda et al., 2002; Kantz and Schreiber, 2004) of Khoomei audio recordings may help to inform the model and might better capture focused filter sharpness and the origin of secondary focused states. Several potential areas for improvement are: nonlinear source–filter coupling (Titze et al., 2008); a detailed model of glottal dynamics (e.g., ratio of open/closed phases in glottal flow [Grawunder, 2009; Li and Hou, 2017], and periodic vibrations in ); inclusion of piriform sinuses as side-branch resonators (Dang and Honda, 1997; Titze and Story, 1997); inclusion of the 3-D geometry; and detailed study of the front cavity (e.g., lip movements) that may be used by the singer to maintain control of the focused state and to make subtle manipulations.

Appendix 1—figure 15.

Brief instability in the focused state.

(A) Spectrogram of singer T3 during period during which the focused state briefly falters (T3_2shortB.wav, extracted from around the 33 s mark of T3_2.wav). (B) Spectral slices taken at two different time points (vertical white lines in panel A at 0.2 and 0.96 s), the latter falling in the transient unstable state. Note that while there is little change in between the two periods (170 Hz versus 164 Hz), the unstable period shows a period doubling such that the subharmonic (i.e., /2) and associated overtones are now present, indicative of nonlinear phonation.

Perceptual consequences of overtone focusing

Although this study did not directly assess the percept associated with these vocal productions, the results raise pressing questions about how the spectro-temporal signatures of biphonic Khoomei described here create the classical perception of Sygyt-style Khoomei as two distinct sounds (Aksenov, 1973). The first, the low-pitched drone, which is present during both the normal singing and the focused-state biphonation intervals, reflects the pitch associated with , extracted from the harmonic representation of the stimulus. It is well established that the perceived pitch of a broadband sound comprised of harmonics reflects the derived primarily from the perceptually resolved harmonics up to about  (Bernstein and Oxenham, 2003). The frequency resolution of the peripheral auditory system is such that these low-order harmonics are individually resolved by the cochlea, and it appears that such filtering is an important prerequisite for pitch extraction associated with that common . The second sound, the high-pitched melody, is present only during the focused-state intervals and probably reflects a pitch associated with the focused formant. An open question, however, is why this focused formant would be perceived incoherently as a separate pitch (Shamma et al., 2011), when it contains harmonics at multiples of . The auditory system tends to group together concurrent harmonics into a single perceived object with a common pitch (Roberts et al., 2015), and the multiple formants of a sung or unsung voice are not generally perceived as separate sounds from the low harmonics.

The fact that the focused formant is so narrow apparently leads the auditory system to interpret this sound as if it were a separate tone, independent of the low harmonics associated with the drone percept, thereby effectively leading to a pitch decoherence. This perceptual separation could be attributable to a combination of both bottom-up (i.e., cochlear) and top-down (i.e., perceptual) factors. From the bottom-up standpoint, even if the focused formant is broad enough to encompass several harmonic components, the fact that it consists of harmonics at or above 10  (i.e., the 1500 Hz formant frequency represents the 10th harmonic of a 150 Hz ) means that these harmonics will not be spectrally resolved by cochlear filtering (Bernstein and Oxenham, 2003). Instead, the formant will be represented as a single spectral peak, similar to the representation of a single pure tone at the formant frequency. Although the interaction of harmonic components at this cochlear location will generate amplitude modulation at a rate equal to the  (Plack and Oxenham, 2005), it has been argued that a common is a weak cue for binding low- and high-frequency formants (Culling and Darwin, 1993). Rather, other top-down mechanisms of auditory-object formation may play a more important role in generating a perception of two separate objects in Khoomei. For example, the rapid onsets of the focused formant may enhance its perceptual separation from the constant drone (Darwin, 1984). Further, the fact that the focused formant has a variable frequency (i.e., frequency modulation, or FM) while the drone maintains a constant is another difference that could facilitate their perceptual separation. Although it has been argued that FM differences between harmonic sounds generally have little influence on their perceived separation (Darwin, 2005), others have reported enhanced separation in the special case in which one complex was static and the other had applied FM (Summerfield and Culling, 1992) – similar to the first and second formants during the Tuvan focused state.

The perceptual separation of the two sounds in the Tuvan song might be further affected by a priori expectations about the spectral qualities of vocal formants (Billig et al., 2013). Because a narrow formant occurs so rarely in natural singing and speech, the auditory system might be pre-disposed against perceiving it as a phonetic element, limiting its perceptual integration with the other existing formants. Research into ‘sine-wave speech’ provides some insights into this phenomenon. When three or four individual frequency-modulated sinusoids are presented at formant frequencies in lieu of natural formants, listeners can, with sufficient training, perceive the combination as speech (Remez et al., 1981). Nevertheless, listeners largely perceive these unnatural individual pure tones as separate auditory objects (Remez et al., 2001), much like the focused formant in Khoomei. Further research exploring these considerations would help close the production–perception circle underlying the unique percept arising from Tuvan throat song.

Materials and methods

Acoustical recordings

Recordings were made at York University (Toronto, ON, Canada) in a double-walled acoustic isolation booth (IAC) using a Zoom H5 24-bit digital recorder and an Audio-Technica P48 condenser microphone. A sample rate of 96 kHz was used. Spectral analysis was done using custom-coded software in Matlab. Spectrograms were typically computed using 4096 point window segments with 95% fractional overlap and a Hamming window. Harmonics (black circles in Figure 1) were estimated using a custom-coded peak-picking algorithm. Estimated formant trends (red dashed lines in Figure 1) were determined using a lowpass interpolating filter built into Matlab’s digital signal processing toolbox with a scaling factor of 10. From this trend, the peak-picking was reapplied to determine ‘formant’ frequencies (red 'x's in Figure 1). This process could be repeated across the spectrogram to track overtone and formant frequency/strength effectively, as shown in Appendix 1—figure 1.

To quantify the focused states, we developed a dimension-less measure to represent the energy ratio of that spanning a frequency range relative to the entire spectral output. This can be readily computed from the spectrogram data as follows. First take a ‘slice’ from the spectrogram and convert spectral magnitude to linear ordinate and square it (as intensity is proportional to pressure squared). Then integrate across frequency, first for a limited range spanning (e.g., 1–2 kHz) and then for a broader range of (e.g., 0–8 kHz; 8 kHz is a suitable maximum as there is little acoustic energy in vocal output above this frequency). The ratio of these two is then defined as , and takes on values between 0 and 1. This can be expressed more explicitly as:where is the magnitude of the scaled sound pressure, is frequency, and  and are filter limits for considering the focused state. The choice of kHz has the virtue of spanning an octave, which also closely approximates the ‘seventh octave’ from about C6 to C7. did not depend significantly upon the length of the fast Fourier transform (FFT) window. Values of for the waveforms used in Figure 1 are shown in Appendix 1—figures 2 and 3.

MRI acquisition and volumetric analysis

MRI images were acquired at the York MRI Facility on a 3.0 Tesla MRI scanner (Siemens Magnetom TIM Trio, Erlangen, Germany), using a 12-channel head coil and a neck array. Data were collected with the approval of the York University Institutional Review Board. The participant was fitted with an MRI compatible noise-cancelling microphone (Optoacoustics, Mazor, Israel) mounted directly above the lips. The latency of the microphone and noise-cancelling algorithm was 24 ms. Auditory recordings were made in QuickTime on an iMac during the scans to verify performance.

Images were acquired using one of two paradigms, static or dynamic. Static images were acquired using a T1-weighted 3D gradient echo sequence in the sagittal orientation with 44 slices centered on the vocal tract, TR = 2.35 ms, TE = 0.97 ms, flip angle = 8 degrees, FoV = 300 mm, and a voxel dimension of 1.2 × 1.2×1.2 mm. Total acquisition time was 11 s. The participant was instructed to begin singing a tone, and to hold it in a steady state for the duration of the scan. The scan was started immediately after the participant began to sing and had reached a steady state. Audio recordings verified a consistent tone for the duration of the scan. Dynamic images were acquired using a 2D gradient echo sequence. A single 10.0 mm thick slice was positioned in a sagittal orientation along the midline of the vocal tract, TR = 4.6 ms, TE = 2.04 ms, flip angle = 8 degrees, FoV = 250 mm, and a voxel dimension of 2.0 × 2.0×10.0 mm. One hundred measurements were taken for a scan duration of 27.75 s. The effective frame rate of the dynamic images was 3.6 Hz. Audio recordings were started just prior to scanning. Only subject T2 participated in the MRI recordings. The participant was instructed to sing a melody for the duration of the scan, and took breaths as needed.

For segmentation (Figure 3), 3D MRI images (Run1; see Appendix) were loaded into Slicer (version 4.6.2 r25516). The air-space in the oral cavity was manually segmented using the segmentation module, identified and painted in slice by slice. Careful attention was paid to the parts of the oral cavity that were affected by the artifact from the dental implant. The air cavity was manually repainted to be approximately symmetric in this region using the coronal and axial view (Figure 3A). Once completely segmented, the sections were converted into a 3D model and exported as a STL file. This mesh file was imported into MeshLab (v1.3.4Beta) for cleaning and repairing the mesh. The surface of the STL was converted to be continuous by removing non-manifold faces and then smoothed using depth and Laplacian filters. The mesh was then imported into Meshmixer where further artifacts were removed. This surface-smoothed STL file was finally reimported into Slicer, generating the display in Figure 3B.

Computational modeling

Measurement of the cross-distance function is illustrated in Figure 4. The inner and outer profiles of the vocal tract were first determined by manual tracing of the midsagittal image. A 2D iterative bisection algorithm (Story, 2007) was then used to find the centerline within the profiles extending from the glottis to the lips, as shown by the white dots in Figure 4A. Perpendicular to each point on the centerline, the distance from the inner to outer profiles was measured to generate the cross-distance function shown in Figure 4B; the corresponding locations of the anatomic landmarks shown in the midsagittal image are also indicated on the cross-distance function.

The cross-distance function, , can be transformed to an approximate area function, , with the relation , where and are a scaling factor and exponent, respectively. If the elements of are considered to be diameters of a circular cross-section, and . Although other values of and have been proposed to account for the complex shape of the vocal tract cross-section (Heinz and Stevens, 1964; Lindblom and Sundberg, 1971; Mermelstein, 1973), there is no agreement on a fixed set of numbers for each parameter. Hence, the circular approximation was used in this study to generate an estimate of the area function. In Figure 4C, the area function is plotted as its tubular equivalent, where the radii were rotated about an axis to generate circular sections from the glottis to the lips.

The associated frequency response of that area function is shown in Figure 4D and was calculated with a transmission line approach (Sondhi and Schroeter, 1987; Story et al., 2000), which included energy losses due to yielding walls, viscosity, heat conduction, and acoustic radiation at the lips. Side branches such the piriform sinuses were not considered in detail in this study. The first five formant frequencies (resonances), , were determined by finding the peaks in the frequency response functions with a peak-picking algorithm (Titze et al., 1987) and are located at 400, 1065, 1314, 3286, and 4029 Hz, respectively.

To examine changes in pitch, a particular vocal tract configuration was manually ‘designed (Appendix 1—figure 6) such that it included constrictive and expansive regions at locations similar to those measured from the singer (i.e., Figure 4), but to a less extreme degree. We henceforth denote this area function as , and it generates a frequency response with widely spaced formant frequencies (), essentially a neutral vowel. In many of the audio signals recorded from the singer, the fundamental frequency, (i.e., the vibratory frequency of the vocal folds), was typically about 150 Hz. The singer then appeared to enhance one of the harmonics in the approximate range of . Taking the 12th harmonic ( Hz) as an example target frequency (dashed line in the frequency response shown in Figure 5c), the area function was iteratively perturbed by the acoustic-sensitivity algorithm described in Story (2006) until and converged on 1800 Hz and became a single formant peak in the frequency response. Additional details on the perturbation process leading into Figure 5 are detailed in the Appendix.

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Acceptance summary:

Tuvan throat singing, in which people are able to simultaneously produce and independently control two different distinct pitches using the human vocal apparatus, has fascinated hearing and speech researchers for decades. This careful study examines the acoustics of the produced sound and offers new insights into why the produced sound results in two distinct, separately controllable pitches.

Decision letter after peer review:

Thank you for submitting your article "Overtone focusing in biphonic Tuvan throat singing" for consideration by eLife. Your article has been reviewed by two peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Barbara Shinn-Cunningham as the Senior Editor. The reviewers have opted to remain anonymous.

The reviewers have discussed the reviews with one another, and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary

We enjoyed this work addressing mechanisms by which throat singers produce dual pitches that assesses the mechanism for this in terms of the ways in which the vocal tract is precisely controlled based on MRI videos. The work mentions other biological examples of dual fundamentals in songbirds for the broad eLife audience. One of the issues that came up in discussion was the control for normal vocalisations without biphonation, but I think the authors make a reasonable case that the singers act as their own controls. Basically, the work shows that the dual pitch mechanism is associated with changes in the vocal tract morphology based on two constrictions that merge second and third formants and is associated with what they call a 'focussed state' in which the harmonics at 1.5kHz to 2kHz are accentuated. The idea as I understand it is that this accentuates a single harmonic of the fundamental glottal pulse rate so that a new high frequency component of Khoomei emerges that is in effect perceptually 'released' from the harmonic series to allow the emergence of the high pitched whistling part of the song.

Major comments

1) From first principles, dual pitch singing could be achieved by a different type of glottal pulse generation in the larynx so that two vibration modes were present (as in avian syrinx). This is not the mechanism suggested here, and it is hard to see how the anatomy and physiology of the human larynx might allow this, but this has not been directly examined in the MRI work. The authors carried out a careful acoustic analysis which shows only one harmonic series before and after transitions to throat singling (without shifting), which I think is adequate. But they might comment on the other possible mechanism for biological readers, if only to dismiss it.

2) Both reviewers thought the discussion of the basis for the perceived dual pitch was not clear. The authors discuss differences in cochlear mechanisms between low frequency regions and high frequency regions. More effort could be made to explain how the dual pitch, which is attributed to a type of spectral emphasis, can be reconciled with current models of pitch perception. The fundamental for the singers assessed was ~150Hz so that the >1.5 kHz region will be unresolved (H10 and above). The greatest contribution to the salience of the low pitch will be the resolved harmonics at frequencies below the focus region, which are well represented. The high-frequency harmonics will usually contribute (weakly) to the low pitch based on the temporal firing patterns due to merged harmonics in frequency bands. The authors appear to be arguing that a different spectral pitch emerges in the high frequency focussed region, distinct from that associated with the lower harmonics.

3) The argument about decreased phase locking at high frequency was not convincing: this occurs in a much higher frequency region that the focussed region. The argument that the high pitch was not easily explained by a non-linear distortion was convincing.

4) In conclusion, we thought the work nicely shows the changes in vocal tract morphology and associated spectrum as an explanation for the dual pitch, but more teasing out of mechanism for the dual pitch perception is required in a way that might be accessible to readers.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for submitting your article "Overtone focusing in biphonic Tuvan throat singing" for consideration by eLife. Your article has been reviewed by a member of our Board of Reviewing Editors and Barbara Shinn-Cunningham as the Senior Editor.

We are afraid we are still not satisfied with the discussion of the basis for the dual pitch at the end of the Discussion. The authors have demonstrated a region of spectral focus as a proposed mechanism for the new pitch. But we still do not understand why the focussed overtones produce a different pitch. They are still harmonics of the same fundamental and interactions between them in this unresolved region would be expected to produce beating at the same frequency as the fundamental, in the absence of non-linear mechanisms. We also do not understand what the additional relevance of a decrease in phase locking in this region would be that the authors highlight. Are the authors claiming that the focused region produces spectral excitation in a region without the usual coding of beating between harmonics (because of decreased phase locking) and that this is the cause of the new pitch? If so an explicit suggestion along those lines might help readers who are familiar with conventional pitch models.

eLife does not usually encourage multiple rounds of revision but this is a critical point in the interpretation of an interesting study, and I would encourage a revision with a much shorter final section of Discussion that explains a clear hypothesis related to the cause of the new pitch.

Major comments

1) From first principles, dual pitch singing could be achieved by a different type of glottal pulse generation in the larynx so that two vibration modes were present (as in avian syrinx). This is not the mechanism suggested here, and it is hard to see how the anatomy and physiology of the human larynx might allow this, but this has not been directly examined in the MRI work. The authors carried out a careful acoustic analysis which shows only one harmonic series before and after transitions to throat singling (without shifting), which I think is adequate. But they might comment on the other possible mechanism for biological readers, if only to dismiss it.

We attempted to further clarify the point (that we saw no evidence for a nonlinear source mechanism) and added an additional line of text as per the suggestion.

2) Both reviewers thought the discussion of the basis for the perceived dual pitch was not clear. The authors discuss differences in cochlear mechanisms between low frequency regions and high frequency regions. More effort could be made to explain how the dual pitch, which is attributed to a type of spectral emphasis, can be reconciled with current models of pitch perception. The fundamental for the singers assessed was ~150Hz so that the >1.5 kHz region will be unresolved (H10 and above). The greatest contribution to the salience of the low pitch will be the resolved harmonics at frequencies below the focus region, which are well represented. The high-frequency harmonics will usually contribute (weakly) to the low pitch based on the temporal firing patterns due to merged harmonics in frequency bands. The authors appear to be arguing that a different spectral pitch emerges in the high frequency focussed region, distinct from that associated with the lower harmonics.

This criticism was given particularly serious thought and consideration. As a result, we totally rewrote this section to make the proposed ideas clearer, as well as accessible to a broad readership. We tried to find a better balance between issues/questions related to pitch coding and those to cochlear mechanics.

3) The argument about decreased phase locking at high frequency was not convincing: this occurs in a much higher frequency region that the focussed region. The argument that the high pitch was not easily explained by a non-linear distortion was convincing.

As alluded to in the comments above, we clarified the nature of the argument (re phase locking) by expanding upon the discussion of pitch coding. While a reasonable degree of phase locking would still be expected around the 1.5-2 kHz region, this is also where temporal coding starts to fall off dramatically (e.g., Verschooten et al., 2018, PLoS Biol.). That facet, that in the 1-2 kHz region of the human cochlea the fidelity of timing information changes, is what is relevant to the narrative thread here.

4) In conclusion, we thought the work nicely shows the changes in vocal tract morphology and associated spectrum as an explanation for the dual pitch, but more teasing out of mechanism for the dual pitch perception is required in a way that might be accessible to readers.

See comments above.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

[…]

eLife does not usually encourage multiple rounds of revision but this is a critical point in the interpretation of an interesting study, and I would encourage a revision with a much shorter final section of Discussion that explains a clear hypothesis related to the cause of the new pitch.

As we would like to see this work published with eLife, we have drastically truncated the highlighted section to create “a much shorter final section” as suggested. Given our lack of expertise in pitch perception, coupled with our appreciation for the comments raised, we instead (succinctly) reframed through the lens of looking ahead at future work. Specifically, we include only what we think are some quite interesting and provocative parallels we have observed between Sygyt song and cochlear mechanics. We believe that providing this as a summary to the narrative will help stimulate crosstalk between emerging viewpoints in cochlear mechanics and central processing (e.g., pitch perception).

As such, hopefully we have a more streamlined “story” that will be sufficient for

publication. We believe the rest of the work paints a clear picture as to how the

morphology leads to biphonation and that can stand on its own without over speculation on other facets.