Multimodal stress detection: Testing for covariation in vocal, hormonal and physiological responses to Trier Social Stress Test.

Examining the effects of acute stress across multiple modalities (behavioral, physiological, and endocrinological) can increase our understanding of the interplay among stress systems, and may improve the efficacy of stress detection. A multimodal approach also allows for verification of the biological stress response, which can vary between individuals due to myriad internal and external factors, thus allowing for reliable interpretation of behavioral markers of stress. Here, controlling for variables known to affect the magnitude of the stress response, we utilized the Trier Social Stress Test (TSST) to elicit an acute stress response in 80 healthy adult men and women. The TSST involves an interview-style oral presentation and critical social evaluation, and is highly effective in inducing psychosocial stress. Participants completed the study in individual 2 h sessions, during which we collected voice, polygraph and salivary hormone measures in baseline, stress, and relaxation phases. Our results show sizeable systematic increases in voice pitch (mean, minimum and variation in fundamental frequency, F0), hormone levels (cortisol) and decreases in skin temperature and hand movement during psychosocial stress, with striking similarities between men and women. However, cortisol and skin temperature only weakly predicted changes in voice pitch during stress, in either women or men, respectively. Thus, while our results provide compelling evidence that psychosocial stress manifests itself behaviorally by increasing voice pitch and its variability alongside simultaneous activation of physiological and endocrinological stress systems, our results also highlight a relatively weak degree of intra-individual 'response coherence' across these stress systems, with dissociations among different stress measures related most strongly to sex.