Acoustic–phonetic characterization of affect in spontaneous Finnish

Abstract

Earlier studies of affect in spoken Finnish have largely been based on acted emotional expression, a relatively small number of speakers, and a narrow linguistic scope paired with predefined emotional categories. While previous studies have reported consistent findings related to acoustic–phonetic features associated with both expression and perception of affect in Finnish, no study has investigated how these findings generalize to spontaneous Finnish spoken by a wide range of non-professional speakers. The current study investigates how affect is encoded in acoustic–phonetic features of spontaneous Finnish. By using a recent large-scale affective speech corpus containing spontaneous speech from thousands of individuals with differing backgrounds and with subjective listener ratings of valence and arousal, we first test whether findings from studies with acted speech generalize to spontaneous Finnish. We also report results from an exploratory analysis with a wider range of speech features. As a result, we find that the intonation, voice quality, and rhythmic findings from studies of acted speech largely replicate in spontaneous speech. In addition, we find statistically significant relationships between affective ratings and several new speech features. In general, the mean of log-F0, speaking rate, the mean and variance of jitter, shimmer, harmonics-to-noise ratio, and variance of the difference of F0 harmonics were found to have the highest correlations in relation to arousal. In the case of valence, the mean of log-F0 and the mean and variance of HNR showed the highest correlations. Overall, the results demonstrate how affective information is richly encoded in various aspects of Finnish speech.