Mobile Microphone Array Speech Detection and Localization in Diverse Everyday Environments
Pertilä, Pasi; Cakir, Emre; Hakala, Aapo; Fagerlund, Eemi; Virtanen, Tuomas; Politis, Archontis; Eronen, Antti (2021)
Pertilä, Pasi
Cakir, Emre
Hakala, Aapo
Fagerlund, Eemi
Virtanen, Tuomas
Politis, Archontis
Eronen, Antti
IEEE
2021
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202202021791
https://urn.fi/URN:NBN:fi:tuni-202202021791
Kuvaus
Peer reviewed
Tiivistelmä
Joint sound event localization and detection (SELD) is an integral part of developing context awareness into communication interfaces of mobile robots, smartphones, and home assistants. For example, an automatic audio focus for video capture on a mobile phone requires robust detection of relevant acoustic events around the device and their direction. Existing SELD approaches have been evaluated using material produced in controlled indoor environments, or the audio is simulated by mixing isolated sounds to different spatial locations. This paper studies SELD of speech in diverse everyday environments, where the audio corresponds to typical usage scenarios of handheld mobile devices. In order to allow weighting the relative importance of localization vs. detection, we will propose a two-stage hierarchical system, where the first stage is to detect the target events, and the second stage is to localize them. The proposed method utilizes convolutional recurrent neural network (CRNN) and is evaluated on a database of manually annotated microphone array recordings from various acoustic conditions. The array is embedded in a contemporary mobile phone form factor. The obtained results show good speech detection and localization accuracy of the proposed method in contrast to a non-hierarchical flat classification model.
Kokoelmat
- TUNICRIS-julkaisut [18638]