Wide-Area 6DOF Rendering of Multi-Point Ambisonic Recordings Based on Interpolation of Spatial Parameters

Abstract

<p>This work presents a method for 6DoF binaural rendering from multi-point first- or higher-order Ambisonic (FOA/HOA) recordings. Previous methods were either based on wavefield decompositions or expansions, with extreme hardware requirements and a limited effective bandwidth, or based on spatial filtering of prominent sources, which can prove difficult in dense multi-source scenes in reverberant environments. The proposed method measures spatial parameters in the time-frequency domain that summarize the scene at the microphone positions. It then estimates the same parameters at the listener position through spatial interpolation of the measured ones. A prototype 3DoF binaural signal, produced from the FOA/HOA recording closest to the listener with high signal fidelity, is then adapted to match binaural cues dictated by the interpolated spatial parameters. The method is designed to support perceptually plausible rendering for scenes of arbitrary complexity even from FOA recordings spaced widely apart in unstructured arrangements. A real-time 6DOF listening preference test demonstrates higher-perceived spatial quality of the method compared to alternatives that mix spatially HOA recordings. This work is related to the upcoming MPEG-I immersive audio standard (ISO/IEC 23090-4) which is defining 6DoF immersive audio technology for AR and VR scenarios.</p>