Abstract

In the context of source separation solutions for virtual reality applications, several techniques in the spherical harmonics domain have been proposed in the literature. The performance of such methods is limited under high reverberation conditions and the rendering of the obtained spatial sound is fixed to the recording location only. Recently, novel sound field works in the literature proposed a global representation that enables both the direct sound (exterior field) estimation and the reconstruction in locations different from the acquisition ones. In this paper, we propose a signal processing framework based on Multichannel Non-Negative Matrix Factorization in the spherical harmonics domain that operates directly over the exterior field coefficients enabling the reconstruction of the direct sound field of the separated sources. To evaluate our proposal, we compared with other state-of-the-art source separation approaches using several setups and including different reverberation conditions, showing promising results in terms of BSS_eval metrics.

Listening tests

Here below we report some audio examples accompanying the paper.
For each example we included the mixture, the exterior field estimation (Eq. 15), which is the dereverberated input of the MNMF algorithms, the estimates obtained by the proposed method varying spherical harmonics order $\tilde{N}’_E$, the estimates using ILRMA and FastMNMF and the reference exterior field for each source.
The collected mono signals are obtained averaging the signals of all the sensors.

EXAMPLE 1: 4 HOMs, 2 Female sources Mixture
T60: 0.3[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2	Sph order: 2 Source 1 Source 2	Sph order: 3 Source 1 Source 2	ILRMA Source 1 Source 2	FastMNMF Source 1 Source 2	Exterior Field Reference Source 1 Source 2

EXAMPLE 2: 8 HOMs, 2 Female sources Mixture
T60: 1.2[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2	Sph order: 2 Source 1 Source 2	Sph order: 3 Source 1 Source 2	ILRMA Source 1 Source 2	FastMNMF Source 1 Source 2	Exterior Field Reference Source 1 Source 2

EXAMPLE 3: 16 HOMs, 2 Female sources Mixture
T60: 0.6[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2	Sph order: 2 Source 1 Source 2	Sph order: 3 Source 1 Source 2	ILRMA Source 1 Source 2	FastMNMF Source 1 Source 2	Exterior Field Reference Source 1 Source 2

EXAMPLE 4: 8 HOMs, 3 Female sources Mixture
T60: 0.6[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2 Source 3	Sph order: 2 Source 1 Source 2 Source 3	Sph order: 3 Source 1 Source 2 Source 3	ILRMA Source 1 Source 2 Source 3	FastMNMF Source 1 Source 2 Source 3	Exterior Field Reference Source 1 Source 2 Source 3

EXAMPLE 5: 16 HOMs, 3 Female sources Mixture
T60: 0.3[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2 Source 3	Sph order: 2 Source 1 Source 2 Source 3	Sph order: 3 Source 1 Source 2 Source 3	ILRMA Source 1 Source 2 Source 3	FastMNMF Source 1 Source 2 Source 3	Exterior Field Reference Source 1 Source 2 Source 3

EXAMPLE 6: 4 HOMs, 2 Male sources Mixture
T60: 0.3[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2	Sph order: 2 Source 1 Source 2	Sph order: 3 Source 1 Source 2	ILRMA Source 1 Source 2	FastMNMF Source 1 Source 2	Exterior Field Reference Source 1 Source 2

EXAMPLE 7: 8 HOMs, 2 Male sources Mixture
T60: 0.6[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2	Sph order: 2 Source 1 Source 2	Sph order: 3 Source 1 Source 2	ILRMA Source 1 Source 2	FastMNMF Source 1 Source 2	Exterior Field Reference Source 1 Source 2

EXAMPLE 8: 16 HOMs, 2 Male sources Mixture
T60: 1.2[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2	Sph order: 2 Source 1 Source 2	Sph order: 3 Source 1 Source 2	ILRMA Source 1 Source 2	FastMNMF Source 1 Source 2	Exterior Field Reference Source 1 Source 2

EXAMPLE 9: 4 HOMs, 3 Male sources Mixture
T60: 1.2[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2 Source 3	Sph order: 2 Source 1 Source 2 Source 3	Sph order: 3 Source 1 Source 2 Source 3	ILRMA Source 1 Source 2 Source 3	FastMNMF Source 1 Source 2 Source 3	Exterior Field Reference Source 1 Source 2 Source 3

EXAMPLE 10: 8 HOMs, 3 Male sources Mixture
T60: 0.6[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2 Source 3	Sph order: 2 Source 1 Source 2 Source 3	Sph order: 3 Source 1 Source 2 Source 3	ILRMA Source 1 Source 2 Source 3	FastMNMF Source 1 Source 2 Source 3	Exterior Field Reference Source 1 Source 2 Source 3

EXAMPLE 11: 16 HOMs, 3 Male sources Mixture
T60: 0.3[s]	Estimated Exterior field (input of MNMF algorithms)	Sph order: 1 Source 1 Source 2 Source 3	Sph order: 2 Source 1 Source 2 Source 3	Sph order: 3 Source 1 Source 2 Source 3	ILRMA Source 1 Source 2 Source 3	FastMNMF Source 1 Source 2 Source 3	Exterior Field Reference Source 1 Source 2 Source 3