Sound source separation algorithm of end-to-end time domain multi-scale convolutional neural network

Abstract

The invention discloses a sound source separation algorithm of a multi-scale convolutional neural network with a real-time processing function. The sound source separation algorithm mainly comprises the steps of making a mixed data set; carrying out feature extraction on the mixed audio through a designed multi-scale encoder so that the separation performance is improved; carrying out modeling on the extracted features through multiple groups of improved TCNs, and replacing standard convolution in a convolution block with depth separable convolution (S_conv); introducing a gating mechanism into each TCN block, and processing a feature information stream by using a full convolution gated linear unit (GLU); and finally, restoring a pure audio waveform through a multi-scale decoder. According to the multi-scale-based time domain audio operation method, a TCN network is used for building a sound source separation model, the defect that phase information is ignored by using short-time Fourier transform traditionally is overcome, a lightweight network is built, information flow is effectively controlled, and real-time and accurate separation is achieved.

Description

technical field [0001] The invention belongs to the audio signal processing technology, and relates to a multi-scale sound source separation algorithm with a convolutional neural network with a real-time function and a gating mechanism, with the purpose of separating human voice from musical instrument performance. Background technique [0002] As an art, music has developed rapidly from ancient times to the present, and many outstanding musicians have emerged, producing many popular works of art. Traditional music is handed down in the form of scores and performances. The audio played by multiple instruments makes the mixed instrument audio emerge as the times require. Mixed audio is not just a variety of musical instruments. Music works are more mixed voice audio mixed with singing voice and musical instrument audio. The research on voice and musical instrument audio in the traditional field is only based on their different acoustic principles. [0003] With the developm...

AI Technical Summary

Problems solved by technology

This limits its applicability to low-latency, real-time applications, increasing the minimum latency of the model

Images