🚀 Feature Request: Audio mix and degrader

[VictorBeraldo]

🚀 Feature

Add mixing feature to torchaudio with an desirable SNR

Motivation

This feature wil be usefull for audio data augmentation by adding noise to the audio signal.

Alternatives

There are a lot of python packages that contains this features, but it will be very useful if we have it inside torchaudio

Additional context

[mthrok]

Hi @VictorBeraldo

I love this idea and I was actually thinking about a new augmentation (degradation) module here (although it turned miniaudio leaks memory, so we cannot do it right away).

Do you have an idea of interface or an inspiration from existing packages? It would help to discuss requirements, pros and cons.

[VictorBeraldo]

Hello @mthrok!
I find two of them here: https://github.com/emilio-molina/audio_degrader and https://github.com/sleekEagle/audio_processing.
The first is more complete with a lot of complex features and the second uses what exactly what i'd like to have in torchaudio.

[mthrok]

Hi @VictorBeraldo

Thanks for the links. I skimmed through them and here is my thoughts

• audio_degrader
  • sox
    torchaudio has direct binding to libsox where you can run most of sox effects directly on a Tensor or on a file so it's already covered.
  • ffmpeg
    It seems that MP3 codec is the unique functionality of this. (other things like speed change, filtering are covered). This is what I was thinking with doing with miniaudio.
  • rubberband
    I am not familiar with this library. Will take a look again later (but I have a feeling that libsox can do this too)
• audio_processing
  • Both functions look valuable and I believe it should not be difficult to add to torchaudio. Though we need to settle on the interface (and module name/location).

I guess we can start from functions to add noises to Tensor object.
Some parameters for this includes SNR and maybe the type of noises.

What do you think?

[dataprowl]

Hello, @VictorBeraldo

The links that you've provided are great resources. I went through them and I found that I'm working on something very similar. So, I thought to give it my side of the effort to contribute to it.

Please do check out the PR for this request. It is a simple utility that is designed to degrade the audio Tensor with noise at a specific given SNR. I've also added a few separate functions that adds white and red noise to the audio which is provided as the input.

Happy to help and collaborate to improve it further! Feel free to suggest any changes, enhancements. I'm all ears!

[mthrok]

I have added a tutorial on how one can use torchaudio for augmentation.
https://pytorch.org/tutorials/beginner/audio_preprocessing_tutorial.html#adding-background-noise

I think the next step will be to define interface for noise mixture function.

[mthrok]

After about 2 years, we have a goo collection of augmentation utilities, so I will close this issue. For people looking for specific audio utility to be added to torchaudio, I encourage to open a new issue. Thanks,