Generating Sound with Neural Networks

Generating Sound with Neural Networks

Learn how to generate sound from audio files 🎧 🎧 using Variational Autoencoders.

Sound Generation with Deep Learning - Approaches and Challenges

0:00 Intro
0:33 Defining the sound generation task
1:17 Classification of sound generation systems
2:14 Types of generated sounds
3:41 Sound representations
4:07 Generation from raw audio
7:40 Challenges of raw audio generation
10:21 Generation from spectrograms
16:12 Advantages of generation from spectrograms
18:07 Challenges of generation from spectrograms
20:26 Can we generate sound with MFCCs?
21:26 DL architectures for sound generation
22:13 Inputs for generation
24:03 Details about the sound generative system we'll build
24:44 What's next?

Mentioned papers:

• Wavenet: A Generative Model for Raw Audio:

https://arxiv.org/pdf/1609.03499.pdf

• Jukebox: A Generative Model for Music

https://arxiv.org/pdf/2005.00341

• DrumGAN: Synthesis of Drum Sounds with Timbral Feature Conditioning Using Generative Adversarial Networks

https://arxiv.org/pdf/2008.12073

• Melnet: A generative model for audio in the frequency domain

https://arxiv.org/pdf/1906.01083.pdf