Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
When I opened my inbox today, I was greeted by a huge deluge of messages about WaveNet.
Well, first, it's great to see that so many people are excited about these inventions,
and second, may all your wishes come true as quickly as this one!
So here we go.
This piece of work is about generating audio waveforms for Text To Speech and more.
Text To Speech basically means that we have a voice reading whatever we have written down.
The difference in this work, is, however that it can synthesize these samples in someone's
voice provided that we have training samples of this person speaking.
It also generates waveforms sample by sample, which is particularly perilous because we
typically need to produce these at the rate of 16 or 24 thousand samples per second, and
as we listen to the TV, radio and talk to each other several hours a day, the human
ear and brain is particularly suited to processing this kind of signal.
If the result is off by only the slightest amount, we immediately recognize it.
It is not using a recurrent neural network, which is typically suited to learn sequences
of things, and is widely used for sound synthesis.
It is using a convolutional neural network, which is quite surprising because it is not
meant to process sequences of data that change in time.
However, this variant contains an extension that is able to do that.
They call this extension dilated convolutions and they open up the possibility of making
large skips in the input data so we have a better global view of it.
If we were working in computer vision, it would be like increasing the receptive field
of the eye so we can see the entire landscape, and not only a tree on a photograph.
It is also a bit like the temporal coherence problem we've talked about earlier.
Taking all this into consideration results in more consistent outputs over larger time
scales, so the technique knows what it had done several seconds ago.
Also, training a convolutional neural network is a walk in the park compared to a recurrent
neural network.
Really cool!
And the results beat all existing widely used techniques by a large margin.
One of these is the concatenative technique, which builds sentences from a huge amount
of small speech fragments.
These have seen a ton of improvements during the years, but the outputs are still robotic
and it is noticeable that we're not listening to a human but a computer.
The DeepMind guys also report that: "Notice that non-speech sounds, such as breathing
and mouth movements, are also sometimes generated by WaveNet; this reflects the greater flexibility
of a raw-audio model."
At the same time, I'd like to note that in the next few episodes, it may be that my voice
is a bit different, but don't worry about that.
It may also happen that I am on a vacation but new episodes and voice samples pop up
on the channel, please don't worry about that either.
Everything is working as intended!
They also experimented with music generation, and the results are just stunning.
I don't know what to say.
These difficult problems, these impenetrable walls crumble one after another as DeepMind
takes on them.
Insanity.
Their blog post and the paper are both really well written, make sure to check them out,
they are both linked in the video description box.
I wager that artistic style transfer for sound and instruments is not only coming, but it'll
be here soon.
I imagine that we'll play a guitar and it will sound like a harp, and we'll be able
to sing something in Lady Gaga's voice and intonation.
I've also seen someone pitching the idea of creating audiobooks automatically with such
a technique.
Wow.
I travel a lot and am almost always on the go, so I personally would love to have such
audiobooks!
I have linked the mentioned machine learning reddit thread in the description box, as always,
there's lots of great discussion and ideas there.
It was also reported that the algorithm currently takes 90 minutes to synthesize one second
of sound waveforms.
You know the drill, one followup paper down the line, it will take only a few minutes,
a few more papers down the line, it'll be real time.
Just think about all these advancements.
What a time we're living in!
And I am extremely excited to present them all to you Fellow Scholars in Two Minute Papers.
Make sure to leave your thoughts and ideas in the comments section, we love reading them!
Thanks for watching, and for your generous support, and I'll see you next time!