[GUITAR STRUMMING]
CHRIS WILSON: Hey, I'm Chris Wilson.
Like me, many of you probably have a guitar or two tucked
away you like to noodle around on.
Or like me, you have a basement full of synthesizer and studio
gear.
But for years now, I've been getting
more and more excited by the powerful music software
available for desktop computers, and lately even mobile devices.
Now, I can replicate vintage synthesizer and processing
hardware and software, which lets me stop filling up
my basement while draining my checking account.
Well I'm here to tell you that all that exciting software can
now come to the web platform with an amazing open web
platform API called the Web Audio API.
The Web Audio API was designed for several purposes.
First of all, it's a great API for building sound for gaming.
When we build games, we need to be
able to make pew pew pew sounds right
when our users hit the button.
You need to hear that massive explosion right
as the screen lights up, or whatever.
Now this was hard to do with the HTML5 audio element, which
wasn't really designed for precise timing,
multiple simultaneous sounds, or memory management.
But the Web Audio API also lets you
crack into the world of digital audio in a big way.
You're in control of the audio path
itself, with powerful but easy to use processing nodes
to create effects like delay, filtering, reverb, and more.
Now the Web Audio API lets you create rich, multiply
layered effects by giving you the primitives you need
to do the job, like delay nodes, gain control, filtering, even
complex primitives like convolution.
The goal isn't to give you all the complex final effects
you could ever want.
It's to give you all the primitives
you need to build more complex effects.
When I first started playing around with the Web Audio API,
I wanted to see just how far I could take the processing
system.
So for a past Google I/O, I wrote a vocoder application.
Now vocoding is a fairly complex process.
It takes one sound source, like me talking to you now,
and it chops it up by frequency bands.
Next, it takes the volumes in those bands over time
and it applies them dynamically to the same frequency
bands in another sound, usually a synthesizer sound.
(SYNTHESIZED) Now you'll be able to sound like a robot.
You'll amaze your friends.
You can even change your voice like a singing robot.
Do re mi fa so la ti do.
You can also use Web Audio to create other audio processing
effects, like chorus effects, or reverb, or pitch shifting.
I'm talking to you, Kent.
Or you can become Alvin, Simon, or Theodore.
Should I give the rest of my talk like this?
No, probably not.
To implement each of these effects though,
I didn't write JavaScript code that
acts directly on the sound buffers.
I wrote code that combines the processing nodes
that Web Audio gives me, and it does all the work.
OK, so you can perform real powerful processing
with Web Audio.
But where does that sound source come from?
Where do you get started?
Well as I just demonstrated, the sound
can come from the user's audio inputs via WebRTC's
getUserMedia API, you can hook the audio inputs
of your computer.
Sometimes, that's just the microphone on your headset.
But you can also have a pro quality
audio interface with a killer axe plugged in.
[GUITAR RIFF]
Web audio can also hook any media elements audio output,
so you can grab the audio output from video or audio elements.
This can make it easy to do just some basic processing,
like gain control or filtering, on an audio element that
may be streaming from the server.
But sound sources can also come from audio samples
that you load through XML HTTP request most likely.
And you can use Web Audio to decode from MP3, wav, or OGG
format to a buffer.
When you play those buffers back in Web Audio,
you can also change the playback rate or detune them.
This makes it easy to take a simple sample, like this one
I recorded of me tapping a wine glass--
[CHIME]
--and turn it into a playable musical voice.
[MUSIC PLAYING]
And finally, if you don't want to use recorded sound,
you can use Web Audio's natively implementing oscillators.
If you're an old synthesizer nut like me,
you'll know the oscillators create a basic waveform,
like a sine, square, saw tooth, or triangle wave.
This makes it easy to produce basic audio blips, bleeps,
and bloops.
Of course, oscillators also make it completely possible
to build much richer synthesizers without ever
loading a sound file, like with this digitally replicated
analog synthesizer I built.
[MUSIC PLAYING]
I tried to dig up a keytar to play this live,
but I couldn't find one.
Now, oscillators aren't just for building synthesizers.
They're also the key to creating a lot of effects
and complex sounds.
Many effects, like the very common chorus effect,
use oscillators in the background.
Chorus is really just a delay with a constantly varying delay
time.
That variation is caused using an oscillator.
This kind of control is possible because of Web Audio's
powerful, flexible audio parameter and routing system.
Most parameters can be controlled and scheduled
at the rate of the audio.
This makes for some amazing effects.
For example, for those of you who remember the '80s,
you can build FM synthesizers similar to Yamaha's DX7
because you can modify one oscillator's frequency
with another oscillator at an audible frequency,
and that creates that classic FM synth bell-like sound.
Working with the Web Audio system's routing and parameter
schedule can be a little daunting at first,
but think of it like a queue of notes
that you need to keep full.
It's sort of like when you sight read sheet music.
You need to look ahead of where you
are to keep your playing fingers primed.
The audio system is busy under the hood
generating and processing the sound stream
and keeping the sound output happy by pushing
a continuous stream of audio bits at it.
So really, the least you can do is give it a little
heads up for what it will need to be playing soon.
I wrote an article a while ago about Web Audio scheduling
for HTML5 Rocks they can help you better understand
this system.
And it can help teach you to create rock solid timing
for musical applications, even with JavaScript's not always
rock solid event delivery timing.
This makes it so you can easily build
drum machine and sequencing applications that
keep a rock solid beat while still responding
in near real time to parameter changes.
One more critical piece to building musical applications
is control.
In the music world, nothing beats
having a real physical instrument to finally control
your sound.
So we've been building support for a standard called
the web MIDI API, that lets music controllers like this,
or this, or even this get plugged straight into the web
platform, so you can use them for direct control of your web
applications.
This lets me have direct physical immediate control
over these great audio capabilities.
Now, the Web Audio API is available on Chrome, Firefox,
and Safari today.
And Internet Explorer recently began development as well.
But finally, I want to leave you with one mind blowing
revelation.
All of these things I've been talking about,
all of the incredible power of the Web Audio API,
all of this works on the mobile web as well with
modern mobile browsers.
iOS Safari and Chrome for Android
both implement the Web Audio API.
In fact, Chrome for Android even has
an experimental implementation of web MIDI,
so you can build amazing web music applications
and take them with you wherever the web goes.
So I hope I've inspired you with some
of the things you can do with the Web Audio and web MIDI
APIs.
Let's get out there and make the web rock.