LOUIS GRAY: Welcome back to Google Developers Live at I/O.
I'm Louis Gray, here with my special guest, Chris Wilson
from the Google Chrome team.
Welcome, Chris.
CHRIS WILSON: Thanks, Louis.
LOUIS GRAY: Chris, this is a special treat.
We both get to stand up a little bit.
And we're going to talk about some fun things,
including Web Audio.
We've had you on Google Developers Live a
couple of times now.
CHRIS WILSON: You have, actually, to
talk about Web Audio.
LOUIS GRAY: Talk about Web Audio.
I mean, this is really a specialty and a passion for
you, as you've told me.
CHRIS WILSON: Yes.
Yes, it is definitely a passion in my
personal life, too.
So I kind of carried that over into my work life.
LOUIS GRAY: And you live up in Seattle, right?
CHRIS WILSON: I do currently.
Although I'm in the process of moving down here.
LOUIS GRAY: Seattle's kind of appropriate, where you let
your hair down--
CHRIS WILSON: Exactly.
LOUIS GRAY: --get the guitar out.
So it all just kind of works very smoothly.
CHRIS WILSON: Get the whole grunge look going, and we'll
see how it works down here.
LOUIS GRAY: Absolutely.
So on the Chrome team, we really talk about three things
most of the time.
And it's speed.
It's simplicity.
And it's security.
CHRIS WILSON: Yes.
LOUIS GRAY: But with the Web Audio, you're also trying to
make the web do more.
CHRIS WILSON: Absolutely.
LOUIS GRAY: Tell me a little bit about what's
going on with Web Audio.
CHRIS WILSON: So one of the goals with Web Audio was first
to make it so that we could do really good audio for gaming,
for any kind of interactive application where you wanted
to be able to precisely time things.
You hit the Fire button, you don't want to hear it sometime
in the future.
You want to hear it right when you hit the Fire button.
You want to be able to do all kinds of
processing on that too.
And you want to be able to synchronize things together,
too, which has been really hard to do in the web
platform, prior to Web Audio.
And now with Web Audio, you can actually do that.
We also have a really powerful processing system.
So you can do reverb and another convolution effects.
You can do filtering effects.
You can place things in a 3D sound field, and all kinds of
other neat stuff that is really interesting for gaming,
of course, like all those things.
Immersive games really need those kinds of features.
But you also really want the ability to do more than that
for kind of regular digital audio workstation synthesis
kind of stuff as well.
LOUIS GRAY: Like professional musicians and
that kind of stuff.
When you look at the verticals that are here, not only games,
but what other type of people are really going to be looking
at Web Audio as a solution for them?
CHRIS WILSON: Exactly.
Well, I think that gaming, first and foremost, was what
we had to hit just to begin, just to get off the ground.
And now we're starting to carry over into some of these
more professional audio applications.
And we're starting to see a lot of people come out of the
woodwork and start building applications that do things
like drum machines, and synthesizers, and other
processing type stuff as well.
And that's really I think what I'm here to talk about today,
is kind of where we've come since last year at I/O. Last
year at I/O, I gave a talk about Web Audio--
still up on YouTube of course.
Got tons of great information.
I think almost all of that is still completely relevant and
really a great--
LOUIS GRAY: But since then, we've introduced new APIs.
CHRIS WILSON: We have, actually.
LOUIS GRAY: We've made phones stronger, faster.
And I think you actually bought a couple demos for us.
CHRIS WILSON: I did.
LOUIS GRAY: Maybe you can introduce us to what this demo
is all about.
CHRIS WILSON: Sure.
LOUIS GRAY: And let's get the instruments out.
You didn't bring a drum machine, did you?
CHRIS WILSON: I didn't bring a drum machine, although I do
have a drum machine app on my demos page.
LOUIS GRAY: No doubt.
CHRIS WILSON: So you can check it out.
But one of the things I wanted to start with was when I did
the Web Audio talk last year, I was
exploring the API myself.
And what I did to explore it was I built a vocoder.
And for anybody who remembers kind of early '80s synth rock,
a vocoder is like the robotic voice effect.
So you got this, really it's a processing effect.
I'm not into tons of details.
LOUIS GRAY: Sort of Kraftwerk to information society.
CHRIS WILSON: Exactly.
LOUIS GRAY: I like that stuff way too much, to be honest.
CHRIS WILSON: The domo arigato thing, that was all vocoding.
And so vocoding, takes a frequency analysis of one
audio sound.
And it maps that frequency analysis over time under a
different audience sound.
And I did this demo as part of my talk.
And I didn't go into a ton of depth, because there was a lot
of code that I built for it.
But one of the exciting things that I was really hoping we
would get in before I/O last year, but it missed.
We were in summer I think, midsummer,
before we got it in--
was audio input.
So not only can you generate these sounds on the computer--
play back pre-recorded audio snippets or build things with
synthesis--
but you can actually get audio input from the computer.
LOUIS GRAY: From live audio, as well.
CHRIS WILSON: Exactly.
So you can get the microphone on your computer or any other
audio source that's plugged into it.
LOUIS GRAY: And I understand you brought an
audio source today.
CHRIS WILSON: You can use that and process it.
I did.
LOUIS GRAY: Let's see it.
CHRIS WILSON: I brought a couple of them, actually.
So we're going to start, I want to show-- the vocoder was
the first app that I was like, as soon as I got a build from
the developer, I was like, oh I got to go add this.
And like my wife was there making dinner.
And I'm like yeah, don't bother me right now.
I'm busy.
So I got this up and running in about 20, 25 minutes.
And basically I took this audio vocoder that I built,
which could work off of a preprocessed sound source.
So let's play and you can hear of what it sounds like.
LOUIS GRAY: You're playing that off the computer now.
CHRIS WILSON: And we're playing it off the
computer right now.
So this is all being--
I prerecorded myself talking and then applied the vocoding
effect to it.
So now, I can take this and just click my little live
input button.
And I have a live audio source.
LOUIS GRAY: Are you going to sing for us?
CHRIS WILSON: Check the microphone, right here.
I'm not going to sing for you.
I actually told one of my coworkers, singing--
not really something I'm prepared to do yet.
LOUIS GRAY: Right.
No singing today.
Sorry guys.
CHRIS WILSON: Sorry, not this time.
So one of the things that we do here of course, is we have
to ask, is it OK if we use your audio input source?
Because we don't want to let any random web page pop up and
say, oh sure, you can listen in to all of your users.
LOUIS GRAY: Oh, absolutely.
Permission first.
Security.
CHRIS WILSON: So permission first.
Security is always the theme here.
But now you can tell, I actually have this vocoder
effect on my voice.
Everything that I say is running through this vocoding
processing effect.
And by the way, all of the code that I'm doing here--
and maybe we should stop doing vocoding effect, because it's
probably a little bit distracting.
We could use this for the rest of the talk.
LOUIS GRAY: Sure.
CHRIS WILSON: But all of this processing is actually being
done by the Web Audio API.
My own code actually just sets up this
whole network of nodes.
And then the audio processing API does everything.
The only bit of my JavaScript code that's running right now
is the stuff that animates that neat little visual effect
on the screen.
So the next thing though, like the next thing that I built
after I added this into the vocoder, was just started
playing with some input effects.
LOUIS GRAY: And which input is this?
CHRIS WILSON: So this is still pulling off the microphone.
And it's just a few simple effects, like delay.
But I'm going to turn this off for a second.
And one of the things that is really important to understand
is that input isn't just microphones.
It's not just the mic that's plugged into your computer.
So what I've got hooked up here is just a standard USB
audio input.
But I have the ability to plug-in
different audio sources.
And I brought along my trusty guitar.
LOUIS GRAY: That's a nice guitar.
Are you going to show everybody?
CHRIS WILSON: It is a nice guitar.
This is my baby.
This is one of my babies.
This is my electric baby.
But so what I've got running off here--
and I'm going to pick a different sound here, just so
we can see it.
What I've got running here is just a regular, quarter-inch
output out of my guitar, run straight into the USB audio.
In fact, I can change the effect mix here.
And you can hear what it sounds like.
It's kind of twangy.
It's standard kind of thing.
But if I crank the effects mix up, what you'll hear is me
playing running through effects that are all being
done in Web Audio.
I just set up a bunch of nodes to do various things in here.
So this is my bass effects.
I didn't want to bring my bass along because that was way too
much to take on the plane.
LOUIS GRAY: And while you're doing that, we can all see the
visual running through the computer at the same time.
CHRIS WILSON: Yeah.
So I've already talked about one of the first things we had
to do was this precise timing of audio.
We had to give you the ability to precisely schedule things,
set them up, synchronize them, do things together.
And I've got a bunch of other really great stuff too that I
don't want to get into today, because it takes a while.
But there's lot of powerful stuff you can do with that
scheduling.
And then we added all this processing stuff.
Like the things you're seeing here are all filters and
convolution effects, distortion using wave shaping,
and things like that.
But the last thing that the Web Audio API really has is a
great ability to do analysis, like we have built in
FFT-based frequency analysis.
LOUIS GRAY: What kind of analysis?
CHRIS WILSON: So you can actually see as I'm playing
here, this is a frequency break down.
In fact, I can switch this to just a straightforward
frequency graph.
And you can see the rich harmonics because this wah
bass effect is actually trimming off all the higher
frequencies.
It clips all the stuff that's on there to the right, because
of course, lower frequencies are on the left here.
LOUIS GRAY: Right.
CHRIS WILSON: But if I switch to something else, like just a
distorted effect, you can see the distorted effect has lots
of high-end energy.
And this is all being done, all of the analysis of this is
being done, by the Web Audio API.
So I have code of course, that goes in and pulls the numbers
out and draws them on the screen--
draws them on an HTML5 canvas.
But I'm not actually having to do the processing of the bits
in JavaScript.
LOUIS GRAY: A lot of this, this is programs that you've
written based on the Web Audio API.
Now, a couple of questions.
Number one, is the Web Audio API always available in every
Chrome install, or is it something else
you have to go get?
CHRIS WILSON: It's a great question.
The Web Audio API is actually available in
Chrome across the board.
It's not something you need to add.
As I showed before, you do have to, your users have to
allow microphone input, live audio input.
They have to say it's OK to be able to get access to that.
LOUIS GRAY: And then the second thing I was going to
ask is, how do developers get started with this?
If there's somebody who really wants to either look through
the Web Audio API to add it to their games, or find some
other way to manipulate sound, what do they do?
CHRIS WILSON: I think there are a lot of
demos out there now.
There's a lot of code out there now.
All of these things by the way, are on
webaudiodemos.appspot.com.
You actually just search for Web Audio demos.
This comes up.
LOUIS GRAY: It can be number one.
CHRIS WILSON: This is number one, I think.
I'm pretty sure this is still number one.
LOUIS GRAY: In all major engines.
CHRIS WILSON: Yes.
And this has a whole bunch of my demos.
You can see most of these have a link to
the Repro and GitHub.
These are all open source stuff.
You can take it in and do whatever you want with it.
But I forgot where I was going now.
LOUIS GRAY: You were talking about how
we're going to develop.
You said it's on GitHub.
You can find it at web.appspot.
Yeah.
CHRIS WILSON: The one thing I did want to say is, one of the
other things and I'm really excited about this year is
we're just on the cusp of getting all this stuff into
Chrome for Android.
LOUIS GRAY: Right.
I was going to ask about mobile.
Yes.
CHRIS WILSON: We're starting to get all of this processing
power on the web platform on mobile devices as well.
LOUIS GRAY: Interesting.
CHRIS WILSON: This is a much more challenging thing to do,
because there's a broad spectrum of devices and audio
hardware and those devices.
LOUIS GRAY: But there's been some of that simplicity.
CHRIS WILSON: But it's super powerful.
LOUIS GRAY: You had Android studio, that just came out
which is going really help people be able to code Android
applications against many different models and types and
see how that works.
And as you look in Chrome, Chrome came off Android not
too long ago.
It's out on IOS as well now.
And it's very interesting to see that once you start
writing for a specific platform, and that platform
becomes available to different people on different operating
systems and different handsets, it's just really
ubiquitous.
And it's available everywhere.
CHRIS WILSON: Exactly.
That's really the thing had me really excited about Web Audio
from the beginning, was the idea that eventually, we would
get this everywhere.
We would get it as a ubiquitous platform.
I don't need to have a particular device.
I actually, as I've been preparing to move, I've been
packing up my studio.
And I've had to think through each piece like, am I really
going to need this anymore?
Because by the time I unpack, I could rebuild a bunch of
this stuff just as Web Audio APIs.
And I don't really need to drag the
hardware around anymore.
LOUIS GRAY: Right.
CHRIS WILSON: Like what I brought here with me today,
what I brought with me down to I/O, was basically just a USB
audio input, one microphone, one guitar, and no effects
boxes or anything like that.
LOUIS GRAY: Awesome.
CHRIS WILSON: And I can still there are tons of neat stuff.
LOUIS GRAY: Thank you for bringing all that stuff over.
Thanks for joining us with your guitar--
CHRIS WILSON: Thank you for having me.
LOUIS GRAY: --and all of your equipment.
It seems like we ran really fast through this.
So I definitely want people to check out
webaudio.appspot.com.
CHRIS WILSON: Webaudiodemos.
LOUIS GRAY: Webaudiodemos, thank you--
webaudiodemos.appspot.com.
Thanks for joining me, Chris.
We have Chris on a regularly on GDL and
all the Chrome team.
So definitely follow us at developers.google.com/live,
and subscribe to Google Developers on YouTube.
We're going to be right back in a couple of minutes to talk
about YouTube, and the Sandbox, and wrap our I/O
programming.