MARTIN GORNER: Hello, and thank you for watching "Google
Developers Live." This is a new show called "root access
Berlin." We are recording at the brand new studio at the
Factory in Berlin.
The startup community is absolutely vibrant here.
The Factory is the place to be.
It's a 16,000 square meter campus that mixes young
startups, seasoned startups, and creates a fantastic
innovation, to stimulate innovation and new
technologies.
Please welcome Chad from 6Wunderkinder.
CHAD FOWLER: Thank you.
MARTIN GORNER: Chad, can you introduce yourself?
CHAD FOWLER: Yeah, I'm Chad Fowler.
I'm Chief Technology Officer at 6Wunderkinder, which means,
partially, that I'm a programmer.
And probably mostly, I am the seasoned guy in the young,
fresh startup, to use your words.
MARTIN GORNER: All right.
So the product you do is actually a very successful
cost manager.
Can you tell us more about it?
And I'll show on the screen what it looks like?
CHAD FOWLER: Sure, OK.
So the product is called Wunderlist.
And this is actually Wunderlist 2, which we
launched in December.
And I'm going to talk about the implementation of it a
little bit later.
But it is a productivity tool that we pride on being simple
and beautiful and just staying out of your way.
So it's a place where you can capture your own tasks for
personal or for work.
And you can share them with others and even assign them to
people, and that sort of thing.
MARTIN GORNER: So here, I have actually done that.
I have created this work list.
And I have shared it with you, Chad.
And you have a smartphone, on which you can see it.
So let me add a work item for you.
All right.
CHAD FOWLER: Easy.
MARTIN GORNER: Easy, yes.
Let's see how this works.
CHAD FOWLER: So this is our native Android application.
You'll see that it synchronized and
the task came through.
Something I forgot to mention is we are a distributed
cross-platform product.
So we have a central synchronization service.
But then, we have Windows, iOS, Objective, Mac OS,
Android, and web clients.
MARTIN GORNER: So yes, this looks extremely simple.
I add a task.
It pops up on my other devices.
It pops up in the list of people who share my list.
It's super simple.
CHAD FOWLER: Yeah.
That's what we want you to think.
Or that's what we want you to feel.
MARTIN GORNER: There must be 20 lines of code behind this.
CHAD FOWLER: Slightly more than 20.
It ends up being a lot more complicated than that, to
implement it and make it be seen less.
Just then, you shared one task with me.
I was not currently working on it.
Neither of us was offline.
But it gets a lot harder when you have multiple clients on
multiple platforms, connected and/or
disconnected from the internet.
So effectively, the problem that we have to solve is
similar to creating a multi-master distributed
database system.
You may have multiple friends and coworkers that you're
collaborating with that have mobile phones and desktop
computers and connections via the web, all of them making
updates all at the same time, on the same lists and tasks
that you are.
So they're saving those changes locally.
When they go offline, they could be saving many, many
changes locally.
When they come back online, they bombard
the server with changes.
And as you probably know, in mobile development, every
connection you make is very expensive, in terms of time
and potential battery life.
So the big problem that we had to solve when we implemented
the synchronization layer is, how can we allow clients to
batch up a bunch of requests and effectively make them all
via one HTTP call and get all of the
responses back all at once.
So we initially implemented that in Ruby.
Our back end was written in Ruby.
And we did it all in memory.
MARTIN GORNER: So this is what it looks like.
CHAD FOWLER: Yes, exactly.
So the black box that says, go, the first implementation
was done in Ruby.
It was all in memory.
And that worked when we had a monolithic API that was all
written in one language.
But we're moving to a heterogeneous distributed
service oriented type of architecture, for many
reasons, including performance and scalability.
So to do this, we decided instead of having an in-memory
batch API layer, we would do a batch API service.
So it takes the same format that the original thing does.
But instead of just calling things in memory, it makes
HTTP requests.
And this is a JSON document.
And it represents the request and the response.
So a client would batch up potentially multiple HTTP
operations.
You can see the methods are listed, Get and Post,
and then the URIs.
And then the server does all of the requests, in the
initial version, only serially, and then packages up
the results, again in a JSON document that
the clients can parse.
MARTIN GORNER: And so you chose to recode your Ruby
server in Go.
CHAD FOWLER: Yes.
MARTIN GORNER: Why Go?
That seems like a no go, to add a new language that nobody
knows about.
You have very little competency in the world for
working with it.
CHAD FOWLER: Yeah, so there are a few
different answers to that.
One of them is more philosophical.
And that is, I think it's a mistake for teams to be afraid
of new languages and new technologies.
I understand why teams can be.
Now, when it comes to Go, Go is really easy to learn.
I learned enough to do this service watching one talk at a
conference, from a Google engineer.
So it's a really good at doing concurrency.
It's really good at doing networking.
And in order to actually deploy Go--
a beautiful thing about it is, although it has all these high
level language things that you're used to from languages
that have virtual machines, there are no external
dependencies.
So it's really, really simple to deploy.
Whereas, if we were to go for JVM stuff, then we'd have to
get a JVM stack running our
infrastructure, Erlang same thing.
MARTIN GORNER: So for you, the ease of deployment, lack of
dependencies, was the key.
I understand that, for other people, having a strongly
typed language is also important.
Go is that.
The performance part is, I think, very important for a
lot of people.
Go is compiled into a native assembler.
So it's close to C performance.
But the big difference from C is that it has memory
management.
So it is a garbage-collected language.
CHAD FOWLER: Yes, I think of it as a modern C, what I would
wish C would be today, is what Go is for me.
MARTIN GORNER: And Go has built-in concurrency patterns,
which we'll dive into shortly, because that is what allows
you to program this service very simply.
CHAD FOWLER: Yes, exactly.
MARTIN GORNER: All right.
So now we want root access.
Let's show the code.
CHAD FOWLER: OK.
Let's start by looking again at the JSON document that we
were looking at a minute ago.
And the reason that concurrency is interesting
here is, as I said, this document could have maybe 40,
50 requests in a single batch request.
And another way to say it is, if you have 10 requests that
you're doing, and each one takes one second on the back
end, then it could either take 10 seconds.
Or it could take one second, just to oversimplify.
MARTIN GORNER: And here, it will take one second.
CHAD FOWLER: Correct.
MARTIN GORNER: But before we go there, you
need to do HTTP requests.
There are handy libraries for doing that in Go.
So let's see what they look like.
So this is an HTTP client.
You import the library.
And really, the only thing there is to
it is Get and Post.
It doesn't get any simpler than that.
CHAD FOWLER: And that's included with Go, as part of
the standard library, which is a lovely thing.
MARTIN GORNER: And here's how you do an HTTP server.
It's the same library.
And first, you have to look at the main function.
You declare a handler.
You associate it to a URL.
And then you launch the serving
service, Listen and Serve.
And the handler is, again, pretty much what you expect
from an HTTP handler.
It has a response writer for writing the response.
And it has access to the request.
CHAD FOWLER: Yes.
And this is a multi-threaded server that can
handle a lot of load.
Just this little bit of code here, that's actually what a
production Go server looks like.
MARTIN GORNER: OK.
So let's dive into the real code here and talk about
concurrency.
So here is the first part of your black box.
What does this do?
CHAD FOWLER: So this is actually the code where, given
a request, one of those JSON documents sent from a client,
off screen, we're calling extract batch operations,
which all it does is parse the JSON of this request.
So we're extracting that into a collection of operations,
which is what we've assigned to ops here.
And then, the interesting thing and the part that
demonstrates how concurrency works is the line in red,
where we see rc equals make chan Response.
What we're doing here is we're declaring a variable called
rc, which is an instantiated channel in Go.
And channels are how, in Go, you have multiple concurrent
threads-- or Go routines is what they're called--
able to talk to each other and send messages to each other in
a thread slash concurrent-safe way.
MARTIN GORNER: So the core is the Go routine.
And that is the Go statement that you see here.
What does that do?
CHAD FOWLER: So basically what that does is, we could have
called request without the word go there.
This is a really neat thing about it.
It's like you add go to the beginning or
right before a call.
And it turns something that was otherwise done serially in
the same thread into running into a new
thread or a new thread.
The threads in Go are called Go routines, because they're
not exactly threads.
They happen to run on operating
system threads natively.
But they're smarter than that.
They can multiplex across operating systems threads, in
case one gets locked up, that sort of thing.
So it's a really high-level environment.
And you can see, with just that Go statement, you take
that was serial before, and you turn it
into parallel code.
MARTIN GORNER: And actually, the Go runtime can also decide
to run a Go routine as its thread or to batch them up
inside one thread.
So you can have many more Go routines than you would have
been able to have threads, in any operating system.
CHAD FOWLER: Yeah, it's a really
nice high-level construct.
So what we're doing here is, we're calling the Request
function, which is something that we built, which just
simply goes off and makes the HTTP request that we've asked
for in that batch JSON.
And then we pass in it this channel.
So that's how the new Go routine is going to be able to
communicate back to the calling code and give it the
response from the batch request.
MARTIN GORNER: So let's see how these calls come back.
What does this do?
This looks a little bit unusual.
There is an infinite loop on a Select statement,
on a channel response.
CHAD FOWLER: Yeah, it's very similar to the Select call in
C. Although, what we're doing here is, we're asking Go to
watch for I/O on channels.
So we say, Select, which tells us it to watch and block for
I/O. And then we say, if we get a response from this RC
thing, which is the channel we made earlier, then we can
process it.
And the syntax that we have highlighted here, where it
says less than dash rc--
that is basically saying, we're taking
something out of RC.
It's a kind of a clever visual syntax.
MARTIN GORNER: So on the previous screen, we have
launched our 10 requests, asynchronously.
And here, we are listening on the channel, if one of those
is returning a response.
CHAD FOWLER: Correct.
MARTIN GORNER: One of those or many of those.
How are you going to deploy it?
Is it physically one box?
CHAD FOWLER: A beautiful thing about this service is, it's
this much code.
And it's like technically maybe three times as much code
as we saw, for the entire thing.
But not only is it a small footprint for the developers.
It's a small footprint for the server.
So the way we're deploying it is, we're actually just
putting it on all the servers, so wherever we have app
servers, which really simplifies our deployment,
because it's not eating up so many resources.
We can just say, it's a part of our standard server build.
And it's just always running.
It just goes extremely quick.
MARTIN GORNER: Thank you very much for being with us on
"root access Berlin." And we hope to see you soon, with
other startups, other interesting technical
problems, and other--
I hope-- interesting technical solutions.
Thank you, good-bye.
Thank you, Chad.
CHAD FOWLER: Thanks.