KASLIN FIELDS: Welcome to Architecting with Google Cloud.
I'm developer advocate, Kaslin Fields.
And today, we're going to explore
the architecture of Apollo with engineering lead, Adam Zionts.
Welcome, Adam.
ADAM ZIONTS: Thank you.
I'm really happy to be here, Kaslin.
KASLIN FIELDS: So I know Apollo provides
an implementation of GraphQL.
Could you first tell us a little bit about what GraphQL is?
ADAM ZIONTS: Definitely.
GraphQL is an API technology that
helps developers provide access to all of their data via graph
interface.
QL actually stands for query language.
So it's a query language to get at the data graph.
And so that differs from the usual kind
of hierarchical pattern that REST provides.
It's been hugely adopted by a lot of major companies,
including Walmart, Expedia, Netflix, Airbnb,
on down the line, and originally came out of Facebook
to help provide an API to the social graph.
KASLIN FIELDS: That makes a lot of sense.
And it sounds like a lot of businesses
are finding benefits in this style of querying.
ADAM ZIONTS: Definitely.
I think that some of the biggest benefits
are just having all of your data in one place helps to unlock
a lot of things.
First of all, it cuts down to the amount of redundancy.
Where, if you have all that data there,
you're not going to end up recreating the same thing over
and over.
And second of all, it opens up that data
to people who might want to have access to it who aren't just
in engineering, people like product management or sales.
You want people to get into that data,
understand how it's being used, who
is looking at it to help drive the right kinds
of conversations and unlock the most value for that business.
KASLIN FIELDS: Cool.
So that's a little bit about what GraphQL is.
So now could you tell us a little bit
about what Apollo does with GraphQL
and what services you offer?
ADAM ZIONTS: Absolutely.
Apollo offers kind of the default implementation
of GraphQL.
So we have an open source client, an open source server,
a number of tools besides those that all
help people get started with GraphQL
and scale it up to production as well.
So starting to write out web applications and APIs, move
data from REST, or even just start out
new projects in GraphQL to prove out the value of it.
Beyond those kind of table stakes
of how to even get started with GraphQL and build,
we've invested a lot over the past couple
of years in the idea of Apollo Federation, which
allows distinct teams to run their own GraphQL services
but then join them together to provide a single, unified graph
of all of their data.
And so that's also completely free
open source, Apollo Federation that helps big businesses scale
GraphQL within multiple teams.
And then we have a SaaS application that helps you--
also has a free version up to paid and enterprise--
that helps you understand how your GraphQL schema is evolving
over time, see who is using your graph and how,
and help you to kind of automate out
that build process and keep those teams in sync
without needing to have a lot of Slack communication or email
communication that generally happens between backend
and frontend in order to get that kind of API work done.
KASLIN FIELDS: So GraphQL is an open source API.
And Apollo offers clients and servers,
which are very popular for use both with open source
and in businesses.
And then you offer all of these additional tools
to help make using GraphQL at scale
in large businesses a lot easier and kind of
make a lot more sense in the enterprise space,
like DevTools, query monitoring, this kind of--
I don't know how to describe it, but like a kind
of an auditing tool, it sounds like you were describing?
ADAM ZIONTS: Yeah, yeah.
So our mission around GraphQL is to make
sure that we provide all of the tools
that businesses need to be able to scale and manage their graph
in production safely.
KASLIN FIELDS: Awesome.
So Apollo provides a lot of tools and services
to help developers at these big companies,
and open source as well, work with data via GraphQL.
And you're serving a lot of users and companies.
How does a user typically start using Apollo's capabilities?
ADAM ZIONTS: Great question.
Typically, users get started, or companies
get started, using GraphQL and using Apollo bit by bit.
We very much believe that the idea of re-architecting
and starting over from step one via some big
like Flag Day and years of development
is not something that anyone is trying to do in 2022.
And we highly recommend that when
people want to get started with GraphQL,
people want to get started with Apollo,
they do that by one feature at a time,
one application at a time, even one page,
one component at a time when you're talking about React.
And so generally, that's how people get
started, as this test case, to say
can we take this maybe complicated feature
and move it over to a graph?
And then once there's success in that,
you can incrementally, iteratively
start adding more and more into that graph.
KASLIN FIELDS: This sounds like a lot of use cases
that I see in the Cloud space, especially.
Whenever a business is adopting any kind of new technology,
it's always best to start with maybe one or two applications
that you can start shifting over to that new technology,
and then shift more and more as you see the benefits,
and as it makes sense, and as your engineers develop
the skills to use those tools, too.
ADAM ZIONTS: Definitely.
And it's very similar to Kubernetes,
even, and very much where that idea of Apollo Federation
comes from--
the idea of being able to move incrementally,
to be able to segment things off separately,
but then bring them together in the end.
All of those things are very prominent in the space
of how to do technology right, given all the things
that we've learned, and very much the direction that we
try to come out as Apollo.
KASLIN FIELDS: Awesome.
So a lot of businesses will start maybe shifting something
over into GraphQL, start looking at Apollo services,
and maybe start moving something over.
How does all of this work on Apollo's side?
And where does Google Cloud fit in?
ADAM ZIONTS: Yeah, so our SaaS application that I talked
about before, that helps track which schemas
are being deployed over which time,
that helps you to craft out excellent queries
and mutations, and share them with your friends
and colleagues, that gives you insights into who
is using your graph how.
All of that data is hosted inside of Google Cloud.
And the services behind the scenes
that help make all of that possible
is coming from Google Cloud as well.
We very much believe that our core competency as a business
should be building excellent GraphQL
tools that help developers build excellent apps,
and not learning how to manage and run
complicated infrastructure.
KASLIN FIELDS: So let's talk a little bit more
about this architecture.
What are some of the pieces that go into it, both Google Cloud
products and other types of technologies
that you use to do what you do?
ADAM ZIONTS: Sure.
So we're running most of this atop GKE, Google Kubernetes
Engine.
And on top of that, we're using a number of different databases
and stories behind the scenes, some
of the things that we're running ourselves,
for instance Memcached, Kafka, and an old data source database
called Druid that helps us manage that time series data.
A lot of things, we also use Google Cloud for hosting.
For instance, Cloud SQL, which are using Postgres,
ThinkTable, BigQuery, Pub/Sub, Spanner
is another really big one that we rely on as well.
We also have a couple of services deployed in AWS
as well, so that we can have multi-Cloud support
for the most critical aspects of our system.
And I can go into more detail on any of those things, again.
KASLIN FIELDS: I think there's a lot of folks out there who
can relate to a lot of those components of architecture--
hybrid Cloud, using Kubernetes is, of course, very popular.
And I'm a Kubernetes specialist.
So of course, I point that out.
But one thing in there that I found really interesting
is that you're running your own Kafka.
I hear Kafka in a lot of architectures these days.
So could you talk a little bit about why
you decided to run Kafka yourselves,
and how that's been going for you?
ADAM ZIONTS: Yeah.
So I laugh because it's like, I wouldn't say a sore subject.
But I certainly have battle scars.
We've been running Kafka over the last few years.
And originally, when we chose to run our own Kafka,
it was because there just wasn't an offering for it
on Google Cloud.
There wasn't an offering with GCP via Confluent Cloud either.
So we started running our own Kafka then.
And we have definitely--
I think we've reached a point now
where things are pretty stable.
We're not-- like, knock on wood--
but we're not having any kinds of crazy outages
or disturbances because we're running on Kafka.
But we had to learn a lot of things.
I remember a lot of sleepless nights,
just banging our heads against the proverbial wall trying
to figure out what the heck was going on.
In particular, there was this one night
where I had to escalate to the CTO
at the time, who was on sabbatical,
because these super blocks inside of the disk
was actually being completely corrupted because we were going
over 16 terabytes, which apparently you can't
do on XT four-file system.
Meanwhile, I'm like--
I've been using Kafka a few months.
I had no idea what was going on.
I felt like I learned much more in that one night about Kafka
and about disks that I had in the previous month.
But that being said, not something I particularly
want to repeat.
So the thing we're actually looking at right now
is how do we migrate to a Cloud provider to make sure that now
that the world is kind of caught up with our usage of Kafka
and there's plenty of options out there, make sure
that as we grow out the size of Apollo,
we're not going to have to have a Kafka team just
to manage that queue.
KASLIN FIELDS: Yeah, that makes a lot of sense to me.
And I think a lot of folks out there
can relate to having some part of our infrastructure
that's just, like, we have to run it ourselves
because reasons.
But it causes a lot of pain.
Maybe eventually, there are better options.
And you start considering moving over
to one of those newer options so that you
can focus more on your core business,
rather than these pieces that you have to run
to make the business work.
ADAM ZIONTS: That's right.
And one thing that we've also started
to do over the last year or so has been just investing
more and more in Pub/Sub.
So we haven't added a lot into our Kafka queue.
That's mostly just quote-unquote "legacy stuff."
And we're starting to think about that migration now
for the things that are in Kafka since they're not
super compatible with Pub/Sub at the moment.
KASLIN FIELDS: Cool.
So one other thing I want to point out
about your architecture is when we
were talking about all of this, I
found out that you are running pretty much your whole business
on a single Google Kubernetes Engine cluster.
And as a Kubernetes specialist, I
found this very interesting and exciting.
I'd love to hear more about how that's going for you,
and some of the tools and kind of tactics
that you use to make a single cluster for multiple teams,
multiple products even, work for you?
ADAM ZIONTS: Yeah.
Another great question, and probably something that
will evolve as we grow.
Like, all of our Kubernetes Engine
and the way that we built our backend
was originally out of one team, as I'm
sure it is for a lot of folks who have seen
huge growth in their companies.
And now we have almost a dozen engineering teams.
We're about to begin doubling that.
So I'm sure that a lot of these things
are going to change as we scale and grow out Apollo.
But ultimately, being able to have
all of this in one GKE cluster has been pretty advantageous.
It means that we can wrap all of our [? kubectl ?] tooling
to all work in one way.
It means that whenever you're inside of the cluster,
you can access logs from multiple different services
or do operations on multiple different services
without needing to log in and log out, and be like,
am I logged into the right one?
A couple of things that we have done to make sure
that we don't have big problems there
is make sure we have separate namespaces to match production
and staging traffic.
And just make sure to keep those separate.
And also, just having different node pools.
Because not every service has made the same.
So we have a bunch of different node pools
where we can use taints, and tolerations, and just
selectors to ensure that those services can be assigned
to the correct node pools.
Or in cases like Kafka, that's just on its own.
You don't have a noisy neighbor problem
where Kafka and some other thing are all on the same pod
and one is being starved of resources.
KASLIN FIELDS: I want to point out
for anyone that's not familiar with GKE,
we have this concept of node pools,
where a single node pool is all virtual machine
instances of the same type, the same kind of hardware
configuration, the same pieces there.
So then you can use Kubernetes taints and tolerations
to manage where your workloads are run,
making sure that they're run on the right hardware,
the right machines for that workload
ADAM ZIONTS: That's right.
And that's a--
I say it's all pretty much good.
Kind of a similar start to Kafka,
like we have learned our--
we've learned our lessons the hard way.
For instance, we, in general, are
OK with having pods from staging and from production
on the same nodes.
That's not a huge deal.
Could be a little scary.
For instance, one time we were using local SSDs
to store a whole bunch of state.
And they were just picked up by the same file path.
And we ended up leaking data between staging and production,
which had this massive effect on wiping out
a swath of our database.
This was years ago.
And we didn't affect any customer traffic because of it.
But it was the most perplexing issue you could imagine.
And so, definitely, there is a huge advantage
in keeping things as separate as possible.
And as I mentioned before, as we grow out more and more teams,
our goal and the way that we think about team management
at Apollo, is that each team should
be able to its own stack of architecture, top to bottom,
as much as possible, and do that independently.
So that might mean having a different namespace or even
entire different cluster per team,
and being able to enjoy those things together
across some kind of [? period. ?]
KASLIN FIELDS: Awesome.
I think as you grow, like you said, a lot of folks
start out with technologies that work
when you're one team that's trying to do a thing.
And then as you start growing more teams,
there are all these types of challenges that you encounter.
And so you have to start thinking about different ways
to do things.
So that makes a lot of sense.
ADAM ZIONTS: Yeah, and for me, that's part of the fun of this.
I remember starting off in engineering,
I had this idea of wanting to do everything perfectly.
And as you get in, months in, you're like, OK,
it's all about trade-offs.
You're not going to do the perfect thing ever.
You're going to do the right thing for the right time.
And six months from now, that's not going
to be the right thing anymore.
Nothing lasts.
And I don't know, I find that problem
really fun and interesting, of balancing
all of those different priorities,
and making good decisions, and then fixing the airplane
as you're in the air, so to speak.
KASLIN FIELDS: Awesome.
So Apollo has a lot of innovation coming up,
it sounds like.
As you grow up more teams, you're
going to consider different ways that you're
going to do your Kubernetes architecture.
We talked about Kafka possibly moving into a managed service.
Anything else you want to point out about ways that Apollo
is kind of innovating, both on their business
and on their infrastructure architecture?
ADAM ZIONTS: Yeah.
Those are all really good examples.
I think that, in general, as you grow and scale
out the number of teams that you have, there is a lot-- like,
a lot of people talk about scale.
And the first thing that usually comes
to mind for me is like, what is the TPS that you're
trying to deal with?
How much more traffic are you getting?
But as I've kind of gone through my career,
I've started to think much more of scale in terms of people.
How many people can you have actively
collaborating on this code base, on this infrastructure?
And a ton of work is going to be done
over the next several-- over the next six months,
over the next year, probably ad infinitum,
to make that more and more achievable,
like we were talking about with trade-offs before.
So that means separating this infrastructure
so that each team can its own architecture.
That means making it easier to deploy things and have
the observability that you want out of the box.
That means starting to use more managed services
instead of hosting things ourselves.
We're also dealing with that other side of scale,
where we have customers like Walmart, and Netflix,
and Airbnb, on down the line, more and more,
who are starting to adopt GraphQL, who
are starting to adopt Apollo.
And being able to lean in a little bit more
to more events-driven systems, making sure
that we have partial degradation rather than complete failures
in places, starting to do more chaos engineering.
There's a lot of fun things ahead of us
infrastructure-wise, to make sure
that we can hit both of those kinds of scale,
both the scale that comes from just the raw growth of traffic
as well as the engineers that we're all collaborating
with now.
KASLIN FIELDS: Awesome.
And speaking of scale and growth,
I know that GraphQL is growing in popularity.
Of course, Apollo's offerings are therefore
getting a boost as well.
So I'm sure a lot of folks out there
are probably starting to think about GraphQL
in their businesses, maybe they're
interested in some of the features and products
that you mentioned.
Could you tell us where we can find more about Apollo?
ADAM ZIONTS: Absolutely.
So the best place to start is just our landing page,
apollographql.com.
We would love for you to check out
Studio, that SaaS application.
That's at studio.apollographql.com.
If you want to just learn GraphQL and get4 started,
we actually have a course that we're offering, Odyssey.
And that's at odyssey.apollographql.com.
And last but certainly not least, we
are doing a ton of hiring.
So if you want to work with us, you can find that Careers page
on our main website, apollographql.com/careers.
Or just connect with me directly on LinkedIn.
KASLIN FIELDS: Awesome.
I love hearing about learning opportunities, too.
So if you want to check out any of those awesome things
that Adam just mentioned, be sure to check out the links
in the description below.
And be sure to like and subscribe.
Thank you so much for being on today, Adam,
and telling us about your architecture
and what Apollo is doing with GraphQL and Google Cloud.
ADAM ZIONTS: Yeah, thank you so much, Kaslin.
It was a real pleasure to be here and get
to share that with you.
And I would be remiss of me to not shout out
the excellent people behind the scenes who have built
up all of this infrastructure.
Because I can't claim very much credit myself,
as I've moved on to the management side of the world.
So a really huge shout out to the team
that has made all of this possible,
that keeps it all running, that is on-call,
and that is doing all this work to make improvements.
And come join us.
KASLIN FIELDS: Awesome.
Thank you so much.
ADAM ZIONTS: Thank you, Kaslin.
[MUSIC PLAYING]