[MUSIC PLAYING]
PRIYANKA VERGADIA: Applications that
help us reduce the repetitive tasks make us more productive,
but building them is hard since it
stitches a lot of different applications together.
To explore one such application, I've
invited David Ting, Senior Vice President of Engineering
at Nylas, a company that's revolutionizing the workflow
automation by collecting data from multiple applications,
compiling those insights, and then creating
an end-to-end automation workflow.
Hi, David.
Welcome to the show.
DAVID TING: Hi, Priyanka.
Thank you for having me.
PRIYANKA VERGADIA: I am so excited to have you
on the show.
Tell us a little bit about you and Nylas.
DAVID TING: I am the SVP of Engineering,
and I'm responsible for Nylas's product
and technical execution.
Nylas's mission is to make the world more productive,
and we built up a very scalable and secure platform
to handle communication data driving business process
automation.
PRIYANKA VERGADIA: OK, so how does it matter to me,
you, as end users?
DAVID TING: If there are enough developers that
write all the applications you need
to automate and streamlined workflow,
the world would be much different today.
Why do I have to enter receipts for expenses?
Why is it so hard to juggle my calendar when I just
need two hours of focus time?
Building applications to reduce repetitive tasks
are really hard, and Nylas's mission
is to make it easier for you to build high-RI application
and make you a star in your work.
PRIYANKA VERGADIA: Wow, productivity
sounds really cool, and yes, I do not
want to upload my receipts into a system.
It should just happen automatically.
How does this actually work?
DAVID TING: So we capture real-time data
from service providers such as Gmail, Microsoft, IMAP.
We have data pipelines power ML models
behind the scene to create intelligent data objects.
Data from these insights trigger programmable workflows.
And the challenge is building up a distributed
real-time architecture for the billions of messages
that we receive each day and keep the end-to-end latency
low and predictable.
PRIYANKA VERGADIA: All right, so now
that we have put the customer expectation in perspective,
and now that we know how Nylas gathers the data
and creates these insights, what are the technical drivers
for working with Google Cloud?
DAVID TING: We came to Google with what we viewed
as a very challenging mission.
We have a large enterprise customer
in the financial services space with very strict security
requirements.
PII can not be stored persistently,
and the lifetime of each server needs
to be short to avoid potential in-memory hacks.
This was a great reason for us to rethink the Nylas
architecture.
The legacy Nylas architecture was built with Python flasks
on top of AWS.
It was difficult and very costly to scale,
and the end-to-end performance is unreliable.
PRIYANKA VERGADIA: Interesting.
Why was it so costly to scale?
DAVID TING: Nylas processes over 20 terabytes of data every day,
and the high volume of read and writes
are mainly hitting against hundreds of MySQL shards.
The workload is partitioned as email accounts
across a farm with a static assignment.
Because volume of the emails is unpredictable for each account,
there are many hotspots that emerge throughout the day,
causing high variability in latency and, in severe cases,
reliability issues.
In order to address this, we had to overprovision the system
by a good margin in order to keep that latency in control.
Based on extensive load testing, with the same traffic
load, whenever we double the infrastructure footprint,
we can lower the P90 latency in half.
And at our current scale, doubling our footprint,
it really means a million dollars in cloud cost,
so that's not a sustainable strategy.
PRIYANKA VERGADIA: Wow, that is a lot of money.
All right, so three big major issues
that you set out to solve--
the security, scalability, performance, and, obviously,
cost.
So why did you pick Google Cloud?
DAVID TING: In my mind, Google Cloud
has the best distributed technology
and has been a true engineering partner throughout the journey.
The security, scalability, and stability of the platform
is second to none.
GKE is currently the best option out of any cloud provider
to run Kubernetes workload securely at scale,
and one of the most performant data stores in Spanner.
PRIYANKA VERGADIA: So now I want to get into my favorite part,
which is the architecture.
How did you approach it, what was it like,
and how is it going?
DAVID TING: We were able to transition
to a brand new architecture and achieve
a huge ROI for the project.
The big bets of using GKE to handle the container
orchestration, Pub/Sub for message bus,
and Spanner for relational data store has paid off handsomely.
We went live on June 4, and we really
knocked it out of the park.
Everything has been scaling effectively.
For example, when an application has
performed over 1 billion transactions in less than a
month--
and then, this is on top of Spanner--
what I cannot believe is that the latency is actually
controlled very well, to 7 milliseconds.
And this really speaks volumes to the robustness
of GCP services.
By working closely with the GCP Account Team
and a great partner in Bespin.
We're able to address most of the issues before the launch.
So our customer's QA team went in
and tested our initial prototype.
They found zero issues.
And we spent the rest of the time optimizing the workload
as we scale on GCP.
PRIYANKA VERGADIA: Great!
Well, how did you go from-- you mentioned
that you went from a Python-based application to Go.
You did an entire rewrite.
How did you do that?
And moving from virtual machine instances on EC2
to GKE, how did that process happen?
DAVID TING: As part of the re-imagination exercise,
we had to take some big bets.
And what we did was we actually decided
to rewrite our services in Go, because we
believe that there's a 10x throughput improvement when
we do this.
The second part is moving, basically,
the orchestration here from Cloud-specific and an AWS
autoscaler group.
We basically want to go with the industry
standard in GKE or Kubernetes.
So we used GKE to orchestrate the containers,
and do to the security requirements,
the nodes have to be cycled very quickly.
So we're able to take advantage of the fact
that we can have up to 15,000 nodes in a Kubernetes cluster
on GKE.
And that's actually something that other cloud providers
don't provide.
In addition, we're also using gVisor
to run the containers to create a strong isolation
between the application and the operating system.
This helps us to lock down the host memory and storage access
and enforce least permission principle at the operating
system level.
PRIYANKA VERGADIA: Earlier, you mentioned 20-plus terabytes
of data being processed in hundreds of MySQL shards--
which, obviously, wasn't really efficient
and was also extremely costly.
How did you solve that as you re-imagined this architecture
in Google Cloud?
DAVID TING: We made some big bets.
And so we used Spanner as the relational data store
to keep states for the application
and also store the keys for each one of the accounts
we need to connect to.
So fast read and write access performance
is extremely critical for a large number of calls.
We need the consistent high performance
to have predictable end-to-end latency.
And that actually has paid off by betting on--
running on GCP services versus doing it ourselves.
Lastly, Nylas does not have to hold PII.
So this information is pushed so that once our customers receive
it, we purge the information from our store for security
reasons.
PRIYANKA VERGADIA: For communication
between those microservices, you used Pub/Sub.
Can we talk a little bit about why and how
Pub/Sub has been helpful?
DAVID TING: Yes, so we went from a very database-centric design
to an event-centric design.
So for us, the message bus is the key and the heart
of our whole design.
So we chose Google Pub/Sub for an event-driven microservices
architecture.
And just as a comparison, in the AWS world and the legacy
architecture, because it's actually
more of a data-driven architecture,
the data is written into Dynamo.
Then, it gets converted by a lambda function,
and then it goes to Kinesis events stream.
So in Google Cloud, we want to simplify this.
So we want it event first.
And so we rewrote it directly on Google Pub/Sub.
So it actually puts a lot of strain on the system,
so there's a lot of trust that we have in how this scales
and how it's going to perform reliably and consistently.
And we were very happy with the experience on Google Pub/Sub,
where the performance latency is minimal,
even when we have to burst to tens of thousands
of writes per second.
So you can even think about Event Bus
to basically be like a data store for us,
because it is that reliable and scalable for us.
PRIYANKA VERGADIA: Yeah, that is amazing to hear.
What are some of the benefits that you've
achieved after migrating to Google Cloud?
DAVID TING: This cannot happen without exceptional service,
so we're very happy with everyone we've worked with
at GCP.
We were able to get over 30 times the throughput per node
after the rewrite in Go, and in addition,
the elasticity of the architecture has been amazing.
It's reliable.
Latency doesn't go down.
Performance is actually stable as we actually
have traffic spike.
It's just outright amazing when you
work with the high-scale architecture.
Our customer has been migrating their entire user
base onto Nylas in June, and the process
has been extremely smooth.
And in addition, during Amazon Prime Day,
where just a lot of bursts in e-commerce transaction
come in within the minute, we saw that a number of nodes
went from a handful to tens of thousands
of nodes within a few minutes.
And we're able to handle the peak consistently,
and the P90 performance actually was stable,
even with those spikes.
And it's just really amazing to think back,
and we were able to accomplish out all
of that in just three months.
PRIYANKA VERGADIA: Wow, three months is a pretty impressive
timeline for a migration this big where you had to rewrite
from one language to another-- in this case, Python to Go--
and then also learning all these new tools--
GKE, Spanner, and Pub/Sub.
And I'm sure it was a very massive team effort.
What was your experience like working
with the Google team and the partner in this process?
DAVID TING: I want to first thank the Nylas engineering
team.
I have not worked with a group that
is as talented and as dedicated in their work in my career.
Secondly, I would like to thank our partner, GCP and Bespin
for making this happen.
We're able to work as one team over emails in Slack
and resolve issues in real time.
Most companies would need at least one
year to pull off an architectural overhaul
of this order of magnitude.
PRIYANKA VERGADIA: This is absolutely phenomenal--
the speed, the collaboration.
What is the road ahead?
What does future technical enhancements
look like for your architecture?
DAVID TING: With the success we experienced,
we want to shoot bigger.
So we had the confidence to handle more challenges ahead
and we're looking to work with GCP
on advanced AI/ML use cases where
we're extracting valuable insight contained
within communication channels while automating workflows
for our clients.
Our customers have planned to increase their usage
several orders of magnitude, and they're
pushing with all their marketing power
to reach more users and customers.
And, of course, we have many more mega clients waiting
in our pipeline, and we have the confidence with the partnership
that we have to basically take on more of them concurrently.
PRIYANKA VERGADIA: Well, all that sounds really exciting,
and I wish you and the team good luck
in implementing some of those enhancements.
Thanks, David, for diving into the details
of this incredible journey and architecture with me.
DAVID TING: Well, thank you for the opportunity
to share our experience at Nylas.
PRIYANKA VERGADIA: Well, we just saw
how Nylas migrated to GKE, Pub/Sub, and Spanner
to achieve the performance, scalability, and security goals
while keeping the costs in check.
If you want to learn more, check out the blog I've linked below.
And if you've got any topic ideas for upcoming customer
stories, put them in the comments below,
and I will find that use case and bring it to you.
Bye!