COLT MCANLIS: In this episode of "Cloud Performance Atlas,"
I helped some IoT visionaries come to terms
with Load Balancer performance.
Will their bet on the future of technology payoff?
Stay tuned to find out.
IoT Roulette is a company which offers
a scalable way for IoT devices to connect with cloud
back ends through a common and simple SDK and transport layer.
A few weeks ago, I met with their CEO
to talk about the biggest thing on her mind.
What's the performance overhead of using GCP's load balancer?
As you can imagine, scale is really
important to IoT Roulette.
With an estimated 50 billion IoT devices
in the world by 2020, scaling your services
to handle the sudden load of a million new devices
is an important feature.
So fast performing load balancers
are crucial to get this right.
But before we start talking about the balancers,
we need to see how things perform directly.
To get a baseline performance, we
set up an instance in US Central and kicked off
a bunch of cURL commands from a machine in Europe.
Now, if you do this test yourself,
you'll notice that the cURL performance can end up
being double what the ping time is simply
due to the overhead of TCP hand-shakes involved
and the transfer of the fetched data.
Now, Google Cloud Platform provides two primary load
balancers, so we need to understand
the difference and profile of each one individually.
The first is a TCP UDP load balancer, which
acts much like you'd expect.
A request comes in from the client to the load balancer
and is forwarded along to the back end directly.
The HTTP load balancer, on the other hand,
has some serious magic going on.
It proxies traffic through Google front end servers
first, which stops the session on the public internet
and continues on using Google's private network.
So depending on which balancer you're using,
we should expect a different set of performance characteristics.
To test this, we set up instance groups for each balancer
in the US Central region and had a load generator in Europe
fetch from them using the same cURL commands as before.
The results show some interesting characteristics.
First, the TCP balancer shows very similar results
to the direct access version.
This is somewhat expected.
In this case, traffic moves from the user
to the load balancers, which are in the region where
the VMs are located.
As such, performance is a factor of the distance
from the requester to the closest entry in the target
region, just like directly accessing the VM.
The HTTP balancer, on the other hand,
did something I really wasn't expecting.
According to this test, it appears
to outperform both the direct connection and the TCP
balancer.
This is odd since the HTTP load balancer effectively
adds another hop to each fetch.
Instead of going from the client to the balancer
to the back end, it has the intermediate step
of hitting the front end boxes first.
Now, at first glance, this seems confusing about why
adding an extra hop would improve performance,
but once we look under the hood, we
can quickly see why this works.
See, when a request hits the HTTP balancer,
the TCP session stops there.
The GFEs then move the request on to the private network,
and all further interactions happen between the GFE
and your specific back end.
Now, here's the important part.
After a GFE connects to a back end to handle the request,
it keeps that connection open.
This means that future requests from this GFE
to this specific back end will not
need the overhead of creating a connection,
and instead can actually just go to sending data ASAP.
The result of this setup means that the first query that
causes the GFE to open a connection to the back end
will see slightly higher response times.
Those are the spikes in the front of the graph.
However, subsequent packets routed to the same back end
will see a lower minimum latency.
So for IoT Roulette, this is fantastic.
The more IoT devices they get in a region,
the better their HTTP load balancers will perform.
Not a bad bet to make.
If you'd like more details on this problem,
check out the full article linked in the description
below, and make sure to subscribe to our Google Cloud
YouTube channel to get alerts when new performance videos
comes out.
Because don't forget, when it comes to performance,
every millisecond counts.
[MUSIC PLAYING]