COLT MCANLIS: In this episode of Cloud Performance Atlas,
we help a semi-aquatic, egg laying, mammal-based calculator
unravel its dependency problem, and then we do nothing.
Stay tuned for more.
I was grabbing lunch last week with a good developer
friend of mine, and the conversation
turned to lamentations around the limits of Google.com
as a calculator.
I mean, frankly, it's one of the more accessible calculators
I've ever used, but I end up hitting the limits of it
pretty quickly.
It won't even compute the factorial of 256.
My friend then revealed that she created
her own version of Google.com calculator called Platy-Calc.
However, her version ran on App Engine,
and evaluated the submitted math expressions
using the Python runtime, which can easily
calculate the factorial of 256.
To me, this was immediately brilliant,
and I started using Platy-Calc as my primary web-based
calculator.
However, I started noticing that the first request of the day
was always a little slower than subsequent requests.
Seeing as I've been talking about cold boot time a lot
lately, I reached out to my friend
and asked to do some profiling on the application.
A first look at Platy-Calc showed pretty much
what I was expecting-- higher than normal cold boot time.
Now, the code for this app was really simple.
I mean, there's no global variables, no locking issues,
and no NDB fetches in the startup path.
The only thing that was occurring globally
during boot time was that it was importing
some complex numeric processing libraries.
Given the fact that we've already timed a simple Hello
World app and seen what startup times look like,
these imports look like a culprit.
Just as a test, we removed those imports
and retimed the startup process, just
to see if App Engine cold boot time had suddenly regressed,
or if there was some other issue going on.
Thankfully, we weren't going crazy,
as our timing showed pretty much the same startup
behavior as I had seen before.
At this point, process of elimination
was clearly porting to the imports
as the source of the startup time issues.
I mean, there was nothing else going on in the app.
However, it felt wrong to simply discard them outright.
We needed more information.
As such, we put together a small code snippet
that timed how long it took for each of these imports to occur.
Our profiling showed us two clear data points.
Number one, the summation of imports
is easily adding about 1.25 seconds to cold boot time
to the application.
And number two, obviously some of the imports
were taking longer than others.
Now, number two is really the interesting point here,
and it's worth taking a second to discuss.
See, during cold boot time, your application
will be busy scanning and importing code dependencies.
The longer it takes to do this, the longer
it will take before your first line of code is executed.
Now, some languages can optimize this process
to be exceptionally fast.
Other languages may be slower at it,
but they provide more flexibility.
And to be fair, most of the time,
a standard application importing a few modules
is less than impactful.
However, when a third party library gets big enough,
we start to see them do weird things
with import schematics that can often mess up your cold boot
time.
A simple Google search revealed that some other developers
saw similar import time issues.
So it was clear that this was the problem
we were running into.
Given the fact that imports were required for our operation,
our hands were a bit tied with respect to an elegant solution.
Nonetheless, we came up with a few ideas to address the issue.
Number one, use a warm up request.
When the website is loading from static assets,
send a quick ping to the calculator service, which
will import the required modules,
rather than waiting for the calculate button to be pressed.
Number two, lazy load our imports.
Python supports this operation by allowing
global versus local importing of modules.
So we could move all of the imports to be function local.
So if you just want to do 2 plus 2,
you don't have to wait to import the entire SciPy module.
Only when we detect some specific keyword,
we'd import those specific things.
And number three, prune the dependency tree.
We've had a lot of Stack Overflow posts and tutorials
about how to manually prune the dependency tree by either
explicitly importing sub portions of it
or by doing some fun hacks that included mirroring the module
and removing the things we didn't need.
But given all that, we decided to do nothing.
Yep, Platy-Calc is not a huge million QPS service.
It's just a small application used
by a few friends who really need to compute
large numbers from a command line interface
for some crazy reason.
So waiting an extra second every few days
for a query that wasn't big of a deal in our case,
we decided that's the right thing to do.
Oh, by the way.
If you're looking for what the factorial of 256 actually is,
there you go.
Thanks, Platy-Calc.
Now, as much as I'd like to beat the performance drum,
truth is that sometimes the result
on the end user for a particular perf improvement
just isn't justified given the work needed to fix it.
So remember, every millisecond counts,
except in the few times where it kind of doesn't.
[MUSIC PLAYING]