[upbeat music]
[logo flipping dramatically]
>>THWACKcamp!
[logo flipping dramatically]
>>Thanks for joining the session,
we're gonna be talking about modern web applications,
a little bit on how they're built
and spend most of the time on how they can they can break.
My name is Adam Hert,
I'm a Senior Product Manager here at SolarWinds,
responsible for some of our application performance
monitoring products and joining me today is Mike Morgan,
our Senior Technical Advisor.
Mike, thanks for joining the session.
>>Great to be here, Adam.
>>Mike, to start off with,
we'll set the groundwork.
The old applications,
I say old but they're legacy applications, let's say,
they used to be monolithic.
And they'd have the database, the code,
maybe the web server all on a single host.
And they did all everything in one.
They had their own issues
but now modern applications are quite different.
Can you tell us,
how are modern web applications built today?
>>Yeah, so modern web applications are less reliant
on monolithic LAMP-style stacks,
and utilize more microservices
that can be scaled horizontally to accommodate loads.
So if your app is under heavy load,
we're gonna be spinning up containers,
we're gonna be using serverless technologies
to handle those requests.
>>Makes sense.
And people are probably reaching for things like Kubernetes,
and running things in the cloud now.
So definitely a different architecture
with its own set of problems.
And over the next 20 minutes or so,
we'll walk through those problems.
Let's start in the user's browser,
and see what's needed
to make these modern web applications work.
We're gonna start with DNS request,
straightforward, we do our TLS handshake,
then we make our HTTP request to the web server
on modern applications that are using single page apps,
instead of all the content being returned at once,
it's gonna return some JavaScript to us,
which is executed in the browser.
And then we make even more Ajax-style requests,
and pull even more JavaScript to do
third-party tracking and all that fun stuff.
So, Mike, walk us through with all this going on
just in the browser, what are some areas
that can break and we need to watch out for?
>>Well, absolutely, starting with DNS,
seems simple, but often not the case.
I mean, an example would be the CloudFlare outage
that occurred in July,
just a router failure in Atlanta,
just knocked off large swathes of the internet,
a massive problem.
>>There's this old term that, it's always DNS,
it's probably a bit overzealous,
but it can cause a lot of issues.
And if you're running in the cloud space,
you're probably not managing DNS anymore,
this is managed by,
or your users are using Google DNS,
CloudFlare DNS in this case.
And even if it's not your fault,
I mean, when CloudFlare went down,
and we had this massive outage across the general internet,
there's nothing you can really do about that,
but it's still a way that these things can break and
you need to be monitoring that
and understand how that can affect you.
And so you're gonna jump into another failure point there.
>>Yeah, DNS, I mean, it's not just limited
to large DNS providers.
I mean, it can be down to individual user issues as well,
I mean, things like, VPN issues,
even local DNS that you host yourself.
I mean, it's really a single point of failure
that can affect any kind of operation
that users are trying to request the site
or something like that.
>>For sure.
You don't--we use internal applications ourselves,
and every once in a while, I'll type in an internal address,
and then the web application
will break down or I just can't get into it.
And I forget, I'm on the VPN, and my DNS isn't working.
So pretty kinda, it's right at the beginning of this flow,
we're just scratching the surface here.
But absolutely, DNS is an issue
for both modern applications,
things running behind the firewall, for sure.
And then moving on to TLS, you need to use TLS today,
that your browser is gonna yell at you if you don't.
What are some things we need to watch out for here?
>>Yeah, absolutely.
So things like is your certificate valid?
So it's very, very easy
if it's not someone's responsibility,
specifically to look after certificates,
it could just expire and then that's gonna be a problem.
Your TLS connections aren't gonna work anymore.
>>Yeah, hundred percent preventable,
but I think we've all had some experience
with TLS certs expiring.
It'll definitely catch us. Also need to watch out
for the versions of TLS we support now
so it seems like yesterday that 1.0 is retired
and now we all need to support 1.3.
So, yeah, definitely a preventable thing
that we need to watch out for.
>>For sure.
I definitely think the certificate chains as well,
I mean, chaining certificates is a complex subject,
something may look like it's valid,
but unless it has the valid certificate chain behind it,
that can also be problematic
and can cause certificates to not validate correctly.
>>Right, that always gets over my head a little bit.
But yeah, something else to watch out for there.
The next byte here, the HTTP request from the browser
out to the actual website.
And there's a couple failures here.
I mean, the browser's making the request,
if there's no network,
if there's a firewall in the middle of the network is down,
that's gonna fall apart.
I guess it could be a server error there,
once the request hits the server.
But we'll dig into that one a little bit
when we dig into the real server side
and all the complexities that are there.
The next one here that's really focused on the browser,
is the contents returned from the server,
it's back in the browser.
But on these new modern applications,
the content is just, it's really just a bunch of JavaScript
and single-page applications.
And so what are some of the problems that can happen there,
when the browser's executing this JavaScript?
>>I mean, certainly, there can be errors in the code itself.
So if there's some sort of problem with the JavaScript,
then that's gonna affect how the page loads,
could ultimately cause just the page to not load at all,
which is absolutely the worst case scenario.
>>For sure.
And then, there's the other thing,
I guess, to think about here too, is,
JavaScript is single threaded,
it's a weird thing that we're using everywhere.
But if you have a lot of logic in your JavaScript code
that's being executed in the browser,
it's single threaded but it is asynchronous,
there's just some complexities there
that if your JavaScript, if you've written it away,
that you think,
"Hey, this stuff is gonna be running in the background
on somebody's second core on their computer."
Not really the case, kinda everything you do in there.
It's kind of degrading the end user performance.
>>And then the last point we want to cover here.
So the JavaScript is returned from the server,
it's executed in the browser.
And then what happens there is,
as this JavaScript is executed,
it's actually going to get the content for the page.
And it's getting, more images.
It's retrieving third-party JavaScript code
just to make this even more overwhelming.
So again, kind of same problems, we can have network errors,
again, there could be firewalls, against this.
It's single threaded, and we start loading all that,
third-party tracking software
and all that cool stuff we want on our webpage,
that's gonna bog down the website.
So some things to watch out for there.
And we didn't cover everything in the browser,
there's other things that can happen.
But let's jump over to the server-side application.
And just in the essence of time,
'cause there's so many ways
that modern web applications can break.
We're gonna skip over load balancers, your API gateways,
and the things that stand in between the browser
and where your custom code is running.
So we'll start at really like the web tier.
So the web tier is one of these,
it's almost like your your edge router,
if you're a network person,
but it's the the application
that your user is actually interacting with,
and making their HTTP web requests to.
So, Mike, when we're talking about the web tier,
where custom code is running.
And really the other supporting microservices
that are running behind that,
what are some of the ways that these can break?
And we'll talk about the database afterwards.
But just in these custom applications,
what are the things that we need to watch out for?
>>Yes, I mean, we definitely want to watch out
for things like errors, of course,
exceptions that your code will throw,
I mean, it's gonna be absolutely critical
to keep an eye out for monitor,
in the event that they happen,
we need actionable data as soon as possible
so we can work on those, resolve them.
>>And that's probably the worst case,
like, we're oh again, there's so many ways you can break
we made a web request to our custom code
and immediately, dark code
is written brilliant through an error.
And we're done.
We didn't make it further than that.
So that totally makes sense.
We need some error and exception handling in there,
it's kind of like a catastrophic failure.
What are some of the other things
where maybe it didn't completely break,
It didn't just throw us an ugly error,
but things aren't going well and it's really slow.
What are some other things we need to watch for?
>>Yeah, I mean, definitely the infrastructure
that the application is running in,
ultimately, if your server is running out of resources,
CPU, memory, all that kind of stuff,
then that's gonna hinder the performance
of the application.
>>Yeah, totally makes sense.
I mean, your things need to be provisioned properly
and I guess, just to add on to that,
you need to watch now auto scaling
when you're running in cloud environments,
that if you're hitting these limits on CPU and memory,
you know what, make sure that
that automatically adds a new server or go do it yourself,
just so you can keep up with the demand there.
That totally makes sense.
And then, if we're talking about microservices,
if you're running in Kubernetes,
it's even more complicated.
Now, we have to watch our memory and our CPU
and make sure that things are allocated properly
across our cluster, which is still a new problem
that some people are dealing with.
Anything else we need to watch for,
kind of on that's that can cause poor performance
and not maybe completely break things?
>>Yeah, I mean, definitely,
if the code is just written badly,
I mean, that's obviously gonna be problematic.
So if you have a code that's just inefficient,
taking too long to work through itself,
I mean, ultimately, that's going to just hinder
the performance of the app, slow things down,
in itself can cause resource issues.
So absolutely something we need that visibility around.
>>For sure, I was working with a customer one time,
that they had this application,
it actually was kind of a hybrid, not super modern.
But they had this one endpoint
that took a lot of data and turned it into a CSV
and was just super resource intensive.
And whenever somebody hit this endpoint,
it would just bog down the entire application.
So yeah, keep an eye out
for those functions and those endpoints
that are inefficient, super resource-intensive,
and might be taking away from even other users' experience.
The one other thing I wanted to call out here
on the web tier is just watching the configuration.
So if you've got high volume,
note, watch how many workers and threads
you have configured to run this stuff.
Sometimes little tweaks in there can cause big problems,
and big improvement.
So just keep in mind the traffic
that you're expecting to run.
And then tweak this stuff,
both the amount of replica sets and stuff in Kubernetes,
that you're running, as well as just making sure
that the number of workers
and threads match the workload.
So you're not creating these bottlenecks
right at the web tier.
Now, we're gonna talk about the web tier a little bit,
and the custom code there.
What about the actual microservices?
So now we've got instead of one big monolith,
there's 10 different services,
or maybe there's 1,000 different services.
We've certainly worked with customers like that.
They're all working together
to make this application run performantly.
Anything, we kinda need to watch between our microservices?
Ways they can completely break
or ways that they can actually make for performance?
>>Yeah, definitely.
I mean, as you mentioned earlier on Kubernetes,
huge thing for our customers,
everyone wants to be able to break their app
into microservices and use Kubernetes to manage that.
But it's a very complex product,
it's really difficult to configure
unless you know what you're doing.
And I'm absolutely certain that many, many people
are not doing it completely correctly.
And that in itself
can be very, very problematic performance.
>>Yeah, absolutely.
And, I was talking with a customer a little while ago,
that they had this old,
monolithic application running on-prem,
they were moving to the cloud and they actually told me,
"Hey, we're moving to Kubernetes next week,
which sounds amazing."
And then I spoke to them about a month later, and they said,
"Yeah, we tried, it didn't work out."
So there's a lot of gotchas in there.
And it's funny when we went through all the stuff
that was happening between the browser
and the web tier, network, DNS, all that fun stuff.
That's happening basically,
every time we're making a request between microservices.
So you know what, there's something
like a dozen different networking options for Kubernetes.
I don't know why we have so many that exist.
And then DNS is its own thing in there as well.
So again, there's just every time you add a new service,
and there's more complexity,
there are more failure points that can catch you
not only completely break the application,
but can cause loneliness as well,
if you haven't configured this correctly.
Okay, so we've talked about the browser,
we've talked about the microservices
and the custom code that you've written,
we are the most way through this session.
And we haven't even started on databases,
and third-party services and all the other ways
that applications can break,
we're not gonna get to them all.
This is not gonna be a comprehensive list,
'cause there are just so many ways
that these applications can break.
And we could have spent this entire time just on database.
But database is important.
So let's talk about the ways
that these can be inefficient, they can be slow,
and that they can completely break our application.
So, Mike, I'm sure you've heard of
lots of different examples.
But what are some of the areas
that these things can can cause us issues?
>>Yeah, well, such a comprehensive list.
So inefficient queries would be the first one
that comes to mind.
I mean, your database performance
is only gonna be as good
as the queries that you're writing.
>>For sure.
And inefficient queries are twofold.
You could make a single query that takes a long time,
I actually spoke with a customer one time
that was using one of our tools.
And they found out that one of their endpoints
in their application, every time a user hit this endpoint,
it made over 1 million queries to a database.
So imagine that, I mean, you could,
I think they must have been efficient queries themselves.
But when you have some sort of N+1 query loop,
and you've got a million queries,
that's gonna cause some issues.
Anything else other than the inefficient queries?
>>Well, just bad SQL statements in general,
something that would just generate errors,
again, gonna hurt the application performance,
perhaps even raise exceptions in your app.
>>Yeah, makes sense.
I mean, even if you've got all the infrastructure there,
if you've written something poorly
in an SQL statement, that's throwing an error
that's gonna go bubble up
all the way up to the end user.
Totally makes sense.
Infrastructure things.
Any worry about things you've seen
from customers run into before
that are common gotchas?
>>Oh, absolutely.
In particular, not enough resources
for your database server.
So, particularly for those unmanaged database services,
if you just have something
that's in a very, very busy application hitting a database
that does not have the system resources to accommodate it,
then that's gonna be just horrible for your application.
Totally make sense.
And, one of the ones I've seen before I have this
is actually on an application that I was building.
It was with a MySQL one, and it returned this error
that said, "The MySQL server has gone away;"
was just a ridiculous error message.
Obviously, the MySQL server was still there,
if it sent me that message.
But sometimes, if these things are so bogged down,
they just give up and they just say,
"No, MySQL is not here anymore.
Go figure it out."
So definitely watching your infrastructure resources,
making sure it's... I guess, provisioned properly there.
Also just wanted to call out quickly
on the the PaaS or platform as a service side,
things are a little bit different there as well,
you have to watch your provision throughput.
Once you've hit your limits there,
you'll start to get rate limited.
And the database will send you back specific messages
to say, "Hey, slow down to these rates."
And if you don't observe those messages
and react to them, it'll cut you off completely.
Again, that more modern, "MySQL has gone away"
but hopefully a little bit more friendly.
We should mention cache quickly.
They go hand in hand with database,
but anything we need to watch there?
>>Yeah, I mean, absolutely.
I mean, there's caches themselves,
can really speed things up,
but they have to be watched carefully
to make sure they're performing as they need to.
So certainly things like underprovisioning
the cache cluster.
I mean, that in itself becomes a bottleneck, right?
>>For sure.
And I guess one of the other things too,
is this cold starts when your cache
really isn't there yet, can the database too much
and just not using the cache enough.
I'm sure that guy that was doing a million database requests
should probably be caching some of that stuff,
I'm gonna hope.
And you know what, again, there's so many ways
that your applications can break
or be inefficient and slow.
We are running against the clock here
but there are some other things
that aren't your application or the database.
Mike, what are those other things
that you might not think about at first
with a modern application that we need to watch out for?
>>Yeah, absolutely.
Things like just network services
and DNS that you might be giving responsibility
to a third party is great,
'cause it takes the onus off you,
but then ultimately, the onus becomes on them.
And then if they break, then you're broken.
>>Totally makes sense.
And even if you're making requests
to third-party APIs, network and DNS becomes an issue.
Once again, every time you add another dependency,
all of these issues can pop up again, and again.
Just some other areas too, like a serverless architecture
comes with its own problems,
we're not gonna have time to cover it.
There's background jobs that can conflict on resources.
You might have data pipelines.
I'm sure I'm missing some things here,
anything else you can think of?
>>Just the speed of your storage
that you're using would come to mind.
I mean, ultimately, the files need to be stored somewhere,
and if the storage isn't up to par, then, yeah.
>>You could've moved to S3.
And that's not the speed you're expecting it to be.
'Cause it's not really designed to be a super-fast database.
Yeah, that totally makes sense.
There's some problems you can have in there.
Okay, so we are racing against the clock.
We've been talking for almost 20 minutes.
And we've been keeping it pretty high level.
I think one of the lessons here is that,
there's almost limitless number of ways
that these modern web applications can break.
>>Yeah.
And regardless of what your stake is
in the application, whether you're the application owner,
the support guy, or the operations guy
who's ultimately maintaining this.
I hope we've conveyed the importance
of having appropriate monitoring,
so you have visibility in these issues,
and gives you the data to fix them when they go wrong.
>>Absolutely, yep.
Talk through these issues with your team,
get some observability in there.
And I hope you found this session useful.
Mike, thanks for joining us today.
>>Thanks, Adam.