Hey, friends! Did you know that you can now run high-scale load tests in the Azure portal?
While the test is running, you'll get live updates of client and server-side metrics.
John Stallo is here to introduce me to the new Azure Load Testing Service,
today on Azure Friday.
Hey, friends, I'm Scott.
And it's Azure Friday.
Today I want to learn all about Azure Load Testing.
Talking with John Stallo. How are you, sir?
Hi, Scott. Thank you for having me on.
Thanks for dialing in from Perth.
Lovely Western Australia.
We appreciate you either staying up or getting up early.
I have no idea what time it is there.
It's early in the morning and it's going to be a hot day.
Brilliant, brilliant.
So, you know, the great thing about load testing and using something like this in Azure
and the cloud is that
not only do you have infinite scale, but you can have that load come from anywhere in the world.
So, I suppose that you could test my website from there or I could test yours from there.
We have a lot of choice, don't we?
Indeed, indeed.
And agility testing is a new service that is really designed to help developers
and testers gain confidence in their applications
performance and the ability to meet demand.
That is a really great point.
You know, very, very often
we don't trust our machines and the whole point of testing, whether it be unit
testing or integration testing or load testing, is to have a feeling of like, yeah, you know,
I trust that this is going to do well on Black Friday
or whenever I go and launch these things.
That's right.
You know, that that confidence that I was talking about can come in many forms.
One is that important event like Black Friday tech season where your application needs
to be ready right from the get-go, but can also be like earlier in the development lifecycle.
It might be that you're working on
redesigning your architecture.
You might be taking a new dependency
using an agile service for the first time.
And that can also be while you're in the thick of it.
And you want to really want to make sure that your
that you're able to
catch and block performance regressions as soon as they happen.
And so we see a lot of teams interested in how can they automate,
load tests in their deployment pipeline.
You know, that is a really great point.
You know, back in the day, 15 plus years ago, even 20 years ago,
for us, the load testing was an event that hadn't happened
at the end of the waterfall and we would all get here.
We didn't do it. We're doing the load test this week. It's going to be a big deal.
But you're describing that this can be a part of that shift left experience.
And there's no reason I can't be load testing, you know,
every ICD pipeline and making sure that that's part of that pipeline.
And then with the right metrics,
make educated decisions about whether or not something regressed or something got better.
That's right.
You know, so
teams are definitely deploying a lot more often nowadays.
And so you really want to have that confidence during your development lifecycle.
And it really starts with being able to simulate your load in a real traffic
in the real world way and then be able to learn how your application behaves under stress.
Okay. Do you have a demo?
You can show me how easy this is.
I'd love to show you how easy it is to get started and then we can look into a few more features.
All right. Let's do it.
Okay.
So, you know, Azure load testing is available right in the portal.
I think I mentioned that it's in public preview right now.
We just launched last December, so we're pretty excited about it.
But to get started, I can go and create a resource
and then I just search for agility testing
and then I'm ready to get going and I'm on my way.
When you create Azure Load Testing resource, what it basically is,
is a way for you to be able to manage your tests.
And you can also manage access to who can actually run the load test.
You can actually access the results.
I'm going to go ahead and go ahead and create
my own test here.
Let's give it a name
and then
in my test plan, I'm going to upload a JMeter script.
Now, JMeter is a very popular
open source load testing framework.
It has a lot of extensibility, built in a lot of enterprise friendly features,
and it has a great community.
So it's really easy to be able to go out and get help,
get resources about how to configure a good load test
and then with as load testing, that's basically it.
I've uploaded my JMeter script and I'm ready to go.
There's a lot of other things you could do about it, not do with it,
but maybe just one of the things to point out is
if you wanted to scale out and be able to
simulate high traffic, it's as easy as just being able to use a slider
to be able to increase the number of parallel
engine instances
I'd just like go ahead and.
If I may.
It's a really great point that it's JMeter here because JMeter, there's a very mature,
open source project Apache JMeter,
who's been around since the soap web services days and when we say test
something, we're not talking about just sending a bunch of HP gets at an endpoint.
You can test Web services, you can test FTP, you can test specific type things.
You can do Native Command, like JMeter is really powerful and you're doing
JMeter in the cloud,
which is going to enable people who have existing JMeter their tests to be really successful.
But if someone is just getting into JMeter here, they're going to be really impressed
with the amount of flexibility that plus the infinite scale of the cloud is really powerful.
Indeed.
So what I've done here is I've actually kicked off that low test.
It's going to actually it's already started to execute and I'll start seeing
some results already show up in real time in the chart in the second.
And the other important thing is, like, any time that I want,
I can actually just stop my load tests just in case things are going wrong.
And I really want to be able to just put everything on pause.
And so you can see that as that low test is actually running.
I can see that and slowly I am ramping up the load
that is being sent to my application
and I can see other important metrics
that JMeter provides, like response time and requests per second.
So far, it seems like it's kind of holding steady and I'm not seeing any errors yet.
Hmm. Looks like a bit of a cool thing.
I can hold start there.
You started out about 2 seconds and it kind of like warmed up very quickly.
Maybe it's catching some stuff you can really understand.
The fact that is happening in a real time is really cool.
Like, I'm watching it update every few seconds with real time information.
And right now on that response time, those averages are high only
because the first one was at 2.5 seconds, but it looks like you're under a fifth
of a fifth of a second there, a little hiccup there.
Something just happened.
Oh, wow. Wow. Something happened here.
And I'm actually now starting to see some errors come in.
Oh, at that and I've actually hit my peak load here.
And so as that's being sustained, I can see my application, you know,
is it going to be able to handle that or are they going to get back on track? Oh.
And it's still.
Yeah, so something's not going quite right.
So click even here.
I can already learn about how my application behaves under certain load.
We thought it was behaving okay earlier on.
Yeah.
But at this time now things are definitely not going as we would want it.
Oh, wow. Okay.
Yeah, looking great.
You were doing a great job.
The left side requests per second was heading up towards 200, and then just something,
something hiccup, and now it's stumbled and it doesn't seem to be able to right itself.
And we've learned that just in a few seconds here by my firing off this test in the cloud.
That's right.
That's going to keep on going.
So I'm going to just switch over to a low test
that I had previously run on the same application.
And you can kind of see the same pattern here that we hadn't seen before.
Everything seemed to be going right with response times.
I wasn't getting any errors.
But then at some point as I was hitting peak load in my low test,
then I started to see those really long response times and errors.
And so this is what we call client side metrics.
This is everything that JMeter is telling me about
how my application
is, you know, is running from a black box perspective.
You know, from a as users go into my application,
they're bound to see some of these response times and bad errors.
But one of the immediate questions then that one would have when you run a low test
like this is within how did my application components
actually behave during the load test?
And this is where we can really take advantage that agility testing is an Azure service.
So it really is optimized for applications that are hosted in Azure.
And so now I can actually go ahead
and add what we call server side metrics.
My application here is made up of an API that is actually deployed at service.
It talks to a backend database.
And so I can go ahead and click on configure app components.
And then here I can just go off and filter by,
you know, different resource groups and subscriptions and so forth.
I know that my target application, the application
that I was testing is actually deployed to this resource group.
And I can see here I've got an app service that's service plan.
It's talking to Cosmos DB
I actually using application insights for telemetry.
So it's easier just selecting these resources
and then add those to my dashboard
Interesting.
So you've got what you're calling client side metrics.
That's kind of the pane of glass.
That's from the point of view of there's an endpoint out there and I'm going to poke
at that endpoint and this is what I'm seeing on this side.
And then now that you've got your introspective, you're able to introspect
into those server side components.
That's right.
You're going to be able to see all of the, the other side
once you've come inside the cloud, what's happening.
Exactly.
And so you can see that I'm getting, you know,
some some insight into my app service
where my
API is hosted or what my CPU and memory looked like.
And that seem to be okay.
But as I scroll down, I can also start to see some other.
There it is.
Like there are other metrics
that, yeah, you probably already saw it here, Cosmos DB.
I'm already maxing out my IQ consumption.
So there's something here that's telling me about
maybe the way that my as a resource is is configured
with a way that I'm using it
is maybe surprising
and that's why I'm actually seeing these results.
This is so this is so significant for people who may have been,
you know, in the industry for a very long time.
If you do a front end load test against a website and the website
has a data structure, a data source, SQL Server downstream,
you know, you're going to see that present itself as 500 errors.
So on the glass, HTTP 500 is going to go look that.
And to figure out that it wasn't the web server or the load balancer or the reverse proxy,
its downstream could take you days we would go digging through log files for days
to try to figure out the kind of insights that you've already figured out
in 5 minutes of using Azure Load Testing with that server side
component.
That's right.
And I can even actually click into these into these metrics.
And so we're pulling in metrics from all over Azure Monitor Container Insights.
If I was using that Kubernetes service
Application Insights in this case, I've actually drilled into the requests
that are reaching my back end database, and then I can just poke around you
here.
I can see that out of all the requests being sent to my database,
I can actually see that my database is now throttling.
And that's why now that's heavy that, that, that, that
and end to end errors that I'm seeing barely in my front end
right API because of what's happening in my backend database.
So in this case, we've learned that 100 users at the Web site
perspective at the JMeter user test perspective is going to equal or
more RUs more resource units from the Cosmos perspective than we are prepared to offer.
Forgive my ignorance, but would you be able to then scale Cosmos
and put in more RUs and this would be a successful test?
Absolutely.
And so then Cosmos actually enables a lot of really good ways to scale
including being able to also configure it for order scale.
In fact, one of the things that maybe I could show you is
after having run this low test for some time,
I had some hypotheses about maybe what I could do
to actually improve my cosmos scale.
And so just now being able to compare results is as easy as selecting
these test runs and compare them.
And then that would actually then give me confidence of whether any of these
changes that I made to my application or my configuration
actually improved my
my application's performance In this case, you can see that
if I can just maybe filter down to one metric at a time,
you can see why after making those changes, my
my response time actually was
was that was healthy after
making the improvements to my backend database, I no longer saw any errors
to compare that very first load test that we saw that we were getting lots of errors there.
So really low testing is is really there to be able to help you run these real world
simulations and then gain that confidence about how to
how to improve your application's performance.
That is brilliant.
So there are folks that are watching right now who may have JMeter tests.
They can bring those up into Azure Load Testing there may be people who are just
getting started with load testing, but they have existing resources in Azure.
This is going to be a very nice complement.
I'm thinking already about how I could possibly put this in my own
psyche pipeline and run this as an epilog to like
I've done my unit test, I've done my smoke tests, maybe I've done my playwright
functional testing now I'm going to go and do some JMeter stress tests.
So for example, I just upgraded from dot net 52, dot net six.
I don't really know whether it got better or not from a load testing defective now I can.
Also I'm looking forward to,
you know what, if our customers use this, we're here in public preview.
We want to learn how to make this better and more useful
for developers and testers to gain their confidence.
One last thing, maybe just to point to anyone who's interested in giving this a try.
We have a great documentation hub
where there's quick stats, but there's also a lot of really good end
to entry trials about some of the features that I showed, but also being able
to automate your load tests and say your GitHub workflow, your Azure DevOps pipeline,
all of those really good stories are all right here in documentation page.
Fantastic. I want to check that out.
Continuous regression testing.
That is exactly what I want.
I want to watch for any performance regressions
or performance improvements, and now I can do it with Azure Load Testing.
Thank you so much, Jon Snow.
Thank you, Scott.
Thank you.
All right.
I am learning all about how to make my sites scale even better with Azure Load Testing today
on Azure Friday.
Hey, thanks for watching this episode of Azure Friday.
Now I need you to like it, comment on it, tell your friends, retweet it.
Watch more Azure Friday.