NIKITA NALLAMOTHU: Today we are going to talk
about what is Azure Load Testing?
How you can integrate into your CI/CD workflows and
how you can identify
performance bottlenecks using Azure Load Testing.
Hi, everyone, I'm Nikita,
a Product Manager in
Microsoft working on Azure Load Testing.
NAGARJUNA VIPPARTHI: Hi, everyone.
I'm Nagarjuna
working on Azure Load Testing.
Azure Load Testing is
a fully managed load testing service
that helps developers and testers
generate high scale load easily,
get actionable insights
and identify performance bottlenecks.
You'll be able to generate
high scale load needed to load test
your endpoints
without needing any complex infrastructure.
You can specify the load
easily identify performance bottlenecks,
and which components are maxing out when your system
is under load and
Start with performance testing
early in the development lifecycle,
and easily identify regulations.
Specify the performance expectations of
your application using different metrics
such as response time,
error rate, throughput,
etc, and make sure that
the pipeline gets executed
only if these performance benchmarks are met.
You can view the load test summaries in
Azure Pipelines or GitHub and
view the details results on
the Azure Load Testing portal.
For this demo, we'll be using an e-commerce application,
Contoso Traders, which
specializes in electronic products.
Logging into the Contoso Traders application using
my Azure Active Directory AD credentials.
Once I am logged in,
I can search for the product.
Scroll through the products,
pick a product, and add it to the cart.
I can go to the cart,
view my items that are
added and go ahead to check out the items.
NIKITA NALLAMOTHU: Let's look at
how we can load test
this application that Nagarjuna just showed us.
This is a JMeter script that I've written for
load testing that e-commerce Contoso traders application.
As you know, Azure load testing
runs JMeter scripts at scale.
You can just upload
the JMeter script and run
the script on a shallow texting.
Looking at this JMeter script,
we have a trade group for a co-authentication.
Remember the AD authentication
that Nagarjuna has showed us,
you can add a trade group and add
a HTTP request to login
to the application and fetch a better token.
As you can see, I'm hitting
the Microsoft login endpoint with
my username and password that I want to login with.
This will extract the token from the response that I get.
This token can now be used for
load testing my further endpoints.
Now, I want to load test the user scenario where a user
logs into the application
and then learns at the product page,
searches for a product, views the product,
then adds a product to the cart and then can view a cart.
They can also search an image on the website.
As you can see, I've configured
my different samplers here for load testing,
each of these APIs.
Now, each of these APIs have
a domain name apart
all the parameters that are required for the payload,
and they also have a Authentication Header.
This is the Beta token that is
required for me to log in using AD.
It's as simple as that.
You can configure AD authentication
and load test each of your endpoints.
I also have some variables here.
I have parametrized my script because I do
want to hard code these values into my script.
I have my API endpoints parametrized.
I also have my tenant ID, client ID,
username, and password that I need for authentication.
These are secret variables
that I can provide in Azure load testing in
a sensitive manner and we look at
that later in the later part of the demo.
NAGARJUNA VIPPARTHI: Let's now go ahead
and see how we can
run the load tested Azure load testing.
We'll get started by creating
an Azure load testing resource.
You can pick a resource group,
give a name to your Azure load testing resource,
and pick the region.
When you choose a region,
the load's generating test engine instances are
deployed in the region that you pick for your resource.
For better results, you can choose
the region where you expect most of
your users to use the endpoint from
so that you can better simulate your test load.
The Azure load testing resource is
a hub of all your load testing activities.
It contains the different tests that you have created.
You can create any number of tests as required,
and you can run each of these tests
multiple times whenever you need to run a test.
You can get started with a quick test where you can just
provide a URL multiple parameters in terms of the load,
the test duration, and the ramp up time or you can do
a full-blown test using
the JMeter script similar to
the one that Nikita just showed us.
NIKITA NALLAMOTHU: Let's look at how
we can load test
your application from a CI/CD work flow.
This is my GitHub repository,
which contains the source code of
the Contoso Traders application.
As you can see, I have my application source code here,
but I also have Azure load testing
folder which has all my scripts
that I need to run the load test.
Let's look at what I have in this folder.
You'll see I have the gv.script
that I just showed you some time back,
and I have converted this into my repository.
I have all of the files that I
need for me to run the load test.
I also have a config.yaml.
Now config.yaml is a way for you
to describe all your test requirements
in our code format.
This is configures code for the load test.
Let's look at what this file looks like.
So you have basic details of your test.
You have your test name,
you have the test plan,
which is the JMeter script,
you have the description of the test,
the number of engine instances that you
want to run your test script on,
and all your configuration files.
What is interesting is you can also define
certain service level indicators for your test.
I can define my failure criteria for
different metrics that I want to monitor
and make sure that my application meet
these expectations that I have for my APIs,
for example the view product API,
I'm setting that my error percentage
should not exceed 10 percent.
I'm also setting some more SLIs
you're having for the product landing page.
I'm seeing that both 90th percentile of my response time
shouldn't exceed 3,000 milliseconds.
Similarly, I have
a 90th response time for the view part,
and the threshold is 100 milliseconds.
I can say these criteria here.
Based on how your application
performs during the load test,
evaluate these criteria, and you can pass or fail
your workflow based on the results of these SLIs.
This makes sure that if you have
a code which does not meet your expectations,
you're not promoting this code into production,
which can result in downtime
and it could not
lead to a great experience for your customers.
That's how you can make sure
that the code that you promote the production,
it was always meeting your expectations.
[inaudible] we
had some secret values
that we needed for the authentication.
It's not good to checking your secret values
directly into your GitHub repository,
and that's why we can store them in
a key vault and provide the secret identifier here.
As you can see I have a client-Id, tenant-Id,
the username and password that I need for
logging in into my AD credentials.
I can provide the secrets from the key vault here.
What Azure load testing would do
is when the script is running,
it will fetch that credentials from key vault in
a secure manner and provide it to
the JMeter engine when it is run.
This secrets are not
exposed anywhere and are very secured will fetch,
and used during the load tests.
All less than one
sensitive parameters like my endpoint for
my APIs or any other parameters
that I would like to pass to the load test,
I can pass as environment variables here.
Now let us look at what the workflow file looks
like and how I can
integrate this load testing into my GitHub workflow.
This result be GitHub workflow that I have that can run.
You can schedule when you can run for this demo purpose,
set the manual run but you can
run it on a schedule or you get run it when
you have a PR or atomic into your branch or repository.
I have defined some variables
which are my load test resource,
name, my resource group, and the location.
These are the ones that we just
saw of a Nagarjuna showed you in the portal.
In a CICD workflow you can
create this infrastructure that we
need to run your load test using
the arm or the bicep templates.
Let us look at how you can run the load test,
so I have logged into Azure action code where I moves
Azure login action and uses
[inaudible] principle to login to my Azure account.
When I use the Azure load testing action,
I can provide the load test [inaudible] that we saw,
and then provide the name of
my resource and the resource group.
Once I'm done running the load test
I get all the results of
the load test which are
exported into your GitHub workspace.
You can upload these results into
your GitHub workspace by just uploading the artifacts,
and you will find them in the virtual run.
Let's look at a run that I had
recently run this workflow and let's see how it looks.
This is a run that I had,
as you can see this run has failed and
we see that the load test step has failed here.
Let's look at why was run failed.
As you can see the test run uploaded
the test plan and we'd started running
a test and the test run also completed.
We have a quick summary of how many virtual users,
what was a step tester status and everything here.
These are my test criteria,
these are the SLIs we talked about.
I see that one of the three passed and two other failed.
As you can see the error percentage
for my view product was very high.
The product run page response time
was below and that is a good sign, so it passed.
But my response time for my view part
was very high and so that failed as well.
As you can see we have all the client went to explore
polygenic risks that I had in my gym district.
I have all the status it's here.
[inaudible] see that we tested they'll fail
because the test criteria did not meet my expectations.
I want to see why exactly my test result fail,
why did my APIs do not meet the expectations?
For now we are going to look at how these metrics look in
Azure portal and we'll take you to
Azure portal and show you how this test run look,
and what were the metrics for this test run.
Taking to portal and I can view
all the results of my test run.
You see this is the dashboard that we provide
you and you can view this in
Azure portal for all your test
runs from whether it is around
trauma CICD pipeline or if it
is run from the portal itself.
You can see the statistics here we have
a high-level overview of what was my are percentage,
my response time looking like,
what was the throughput
that application was able to give.
I see that the test criteria as well,
two have failed and one has passed.
Now we look at what is the client-side metrics.
Client-side matrix are the matrix
that GitHub provides from
the client side to tell what was
the response time and
how many edits were reported back by the application.
As you can see as my load ramped up,
at certain point my application started throwing
errors and do that all [inaudible] errors,
so there's something wrong with my application.
I can monitor the Azure components that I have.
For example, the Contoso trader application
has any case tested,
it has some databases and has an app service.
I can always configure that in
my test run to show me the server-side metric [inaudible].
As you can see now I can monitor
what went wrong with my application components.
I see here that the Kubernetes cluster was
high and I'm looking at what went wrong.
I see that my Cosmos db for my car xAPI,
the ALU consumption was very
high at certain point it reached 100.
That could be a bottleneck in my application.
I also want to see what happened with the products API,
so I see here the SQL db for my products
db is reached 100 percent
in the CPU percentage utilization.
These could be bottlenecks for
the products and look [inaudible] and I can
go ahead and look at what can do maybe I can increase
the RUs for my cosmosdb or
I could also increase the skew,
add one more instance or one would go to my SQL db,
and that might help me get a better performance.
This is how you can co-relate
the application component self-assigned
metrics with the load that you have generated,
and seeing that at certain point is
your application component not
responding because of the high load.
NAGARJUNA VIPPARTHI: In this session today we looked
at the Azure load testing service,
how it can be used to generate high-scale load against
endpoints hosted on Azure or on-premise
or on any other Cloud.
We looked at how to set up a JMeter script to load
test an e-commerce application, the Contoso Traders.
We looked at how to integrate Azure load testing in
the CICD pipelines and we
analyze the results to identify performance bottlenecks.
You can try out the demo by using
the sample repository and the artifacts available there.
You can use the resources available
at aka.ms/MALT-resources to get
started with Azure load testing.
Please do share your feedback on Azure load testing
at aka.ms/MALT-feedback. Thank you.