>> Hey, Nikita. What are we talking
about on this episode of DevOps Lab?
>> We're talking about integrating
Azure load testing into Azure DevOps.
>> Join us for this episode as we
show you how to integrate these tools together.
[MUSIC]
Welcome to this episode of DevOps Lab.
We're back again talking about Azure Load Testing.
This time with Azure DevOps, Nikita, welcome back.
>> Hi, April. Thanks for having me again.
>> Great to have you back. You are
our Azure low testing expert at Microsoft, definitely.
Tell everyone out there what else it is you do for Microsoft.
>> I'm a product manager with Azure Load Testing team.
I've been working for Azure Load Testing for awhile now.
Now, Azure Load Testing is
generally available for everyone to use.
Pretty excited about that
and we'll talk about how Azure Load Testing
can be used in Azure DevOps and integrated in your CI/CD pipeline.
>> Awesome. We've already done
a couple episodes from an out there watching.
We've talked about Azure Load Testing depth.
You showed us how to get started quickly.
Then you went more in depth and shows how to configure it.
In the last episode,
we deploy with GitHub Actions.
Now we're doing with Azure DevOps,
because it is supercritical,
we get this deployed in our pipelines and make it
part of our development process. Go ahead and take it away.
>> Let me start by showing you
what we're going to load test today.
This is the.NET podcast web app that
is basically a web application which has a bunch of podcasts.
You can look on what podcast it has,
and then you can listen to podcasts.
It has a lot of content over here.
Since we expect a lot of uses to use this,
we want to load test and see if this is performing fine.
But with a twist,
we want to test the web app without access restrictions.
As you can see, this is another version of
the same web application and this
is not accessible from a public Internet.
This is a private web app.
We can do this as well as using Azure Load Testing,
and I'll show you how to do it today using Azure DevOps as well.
>> It make sense not to
expose our demo environments to public domain.
>> That absolutely makes sense.
Let's start by looking at the repo where we
have source code for.NET podcast as you can see.
This is our repo that we have.
There's the source code of the application,
but there's also a test folder that I have configured here.
The test folder contains all of the files that are
required for me to run my load test from Azure DevOps.
As you can see, there's an arm template folder here which
contains an arm template for you to manage the infrastructure,
which is the Azure load testing resource.
It allows you to create and manage the resource from the pipeline.
Once you have deployed
your resource and you have resource up and running,
you can just run load testing.
I configured all the files that are required here.
As you can see, you require JMeter Script
for you to run the load test.
I prepared a JMeter Script,
let me just take you quickly to the JMeter Script.
Reference the JMeter Script for the podcast application.
As you can see, I've configured
four different pages that I want to load test,
the podcast page, the home page,
discover page and the search page.
There's also a CSV input data file that we are
providing for our input data as parameter.
I have the CSV data set config for element here.
I also have a user-defined variable which uses a custom functions.
This custom function grabs environment variable.
Custom functions, custom plugins are where you
can extend the core functionality of a JMeter.
You can write custom code and you can
basically use much more than what a JMeter provide out of the box.
Azure Load Testing allows you to also run
custom plug-ins on your JMeter Script.
I have this script configured and I have it virtually
controlled here in my repo.
As you know, the benefit of virtual controlling
is that you can keep track of changes overtime,
so I can always look at what happened to my JMeter Script,
what changes were made,
and I can always keep track of those changes.
>> Absolutely. That's so critical when we talk
about automating our deployments and testing our environments.
You can see on your screen we have the commits.
We can go to the commits and if somebody changes it,
we can see how the changes transpired,
we can revert those changes,
maybe just whenever JMeter Script
is incorrect, we can revert that.
We have different versions of it. This is fantastic.
I love having the ability to virtually
control my testing suite because if we make changes,
we need to know why it changed,
when it changed because that could
affect the output of our tests as well.
>> Absolutely. As you can see,
I have my JMeter Script.
I have the jar file which is basically
the custom function that we have configured in our JMeter Script.
We also have a CSV file to providing for data towards load test.
I have all of these virtual control,
but I also have a config.yml.
This is basically config.yml.
It's how you configure all the settings of your load test.
This is a configuration as code.
As you can see, we have
defined different parameters that
are required for you to run your load test.
Like you provide what is your test stan,
which is basically pointing to the JMeters script.
You have configured what engine instances,
how many engine instances you want to run your load on.
You can provide all the configuration files,
like the jar file, the CSV file.
You have many options like for example you can say split
the CSV file evenly
across all the loading Jin setting up configure,
which allows you to basically
provide unique at all for every engine.
This is very important in scenarios where you are using
login functionality and you don't want
the same user to be logged in in different engines, right?
>> That makes sense.
>> Just clicked, although it was equally.
Then we also have environmental variables.
Now this is the environmental variables that
would be picked up by the custom function that I've written.
I've provided the URL for
the web app as environment variable should.
Now, my web app also requires a certificate for authentication.
Client certificate authentication is one way for
you to secure your in bytes.
We have secured it by using a certificate.
As you can see, I've just provided
the name and I've stored those certificate in
a a keyVault and I've provided the URL for the certificate here.
Once you do that, you can also see that I provided the subnet ID.
Now, remember when we said that the web app is
not publicly accessible and
it's only accessible from accessible network.
>> This is how you can just provide the details of the subnet ID.
You can just provide the resource ID of
the subnet and you're good to go.
You can just load test a private application.
It's as simple as providing a subnet ID.
Coming to failure criteria.
Now this is very important for
a load testing scenario in a CI/CD pipeline.
Basically this allows you to set 10 thresholds,
based on your expectations of
how your web application should perform.
It's important in a CI/CD pipeline because this two goes,
the automation of either failing or passing
your pipeline based on the criteria that you will set.
As you can see, I've set some criteria here for the podcast page.
I'm seeing that my 90th percentile
of response time should not be greater than, 5,000 degrees.
I've said similarly for the homepage at an aggregate level,
I'm seeing that the percentage error shouldn't be greater than 10.
This is one way of me telling that these are my expectations from
my web application and if this does not qualify,
then that's failed pipeline and I'll be
alerted of what action I can take despite mine has failed.
>> Absolutely. I love the failure criteria,
it gives us a very defined way to say,
our app is working anyways,
it's not because it won't scale to what we needed
to and actually underload, it won't work.
This is fantastic. Now Nikita,
I have a love hate with YAML files.
Can you tell me,
how do we generate these YAML files for configs?
Because I struggle with YAML.
What's the easiest and best way to get started with this?
>> Yes, vitally have factors of
the YAML available in our reports,
but you also can download this if you
have run a load test from portal,
which is very easy for you to configure and get started with.
You can always run a load test from
there and if everything is looking fine,
you can download the input artifacts
and I'll show you how to do that.
You can just download the artifacts,
then you get an auto-generated YAML file.
You just have to commit the studio eligible
and you are good to go and you
can just start running all of these.
>> Awesome because I struggle with YAML configuration.
Being able to just configure what I need,
see it, seal the options output that YAML,
that's such an easy way to get started getting
my source control and then we have
ability to make sure that
it doesn't get changed or some changes, that we can revert it.
>> Yes, absolutely.
>> Cool.
>> Once you have all your files that you
need for your load test in your repo.
You can just go and configure your pipeline.
This is a pipeline basically which runs a load test.
You can as even see
this load test first deploys ARM template that we
had to create the Azure load testing results
and once you have created load testing results.
You have an Azure load testing task.
In this task you just provide what is
your Azure service connection,
what is your test config file that we saw
here and what is
your resource group and
your load test resource for which you want to run.
You can just provide these inputs here and you're good to go.
Now we also export the results once load test complete into
folder in your pipeline book space
and this is a folder where we export the results.
As you can see, you can just publish
these as artifacts into your pipeline.
They will be available for you to consume.
You can download those and share those across Azure.
Now that we have a pipeline, all our files,
we can just run this and it will just trigger the load test.
As you can see, my pipeline has started.
While this runs, I can just take you to
a previous runs and this might take some time to run.
I'll just take you, as you can see,
this is the one which is running,
but I also run one few hours ago and as you can see,
this was the pipeline that failed,
as you see the first that failed is the Azure load test in task.
This is the output that you get from Azure load testing.
First, things are going fine when you created a test run
and you've got a portal URL
where you can just go and click and see what is the progress.
We'll just look at that in a moment
and we also have the summary once the test run is completed,
you have what was
the start time and how many virtual users and what was
the status of the test that
is done which says that the test run completed.
But let's look at the test criteria.
As you see, we had defined
three criterias and two of
them failed and that isn't viable of testing task failed.
Now, when we pass or fail based on the test criteria,
this allows you to make sure that
a bill which is not supposed to go into production,
gets stopped by filling the pipeline.
It makes sure that if your bill is not passing your expectations.
Then it stopped at this stage
and you're not introducing
a non-performing code into your production.
>> Absolutely. This is great because it fails the pipeline,
we can start, we can address the issues.
We can see the issues there in the output from
the pipeline run and then you have a link to
go to the Azure portal and see that.
Can you take us to the Azure portal and show
us what that output looks like because
we can get a lot more detail
into what actually happened and why that happens?
>> Definitely, as you can see,
this Azure test run that I clicked from
the pipeline logs and this is how it shows.
It shows that it failed.
You have all the best stylistic field and the test criteria date.
You also get a very good visualization of what happened over time.
You have, how did the response time look over time,
how much was the throughput for
the application that it could handle?
Then as you can see, there are bunch of errors
here and these are 500 errors.
Basically the application was not able to take the load.
You also have the option of configuring server-side components.
Now you can just see
what other application components of
your target application and you can basically monitor those.
I'm selecting.
>> All your infrastructure and we can see
exactly where the bottleneck is.
All those dependencies in our infrastructure we can select,
we can pull them through and it will pull the reporting
out so we can define where this is failing. That's awesome.
>> Yes. As you can see,
you get all the metrics from
Azure Monitors for the app components that are configured.
I can see what happened over time.
You can also configure custom metrics yet I've selected using
those foldments but you can also configure if you're
looking for something specific as,
you can see the app service CP
was high but it is still high but as you can see on
the sequence over the CPU percentage was very
high and that could be one of the reasons
why the app was a non-responding day.
They were not high end data.
This way you can just debug and see what other issues,
is it a code issue or is it a issue with your infrastructure,
do you need to upgrade the skill or
both measures can be to improve the performance.
>> Also. Nikita, thank you so much.
I have love seeing
Azure load testing go into general availability.
The integration to both Azure DevOps
and GitHub actions has been awesome.
Thank you so much for showing us how to put that into
our pipelines in Azure DevOps and for
everyone out there we'll have all the resources
available for you in the description
so you can follow along and get started with Azure load tested.
We'll see you next time on the DevOps Lab.
>> [MUSIC]