[MUSIC]
>> Hi there. My name is Mark Downie.
I'm a program manager on
the Visual Studio production diagnostics team.
Today I'd like to talk about recognizing and
diagnosing sync over async issues in your applications.
Asynchronous programming has been around for
several years now on the.NET platform,
but has been historically been very difficult to do well.
Async await concepts were introduced in C# 5,
and since then, asynchronous programming
has become incredibly popular.
Today, most modern frameworks are fully asynchronous,
and it's frankly nearly impossible to avoid
the async keyword when writing web services.
Today, I would like to go over some of the best practices to
avoid sync over async, and more importantly,
show you how to recognize the anti-pattern in production and use
Visual Studio to avoid
these problems when developing and updating your code.
Today, I'm going to show couple of
really common examples of
how we can do asynchronous programming badly.
We're going to start with this first action on this controller.
This is a prototypically bad way of dealing with task.Result.
Frankly, task.Wait has the same type of issues.
Using task.Result or task.Wait to block wait on
an asynchronous operation is actually worse than
calling a truly synchronous operation.
Let's look at what's going on in here.
I have a action on
this controller that is calling into this service,
and this is an asynchronous call to this method.
In fact, if we do a Control Left 12,
we can see that it's doing simply
a random and waiting for some period,
representing a database call or
some other request to
a service and then we're returning some results.
Control minus to go back to our action.
Because this isn't setup as async await,
I have to do a.Result in order to make this compile.
It seems like a reasonable solution here.
The problem, when we take a look at this
from a big picture standpoint,
what's happening here,
while this will compile, and it will run locally,
and it will probably run in production under low load,
as we get higher load,
there's going to be problems, and I'll describe why.
An asynchronous operation is been
kicked off essentially from here.
But the important thing is that the calling thread is blocked,
waiting for the operation to complete by using.Result.
When the asynchronous operation completes,
it will unblock the code waiting on that operation.
But this operation is taking place on another thread.
This is what we mean by sync over async.
The result is that we now need to use two threads
concurrently instead of one
to complete what is essentially asynchronous operation.
From the perspective of your end users and your customers,
there's a whole bunch of symptoms that are coming here because we
are essentially going to starve our thread pool,
so the leasing mechanism for
deciding which threads are going to execute which code,
that whole leasing mechanism becomes starved.
That will lead to long process times for
your running applications and your requests.
It'll lead to timeouts.
It will have your customers asking why everything is so slow.
Again, this typically only occurs under load,
so as you're developing locally,
you'll never see this.
Then when you have reasonable load on your servers,
you probably won't see this.
This phenomenon will result and
manifest at the worst possible times.
There are unfortunately lots of ways to get
this particular pattern wrong.
If we dive down a little bit here,
we see another example.
This time we're using a Task.Run
in combination with GetAwaiter and GetResult.
Unfortunately, comes to the same unfortunate predicament.
Here's another example with Task.Run with a few more GetAwaiters,
but it ultimately comes down to the same problem.
Let's go all the way to the bottom,
to the right way to do this.
In this example, this final example,
in this action, I use the async and await keywords.
I wrap my result in a task.
Then now, as you can see,
I no longer have to have a result or await for this operation,
and I can return the result
immediately at the bottom of this method.
A reasonable question is, what actual difference does
this make to an application running in production?
What I was able to do was set up a series of tests,
where we run under
similar load for our good code
and what is essentially our bad code.
The two pieces of data I want you to focus on
in this.NET counters results here,
we're looking at system runtime statistics here,
and the two pieces of information that are really important are
thread pool queue length and thread pool thread count.
The thread pool queue length is really
important in that it demonstrates
how many requests are actually waiting to even be executed.
Over time, the thread pool queue length is increasing.
The second really important one is the thread pool thread count.
Essentially, if we see this increasing,
we know that the thread pool is having to add new threads to
manage the increasing load against your application.
Seeing the thread pool queue length as long as it
is and increasing to as much as a depth of
60 tells us that we have
60 requests that are not currently being serviced.
Looking at this,
again, with the same load,
but with the code now corrected to use async await correctly,
our thread pool queue length remains at zero,
meaning there are no requests that are waiting to be serviced.
In fact, our thread pool thread count,
the active thread count,
stays relatively low, in fact,
almost half of what it was before.
Again, this is typical for
scenarios where our services are running correctly.
When it's running incorrectly,
we tend to see an increase in the thread pool queue length and
we tend to see the thread pool thread count increase over time.
In addition to using.NET counters
to validate the state of the thread pools,
you could also take a memory dump
using another CLI tool like.NET dump.
That memory dump would contain
sufficient information for us to analyze
the state of your application in a safe way in Visual Studio.
Here I'm going to go ahead and hit "Control O" and
open up a memory dump that I have captured earlier.
I'm going to go ahead and run diagnostics,
which is one of the actions you can take here.
Running diagnostics allows me to
start a diagnostics within Visual Studio.
Then I can get access to a bunch of tools that
might be helpful to me in this context.
Because it's a process responsiveness issue,
the symptoms we are likely to encounter would be,
my application isn't responding quickly,
or there's too many timeouts.
Process responsiveness would be a great option for us to look at.
I'm going to go ahead and analyze that.
Immediately, it detected a couple of things.
First is that the thread pool threads are out of threads,
meaning it hasn't got the opportunity
to respond or to add new threads,
which takes several seconds to do.
That's a first indicator that there's something wrong here.
We've starved our application of
the ability to execute additional work.
More important than that,
it's detected that there is actually a sync over
async issue here that, in fact,
not only have we run out of the threads to service our requests,
but we've identified the code
and the call stack that is also problematic here,
where we are synchronously waiting inside an asynchronous context.
Then we can use this tool to dig a bit deeper into this problem.
While those are the tools to capture this issue in production,
there are tools that allow you to capture
this issue earlier in your life cycle.
I have the same application open here and obviously,
I have multiple areas where I've
done this incorrectly in my application.
What I could do is actually apply code fixes and refactoring,
powered by analyzers,
to help me while I'm in my IDE.
While I'm developing my code,
it will tell me straight away that
I'm developing something with an issue.
I can just apply this to even existing code.
I can go to Tools go to
NuGet Package and Manage NuGet Packages for Solutions.
I can then search for Microsoft Visual Studio threading analyzers.
This threading analyzer can be
installed against my existing project.
What this does is it takes advantage of the.NET compiler platform,
which is Roslyn,
and there are several analyses listed in here
that allow me to understand where
my app has gone
against a standard way of doing a particular thing.
Now I have that familiar squiggling green line
telling me that I have some problematic code.
In fact, it's able to detect every single one
of the anti-patterns that I have described here.
You can click on the links and get additional help,
specifically about each of these and why they're problematic.
In today's session, we looked at
best practices to avoid sync over async issues.
We demonstrated and discussed how you can detect
sync over async in production using.NET CLI tools.
We concluded with tips for using Roslyn analyzers to help
detect and avoid the issues in the code
in the first place. Thanks so much for joining.