We are live. Good afternoon. How are you today?
Great, great. So my name is tony.
I'm a senior product manager of cloud ark i group within microsoft.
So i have been working on big data project past several
Years. Now we are nounsing the live or
The project reminder as a lab service.
So before i dive into detail about the project finder, just
Want to step back and provide a lot of some of the context about
The motivation why we are doing this. Nowadays, a lot of companies
Using data to make decisions and improve their products and
Understand their service better and understand the product better.
The problem is there's a lot of data out there.
Inside very quickly, especially the passing.
The pathing happens. So how we can automate this
Process to find out whether there's something so that's the
Anomaly detection. Come into play.
Anno, ma'am alley detection is the technology behind project
Anomaly finder. So before i go too anomaly
Finder, just a quick survey showing of hands how many have
Heard of anomaly detection and understand this type technology.
Several people so anomaly
Detection whether use application in this
Industry. Detection.
The problem is that is there anything strange, anything rare,
So whether my value as of now is now supposed to be.
That is the problem anomaly detection you're trying to
Solve. You can picture that in this
Screen. So it's basically a line chart
For people of human beings, there's a huge strike.
You can see that as a anomaly. So the anomaly detection is
Really how we can automate this process for the machine to
Understand whether any data point is anomaly or not.
So basically for doing those, we basically need to take the input
Of time series of of past history and use that to build a
Model and use the model the prodiction. If there are anything
Significantly different from the rediction, we think that is anomaly.
So then what exactly is project anomaly finder?
What so it's a new lab of dem he had he's service lunched on build.
This service build on azure so it's a rest api you can use
That. If any language whether, for
Example, python, c#, java, everything so we have that.
Enough sample of our documentation site. The input/output is pretty much
Very straightforward. So basically, you give us a time
Series, given time range. Then anomaly detection engine
Hosted in service is basically take that historical input and
Build model, use the model to do the prediction detection.
Went to see if there anything out of historical pattern.
Then we just give you the enough input to use that to do host
Processing. The output we provide include
Original time series and also whether a given point is anomaly
Or not and then the value and the upper and lower bond you can
Use to do the visualization. That can show you later on with
Some of the co-sample.
So what you can build with this api. So i can.
The a pi is pretty much very generic. It can be used across a lot of
Industry like finance, technologies, you know,
Transportation, healthcare. So any industry they care about
In matrix and care about when something goes wrong, they can
Actually leverage this. So i'll give you two examples.
For example, if you are advertising company, you try
Your daily revenue treatments and you want to be alerted if
Anything goes wrong. So because those are money, you
Don't want to lose money. So basically, the first is
Basically we can visualize and detect how if there are any data
Point has been shown, some of the dips or spikes in some cases.
We are also provide you with the upper and lower bond values so
You can draw the purple shade over there.
Anything out of that shade is pretty much anomalies.
The other example is let's say if you are technology company,
You care a lot about your service health and there's, for
Example, you can give us the input of at the arrow counters.
Then we can pretty much detect anything has been wrong it and
There's a sudden spike or level changes. So basically the pattern changes
We can detect that with apis.
So let me give some example about what's the input data
Looks like. For example, the input is very
Straightforward. It's just decent format.
Has two field. One is the period, the other is
Basically a read of data point, a time series.
Then you give that, this, as input in the request body.
So you post that to our api end point then we will be able to
Output the following in the response button.
So basically, the period part is that we can detect automatically
What seasonality of this time series. For example, we think that is a
Weekly pattern. S is the seven.
We will give then the expected value and whether this is anomaly.
So here is several category of anomalies.
Anomaly underscore positive is basically, for example, the
Sound use cases only care about the spikes. You may want to use that.
Some some use cases you only care about the tips.
It's auto negative. You if you care about both bag
And dips, you use this anomaly. We also provide up and he lower margin.
Those parameters are for you to draw on the chart the
Visualization. To help you to visualize what's
The deviation from the expected value, the boundary should be
Like. Okay. And then here are just a switch
To the jupiter notebook and let's do some demos with jupiter
Notebook. As i said earlier, so basically,
Our api is rest api. You can pretty much use all the languages.
So here here is json languages.
Not -- sorry. Python languages.
So there's some prerequisites. You need a subscription key to
Be able to cause this api. Right now as lab, so this api is
Totally free.
You can try without any cost. This is our project main page.
So to get a subscription key, you click here.
Since already been subscribed to that, so there's actually ape
Sign-in step. You use your microsoft account
To sign in and this will create those subscription keys. So, for example, the project
Anomaly finder will give two keys you can use either of them.
And the copy of those keys and paste that here.
Yeah. So basically here is you paste
Key here and input is you can find is our end point, you can
Find our documentations.
So here we will analyze the assemble data called
Simple.Json. Load the format disc.
It's pretty much contained the similar format i show you
Earlier. It's basically a time series
With one month of data or more than one month of data.
So this first block of code is pretty much just a load called
The api and load samples and
Call the a pick i and get the response.
So that is the pretty much what you are interacting with our
Api. And then the address part is
About, you know, how this looks like, the response it looks like.
Before i go to the response, so it's look like this, so one of
The interesting aspect of time series anomaly detection is
Different from other, you know, the machine learning project
Like the face api. Pause for the fifth api, you
Pretty much can determine whether this is sad or happy.
Different people have pretty much the same opinion.
Right? but for the time series
Anomalies, depends being on different use case, different
People actually can have different opinion, whether it's
A point anomaly or not. We have to have the sensitivity
Come into play. Because even for the same time
Series, if you put it, you know, revenue stream or if you compare
With the same time series, you know, service health domain,
People could get different perception about what's anomaly
Or not. So here are code, just not only
Generalize , you know,, not only
Show you this anomalies and also we provide an upper bound, lower bound.
You can adjust the sensitivity and you can see it can resolve
About, you know, so that you can adjust the different tolerance
Of the anomaly detection. And then you could potential
Ly. So how do you do that?
Let's go back too the code. So remember, our code return to
You the upper imaginerrer and lower margin.
That is something we provide to the developers that you can
Define. Can use to times the upper
Margin and generate different sensitivity
Boundaries. For example, from the code here,
It is very straightforward. Sensitivity times the upper
Margin and lower margins. And plus or minus from the
Expected value and then we can just dynamic adjust this upper
Bond lower bond so you can have different sensitivities to even
The same time series.
Right? then let me see whether i can
Show another example. Let's load a different
Run again. Yeah.
So this one is actually a time series with sort of granule art
Of a data data point.
We can do the same by acing sensitivity.
So the code i show here is now the only way you can adjust.
You can define by your own. Depends on your business case.
So and you can define different steps to tunes anomaly
Detection. Right? so it's all up to your own.
We -- our api about l just first return to you by our own tee
Fault settings. Then you can tune that based on
Your own business needs. Okay. So that's all for my demo and presentation.
So i'm not sure if you have any questions.
Okay. If no further question, then