[Music]
Hello and welcome to Microsoft Mechanics Live.
Coming up on how we built it we meet the head of data science fromAccuWeather
to learn how they're using Azure Machine Learning to both train and extend
their weather models comprising over 55 years of predictive weather knowledge
to help analyze weather data with business data to assess
and plan for specific business outcomes.
But please before we get started I'm joined by Mary Radich give her a warm round applause.
Great to be on the show.
[Applauding]
Alright, so AccuWeather is really known as an early adopter.
They started their mobile transition to the Internet in 1997 it really has an extensive data set, forecast models
AccuWeather is running a global forecast engine that can predict weather behavior around the earth.
This is a pretty topical subject today given all the recent extreme weather events were having.
That's right we're able to provide a hyperlocal specialized forecast with superior accuracy
using global real-time and historical data
to provide forecast to you and to businesses down to GPS location.
We process a petabyte and a half of data every day derived from various sources.
That includes hundreds of thousands of real-time weather observations
updated every few minutes from land, ships, and aircrafts.
Satellite observations and lightning strike data for locations all around the world.
We use over a hundred and sixty different global forecast models consisting of more than 12
terabytes of data run daily on supercomputers by governments and all over the world.
Including our own proprietary models and also commercial models that we license.
So you're really collecting massive amounts of data from sensors all over the world.
How is it all stored?
So we use a combination of SQL Server for historical and forecasted data.
We have data on prem and data centers and also in the cloud
using services like blob storage, Data Lake and Data Warehouse.
Like many of you out there we're also looking to move more of our data to the cloud for scalability.
And right and these are these are draw data streams that you have to process.
How do you go about processing and analyze and cleaning up the data
as it comes in to make that into forecasts?
Of course we have meteorologists worldwide that are analyzing data 24/7 and 365.
We're looking at ways in which we can blend models to drive the accuracy of forecasts all around the world.
We use proprietary techniques
to continually modify the output of our forecasts.
We're also working with Microsoft and dynamic data on migrating more of these processes
into production using Azure Machine Learning.
And one of the things I've noticed is that your forecasts actually go quite extensively three months
into the future, for example.
These are pretty extensive and in longer-term forecasts that you're making.
That's right many of our clients want a plan for more than just a few hours or a few days.
This could mean the next month, the next quarter or even planning for an entire year.
Right so after I've analyzed all this information and I've developed models
for forecasting and weather, how do you then make this available for me if I'm a developer?
Or say a customer that wants to use this?
Of course we have accuweather.com.
We also have over 1.5 billion devices that have our mobile applications.
We're available in over a hundred different languages and dialects
We support 30 billion data calls per day,
and we serve almost half of the fortune 500 with site-specific warnings, data, and analytics products.
All right so let's convert this into business data.
So how do we bring AccuWeather's weather data then into businesses?
Sure we utilize Azure Machine Learning to analyze the relationship
between weather and businesses and consumers.
For example, knowing when people consume more dark beer or more light beer.
In Dallas it takes an increase of 15 degrees before people start drinking more light beer.
But in Seattle where it's very cloudy, very rainy it only takes the shift of 5 degrees
before people start consuming more light beer.
Weather affects the type of coffee you drink, what restaurant you go to,
your outdoor activity levels and even health risks like asthma.
So by knowing where you're at, we know what weather is happening in your area.
We know what type of drink you're likely to order, what restaurant you may be going to,
if you're going to be walking there and what type of music you'll be listening to along the way.
All of this information is very valuable to businesses.
So can we look at some of the commercial offerings that you have
and how you're leveraging Azure Machine Learning?
Yes, we have two main enterprise services that run on Azure.
We have AccuWeather g3 Express
which is self-service application available on d3.accuweather.com
Anyone can use this, you plug in multiple locations and you get back
a real-time forecast using our proprietary impact indicator
which tells you on a scale of 1 to 10 what the upcoming impact of a weather event will be.
This can range from no impact to extreme impact.
We also have an advanced offering where you can upload your specific business data
and you can use advanced statistical techniques to analyze trends and relationships.
And then you get forecasts that are customized based on your data
and this will eventually be a self-service application.
So what do I need to do today if I want to be able to take as a business
advantage of this real-time intelligent information
and make my own models then that apply to me?
Sure, well let me show you.
So here we have a CSV file where you can see daily sales for a number of locations.
And we have to make sure that we have a unique identifier for each one of these locations.
So you can see staff count, you can see store ID, and you can see all of those sales that occur every day.
CSV file, what many of you are used to working with yourselves probably.
So once we have that client data,
we have to merge it with historical weather data using a joining script.
We also test for seasonality.
Weekdays versus weekends, month, etc.
Then we test different types of transformations.
So this can include lags thresholds interactions.
For example, one day of cold weather can have a much different impact
than if you're experiencing several days of cold weather.
Especially if that happens over a weekend or a holiday.
Then we have to determine the most appropriate statistical technique.
This can include neural networks, decision tree, regression modeling.
Once we have our models scored then we use the most appropriate model.
So you get an ideal forecast of your client sales.
So here in Power BI I've got on the left hand side
if you kind of focus there in terms of what the impacts are.
Some of the things that we can do with this are actually pretty powerful.
So if we look at things like the bottle water which is the blue stripe and the battery sales,
we have a nice kind of overall view of the the entire hardware chain.
We can also see kind of the the impacts on other other factors like power backups and employee
and also forecasted sales.
So I've got all this information at my fingertips which is really cool.
Of course I can go ahead and filter that if I want to.
Someone go ahead and filter that by Maryland because that's actually been an area that's been hit
by some highly impacted weather in this case.
And you can see here on the eighth and ninth of the month,
we have these high impacts this doesn't mean that it's really cold or really warm.
In this case it's really cold.
So we see that there's ice impact.
We can see the the sales really spiking up right until that point.
Then going down because people can't get out of their homes probably.
And then we can see even down to the store level if we want.
Here we've got a nice store number pivots.
So we're going to show store number 55.
That's going to update the map and do all of that and show me the impacts of that specific store.
So now I've got drilled down impacts for that area of Maryland in that specific store in that region
Here's the most impacted store though.
So we see here as we as we zoom over that
the zip code so this is the key right?
Right definitely, so think about it you may want to look at a particular store location
especially if you're the manager of that store.
So you can have different views of this for whatever your role is whether you're chief operations office,
a regional manager or a store manager.
So you can use this information then for example to look at inventory levels to make sure you get the right
inventory leading up to that snowstorm or that ice incident in this case in that impact.
And you can use it for things like staffing to make sure that you
don't have too many people go in because you can see that the amount of people that will be make it
to the store because of so much ice and so much weather is going to be reduced.
So this gives you some really powerful information.
How does the team go about building something like this and really creating the model?
Sure so of course it's a lot easier because you can connect Azure machine learning streams
directly into Power BI.
So being in a Microsoft environment makes those connections a lot quicker and a lot easier.
And then you just need to be able to go in understand a few basics about the software
and be able to build a beautiful like you see here.
Now what's happening behind the scenes to ensure that the clients receiving the most accurate forecast
and stays up-to-date maybe as forecasts change?
Sure, Azure enables us to quickly call in the latest forecast in real time.
So as weather changes so does the client output.
As more client data is uploaded as more historical weather data is captured
we're able to retain and rescore models.
That way the information that you see is the most accurate representation
of the relationship between weather and your data.
So putting this all together means you have a model that's completely unique
to your data, your locations and to the client.
So this application could have been used for example in recent events.
Or we could maybe help plan for fuel supplies
or other things in the event of say hurricane weather that we've had recently.
That's right so here in Florida and neighboring Georgia we were able to predict ahead of time
which areas would receive massive amounts of property damage and power outages.
That way consumers and businesses could be prepared ahead of time.
For consumers this means possibly evacuating or stocking up on supplies
like batteries and bottled water.
For businesses you want to make sure that your employees are safe.
But, you also want to make sure that you have supplies for those consumers who are really depending on you.
And so when people think of impact they often think of extreme events
like Harvey and Irma.
But oftentimes those small events that happen over time that can have a major impact.
Think about a home improvement retailer who experiences a prolonged winter.
A few weeks of delay of spring can cost millions to that home improvement retailer.
Right so as AccuWeather moves most of its data to the cloud,
where do you see the potential for machine learning going?
Sure well we're continuing to centralize all of our data
as many of you know that so quite an undertaking.
We're looking at centralizing both weather and non weather data that way we can continue to analyze
the relationship between weather and consumers.
We also have groups working on BOTS and providing insights in a more human fashion
combining machine learning and natural language processing to deliver more actionable insights.
And also delivering those insights all across the company
So optimizing our forecasts, our products, our advertisements for consumer audience
and of course for our business clients to protect, people, property and profits.
So we recently do the show and I machine learning which was actually released this week
which currently that looks at Azure Machine Learning, the workbench, it's in preview now.
To build out an AI model.
In this case we used it to actually help researchers to build out whether or not
a snow leopard was in an image because they're part of the conservatory to conserve snow leopards.
So this is something maybe that we can use here in the wild.
Here we can see for example the the slides.
It's really hard even as a human to detect where the Snow Leopard is in that one image.
Now with kind of the the machine learning algorithms,
some of the things that we've done with the ResNet 50
model we can then detect automatically whether or not there's a snow leopard in the photo.
Do you see something like this maybe applying to weather patterns,
satellite images, etc that you'd be able to use like this?
Definitely we've been working with AI on the consumer and the business side
using machine learning, natural language processing, and of course imagery analysis.
Imagery analysis can be used to determine what type of cloud is present what type of precipitation.
And in turn that can help us improve our historical, current and our forecasts of weather data.
So how does this then compare to what you had before?
What were you doing prior to this in terms of making these types of decisions?
So before we used to require one data scientist per customer.
It's very expensive, resource intensive, customized service.
AML Modules allow us now to take that data science process
and replicate it in just a few clicks or even automatically.
And we were also able to customize those streams by using our own proprietary techniques
using R, for example.
So really fascinating but what tips can you embark upon everyone here and people watching online
in terms of what they can do to apply machine learning to their own data?
So you know your data.
You have great domain knowledge that should be applied to this machine learning process.
So make sure that whoever is in charge of doing statistics implementing machine learning
make sure they really understand the data and they take that into consideration
when they're doing this process.
Also know your modules.
Don't go reinventing the wheel and custom coding everything if it can be done in just a few clicks.
However really take the time to learn like R or Python
that way you can have flexibility and how you integrate and
customize it to your needs when needed.
So really great tips R and Python we're hearing that from basically every data scientists that we talked to.
Look forward to seeing the progress with AccuWeather and kind of more
implementation of Azure Machine Learning other Azure capabilities there.
Thanks for joining me Rosemary.
You can also check out AccuWeather service at the link shown.
And you can start actually plugging that into your applications to
be able to see these impact indicators throughout your own apps.
That's all the time we have for this show thanks for watching we'll see you next time
[Music]