On September 11th, 2023,
weather predictions in the northeast
of the U.S. sounded like this.
All eyes are on Hurricane Lee.
The storm has strengthened back to a category 3.
We're expecting it to make
a northward turn over the next few days.
By September 16th, after being
downgraded, storm Lee made
landfall in Nova Scotia, Canada,
flooding roads, downing trees and
cutting out power for tens of
thousands of people along the East Coast.
At least five days before Hurricane Lee struck
land, weather forecasts had roughly
predicted its trajectory.
But another forecast beat them
three days before weather stations,
an AI model created by Google
predicted the cyclone's path.
The AI revolution has reached
meteorology, and it's at a
time when we are responding to
extreme weather more than ever.
We're about to find out if it will
help us prepare by bringing the
future into clear view.
Predicting future weather more than
a few hours out starts with creating
a snapshot of Earth's current
atmosphere.
Scientists do that by collecting
data from sources like satellites
and weather stations and buoys
located around the world, taking
pictures of clouds and measuring
temperature and pressure and wind
speed and humidity.
All that disparate data gets
fed into computers, which generate
a 3D grid of boxes that represent
the atmosphere, both vertically and
horizontally.
Computers then do a lot of physics
to determine how these conditions
interact with each other, and they
produce a forecast.
I think any forecast has like
150 trillion calculations.
It's pretty amazing.
All that math requires some of the
world's most powerful
supercomputers.
The two big ones are run by the
European Centre for Medium-Range
Weather Forecasts and the National
Weather Service in the U.S.
To make a local weather forecast
from this global model,
meteorologists zoom in and refine
their own forecasts with their local
expertise.
Like if they live in a hilly area
or a flat area or near a lake,
they'll adjust those models and do
their own professional
interpretation based on their
area.
No matter what, this initial 3D grid
of that sphere is never going to
exactly replicate reality.
There's too many gaps in the data we
can measure.
That means forecasts get blurrier
the further out you go.
Which is why the big weather centers
don't just generate one forecast.
They tweak the initial data and
produce up to 50 forecasts.
It's called ensemble forecasting,
and it helps meteorologists measure
uncertainty.
If all 50 forecast looks similar,
there's a higher certainty in the
prediction. But if there's a lot of
variation there's much less.
We got to kind of keep an eye to the
sky. There's a potential of another
storm in the works.
This went from 0% chance to
40% at two o'clock,
70% at 8:00,
and at 11:00.
We have a tropical storm.
Weather centers only release their
forecasts every six hours.
Because that's all today's computing
power will allow.
But what if that limit didn't exist?
Before we explain that, we'll hear
from the sponsor of this video.
This episode is presented by
Microsoft Copilot for Microsoft
365, your AI assistant at work.
Copilot can help you solve your most
complex problems at work, going
far beyond simple questions and
answers. From getting up to speed
on a teams meeting in seconds
to helping you start a first draft
faster in Word, Copilot for
Microsoft 365 gives
everyone an AI assistant at work
in their most essential apps to
unleash creativity, unlock
productivity, and uplevel skills.
And it's all built on Microsoft's
comprehensive approach to security,
privacy, compliance, and responsible
AI. Microsoft does not influence
the editorial process of our videos,
but they do help make videos like
this possible.
To learn more, you can go to
microsoft.com/copilotforwork.
Now back to our video.
In 2020, a group of researchers
published a data set called Era5
five, which contained about 40 years
of the Earth's hourly weather data.
The data set was just primed for
using AI because it was huge.
The data is nice and smooth,
there's no missing values, and
it's free just to step in.
And say, "Hey, you have terabytes of data?
I can learn how weather moves."
AI models learned how weather moves,
not through applying trillions of
physics equations to the Earth's
atmosphere, but by being trained
on Era5's enormous historical
data set.
Researchers gave the models a
snapshot of weather conditions,
ask them to make a prediction, and
then scored them on how closely that
prediction matched what really
happened.
After a while, the models eventually
got really good at this.
By 2023, the tech companies
Google, Huawei and Nvidia
had developed models that rivaled
traditional forecasting on variables
like surface temperature, humidity
and wind speed, and on some extreme
weather events like the paths of
tropical cyclones, atmospheric
rivers, and extreme temperatures.
These AI models still rely on the
same observation data from the big
weather centers, the data that
creates that initial 3D grid
snapshot.
But they don't require anything
close to six hours to produce a
prediction. Huawei's PanguWeather
model, for example, can produce a
week long forecast in 1.4 seconds.
Which means that we spent over a century
figuring out the physics, the
atmospheric science and the
computational skill to bring us our
modern day weather forecast.
And now suddenly we have these AI
models that have come out of, you
know, the past 2 or 3 years,
and they're getting the same skill,
and now they run on a
modest laptop.
Despite the impressive results from
these first AI models, there's still
lots of work to do.
Google's GraphCast predicted
Hurricane Lee's path faster than
traditional models, but it didn't
prove it could predict a hurricane's
intensity, which is a trickier
calculation to make.
These AI models are incentivized to
get as many correct answers as they
can through the scoring system.
If you swing for it, swing for the
fences, right?
If it misses, the model is
penalized very large. It says "No, you
should never do that.
Don't swing for the fences because
the error is going to be huge."
But by prioritizing safer,
correct answers to boost the model's
score, it could miss rare,
outlier weather events.
Plus, they are learning from 40
years of history, and historical
weather has fewer extreme events
than we do today or will have in
the future due to climate change.
But a big reason for optimism with
these AI models comes from their
ensemble forecasting.
Instead of the traditional 50
ensemble forecasts, they can predict
a thousand or more because they're
freed from computing and time
constraints.
There's always going to be
uncertainty in a weather prediction,
but larger ensembles will help us
measure that uncertainty better.
That's an extremely useful context
to say you are a emergency manager
down in Florida who's dealing with
the very difficult decision:
Are you going to, you know, order an
evacuation or not?
You want as much information about
the uncertainty as possible.
Large ensembles might also catch
a rare weather event that a 50-member ensemble would miss.
Or, measure the probability of
weather events even further into the
future than our 10 day forecast.
Are we magically going to get a
crystal ball that lets us foresee
perfectly into the future?
Probably not.
I think especially on like sub
seasonal timescales, like multiple
months out, we're going to be able
to frame the statistical question
with a lot more specificity
and probably a much better
quantification of the uncertainty.
We do have a new winter storm
warning.
One thing we shouldn't
expect to change any time soon is
the role of the meteorologist...
at least the ones you see on TV.
If only because we fundamentally
have to communicate uncertainty, and
we have to walk through all the
various what-ifs.
And a human is the best tool that we
have today to effectively
communicate that and help somebody
else make a decision.
AI forecasting models are still in
an experimental phase, but the
European Centre for Medium-Range
Weather Forecasts has started
publishing AI forecast alongside
their traditional ones for the
public to compare.
When we check the weather in the
very near future, it might be
powered by AI instead of physics
based models or a combination
of the two.
And if we get things right, we'll
have a sharper view of the weather
events that we need to prepare for
the most.