- So in this next video,
what I'd like to do
is think about
the other dimension of uncertainty propagation.
So in the past two videos,
we focused on analytical methods
for propagating uncertainty into forecasts.
I'd next like to switch
to thinking about the numerical methods
we use for propagating uncertainty into forecasts.
So the key conceptual idea here,
is that one of the challenges we face
in analytical methods is called Jensen's Inequality,
the idea that we put the mean of our inputs into a function,
that that does not give the same prediction
as the mean of the function evaluating
over a full probability distribution.
So you can't transform summary statistics
through models to get the summary statistics
of the predictions.
But what you can do
that doesn't violate any statistical principles
is you can always take samples from distributions,
which then give us exact numbers
and then transform those exact numbers
into specific predictions.
So if I have a distribution for a parameter,
I can take a sample from that parameter distribution,
and I can run the model under that specific parameter value.
Or I might have a specific distribution
of possible initial conditions
that I can sample from that distribution
and run the model under that specific initial condition,
or it can run under combinations of parameters
and his conditions and co variates and,
and what not.
So one of the things that we rely on a lot
in a lot of the numerical techniques we use
for calibrating models,
such as Basie,
and MC MC approaches
is this idea that we can approximate
a probability distribution
with samples from that distribution.
And so for example,
when we do our model calibration,
in a basic sense,
usually what we get is not a written down distribution
for the parameters,
but samples from that parameter distribution
that we can then summarize as a histogram,
and then calculate means and variances and quantiles from.
We're going to rely on that same principle
when we do the propagation of uncertainty in the forecasts.
If we have samples from our inputs,
and we can transform those samples,
will we get out our sample of our prediction.
So the key idea here is we're gonna
represent the uncertainty in our forecast
in terms of the samples of the predictions.
One of the real strengths of this approach,
which is known as Monte Carlo Uncertainty Propagation,
it deals with all the non linearities in our models
and all the interactions between different things
in a way that is valid and justifiable,
and also one that is very robust.
And the other real advantage
that's a fairly easy algorithm to implements.
We have an algorithm that can be implemented very easily.
We can just sample things and run models repeatedly.
We can handle,
some fairly complicated uncertainty transformations,
while maintaining complex interactions
and complex covariance structures.
The other advantage is that what comes out of
that uncertainty propagation can then seamlessly be fed
into later analysis.
So if I take uncertainties about my inputs
and propagate them into an ecological forecast
and get out this sample distribution,
that someone downstream such as the decision maker
decision analyst can feed
that sample of ecological forecasts
into a later say economic analysis or policy analysis,
where they propagate that uncertainty further.
So to summarize the pros and cons
of this Monte Carlo approach to Uncertainty Propagation.
The major advantages
is that it gives us the full probability distribution.
It captures all the covariances
and captures the complex structures
and interactions among parameters and processes.
The main disadvantage is that we may require thousands
to tens of thousands of model runs
to get a good approximation
of our predictive probability distribution.
So the computational cost
for full Monte Carlo Error Propagation,
can be prohibitive for more complex models,
or for when we're dealing with bigger data.
That then brings me to my last approach
to error propagation that I would like to talk about.
Which is, if we want to rely on the Monte Carlo approach
to propagate uncertainties,
but,
we can't afford the full computational cost
of running models of thousands of times,
is there a way to approximate
that with a smaller number of model runs.
This approach to uncertainty propagation
is usually referred to as an Ensemble Error Propagation.
So the main difference between Ensemble
and a Monte Carlo method
is just that Ensemble usually deals
with smaller numbers of runs.
So they may be tens to hundreds of models runs.
And they may,
again propagate over multiple sources of uncertainty
such as alternative model structures,
alternative model inputs,
alternative model parameters,
When you're only sampling tens to maybe hundreds of inputs,
and do tens to hundreds of models runs
if you make a histogram of those predictions,
the histogram itself is not an adequate description
of a probability distribution.
So if I take,
if I sample ten random numbers from a normal distribution,
the histogram of those ten numbers is not sufficient
to calculate the quantities such as the confidence interval,
or specific quantiles that I might want.
It doesn't really characterize that district shape
that distribution well.
But what I can do is to make a simplifying assumption,
which would be I may assume
that those samples
fit a specific named probability distribution.
So for example,
I may look at that histogram of predictions and say,
I can't work with those samples directly
to represent the distribution
but they appear to be approximately normal.
So for example,
if a predictive distribution looks approximately normal,
you could use the sample mean and variance.
So parameterize,
a normal distribution using the same Ensemble samples.
Then when we do any further analysis,
for example,
calculating a confidence interval,
we will do that using that assumed normal distribution.
So to summarize the Ensemble methods,
we're trying to reduce that computational cost
of the Monte Carlo method by making additional assumptions.
So the cost is additional assumptions
about specific probability distributions
that we're going to fit our Ensemble to.
The benefit is that we can
get away with a smaller number of Ensembles.
So for example,
if I need to get a solid stable estimate
of a mean and a covariance,
I may be able again be able
to get away with that which is tens to hundreds
of Ensemble members.