Hey guys, in this video I'm going to do another time series topic and it's going to
be on determining the stationarity, causality, and invertibility of any ARMA(p,q) stochastic
process. An ARMA(p,q) is a stochastic process that has an autoregressive and a moving average part.
The p corresponds to the autoregressive part and the q corresponds to the moving average part. And
there's an easy way, or a straightforward way, of determining for any general ARMA(p,q) for any p
and any q whether these stochastic processes are stationary, causal, or invertible based on the
formula. We're going to be given a stochastic process and depending on how it's written,
we're going to write it in this general form where we have phi, or 'fee', however you pronounce it,
B times the stochastic process Xt is equal to theta B Zt, where Zt is going to be white
noise with mean mu and variance sigma squared and the B in this formula refers to what's
called the back shift operator. And the back shift operator works as follows. You have B to
any integer j times Xt and that's going to be equal to Xt minus a certain lag j. For example,
if you have just B to the 1 times Xt, that's equal to Xt minus 1. And you can also apply
the backward shift operator to a white noise Zt, so it's going to be the same thing. B to the j,
where j is any integer, times Zt is going to be equal to Zt minus j. And once we write
our ARMA(p,q) process in that general form, what we're going to do is plug in Z for the
backward shift operators. We're going to have phi Z times Xt is equal to theta Z times Zt
where Z--the Z in these two functions--is going to be any complex number. It can be a complex
number or a real number, so it's going to be X plus Yi. And the way we determine stationarity,
causality, and invertibility will be based on how we solve for--so basically this will give
you a characteristic polynomial for phi Z and theta Z and then we solve for Z and, depending
on the solutions of the roots of Z, that will tell us whether we have an invertible, causal,
or stationary process. The rules for determining these three properties is: for stationarity, the
phi Z roots are not on the unit circle, and I'll go into more detail what that means in a second;
for a causality, the phi roots are outside the unit circle; and for invertibility, the theta
roots are outside the unit circle. And as far as what we mean by the unit circle, that's going to
be any value that has a length of radius one. If we imagine that this is the X-axis and this is 1,
-1 and this is a Y-axis with value 1, -1 and if we draw a circle where every value--pretend that's
a circle--where every value has a radius of one, that's a unit circle. And so we just have to check
that, for stationarity, the values do not equal one or are not on the unit circle; for causality,
the values are outside the unit circle, so greater than 1 or greater than these radii; and for
invertibility, it's the same thing but checking the roots for theta instead of phi. And so if you
have a real number, you just have to check that that number is greater than 1 or less than -1,
but if you have a complex number and I'll show you an example of a value that is a complex number,
we're going to check the values in between the major axes. Okay, so I have two examples and
the first example is a simple, straightforward example and the second one is a little bit more
involved. Let me start with example one. This is just an AR model. This is an AR(1), which
is equivalent to writing that as a ARMA(1, 0). So q is equal to 0 in this case. And the stochastic
process is Xt is equal to 0.9 Xt minus 1 plus Zt. The first thing we want to do is we want to write
the stochastic process in this general format. We want it to be phi backward shift operator,
so a function of phi times Xt is equal to a function of theta times Zt. We're going to
move the Xt's to the left hand side and keep the Zt's on the right hand side. That's going to be Xt
minus 0.9 Xt minus 1 is equal to Zt. And then we want to write Xt times a function of the backward
shift operators. You don't need a backward shift operator for Xt. That's just going to be 1 minus
0.9 B times Xt. If we distributed this, this would be Xt minus 0.9 B times Xt and that would give you
this line here. And then this is just Zt. We don't have to make any changes to the right hand side.
And now the next step is we're going to plug in Z for the backward shift operators. We're going
to write out our characteristic functions for phi and theta. Phi Z would be equal to 1 minus
0.9 Z and theta Z is just going to be equal to 1. And now for the next step, we have to check
the stationerity, causality, and invertibility based on determining the roots of Z. First we can
check for invertibility and the rule for that is that the theta roots are outside the unit circle.
But here we don't even have any roots to check because theta Z is just equal to 1 and there's
no Z. Right away we know that this stochastic process is invertible. This is a check mark,
by the way I'm left-handed so it looks weird. This was easy, we're done with that first part. And
then we have to solve for Z, find the roots of Z, to check for stationarity and causality using phi.
We're going to set this equal to 0 to find the roots. That's going to be 1 minus 0.9 Z is equal
to 0. That's going to be 1 is equal to 0.9 Z and Z is going to be 1 divided by 0.9. One divided by
0.9 is going to be greater than one and it's also not equal to one. Since it's greater than 1, we
have causality and since it's not equal to 1, we all so have stationarity. We've effectively proved
that this stochastic process, this AR(1) model is invertible, causal, and stationary based on these
properties. That was a simple example because it was just an AR(1) model, which is pretty simple.
Now let me go to the second example, which is a little bit more involved. For example two, we
have an ARMA(2,1) model and it is the following: Xt minus 0.75 Xt minus 1 plus 0.5625 Xt minus 2 is
equal to Zt plus 1.25 Zt minus 1. Again the same thing as before, the first thing we want to do is
get this in terms of the backward shift operator, a function of backward shift operators times Xt on
the left hand side and function of backward shift operators times Zt on the right hand side. That's
going to be equal to 1 minus 0.75 B plus 0.5625 B squared times Zt, and that's the left hand side.
And the right hand side is going to be 1 plus 1.25 B times Zt. Now that we're done with that step,
the next step is to get the phi Z characteristic polynomial and the theta Z characteristic
polynomials. That's going to be 1 minus 0.75 Z plus 0.625 Z squared for phi Z, and for theta Z,
that's going to be 1 plus 1.25Z. The first thing I did was I checked the theta Z roots,
since that's a little bit simpler than checking the phi Z roots. I'm first going to check for
invertibility. That's going to be 1 plus 1.25 Z is equal to 0 and we're solving for Z. That's going
to be 1.25 Z is equal to -1 and then Z is going to be equal to -0.8, if we divide -1 by 1.25. We have
Z is equal to -0.8, which is greater than -1. If we think about the unit circle again, this point
is going to be within the unit circle because it's greater than -1. This is not invertible. Next, I'm
going to check the phi Z roots for stationarity and causality. To do that, I have to solve for
Z here, and this is just a quadratic equation. We have 0.625 Z squared minus 0.75 Z plus 1 is
equal to 0. I'm going to solve for that using the general quadratic formula. That's going to be 0.75
plus or minus the square root of 0.5625 minus 4 times 1 times 0.625. And that's all over 2 times
0.625. If you do the math on this, this is going to give you 2/3 plus or minus 2 times the square
root of 3 i divided by 3. As you can see when you do the quadratic formula and you simplify,
this is a complex number because it has a real part and an imaginary part. This is going to be
not on the major axes. It's going to be somewhere in between. We have to determine the length of
this Z. We basically have to find the length or the radius to determine whether it's greater than
the unit circle or within the unit circle. To do that we're going to find the length of Z, or you
can think of this as the L2 norm and the formula for that is just the distance equation. It's going
to be the square root of X squared plus Y squared. You can think of this as X and this part as Y. If
we do that, that's going to be equal to the square root of 2/3 squared plus 2 times square root of
3 divided by 3 squared. And if you do that math, that's going to be 4/3. Because 4/3 is not equal
to 1, we have stationarity and because 4/3 is greater than 1, it's outside the unit circle,
and we again have causality. And so for this stochastic process we have that this ARMA(2,1)
is not invertible, it's stationary, and it's causal. And we're done with this problem.
*chiptune music*