So now let's talk about Markov Models.
And my apologies if you already know what Markov Models are,
because we're going to go through this a little bit, then move into HMMs and
maybe that'll be new for you.
And if it's not, you can just write in a postcard telling me that you're unhappy.
Send it to Megan.
All right. We're going to pretend for
the moment that weather, as in the weather outside is a Markov Model.
And if it were,
I'll also pretend that there's only three types of weather you could have.
You could have a sunny day, a rainy day, or a snowy day.
It's kind of like a little kid's book.
And the Markov Model says we have to specify, for
any given one of these, what's the likely next day going to be?
So for example, suppose I tell you that if it's sunny today there's an 80%
chance that it'll be sunny tomorrow,
15 that it'll be rainy, 5% that it'll be snowy.
Likewise if it's rainy,
I may tell you well there's a 60% chance that it'll rain tomorrow,
if it's raining today there's only a 2% chance that it'll be snowy.
So it's a place that gets very cold so, so once it's rainy,
when it's rainy it's not going to snow and vice versa maybe.
And 38% of it will be sunny, and likewise, if it's snowing
there's a 20% chance that it'll keep snowing, 75% chance that it'll be sunny,
and for the same reason before, very unlikely that it'll rain.
This is referred to as a Markovian system.
And in particular, if the only thing you need to know to make a prediction
about what the weather tomorrow will be, is what the weather today is,
that's referred to as a 1st Order Markovian System.
So I don't look at today, and yesterday, and the day before.
I'm just looking at today in order to make a prediction about tomorrow.
If I was looking at today and yesterday, that would be a 2nd order Markovian.
All right?
So the probability of moving, of being in a particular state of getting to
a state, depends only on the state I'm currently in.
In order to specify our Markovian Model we need a couple of things, right?
So, we need a set of states and these are labeled here as S1, 2, through SN.
And those, the S's, those are, there's state one, state two, state, that's not
a particular time, or particular day, there's three different states.
So in this example, N would be three.
We also need transition probabilities, and that's written as AIJ.
So that is the probability that if you're in some state I,
that the next time you would be in state J.
And then finally to kick things off, we need a initial distribution.
That's just the probability that Q1, and I'm going to talk about Q a little bit
more in a minute, that Q1 is equal to Si.
So, Q sub t Is whatever state I'm at, at time t.
So q sub 1 is the time, is the state at the first day, or time one.
So for this particular example, as I showed you here,
you've got three states, sunny, rainy, and snowy.
We've got the state transition probability.
So, this matrix A, that's the encoding of all of these links,
of all of these transition values.
And then, I have an initial distribution which just says well to start off with,
maybe, 70% chance that it's sunny, 25% chance that it's rainy and,
if my math is right 5% chance that it starts out as being snowy.
So, given this Markovian Model, you can ask certain questions.
For example, you could say if I give you this series so
that's sunny, rainy, rainy, rainy, snowy,
snowy, you could say what's the probability of getting that series?
Okay.
Well you have all the information you need.
In fact it's written down here below, I've got A and pi, right.
It's just, the probability that I start out sunny, that's here,
times the probability that it becomes rainy given that it's sunny, so that
would be here, times the probability of rainy given rainy which is there.
All the way through, I just have to chain it through, right?
And when I multiply all those number together, 0.7 times 0.125,
times [INAUDIBLE] I get some number, 0.0001512.
Okay. They tend to be small numbers,
by the way, probabilities and HMMs, or in, in market models,
because whenever I multiply a lot of numbers that are probabilities together,
since all numbers that are probabilities are less than or equal to one,
when I multiply a lot of them, I get small numbers.
And when you implement these things, you sometimes have to be careful and
do things like take the log but that's all I'm going to say about that.
All right, so that's a Markov Model.