In probability theory and statistics, the term Markov property refers to the memoryless
property of a stochastic process.
It is named after the Russian mathematician Andrey Markov.
A stochastic process has the Markov property if the conditional probability distribution
of future states of the process depends only upon the present state, not on the sequence
of events that preceded it.
A process with this property is called a Markov process.
The term strong Markov property is similar to the Markov property, except that the meaning
of "present" is defined in terms of a random variable known as a stopping time.
Both the terms "Markov property" and "strong Markov property" have been used in connection
with a particular "memoryless" property of the exponential distribution.
The term Markov assumption is used to describe a model where the Markov property is assumed
to hold, such as a hidden Markov model.
A Markov random field extends this property to two or more dimensions or to random variables
defined for an interconnected network of items.
An example of a model for such a field is the Ising model.
A discrete-time stochastic process satisfying the Markov property is known as a Markov chain.
Introduction A stochastic process has the Markov property
if the conditional probability distribution of future states of the process depends only
upon the present state; that is, given the present, the future does not depend on the
past.
A process with this property is said to be Markovian or a Markov process.
The most famous Markov process is a Markov chain.
Brownian motion is another well-known Markov process.
History
Definition Let be a probability space with a filtration
, for some index set ; and let be a measurable space.
A -valued stochastic process adapted to the filtration is said to possess the Markov property
if, for each and each with ,
In the case where is a discrete set with the discrete sigma algebra and , this can be reformulated
as follows: .
Alternative formulations Alternatively, the Markov property can be
formulated as follows.
for all and bounded and measurable.
Strong Markov property Let .
Suppose that is a stochastic process on a probability space with natural filtration
. Then is said to have the strong Markov property if, for each stopping time , conditioned on
the event , we have that for each , is independent of given .
The strong Markov property implies the ordinary Markov property, since by taking the stopping
time , the ordinary Markov property can be deduced.
Examples Assume that an urn contains two red balls
and one green ball.
One ball was drawn yesterday, one ball was drawn today, and the final ball will be drawn
tomorrow.
All of the draws are "without replacement".
Suppose you know that today's ball was red, but you have no information about yesterday's
ball.
The chance that tomorrow's ball will be red is 1/2.
That's because the only two remaining outcomes for this random experiment are:
On the other hand, if you know that both today and yesterday's balls were red, then you are
guaranteed to get a green ball tomorrow.
This discrepancy shows that the probability distribution for tomorrow's color depends
not only on the present value, but is also affected by information about the past.
This stochastic process of observed colors doesn't have the Markov property.
Using the same experiment above, if sampling "without replacement" is changed to sampling
"with replacement," the process of observed colors will have the Markov property.
An application of the Markov property in a generalized form is in Markov chain Monte
Carlo computations in the context of Bayesian statistics.
See also Markov chain
Markov blanket Markov decision process
Causal Markov condition Markov model
Chapman–Kolmogorov equation References