This is a rather theoretical exercise that has two purposes.
One is to verify that the notion of convergence in probability
is quite natural and that it has properties
similar to the notion of convergence of sequences.
And the second purpose is to get a little bit of practice
with the formal definition of convergence in probability.
So what is the statement saying?
It says that if we have a sequence of random variables
that converges to a certain number, a,
and this basically means that when n is large,
the distribution is highly concentrated around a.
And if we have another sequence of random variables
that converges to a certain number,
b, which means that the probability distribution of Yn
is heavily concentrated around b.
In that case, then the probability distribution
of the sum of the two random variables
is heavily concentrated in the vicinity of a plus b.
So what are we saying?
If Xn is very close to a with high probability
and Yn is very close to b with high probability,
then the sum will also be close to a plus b
with high probability.
This is the intuitive content of the statement.
Now we want to establish this formally.
Before establishing this statement,
however, it will be a good practice
to verify a property of this type
for the ordinary convergence of sequences of numbers, not
random variables.
So let us do that.
What we want to show is that if a sequence of numbers, an,
converges to some number a, and another sequence converges
to some number b, we want to show that
in that case, an plus bn converges to the sum of a plus
b, and we want to do this formally.
So let us start with the definition of convergence.
What does it mean that an converges to a?
It means that if I fix some positive epsilon,
then there exists some number or some time
such that if we consider some n bigger than n0,
then an is close to a in the sense
that this difference is less than epsilon.
Now this is true for any positive epsilon,
so if instead of epsilon, I take epsilon over 2,
this would also be true.
Eventually, after some time, we will have the property
that an minus a is less than epsilon over 2.
Similarly, if bn converges to b, then
we will have the property that there exists some time-- let's
call it n0 prime-- such that if n is bigger
than that particular time, then bn minus b is going
to be less than epsilon over 2.
So after time n0 and after time n0 prime,
these two inequalities will be true.
So if we wait long enough so that both of these inequalities
are true, that is, if n is bigger
than the maximum of n0 and n0 prime,
then we will have the following.
We will have that an plus bn minus a minus b
which, by an elementary inequality,
is less than or equal to an minus a plus bn minus b.
Where is this inequality coming from?
This is a general inequality about absolute values.
If I give you two numbers, the absolute value of x plus y
is always less than or equal to the sum of the absolute values.
So we're using this inequality where
x is an minus a and y is bn minus b.
So we have this inequality, but when time is big enough,
an minus a is less than epsilon over 2.
bn minus b is also less than epsilon over 2.
And putting everything together, this is epsilon.
So what have we shown?
That if an converges to a and bn converges
to b, so that all these relations hold,
then if time n is large enough, then
the difference between this number and that number
is going to be less than epsilon.
And this is true for every positive epsilon,
but that's just the definition of convergence
of this quantity to that quantity.
And this is the proof of this elementary relation
about convergence of numbers.
Now let us turn to convergence of random variables.
We fix some epsilon that's positive.
In order to show convergence in probability,
we want to look at this difference
and look at the probability that this difference is
bigger than epsilon in magnitude.
And we want to show that this quantity converges to 0.
If it does, then we will have established
convergence in probability because that's just
the definition.
Now, this is the event that the sum of the random variables
is close to a plus b, and we want
to use the fact that xn is close to a and yn is close to b.
So this is the event that-- let's write it
in a somewhat different way-- is the probability of the event
that xn minus a plus yn minus b is
bigger than epsilon in magnitude.
Now, for a sum of two numbers to be bigger
than epsilon in magnitude, it has
to be the case that either one of them
is larger than epsilon over 2 or the other number
is bigger in magnitude than epsilon over 2.
So if this event happens, this event must also happen.
This means that this event is a subset of this event.
This is a smaller one.
If this happens, then this one happens.
So since it's a smaller event, it
means that its probability is less than
or equal to the probability of that event.
Now we use the union bound.
The probability that something happens
or something else is happening is less than
or equal to the sum of their probabilities.
And now, since Xn converges to a in probability,
then by definition, we know that this quantity converges to 0
as n goes to infinity.
Similarly, since Yn converges to b in probability,
this quantity converges to 0 as n goes to infinity.
This is a sequence of numbers that converges to 0.
This is another sequence of numbers that converges to 0.
Therefore, the sum of these two sequences also converges to 0.
In essence, here we're applying what we established earlier
about convergence of numbers.
If a sequence converges to 0 and another sequence converges
to 0, then the sum of these sequences
also converges to 0 as n goes to infinity.
But this is exactly what we need to show in order
to establish convergence in probability of Xn plus Yn.
We have shown that if I fix any epsilon, positive,
no matter how small, the probability
that I am more than epsilon away,
the probability that Xn plus Yn is more than epsilon away
from the supposed target or the limit,
this probability must go to 0.
And that's exactly what we established here,
and this completes the derivation.