When you are cleaning your raw data, you will want to check for outliers...
particularly multivariate outlives because they can really mess up your
analysis. Finding an outlier in a single variable is relatively easy, but how can we
identify if we have a problem-child who is an outlier among multiple variables?
We are going to dive deeper into how to identify multivariate outliers using a
Mahalanobis distance test in SPSS.
in this episode, we're going to talk about finding outliers, which outliers
matter, and what you should do about them. Let's begin with a review of outliers:
If we are measuring height, then our group will have an average height. We will,
of course, have someone who is a little taller than the others. That is not an
outlier. That is someone with a legitimate score in the upper tail of
the distribution. Not everyone in the tails of a distribution is an outlier.
On the other hand, when we measure height, we may have someone whose height is
dramatically taller than the others. That person is an outlier. Their score lies
outside the range of the other scores. Not all outliers are a problem, then
again, some of them are. What we know for sure is that when you have multivariate
outliers in your data set, you are going to have problems. A multivariate outlier
is a person who scores are outliers, not on just one, but on multiple variables.
It's like this: we all have friends who are a little bit odd... and we love them
for it. Being a little bit unusual, eccentric, idiosyncratic, or being out of
the ordinary in one way can be charming. Because "Outliers are people, too!"
Just because someone is an outlier does not meant that you must remove them from your
data set. They may be a legitimate score who's just odd on one variable. So if our
exceptionally tall person scores similarly to others on blood pressure,
anxiety, and a personality test, he is only a univariate outlier. And if
our research only concerns measures of blood pressure and anxiety, then being an
outlier on height is unrelated to our study, and we could just ignore it.
Certainly, we would leave that person in the data set.
But if our exceptionally tall person is also hypertensive, highly anxious, and has
an antisocial personality disorder, he is a multivariate outlier. He's not like the
others in the data set and his presence in the data may distort other findings.
It is best to remove multivariate outliers from the data set. Here is how
to remember this: if you have a friend (or a data point) who is a little bit crazy
in one way, probably leave them in the data set. But if you have a friend who is
crazy in a LOT of different ways, you may want to rethink having that person as a
friend... and remove that score from the data set. We tend to keep univariate
outliers - because "outliers are people, too" - but we delete multivariate outliers.
The Mahalanobis distance test identifies those subjects who are outliers in a lot
of different ways. When a score is an outlier, that score will lie some
distance away from other scores. In this example of movie franchise earnings, we
see that a few franchises stand out: Star Wars, for example, stands out for having
about the same number of movies as the Friday the 13th franchise, but banking a
lot more money. The Mahalanobis distance test looks at the distance that a given
score lies from all of the other scores and asks what is the probability that
someone is THAT distant from all of the other scores just by chance. And that
answer will help us to decide if we should keep that score in our data set.
For this assignment, we will be working with the dataset Mahalanobis_PH.sav.
And to be clear, the Mahalanobis test does not require a dependent
variable; however, the only way to run a Mahalanobis test in SPSS is by using a
regression analysis, and the only way to run a regression analysis in SPSS is to
specify a dependent variable of some kind. We will completely ignore the
regression output, but we still need to have some kind of continuous variable to
use as a DV. I will use the random ID number as a dependent variable.
if you do not have a unique identifier to use as a dependent variable, you need
to create one. It's easy to do. Go to Transform -> Compute Variable... I am going to
call this variable "DV", even though it's actually numbering the cases in my
dataset. Among the Functions, click All, and the first option is a function
called $Casenum. Double-click the function to move it into the numeric
expression box. Click OK, and that is all that we need to do. We now have a random
ID number that we can use as a dependent variable for our regression equation.
I am going to move it to the top of my variables list. Now we will do the
Mahalanobis test. Go to Analyze -> Regression -> Linear...
we will begin by moving our ID number (or other random continuous variable) into the Dependent:
variable box. Next, we need all of the variables that we will use in our
analysis and that may contain multivariate outliers.
Notice that we have eight scales or eight variables. These scales were each
created by taking the mean of three items that were written to measure that
construct. Those individual items have been removed from this data set for
simplicity. Move our eight variables into the Independent(s): variables box.
Remember that we're going to ignore the regression output, so there is nothing to
set or adjust in the output, but there is one test that we want, and it's found
under the Save button. The Save menu does not add anything to the output, but
instead, it *does something* to the variables, and then saves those results
to your data set. Under Distances, click "Mahalanobis". This will save the
results of the Mahalanobis test to our data. Click Continue and then OK.
The output window pops up but we do not need anything in this output, so we will just
close the window, and we will ignore all of this. Instead, we are going to focus on
the data. In Data View in our SPSS data set, you will notice that we have a new
variable: MAH_1. Those are the Mahalanobis scores that SPSS saved
for us. Each case has its own Mahalanobis score. It tells us the distance between
it and the other scores. Unfortunately, this does not tell us enough. We want to
know the probability that a particular case would be THIS distance, just by
chance. There are two ways that we might go about interpreting Mahalanobis scores.
The first begins with sorting the data. Larger Mahalanobis scores mean greater
distances.(i.e. more extreme outliers), so we want those largest scores at the top of
our dataset. Right-click (or control-click on Mac) on that MAH_1 variable and choose
Sort Descending. This will move the largest scores to the top. To interpret
these scores, we could use a chi-square distribution table. You can usually find
one of these tables in your statistics textbook or I have one posted in a
Google folder for this channel. See the description or click on the link in the
channel artwork.
On this page, we have a chi-square distribution table, just like one you
would find in a stats textbook. But, I'm going to use THIS tab, where we can have
more control over the critical values. There's something that I want to teach
you here. We want to "Determine a chi-square critical value for any
degrees of freedom", so we will begin with our typical .05 alpha level.
But then, what should we use for our degrees of freedom?
Typically, the degrees of freedom for a chi-square is the number of categories
minus one, but for the Mahalanobis scores, the degrees of freedom are the total
number of variables. Not variables minus one; the total number of variables.
In our case, we had eight variables. That means that any Mahalanobis score greater than
15.5 would be a multivariate outlier. But that is for an alpha level of p = .05, or
five times out of 100, which is really a lot of the scores. 5% of our data set
could be outliers. When you are doing data cleaning - and especially when you
were considering whether to throw out data - you should not use .05
as your cutoff. In fact, I wouldn't even use .01. I want scores that
are REALLY out there... really different from the others, so I use a cut-off of .001
Now, Mahalanobis scores greater than 26.124
are multivariate outliers. We can go back to our data set and begin
looking, but I want to show you a shortcut. It is a lot of work dragging
out these tables, and we can get SPSS to do all of this work for us. And now that
you know about the cutoff scores and the degrees of freedom, this will make sense.
We are going to convert these Mahalanobis distance scores into
probabilities. We are going to create a new variable in SPSS.
Go to Transform -> Compute Variable... In the target variable box, type "pMAH_1".
This will stand for the probability of the MAH_1, (a.k.a. the
Mahalanobis score). Under Function group:, select the group CDF & Noncentral CDF
These are the cumulative distribution functions for various distributions.
We want chi-square so select Cdf.Chisq. This will calculate the probability of a
variable which follows the chi-square distribution, such as the Mahalanobis test.
We need two values here: first, we need the variable name, which is "MAH_1"
and then we need the degrees of freedom, which is the number of variables, or 8.
And there's one more thing to do: because the CDF function in SPSS
computes cumulative probability from the left end of the distribution, we actually
want the inverse of this probability. So we will subtract the whole thing from 1
to obtain the probability in the upper tail of the distribution. Click OK. In the
data set, we only have two decimals, so let's go back to the variables - which you
can do by double-clicking on the name of the variable - and we will increase the
decimal points to 4. Now, return to the data - again by double clicking on the
name of the variable - because we previously sorted the Mahalanobis values,
we see the smallest Mahalanobis probabilities are at the top of the
column. Had we not previously sorted, this is where you would right click on the
new variable and Sort Ascending. For this kind of data cleaning, we will use a
cut-off probability of p = .001. Which of these
Mahalanobis values have a probability less than p = .001?
Clearly, this first one is WAY below p = .001, so it is a
multivariate outlier and it should be removed from the analysis. The second one
is exactly .001. Although you could make the case for it not
actually being LESS than .001, because it is EQUAL to .001,
the reality is that this case is going to cause problems, so I removed
it from my data. I figured that I was starting with 146 cases,
so I could afford to lose that one. Now then, we have a case that is
p = .0014; this is not less than .001, so I can make an
argument to leave it in: in fact I did leave it in,
based on this Mahalanobis probability, but it later popped up in other
assumptions testing, so I ended up removing it before the analysis - and
those are the kinds of decisions that we have to make when we do exploratory data
analysis. I would probably make a note and keep an eye on these two cases,
as well. The rest of them, however, look like they will be fine.
When you write up your results for data cleaning you can include a statement
like this: "Data were assessed for multivariate outliers using a
Mahalanobis distance test (Tabachnick &Fidel, 2013). Three multivariate outliers were
identified and removed." - OR - "No multivariate outliers were identified."