Hi! I am Mike Marin and in this video we'll talk about calculating correlation
and covariance using R programming language. Pearson's correlation in statistics
is a parametric measure of the linear association
between two numeric variables. Spearman's rank correlation
is a nonparametric measure of the monotonic
association between two numeric variables. kendall's rank correlation
is another nonparametric measure of association
based on concordance or discordance of x-y pairs.
We will be working with the Lung Capacity data
that was introduced earlier in these series of videos. I've already gone ahead
and imported the data into R statistical software and attached it. we will explore
the relationship between Age and Lung Capacity. We will use the
"cor", "cov" and "cor.test" commands/functions in R.
to access the help menus in R
you can type "help" and in brackets the name of the command you would like help for
or simply place a question mark (?) in front of the command/function.
the first thing that we will want to do is produce a scatterplot
of Lung Capacity versus Age. We can use the "plot" command/function and
on the x-axis we place Age and on the y-axis
Lung Capacity. We can add a title, as well as rotate the values on the y-axis in R.
you can see my series 2 video
on making scatter plots or modifying plots in R to
learn more about how to change the look of this plot. we can see there is a
positive association
between Lung Capacity and Age. we can calculate the correlation
using the "cor" command/function. here we will calculate the correlation between
Age and Lung Capacity. we can set the "method" argument equal to "pearson"
to have Pearson's correlation returned. you can also note
that Pearson is the default so if you leave the "method" argument out of
this command, by default Pearson's correlation will be returned.
you can also note that the order that these variables are entered in R
does not make a difference. the correlation between Age and Lung Capacity
is the same as the correlation between Lung Capacity and Age.
if you'd like to calculate Spearman's correlation in R programming language,
we can set the "method" argument equal to "spearman"
and similarly if you'd like to calculate kendall's rank correlation in R statistical software,
we can set the "method" argument equal to "Kendall". if we'd like
we can have a confidence interval returned in R for the correlation
as well as test the hypothesis that the correlation is equal to 0
using the "cor.test" command/function in R. here we will do this for Pearson's correlation
and we use the "cor.test" command/function. We can see
we're returned the estimate of the correlation. We can also see a 95%
confidence interval for the correlation
as well as the test statistic and p-value
for the test that the correlation is equal to zero. again
we can do the same for Spearman's correlation
we can see that
R statistical software returns a hypothesis test although it does not return
any form of nonparametric confidence interval for the correlation.
we can also see we're given a warning message that R cannot compute an exact
p-value
when there are ties: meaning we have a few people of the exact same age
in this dataset. this isn't really a big deal
but if we like we can use the "exact" argument
and set this to "False" letting R know
to only approximate the p-value for us. now let's go back again to the Pearson's
correlation test we did
as we've seen with other tests in R we can use the
"alt" argument to change the alternative hypothesis. here
we can set it to "greater" to have an alternative hypothesis that the
correlation is greater than 0
by default the alternative will be a two-sided test
we can also use the "conf.level" command
to change the confidence level we're using, for example we may want a
99% confidence interval returned.
while covariance is often of less interest in applied statistics,
we can calculate this using the "cov" command.
here we would like the covariance between "Age" and "Lung Capacity"
we can produce all possible pair-wise plots using the "pairs" command.
here if we ask for "pairs" plot of the LungCap data
this will produce all possible pair-wise plots
you'll notice that a scatterplot is not really appropriate for categorical
variables
you can also notice that the first three variables
in our dataset are the numeric ones. so let's produce a "pairs" plot
only for those variables. here we'll produce a pairs plot for the LungCap data
subsetting taking only columns 1 to 3
to learn more about subsetting using square brackets in R
check out my video in series 1 on subsetting using square brackets.
taking a look at the plots we can see
this plot here is a scatterplot with Lung Capacity
on the x-axis and Age on the y-axis;
this plot here is a scatterplot with
Age on the x-axis and Height on the y-axis;
the "cor" command/function can also be used to produce a correlation matrix
for all the variables. here we can try and calculate a correlation matrix for the
LungCap Data. if we enter this command/function,
you'll notice R returns an error. this is because
R will not calculate a correlation for categorical variables
or factors. like we just did with the pair's plot
we can subset the data in R and make a correlation matrix for only variables
in column 1 to 3 or numeric variables
we can see Pearson's correlation between Age
and Lung Capacity. Here we can see Pearson's correlation
between Height and Age, as before
we can also ask R statistical software for Spearman's correlation using the "method" argument
and setting this
equal to "spearman". If desired
we can also go ahead and produce the covariance matrix in R
using the "cov" command. In the next video in the series
we'll talk about fitting a simple linear regression using R.
Thanks for watching this video and make sure to subscribe to marinstatslectures for more videos on R programming and statistics.