Multivariate statistics is a form of statistics encompassing the simultaneous observation
and analysis of more than one outcome variable. The application of multivariate statistics
is multivariate analysis. Multivariate statistics concerns understanding
the different aims and background of each of the different forms of multivariate analysis,
and how they relate to each other. The practical implementation of multivariate statistics
to a particular problem may involve several types of univariate and multivariate analysis
in order to understand the relationships between variables and their relevance to the actual
problem being studied. In addition, multivariate statistics is concerned
with multivariate probability distributions, in terms of both:
how these can be used to represent the distributions of observed data;
how they can be used as part of statistical inference, particularly where several different
quantities are of interest to the same analysis.
Certain types of problem involving multivariate data, for example simple linear regression
and multiple regression, are NOT usually considered as special cases of multivariate statistics
because the analysis is dealt with by considering the conditional distribution of a single outcome
variable given the other variables.
Types of analysis There are many different models, each with
its own type of analysis: Multivariate analysis of variance extends
the analysis of variance to cover cases where there is more than one dependent variable
to be analyzed simultaneously: see also MANCOVA. Multivariate regression analysis attempts
to determine a formula that can describe how elements in a vector of variables respond
simultaneously to changes in others. For linear relations, regression analyses here are based
on forms of the general linear model. Principal components analysis creates a new
set of orthogonal variables that contain the same information as the original set. It rotates
the axes of variation to give a new set of orthogonal axes, ordered so that they summarize
decreasing proportions of the variation. Factor analysis is similar to PCA but allows
the user to extract a specified number of synthetic variables, fewer than the original
set, leaving the remaining unexplained variation as error. The extracted variables are known
as latent variables or factors; each one may be supposed to account for covariation in
a group of observed variables. Canonical correlation analysis finds linear
relationships among two sets of variables; it is the generalised version of bivariate
correlation. Redundancy analysis is similar to canonical
correlation analysis but allows the user to derive a specified number of synthetic variables
from one set of variables that explain as much variance as possible in another set.
It is a multivariate analogue of regression. Correspondence analysis, or reciprocal averaging,
finds a set of synthetic variables that summarise the original set. The underlying model assumes
chi-squared dissimilarities among records. Canonical correspondence analysis for summarising
the joint variation in two sets of variables; combination of correspondence analysis and
multivariate regression analysis. The underlying model assumes chi-squared dissimilarities
among records. Multidimensional scaling comprises various
algorithms to determine a set of synthetic variables that best represent the pairwise
distances between records. The original method is principal coordinates analysis.
Discriminant analysis, or canonical variate analysis, attempts to establish whether a
set of variables can be used to distinguish between two or more groups of cases.
Linear discriminant analysis computes a linear predictor from two sets of normally distributed
data to allow for classification of new observations. Clustering systems assign objects into groups
so that objects from the same cluster are more similar to each other than objects from
different clusters. Recursive partitioning creates a decision
tree that attempts to correctly classify members of the population based on a dichotomous dependent
variable. Artificial neural networks extend regression
and clustering methods to non-linear multivariate models.
Statistical graphics such as tours, parallel coordinate plots, scatterplot matrices can
be used to explore multivariate data. Important probability distributions
There is a set of probability distributions used in multivariate analyses that play a
similar role to the corresponding set of distributions that are used in univariate analysis when
the normal distribution is appropriate to a dataset. These multivariate distributions
are:
Multivariate normal distribution Wishart distribution
Multivariate Student-t distribution.
The Inverse-Wishart distribution is important in Bayesian inference, for example in Bayesian
multivariate linear regression. Additionally, Hotelling's T-squared distribution is a univariate
distribution, generalising Student's t-distribution, that is used in multivariate hypothesis testing.
History Anderson's 1958 textbook, An Introduction
to Multivariate Analysis, educated a generation of theorists and applied statisticians; Anderson's
book emphasizes hypothesis testing via likelihood ratio tests and the properties of power functions:
Admissibility, unbiasedness and monotonicity. Software and tools
There are an enormous number of software packages and other tools for multivariate analysis,
including: High-D
JMP MiniTab
Calc PSPP
R: http:cran.r-project.orgviews/Multivariate.html has details on the packages available for
multivariate data analysis SAS
SciPy for Python SPSS
Stata STATISTICA
TMVA - Toolkit for Multivariate Data Analysis in ROOT
The Unscrambler SmartPLS - Partial Least Square
MATLAB Eviews
See also Estimation of covariance matrices
Covariance mapping Important publications in multivariate analysis
Multivariate testing Structured data analysis
RV coefficient References
^ T.W. Anderson An Introduction to Multivariate Analysis, New York: Wiley ISBN 0471026409;
2e ISBN 0471889873; 3e ISBN 0471360910 ^ Sen, Pranab Kumar; Anderson, T. W.; Arnold,
S. F.; Eaton, M. L.; Giri, N. C.; Gnanadesikan, R.; Kendall, M. G.; Kshirsagar, A. M. et al..
"Review: Contemporary Textbooks on Multivariate Statistical Analysis: A Panoramic Appraisal
and Critique". Journal of the American Statistical Association 81: 560–564. doi:10.2307/2289251.
ISSN 0162-1459. JSTOR 2289251. (Pages 560–561) ^ Schervish, Mark J.. "A Review of Multivariate
Analysis". Statistical Science 2: 396–413. doi:10.12141177013111. ISSN 0883-4237. JSTOR 2245530.
Further reading Johnson, Richard A.; Wichern, Dean W.. Applied
Multivariate Statistical Analysis. Prentice Hall. ISBN 978-0-13-187715-3.
KV Mardia, JT Kent, and JM Bibby. Multivariate Analysis. Academic Press. ISBN 0-12-471252-5.
A. Sen, M. Srivastava, Regression Analysis — Theory, Methods, and Applications, Springer-Verlag,
Berlin, 2011. Cook, Swayne. Interactive Graphics for Data
Analysis. Malakooti, B.. Operations and Production Systems
with Multiple Objectives. John Wiley & Sons. External links
Statnotes: Topics in Multivariate Analysis, by G. David Garson
Mike Palmer: The Ordination Web Page InsightsNow: Makers of ReportsNow, ProfilesNow,
and KnowledgeNow