statistics is all about data in fact data are the life and blood of
statistics I would argue that without data in fact there is no statistics
when it comes to data it comes in all shapes and sizes and if you have been
paying attention to what is happening and what has been happening in the world
as of late especially around since 2010 and 11 now you will notice that the real
buzz is around big data and corporations governments individuals they all want to
do something with big data and they don't want to be seen without big data
in fact a colleague and I wrote a paper in the International Journal of
Information Management and we critically looked at this concepts of big data and
we called it beyond behind because we realized that it was more hype to big
data than what was in fact possible with it in fact I would argue
that most companies and individuals would have what we call small data that's
something that sits on your hard drive or through the hard drive of your
computer or something that you can work with with a small server and if you
notice a lot of data that we work with in real life not just at universities
and schools is small to medium-size data and thinking of data you have to realize
how to it's collected and how its generated a large number of data sets are
collected through surveys and you have been recipient of such unsolicited phone
calls or people stopping you on the street asking you can use for your five
minutes for a survey there are cross-sectional data in fact what
happens is that from a random sample of individuals representing some population
we conduct a survey and ask them a question at a point in time and in
fact we get a cross-section of opinions at that one particular time and another
way of collecting data is to ask the same question to the same people
repeatedly over over a longer period such as tracking children
from the age of say five into their adolescence and all the way to
their adulthood so a 30 year period you test their IQ scores their
behavioural attitudes their attitudes towards life the choices they made for
education and careers and family and that kind of data would be the
longitudinal or panel data another kind of data that you are most likely to be
familiar with is called time series data if you look at housing prices a graph of
housing prices going up and down or unemployment rate fluctuating over time
these are these are called time series data you're collecting this information
about the same entity over time what what is called the big data is
essentially the web-based data or data that has been being continuously
generated by smart forms or information and technology enabled devices that keep
generating these this data automatically and and in the size of that data is
humongous and that's why it's called big data the data set once it is in a
tabular format and that would be the focus of this this statistics 101 that
data has variables and those variables would have certain types and we'll
discuss them later since business analytics and social
sciences rely on to a great deal on survey data let us discuss it briefly if
you have been paying attention to the news or to television you know that
the United Stated states you have the presidential elections and what happened
is that for a very sustained period of time the polls that is the surveys
done of of eligible voters suggested that one candidate would win over the
other the odds of one candidate was significantly higher as for the survey
data but then it turned out at the night of election election night people
realize that evening that the polls were horribly wrong so survey data is very
useful but I would start with the caveat that the quality of data determines the
quality of analysis and while I'm mentioning these poles
went wrong I would like to mention Warren litovsky and who is one of the
pioneers of exit polling data and as we go along I hope that we could get a
chance to revisit some of his work but in the end of the day you have many
companies individuals who collect data even the government collects data
through polls and the United States Census Bureau
not only does census which essentially is a survey of all individuals over
supposed to be a survey of all individuals in the US and same is the
case with the Census Bureaus and all agencies in Europe and in other parts
and then there are professional organizations that do service for
businesses and governments and Gallup is one such agency and then they will think
tanks of fact tanks such as the Pew Research Center that collect data on
matters of that are of importance sociologically socially or politically
or economically and one good thing about Pew Research Center is that it makes its
data available to all in this particular course we will routinely use a data set
from University of Texas provided by professor Daniel Hammermesh professor
Hammermesh is an author and it celebrated economists who have been
looking over the years as to how your ones appearances or looks impact your
economic or other outcomes history book beauty page is an interesting read and I
highly recommend it the data set looks as follows we have
Beauty which is the rating of the instructors physical appearance by a
panel of six students and then teaching evaluation which is the lowest could be
one and highest could be five notice that beauty and evaluation are
continuous variables of real numbers so evaluation could be 4.1 4.2 they are not
integers whereas minority status is sort of a factor variable or a categorical
variable it's either 1 or 0 if a person is a is a
non Caucasian or visible minority 1 0 otherwise then gender would also be a
categorical variable or a dichotomous that could take only one or the other
value so gender native tenure are all dichotomous variables so we will discuss
these types as we go along and as we work with data but understanding the
type of variable and the kind of analysis one can perform based on the
type of variable is very important to get right from the word go