>> Chances are, staring at a bunch of numbers isn't
necessarily going to give you a lot of insights.
Sometimes that's what you need
but a lot of times it's nice to have a picture.
The old saying, "A pictures worth a thousand words," I
guess it's worth some level of numbers as well,
and that's what we want to explore
here is how we can lay down some charts
and start to see what's going on with our data.
Now, one big thing that we're going to,
one be on the lookout form,
is some level of correlation in our data that
sometimes we're looking for
something that's relatively high level of just,
"Hey, real quick, does X impact Y?"
Sometimes it's because we want to remove items out
of our training data
because that could wind up skewing our results.
So for example, if we were
looking at something where we were trying to
predict something and we had
both maybe the amount that somebody
spent on a plane ticket and their fare class.
Well, chance are there's going to be pretty tightly correlated.
Because generally speaking, the more money that you spend,
the higher of a class
that you're going to wind up with on the plane.
So sometimes we need to remove that data or sometimes we just
want to go in and explore and see those types of things.
So that's what we're looking for it.
One way that we can do this is through the use of a Scatterplot.
The scatterplot is a really common chart type
where what's going to
happen is a dot is going to be drawn out on your chart,
that's me drawing dots.
A dot is going to be drawn everywhere
on that chart where X meets Y.
So now, I can see based on where all of those dots are,
what the values are,
what the scatter is,
what the full range is of
all the values that are going to be on there?
It's very useful for then checking correlation between columns.
So if you're going to notice that there is that correlation
where the value of one changes along with the value of another,
then we're going to see that inside of our scatterplot.
So if we break this down to a couple of a real-world examples,
you are going to notice that there's a correlation between,
say, car accidents on a particular day and snowfall.
But you're probably not going to notice
much of a correlation between
snowfall and the stock price of Microsoft on any given day.
We can stop here and have
a conversation about correlation and causation and so forth,
that's a little bit beyond what we're looking for.
What we're really looking forward today is just,
"Hey, is there correlation?"
So a couple of scatterplots,
you can see right there.
Correlation, you'll notice that real nice tight band
that's going from the lower left towards the upper right,
and then no correlation.
You'll notice that my values are just simply well,
scattered and so that's
a really nice indicator of what's going on there.
We can use Matplotlib to draw out our graphs.
So if we have a DataFrame,
we're going to be able to use that to draw everything out.
We are going to need the Library.
So we'll pull that in right there and then,
we'll be able to actually draw chart directly from our DataFrame.
So what I want you to notice about this code plot here,
is the fact that,
we are and I'll just go ahead and draw this out here.
So what we're going to do is we're going to load up our CSV file.
We've already seen how to do that.
So not really a whole lot going on there,
and now we're going to plot everything out.
So we're going to indicate, "Hey,
we want a scatterplot.
We're going to indicate what we want
the X and the Y values to be."
So the X is going to be distance and then the Y
is going to be our arrival delay.
You'll notice the color,
you'll notice the title.
Both of those are pretty straightforward.
The one that I want to highlight here is the Alpha.
What the Alpha is going to do,
it's a zero to one base number.
Is it's going to indicate how lightly or how
heavily drawn each one of those dots is going to be.
By default the Alpha is one,
meaning that's going to be a just a completely dark
dot if we lighten that though.
Now what we're going to notice is that a single dot is going to be
a lighter color and if there's like
two dots that are top of one another,
or maybe they're just slightly offset,
then what we're going to notice is
that it'll get a little darker with two dots,
and then three dots, and then four dots, and then five dots.
So that way if I'm looking at data
where I've got a lot of points that are
going to be inside of
my chart and I'm trying to get a better sense of,
"Hey, where are things grouped together?"
I can do that by setting a lower Alpha.
So now, I'm going to see
darker colors where there's more overlap values.
Then we go ahead in call show,
and then what will happen is it will then display everything out.
What I want you to notice here, first of all,
is the fact that there's not a correlation
between distance and arrival delay,
which might have been a little bit surprising.
I figured there would have been some level of correlation,
but we can see that there's
just dots scattered all over the place,
so no real strong correlation there.
The other thing that you're going to notice,
is the fact that we've got just
a lot of values really down towards the bottom.
So there's just not a lot of delay for a lot of
our flights and those outliers
are then just that really faint color there.
So it's a really nice way to get
an overview of how our data looks.
Now we can also go in and
break things down across multiple lines as well.
So we don't necessarily have to go directly from the DataFrame.
Maybe we want to be able to break this down into multiple lines.
Maybe because we want to be able to reuse things,
maybe because we want to make it more readable.
Maybe because we just want
a little bit more power and we just want to be able to
control exactly how our chart is going to be laid out.
So this will actually do almost the exact same thing.
The only difference is the fact that we're going to be using
the Departure Delay and
the Arrival Delay to display everything out and so now,
we can see that nice tight correlation there.
We can already start to see, "Hey,
there's an obvious align that's forming there."
Because we've got our departure delay,
our arrival delay, and obviously
makes sense that there's a tight correlation.
But we also have a little bit of,
I don't know, like a little something going on over here.
So it'd be nice to be able to just slap
down a line right across here.
So that way I can hopefully better visualize,
how all of those numbers are coming together?
So this is actually where we
could use a little bit of linear regression.
That linear regression, isn't necessarily
only used to try and be able to predict values.
Sometimes it can be really
helpful to be able to just have it draw a line.
The name linear there is not just a clever name.
It's all about being able to draw a lines there.
So let's take our data and let's feed that
then into a linear regression model,
and then when we get our value back,
now let's go ahead and snap down that red line.
So this is really that same data that we saw previously.
The only difference is now we run
it through a little bit of linear regression.
So the y-value, that y predicted,
is going to be the prediction of
the arrival delay and
that's what's going to give us that nice red line.
Then we can use that model to then overlay
that on top of our data and see that nice neat little line.
So there one more time is a little bit of code.
You're going to notice that we're going
to draw everything out with
that nice red color and then we're going to
create this scatterplot down below,
where we've got everything laid out in blue.
So the end result is going to wind up being
a nice neat little chart here where we've got
all of those row values and then
the linear regression model laid right on top of that.
Now we can see can a nice and clearly how it is
that our departure delay is impacting our arrival delay?
Let's turn our attention to the code
so that way we can see all of this in action.