Welcome everybody, you're watching mr fugu data science. Today we're going to do some
exploratory data analysis, panda plotting and some multi-indexed tables. I decided to do this
video because, i've noticed lately i've had some people who are very new to python and i thought
this would be something very useful for them. As always feel free to hit me up here on instagram
and twitter and if you like to help support this channel feel free right here. Let's get into
this. You want to follow along, the data came from Kaggle right here. You could download the data set
or go on my Github, as usual where i post all my code. It will be in the link in description below.
Here's your imports. Let's evaluate this data real quick, and see what we're actually dealing with.
This is some sales data and some of the columns that we're going to be interested in . First
will be sales subcategory category, as well as the postal information, the state and then the segment
that you're going to see here as well as shipping mode which is this as well all right. Let's zoom
in just a little bit, so we'll notice that we want to create some plotting from pandas directly.
While these aren't publication quality plots; we will be able to see a few things. So let's block
out some of this real quick so you can see what's going on here. What i did was i took two columns
of interest which were your shipping mode and your sales and i decided to do a group by your shipping
mode. And then some along all of the sales relating to each one of those modes and then i
sorted the values in descending order. Let's look at this real fast so here's what we're dealing
with in descending order for our sales standard shipping, second class first class and same day.
You'll notice the standard shipping turned out to be the highest sales because, it's the cheapest.
Now how would you actually enter inside of this table? Let's look at this real quick if you've
never done this before. Let's find out what the columns are the columns are sales, well then what
is the other information that we're dealing with it's called an index. And the index is all of our
row indexes. What we're just looking at. Let's get into the plotting of this, so let's scroll
back up let's undo all of this and plot it out and see what it looks like. Before we scroll down, let
me explain what's going on. I decided to just do a regular plot, increase the font size, the figure
size and rotate the labels or the excuse me the x tick values by 50 degrees. Then i set an x label,
i create a grid pattern which you're going to see the y label, and then the title for
the plot and then i just print it off again, the table information so you can see it. Here's our
table and here's our plotting let's look at this a little better. So here's our plotting which is
our sales on the y versus each one of the modes of shipping okay. You'll see that the standard
shipping has the highest sales and then it goes down from there for the more expensive shipping.
From there we'll look at doing a separate table but, we're going to set it up a little bit
differently with this plot. So instead we're going to have two columns with our index of shipping
mode. This is pretty cool because, now we're able to see how much money they make for each shipping
type, and then how many people actually use this. Like how many packages accounted for standard
class shipping, how many counted for second class shipping, etc. We did this with the group by,
relative to our shipping mode. Then we're calling our sales and running an aggregate on this. An
aggregate function allows us to do two separate operations here. Then we just sorted these values
in descending order again. Ran a plot, and i didn't have any parameters inside of here. And
then i just printed off the table and i didn't put anything inside of here. To show you next
we scroll down and i didn't play with any of the functionality here on purpose. Because this count
right here while it would have been useful if it was on a different scaling you know. Since all the
values were fairly similar it just looks like a straight line. This would be relevant if you had
some data lower on a different scale so you would have been able to plot two different variables at
the same time with respect to whatever you're having on your x-axis. This is pretty useful.
Then what we'll do is something a little similar but, we're going to use the aggregate.
And basically take the count which is this length. And we're going to unnest everything. Ao what does
this even look like if we're doing this with a table? Let's even look at it without that.
Here's what this table; nested what we're looking at by the segment type before a
corporation or is it for a home office. And then what shipping type and how many items
were shipped with respect to consumer for first, second, same day and standard class shipping.
Now we'll do our plotting. See what this looks like, where i had to rotate the labels and i did
a bar plot. If you notice, we do a subplot equals true. Scroll down and we'll see that we're going
to have four plots which is pretty cool. Because, it's with respect to each of the three, each of
the three types consumer corporate and office. And within each one of those you have your standard,
second class, same day, first class shipping. And you'll notice that each one of the y-axis are
consistent with the count values for each one of those classes. The next thing we can do is
a bar chart allowing us to do the same thing with respect to the bars here. And you're going
to see something interesting. This is a pretty simple plot, you could do the same thing. But,
set it up as a bar chart. That's what's going on with this and all we did was change up the labels.
Here you go with this. This is a little bit more readable where we have on the y-axis the quantity
sold. And then we scroll down and we can see that we're grouping by consumer corporate and office
type. And for each one of those we have with respect to the first class, same day, second date
and standard shipping. This is very useful and intuitive to interpret. So here we're doing our
subcategory sales in US dollars, let's make this a little more legible for us. Where we're taking our
data subsetting within two particular columns that we want, and running a group by function and
summing each one of the categories by their sales, sorting in descending order as usual, in running
up our chart. Then we just chuck in here. All of our label information what we're doing which
is fairly simple. These are all pretty much the same kind of idea when you're plotting this. We'll
scroll down and look at on the y-axis. We have our sales information in descending order relative to
each sub-category for our sales information. We can use a cross-tab which is interesting because,
you don't need a data frame. You can use an array or a list instead. And you have the option of
normalizing your data as well which is different than a pivot table. I decided to substitute my
data and arbitrarily convert it into a numpy array instead of a data frame or a pandas series so you
can see what i'm talking about. Let's look at this real quick so just here's what our data looks like
for our subcategory names. Let's run our crosstab real quick and you'll see that for your indexes.
We have all of our subcategories here, and then going along this column. We have the actual count
number that this particular person purchased zero accessories. They purchased one piece of artwork,
one book case. This person purchased one piece of accessories, one piece of artwork, etc. This is a
useful little table. The next thing that we can work with is nesting. Further to show our sales
information by count, relative to the customer and the subcategory that we just showed above. Doing
it this way with the group by. Instead, we need to take these particular columns with all rows and do
a group by relative to the person's name and the category. Here, which are our index values and our
column which is going to be our sales. But this is going to be a count instead which is this. it's
doing the same thing. Just set up differently. We could scroll down perform another cross tab
where this is going to have an index of our customer name which is here. And then we're taking
our columns which are our subcategories and then we're taking the values from the sales. And that's
going to be these right here. Then i'm using the sum for each of the sales. You could do the mean,
you could do whatever you want there. You have a lot of functionality. So we're still using the
people. For each person, they bought a piece of art which was almost 18. And remember from above
they only had one item. The next thing we could do, we could look at another crosstab example
to get a little more functionality. This one's relatively simple, where we're only looking at
after we subset Florida. Then we'll do a crosstab where the city is the index value and the category
or subcategory for our items is going to be our index. So we scroll down and we can see that in
the city of Apopka. We have one item sold in Boca Raton. We have one binder in Apopka. We have three
binders, etc. Let's do something so we could have some plotting with a bar chart and we're going to
take the cross tab for the cities in Florida and look at these sub categories here. And
we're going to do a stacked equals true, you'll see why in a second. This is just plug and chug.
And we're doing a bar chart. So we scroll down. i got to zoom out, and we're going to see we have
all of our categories along this legend. We have on the y-axis all of our quantities sold and then
on the x tick marks we have each city in Florida, there's like 36 i think. You'll be able to see
what was sold within each one of those cities, quantity wise pivot tables. But if you set them
up right you could have a pivot table form the same layout as a cross tab. this case i want the
city state and sales subset from our data. Then i'm going to do a pivot table where the index
position is our state and our city. And the actual values are the sales we're going to take the mean
for those index of state followed by city, and the mean sales for that city this is a pretty useful
option that you have. We could then do a little bit of fun and take our Florida data once again
and play around with this. Let's do another plot. First we're going to do a pivot table where we're
taking the city and the subcategory as our index. Sales is going to be the columns and then we're
going to use the mean of those. So in the city of Apopka they sold 671, and a half dollars
on average for appliances. That's how you would interpret this. Now let's do some plotting with
these data. We're going to take the same thing that we just did above. We're going to plot it.
But we're going to sort the values in descending order for the sales, do a bar chart rotate the
labels play with the sizes of the figure and the font and chuck in our label information here. And
this is what you end up with. We have our sales on the y-axis for every single county in descending
order for our bar chart. That'll be the conclusion of this video. I hope i brought utility to
someone. But, please like share and subscribe and if you subscribe turn on that notification bell.
Feel free to hit me up right here on instagram and twitter. And if you would like help support
this channel on buy me a coffee. there's my handle. i'll see you in the next video. Bye