YUFENG GUO: Data is messy.
It's often unbalanced, mislabeled,
and sprinkled with wacky values that throw off your analysis
and machine learning training.
The first step to getting your dataset cleaned up
is understanding where it needs cleaning.
Today, I've got just the tool for the job.
Understanding your data is the first step
to getting your dataset cleaned up for machine learning.
But this can be tricky to do, especially
in any sort of generalizable manner.
An open source project from Google Research
called Facets can help us visualize our data
and slice it in all sorts of manners
which can help us begin to see how our dataset is laid out.
By allowing us to find where our data doesn't quite
look the way we're expecting, Facets helps reduce mishaps
down the road.
Let's see what Facets looks like.
The team has a demo page on the web
so you can try out Facets from Chrome
without installing anything.
Moreover, Facets visualizations use Polymer Web Components
backed by a TypeScript code so you can easily
embed this in web pages and into Jupyter notebooks.
There are two parts to Facets--
Facets Overview and Facets Deep Dive.
Let's take a look at each one now in more detail.
Facets Overview gives-- well, an overview of your dataset.
In previous videos, we saw how tools like pandas
can help us get a sense of how our dataset is distributed.
We can get a slightly upgraded view
of this type of information using the Facets Overview tool.
Facets Overview splits the columns of our data
into rows of salient information,
showing stuff such as percentage missing, min and max values,
as well as stats like mean, median, and standard deviation.
It also has a column that shows the percentage of values that
are zeros, which totally helps with catching cases where
most of your values are zeros.
You can also see these distributions on your test
dataset as well as your training set for each feature.
This way, you can double-check that the test
set has a similar distribution to the training dataset.
Yes, indeed, technically this is best
practice to do at least this level of analysis on your own
anyway, but I have definitely forgotten
to check all of these aspects of every column of my data.
This tool will help ensure that you
don't miss this crucial step, and it
highlights some abnormalities.
Now let's take a look at Facets Deep Dive.
This is where things get really interesting.
It allows you to have even more clarity into your dataset
and zoom all the way in to see an individual piece of data.
You can "facet" the data by row and column
across any of the features of your dataset.
This will look familiar to views when
you're shopping online for, say, shoes, and filter
by different categories, such as size, brand, and color.
Let's take a look at a concrete example of Deep Dive in action.
The interface is divided up into four main sections.
The main area in the center is a zoomable display of your data.
On the left-hand panel is where you
can change the arrangement of your data
with various dropdowns to control
things like faceting, positioning, and color.
Directly below that is a legend for the center display,
and on the far right is a detailed view
of a specific row of data.
You can click on any dot of data in the center visualization
to see a detailed readout of that particular data point.
Now let's see how this all comes together.
To do this, we'll use the "Census Dataset,"
a classic dataset extracted from the 1994 US census
by Barry Becker.
Its target is to predict whether a household's annual is
above or below $50,000, based on various census statistics.
We'll first split the data by age range and color
the data points based on the target value.
Here, blue means less than 50K, and red means greater than 50K.
Now we can look at the breakdown, by age,
of another feature of the data--
say, do different number of hours per week
yield different results across age ranges?
Let's find out by faceting columns by hours per week.
We can see that there is a larger chunk of the age 17-26
population working 15 to 29 hours per week, which may just
be the result of kids doing summer jobs, let's say.
And we can also see the trend of fewer and fewer people working
29 to 43 hours per week as they get older, while the 43 to 57
hour segment stays relatively constant in the middle years
of the chart.
But this still doesn't quite show us
what we are looking for.
So let's try changing the positioning of the plot
and getting a more detailed look.
We'll switch the positioning to "Scatter"
and only facet by age.
I'm also going to select "Hours per Week"
as our vertical sort order to make
it easy to see the working hours across different age groups.
Now we can see that indeed, the hours per week rise
in the middle of the chart and lower on either side.
You should continue exploring this data
and see what trends and relationships you can find.
For instance, you might facet by country
of origin, which would show that the data is heavily skewed,
and this might tell you that you need
to collect more data points to have a more balanced dataset.
If this was interesting, you're probably
wondering how to get your dataset loaded into Facets.
Well, here you have two options.
You can either use the web interface and just upload
the data directly and play with it in the browser,
or you can install the library as a Jupyter notebook extension
using the instructions on the project's GitHub page, which
I'll link below.
Facets is a useful tool for peering into your data
and seeing the relationships between different features,
as well as ensuring that there aren't missing
or unexpected values in your dataset.
Thanks for watching this episode of Cloud AI Adventures.
Be sure to subscribe to the channel
to catch future episodes right when they come out.
Now go out there and facet your data,
and see how you can improve your machine learning.