YUFENG GUO: Natural Language processing is not an easy task.
And when it comes to domain-specific language,
things get even trickier.
Standard pre-trained benchmark models
aren't always useful in industry-specific applications.
So how can we train a text classification system
on custom data?
Stay tuned to find out.
Welcome to AI Adventures, where we explore the art, science,
and tools of machine learning.
My name is Yufeng Guo.
And on this episode, we're going to look
at how to use AutoML natural language
to build a custom text classifier,
using a dataset from Stack Overflow.
Stack Overflow is an incredibly useful tool.
And it has saved me on many occasions.
But sometimes it gets tedious tagging questions.
And I wish it would automatically
generate or at least suggest some tags when
posting a question.
So today, we will try to build a machine learning model
for recommending tags for Stack Overflow questions
about machine learning.
To get started, we're going to need some training data.
I have spoken about the BigQuery datasets program before,
and it comes to the rescue here again.
With over 25 gigabytes of Stack Overflow questions here,
I think we've got enough material to work with.
Using AutoML Natural Language is quite convenient for our use
case because I don't necessarily know
what I want to build initially.
It would be great to be able to easily experiment
with a couple of different options.
So to start out, let's just try building
a simple classification model that just differentiates
between whether a question is asking about TensorFlow
or other, something else.
The SQL query we're going to use to pre-process this data,
it's a little bit involved.
We'll combine the title and the text fields together
into one long string of text, strip out
all that HTML formatting tag nonsense.
We don't need that.
And we'll replace those vertical bars in the tags with commas.
So we'll limit our questions to only
be those that are tagged with TensorFlow, Keras, Matplotlib,
Pandas, and Scikit Learn.
That way it's just machine learning topics.
So at this point, you might be wondering if we shouldn't just
make a tag recommendation system by writing,
say, an IF statement.
In some cases, you'd be right.
But there's definitely still questions out there
which don't mention the word TensorFlow, even though they
are about TensorFlow.
So to even things out and prevent the model
from using the word TensorFlow to basically cheat and find
those questions about TensorFlow,
let's replace all of the obvious words with, let's say,
the word avocado.
For your work model, you can choose a different food.
Now that our Stack Overflow questions are all
about avocado datasets and avocado classifiers,
we can use AutoML Natural Language
to train a model that decides whether a question is
about TensorFlow or something else.
Training AutoML Natural Language is very straightforward.
And you just click Train.
It seems to have yielded good results for us.
And we can test it with a question from outside
of our training set.
Here's one about the .net framework.
And the model seems to correctly recognize that it is not
about TensorFlow.
Great.
So now we can look at something more useful
than just TensorFlow or not, labeling the questions
with up to five different tags all about those different
machine learning libraries.
We can use our dataset, just as before,
and replace the library names with avocado.
And we'll need to update the labels so that instead
of just TensorFlow and other, we have all five libraries
represented.
And we can see in AutoML Natural Language
that the datasets are roughly balanced.
And once we wait for the training to complete,
we actually have quite some good results from the training.
I was very happy with this.
AutoML Natural Language shows not just the training scores,
but it also shows false positives and false negatives
for each of those five classes.
So this helps us spot potential errors in our data or errors
in the model.
We can try a couple of different sample questions
and paste them into the Predict tab of the interface
and see if those tags and the scores are reasonable.
And in our case, they seem to have turned out pretty well.
AutoML Natural Language allowed us to quickly and easily test
out some ideas about our data with different ways of labeling
and training that model.
We didn't have to deal with tokenization,
word representation models, or embeddings.
All we needed to do was have an understanding
of our dataset and the task we were hoping to achieve,
and then see if the system could make a model that
was suitable for our use case.
This meant that we could focus on the problem we are trying
to solve, tagging Stack Overflow questions,
rather than figuring out how to model text.
Now that we know it's possible to build this model
and that the data is there, we can weigh the trade off
between building a custom model ourselves
versus just using the model that we already made
from AutoML Natural Language.
Thanks for watching this episode of Cloud AI Adventures.
And if you enjoyed it, click that Like button
and be sure to subscribe to get all the latest episodes right
when they come out.
For now, check out the expanded blog post
I have linked below in the description for more details
and examples.
And be sure to check out AutoML Natural Language
and see what interesting models you can build.
[MUSIC PLAYING]