YUFENG GUO: Getting high quality data
is paramount for good machine learning outcomes.
But public data sets aren't always
suitable for custom use cases.
So how can you get your data labeled
efficiently without spending days in front of the computer?
Stay tuned to find out.
[MUSIC PLAYING]
Welcome to "AI Adventures," where
we explore the art, science, and tools of machine learning.
My name is Yufeng Guo, and on this episode,
we're going to take a look at the cloud AI platform
data labeling service and see how
it can help you create labeled data
sets that you can use to train machine learning models.
While you should always try to collect data in such a way
that the labels are known at the time of collection,
that isn't always possible.
Especially in the case of, say, crowdsourced data,
or unstructured data like images and video or text,
labeling or data after collecting it
might be necessary.
The data labeling service lets you work with human labelers
to create labels for data that you can then
use to train machine learning models.
Let's learn a little bit more about how this system works.
Say you've collected a representative data
set that you want to classify or analyze,
and you're ready to try out creating a data labeling task.
What does it take?
Let's dive in.
The data labeling service has three core resources
that it uses to define a labeling task--
data sets, labels sets, and instructions.
You'll need all three in order to create
a labeling task for your data.
Now let's look at each of these in turn.
The data set resource is, of course, your data set.
It's represented as a CSV file on Google Cloud Storage.
Each row of the CSV file points to an individual element
of data.
For instance, if you a bunch of images,
each row is a GCS path to an image file.
For text data, each row should be a path to a text file.
In addition to the CSV file, you'll
also need to give the data set a name
and select the data type-- images, video, or text.
Once you've created your data set,
you'll need to create your label set.
A label set is the set of labels you want your human labelers
to use to label your images.
Each label set can contain up to 100 labels.
You can mix and match data sets with different label sets
depending on what model you're trying to train,
or you might even use the same label
set for several different data sets.
Try to avoid using labels whose meanings might overlap,
as this can confuse labelers.
It is also worth noting that you can
have a label called neither and both,
which might be useful especially in, say,
classification scenario.
Once you have a data set and corresponding label
set created, it's time to create some instructions
for your labelers.
Here's some tips for making great instructions.
First up, remember that labels don't have your domain
knowledge so make sure your instructions are easy enough
to understand.
Secondly, be sure to include a list
of all your possible labels with a description of each label.
And third, in your instructions, include
examples for every single label, making
sure to give at least three positive examples and at least
one negative example covering different kinds of cases.
Fourth, clarify as many edge cases as you can.
For example, with say bounding boxes and images,
be sure to state whether a partially covered item should
be included and how tightly that downing box needs to be drawn.
The system requires that you submit PDF instructions.
So be sure to have some way to output to PDF.
There is lots of ways to do this.
One option that I would recommend
is to just write it up in Google slides or Google Docs
and save as PDF.
Now that we've created a data set, label set,
and instructions, we're ready to bring them all together
to create a labeling task.
The data labeling service supports many different types
of labeling tasks.
For image data, you can do six different types of labeling.
There is classification, bounding boxes,
oriented bounding box, bounding polygon, polyline tasks,
or segmentation tasks.
For video data, the data label and service
supports classification, object detection, object tracking,
and events.
Now, finally, for text labeling, the service
supports classification, classification with sentiment,
and finally, entity extraction.
I want to close this video with some tips for success
and common questions about the data labeling service.
There is always a concern about how to ensure
high labeling quality.
One way to ensure the best labeling quality possible
is to request multiple human labelers to annotate
each piece of your data.
In cases where there is disagreement on labeling,
additional opinions from the other labelers
are gathered until there is a consensus
or the maximum number of labelers have been reached.
Let's take a look at an example.
Say we chose to have three labels.
For image classification tasks, all three labelers
will classify each image, and the majority vote
will decide the final answer.
For image bounding box tasks, the first labeler
will draw the box and the second labeler then can verify them.
If the second labeler disagrees and makes edits,
continue on to the third one to get a majority opinion.
Another potential area of questions
is around who will label my data.
All data sets are labeled by officially onboarded
subprocessors, and you can find out
more information and all the details
at the GCP data processing and security terms.
Anytime you're transmitting data,
the security and protection of that data is paramount.
It's worth noting that all data stored
in Google Cloud is already encrypted by default
both in transit and at rest.
Human labelers can only view your data during labeling.
The data labeling service is also HIPAA compliant,
so for those of you working with sensitive health care data,
rest assured that it will be handled in a proper manner.
One final tip, ramp up your data labeling jobs incrementally.
Start your first labeling job with a small amount of data
then see whether the results are what you expect.
You can then revise your instructions
and then create subsequent jobs and iterate
until you feel comfortable sending
larger and larger quantities of data to be labeled.
This will help you get high quality results
and make the best use of your budget,
both in terms of time and money.
I hope this was a useful overview of the data labeling
service and how you can get going with it.
And even if you don't need to use a data labeling service,
these tenants of label creation and instructions
are important for creating accurate data
sets for your machine learning training.
As always, I've linked a number of useful resources
down in the description below.
Thanks for watching this episode of Cloud
"AI Adventures," and if you enjoyed it,
click that Like button and be sure to subscribe
to get all the latest episodes right when they come out.
For now, check out the data labeling service
and create your first labeling task today.
I look forward to seeing you next time on "AI Adventures."
[MUSIC PLAYING]