You know what is the most difficult part of any machine learning project? It is
collecting the training data set. Self-supervised learning is a technique
that allows you to generate the label data set from unlabeled data
set. In order to understand that we need to
understand supervised and unsupervised learning first and then we'll look into
what is self-supervised learning. Let's say you're working on house price
prediction problem. Here the square foot of the home bedroom edge
etc are called independent variable and you
have a dependent variable which is price. Price is something you are trying to
predict, but in the training data set you have
the prices available for this training data and that those prices are called
basically your target variable or label variable
whereas the square foot bedroom image are called x
or independent variable. If you are working on a
image classification data set you have images of cats and dogs
and the label that you put that this is a cat image this is a dog image
is also called target variable or label and
annotating these images with these labels is expensive task
you have to have a human workforce you know that working on annotation it is
expensive. I unsupervised learning for example
here I have this data set where I have in income and
age of people I'm trying to group them so we use
k min clustering to form different groups here
and we can do certain predictive analysis or certain
descriptive analytics based on these groups.
This is called.. k means here you don't have any label data set
everything he that you have here is independent variable.
Now you might have seen in gmail when you type any sentence gmail will try to
auto complete it. In NLP or natural language processing world this is called
sentence autocomplete task and if you have to
build a model for such a sentence auto complete task
where would you get your training data set? Because this is a supervised
learning problem. You have your label I am not interested
and machine learning model is generating at this time. One thing you can do is go
to Wikipedia collect all the articles let's say I
have this article for elon musk from this
which is an unlabeled data set I can generate
a labelled data set where I have a statement Elon Reeve Musk is an
and the autocomplete for that will be enterpreneur.
So from unlabeled data set you are generating a label data set you have x
and y and you are saving time on annotation
this is called self supervised learning and this is the
technique used by popular NLP models such as word
to Vec and BERT. What they do in these models
is they they go through a lot of Wikipedia articles or
google news or books and they collect all these paragraphs
and they generate these kind of labeled pairs. You know pairs of training samples
and here the task is autocompleting the sentence
but there could be a task you know where you want to let's say
maybe in a BERT they mask these words let's say you mask enterpreneur Tesla
etc and then the model will try to predict those
missing words. Basically so you could be solving a real NLP problem which is a
sentence autocomplete or you are solving a different problem
but by working on this fake problem the word
to Vec and BERT models will generate sentence or word embeddings. If you don't
know what word embeddings are I have some separate videos which you
can check but the summary of this session is these
NLP models such as virtualbackboard they use this self-supervised technique
and self-supervised learning is nothing but a process of generating
label data set from unlabeled dataset. You basically generate this label
dataset and then you do learning that's called
self supervised learning. I hope that video gave you a quick
understanding of the concept. If you have any question
post a comment below.