in deep learning you should always aim to start your neural network training with a pre trained
model and fine-tune it you really don't want to be starting with random weights because it means that
you're starting with a model that doesn't know how to do anything at all with pre training you
can use 1000 times less data than starting from scratch however sometimes a pre trained model
that is appropriate for your case doesn't exist so we would like something which works better or
at least as similar results but doesn't need the huge amount of Anna today to pre train models use
the secret for this purpose is self supervised learning this is where we train a model using
labels that are naturally part of the input data rather than requiring separate external labels
self supervised learning is a relatively recent learning technique where the training data is
autonomously or automatically labelled it is still supervised learning but the datasets do
not need to be manually labeled by a human but they can for example be labeled by finding and
exploiting the relations or correlations between different input signals that is input coming from
different sensor modalities a natural advantage and consequence of self supervised learning
is that it can more easily be performed in an online fashion given the data can be gathered
and labeled without human intervention where models can be updated or trained
completely from scratch therefore self supervised learning should also be well suited for changing
environments day to end in general situations as Yan Lacan puts it one of the godfathers of
AI reinforcement learning is like the cherry on a cake supervised learning is the icing and self
supervised learning is the cake as he says in self supervised learning the system learns to
predict part of its input from other parts of it input self supervised learning is supervised
learning because its goal is to learn a function from pairs of inputs and labelled outputs like
supervised learning self supervised learning has use cases in regression and classification
self supervised learning is like unsupervised learning because the system learns without using
explicitly provided labels it is different from unsupervised learning because we are
not learning the inherent structure of data self supervised learning unlike unsupervised learning
is not centered around clustering and grouping dimensionality reduction recommendation engines
density estimation or anomaly detection using self supervised learning machines can predict
through natural evolution and consequences of its actions similar to how newborns can learn
incredible amounts of information in their first weeks months of life by observing and
being and curious self supervised learning has the potential to scale learning to levels required by
new use cases including but not limited to use cases in medicine autonomous striving robotics
language understanding and image recognition self supervised learning has found success in multiple
fields for example in estimating relative seen depths without human supervision by using motion
segmentation techniques to determine relative depth from geometric constraints between the
scenes motion field and camera motion in medicine it has found use cases robotic surgery and in
dense depth estimation in monocular endoscopy in autonomous striving self supervised learning is
useful for estimating the roughness of the terrain when off-roading in short self supervised learning
brings us closer to human-like autonomous learning note that self supervised learning is defined
slightly differently depending on the context or area all the examples and more information about
self supervised learning teach from yang Lacan is linked in the description below please leave
a like if you learn something and subscribe to the channel to not miss any terms clearly explained