do you find it hard to quickly summarize
a movie you just watched or a book you
read a few weeks ago sometimes you love
a book and if you can't manage to
remember its content which i often can't
you may end up boring your friend for
talking for an hour describing the many
chapters and important parts while your
friend just want to have a quick and
concise summary this is because doing a
great summary is challenging even for us
but necessary how useful it is to be
able to quickly know what the book is
about before buying it or simply to help
you go through all your emails and
documents in seconds taking the time to
recap and summarize your work is also
very important if you want to be taken
seriously at your job and weights and
biases the sponsor of this video will
help you with that with my favorite
feature weights and biases reports it
helps you create beautiful reports about
your previous runs with your hyper
parameters and any matrix tracked in
seconds it even allows you to create
clear and dynamic plots comparing the
runs your team members can leave
comments right on the report if this
doesn't make you the coolest ml engineer
on your team i don't know what will your
teammates will understand what you are
working on in a split second it will
allow them to help you out if they can
saving valuable time and possible
misunderstandings try out weights and
biases for free with the first link
below and profit from these benefits by
stepping up your engineering game
just like for creating clear reports you
need to have a great understanding of a
book movie or any content you are trying
to summarize to do it well omit all
unnecessary information while keeping
the essential making something as
concise as possible can be really
complicated or even impossible here i
try to explain research in a few minutes
and i often can't manage to make it
shorter than 5 minutes even if it's only
a summary of a 20 page piece it requires
hours of work and fine tuning and now i
may be replaced by an ai that does that
better in milliseconds indeed google
recently announced a new model for
automatically generating summaries using
machine learning released in google docs
that you can already use the model will
try to understand the whole document and
generate a short summary of the piece
something some movie professionals
clearly still can't do the model needs
to achieve two things understand the
text in the document called natural
language understanding and generate
coherent sentences using a natural
language or in other words perform
natural language generation but how can
you achieve that you guessed it with a
lot of data and compute power luckily
enough this is google research they
trained our model to replicate our
thought process for generating summaries
using way too many documents with
manually generated summaries seeing all
these examples the model does like any
good student and ends up being able to
generate relatively good summaries for
similar documents as it has seen during
its training phase you can see why we
need good quality data here the model
will learn from them it may only be as
good as the data that was used for
training it it will be like having a
really bad coach that doesn't know
anything about basketball trying to
teach a new player how could this new
player become any good if the coach
doesn't know anything about the sport
the newcomer's talent won't be optimized
and might be wasted only because of the
poor coaching the challenge comes with
generalizing to new documents
generalizing is something even difficult
for students that only learned how to
perform the given examples but did not
understand how to apply the formulas
it's the same thing here the model faces
difficulties as it cannot remember all
documents and summaries by heart it has
to understand them or at least know
which words to put its attention on in
order to produce a summary that reflects
the document well the latter will most
likely happen as the model doesn't
really understand the document it only
understands how to perform the task
which is unfortunately still far from
human level intelligence but well enough
for such a task i just mentioned
attention well this was not a
coincidence attention may be the most
important concept behind this model
indeed just like gpt3 this new model
also uses the transformer architecture
and attention mechanisms this is where
high computation is required as you know
transformers are big and powerful
networks but most time a bit too big for
fast and efficient tools that need to be
available online in seconds transformers
computation complexity also scales with
the input size which means that the
longer the input the heavier the
computation will be causing big issues
when we want to summarize a whole book
gpt3 works well for small inputs like
question and sharing tasks but the same
architecture won't be able to process
whole books efficiently instead they had
to use some tricks in order to have a
smaller and more efficient model while
keeping high quality results this
optimization was achieved by merging
transformers with rnns or recurrent
neural networks which are two concepts i
explained in previous videos that i
highly recommend watching for a better
understanding both videos are linked in
the description below in short it will
act similar to gpt3 which you should
understand by now from my video about it
but with a smaller version of the model
iterating over and over until the model
finishes the book the transformer part
of the architecture will be responsible
for understanding a small section of the
text and producing an encoded version of
it the rnn will be responsible for
stacking and keeping this knowledge in
memory iterating through the whole book
to end up with the most concise way of
summarizing its content working together
the attention mechanism added to the
recurrent architecture will be able to
go through long documents and find the
most important features to mention in
the summary as any human will do of
course the model is not perfect since
even professional writers aren't perfect
at summarizing their work but the
results are quite impressive and
produced extremely efficiently i'd
strongly recommend trying it for
yourself in google docs to make up your
mind about it
and voila this is how google docs
automatically summarize your documents
with their new machine learning based
model i hope you enjoyed this video if
so please take a second to leave a like
and comment your thoughts on the video
and on this new model will you use it
thank you for watching until the end and
i will see you next week with another
amazing paper
you