cover bi-directional RNN in this video if you have not
seen my video on simple RNN which is single direction RNN then
I suggest you what that because it's kind of a prerequisite.
In this video we'll just discuss theory behind bi-directional RNN in the future
video will cover coding say you are working on a named entity
recognization problem and you want to figure out the apple
word that we're using is what kind of entity is it so here
apple is fruit and Dhaval is a person but when
you look at a different sentence like this
apple here is a company it is not a fruit
now if you observe the statement carefully
the words which come prior to Apple they are same so until you look at the
sentence after apple you won't know if apple is a
fruit or a company now if you look at your simple rnn which
is a single direction okay left to right then
the word apple will have influence only the previous words which
is Dhaval loves now people have a common confusion
when they look at the the graph like this
the picture like this they think that you know it's a three and three six
seven layer neural network but no look at this axis. X-axis is a time axis
so this is something you have to keep always in
your mind that the actual network is just this
it's a simple network. But we feed word by word
when we get a sentence we feed every word one by one to this network
and then it kind of has a looping effect okay so when you unroll that in time
you get a network like this. So that thing you need to keep in your mind all
the time that this is a time access at time this is a time axis. at time t1 i am feeding
word Dhaval getting my activation a1 then at time t2
when I feed this loves actually i'm feeding it to the same network
same same layer okay so the actual network is this but since it has a
looping effect I unroll this in time and then
I get this picture now simple RNN okay so i'm just
gonna show you the simple presentation so I'm removing all those neurons and
everything and this rectangle box that you see is
like a cell so if its a simple RNN, its a simple RNN cell.
if it is a LSTM or GRU its a you know LSTM cell or GRU cell
single direction RNN then the output for word apple will be influenced by
all these edges that you see in this different color
so Dhaval loves apple so these these are the words that influence the
final outcome but we already saw that in order to
decide whether apple is a fruit or a company
you need to look the at the words which are coming after
apple in the future. So you need to make a
change in your neural network for this. So one
one way you can do this is you add another layer which processes
the words from right to left. So think about this layer
okay so I already have this the previous layer
layer you know the forward basically single direction left to right a layer. I
add a new layer which is in blue boxes here
so it's a same kind of RNN cells but the only difference is the
activation is going from right to left okay so now what happens
is
when I decide whether apple is a fruit or a company
see the output y3 hat that output is a function of
a1 and a2 okay so basically you have the influence of
Dhaval loves the previous words on your y3 hat
but be watch this blue line but because of this line
now you have influence of your future words as well
so it keeps him healthy so that sentence will have an impact
through this direction you know through the direction that goes from right to
left you get all the activation results and
these results feed through this particular arrow
and you can make a decision so that's all I had for
bi-directional RNN we just covered a basic theory I did not go into math
a little bit you can maybe check other resources for it
but I hope this video gives you a simple you know simplistic understanding
of how bi-directional RNN works. In the future video we are going to
cover the vectorization or word embedding
techniques and then we will get into coding. I know
in this series so far we have been doing some theoretical videos.
You all have been waiting for coding we'll get to that
but before we get to the coding we need to cover some essential concepts.
So we'll get into word embeddings and after that
we will write code to implement LSTM, GRU or you know bi-directional
RNN to solve some NLP problems. all right so
thank you very much. Goodbye.