what is going on everybody and welcome
to another deep learning with Python
tensorflow and Cara's tutorial video in
this video we're gonna continue building
on our a future crypto currency price
movement predictor recurrent neural
network okay so where we left off we've
got this pre-processing happening we've
built the sequential data and we've
separated out our validation data we
have we normalized the data here we
scaled the data here the next thing we
need to do is balance the data we need
to have as many byes as we have cells if
you have any more like if you're like
52% buys and 50 want 52% buys and let's
say 48 percent cells let's make sure it
adds up to a hundred if you're if you
have data like that it's probably ok you
probably need to worry about balancing
it but if you've got anything more than
like a 60/40 split or even more
egregious it's really important you
balance your data set and arguably it's
just always gonna be better if you just
balance it because the the model is
gonna learn probably pretty quickly if
there's like a 510 percent Delta there
that oh just always break this one class
and immediately we make huge improvement
and that's the easiest change that the
model can make so it's important that
you have a perfect split and imbalance
of your data so that the model doesn't
waste time doing that and then also kind
of get stuck in a rut like it's gonna
start doing that and then from there it
might stay stuck and lost and so you
don't want that to happen now there is a
way you can pass class waits to care us
and tell it like hey one example of this
is worth 1.5 examples of this or
something like that and you can tell it
hey wait these a little different when
you go to calculate the loss for
mistakes made and stuff but to be honest
I've not found that to solve balanced
issues so I think that's a great idea in
theory it just doesn't seem to work so
anyway
balance your data so so that's what
we're gonna do now we've shelled the
data and now what we want to do is
actually
you know balance it so what we're gonna
do is and again there's probably better
way to do this but I'm just gonna have
two lists here we're gonna say buys as a
list and sells is a list clear that out
we're gonna zoom in a little bit okay
then we're gonna say is for sequence
targets in sequential data if Tarija is
a zero that's a cell so then what we're
gonna say is cells not append sequence
target Elif target equals a1 bys append
sequence and target okay um so now we
have buys and sells the next thing we
want to do is before we start slicing
things and dicing things we're gonna go
ahead and do a random shuffle buys and
dot shuffle cells we probably don't
really need to do that because we've
already shuffled here but shuffle for
good measure
so so then what we're gonna find out is
which one is lesser we don't actually
need to know if it's by ourselves we
just need to know what's the minimum
value of these two lists so the way we
can do that is lower equals the minimum
of the Len of buys or the Len of cells
and that'll tell us you know like maybe
by cz's 30,000 in cells a 60,000 well
lower would be 30,000 then what we can
say is buys equals buys up to or I'm
sorry let's see uh how do we do this up
yeah up to lower so if the Len is 30k it
would say up to 30,000 buys and then
sells equals sells up to lower I'm gonna
remove 30k because that's not true I
don't think it's just a value okay
okay so now we've got that so then we're
gonna say sequential data now
actually is just equal to byes plus the
cells and then random dot shuffle
sequential data why are we doing that
shuffle again at least this one is so
like the data isn't all byes than all
cells cuz that'll really confuse the
model you definitely want to shuffle it
up okay so now we've got that what we
need to do is now it's in sequential
data we need to convert it from you know
it's bison cells is containing the
guitar you know the features and the
labels what we want to do now is split
those out to be you know the X's and Y's
as diff you know as different lists I
guess because the way that we're gonna
feed it you know it's going to be a
model dot fit XY so we have to split
these out into X's and Y's so we're
gonna say x equals a list y equals a
list and then we're gonna iterate over
the sequential data so full sequence
targets in sequential data we're gonna
say capital X can we type dot append the
sequence and then Y dot append the
targets okay at this point I think we're
closing in on being complete we're going
to return the number array of x and y so
our pre-processing data frame function
should be complete and now we're ready
to do is come on down here and comment
out that uncomment those get rid of this
and I think what I want to do is let's
print out some statistics I'm not sure I
really want to type all this out I'm not
going to I'm gonna copy and paste this
from the text-based version of this
tutorial this is just wastes Aria but my
dog so long I think our trash people are
here and they're making a bunch of noise
oh no we got a huge
I have no idea what's going on someone's
putting a ladder up to my roof I
probably should be somewhat concerned
okay it's so the so what we can see is
the training data is 69,000 validation
is 3000 there's a good balance between
the buys and the don't buys in that we
have a balance between the validation
buys and don't buys so everything is
nicely balanced we've got it split apart
the way that we're expecting to and in
the next tutorial I think we are ready
to build the model and train the model
so that's what we're going to do in the
next video if you've got questions
comments concerns whatever you know the
deal feel free to leave them below
otherwise I will see you guys in the
next video