what is going on everybody and welcome
to yet another deep learning with Python
tensorflow and Cara's tutorial in this
tutorial we're going to continue with
the last tutorial where we are
attempting to predict future price
movements of a certain cryptocurrency
based on the sequence and the historical
prices and volume of that cryptocurrency
as well as other major crypto currencies
and we're trying to do this with the
recurrent neural network so let's
continue the next thing that we want to
do here is we're gonna import time so
we're going to use that just in a moment
and we need to create a few more
constants here the first one we're gonna
do is a pot so how many efox we want to
train this model for then we're gonna go
with a batch size and we're gonna go
with 64 to start we can tinker with that
later if we wanted and then finally
we're going to go with a name and we're
gonna make this an F string and what you
want is a name that is descriptive of
the model because generally you're gonna
tinker a little bit with the model tweak
it a little bit here and there
rerun it again and then you're gonna do
the same thing again hopefully you don't
have eyes as I do anyway so you want to
have a unique name both for the model
that you save as well as in tensor board
so later you can compare the results of
a bunch of different models and you
don't have to be like I don't know what
model that was so or worse you overwrite
the other models so anyways yeah so
let's come up with a good name so I'm
gonna go with sequence length - sequence
- future period predict - predict - and
then we'll just throw in an int time
that time here and that should be good
enough that'll give us a nice unique
model name so now we're gonna do is
import all the tensorflow stuff that we
need so import tensorflow as TF as TF
stop it sublime from tensor float chaos
dot models we're going to import in
or sequential there we go
ah you're such a hmm
it's a quince dude oh this might be a
hard one to get through from tensor sir
flow dr. oz layers we're going to import
dense drop out LS TM probably also coud
kudi NN LST dude you know I had it right
LST em and then batch batch
normalization all of these we've seen
except for I don't remember if we've
done batch normalization I've apologize
if I've already covered it basically
just normalization but between the
layers so for the same reason you want
to normalize your input data batch
normalization can be useful from layer
to layer because you because really you
can think of each layer if it's going to
another layer that layer its output is
kind of like your it's it's a new input
for a new layer so you would want to if
you could normalize that data so we're
gonna do batch normalization there okay
once we've got that more imports from
tensorflow dr.oz callbacks we want to
import tensor board tents oh my oh why
can't i type right now uh anyway tensor
board in model check point I think B and
tensor board probably needs to be
capitalized it does okay
tensor board we've really already seen
this callback model checkpoint is a
fancy dancing a little callback where
basically you can set various parameters
as to when you want to save certain
checkpoints so I like to use validation
accuracy I think it's a pretty good one
validation loss would be another good
one basically even each time it reaches
a new max you could have you can have
the model actually save that checkpoint
that way you know you can run for a
hundred epochs now probably
in the end of a hundred epochs before
you got there the model overfit and it
kind of went crappy and you could save
every single epoch but this might waste
space or whatever anyway this just makes
a nice way to always save the best one
okay once we have that we're gonna
scroll on down we've got our training
extra Hawaii validation X validation Y
ready to build the model so make some
space here and we're just gonna this is
a lot of this coded stuff we've seen
before so again model is a sequential
model we've seen that one model dot add
and we're gonna add I'm gonna use cout
DNN Isles TM if you're on CPU version of
tensorflow use a regular LS TM it's
gonna be a 128 nodes in this layer and
input shape and something bangs on this
desk and it makes me mad that what is
that noise
I can never find what that noise is
struggle judge me nuts and then
sometimes it like stops happening for
like months and then it comes back
anyway input shape train X oops training
exercise shape 1 : that was about to be
an error that was not going to be fun to
debug
I figure out where we are now so we're
inside the cadena Nell's TM still yes
yes yes
so then return underscore sequences
equals true so you should already know
that what the next layer is gonna be are
we good why is it do it what is it cuz
we don't have a new line this is maybe
Pepe yeah okay all right so then model
that ad and we will throw in a dropouts
and we're gonna do 0.2 then we'll add a
batch normalization so model that add
batch normalization and no parameters
there and then I am going to copy this
and then paste it twice paste and paste
and then we're gonna have a dense layer
so because we're gonna have a dense
layer I'm going to remove the return
sequences
and so now it's a 128 to another 128 and
then to a final 128 now for whatever
reason in my notes I have this as a
dropout of 0.1 so I'm gonna leave that
there but I'm pretty confident 0.2 would
probably work just as good anyway
I love a movie about super 1 and I think
we'll leave everything else as the same
now we're gonna add a dense layer but
not the output layer just a dense layer
model that add yes that one dense 32
activation will be rectified linear
which reminds me if you can't use kudi
and NL STM make sure you throw in some
activations I would do either tan H
because that's what cootie and NL STM is
using or you could just use rectified
linear so we've got that dense layer
cool model dot ad we'll throw in a
dropout 0.2 I really wonder if this was
like a typo or what everybody else is
0.2 except for this layer so anyway and
then finally we need the final dense
layer there's this is a binary choice so
it should be only two options there
activation because it's the output layer
softmax okay now that we've got that
we're ready to specify the optimizer
will go with TF caras optimizers dot
adam with a learning rate of 0.001 20
negative 3 and a DK of 1e negative 6 and
then we'll do the model dot compile and
we'll go with loss is sparse cat
categorical cross and tropi you can also
go with like binary cross entropy next
what we're gonna do is optimizer is the
optimizer that we defined and
and metrics we will go with accuracy
okay cool
the next thing is we need to define our
callback saying I guess we can fix this
make make sublime happy so we have two
callbacks that we want to do that one is
tensor board which we've really already
seen and so that's just going to be a
tensor board object log dirt dr. equals
logs and then we'll do some formatting
here and in my text-based version I
haven't made this an f string I'm trying
to move completely to F strings they do
they are better it's just I'm so used to
you know doing the format but it's so
much nicer to use F strings to be honest
to like build it out and then not have
to remember what order things are in and
then and then later the the worst thing
is like when you build it for the first
time it's really no big deal but then if
you want to like go in and move things
around or like add a bunch more it
really does kind of it's kind of
challenging with with string formatting
whereas with like the f strings just
it's just a breeze okay so that's our
tensor board object the next thing we're
ready to do is do the checkpoint object
so here a file path and to be quite
honest with you this is not my code I
just I basically searched for the check
like an example for the checkpoint
things so this might be from like Karis
documentation I don't even remember
where I found this but basically we'll
do Arnon final obviously they didn't
have this exact name but then it's epoch
and then 0 to D and the reason why I'm
doing this is I kind of want to just
like copy and paste this in so I can
talk about it and not waste a bunch of
time here paste so like I'm not sure how
how you're formatting because obviously
this is this here is a string formatting
object to this file dot model and then
we dot format the file path
but then it's like monitor I guess I
just don't understand how these values
are eventually getting passed into here
with a dot format I don't know anyway
what I want is epoch and validation
accuracy and this code works and again I
don't because obviously you know format
file path it throws in the file path but
then it's like these other things in the
format anyway I don't get it I don't
know how it works but it works anyways
so now we're going to do a history
equals model dot fit we're gonna fit
train X train Y the batch size is just
batch size epochs is going to equal a
pox capitals let's fix this they are
parameters therefore no spaces after
epochs we have the validation validation
underscore data and then in here we have
validation X validation Y and then
finally we have our callbacks that we
want to run so that's a tensor board and
checkpoint ok and then once that's done
I do run a value and evaluate in the
text-based version but really every POC
should be running validation data
anyways so at the end I guess we could
just do a model not save although even
here we're not really sure what the
point is because the model checkpoint is
gonna be saved so I guess we're done at
this point ok save that and now let's go
ahead and run this thing
- three six crypto aren't in tight let's
see if we have any errors faster please
impatient nothing yet
come on okay it starts training already
at 51 ish accuracy the epochs are not
too too slow I I think I'll probably
posit or just chop this out but we can
also watch it as it trains with a tensor
no what have you done it went to save it
in that Oh probably because we don't
have a models directory let's just throw
it in real quick models darn good well
here's a perfect example why you want to
throw in time to your name because
you're gonna fix the bug you're gonna
rerun it and then you're gonna be like
dang it I didn't change the name and
then it's gonna overwrite it in the logs
and now it doesn't matter now there's
two of them by timestamp although
actually why that's oh no it's not the
same earth dude I can't see at all
clearly anyways those look is the same
you know give me a break
there you know three versus eight
anyways I think I need some some alcohol
anyways that would definitely help the
situation okay so back to our tents our
board
tents our board log der equals logs so
why is this - - log der with no
underscore and then this here is oh
shoot yeah yeah this here has an
underscore drives me nuts
because it really always confuses me
every time okay so then we can go to our
browser go to our
pchew could always go to one 27001 : six
thousand six if you don't know but
anyways HPC for me smite
okay so the blue line is is what's oh we
might I might my CPU might just crashed
hopefully this recording isn't borked
what is happening like what is this you
guys can't see it but my entire desktop
just like went black and all my all of
my icons went bonkers anyway hopefully
hopefully still recording
okay so we're currently five epochs and
let me refresh the page hopefully we get
proper scaling here there we go okay so
this is our model and we're actually
doing pretty good validation loss has
fallen for the first five epochs at
least validation accuracy has gone up
again we are seeing what appears to be
in general better validation accuracy
than in sample accuracy which probably
shouldn't be the case because this is
always going up we'll see if that
remains the case but like I said before
a lot of times you will have like a
higher validation accuracy if the model
still has a lot of learn or it's
learning a lot per epoch you might find
yourself having at least initially ass
liar slightly higher validation accuracy
because the validation accuracy is
calculated at the end of the epoch
whereas the accuracy here is for the
entire epoch so the initial bit of the
epoch is kind of weighing it down
anyways accuracy in-sample and
out-of-sample I'm just going to do this
both look pretty good and then loss
looks really great here and validation
accurate or validation loss rather looks
pretty good for the first really like
six epochs and then it looks like it's
turning turning around and then you can
even see validation accuracy is kind of
doing the same thing after about six
epochs it's
it's falling but we had about as high as
50 7.28
percent accuracy which is pretty darn
good so the other thing we probably
should have in our our name here is the
thing that we're predicting so I would
also throw in ratio to predict so now
what we could do is like let's change
this to a theorem and do aetherium USD
and then come over yeah and we run that
hopefully my GPU won't cry too hard
we probably probably should have said
GPU usage to like 75% my mistake let's
see hold on do we have even less data
here oh no no we actually have more data
for this this one so this is for a
theory 'm by watching the other prices
so I already have 51 percent so it's
pretty good okay so here is a theorem
done so a theoria ms our red line here
blue is the light coin so at the end of
validation accuracy is about the same
but for the most part before it strikes
things started to turn around a little
bit below but pretty pretty similar
results validation loss similar they
kind of diverge in a lot of places but
still it's pretty similar trends after
about the sixth epoch things just kind
of go south anyways I am now running
Bitcoin cache we'll see how that does
and then after that I'll run the Bitcoin
and then we can kind of see how
everything relates okay Bitcoin cache is
done and actually it's the best
performing one of all of them in terms
of validation accuracy and loss on in
both that has the least loss and the
highest accuracies pretty cool finally
we're gonna run Bitcoin and see how that
does I'm interested to see Bitcoin now
again I actually took two other results
I probably should have founded this out
before but pretty good results I managed
to keep this all to myself okay and
Bitcoin is done very interesting results
on Bitcoin initially it was just kind of
50 50 50 just 50 percent the whole
it's obviously somewhere around the
fifth epoch it figured something else
something out started dropping and then
obviously validation accuracy started
going up validation law started going
down
although briefly did spiked up so that's
actually interesting that's way better
validation accuracy than I thought
Bitcoin would ever get just because
Bitcoin there's probably just a whole
lot less alpha involved with Bitcoin
nowadays but also I think Bitcoin is
more of the trendsetter but then again
maybe not I mean I think a lot of these
other crypto coins have a lot more of an
established presence than just relying
on the price of Bitcoin so anyways
pretty cool results that we were able to
seemingly predict forward movement to a
pretty good degree I mean I kind of draw
a threshold around 60% for like a pretty
impressive classifier when it comes to
predicting finance and in this case this
is predicting just simply an upward or
downward movement so I think you could
probably tweak this model I mean I
didn't spend too long on the model all I
did was I just wanted to find a model
that started to learn from here I would
probably write a script to just kind of
automate the rest of everything but I'm
pretty confident we could get probably
all of these over 60% and that's just
off of pricing and volume features so
then you'd pry start bringing in other
features besides that but anyways pretty
cool I will just say though this is
seemingly good as I said before probably
a few times it's so easy to make
mistakes on projects like this so
obviously use this at your own risk
I'm not responsible for anybody who
actually uses this and destroys their
trading accounts I'd go so far as to say
there's almost certainly a bug in here
so so yeah educational use only please
so anyways pretty cool results let me
know if you guys tweaked it and can get
better results and all that or if you
don't want to talk about it I guess
that's fine too anyway keep it all to
yourself pretty cool that's it for this
project for now I don't really know what
I want to do next in this series I'm
probably done with like basics chaos
because I'm running out of things to do
besides getting like super complicated
like we did here
the next thing I'd really like to get
into as far as just kind of playing
around with things is audio either with
songs or speech or something like that I
want to do audio so anyway let me know
what you guys think about doing audio
with this stuff also if you have any
anything else you really want me to get
into let me know down below if you have
questions about anything in here feel
free to ask you can also come and join
the discord it's just discord dot GG
slash Centex
if you spot an error or have something
you'd like to improve upon the script
you can let us know in the comment
section or by discord again otherwise
that is all for now I will see you guys
in another video