what is going on everybody and welcome
to an exciting channel update both for
me and hopefully for some of you so stay
tuned for that what you see behind me is
the box apex 4 although box
everything is so customizable there's
really no specific model I can give you
but what I can say is the interior of
this PC is very special so let me show
you so this is from a company called box
they're Bo X X that's two x's or vo xx
comm there our company based out of
Austin Texas and they build like
high-end data analysis video editing
rendering those kinds of machines they
also have things like GPU servers
anything you could want for your
high-end high performance computing
tasks they definitely have it so what
you see here is a beautiful to RTX 8000
GPUs also a Titan RT X this I just
happened to slap in there I had one
lying around as one does so I threw that
in there so yeah the cable here and also
the cables for this hard drive that's on
me that's my job don't hold that against
them and so yeah that's pretty much the
heart of this machine at least to me
then you've got two Intel Xeon Gold CPUs
I can never keep it straight what the
model numbers are for Intel Xeon Gold
Jeep CPUs rather what I can say is
that's 12 cores each so 24 threaded
cores there for like it's 48 core
threads basically which is indeed useful
for pre-processing but I will just admit
most of my work involves the GPUs for
sure now the RAM here is 256 gigabytes
of RAM again very useful for
pre-processing also very useful for
things like reinforcement learning for
like memory and and such so yeah quite
the honkin machine there's also a one
terabyte M to drive the CPUs are
water-cooled I'm trying
think if I missing anything it's a 1300
watt PSU fully customizable unlike so
not too long ago I reviewed the Lenovo
data science workstation and my biggest
complaint there was you you couldn't
upgrade or change like GPUs you couldn't
even add GPUs even though the slots were
there to add GPUs you couldn't do it
because the the PSU was like all
hardwired there was no you couldn't
customize it with like extra plugs and
stuff whereas here there's extra plugs I
could do whatever I wanted to do from
the point of buying the machine so
that's really cool now what the heck do
we or what does somebody use a machine
like this for so a little bit ago I did
a cloud versus local computing kind of
comparison to kind of talk to you guys
about like at what point should you buy
actionable local hardware I'm not here
to sell you one a machine like this um I
will say I mean if you're interested in
high-end machines you can you can go you
can get anywhere from like a thousand
dollars and beat have a decent issu
machine to start off doing deep learning
all the way up to something like this
which is closer to like thirty thousand
dollars now the argument to should you
buy should you not buy is the same no
matter how much you know computing power
there is there it's all kind of the same
argument and generally it boils down to
you can check out that video but I'll
summarize it so in the video I kind of
show you based on the actual prices on
websites you know like AWS and Linode
and Azure and all that but the gist of
that is after about a cumulative of two
months so either if you're training
something for two months straight or
you're running a model for two months
straight although I will just remind
everybody that to train a model you
definitely need GPUs to run a model
oftentimes unless you have just an
incredible amount of queries per second
to this model to run a model you can
usually get away with running it on CPU
and RAM so just keep that in mind but if
you're training models for two months
cumulatively over the course of maybe
one or possibly two years depending on
how acceptable it is to you to kind of
run on older hardware then and that also
has a bit of a stipulation because the
cloud GPU providers most of them are
still on K 80
gpus some of them have some V 100's and
nothing more right Linode has the most
recent I believe they're offering still
the RT X 6000 GPUs and you can get up to
four of those on a single machine anyway
so that's kind of the argument of
whether or not you should buy it but
then you know what do we even do with 96
or actually with the Titan RT X I threw
in there that's actually a hundred and
twenty gigabytes of vram what the heck
do we do with that obviously they you
know the clearest thing is to have just
like a monster model okay so a model
that you could not have on smaller bits
of hardware so right now what I'm
working on is a chatbot model this has
kind of been the the white whale for me
because we we were I was able to train I
say we because now it's Daniel and I are
working on this but the first chatbot
model that I ever built was the one that
we used for the twitch stream and that
model is the best one I've ever gotten
and it was very small and it didn't have
very much data and it's not the greatest
model it's not actually that great and
so trying to make that better has just
been like the most impossible thing ever
so right now what I'm working on is this
this model here that is currently in
training that has had some pretty good
results is 20 total layers of 1024 units
per layer I mean this is a monster of a
model it is maxing out both the r-tx
eight thousands I'm not using the our
Titan r-tx only because so these cards
are 48 gigabytes of vram 48 gigabytes of
vram 24 gigabytes of vram so I'm
actually not using the Titan r-tx
because just because of the architecture
of what we're actually training here it
just makes sense one of these GPUs
becomes the encoder one of the GPUs
becomes the decoder so 10 layers on this
one 10 layers on that one it just works
out that way which is why having multi
GPUs isn't necessarily always going to
work or be faster if it is all on the
same machine it does it does work pretty
well but you do pay a price for sending
data from this GPU to this GPU so if
it's just one time it's okay but as you
scale out GPUs it does
come a little more challenging to get
things to keep being fast but again you
can build models that you just simply
could not build without multiple GPUs
in this case especially if you're able
to utilize utilize the envy link it is
effectively pretty close to being one
hole GPU so anyway yeah so very exciting
lots of things to look forward to on the
channel for what I can do and the other
thing I was thinking about was with a
machine like this I know that most
people watching this video will never be
either they won't have a machine like
this or they won't even have access to
it even in the cloud this machine would
be quite expensive to run hourly but um
I'm sure I think what this would be I
mean it would probably be maybe close to
like 10 dollars an hour to run this
machine in the cloud I mean so so you
really don't have time to train a model
and for example since I've received this
machine I've been running it 24/7 for
two months straight just constantly
doing rnd on this chatbot that that adds
up really quick and like I said um that
after about the two month mark you start
having to wonder is this making much
sense so you know to run in the cloud
versus manoa I could just actually own
the hardware and it would it would be
cheaper so anyway so that's what I've
been doing but then I started wondering
what if I now I don't know how this will
work I don't know I know how it will
work but I don't know if it will work
and that is I'm considering opening the
door to allowing you guys the community
to make a submission of code it needs to
be code that's on github send me a link
to the github to either Harrison at
Python programming net that's probably
best thing to do is just email it to me
so Harrison at Python program a net but
you can come into discord you could post
a comment or whatever make a submission
if something you think either it's your
your code or maybe some larger project
so like the the the chat bot thing at
least it started off as being basically
it was no I don't think I made any
change to the the Google's nmt and then
that's neural machine translation and
that's really for languages like one
language to another language and that is
what the
best chat bot has ever been was it was
just straight nmt and instead of doing
you know English to German I did English
to English so in English comment to an
English response and that works it works
pretty well for a deep learning chat bot
model but I want more and so yeah we now
we as in me and Daniel have been trying
to slowly kind of tweak things and
change things and see can we do a little
better and recently I actually had a an
amazing output from one of these models
and I know what I kind of want to do in
the next few steps and try to figure out
can I eat out slightly better responses
there but anyway it just takes a long
time that's why we want a machine like
this but getting back to what I was
saying I'm willing to open the door to
just giving the opportunity to the
community to submit some idea not ideas
no code it has to be running code but if
you have something like that either it's
your project or someone else's project
that you think makes sense for a machine
like this so it needs to be you know
don't send me cats versus dogs don't
send me some eminence you know send me
something that you that either is your
project or someone else's project that
you think would benefit from this
machine and I will at least consider
putting it up if I can't read the code
and understand it to a hundred percent
it's not being run on the machine but
just thought you know that would be kind
of cool if it can work I don't even know
if that will work I don't know if that
is just going to be too much wasted time
or something but I'm willing to just see
what do people have because I know there
has to be a separation between people
who have really cool ideas either in
like reinforcement learning video stuff
audio stuff all kinds of really cool
projects encode but it just they don't
there's no access to a machine like this
so I'm very interested to see like I
said no promises can't say that anything
will for sure happen there but I just
want to see so anyway that opportunity
is there
finally just has a channel update the
neural networks from scratch series
probably one to two months one month is
really
close I highly doubt the videos will
start coming out my goal there is to
finish the public well public it's for
backers but finish that that draft from
cover to cover so get a full first draft
done and then my plan is to go start
doing the videos and then as we're doing
the videos keep kind of honing in on
that draft and improving that draft
because my main focus is delivering the
book on time to people who have
purchased the book if you have purchased
the book you should already have access
to that draft and that draft just got a
huge update basically everything from
beginning to training a model is done so
now we're getting into kind of the
actual you know tweaking a model doing
having like training and testing data
and what what to look for as their model
trains so in terms of actually coding
the neural network that's all done now
so so if you are on that I definitely go
check out the draft if you're not aware
of that update also if you want access
to that draft if you're just dying to
get access to the neural networks from
scratch series you can pre-purchase the
or pre-order the book and you get access
to the draft it's in a Google Doc and so
you can not only see what other people's
comments were you can ask questions and
stuff yourself like in line with the
text and that's actually been working
pretty well so a couple of hiccups on
the way getting that many people I
didn't expect there to be like I don't
even know what we're like maybe 1300
people now so getting that many people
into a Google Doc was actually quite
challenging which is sort of surprising
for it being a Google product but anyway
that was interesting but that's all
solved so yeah make sure if you do have
access that or if you should have access
that you do have access and again you
can either email me or post a comment
below I suppose but the best thing is
probably to email me in that case so
anyway I think that is everything if
you've got questions comments concerns
whatever on any of this stuff the chat
bot stuff is submission stuff neural
networks from scratch stuff feel free to
leave those below otherwise I will see
you guys in another video