Hi this is Jeff Heaton, welcome to Applications of Deep Neural Networks, with
Washington University. In this video we're going to have an introduction to
TensorFlow and Keras and we're going to see how TensorFlow can be used
directly to perform mathematical calculations and how Keras becomes the
neural network layer on top of the low level TensorFlow compute engine. For the
latest on my AI course and projects click SUBSCRIBE and the BELL next to it
to be notified of every new video. TensorFlow and Keras so TensorFlow is the
low level mathematical library that gives you access to CPU, GPU, and grid
computing capabilities and Keras is the high level abstraction that lets you
think of these mathematical constructs as neural networks. I give you some links
here that point to the various locations that are useful for dealing with TensorFlow
and Keras. One of the first things that you need to deal with is there's a
lot of different versions of TensorFlow and Google is very good (or bad as the
case may be) at breaking changes. There's a lot of very smart people working on
TensorFlow who are very opinionated about how the API should be and it's
different factions of them win and lose their respective battles, breaking
changes happen to the API. A breaking change means that a new version will
basically cause previous code written for previous versions to not work. They
change the names of things. They change the capitalizations of things. Lots of
lots of little sometimes, just little annoying things, sometimes big structural
changes, so it's important that you're running the version of TensionFlow that
is specified for this class. This video was recorded with TensorFlow 2.0, which
was in its dev stages as this video was recorded. But as long as you have a
version that is 2.0 or later. I will be updating the course videos for the
different parts, if something gets broken as a result of
newer TensorFlow. So you can simply run this and it will show you the version
that you that you have. Now installing TensorFlow,
I have an entire video just on how to install TensorFlow on Windows and
another video for Mac. Those deal with installing TensorFlow with CPU to get
really good performance you need to use your GPU and your GPU it needs to be the
right kind of GPU. Some of the assignments and examples that will run
in class will require a GPU, run on a CPU but it's going to be slow, and installing
a GPU is is not the easiest thing in the world. I'll probably do a video on that.
I mostly, personally, run GPU deep learning in the cloud. Where I don't have
to install all the hardware drivers and and other things, but it can certainly be
done. What I recommend for this class, especially on the assignments, and parts
of this class that require a GPU, is Google CoLab Google CoLab is
cloud-based you don't have to install Python or anything. It comes completely
ready to go. I have a video that is linked on the first module on how to use
Google CoLab with this class, and we probably talked about it in the first
class meeting. So, this is this is probably the recommended way to actually
run all of the examples and to complete your assignments is Google CoLab.
It's a powerful instance it is a dual core with I believe 12 gigabytes of RAM
and a GPU. You can't beat it so that's that's my recommended route
for for working on this class and why did I choose TensorFlow for this class.
Well TensorFlow is supported by Google, has excellent support in Google Google
Cloud it has great CPU... CPU and GPU support and it's in Python. Python is
very quick becoming sort of the the high-level
language for machine learning and AI. So your well served in learning Python and
learning learning deep learning through Keras and and through Python. Now you
can use TensorFlow directly and if you're writing very custom sort of
machine learning models, that might be the way to go, but by and large the way
to the way to access this is through Keras it makes it makes the deep
learning quite a bit easier to to set up and there's really not much of a
downside, unless you truly do mean complete control over the the
computation of the underlying neural networks, and then you would need
TensorFlow and there's other deep learning tools, I have a number of them
listed here. If you're particularly into Java DeepLearning4J is definitely
one to look at, as is H2O, TensorFlow is, at least in previous versions, it was
very graph oriented, so that it would it would compute later. A big thing about
TensorFlow 2.0 is your execution they made a lot of changes so this is very
much. If you haven't worked with TensorFlow before don't worry it's
completely changed. Anyway so, this, this will get you ready for the for the
latest version and this is just showing Tensor Board, which helps you visualize
the neural networks that you that you create. We're gonna look at how to use
TensorFlow directly and we're only going to go through that in in this in
this video. The rest of the videos will be completely in Keras, so what this is
going to do is do something called a Mandelbrot plot. You can see it here. If
you've ever work with Mandelbrot's before you definitely recognize that
that very classic looking plot that you have there, essentially. This is a set of
equations, very simple equations that give a you can keep zooming in on
regions of this and it's it's almost infinite. And the complexity of the the
very cool sort of land escapes that you could zoom into here.
I've written the code to basically do a Mandelbrot plot, now you can see it's
it's very simple. That's that's the code to produce the the Mandelbrot and the
the functions the display fractal. That's actually used after it's it's completely
rendered, what we're doing actually in TensorFlow is we're defining essentially,
it's like a it's a complex number it's a complex plane and you plot out the
Mandelbrot across the entire plane. And this I'm not going to go into the actual
math and the Mandelbrot but there's plenty of tutorials on that. And we're
essentially sweeping across this entire this entire plane, we're doing 200
iterations of it so this gets drawn 200 times each time it gets drawn a little
bit better, and it's very much an iterative process so each time through
this it's being sent off to TensorFlow for processing it could use the GPU and
if you wanted to it's so fast that you really you really don't need the you
don't need the actual GPU for this. The CPU alone can do it and this is
essentially the equation here that is that is being calculated to actually
plot the the Mandelbrot, as it goes through and if you run this you can see
that it actually goes very quickly. Now to see a little more exactly what's
going on this is also using TensorFlow directly I'm creating two matrices here.
So, these are actually vectors are you I guess you'd call this a row matrix this
is a column matrix because this goes straight across this is two, two values
that that are on top of each other and we're going to do a matrix multiply.
Matrix 1 by matrix 2 and we print it out and as you can see the results here it
does the matrix multiply and results in 12.
Convert it to a float so that you don't feel the the tensor decorations around
it these were two constants so that was constant times constant. You can tell
this is all very linear algebra oriented. Now we're doing two variables and we're
going to subtract, well we're subtracting a variable from a constant and we do
that and we can basically see that we get the -2 negative, 1 row matrix.
Ee can now reassign X, because X is a variable so we reassign it and we're
able to basically recalculate that that calculation from up there and get a
different value. So this is the function, this is the building blocks upon which the
the neural networks are built and TensorFlow Keras lets us start to
think about the layers of the neural networks, like we had before so we're
going to use a classic data set, the miles-per-gallon data set. This lets us,
that data set gives us statistics or calculations on on various cars and the
miles per gallon so the idea is to build a model that uses cylinder count and
displacement and other stats on cars to actually come up with the miles per
gallon. To produce a model that does that so here we are going to basically read
in the auto miles per gallon we're gonna keep the names of the cars the cars
themselves the names of the cars aren't predictive they're just the car names.
Horsepower we do have some missing values so we're going to fill in the
median just. We've seen this before so I'm going through this pretty pretty
quickly and then we create our X and our Y axis the predictors. So we're
predicting on cylinders, displacement, horsepower, weight, acceleration, year, and
origin, what country that's produced in and we're calculating this on or we're
trying to predict the miles per gallon so this is your X, which is your
values used to predict and Y is what you're actually predicting. And then this
is what Keras actually looks like. We create a Sequential so that's just
layers that go in sequence, once we create our first dense layer, a dense layer
simply a layer where every neuron in the first layer is connected to the next
layer, and that's that's very common that's usually how layers actually work,
especially for tabular data like we have. Now this will be different when we get
into images the input dimensions are going to be the X shape one so that is
the number of columns that we have these X is the input value shape 0 would be
how many rows shape one is how many columns. Activation is the rectified
linear unit, and we have 25 neurons. We have 10 in the next hidden layer, we
don't have to tell it how many inputs we have is, it's, it's basically the 25 from
the previous layer and also activation is row output is just 1 for a
regression neural network, the output is always 1. It's also important that for a
regression neural network that you always have the mean squared error is your loss.
An optimizer we'll usually use Atom. In this class there's
other ones you can use that we'll see soon and then we fit it we're going to
fit it for a hundred epochs epochs is just how long the neural network trains
for. It may or may not be enough will later see how we can determine how many
epochs is actually enough. Verbose means to print out data as we go, and if we run
this we can see you see the loss shrinks as it continued, so it gets better and
better and more effective as it goes. This is what the verbose value up there
says to do so this lets you that that lets you control how much data is being
dumped to the screen. On now, it's not doing particularly good here, this is the
hundred is probably not enough for it will later see that we get more advanced
and this actually gets quite a bit better predictions and we calculate our
root mean square error, 24 believing is not particularly good on
miles-per-gallon because that's that's the range so you can in cases like that
this is not that uncommon you can retrain. It it might give you a better
result yeah that's that's a much better result so it just took a retrain in that
case but we'll see later how we can control automatically the number of
epochs that's needed. And you can see here this should drop yeah that's much
better that's that's about as good as it gets you'll see it sometimes good as low
as 3, and then we can print out the various cars and their miles per gallon
or predicted miles per gallon. And yeah you can see now they're much
much closer you can also do classification we'll use the classic
Iris datasets for that that's where you have four measurements that give you any
of three different types of iris flower you're using sepal length, sepal width,
pedal length and petal width. This is what a Keras classification neural network
looks like. It's very similar you have the inputs that would be four inputs you
have the hidden one or more hidden layers. However, with a classification
especially a multi-class classification, which means you have more than two
classes, you'll do softmax and categorical cross-entropy if this was a
binary classification which we'll see in the in the next module that would that
would be a little different you would have a binary loss function here, but
we'll see that when we when we get to that training this one it trains and
decreases the loss you can also print out the species those are the different
types of virus that we're trying to classify. And this is what the output
from it looks like. It's in scientific notation, which makes that a little bit
difficult to read you can tell it not to use. You can suppress the scientific
notation but essentially these three values are the three irises and
whichever one's the biggest which is this one it was classified to be that
first iris which was satosa you can also suppress the scientific notation
and then you see the values that are one or at least very close to one that means
it's all that first class now iris file is sorted by the by the individual
irises, so that's why at the top of the file they're all they're all the same
and you can also turn these predictions into just numeric values, so zero means
the first iris one is the second and two is the it's the third, so that's if you
just want literally a number that tells you which of the classes
was actually predicted. You can also print out using this array notation you
can print out the actual textual name of the irises of the classes that were
chosen and you can see the accuracy is 98% so it's it's very very good.
You can also feed in just ad hoc values and have a prediction so this is how
you'd feed in one value and it predicts iris virginica for these measurements,
and you can feed a bunch of them at once this is how you'd feed too.
Thank you for watching this video. In the next video we're going to see how to
load and save Keras neural networks. This content changes often, so
subscribe to the channel to stay up to date on this course and other topics and
artificial intelligence.