>> Hi Python data scientists.
On a recent episode of VS toolbox,
we talked about Jupyter Notebooks and VS Code and
how it can make your data science needs a lot better,
and this time we're going to take it a step
further with Jeffrey Mew as he
comes to talk about
PyTorch on this episode of Visual Studio Toolbox.
[MUSIC]
>> Hey everyone, welcome to
another episode of Visual Studio Toolbox.
I'm your host, Leslie Richardson,
and joining me today is Jeffrey Mew,
who's going to help us continue that trend of
all the cool Python things that you can do
in Visual Studio Code today. Welcome.
>> Thanks for having me.
>> Sweet. So what are we going to talk about today?
>> Yeah. Today I'm just going to show off
all the cool PyTorch features within
VS Code that we've been building in the past few months.
I'm going to be going through
a quick demo project and as much as the demo,
just to show the cool features.
That sounds good. Are we going to get started?
>> Yeah. But first, what's PyTorch?
>> Yeah. For sure. PyTorch, if you're not as familiar,
is an open-source Python machine learning framework
that makes it a lot easier for
data scientists and Python developers
like to build and work with machine learning models.
According to the actually the latest
and largest annual Python survey
and Stack Overflow survey as well,
Pytorch is a now actually, one of,
if not the fastest growing
Python machine learning frameworks out there.
Because of this, we want to make PyTorch
development a first-class experience with a VS Code.
>> That's awesome. I have got some big shoes to fill.
We had Claudia Reggio process from
the Python team last time we were talking about Jupyter Notebooks,
and I'm wondering if there's some overlap between
the Jupyter Notebooks that you can use VS Code alongside PyTorch?
>> For sure. Yeah, so Claudia is actually on my team.
Somehow, my colleague as well.
I work closely with her,
but a lot of the features I'm trying today,
I'm actually showing within Jupyter Notebooks,
but you can also find these features within
regular Python puzzles [inaudible]
if you're more familiar with that,
but I'm just assuring also
that Jupyter Notebooks features as well.
>> Awesome. Well, I'd love to see it in action.
>> Yes, so today I'll just be going through
a typical Python or
PyTorch machine learning project within VS Code.
Let me walk you through step-by-step
through the project of getting
started to gathering your data and finally
to development and training the models.
And alongside I'll be showcasing
all the PyTorch and data science features
that VS Code has offered like I mentioned.
To get started with VS Code, if you don't have it already,
you can go to code.visualstudio.com to download it.
It's free open-source and cross-platform.
But once you actually have VS Code you'll
need to go into the extensions tabs.
If you look in the left-hand side,
there's an extension tab, and you click on this,
you can search for the keyword PyTorch,
and that is where
all the PyTorch and data science features reside.
All these are going to be there. Great.
>> Awesome.
>> Now that we actually have everything set up,
your Visual Studio Code download,
a PyTorch extension downloaded, we can now get started.
So everyone's talking about stocks nowadays.
I'm actually using a simple PyTorch project
to try and predict the model of the Microsoft stock phase.
Yeah, we'll be using
the historic stock market price of Microsoft as a dataset,
and we'll also be using PyTorch inside of
Jupyter Notebooks with MDS code to code up the model.
Again, quick disclaimer, I'm not
a financial expert at all this project Is
just a plan so if they worked I'd probably be rich by now.
>> You got in really quickly on the GameStop stock.
>> For sure. But yeah, to get started.
I have my project already opened up.
I have my Jupyter Notebook called stocks that [inaudible] where
I'm actually doing the PyTorch coding
and the training of the model here.
Again, I'm really stepping through the code very
quickly since the focus here is mostly around tooling,
but feel free to pause this video
anytime you want to look at the code in more detail.
>> Awesome.
>> Of course. One of the first things you'll need to
do with any PyTorch project or just data science project,
in general, is to import the dataset.
That's what I've done in this first cell here.
I'm actually importing.
I'm using the Yahoo Finance API to
import all my Microsoft stock price data,
and then to run it, I can just quickly click this "Run" button
or a "Shift-Enter" if I want to use the [inaudible] keys.
One of the first things, once
I actually have my data in my Notebook,
rather than just printing out
the data frame because you actually want to see what
the data looks like right now
they don't really know what the data is.
Rather than running a print statement,
look at it crudely, within our notebook.
I can actually use VS Code is
built in data frame viewers will look
at my data in a much more human-readable format.
I can figure out my game plan of how want to clean
the data and what columns to look at, for example.
To do so I can just open up
the Variable Explorer so you can see the Variable Explorer here.
I got there by clicking this button up top,
and then I can actually find
this DF variable which
I've created here so it's the same variable here,
and then if I click on the left-hand side,
it opens up the data there, which I've
previously just opened for the sake of this demo.
>> Awesome.
>> Once you click on this,
this is what it looks like.
You see it's like an Excel-like interface for your data.
So it's much more human-readable than having to print that out
>> Right.
>> Yeah. In this data here,
it's pretty nice because I can actually go and
quickly filter and sort my value.
For example, if I want to see the earliest and latest dates,
I can just open this up.
I can see the latest date or this earliest date of
2016 and the latest date I'm getting is today's date,
which is April 20th.
I can also filter by,
if I wanted to see the highest price I'm
looking at and lowest pricing and [inaudible] as well.
I can make sure that all my data is on a high level,
as a sanity check to see that my data is looking correctly.
>> It just feels so natural seeing this table [inaudible]
You think it would be super busy when you say,
it's an Excel sheet that's going to add it into
this code and it's not, so that's cool.
>> But yeah, once you actually have
this in this Data Viewer, I almost called Excel sheet,
but in the data viewer,
it's much easier to look at your game plan is figure out what you
want you next because oftentimes you don't have to work,
you don't have to use all the data.
Seems like a lot of data here.
But based on what we can see,
we're probably going to need the dates column because we
need to know the stock price on that date.
But often we're only using this closed column here
because this is actually the closing price
of the stock on that day.
This is where we're actually going to predicting it.
After looking here, I can see in my data and I'd be like;
oh I only need to get the date and close columns so,
I can just quickly jump back to my code here and just run
this cell saying that I only want the date and close columns here.
I just hit "Shift-Enter" and run it.
Then if I want to see it up-to-date,
I can just click on this refresh icon, and boom,
you can see as
a quick sanity check that it only selected those two columns.
>> Great.
>> I've also written some quick codes because
again looking in the database,
it's really nice and all but
sometimes you want to visualize your data.
I have a quick graph.
If I just want to see the last six months or if I want to
look at the range of prices on certain dates.
I have this partly graph, and this is just showing
that VS Code of bucks supports
interactive widgets or interactive plots
out of the graph or out of the box so you know how to [inaudible]
>> That's really when you know your data is
official and you have graphs and stuff like that.
>> Exactly.
>> It's like this person knows what they're talking about.
>> For sure. Now that I have
a pretty good understanding of
my datasets and see what the values are,
etc, and sounds like pre-process the data.
We want to apply some transformations to
that data to better normalized it because
you can see the arbitrary values are between 250-260, right?
We want to apply those values for the model.
Oftentimes it's like if you have
many transformations that are routes,
and you usually want to see what happens.
I'm trying to see if you applied those transformations correctly.
You can see here a decent transformation such as
I'm trying to fill in all the NA values.
If there's any missing values, I think I'm going to throw it out.
I'm going to scale my data between negative 101.
I can run that cell again really quickly here.
Hit "Shift-Enter", and then again,
I can go back to my data viewer.
This is why the data viewer is super-useful.
I can jump back into the data viewer and this time,
I'm opening df_Microsoft because I'm in the variable here.
If I jump back into here, here's
where the data viewer comes in really handy again.
Like I mentioned before, I was trying
to scale my values between negative 101 and
I can just quickly go and use this filter column as
a sanity check to make sure this
is exactly what I expected, right?
I can see that the max value is one,
the minimum value is negative 1 so there's
no erroneous values started doing thing really wrong over here.
>> That's awesome, seems pretty fast and simple.
>> Yeah. The next step is,
once I actually have all my data pre-processed,
I actually want to convert it to tensors now.
If you're not familiar with
tensors or if you're new to PyTorch in general,
you could just think about tensors are like
machine-learning data type that
Pytorch specifically understands and works with.
It's just like a wrapper and it
can come like a matrix essentially.
But again, I've written a basic function to do so.
I have this function here which splits
the data into training and testing sets
but also converts it to tensors.
Again, I can hit "Shift-Enter".
>> This is when Torch really comes in, right?
>> Exactly. Yeah. This is actually
doing more hydrophilic stuff now.
If I want to actually look at the data and
now I can go down scroll here.
If I go find x train x-test.
I can use the variables I would have just created,
and I can see boom,
they're now tensor types, right?
Then we can actually see that the size of them.
I can see the testing and training split I have via
a test size which is 237 items in it,
and then the training size, which is now
947 as well so I can see that testing and training set size.
As well, if you're still interested
to make sure that nothing went wrong,
we also now recently added support for
opening tensors within your data viewer as well.
So you can actually see the tensor data as well,
which is one thing we didn't support for.
Also, if you can see in the bottom right here,
this tensor is actually three-dimensional.
You'd be originally thinking like;
how do you look at three-dimensional in here,
but we have just recently added support for
three-dimensional data by adding the Slicing panel.
Slicing panel, Leslie basically lets you look at
any two-dimensional slice of your data, right?
So just take any two-dimension or three-dimensional data show.
This is way more easier and nicer to look at than having to
look at just a razor within a race right? [inaudible]
>> Okay, yeah.
>> Then this definitely supports them,
almost all your typical slicing things so you
can do ranges as well if you want to do as well.
Cool. Now that we've actually converted our data into
Tensors and we're ready to start training,
when it comes to Tensor, what you probably have to make with
your PyTorch project is to select the model.
There's different models I can use to
train the stock price predictor but the two
most common are well-known ones are
called LSTM and GRU based models.
I'm not going to go into too much detail about what
they are just because I'm pretty sure
some online article can probably explain
it a much more quicker way than I can right now.
But basically, just right now,
just know that we don't really
know which one's actually is the better one.
One of the common things you have to do
is figure out which model is better.
We've actually created both models
just to train them and see which one does better.
>> Awesome.
>> You can see here, I've created
this LSTM model and this all PyTorch code as well.
But before I do so, especially if
you're new to PyTorch or even if you know PyTorch itself.
One of the more knowing things is trying to
understand what this model architecture looks like.
Just looking at the code, this's is
a pretty simple model but it might be a
little bit confusing for some folks.
This is where we're going to make use of
a tool called TensorBoard.
TensorBoard is an open-source dashboard companion to help with
PyTorch or TensorFlow developers visualize
different attributes or aspects
of their Machine Learning models and training.
It's one of the more popular tools that's
actually used for PyTorch right now.
Throughout my code, again, it's hidden the back-end,
but throughout my code I've been logging
information to TensorBoard log files,
which actually gets read by
this TensorBoard dashboard, like I mentioned,
and parts of the UI components that
you'll see as you open on the right.
Actually, what's really great
is one of the newest PyTorch features
that we built into VS Code is TensorBoard Integration itself.
As you can see already here.
>> I was very curious about that.
>> Watch it, it's super-simple.
You can either click on the code lengths if you
remember at the top it printed texts,
you have any text support imports.
On my feed, you'll see things Run button here,
so you click that to watch it.
Or if we just go back to our code down here,
you can watch it anytime from the Command Palette.
By hitting ''Control Shift P''
or "Command Shift P" if you have a Mac,
and it's searching for the keyword launch TensorBoard.
If you click on this, what's really smart is
that you can use the current working directory.
Whatever workspace you have open,
it's smart enough to automatically
search through all of the subdirectories
to look through the files,
so you don't have to manually specify the exact folder,
it'll automatically do that for you, which is why it's great.
But if you have your log files in
another folder for some reason, you can also use this as well.
But again, I want TensorBoard
just for the sake of time for this demo as well,
and you can see what it looks like on the right.
Getting back to the grass problem like I mentioned in the report,
it's somewhat hard to visualize it based on the code.
I've actually logged what the graph LSTM model looks like.
Then from TensorBoard, we can just go to this Graph Tab,
and then here you can see the model,
but if I want, I can double-click it.
It's like a much more human-readable version.
Does that makes sense?
>> Yeah.
>> But my zoom is really big just for this show,
but if I just close this,
you can see at a very glance.
You can see, boom, there's two layers,
I can see it corresponding to the two different layers here.
There's LSTM layer and a Fully Connected layer.
You can see here this is the LSTM and
this is the fc which corresponds to that.
What's really great about this is especially if you're looking,
usually, you're not developing models from scratch.
Usually, you are looking at other people's models;
you want to base on other people's models.
This is really useful because you can quickly see
or get a better understanding of how they built
the model and what the model actually does.
>> Right. Yeah. You're not wracking your brain trying to figure
out what crazy code do they have to write to give me this graph.
>> Exactly.
>> Yeah. I can't imagine not having something like this,
a TensorBoard like this right away.
>> Exactly.
>> I'm personally a visual learner.
>> Yeah. For sure. That's why TensoBoard is so popular right now,
it's trying to help those visual learners with PyTorch.
>> Yeah.
>> Now that we have a better understanding of our model,
we can actually go on with the training and pushing of the models.
Again, I previously ran the training because often model training,
I don't know if you're familiar but often
model training takes ours for
[inaudible] I don't want you guys
to just here waiting for hours to [inaudible].
>> Welcomes to the pre recorded stream where you
are going to watch a model be trained for two hours.
>> What I've actually done is if I want to increase the training,
so here I just have some training steps which I've previously run.
If you can see here, I've actually connected
to a Remote Server in Azure.
My local machine, just some laptop,
is obviously not going to be that powerful.
I ran on my local machine, it takes a few hours to run but I
actually leverage the power of
Azure and this extension in VS Code,
which is free call remote SSH.
You can see I'm actually running all this code right
now this entire time on
this remote server but it looks the exact same
as if I were just running this on my local machine.
This train actually took,
I'll say a minute or less because I'm
leveraging GPUs in the Cloud.
I'm way 1000X compute power comparison to my local machine.
>> Awesome.
>> Once the training is completed, I've done
the training for LSTM,
ran it again for the sake of time, ahead of time.
I can see how that looks in
comparison to the actual plot national prediction.
This is what the predictors also you can see
it's not that great comparison with [inaudible].
>> It's a pretty model
>> I've done the same thing
for GRU which is the other model I was talking to you about.
Then here, you can see this falls in much more accurately.
Not super accurate but you
can see this is just a super basic model.
If I've had it a little more complex,
maybe if all it better but this is
just a grip concept, just trying to show that.
Obviously, this is a way better model.
>> Right.
>> Then the other way to see if
your model is actually correct is you want to make sure that
your loss function is actually converging to zero because that's
a sign that your model is actually
picking up on the patterns, learning and converging.
This is where we can leverage TensorBoard again,
because I'm actually been logging in
TensorBoard my training results.
If I jump to the Scalars tab,
you can see I just have different lines because I just
ran the runs different times
to show you that's it's always going to be consistent.
You can see as I run on the x-axis is the epochs,
the number of times I've run it.
As I run it more, you can see it's always converging to zero,
so it shows that the model
is converging at least so it's a great sign.
Then from the use plots that I've plotted, again,
you can see that, the GRU model is way more accurate.
So we're probably going to be used this GRU model.
Then once we're happy with this model,
we're basically done the project.
All you need to do is just save this model
and then from here you can just start
deploying it to the Cloud via Azure or within VS Code.
Then from there you can start making all your money.
>> Awesome. I do like my money.
>> Yeah. Those are
the key features that PyTorch has to
offer or VS Code has to offer for PyTorch developers.
If you're interested in more, just go search for
the Python extension within VS Code because
that's where all the goodies live,
and Jupyter notebooks as well.
>> That's awesome. What can we foresee
in the near future with PyTorch? What's next?
>> Right. I would say that PyTorch is one of the newer endeavors,
as I was saying, within VS Code and Microsoft.
We have a lot of things in the pipeline.
I would say some of the next things without giving
too much information away is a lot of data tooling.
Things to help with
enhancements to the Data Viewer like you saw earlier,
better ways to basically clean your data, process your data.
Because that's something that all data scientists have to do.
That's something else that's coming into the pipeline.
But stay tuned because I'll probably be back in
the future because there's
going to be so many of you just coming along.
>> Good stuff.
>> Yeah.
>> Yeah. Thanks for sharing that.
I think alongside Jupyter notebooks,
Python really is super powerful and so is VS Code.
>> Thanks so much for having me.
>> Awesome. Definitely go try this out you all.
It's some cool stuff, especially if you're a data scientist.
You're less frustrated.
>> Exactly.
>> Cool. With that, happy coding.
[MUSIC]