Welcome everybody. You're watching mr fugu data science. Today, we're going to do numpy basics
for python newbies. I want to mention something to you. Please, look at this right here okay. That
would be valuable in your life and your career. You just need a couple of installs real quick.
These are going to be pretty simple and painless. If you don't have numpy do a pip install or conda
install. Depending on your situation you may need to do pip3. There's a couple things i want to
mention with why you should use numpy. One, you can use it for logical or mathematical operations but,
the fact that if you're dealing with regular old python lists. This can slow you down.
If you're trying to do computation such as looping things like this can be problematic
if you have a lot of data. That's where numba can help you out. Because in the background it's using
c or c plus plus and it's storing your arrays in one location. Whereas a python list it's not
going to store it in one contiguous location. You are able to use vectorized code with numpy.
Which allows you to avoid explicit looping and behind the scenes is using that c
compiler. Now, since you're using vectorize code, you're able to compact this in your memory. Okay
and one of the other issues that python has is type checking. Python uses dynamic type checking.
So at run time when you run your code then it'll start evaluating to figure out what the type is
for everything. One of the caveats that you have to notice is that when you use numpy: you need the
same data type. Python list could have mixed data types you could have some integers, you could have some
strings, you could have whatever. That's one thing you need to pay attention to.
I want to show you for comparison with speed. Two examples where we're going to run the size of a
billion. First, we're going to go through a list comprehension which is faster than a regular for
loop. Then we're going to compare it against a numpy array of the same size. We're going to do
some multiplication here for our list. While we're iterating through and doing numpy multiplication.
You're going to notice something right here, when you run this: it may vary on your machine.
But you're going to see that speed up of 10 times. That's quite a bit. There's ways that you could run
this and actually make it even faster using numba alongside numpy. And if you wanted to make it
even faster. You could also work with a C compiler using cython. We won't cover those today. The next
example, i want to show you is going to run through a few different examples. Let's get into what those
are. I'm not going to show this code you could pull it off of my Github. Because, it's quite lengthy, so
if you're dealing with concatenation, dot products, scalar addition or deletions: you can notice a few
things we're not quite 10 times as fast doing the concatenation. The dot product on the other hand is
more than 10 times we have quite a bit in scalar addition. And if we're making any deletions. So this
is another step towards going in the right way of using numpy. For some of your operations especially
if you're a data scientist or data analyst or someone who's doing computational work. Also
consider scipy. Now, we can do conditional operators. For instance, we can use this arrange to see what
this is doing first you're creating an array from 0 to 11. Where it's 12 entries okay. And we're
looking for something that's greater than nine. We could also do multiple conditional statements
and throw those in as well. The first one we're looking for: greater than nine. it's returning a
boolean value. Whereas the second one is returning a actual value. Just like if you're throwing this
into a data frame and doing some conditionals the same way. Just depends on how you're actually
calling this. If you're just calling the array it just returns a true and false. If you're returning
the array like this then it will return similar to if you had a python call for your data frames.
You can do various indexing as well. We can set up arrays, in descending order or a regular ascending
order. Where i'm doing a step size of one and i'm going reversed. We can pull two through six
exclusively. Which means that we're going to negate the the sixth one. We can go through the elements
one through eight, in steps of two. And return those. And you can also do specific values that
you want to pull. For instance, you could pull the fourth element twice, the second and the ninth
one. And these are index positions okay. We can look at 2D and 3D arrays for indexing as well
where we can throw in our array and we're going to take the second element of the first dimension
our first array. This is the first array that we're dealing with the first dimension
and we're going to take the first element that's inside of there. Which happens to be one, here.
We can do the same thing and look for the sixth element of the second dimension which would be
here. And we'll take the sixth element of that here doing the 3d
one gets a little tricky. You have to figure out where you're going. We're first going to go
into this. Pay attention to the notation so we're first going into this one then we're going to
go here which is this part. And then we want the second one which is out of there. Which is this one.
Remember these are all by indexes. You can also do some importing, exporting of files. You could load
text you could generate from one file type and convert it to another. You could save the text. Just
chuck in your file type that you have here, your array and then if you have a delimiter. For example
we can inspect the properties. These three you use quite often to figure out what's going on. You can
convert your data types just like you can with a data frame. If you had to change a column, you
can convert to a list just like you would do for a data frame. You could do the same type of thing
and you could figure out what's going on with something in particular. If you just chucked it in
to the parentheses you could do linspace which is giving you the ability to create intervals within
a range, where we're going from zero to three. And we want six items printed off and they're evenly
spaced. We could create what's also called a copy or a view. If you're making a copy, you're making a
new array and if you make any changes to that copy it will not affect the original. The view is just
like what it's called it's just a view. Now there's something you can pay attention to. You can verify
for yourself if you've never seen this before. If indeed your array, your copy versus your view
will own the contents what's here. And when you print this out using this dot base, for your copy
and your view. You're noticing that that copy does not own that array. We have reshape: which is very
useful. Because there's situations where you need to change the dimensions of your data. For instance
we just have this 1d array, and we want to reshape it into two dimensions where we're going to have
two rows and two columns. We're originally starting off here, where your rows are four.
But you just have one column and you're converting it. We can do sorting with various algorithms here.
Depending on what you need. You have to pay attention but, you still have the axis which
is zero or one as well to do your manipulations. Let's look at our array that we have right
here. And we notice that we have these two rows, four columns. And if we do a merge sort
based on a row you'll notice that this now becomes ordered and so does this. Well that seems kind
of odd. Because, it's a one not like in a data frame or where its zero. You have to pay attention to
how that actually works when you print this and see what's going on. Here's the reverse if we're
working with the columns. These were in order except this one. The second and third column or
pardon me. The second column and the first column were changed from this original right here. We can
add and remove elements if we need to. For instance, we have our array that we have right here. This two
dimensional array and we want to add this in. And we're doing it row wise. We can also
delete. For instance, if we wanted to delete based on the index position of 1 for our column. We're
dropping this right here this 977 or 12. You can do the same thing but, go row wise: where
basically we dropped the first row right here. You can get more specific and you can drop the
the first row and the third row. Which means we're just left with the middle row now. You can use
something to change your dimensions using squeeze or expand. Expand allows you to add a new axis. A
row or column depending on what you're needing. Squeeze is the opposite. But is very distinct.
Because you will shrink it only. If the axis has a single entry. This single entry is extremely
important. For instance, if we have this shape the single entry for the shape means this.
And this you could choose the zero axis. The second axis. But you cannot use this first one. Because it
does not equal one. Look at this example real quick. We have this multi-dimensional array
and it's four dimensions. We decide that we're going to use the squeeze. We're going to use
it on axis 0, one, and three. We cannot use it on axis two because, we'll get an error in this case.
Now pay attention to something. Notice when you print this off; the amount of brackets you have.
This is the original 4D to the next spot where we're going to zero. We're going to go
in one layer and take it away now. When you go to the axis equals one; it looks like the same thing
again that you're printing off. But, this get a little more specific and see what's going on here.
So we're going to look at this and get the shape of it so it makes a little more sense. We have
one, one, four and one. Okay fine. We're going to take away for axis zero. This one. That's what's going on,
you're left with this axis one. You're taking away this one. That's why these two look the same
and then axis three you're removing this one and this is what you're left with.
okay they're all three dimensions and you have the same amount of elements. They're size four. So
breaking this down could help you visualize what's going on. Because it could get quite
confusing adding dimensions to your data. Now the expand. So we have this 1D array we're going
to work with our columns. We're going to work with our rows okay. Let's look at what this is
so the sprint is off and we have our 1D array which is five rows and just our single column. Here
our original data. Now we change this to work along the columns. And now we'll have a one by five. We
rotated this. We can go along the rows and rotate this into a five by one. Because you notice there's
nothing here. Well that's fine. We can look at basic mathematical concepts. i'm not going into
the whole list of these the things you're probably going to be more interested in the linear algebra.
Things that come up with matrices because you could do an identity matrix, you can
do matrix of all zeros, all ones, change up the size of the matrix that you need. You could do
anything with random values for instance. You could do random integers, you could do sampling.
Just depends on what you need. i'm letting you just see what is available to you. There's various
sum operations that you need to pay attention to. For instance, you could sum along a 5 by 5 matrix
which is from 0 to 25. And you could sum everything together to get this value. You could get the
column sum, where you're summing the long axis 0. And then you can get the row sum. You have to
pay attention to what you're doing. Because, that can mess you up later okay. You can use something
called flatten and ravel. Flatten gives you a deep copy. If you use regular python in your first
course your, your first few courses: it should cover something about shallow and deep copies. Your deep
copy is creating a new nd-array in memory. So any changes that you make will not affect
the original. The shallow copy is using a pointer for the memory location and anything you do will
change it. Let's have our two by two of all ones and let's flatten it out and see what happens.
So we have our shape which is starting with the two by two. It's two dimensions size four. It gets
flattened out into a one dimensional array that you see right here. That's the deep copy
that we're using with the flatten. We can do the same thing but, with ravel making our shallow copy.
The formatting is still the same you just do dot ravel, to whatever your numpy array is. Nothing
else goes in there. It's easy and you'll notice it's doing the same thing. Going from two to one
dimension. Except you have a shallow copy. You can run different statistics. Here's an idea of some of
the basic things that you can do only distinctions. Pay attention if you're using the arg max versus max.
One gives you a max value. The other one gives you an index position for that value. We can use
stacking and concatenating. For instance, you can increase the rows, you can increase the columns, you
can also create a third axis throwing this into three dimensions. And we could do something similar
to concatenating if you're looking at regular python. Let's create two arrays with these
values and do a v stack. But, what's really going on with this. Well the v-stack is allowing
us to create rows. Let's see what these look like. This is just one row and it's getting
chucked in with the other row after it. The reverse. We're making a bunch of columns and it's only one
row. The third option is using this d stack which is going to create a 3D object. We have
2D arrays here. We have two of these 2D arrays that we're going to convert into a 3D. Let's look
at this. Pay attention to something. Since, these two arrays right here are equal in
size and shape. We're going to print off the fact that they're in two dimensions.
And this is what we're dealing with for the first one; a 2d. Now if we combine these together you're
going to notice that we're doubling the size and we're increasing the dimension. So now we have a 3D
object. That's what's going on here. Notice, the the number of brackets that you're dealing
with. The other thing that we need to consider is to concatenate and that's just like if you
are similar to doing concatenation and data frames. Very similar you throw in the axis, we're going row
wise and we're putting both of these together. Just remember to do the parentheses okay. And
that's what's going on here. Here's the first one. Here's the second one. Easy enough. We're saying
that we're starting with this two by two which is two dimensions and there's only four items.
For each one of them after we concatenate we're still keeping the same dimensions.
We're just adding rows. That's what's going on. You're just adding the rows to it. For the way
we set it up with axis equals zero, axis equals one. Doing the same thing; you're going to add
columns that's what's going on here. We take the same thing, we change the axis to one
and we pluck out four columns instead of two. Cool so that means that we
added these two and these two. We can think of doing an extension onto a list. If we are iterating
in regular or python and you can use axis equals none and it just makes one big old list
or part of me array. If you notice here's our 1D array that we have now which is eight items.
You can use something that's very useful called broadcasting. There are some limitations with this
i suggest you look into. But it's allowing you to do arithmetic operations on arrays of mixed,
mismatched sizes. This is done by stretching the smaller array to fill the void. It will basically
wrap around the values in the smaller array to make it match the size of the larger. But you
need to consider something. This right here. To make this work either they need to be the same size or
one of them needs to be one. Otherwise, you get a value error. Let's see what's going on with this.
We have an array that's a 4 a 1D array of 4 elements. We have a 2D array with 4 elements each.
We add them together, that's fine. Because we're starting with the same sizes that we're going to
put together and end up with a 2x4. That's fine, we expand this and move a little further, where we're
dealing with the 2d and a 1D object. Where we have a 4x3 and we have a 3 item. And what happens here?
We can still add these together because, we still got these threes that are jiving with this. We
can make our four by three. This is doing element wise addition. Okay so this element with
this element, this element with this one, this one with this one, and then the same thing this one
with this one and so on. Let's do some mental gymnastics that's gonna throw you
for a quick loop if you don't pay attention. You got your 1D. You got your 2D. You got six elements
in here. You have a 2x4 but, you got an issue. Let's say we transpose this real quick and we try to fix
the dimensions to try to make this work. i decide to add a numpy array of zeros. That's a two by six
onto this. So instead of having a six by one. Now i have a two by six okay coo.l We do a transpose
of this and we end up with a six by two by one here. Because we're using something new called a
new axis all right. We're adding that in so we can increase the dimensions and use these data
because we need either everything to be the same or have a one. And that's what's going on here.
Ao what i did was i took the new value that i needed for the shape. And i wanted to show you
that if we're taking the shape of b. We have a two by four and a six by two by one. And we end up with
the result of a six by two by four. You're using this and you're using this. We're taking a three
dimensional object. Adding it to a two dimensional, and ending up with a three dimensional result.
that's what we get from this. That's how we can move a one dimension and two dimension end up
with a three dimension. Starting with a six by one and a two by. It's really confusing. But
i suggest watching this again to see what's going on. Because something like this will probably come
up in your life. The next thing we could take the same thing we had before and we could do element
wise multiplication. Works out and remember these are the outer products. That we're dealing with.
the next thing we could deal with is inner product, with a dot product. Using the same thing. But doing
a transpose on b that i did instead. i could show you that this works as well. but i just
wanted to do a quick rundown so you get some ideas. It was kind of fast. That's why you have
to pause the video go back look at the code see what's going on. This is just a primer. But, i hope
this brought utility to someone. Please like, share and subscribe. And if you subscribe turn on that
notification bell. i would like to say thank you to the recent subscribers and viewers i greatly
appreciate it. Don't forget i started to buy me a coffee. Shameless plug but, i need to. I spend
a lot of time on these videos and if you decide to help out i'd greatly appreciate it. i'm also
on instagram and twitter. Look for those in the link in description below thanks have a good one.
bye