hi there sardar here with infoworld at idg today in smart python i'm going to introduce one of
python's most powerful and widely used third party modules numpy by now you've probably heard how
python is a big fixture in the math and statistics space you've probably also heard that python is
convenient if not the fastest language under the hood so how come people choose python for math
and stats when it's computationally slow well enter numpy this is a third party library that
provides accelerated math operations to python the actual math is done behind the scenes as it were
using lower level languages for speed one uses python as a convenience interface to those methods
so you get the best of both worlds you get python for its ease of use and it's user friendliness
and you get the machine level functionality of numpy for raw speed but there are limits to
where numpy pays off for your work and in this example i'll show you how that operates let's
begin with a simple example this is a version of the sieve of eratosthenes a simple way to find
a range of prime numbers the first function uses pure python to perform the calculations
and uses the set data type native to python to speed things up a bit the way it works
is for each number in the range that we want to calculate we check to see if the number in
question is not already in our existing set if it isn't the function yields up that number and
then changes the set so that any multiples of that number up to our limit are also not to be included
all of this is again done with python's native functionality for handling numbers
the second function uses numpy's library functions to accomplish the same things many of numpy's
functions revolve around matrix math creating and manipulating areas of numbers at high speed
the first thing we do here is we create an area of booleans ones are zeros that is the
same size as the number of the integers that we want to search so for 5000 we created an array
with five thousand members in it we then follow the same practice as before we check to see if
each number is marked as prime and if so mark off all of us multiples as not prime because numpy is
designed to deal with matrices of numbers at high speed this operation happens a lot faster than the
pure python counterpart if we run both of these we can see just how much faster we're talking
the first of these the pure python version takes about two one thousandth of a second to complete
which is respectable the numpy version is at least an order of magnitude faster this is because the
most time-consuming manipulations are all handled internally not in the realm of python itself we're
just using python as a kind of command and control system for functionality that would otherwise need
a language like c plus or fortran to work with and python is far easier to work within those
languages now one key issue with numpy is that it works best when dealing with machine level
numerical types this means numbers that the hardware in your machine deals with
directly numbers that can be represented by 32 or 64-bit data like integers and floats
and with integers these numbers can only be so large or so small and with floats there's
only so much precision you can have with them so there are constraints python on the other hand is
a different way to represent numbers internally one that is not directly compatible with numpy
with python an integer can be unlimited in size it can have as many digits as you want
a numpy 64-bit integer has an upper bound and a lower bound as well so while
python gives you a lot more convenience that convenience comes at the cost of performance
let me give you an example of how this plays out with numpy i have here an example program
that creates a list or an array of the integers one through five thousand it then finds both the
sum and the product of all those numbers in the array the sum of those numbers is a number that's
easily contained in a 64-bit unsigned integer but the product isn't it's a number that has 16
000 digits now python can natively handle that number just fine it just handles it slowly
now if i try to use numpy's built-in product method to compute the product of such an
array i get back nothing that's because the number in question quickly becomes too big
to be handled as a 64-bit integer what's more i can't tell numpy to use a python-style integer
to handle that kind of computation under the hood at least not that way what i can do as a
fallback is tell numpy not to use 64-bit integers but instead use python integers
by way of the object keyword the downside of doing this is that we lose just about all of
our performance improvements for doing so as you can see in this third example the performance
of the operation is about the same for the pure python version and in fact in this fourth example
i use the same code for generating the product as the pure python example and
it's not significantly slower than doing things with the prod method using python objects under
the hood that's because the prod method is falling back under the hood to the python way of working
with numbers the slow way so we're not getting any real advantage from using numpy in this case
so here's the takeaway numpy gives you faster math with numbers chiefly in matrices provided
that they are machine level number types like bytes or 16 or 32 or 64-bit integers or floats
but numpy doesn't give you those improvements in speed when working with python's own native
number types so you have to choose the tool that fits the job that's it for this video if you liked
it please leave a comment below don't forget to subscribe to the idg tech talk channel on youtube
and for more smart python be sure to follow us on facebook youtube and
infoworld.com you