foreign
here for infoworld in this episode of smart python I'm going to demonstrate how
two popular python packages for accelerating math operations can work well together numpy and Numba
numpy should be familiar to most everyone who uses python for Math and statistics work it's the chief
go to library for working with Matrix math at high speed number however might not be as familiar
it's a library for accelerating python code using a just-in-time compiler designed mainly
to speed up numerical code that's normally slow in Python now in a previous video I demonstrated
number by itself now this time around I'm going to show you number and numpy working hand in hand
for this demo I'm going to use a slightly modified version of some code that I used in a previous
video where I showed you how you can accelerate numpod using cython in that video I took two numpy
arrays and I generated a third one from them by iterating through the arrays element by element
the actual iteration is what's slow if we do that in Python all the performance benefits of numpy
are lost but if we iterate in cython we speed things up drastically although at the cost of
using cython's special syntax to decorate our code and that syntax can be counter-intuitive
so in our new version of the program we're going to take what we're formerly The cython Accelerated
bits and we're going to bring them back into regular old python as you can see from taking this
code and running it it takes several seconds for a plain python version of this code to complete
now the idea behind number is that it takes in regular python functions analyzes them at runtime
and translates them into machine native code that's much faster the key here is
the runtime analysis because number sees what's actually happening at runtime you don't have to
provide type decorations or other hints to get a performance boost you can just write Python and
let nuba sort everything out so how do we unleash this magic on this code well all we have to do is
add numbers jit decorator this takes in a function we want to accelerate and transforms it to get the
best possible speed however the Transformations for maximum speed only happen if we're using a
safe subset of Python's functionality now this code does not do anything terribly exotic it's
just looping over the two arrays performing a transformation on the values found there and
storing the results in a third array all of this is stuff number can accelerate without any problem
now number is smart enough to recognize that when we work with numpy objects they're not generic
python objects Namba understands that we can work directly with the underlying raw Matrix data
things that are exposed to What's called the python buffer protocol
and to get the best part of this is you don't have to do any additional work to get the speed up
it comes free with number number can recognize that the raw data matrices
are there and work directly with them all the functions that numpy
exposes get a speed up without us needing to annotate types or do anything else like that
also as an aside number can recognize things like when a function can be inlined for better
performance in this case we have the function clip and this function is only called in one place
so it makes sense to inline this to take the code in question and place it directly at the call site
this is something we might normally have to provide hinting for in cython number can
account for it automatically now once we have these decorators in place we can rerun the code
and we will see a massive speed up we've gone from about five seconds to about half a second
but actually this isn't even the full speed up because the first time you run a number
decorated program the performance results are going to be skewed because of the time
that it takes for numbers compiler to warm up and generate code so as you can see from
subsequent runs of the same program we're now much faster we've gone down to 0.02 seconds
now you'll note that with our decorators for the for the compilation there's an argument
cash equals true and what this means is that number preserves the compiled code for each
function whenever possible so whenever we rerun this code assuming that there
have been no changes to it it's not going to be recompiled and that further reduces startup time
now when I tried out timing this code against my cython compiled version of the same code
I found in my dismay that numbers version came out somewhat slower or so I thought
see when I ran this code with scython the time to complete was .008 seconds versus numbers 0.02
but the reason number came out slower was not because the performance sensitive part of the
program is slower it's not because this inner loop here isn't being optimized it's because with
number there's overhead involved in setting up the compiler and getting results from it when we first
run the program and that overhead tends to be a one-time thing per program run not once per call
so the benefits of the speed up far outweigh the drawbacks in this case so to sum up the
benefits of using a number with numpy are that you could write idiomatic python code with your
numpy objects and speed them up by just applying a decorator you can do things that would normally
kill performance like iterating over a numpy array manually in Python and still get good results
and you don't need to do any additional work to make a number recognized numpy Library functions
and accelerate their access you could just use them as is so that's it for this video if you
liked it please leave a comment below don't forget to subscribe to the tech talk Channel on YouTube
for more smart Python and more Devastator be sure to follow me on Facebook YouTube and infoworld.com