hi there serdar gagalow here for infoworld at idg in this episode of smartpython i'm going to
show you how you could create a standalone python executable with cython which compiles python to c
but i'm also going to use this as a general discussion of why it's tricky to create
standalone executables from python programs because of the way the python ecosystem works
first a few words about cython this is a third party package that takes python code and turns
it into c and which can provide major speedups for math operations if you use type annotations but it
was originally devised to make it easier to write python extension modules instead of writing them
in c which can be cumbersome one can write them in python and just compile that to c i've created
an earlier video that works as an introduction to psython and i've linked that in the article
description now normally psython creates a library an so in linux or a dll in microsoft windows but
it's also possible to create an executable an exe in windows which is what i'll use for this
demo if you view certain options when you run psython the generated c code will include an
executable entry point which you can then enable by using the right compiler and linker options
to demonstrate this i'm going to put together a simple sython module that will run as an
executable this module does two things it prints a couple of messages to the console using python's
own print command and it invokes a message box using win32 api this last is a little easier
to do in cython than in python because python has more native provisions for working directly
with system and library function calls to compile this we need to do two things first we need to run
cython on the module we can do this by invoking cython from the command line the important thing
is that when we do so we have to use the embed switch this will generate in the resulting c code
the functions needed to allow the code to run as an executable instead of a linkable library
next we run the microsoft compiler and linker on the generated file
recent builds of microsoft visual studio allow you to download the command line build tools
including microsoft c compiler without having to download all of visual studio to go with it
so we'll be using the command line build tools to compile this project
i've created a little batch script here that does all of this for us it will set up the visual
studio environment variables and run the cl.exe compiler and linker with all the needed options
when we run the linker by the way we have to link the executable against the user 32 and kernel
32 libraries and against python's own library so once we do this we will have an executable
named main.exe in our directory and if we run that we get a printout to the console in a dialog box
it all seems easy enough right but in fact this is deceptive if we take this executable
and copy it to another computer it won't run at all what we've done here is not nearly enough
for many reasons first let's go back to the way psython works psython builds python extension
modules these are libraries loaded by the python runtime in other words cython modules
including those built as executables won't run without the python runtime present plus
our example used python's own print function which would require the python runtime also
the whole reason this seemed to work before is because we had the python runtime in this case
python38.dll on our system path to demonstrate this i'm going to open a command session and set
the system path to a blank now when i run the executable you'll notice that nothing happens
but if i start another command session with the original path where python3.dll is available our
executive magically starts working again so if we want to make a standalone python executable with
cython we have to bundle the python runtime with it but this isn't as simple as just
copying the python dll we also have to include a few other bits that allow the python runtime
as a whole to initialize correctly this often includes python's standard library
this is important in cython's case because the psython compiler treats python imports in a script
anything that you would import during the course of a syphon generated application
it treats those things exactly the same way a regular python script does in other words
it uses python's runtime import mechanisms to bring those files in
so those files need to be present at runtime there's no provision in python for automatically
bundling imports into the compiled extension module now python's creators have a way to deal
with needing to have an entire python runtime available if you go to the python site and look
at the downloads section you'll see one of the options for windows is an embeddable package
this is a zip file that contains a minimal self-contained distribution of python
including a copy of the standard library i've unpacked a copy of the redistributable into
a subdirectory of this project folder and i've called it embed if i copy our main.exe into it and
then open a command session in that folder with no path i can run main.exe on it and it will work
the downside of this approach is that the redistributable here is at bare minimum about
15 megabytes uncompressed it's around half that if we compress the whole folder into a zip file
on the plus side all we have to do is give someone else this bundle let them unpack it somewhere and
run it and it'll work side note in windows if you download a zip file that has an executable
somewhere in it you may need to right click on it select properties and check the unblock option to
be able to unpack it successfully if we want to run our psython executable we will need at least
some of the python runtime but how little of the python runtime can we get away with having
well to answer that question i wrote a custom build script
this compiles our file and then places it in a directory along with a copy of the python runtime
and an extremely minimal copy of the standard library just enough to get things running with
no guarantees about what other packages we might need i've made this script available in a github
repo it's linked in the video description so you can try it out too now when we run this script it
builds our executable it adds the python dll and the visual see runtime dll which we often need
and we'll compile a few modules from the standard library into a zip file to go with it
so if we open this directory and launch a command session with no path initialized it'll
run and this whole package comes to about four megabytes total or two megabytes when compressed
well that's not bad at all but we're still missing a major ingredient of the way python's ecosystem
works in which psython's compilation process can't account for it's what happens when we
use third-party libraries for an extreme example i'm going to turn to a library
that's a popular python item but is not svelte at all i'm talking about numpy with all of its
code it weighs in at some 50 megabytes and that's because a big part of it is a single binary blob
the fortran libraries that it uses at its heart so it's really hard to make it smaller
here i have a copy of our cython project which i've modified to import numpy
and print out some information indicating numpy's been loaded and ready for use if i compile this
with our script and try to run it it can't perform the import and it crashes fortunately i had the
sense to wrap the import in a tri-accept block so we can get a good error message as to what's wrong
now let's take our compiled executable and copy it into the embed folder
next we're going to go and take the copy of numpy that's installed in our virtual environment
and make a copy of that into the embed folder as well again numpy is not small and
it's difficult for us to do anything about that without recompiling it from scratch
but once we've got it all copied in there all we have to do is run our executable
and there you have it we now have numpy available as a library in our self-contained python app
so to sum up you can use psython to create standalone executables from python scripts but to
create a deployable redistributable application is much harder it requires having many other bits of
the python ecosystem present and you can get most of those by reusing the embeddable redistribution
but you'd still need to provide any third-party dependencies on your own finally as a closing note
a number of projects exist that make it easier to bundle python scripts as redistributable
applications the most popular of these pi installer i covered in a previous video an article
which i've linked in the description and it too creates fairly large redistributable packages
and after this walkthrough of the way things work it's probably easier to understand why the python
ecosystem was never created with the scenario that scripts could be made into standalone apps
but now there's growing pressure to make that a standard feature in python of some kind so we'll
see how this shapes up in the times to come that's it for this video if you liked it please leave a
comment below don't forget to subscribe to the igg tech talk channel on youtube and for more
smart python and more dev with serdar be sure to follow me on facebook youtube and inforw.com