WESLEY CHUN: Hi, and welcome to Serverless Migration Station,
a miniseries dedicated to helping developers modernize
their apps running on serverless compute platforms like App
Engine.
This is Wesley from Google, joined by my colleague
Martin, and Porter the Train, to help
you get to your destination.
MARTIN OMANDER: Glad to be here, Wes.
Where is Porter taking us today?
WESLEY CHUN: Well, in module 17, we showed developers
how they can upgrade runtimes, like from Python 2 to 3,
Java 8 to 17, and so on, but continue to use App Engine's
legacy bundled services.
MARTIN OMANDER: Ah, I do remember that episode.
So instead of migrating away from those bundled services,
you showed us the code tweaks necessary for Python 3
apps to access those bundled services.
Right?
WESLEY CHUN: That's right.
I also mentioned that most runtimes like Java, Go, and PHP
don't require code changes-- only Python,
and that for most bundled services,
code changes are minimal with some exceptions-- which
brings us here to module 22, to look
at bundled services requiring more code
changes than the others.
MARTIN OMANDER: That's good.
I don't want to leave those users hanging.
By the way, what are those bundled services
that require more code changes?
WESLEY CHUN: So here's the list of which
bundled services are available in the second gen
runtimes like Python 3.
And the ones we're looking at today
are Blobstore, Mail, and Deferred.
MARTIN OMANDER: Got it.
Now why are these services so special
that they need extra code changes?
WESLEY CHUN: That's a good question,
and the answer has to do with their legacy.
In one way or another, it involves a part
of App Engine that changed between the first and second
generations, whether it's a dependency on the web app
or web app 2 frameworks that are not available in Python 3,
or parts of app.yaml that are no longer supported.
MARTIN OMANDER: Ah, that makes sense.
I can't wait to see the code changes.
But I'm curious if more information is
available in the documentation.
WESLEY CHUN: You just read my mind.
Our documentation features pages specifically
dedicated to these three services
and how to use them in Python 3.
They're also linked below in the video description
for your convenience.
But today, we're diving in a bit deeper than what's
featured in the docs.
MARTIN OMANDER: And you told me earlier that the code samples
are different than the ones featured
in previous videos in the series.
Why is that, Wes?
WESLEY CHUN: Well, that's right.
So the code samples today are different in that they're not
based off of the module 0 or 1 baseline sample apps
like the others.
The main reason is that there are no migrations other
than porting from Python 2 to 3.
So we're going to go to the computer
where I'll show you shorter, but still complete, sample apps,
put up the Python 2 and 3 versions up against each other,
and then highlight the differences between them.
With that, let's start this conversation off
with Blobstore.
In module 15, we showed you how to add Blobstore usage
to the module 1 sample app to allow
visitors to upload a visit artifact, like an image.
Instead of migrating to cloud storage
like we did in module 16, let's take a shorter Blobstore sample
and show how to port it to Python 3.
Today's module 22 basic Blobstore app
lets end users upload a photo.
Pick something to upload with a file picker.
Once selected, click on the Submit button
to make it happen.
If all goes well, you'll see the image you uploaded.
The app is fairly basic, as it doesn't provide for downloads
other than performing the one download to confirm
that your upload succeeded.
Now let's compare the Python 2 webapp2 version
to the Python 3 Flask version starting with app.yaml.
The updates to app.yaml are similar to those we've already
seen throughout this series.
Change the runtime to a supported Python 3 version--
today, that would be 3.9 or 3.10--
and delete everything else.
To use bundled services in Python 3,
add an app_engine_apis directive set to true.
Finally, Python 3's pickling protocol
is backwards incompatible to Python
2's, so add this environment variable,
telling App Engine to use a lower level protocol
that both understand.
This is needed for Python 3 apps that use App Engine
NDB to talk to Datastore.
You'll see this environment variable
in all of the Python 3 app.yaml files today.
Migrating away from webapp2 is required to move to Python 3,
so we arbitrarily chose Flask.
Webapp2 apps run almost exclusively on App Engine,
but Flask apps run on App Engine and most other
hosting services, increasing the portability of your app.
To use Flask or other wsgi frameworks,
create a requirements.txt file specifying third party
packages, namely Flask and the App Engine SDK,
the latter of which we enabled in app.yaml a moment ago.
That's it for config.
Moving to main.py, replace the webapp2 import at the top
with what we need from Flask.
Also import the App Engine SDK wsgi
wrapper to access bundled services in Python 3.
Blobstore functionality has been moved out
of the original webapp framework into the App Engine SDK,
so swap the import of its handlers
from webapp to the Blobstore component of the SDK.
Next, instantiate the Flask app and wsgi wrap it.
The photo upload data model stays identical,
containing just the blob key for the uploaded image file.
The photo upload handlers base class
now comes directly from Blobstore instead of webapp.
And finally, so the user can view what they uploaded,
switch the redirect style from webapp2 to Flask.
Flask routes to functions while webapp2 routes
to handler classes.
So to do this with Flask, a new upload_photo function
is added to handle the POST request.
It then instantiates the handler class
and calls its POST method.
Now do something similar with the viewPhotoHandler.
Switch the base class from webapp to Blobstore
and try to get the blob key.
If successful, send the blob to the user.
An HTTP header for the content type
is set so that App Engine can provide its best guess rather
than taking Flask's default.
And like the upload handler, we need
to create a function for the Flask route
which instantiates the view handler
and calls its get method with the images blob key.
Finally, replace the webapp2 main handler class
with a Flask equivalent.
Flask functions are route decorated,
so the webapp2 routing dispatch table isn't needed,
so remove it.
And that's it for Blobstore.
Feel free to try either the Python 2 or 3 versions
of this app, or both, using the usual gcloud app deploy
command followed by visiting the app in a browser window.
You'll find that both apps work identically.
Moving on to Deferred-- if you're new to this service,
it's a higher level abstraction for spawning extra work
to run later.
It uses the default push task queue
but doesn't require you to know how
to set up or use push queues.
All you do as a developer is specify a function
to run later, along with required parameters.
The sample app is just a basic auto incrementing counter
starting at 0.
If you visit the app, it'll go up by one.
Hit it again, it's now a 2, and so on.
Switch the runtime in the app.yaml file from Python 2
to a current Python 3 version like 3.10 and delete everything
else.
The handler for Deferred comes on by default in Python 3,
so you don't need to specify this in the built-in section
like with Python 2.
Add the App Engine SDK directive and pickling protocol
environment variable.
As far as the app goes, the updates
will mostly be familiar.
Replace webapp2's import with Flask
and add an import of the wsgi wrapper.
Instantiate Flask like usual, then call the wsgi wrapper.
But one difference here is to pass
in the use_deferred parameter with a true value
to enable the Deferred service.
The basic counter and its increment function
stay identical.
Now replace the webapp2 mainHandler class
with a Flask mainHandler function.
The call to defer execution of the counter bumper
is identical on both sides.
Last but not least, remove the webapp2 routing table.
As with Blobstore, feel free to deploy either Deferred app
and discover that they both work identically.
The most interesting one today is email.
Taking a step back, the mail service
gives App Engine apps the ability
to send or receive email.
Sending email stays the same, meaning
there are no code differences upgrading from Python 2 to 3.
What is different is in receiving email.
Now you may be thinking, yeah, sending email, I get it.
But receiving messages means your users are emailing an app.
And what can you possibly do with that?
Boy, have I got a video for you.
I made this one a while back, showing
how you can change the world in just 10 lines of code
supporting inbound email.
Check it out for your next startup idea.
The basic app feature today saves the last email
it's received.
When visiting the app, it will display that email message
along with expected metadata like the sender, time stamp,
and subject line.
Now let's take a look at the app.
In app.yaml, replace the runtime with the current Python 3
version, like 3.10, and delete almost everything else,
keeping the inbound services section.
Add the App Engine SDK to access the bundled services
and the pickling protocol environment variable.
In the main app, import the Escape function.
In Python 2, it's in the CGI module,
but it got moved in Python 3 to the HTML package.
Now replace webapp2 with Flask.
Like with Blobstore, import the mail handlers
from the mail component of the App Engine
SDK instead of webapp.
Lastly, import the wsgi wrapper.
There are constants for the entity key string
and the email headers we're interested in.
The email body is also important,
even though it's not in fields, but we
need to do some processing to get it.
So it's not as simple as just copying a string
like the other ones listed.
The final constant is the HTML template
for displaying the last received email with.
Below that, instantiate the Flask app and wsgi wrap it.
The Python 2 EmailHandler class and its
receive method are replaced with a Flask mail handler function.
When an email is received, the App Engine system posts
the message to your app at the /_ah/mail endpoint,
including the receiving email address.
And the post handler calls the receive method
with that message.
In Python 2, the MailHandler class extracts the message
and gives it to you by the time receive is called.
But in Python 3, you have to instantiate
the message yourself by passing it the Flask request payload.
The last_msg Datastore entity is fetched or created
and the basic fields filled in.
Getting the email body requires extra processing.
The message components are encoded.
So to keep it simple, grab the first plaintext chunk found,
decode it, add it to the entity, then save to Datastore.
Like the other apps, the MainHandler
class and get method are replaced by a Flask handler
function.
It fetches the last saved message entity,
turns it into a hash or Python dictionary,
then passes the values to the template,
HTML escaping the values.
Finally, remove the webapp2 routing table, and that's it.
Like the others, deploy either Python
2 or 3 versions of this app, then email your app.
And now you've got that next big thing that goes viral.
For today, our journey is done.
Now you know how to use the Blobstore, Deferred,
and mail bundled services in both Python 2 and 3.
MARTIN OMANDER: That was a useful demo
of using those special bundled services in both Python 2
and 3.
While they needed a few more code changes
than the other bundled services, it
seemed fairly straightforward.
And I like that I can upgrade my app from Python 2
to 3 without having to migrate off the bundled services
and without having to rewrite all my code!
WESLEY CHUN: That's right, Martin.
It's good to be able to minimize major rewrites, which
makes it less likely you'll break something.
On the other hand, one challenge is that your app can only
run on App Engine.
MARTIN OMANDER: Right, you did mention last time
that moving off the bundled services
does make your app more portable--
meaning developers can consider moving to modern serverless
platforms like Cloud Functions and Cloud Run, right?
WESLEY CHUN: That's right, Martin.
Cloud Functions is a good choice if you
wanted to break up a large App Engine app
into multiple microservices.
And if containerization is now part of your software
development workflow, there's Cloud Run,
which you can use even if you know nothing
about Docker or containers.
Finally, also consider VM-based compute platforms
like Compute Engine VMs or GKE, or managed Kubernetes service
if you want even more control like being
able to SSH into servers.
So while it is more effort to making your apps more portable,
it does extend your app's capabilities in the long run.
MARTIN OMANDER: Yes.
And if our viewers do decide to migrate off
the bundled services, they can take
advantage of all the Serverless Migration videos
we've already done.
WESLEY CHUN: That's right.
And we've got plenty of migration models
to help you move to Cloud Storage, tasks,
Pub/Sub, Datastore, Memorystore, and Identity Platform.
We also have content if considering
moving to one of our serverless 2.0 platforms,
like Cloud Functions or Cloud Run.
Or we have content for those who already know
Docker as well as those who are new to containers.
This wraps up our part 2 content on using bundled services
in Python 3.
Be sure to review module 17 if you missed part 1, which
includes special guest Aaron, who's
crafting a parallel series for Java App Engine developers.
MARTIN OMANDER: Wow, we've certainly
built out a whole rail system of different migrations
to help App Engine developers modernize their apps,
haven't we, Wes?
WESLEY CHUN: Yes, we have.
And on behalf of Martin and I, thanks
for staying with us on all of these modernization
journeys, where Porter takes you to where you want to go.
We hope to see you at the next Migration Station
or on another Serverless Expedition soon.
In the meantime, happy travels.
[MUSIC PLAYING]