[MUSIC PLAYING]
WESLEY CHUN: Hi, and welcome to Serverless Migration Station,
where we show Google Cloud users one
step they can take to modernize their serverless apps.
I'm Wesley, joined by my colleague
Martin and our little friend Porter to take you from point A
to point B.
As a long-time App Engine developer,
you're probably aware of using its default database, the App
Engine Datastore, via its NDB client library.
To make user apps more portable, the next generation App Engine
service has moved away from bubble services like Datastore.
Since Datastore has matured into its own product, Cloud
Datastore, we're here to help you
take the key step of migrating from App Engine's and NDB
to Cloud NDB so your app can basically use the same NDB
code, but accessing Cloud Datastore instead.
MARTIN: My app uses Datastore quite a bit,
so I'm interested in this.
How do I get started with this migration, Wes?
WESLEY CHUN: Well, in the previous video for module 1,
we migrated our base LAN webapp2 NDB Datastore app to Flask,
a popular framework in the Python community.
Today we pick up right where module 1 left off,
a Flask NDB Datastore app.
And we're going to migrate that to a Flask Cloud NDB Datastore
app.
MARTIN: Ah, yes.
I remember that migration.
But I forgot where the module 1 code is.
Can you point me to it again?
WESLEY CHUN: Definitely, Martin.
So go to the Migration Module video, and clone it
or download the zip file.
Its link is also in the video description below.
We'll start with the module 1 code, perform the NDB migration
steps, and voila, that will result in the module 2 code.
Since this app uses only one App Engine service,
after moving to Cloud NDB, you can easily
port your app to Python 3.
So the code lab that features that bonus migration.
That's why you'll also find a pair of module 2 repo folders,
one each for Python 2 and 3.
But hold on, partner.
Let's do the Cloud NDB migration first.
So hit pause now to go get the module 1 code.
OK, if you're ready, let's walk you through this migration.
As in previous modules, let's ensure
we have a working app before migrating.
From your model 1 folder, check you've got Flask in your lib
folder, or run the pip install command if you don't.
Upload to App Engine with gcloud app deploy,
then once it's live, point a browser or curl to it
and confirm the output's the same for module 0 and 1.
Next, we're ready to migrate to Cloud NDB, this time starting
with the config files.
Any time you use a Google Cloud client library in Python 2,
you need a pair of third party helper libraries.
Fortunately, they're already on App Engine servers,
so do not add them to requirements.txt nor use
pip install.
Built-in libraries like this are declared in app.yaml.
Here, create a new library section
with grpcio and setuptools.
If your app's app.yaml has a library section,
and these helper libraries, if they're not already there.
The good news is that third party packages
are way easier in the next gen, where
it's just requirements.txt.
In the module 1 video, we explained
you need to tell App Engine where
to look for your self bundle or copied third party libraries.
That's done in appengine_config.py.
It's also true of built-in third party libraries,
like grbcio and setuptools, but requires a different code
snippet in App Engine config, where you reference them
to lib in the setuptools.packageresources
working set, just like you see here.
This App Engine config can now handle
both built-in and copied third party libraries.
So this is probably pretty good boilerplate
to have for all of your Python 2 App Engine apps.
We mentioned earlier how third party package management
is way easier in Python 3, and not even having an App Engine
config file is part of that.
Next are the third party packages
we do need to copy or bundle.
In module 1, we created requirements.txt
with a single reference to Flask.
Now add the Cloud NDB library spelled google-cloud-ndb, all
lowercase with hyphens in between.
When making this video, Flask was on version 1.1.2
and Cloud NDB was on 1.8.0.
Since the repo is actively maintained,
you may see newer versions there.
Now delete any old lib folder you
may have and run the pip install command
to get those libraries in there so we can upload to App Engine.
That's it for config.
Let's jump into main.py, starting with the imports.
It's a one-for-one swap to switch from App Engine
NDB to Cloud NDB.
As with most Google Cloud client libraries,
you'd need to create a client to access the API.
So that's the new line you see.
The best news is that most of your NDB code stays as is.
The only difference is that it's more idiomatic Python,
requiring use of the client's context manager.
If you're new to those, it's just
saying that you need to acquire those resources before you
can use them.
You'll typically see with statements with files,
because you've got to be able to open a file
before you can read from or write to it, right?
So add a new with block and put all of your Datastore calls
inside, like what you see here.
Everything else is the same, making
this migration fairly painless.
One more tip is that transactional tasks isn't
a feature in Cloud NDB yet, meaning you
can't create tasks only if the transaction succeeds.
In other words, task queue calls won't
get rolled back if the transaction fails,
only Datastore operations.
Ignoring the readme, all other source files stay unchanged.
Index.html template file, gcloudignore, and really,
that's it.
So that's the entire migration to Cloud NDB.
Before we go back to the presentation,
let's talk briefly about a Python 3 port.
The language updates you have to make may be considerable.
If there's any good news on the App Engine side,
it mostly involves deleting stuff,
like deleting a bunch of lines from app.yaml,
deleting the lib folder and appengine_config.py file.
Our super simple app is already Python 2/3 compatible.
So the only file that needs updating is app.yaml.
Change the language version.
Now, that's obvious.
Next gen App Engine requires a web framework
that does its own routing.
So you have three options here.
Set all script handlers to auto, because the framework
does the routing now.
Two, delete all script handlers, because if everything is auto,
they're all meaningless, right?
Or three, replace handlers with an entry point section
if you have specific startup requirements.
Read more about these options in the docs.
Note, I only said script handlers.
Static file handling stays as is.
Lastly, I've already said third party package management
in the next gen is way simpler, allowing you to delete
the library section entirely.
Again, the real work for your app is the language update.
OK, back to the show.
MARTIN: Wow.
That was a pretty straightforward migration, Wes.
I'm glad to see most user code doesn't have to change.
I have one question, though.
What if I have an older app, say a Python 2.5 app?
The 2.5 runtime was deprecated in 2013,
and then it was completely shut down in 2017
when Python 2.7 took over.
So my app's been dead ever since.
Can I somehow bring it back to life?
WESLEY CHUN: The answer is yes, Martin.
But there is a catch.
You have to migrate from 2.5 to 2.7
and DB to NDB first before applying the migration we just
discussed.
Check out these Python 2.5 DB docs as well as the NDB
migration guide to get started.
MARTIN: That's great.
Are there docs for NDB too?
WESLEY CHUN: Well, here are the links to App Engine NDB
as well as its library repo.
We'll also post them in the description below.
MARTIN: Thanks for those DB and NDB resources.
Can we round out this trifecta with the Cloud NDB docs?
And while you're looking that up, why
is it so important to migrate to Cloud NDB anyway?
WESLEY CHUN: Of course, Martin.
Here are its library and its docs links.
As far as why it's important, you
know that modernizing your app and making them more portable
is one goal of the latest App Engine platform.
For Python developers, the key reason
is that Cloud NDB works in both Python 2 and 3, which
means that once you're on Cloud NDB,
you can finally upgrade your app to Python 3.
And with that, you've crossed the datastore finish line
literally.
This is the last of the required migrations,
as far as the Datastore goes.
It's not necessary to migrate further
to Cloud Datastore's native client library nor Cloud
Firestore unless you've got specific reasons to do so.
MARTIN: OK.
What would be some of the reasons
I'd consider those optional Datastore migrations?
WESLEY CHUN: Well, using the official Cloud Datastore client
library means moving away from everything you know about NDB
and how it works.
The primary reason to do this is if you have
other apps that already use it.
For example, Python 3 App Engine apps or non App Engine apps.
And you want to consolidate so that you're not
using two different client libraries
to talk to the same Datastore.
It simplifies and reduces your maintenance costs too.
Anyway, if you want to look into it,
see the module 3 code lab and video.
The next generation of Cloud Datastore is Cloud Firestore.
Based on the name change, you can probably
guess that it now has some features from the Firebase
Realtime Database.
And you'd be right.
This migration is for those who must
have some of those Firebase features for their apps.
Check out module 6, if interested.
MARTIN: OK.
So if those other Datastore migrations are optional,
where can I go now?
WESLEY CHUN: Yeah, great question, Martin.
Well, users should continue to migrate from App Engine's
bundled services to standalone services outside of App Engine,
like Cloud NDB.
What other App Engine services and APIs are you using?
If you use push task queues, consider
migrating to Cloud Tasks, and see module 7 to get started.
If your organization is consolidating apps
into containers for flexibility, CI/CD, or consistency,
consider migrating to Cloud Run.
If so see, modules 4 and 5.
Finally, it goes without saying that porting to Python 3
should always be a consideration, since Python
2 has been sunset.
We add new migration modules periodically,
and most will have videos too.
So keep checking back.
MARTIN: Cool.
What other migration resources do you have for Cloud NDB?
WESLEY CHUN: Well, the best place to look
is in the docs for Cloud NDB, especially its migration guide.
The code in those docks has its own repo,
so here's a link to that.
And finally, check out the last link
that points to migration samples contributed by the community.
On behalf of Martin and Porter, this is Wesley Chun.
Thanks for tuning into this episode of Serverless Migration
Station and allowing us to show you different ways
to modernize your apps.
And we'll see you the next time on another Serverless
Expeditions.
[MUSIC PLAYING]