WESLEY CHUN: Hi, I'm Wesley Chun from the Google Cloud team,
joined by my colleague, Martin, to welcome you
to "Serverless Migration Station," where
each video shows developers one modernization technique for one
of our serverless compute platforms.
Today, our little friend, Porter,
will take Python 2 App Engine developers from Cloud NDB
to Cloud Datastore.
This is an optional migration, so you
don't need to make this journey unless you have a good reason
to do so.
MARTIN OMANDER: That sounds good, Wes.
But wait.
My time is precious.
Why is this migration optional?
WESLEY CHUN: Yeah.
That's a very good question, Martin.
Well, legacy Python 2 App Engine developers
have traditionally used App Engine's NDB library
to talk to Datastore.
In the previous module 2 video, we
showed you how to move from App Engine NDB and App Engine
Datastore to Cloud NDB and Cloud Datastore.
That migration had one specific goal--
modernizing App Engine apps means
moving away from legacy services to unbundled services,
because the original services are feature complete.
So all innovation moving forward only
comes with the newer, standalone services.
You can move to Python 3, the next-generation App Engine
service, and that's why this additional migration
to Cloud Datastore is optional.
By the way, it's just app migration.
Since App Engine Datastore and Cloud Datastore
point to the same database in your project,
no data migration is needed.
Review the module 2 video and codelab
to refamiliarize yourself with our starting point.
The links are below, as well.
Let's pause for a moment to let you review before moving on.
Because App Engine Datastore and Cloud Datastore
point to the same database in your project,
one great feature of that migration
is that you didn't have to migrate the data, only doing
the app migration in the video.
MARTIN OMANDER: That's right.
And not only did I watch that video,
I helped you make it, remember?
Anyway, I know how to migrate from App Engine
NDB to Cloud NDB now.
I'm also pretty sure you said, by migrating to Cloud NDB,
I'd be accessing Cloud Datastore.
But didn't you just say we're migrating to Cloud Datastore
today?
If I'm already on Cloud Datastore,
why would you want me to migrate to it?
I'm confused.
WESLEY CHUN: Oh no, no.
It's not that.
Apologies for the confusion.
You're right.
In both cases, we're still on the Cloud Datastore database.
But here, we're talking libraries.
When Datastore matured to becoming its own product,
the Google Cloud team created a new native client library
for non-App Engine apps.
That's also where we point Python 3
and other next-generation engine runtimes.
Cloud NDB is made just for Python 2 App Engine developers.
Today's migration is for those considering that extra step
to going from Cloud NDB to the Cloud Datastore client
libraries.
MARTIN OMANDER: OK, I think I understand now.
You're saying that there are two client libraries to access
Cloud Datastore, one for long-time Python App Engine
users who prefer the NDB style, while the other is
for those completely new to Cloud Datastore.
Is that right?
WESLEY CHUN: Yes.
Essentially, that's right, although Cloud NDB
can use with the next-generation and Python 3 App Engine apps,
as well as non-App Engine apps, meaning Python developers
have two options for accessing Cloud Datastore,
depending on their preference.
Now the real question is why you would consider migrating.
MARTIN OMANDER: Yeah, that's my point.
If both can be used for App Engine and non-App Engine apps,
why should I even consider migrating?
It'll require some effort without much gain,
since I'm not getting any new features
and generally expect no changes to my app.
WESLEY CHUN: Yep.
That is absolutely correct, which
is the main reason why this migration is optional.
If your older App Engine apps uses Cloud NDB to access Cloud
Datastore and newer non-App Engine apps
use its native client library and satisfy that everything's
working, there's really no need to change.
The only reason you would consider it
is if your organization wants to consolidate to a single client
library for consistency shared across all of your apps.
It'll reduce the size of your code base
and the one common library would become robust, because they'll
be used by many apps.
It would be fully-featured, fully debugged,
and increased code reuse.
This makes code across apps easier to manage and lower
maintenance costs overall.
However, if that's not worth your effort, just stop now.
MARTIN OMANDER: OK, I'll bite, Wes.
Let's say I want to do that, or maybe I'll think about it
and I want to at least see what this migration entails.
WESLEY CHUN: All right, sounds good.
OK, everyone, same drill as before.
Copy your module 2 cloud in the app code.
Or if you don't have it, clone the repo
and download the zip file.
The only decision you have to make
is whether you want to migrate the Python 2
version of the app or Python 3.
These are in the module 2a and 2b folders.
You can also migrate the Python 2 version first,
then port it to Python 3, which is how the codelab does it.
We'll show you how to do both in a minute,
but let's pause now so you can go grab the code
and also pull up the codelab so you can get hands-on experience
with this migration.
Let's go to the computer and walk you through it.
It's always good to start with a working app.
From your Python 3 module 2 folder,
run gcloud app deploy to upload to App Engine.
If you're still in Python 2, double
check that Flask and NDB are in lib.
If not or you're unsure, it's fine to delete
the lib folder and reinstall, then deploy.
Your motivation to upgrade to Python 3?
No more manual bundling.
Regardless, when deployed, point a browser or curl to your app
and confirm the output is the same
as from before, whether modules 0, 1, or 2.
OK, now let's migrate to Cloud Datastore.
The best part about migrations is what doesn't change.
app.yaml and appengine_config.py stay as-is.
Same goes for index.html and templates.
For requirements.txt, switch from cloud-ndb
to cloud-datastore.
Also add versions here.
I left them out of the video, since version numbers change
all the time, and also, it's different between Python 2
and 3.
Regardless, the repo will be updated, unlike this video.
Python 2 developers must delete their lib folders and reinstall
now, while Python 3 users can just chill.
When done, we can move on to the main application file.
At the top of main.py, we're importing and initializing
the API client for Cloud Datastore instead of Cloud NDB.
We also have to add a new import for datetime,
because Cloud Datastore doesn't do automatic timestamps,
so we have to do it.
For store_visit(), NDB acts more like an ORM,
or Object Relational Mapper, where you declare a data model,
instantiate an object with the desired data, then save it.
On the other hand, with Datastore,
you create an entity, specifying its kind as a key.
You get back a dict-like object to add to your data
as key-value pairs.
Or in our case, we merge an entire Python dictionary
into it, then save.
There's also no context manager when using Cloud Datastore.
In fetch_visits(), you can see querying is similar,
but Datastore is a bit easier to read.
NDB favors chaining methods together,
but Datastore has a more step-by-step approach,
plus the convenience of getting Python dictionaries back
without having to do additional conversion like NDB.
And that's all that's needed to migrate this sample
app to Cloud Datastore.
Save, gcloud app deploy, and confirm
it still works just as before.
Let's go back to the main presentation.
MARTIN OMANDER: Thanks for showing us
that immigration, Wes.
It was good to see that Cloud Datastore isn't
that different from Cloud NDB.
WESLEY CHUN: Of course, Martin.
It's true the code is similar, but this is a sample app.
In real life, there are likely more challenges, especially
related to complex queries.
So I've linked to the queries pages for both libraries
so you can see more differences between them.
MARTIN OMANDER: Well, I'm doing the codelab on my own time.
I might need to see the Cloud NDB and Cloud Datastore docs.
You got those links handy?
WESLEY CHUN: Yep, I thought you'd ask.
Here are the links to both the Cloud NDB and Cloud Datastore
docs, as well as to their client library repos.
Finally, we linked to the App Engine Datastore docs.
Don't forget the repo has both the start and finish folders,
so you can reset back to the 2a or 2b code
if you take a wrong turn during the codelab
or want to compare your answers with ours
in the module 3a or 3b folders.
By the way, if you're still wondering
whether to choose Cloud NDB or Cloud Datastore,
here are our recommendations.
Creating a new app in Python 3, App Engine or otherwise?
Go straight to Cloud Datastore.
Got a Python 2 NDB App Engine app?
Migrate to Cloud NDB, then port to Python 3.
Lastly, if using Cloud Datastore elsewhere
and want a consistent codebase using only one client library,
consider this migration.
If not, don't bother, because Cloud NDB
works in both Python 2 and 3 with App Engine
or outside of App Engine.
So there's really no need to move off
unless you have Cloud Datastore code elsewhere.
MARTIN OMANDER: Ah.
Thanks, Wes.
That was a really good explanation.
It helps me make my decision.
By the way, on another note, I also
heard that Cloud Datastore was renamed.
Is that right?
WESLEY CHUN: Yes, Martin.
It's a bit more than that.
And I didn't want to distract users from the main content.
But yes, it's true.
The next-generation datastore service launched in 2017.
As it inherited some features from the Firebase Realtime
Database, it was rebranded as Cloud Firestore.
All existing Datastore databases have
been upgraded under the covers to Firestore in Datastore mode.
For new applications, consider using the Cloud Firestore
in native mode.
We've given you a link to learn more about when you would
choose one over the other.
Neither the Cloud NDB nor Cloud Datastore libraries,
however, can access those Firebase features.
You do really need to use the Cloud Firestore client library.
If you have an app, App Engine or otherwise, and at some point
down the road want to take advantage of those Firebase
features, consider further migrating to Cloud Firestore.
If interested, check out module 6.
MARTIN OMANDER: Oh, that's pretty interesting.
I didn't even know about Cloud Firestore
and I will take a look at that module.
That sounds like another "Serverless Migration Station"
episode we can work on together?
WESLEY CHUN: That'll be fun for sure.
What other migrations should you consider for your App Engine
app?
If your app uses the original App Engine task queues,
consider migrating to Cloud Tasks
and check out modules 7, 8, and 9.
If you're curious about containerizing your apps,
check out the App Engine to Cloud Run, modules 4 and 5.
Finally, if your average app is simple, like our sample app,
consider migrating to Cloud Functions.
More on that in module 11.
MARTIN OMANDER: Sounds like a lot of options for our users
out there.
Happy to be working with you to help guide our users
and show them best practices, Wes.
WESLEY CHUN: Sounds good, Martin.
And that wraps today's optional migration to Cloud Datastore.
On behalf of Martin and Porter, this
is Wesley Chun from Google Cloud,
and we look forward to seeing you
at the next "Migration Station" or on another "Serverless
Expedition" soon.
[MUSIC PLAYING]