Let's now discuss how you can make your App Engine
application more performant and cost less using Memcache.
So far, you've learned a lot of things about Google App
Engine, including the architecture, the services,
the datastore, and lots of APIs.
If you say that those features are like the bread and butter
of App Engine, then you can also say Memcache is like
coffee for App Engine.
Your App Engine application can be very functional
without, just more performant using Memcache.
That's why Memcache is such an addictive feature for a lot of
developers.
Just like caffeine has side effects, however, using
Memcache has some caveats and limitations.
You must use Memcache with care.
Here are the topics we're going to cover in this lesson.
First off, I'll talk about what Memcache is.
What are the general uses?
And then I'll talk about some specific benefits of using
Memcache, and to highlight why we need it.
Then I'll cover how to use Memcache.
I'll talk about the Java and Python APIs for your
application to access Memcache.
Then we'll move to some more advanced topics.
We'll talk about some caveats with using Memcache and what
the general solutions will be.
What is Memcache?
Simply put, Memcache is an in-memory
key-value pair datastore.
The datastore is sitting in memory.
That's why it's much faster compared with other datastores
sitting on disk.
It's also very easy to use.
The only thing you can put in Memcache are key-value pairs.
So you put a value into the Memcache associated with a
key, and you expect to get that value back later on.
You can actually put anything that is serializable into
Memcache either as a key or as a value.
It's really kind of language independent.
In both Java and Python, you can put in any object with its
serializable interface.
In Python, the object has to be serialized for using the
pickle module.
Watch for some future enhancements on the roadmap to
support non-serializable objects in the near future.
You could almost put anything into Memcache.
In general, there are only two major use cases, however.
The first use case for Memcache is to do caching.
For example, you can cache in the datastore query results,
or some user authentication tokens, or session data.
One typical example is a URL fetching.
You can actually cache the content of
the page into Memcache.
You can also cache some other computational results.
The second use case is to share data across different
application instances.
One typical example is an application level counter.
Because multiple instances access the same counter, you
have to pay attention to race conditions, for example.
Fortunately, Memcache actually provides the mechanism to help
you to implement the counter very easily.
Why do we need Memcache?
You can use datastore to share data
across different instances.
You can even use datastore to do the caching of
the counter API call.
So what kind of benefits will Memcache bring to us?
We can summarize this into two points.
The first is to improve application performance.
The second we don't talk enough about, is that it can
actually reduce your
application's operational cost.
Unlike most technologies, you can actually obtain both
benefits together.
Let's look at performance first.
Notice on the left here with a simple data store query that
it takes about 100 milliseconds in App Engine.
But for a Memcache read on the right, it takes less than 10
milliseconds.
Therefore, Memcache is actually more than 10 times
faster than the data store query.
If a request is already in Memcache and finds the
results, there's no need to go to the datastore.
This means the request and the response
can be 10 times faster.
If you have a very good hit rate from Memcache, you can
experience a huge performance improvement for your
application.
How about from a cost point of view?
Zero.
That's simple.
It's free.
But for the datastore query, you have to pay for it.
For API calls, you have to pay for it.
For your application, that CPU memory, you
have to pay for it.
So think about this--
if 90% of the requests can be handled by Memcache, and
there's no need to go to the datastore or do the API call
or do the computation, that could be a huge savings for
your application.
That will make your manager very happy about it.
So how do we use Memcache?
There are different ways to use Memcache depending on the
language you're using.
If you're using Java, you can use a high level JCache API,
which is actually a Java standard for caching services.
App Engine implements most of the JCache API, so people who
choose to use it are mainly interested in application
portability versus using a different caching service.
But App Engine is only one caching service.
For a lot of developers, they actually choose to use the
lower-level Memcache APIs.
The reason is, you can access some extra functionality
provided by Memcache, for example, the byte operation,
the automatic counter manipulation, and also the
synchronized core.
Some developers don't want to deal with
Memcache directly at all.
But they still want the benefit of Memcache.
So they might use a third-party library, like
Objectify for datastore, which uses Memcache internally.
Similarly for Python, you can choose to use a lower-level
API, like Memcache module, which has a very similar
syntax as Open Source Project Memcache D. You can also
choose to use NDB for datastore.
I said you can use Memcache to cache a datastore or to cash
API computations.
But actually, from a programming point of view, the
usage pattern is all the same.
For the data read operation, what you need to do is first
check to evaluate the Memcache.
If it does contain your data, perfect.
You just return the value.
If it doesn't--
for example, if it's a datastore--
then you have to fetch the data from the datastore and
update the cache entry.
For the write operation, you also need to evaluate the
existing value in Memcache first.
You can either evaluate the specific entry, or you can
simply evaluate the entire Memcache.
Because you want to write a new value, then you can write
it to the datastore and optionally load the value to
Memcache, as well.
Here is some Java sample code that explains the read flow.
Similar to other App Engine services, you get the Memcache
service object back.
Then you can start to handle the class.
You try to do the read from the cache.
And if the value is there, perfect.
Otherwise, if the value is not, then you have to get the
value from the datastore and update the cache.
Similarly, this is the Python code.
You implement the same read workflow.
Also you can implement the put operation for the write flow.
This is another feature that's provided specifically by
Memcache that's basically a batch operation to further
improve performance.
I talked about Memcache being a shared service running on
some server.
Every API call is actually a network call.
So if you try to read one object, it's fine.
If you read 100 objects, that's 100 network calls.
The network overhead added together could be
more than you want.
So what you can do is deal with this common use case.
Memcache actually provides a mechanism to do a batch.
You can batch 100 network calls into a single network
call that further improves Memcache performance.
The only limitation is the batch size for your
operational API must be less than 32 megabits.
Another important feature is that Memcache has some generic
built-in support for atomic operations.
The first are increment and incrementAll that can
automatically increment numerical values.
That's very useful feature to implement some kind of counter
in Memcache.
Because Memcache is not transactional, the API cannot
participate in the transaction.
It does, however, provide some very basic optimistic lock
mechanisms.
It provides the method getidentifiable and
putifUntouched.
For example, you want to update something, but worry
about others updating at the same time.
What you can do is you can use getidentifiable first.
What it does is return you an identifiable object.
In this object contains the value of what you need and
also a version of this value.
You can treat the version as a time stamp.
Later on, when you want to update the value, you don't
call put, but you call putifUntouched and then pass
the identifiable object as a parameter.
Memcache is guaranteed only if this value has not been
changed or modified since the last time you called
getidentifiable.
You will be able to update it.
If somebody updated in between, it
will not do the update.
Here are some other features provided by Memcache.
One is an asynchronous call.
Memcache is a shared service, so a Memcache server could be
overloaded by either your application or someone else's.
Your API can actually block on the Memcache API.
For some applications, they are very sensitive to latency.
What they can do is call the asynchronous
version of the service.
What that does is the API code is not
blocked it all by Memcache.
Then you can continue.
And then only when you need it, you return back to sync
back with the API call.
Another feature is namespace.
Actually, namespace is a generic feature that's
supported by the App Engine itself.
There are many use cases for namespace.
One typical use case is to support multi tenancy.
For example, you have one application, but you want to
support a different domain.
But you also want to isolate the data between
the different domains.
So what you can do is use namespace to put all the data
into one domain so they cannot access each other.
We've covered the basics about Memcache.
Now, let's discuss some caveats and some solutions to
work around them.
The first caveat is Memcache is volatile
compared with datastore.
Basically, if you put something in Memcache, it does
not mean you can get it back later on.
The reason is either the entry has reached expiration, or the
Memcache is full.
You either put too much data into it, or old
data will be evicted.
The Memcache server might also crash, so you lose all your
data in Memcache.
It's therefore very important for your application to handle
cache miss gracefully.
Either it's in the cache return or not, or you get an
exception if the server crashes.
If you really want persistence behavior, you can implement
the write-through logic backing
Memcache with datastore.
Actually, that's exactly what Objectify and NDB do, but you
can implement your own version if you want flexibility.
Another caveat is Memcache is not transactional.
Consider two running instances where the first reads an
account in Memcache was $100.
Instance two deducts $20, and then the Memcache becomes $80.
The first instance also deducts, say, $30, and it
writes the incorrect value of $70 to the Memcache.
I think you can see this transactional problem.
You need to pay attention not to fall into this caveat.
The mechanism provided by App Engine, you can simply use
increment, treating the value as a counter to do that.
But for other types of data, or if you want more
flexibility, for example, you can use getidentifiable.
Each instance, one and instance two, both use
getidentifiable.
Then you can do a lot of things.
When you do the update, using putifUntouched, instance two
will succeed.
But instance one will fail, because during that time the
value has been modified.
A lot of people find that they don't have enough Memcache.
Even if you're willing to pay for it, they ask,
can you give us more?
But until now, we couldn't.
In the future, maybe there's some feature,
but not right now.
What we like to say is that your application should
function well even without Memcache, so that Memcache
only gives a boost on the performance.
Also don't put too many items into Memcache, because you can
make it less efficient if you do that.
Think about one use case.
Memcache is one size.
You can only put one item into Memcache, for example.
So you put item A in.
Then you try to put item B into it.
Then A gets evicted.
So we need to try to read A. Then A gets read in from the
datastore and then put into Memcache.
Then B gets evicted.
Then you try to read B again.
And if you get that kind of situation, Memcache doesn't do
you much good.
So what we suggest is your application should put only
useful and necessary information into Memcache.
More specifically, some developers use
optimization like this.
They don't just blindly put the entity read from the
datastore into Memcache.
Because probably only 20% of the information is really what
you need cached, they actually create
their own kind of objects.
And it combines information from multiple entities and put
into the cache.
They even use some compression to make
the object size smaller.
Those kinds of optimizations you can do to make Memcache
more efficient.
But that's the cost you have-- you're coding it more.
These are some of the key takeaways you should have
learned from this lesson.