Welcome to the second session on database multi-tenancy in Azure.
My name is Paul Burpo. I'm a Principal Customer Engineer
on the Azure Fast Track for ISVs team.
In this session, we're going to focus on Azure Cosmos DB.
We'll cover the design considerations for multi-tenant architectures, Cosmos DB basics,
and wrap up with design patterns and trade-offs for each pattern.
First let's discuss the design considerations for multi-tenant database architectures.
You can organize multi-tenant database design considerations into
four categories; scale, isolation, complexity, and cost.
For scale we want to understand the number of tenants.
If my company has thousands of tenants, that is likely going to result in an architecture that
is dramatically different than if I only have five tenants.
Likewise, you will want to understand the storage requirements per tenant, and the
total storage of all tenants.
You will also need to understand the characteristics of the workload.
All of these will help you evaluate different models.
Isolation is one of the most fundamental considerations, and frequently the
requirements around tenant isolation drive your design decisions.
Data isolation is used to separate one tenant's data from another.
This is frequently the result of contractual, regulatory or compliance requirements.
Performance isolation is simply preventing one customer's workload
from impacting the performance of the other customers' workload[s].
Complexity is probably one of the most overlooked design considerations,
yet, it has a direct long-term impact on the cost to operate a solution,
and potentially other aspects, such as availability.
You will need to think beyond the simple performance impacts, and consider things like:
"How am I going to implement a schema change after this system is up and running?"
"Does my team have adequate development skills
to build and maintain a solution of this complexity?"
"How will I monitor not only the entire system, but at the individual tenant level?"
If you require high availability and disaster recovery beyond the default configuration
of the services, what will that look like?
Another often overlooked area is tenant customization.
How do you allow customization of the data tier in your application?
If so, how will that be implemented?
Looking more closely at performance isolation, let's consider that we have seven workloads
running on a single database instance.
The blue boxes here represent the amount of system resources each workload is using.
What happens when one of the workloads, in this case workload 4,
starts consuming more system resources?
This is known as a "noisy neighbor" problem. This is a potential problem for
almost all database management systems, but is especially true for multi-tenant systems.
One way to deal with this resource consumption skew is to implement a
"bounding box" pattern into your design.
This limits the individual workloads' consumption of resources
to the specific limits allotted to them.
To understand multi-tenant design patterns, we first need to understand the basics of Cosmos DB.
First, let's look at the Cosmos DB resource model.
When you create Cosmos DB, you'll start with a Cosmos DB "account".
The account is the fundamental unit of global distribution and high availability.
It provides you a unique endpoint for connectivity, and will define the API you plan to use.
There is currently a soft limit of 50 accounts per Azure subscription,
which can be increased via support request.
Within an account you'll have one or more databases, and within each database
you will have one or more containers.
The container is where your data and any server-side code is going to live.
It's important to understand that some of the settings you might configure are
set at the account level.
For instance, global distribution is set up and configured at the account level.
Digging a little deeper into the Cosmos DB resource model,
we have database accounts, which in turn contain zero or more databases.
Databases, in turn, contain zero or more containers.
The containers will hold your data and any server-side code.
The data that is stored will be described differently, based on the API that you choose.
In most cases, the container will be a collection of documents,
related to either the SQL API or the Mongo API.
Performance of Cosmos DB is governed by "request units".
Request units are an abstraction of memory, CPU, and IO.
One RU, or request unit, equates to a point read of a one kilobyte item.
Inserts, updates, deletes, and queries will result in more RU consumption, and hence, higher cost.
The cost of RUs for a given operation over a given dataset is deterministic,
meaning that if you run the same operation, over the same dataset,
with the same configuration, you will get the same request units used for that operation.
Request units are allocated at either the database level or the container level.
Provisioning at the container level provisions dedicated RUs for that container.
Provisioning at the database level allows us to share our RUs across containers in the database,
up to a maximum of 25 containers.
It's worth pointing out that shared provisioning does not have any resource governance applied,
meaning I cannot set minimum and maximum consumption targets
for the containers participating in shared throughput.
This means that my containers are susceptible to noisy neighbor problems.
Before we dive deeper into database- versus container-level provisioning of request units,
we need to discuss how partitioning works in Cosmos DB.
Cosmos DB partitions your data with logical and physical partitioning.
Choosing a good partition key is crucial for even distribution of your workload
across all partitions of a container.
A logical partition consists of a set of items that have the same partition key.
In our example here, the partition key is "city",
so each unique city name would be in its own logical partition.
There's no limit to the number of logical partitions.
There's no limit to the number of Each logical partition can store up to 20 gigabytes of data.
Good partition key choices have high cardinality,
which means that we have a wide range of possible values.
Cosmos DB will scale your container by distributing your data and throughput
across physical partitions.
Internally, one or more logical partitions are mapped to physical partitions.
Physical partitions are managed internally by Cosmos DB,
so you do not have any direct control over the physical partitions.
Each physical partition can provide up to 10,000 request units,
and up to 50 gigabytes of data storage, so the number of physical partitions in your container
depends on the throughput that you have provisioned,
and the total amount of data storage required.
Throughput provisioned for a container is divided evenly among physical partitions.
A partition key design that does not distribute data requests evenly might
result in too many requests directed to too few partitions, resulting in "hot" partitions.
This might result in throttling and higher costs.
Likewise, poor partition key design that results in excessive query fan out
can also result in inefficient utilization and excessive RU consumption.
Setting throughput at the container level is the most common option,
as it provides the most predictable performance.
Provisioned RUs are distributed evenly across the container's physical
partitions and, if an appropriate partition key is chosen,
the work will also be evenly distributed across all the physical partitions.
Because one or more logical partitions of a container are hosted by a physical partition,
the physical partitions belong exclusively to the container,
and support the throughput provisioned on the container.
When you provision throughput at the database level,
the throughput is shared across all the containers in the database.
At the time of this recording, the number of containers that may participate in
database-level shared throughput is limited to 25.
However, you may provision additional containers using dedicated throughput,
even though some containers are using shared throughput.
Because the provisioned resources are shared across all the containers,
Cosmos DB does not guarantee predictable performance for an individual container.
The portion of the throughput that a specific container can receive
is dependent on the number of containers, the partition key choice of the various containers,
and the distribution of the workload across the logical partitions of the containers.
In addition to the typical, provisioned throughput model for allocating request units,
we also have an auto-scale option.
The auto-scale option scales request units automatically.
The throughput will be scaled based on usage
without impact to availability, latency, throughput, or performance.
You can switch between auto-scale and fixed provisioning.
Auto-scale works well for variable or unpredictable workloads.
It's ideal for greenfield applications, where you might not yet know the workload demands.
It's also great for other types of intermittent workloads.
Cosmos DB serverless allows you to leverage "consumption based billing",
which is ideal for smaller, low average-to-peak ratio scenarios.
Serverless currently uses a different backend from provisioned,
which brings some limitations as well.
For starters, you can't switch between serverless and provision billing model.
Serverless supports a maximum of 5000 request units per second
and 50 gigabytes of storage.
Serverless is also limited to a single region,
and cannot take advantage of the Synapse Link feature.
Serverless is ideal for smaller, unpredictable workloads.
We'll cover the provisioning aspects and the pros and cons of each one later in this session.
Next, we're going to talk about the design patterns in multi-tenant Cosmos DB implementations.
When we talk about design patterns for Cosmos DB multi-tenancy,
we're talking about how we're going to allocate resources for our individual tenants.
Many times, the design pattern will be chosen for you based on your requirements,
but usually you have to make some decisions and trade-offs
to get the optimal price/performance ratio.
Based on your requirements, you'll need to decide
if you can put all of your customers' data in a single container,
or if you need to split across multiple containers,
or give each customer a dedicated account.
So, let's talk about each one of these patterns in a little more detail.
The first option you have is to give each customer their own Cosmos DB account.
This gives you maximum flexibility to provision throughput.
It allows you to provision shared throughput at the database level,
or dedicated throughput at the container level, or both.
You won't need to worry about the noisy neighbor issues or tenant isolation.
Tenant management will be very easy as well.
To drop a tenant, you can just remove the entire account.
The billing will also be easy to track,
as the Cosmos DB account will be a line item on your bill.
This model is good for very large tenants,
or in situations where you need to allow independent control of account-level features
such as geo-replication and Synapse Link.
For example, let's say tenant two requires geo-redundancy for disaster recovery,
but tenant one does not.
If you provision at the account level,
you'll be able to provide tenant two with the geo-redundancy,
without having to incur cost of providing geo-redundancy for tenant one and other tenants.
It would be a similar story for other account-level features, such as Synapse Link.
The next model to consider is provisioning a dedicated container with dedicated throughput for each tenant.
This is better for large or busier tenants that will not require independent control of account-level features.
Noisy neighbor problems are eliminated in this model, as each tenant will have dedicated throughput.
Tenants are easily managed, but you will not see a line item for each tenant on your bill.
In this model, you can also group tenants in different accounts based on region,
to better control which regions they're accessing the database from.
The next model is provisioning sharded database throughput across multiple dedicated containers.
This is a lower cost option for spiky workloads and medium tenant sizes.
Tenants are still easy to manage, and the isolation is still excellent,
since each tenant is in their own container.
You will retain the ability to group tenants in different regions.
Note that there is no performance governance here,
so these containers will be vulnerable to noisy neighbor issues, due to the shared throughput.
The maximum number of containers that you can provision using shared throughput
is currently capped at 25, but that number is subject to change.
You can also mix shared throughput with dedicated throughput containers,
so if you hit the max, you can add additional containers using dedicated throughput
at the container level.
The final option we're going to discuss is putting all customers in the same container,
and partitioning them by some partition key, usually a tenant ID.
This allows all the customers to share the same resources.
This model has the highest tenant density, which makes it very attractive from a cost perspective.
You get easy querying across tenants, since the boundary of a query is the container.
The big downside of the partition key per tenant is
is that it is susceptible to noisy neighbor issues, and each logical partition is limited to 20 GB,
so you might need to add some additional complexity to handle the skew that your
larger tenants might cause in this model.
Because of these issues we typically recommend this model for smaller workloads,
such as B2C applications, where an individual tenant will have a small amount of data,
and access that data somewhat infrequently.
So, it's common that a SaaS provider has many customers of many different sizes.
This is generally described as "tenant skew",
and can refer to the throughput a tenant requires,
or the raw size of the data for a given tenant.
There are different ways to deal with tenant skew,
depending on your provisioning model and your business model.
The first thing to consider is a hybrid strategy.
Hybrid strategies are great, because they allow you to offer different tiers for
your application that correlate with increased scale or features in Cosmos DB.
This is typically something similar to the Azure offerings with "standard" and "premium" tiers, or
billing based on a particular number of seats or users,
or just "T-shirt sizing" like "small", "medium", "large" offerings would be.
So, depending on the tier your customer purchases, you might choose to put them
in a container with dedicated throughput, or shared throughput, or serverless offerings.
It also allows you to more accurately tune performance,
based on what your customer is willing to spend.
If you're using a partition key per tenant,
you might run into this situation where a large tenant's data reaches the
maximum amount of data that a single logical partition can hold,
which is currently 20 GB. One way to deal with this is to use a "synthetic" partition key
One way to deal with this is to use a "synthetic" partition key
that distributes a single large customer across multiple partitions.
You can see in the example here that we're running a "SELECT *" query,
where the TenantID is in "Contoso1" or "Contoso2".
But you might also create a key from an existing field in the document,
like TenantID combined with UserID.
However, there's a trade off here, because now, tenant-wide queries are always cross-partition,
unless you include all the users as well, which is unlikely to occur.
These types of cross-partition queries are called "fan-out queries",
and typically come with a much higher cost than a single-partition-key query.
This also increases the query and app complexity,
which lends to one of the other key pillars that we talked about at the beginning,
which was complexity of the environment.
A more elegant solution is to use an upcoming feature called
"hierarchical partition keys", sometimes referred to as "sub-partitioning".
This feature will allow you to partition a container with up to three levels of
partition keys, replacing the need to use synthetic keys.
So taking our previous example, your first partition key would be TenantID,
and your second partition key would be UserID.
The queries executed at the tenant level would then touch only the partitions
related to that tenant, not every partition related to every tenant,
keeping the cost much more reasonable in the process.
A final note on handling tenant skew is around resizing and migrations.
In some cases, you may need to migrate a tenant, for example.
If they are on a serverless, and you need to move them to dedicated throughput,
you'll need to migrate them.
To do this, you can take advantage of the "change feed".
Every deployment of Cosmos DB also includes a change feed.
It's automatically configured, and it's always on.
You can use this change feed to read all the data changes in history for that Cosmos DB account,
and that helps you facilitate an online migration and cut over.
The last thing I want to discuss is the provisioning models that are supported with Cosmos DB.
We touched on this already, but just to rehash this,
to rehash this, there are three provisioning models in Cosmos DB:
there's serverless,
and then there's provisioned, which is broken down into fixed provisioned and auto-scale provisioned.
Serverless uses a consumption-based billing model,
and is best for intermittent workloads with low average-to-peak ratios.
There are no RUs to provision in this model, but you are limited to a maximum
of 5,000 request units per second.
The storage size is also currently capped at 50 GB,
and the SLA for performance is lower than that for provision throughput.
Serverless runs on different hardware than provisioned Cosmos DB accounts, and
because of this, you cannot mix and match serverless databases and containers
with provisioned databases and containers.
Serverless also supports only a single region.
These limits are subject to change, so it's best to check the documentation
if you believe these limits are going to affect you.
Provisioned throughput is for high traffic scenarios.
You configure the request units and auto-scale options yourself.
You get full support of all Cosmos DB features,
and the sub-10 millisecond latency that Cosmos DB is famous for.
There's currently a 1 million RU soft limit that can be raised by opening a support ticket.
Within provisioned throughput, you have a choice between autoscale and fixed provisioning.
This is merely a billing change, so you can mix and match these provisioning models
across containers and databases.
Auto-scale automatically scales to the maximum RUs that you configure.
It's billed on an hourly basis, based on the highest usage that hour.
Autoscale request units cost slightly more than fixed provisioned RUs,
Auto-scale request units cost slightly more than so you should consider autoscale when you will use 66% or less of the maximum RUs
that you're going to configure over a given period of time.
Fixed provisioning allows you to set a static amount of request units,
and you're billed per hour for those request units that you've configured,
regardless of whether you consume them or not.
It's worth noting that both of these models qualify for reserved pricing,
in which you can pre-purchase request units for one or two years at a significant discount.