>> Hey friends. Azure Cosmos DB is a great way
to store both structured and unstructured data.
But when you combine it with Azure Functions,
Azure Cosmos DB makes storing data quick and easy,
with much less code than storing data in a relational database.
Matthias is here to show some of
the best practices today on Azure Friday.
[MUSIC].
>> Hey folks, I'm Scott Hanselman,
and it's Azure Friday.
I'm here with Matthias bringing me some of my favorite things,
Cosmos DB and Azure Functions.
So like chocolate and peanut butter,
only good things can happen. How are you, sir?
>> Pretty good. Thank you Scott.
So for today's demo,
I want to show you how you can leverage
Azure Cosmos DB Multi-master,
and change these features to create
serverless sure replicated even based architectures.
So what I did is, I deploy two groups of devices.
One in North Europe, and one in Central US,
and I'm sending that telemetry into my Cosmos DB Database.
But I want to do it as quickly as possible.
So what I did is, I created my Cosmos DB Database.
I deployed it in distributions,
both in North Europe and in Central US,
and I've enabled the Multi-master capability.
This make it so. Both endpoints are both,
read and write enable.
>> Okay, so let me get my head around this.
You're writing data from
devices all over the world, like globally writing?
>> Yeah.
>> In my mind,
which is like how we used to do things,
you can have lots of reads,
but you only get to write to one place.
>> Exactly.
>> You said you're writing to multiple places too.
>> At the same time. That is quite amazing.
So what I also did is,
I took this code that I used to build an Azure Function,
I'm using the HTTP trigger,
to receive the telemetry from the devices,
and then I'm using the Cosmos DB output binding to save it.
Now there are two important properties
that I'm sitting on the output binding.
One is the UseMultipleWriteLocations to two.
This will make it so,
the binding will know that I want to leverage
the Multi-master capability in my Cosmos DB Database,
and the other one is the prefer locations.
Now if you take a closer look,
I'm not using a constant value,
I'm using the placeholder.
That makes it so, I can take
this code and I can deploy it in as many function apps,
in as many regions that I want,
and if I set an environment variable that will take the [inaudible].
Where in which reason this is running, it will make it,
so the output when it will connect to
the local Cosmos DB endpoint and two local rights.
>> Okay, so the rights are as close to
one of the Writable Locations as possible.
>> Exactly, and I can even use App Insights to
monitor both my Central US,
and my North Europe function instances,
to see what is the latency in the operations in both endpoints.
The important thing with function is that,
since these are different functions app,
they can scale independently based on the needs of my devices.
So if I get more throughput on a particular region,
then that function app can scale into
more instances to absorb that data flow.
>> Interesting. So if I'm going to do
a global application like this,
depending on what I'm trying to do
whether it's IoT data, financial data,
or website data, I might have it done by time zone.
>> Yeah.
>> Europe is waking up,
and then Cosmos will wake up and scale with Europe-
>> Yeah.
>> -and then they go to sleep, and then Asia wakes up.
>> Exactly.
>> I can deploy these functions up,
and I can use the same code in all the regions.
So another thing I wanted to show you during this demo,
is a Real-time Dashboard.
Because let's face it,
Real-time Dashboards are pretty cool.
So what I did is,
I created this very small and simple console app,
that it's receiving all real time,
all the writes that are happening in all Microsoft DB regions.
Now how did I build this?
I went and use the Cosmos DB trigger in Azure functions.
The Cosmos DB trigger is leveraging the change feed feature.
The change feed is like a timeline,
where all the events that happened in
the past and now in your container are stored.
So technically you could hook into
the latest known endpoint on this timeline,
and start varying up functions
whenever there is something new in that timeline.
So what I'm doing here is,
I'm varying up my function,
whenever there is new telemetry data in my container,
and then using the Singular Output Binding.
As you may know, Signal R uses
WebSockets connections to send push notifications on real time.
So my war application is just opening
a WebSocket to my singular hub,
and it's receiving all real-time,
all the changes that are happening in my Cosmos DB Database.
>> Interesting, and also you've effectively got
a change log that you're listening a firehose,
that you just listen into,
but your hearing the events all over.
>> Exactly. It doesn't matter which region is being written to,
I'm listening to all the events.
>> The other thing that I'm noticing by the way,
which I think it's just interesting
because of the way that you're choosing to architect
your application using Cosmos DB
a Serverless Global Database in Azure functions.
A Serverless Compute Mechanism.
You've really shown me very little code.
You're using input and output bindings
in a very clever way that's allowing
you in multiple places to lose a dozen lines of code,
and not interesting code just boring.
Read this, write that.
The output bindings for Cosmos DB is very clean.
>> Yeah I can focus on what I really want to do with my function,
and all that orchestration
that is really happening in debug like opening the connection, [inaudible].
The connection is entirely done for me.
>> That's cool.
>> Another thing I can also do,
is I can also use
Application Insights to monitor this trigger and see which are
the course remaking both
to the cognitively instance onto signaler,
and I can see that I'm only reading from one Cosmos DB region.
But I'm getting all the data that is coming from both regions.
>> You're using the Azure SignalR Services as well,
which is again another serverless.
>> Exactly. So basically I have
no applications running in terms of the user-
>> Additionally.
>> Exactly.
>> Yeah.
>> I have all these serverless functions running constantly.
Finally one thing that I wanted to show you is,
it's really common for our customers to have,
when they want to analyze the information,
they have different query needs,
so that the queries are different from the queries,
and the operations you do on the record data path.
So what I did is,
I have this materialized view container,
and my material is view documents or the schema,
is different from the schema that I'm
using to store the telemetry information.
Maybe I want to do like an aggregation over the data,
and my partition key on that particular container is different.
So how do I make this synchronization between
two containers online and on real time.
What I use to do this,
I have these very simple,
another Cosmos DB trigger that is working in parallel,
and totally independent from
the previous trigger for the notifications,
and I'm mixing it with the Cosmos DB output
binding targeting the separate container.
So basically what this function is
simply doing is just taking all the telemetry,
then group with the information candidate in my aggregation,
and then sending that new scheme
of information to my second container.
So basically I'm maintaining both containers in sync on
real-time and with probably like five lines of code.
>> So is the reason for doing that.
I mean, I'm thinking simplistically
you're duplicating a lot of data,
but at the same time you're also optimizing it that,
the data ingress for the multiple writes in multiple locations,
is optimized and then,
the heavy query, the heavy reads,
the complicated reads are in a totally different collection which
allows Cosmos to be smarter and not fight with itself.
>> Another reason is, maybe the queries that I want to do on
my materialized views have a different partition distribution.
So I want my queries to be as fast as possible,
regardless of how I'm storing the hot data.
So I can separate both of my units.
>> Then you could ultimately build a data pipeline,
and move that data farther and farther,
into more appropriate places
and data breakers your data factories.
In keeping everything partitioned away,
and even put it into a Dashboard another dashboard.
>> Another dashboard.
>> Power BI dashboard which is also serverless.
>> This combo fusing the trigger and the output binding,
is also useful when
people create a container that you find a partition key,
and then after a while they realize
that they picked the wrong partition key. It's really common so-
>> Yeah, that is very common.
>> Exactly, and one thing they can do is,
they can create a function using the trigger.
Said these particular flag call start from beginning to true,
into a trigger and mix it with this output binding to
a separate collection with a new and correct partition key,
and they start from beginning through a flag will make it so.
The trigger instead of starting from now to listen for changes,
it will go back in time and replay
all the inserts that happen on that container ever.
>> Wow.
>> So it's a hot online migration of your data,
from one container to another,
and you can do this without any downtime.
>> [inaudible] in my mind while you were saying that,
I was imagining all of these cups of liquid,
and how there you're moving from one cup to another cup,
but you're not dropping any liquid at all.
>> Exactly, and I can do this on
real time without any downtime on my containers.
>> Fantastic. So we can learn more about this where,
and we can play with Cosmos on.
I know we can do Cosmos for free, right? Try Cosmos.
>> Yeah. There is a try Cosmos offering that lets you play with
Cosmos DB for 30 days without
any credit card or actual subscription.
You just need a Microsoft account.
>> Do you have this particular application
in GitHub, or can you put it there for me?
>> Yeah sure thing, I can do that.
>> Oh, fantastic. All right.
I am learning all about best practices with
Azure Cosmos DB and Azure Functions,
today on Azure Friday.
[MUSIC]