(upbeat music)
Hello and welcome.
Please enjoy this short little video on Azure database
for PostgreSQL.
All right, Umur, how you doing today?
Very good, Sanja, thanks how are you?
Great to have you with us again.
It's a pleasure.
Awesome.
So can you please start with,
what do you do at Microsoft?
Of course.
So I head up our PostgreSQL product team at Azure.
And before that, I was the CEO at Citus Data,
which as of earlier this year,
it's acquired by Microsoft
and now we're part of the actual team.
Awesome, welcome.
So let's get started.
So why, first of all,
what are you gonna talk about overall?
Well what I'll cover, Sanja,
is just a baseline of,
why PostgreSQL in the first place?
And then diving into our PostgreSQL offerings on Azure,
as a fully managed service
and then how we are changing
what's possible with PostgreSQL with Hyerscale Citus,
being able to scale that horizontally,
in a very unique way
and then putting that all together for
what makes Azure unique?
When we say it's the best destination for PostgreSQL,
why is that the case? And then I'll wrap up with some
use cases and resources to look at.
All right, my first question is already like,
you have so many choices for databases, why PostgreSQL?
Yes, of course. (laughing)
Why not is my answer. (laughing)
But I'll give you, No that's awesome.
A more nuanced one.
Because, you know, here on this slide,
we have you know relative popularities of databases
and then the orange bar is PostgreSQL
and the blue one is Oracle.
I'll make two immediate observation.
One is that PostgreSQL is trending up very strongly
right, over the past five, six years,
but also, it's more than a 20 year old project
that's very mature.
The other is,
on just absolute terms, it's very comparable to
kind of a database like Oracle,
which is of course,
you know, extremely popular.
So PostgreSQL has this huge footprint,
trending very strongly upwards and loved by
developers when there's a recent stack overflow
with a survey,
that ranked PostgreSQL as the most loved database.
I see.
Yeah and the question 10 years ago was,
why would you use PostgreSQL, right?
Or maybe a longer, other than some you know,
people who were really in to open source
and now the question is, almost, why not use it?
You know, what are,
and I'll frame that in,
really three big buckets.
Yeah. And I could talk
entire session on this,
Of course. But I'll just keep it brief.
On first, it's open source.
Not just open source though,
it's very popular, which means it has
tons of developers,
tons of you know, ecosystem around
with all the tooling, all the integrations
so the database is a lot more than just the database itself
and PostgreSQL is huge in that regard
and it's very extensible, very integrated.
The other is,
it's reputation for reliability, right,
so there's a lot of things in your day,
you know, stack that you might be a little adventurous with,
but your database is probably not one of them.
Yeah. And then PostgreSQL
has a tremendous reputation for not dropping data.
Being very reliable.
And last is,
a huge you know,
rich feature set.
Whether that's (mumbles) sequel compatibility,
whether it's all the indexing.
You know, tons of things.
But I'll touch on the data types in particular,
which comes with a very rich,
they have a type for almost anything,
you know, like in PostgreSQL.
Postgis is a popular one for example,
for geospatial data and JSON, which is semi-structured data,
which you can store within the PostgreSQL database,
next to your structured data in a binary way,
index it, access it really fast
and get a lot of kind of modern relational if you will,
structured, semi-structured all together.
It made a powerful package.
So that's why.
So JSON extracts like tweets.
It could be tweets, it could be
signal data that you get from IOT.
It could be things that any device generates.
And it could be very structured, traditional data.
So mainly the details (mumbles) sensors.
Any information.
You got it.
Okay. Yep, yep.
It can store in PostgreSQL. It could be patients,
it could be you know, like a manufacturing, you know,
industrial like B to B businesses.
Awesome. Pretty much everything fits.
And then the next logical question,
why PostgreSQL in Azure?
Oh, yes.
In fact, so what we do on Azure is,
you know the cloud makes PostgreSQL only better, right?
So you get those core benefits
of a reliable database that has very rich data type support.
That's very performant and functional
and then we make it easier to manage, right?
We'll monitor the database,
we'll take the stack up through a team lead
when something goes wrong, we'll fail the over
and one of the things that
you know, wouldn't be surprising,
perhaps, but you drop data accidentally you know,
because sometimes you think you're doing stuff
on the data instance, but you're using it on (mumbles).
Believe it or not that happens
quite frequently. Oh I've done that before,
(mumbles). (laughing)
So there you go, so it happens all the time
and what you wanna do is,
it's a kind of, oh shoot moment.
(laughing) And you wanna
go back in time, which is a point in time to correct.
These are all features that PostgreSQL provides
and then we make it very easy to access.
And beyond those,
you know,
platform as a service pass type of capabilities,
I'll highlight perhaps a couple of things
that are very unique on Azure,
that things that leverage AI around that data.
One example is for example,
what we call advanced threat detection.
Right, so this is looking at the access patterns
to your database and then if there's something unusual,
then it'll raise a flag, it'll give an alert.
So you look kind of more closely into it.
Similar thing goes with performance.
It'll look at your access patterns and create perform time
is going back and then make recommendations
on what indexes to add, which ones to drop, et cetera.
So it just you know, leverages all of that,
with the entire goal of,
you focusing on your application.
And then we worry about the database,
patching it, all the security updates, everything,
keeping the lights on.
That's awesome. (mumbling)
So you know having walls in my keynote.
Yes. I was super excited
about the Hyperscale, you know?
(laughing) Because we have
created a demo together, as well,
Hyperscale. Exactly.
That one together
on PostgreSQL. It was very (mumbles).
So can you please share more about that?
Of course.
So, what we're doing with Hyperscale Citus,
is actually, changing what's possible with a database
like PostgreSQL and when you think about it actually,
relational databases in general,
and PostgreSQL is an example of that,
is criticized for sometimes being kind of harder to scale.
And what that means is, you know,
it provides all this functionality,
you put it in an instance
and then as your data grows,
you put it into a bigger instance.
And then a bigger one.
Of course. And then what.
You know, so that question kind of
lingers at the back of your mind, or can,
depending on your application.
Because, as the data gets larger,
performance becomes an issue,
taking back ups, doing the restore,
like maintaining the database.
Everything becomes progressively harder and harder.
And this is true for not just PostgreSQL,
but pretty much kind of all relational databases.
And what we do with Hyperscale Citus,
is we change that equation.
Like we say, "Instead of scaling up,
"or rather in addition to scaling up
"you can also scale out."
Yes. You know, horizontally.
So it's simple as taking a slider,
dragging it from two machines to 20.
Or to 100, if you wanted to. Yes.
And then have all of that behave
as a single PostgreSQL to the application.
And that's what really
and that's actually a very hard problem to solve.
You know,
you know prior to our Microsoft days,
we spent more than eight years at Citus Data,
building that technology
and then bringing that to market
and now, kind of as part of Microsoft,
keep investing in it,
because what you're really doing is,
preserving all the relational semantics,
Yeah. That PostgreSQL provides
to the application, so you don't want
the application to,
have to know, what data is placed where
and when failures happen,
how will it roll back?
How will it commit?
You know, is it consistent?
Is it inconsistent?
All of those things belong to the database.
And then we enable that.
So you don't have to get a PhD in distributed systems,
to be able to scale your relation database.
I see.
Yep. And I know
we have some live questions
in the chat window as well for you.
Uh-huh. So one is,
what are the scale of storage?
Great question.
In fact, and that's been
that's pretty commonly done, right,
like you can add more storage to a database like PostgreSQL.
You know could have,
tons of terabytes, up to 64 terabytes or more,
if you wanted to.
But what that doesn't do,
is it doesn't scale the compute,
or the memory that goes with, right?
So you're creating a shared storage underneath
and then you're putting replicas of the database,
you're not really solving this fundamental problem
of a single instance getting larger.
You're just giving it more storage.
What that Nice.
Means is, for example,
when you're a query, Yes.
That needs to touch upon multiple data sources.
I see.
You can let them pile it into the Hyperscale Citus
can do it really fast
and you can actually add nodes in real time.
Without any absolutely taking any downtime,
you can add in your nodes, accentuate them,
that's the core difference.
And I know you'll see some live demos today,
Yes. Sometime.
Yes, exactly. Show us as well.
Okay, (mumbles).
Now, putting all this together,
Okay. Going back
to the top of our talk, why does it
Yes. Kind of
making, Azure unique for running PostgreSQL?
First off, are these kind of three core pillars
that makes really PostgreSQL strong.
And I'll get to the open source, a bit later on.
But we add the Hyperscale Citus piece,
right, this ability to scale horizontally and to do so
with zero down time.
Basically, the promise of the cloud, if you will,
for PostgreSQL and for that relational database.
And then, I talked about
the intelligent cloud management features,
early on, which are only available on Azure.
And then integrations that go with it,
Yeah. Which are
the older tooling that you like on Azure,
whether it's Azure Data Studio for kind of like
running your creative interface database,
or running power BI to visualize things,
or Azure Active Directory,
Azure Data Factory, DIA Data Migration Service,
all of those things,
PostgreSQL integrates very neatly,
to that entire ecosystem.
And we do all of this,
going back to this kind of open source tenant.
Preserving the open source.
So we run the community version of PostgreSQL
and actually the Hyperscale Citus product
is also an open source extension on PostgreSQL
that's available that you can take,
that you can run anywhere.
What that means is,
you're not locked in to a particular ecosystem.
That you can actually,
take it and run it wherever you like.
That's one aspect of open source.
The other is, actually giving back
and that's something that really is close to our hearts.
You know, both outside (mumbles).
And also, you know at Microsoft,
which is giving back to the community
that's contributing code to the core project,
contributing code to Citus itself is open source.
Actually at Microsoft, we have the most number
of PostgreSQL committers among,
compared to all other cloud providers combined.
I see. So we are actually
very serious about our commitment to PostgreSQL.
And, making it better.
So that's it. So as you know,
I love power BI actually, you can (mumbles).
Absolutely. Have (mumbles) real time,
is coming. Yes, exactly.
You will have standard connectors,
you don't even have to have a special connector.
It's in kind of your standard ODBC, JDBC drivers.
You're good to go.
Awesome.
All right. Yep.
There are a couple of deployment options.
Yes.
Which I've alluded to.
You can go to the actual portal.
There's a single server and the Hyperscale Citus
Yes. Server group.
You know you can accentuate those kind of
in many regions across the world.
There is another,
kind of short plug in here
for something that I'm super excited about.
All right. Which is called Azure Arc.
Which we announced just again, last month,
at Ignite. Of course.
It's not in production yet.
So it's an early access program,
that you can kind of apply to.
But what it does is,
it gives you all of these abilities that we talked about,
not just on Azure, but anywhere, right?
In your own data center.
Okay.
And it really kind of under the hood
uses capabilities to do the orchestration.
I see. And to do all the
kind of database as a service like you know capabilities.
I think there's another session tomorrow,
that we'll dive deeper into it.
But it's something that I'm really excited about.
Absolutely, I was (mumbles).
It was fun.
Azure Arc with PostgreSQL
(mumbles). Yeah, yeah, yeah, yeah.
And I think there's a lot more,
that's gonna happen.
All right.
So given this kind of set of tools, capabilities in Azure,
they'll talk about use cases,
just a little bit.
For the single server,
your post standard kind of PostgreSQL if you will,
if you see kind of three large groups of usage if you will.
One is, if you're building new apps,
right, that's a very common new modern,
cloud native apps.
In fact, more than I think 70% of all apps,
are getting built on open source relational databases,
right, and then PostgreSQL is an excellent choice for that,
you know, as is MySQL in kind of in other areas.
You can just accentuate on Azure,
you know community service.
You can get it on App Service,
you can run it as a stand alone,
using Python, or Django, or Rails Active Record,
all of those frameworks that PostgreSQL supports,
is very rich.
I see. So you can get
going very quickly.
The other is a more basic lift and shift motion, right.
You're running PostgreSQL on Prim.
You run it kind of on Azure,
you take advantage of the manageability features.
And then a very common pattern we see
is migrating or off loading Oracle, right?
So there's a lot of Oracle databases
which PostgreSQL is functionally similar to.
I see. And then you can
migrate the whole app,
which depending on the app,
could be relatively high or a little more effort,
that's one path.
Or, you could actually offload the application,
keep the Oracle app as is,
but then, move data to the cloud
and then run analytics applications on it.
Or, (mumbles) because there are other things,
which basically reduces the burden on the kind of
the existing database.
And PostgreSQL is an excellent choice
if you think about it,
many companies going digital transformations,
modernizing their infrastructure,
looking at lower costs,
looking at open source,
PostgreSQL is smack in the middle of that.
So they can migrate and then no wait.
Exactly, exactly, to do both
and do that on PostgreSQL and on Azure.
The use case is for Hyperscale Citus,
touch upon again,
kind of three buckets.
And one of the most powerful ones is,
because now I have this ability
to add as much (mumbles) memory,
these hybrid transaction analytics applications, right?
So then you have both analytics,
that are quite easy to acquire
and short request transactions,
sometimes called HTAP hybrid transaction level processing,
Citus is excellent for it's multi-tenant applications
B to B apps, that are relational in nature
that it can scale easily from one tenant to 100 to 1,000.
That's very powerful
and then preserving the relational semantics
very large data sets. Nice.
Again, whether you have 100 gigs or a petabyte
and I'll just, last slide,
talk about some of the kind of customers
in action. Yes, of course.
Whether it's Microsoft itself,
Windows team running more than a petabyte,
a large PostgreSQL,
Hyperscale Citus, you know, like a cluster.
Petabyte?
Petabyte, exactly, which again, when I said,
changing was possible with PostgreSQL, we really mean it.
You know, you wouldn't be able to do those things before,
but now, all the good news of PostgreSQL,
structured data and you know, semi structured data,
and at that scale,
and this system is getting more than six million
queries per day, and responding in the subsecond,
to a vast majority of them, right?
And that is all the (mumbles) different hundreds
of million of Windows devices.
It's such a very powerful use case.
Finastra is one of the largest
Fintech companies in the world
and they have a multi-tenant database,
that powers all of their core APIs.
And being able to modernize that
and scale that out,
without having to re-architect the application from scratch.
They are relying on That's nice.
Citus, Hyperscale Citus, to do that.
And Enlyft, I think there's a session
that talks about, but in that case,
it's as low as a few hundred gigs,
or it's sometimes it could be smaller.
So it doesn't have to be a petabyte scale
to take advantage of this.
Yeah, of course resources. We can be as, yeah,
tons of gigs, will kind of get you there.
So, yeah.
There's a lot of resources out there.
I won't go through all of them.
But if I leave with kind of three parting thoughts.
Yes. Sanja.
Yes. When you think of
if you're thinking of PostgreSQL,
you know, think of Azure for three reasons.
One is Hyperscale Citus. Of course.
Right, like being able to scale
Love it. That's really
you need Azure.
Two, is the open source aspect of it,
that we're not locking you in
and then we're actually contributing back,
to open source as we do that,
all these capabilities and third,
is all the Azure capabilities,
whether it's the intelligent you know,
type of threat detection.
The query performance,
or it's all the Azure integrations
that are out of the box.
Those would be three very good reasons,
and more in the works.
That's awesome. Arc and others.
Thank you so much, Umur,
for (mumbles) Absolutely.
Why PostgreSQL and Azure.
Uh-huh, beautiful, thank you.
Thanks for watching this short Azure Readiness video.