>> SQL Server big data clusters provides
embedded administration experiences to manage the platform.
There's a lot going on at the platform so
we made sure to build a lot of
administration experiences to make
it easy to understand what's going on.
Mihaela here will tell us all about it today at Data Exposed.
>> [MUSIC]
>> Hi and welcome to another episode of
Data Exposed. I'm your host Jeroen.
Today, I have with me Mihaela to talk about some of the Built-in
administration experiences that are
available for you in SQL Server big data clusters.
So Mihaela, back on the show, welcome.
>> Thank you for having me here today.
>> So administration experiences, right?
So what can you tell us about it?
>> Yeah, that's one of
the key value product for big data clusters.
We're going to go through some of
those experiences today highlighting what are some of
the management built-in cluster administration experience
that we have available for you to try out.
As you know, big data clusters are
deployed as containerized applications and talk of Kubernetes.
That is giving us some of
the flexibility to enable some of these experiences.
It's very easy to deploy for some of you that
had tried to deploy big data clusters so that it's very fast.
Similarly for upgrades, that's going to be very easy.
Leveraging, elasticity and scalability of
Kubernetes containers that transfers
to big data cluster experiences where
it's very easy to scale out and up and down and do have
all this experience in
a predictable and consistent way
irrespective of where we are going to deploy it, right?
Because when you deploy big data cluster,
the underneath platform of Kubernetes
irrespective if it's On-prem or in Azure or anywhere else,
we are agnostic to where that Kubernetes cluster is running.
>> Cool. Sounds good.
>> Again, transfer to
this experiences that we built for you to manage
this platform because that's something that you deploy on
your own infrastructure and you have to manage it in.
>> Sure. That makes sense.
So what do we mean when we talk about broken administration?
>> Yeah. So that means that we deploy
a set of components and services
in the cluster to be able for you to deploy during the cluster,
of course, to scale to upgrade.
Similarly for security and I'm
going to mention a bit later what does that mean,
you have built-in HADR
and as well as cluster health system that is
used for other workflows like
when we do deployment on or do upgrade,
we are reading the signals from the cluster automatically for you
to be able to make sure that we're upgrading in the right order,
we are listening to health signals from the cluster to
not roll forward services that are not healthy and so on.
>> Okay.
In the same time, we have components in
the cluster that are collecting metrics, collecting logs,
store them and then expose them through dashboards,
again we're deploying automatically for you to make use of that.
>> Cool.
>> How exactly you want to ask?
What exactly that works and what are some of the services that are
deployed in the cluster and we are making use
of to further enable all these experiences for you?
We call all these components that are part of
the management or administration experience
are under cluster management service umbrella or control.
We're going to hear control plane sometime.
I would split them among monitoring services,
as I was mentioning earlier,
we have components that are collecting metrics.
Let's take a very simple example for
SQL Server has DMVs that is producing a lot of metrics.
They have components in
the cluster that are reading from those DMVs,
it's storing them in InfluxDB,
and then we have Grafana that is sitting on
top of that to expose those metrics.
>> Cool.
>> Similarly for note host in Kubernetes,
we collect some of those resource consumption like
memory and so on and expose dashboards on top of that.
Another set of services are helping to deploy and to upgrade
to set up security or high availability, right?
Those are components that are working together to ensure
cluster health to ensure that
all those things are working properly.
>> Okay. So basically, we have two groups, right?
Monitoring and more of management control?
>> Yeah.
>> Okay. Cool.
>> So for example for security,
to go through what
exactly does it mean that we have built-in security?
As with any SQL Server release,
security's mission critical for us to enable for our customers.
It was very important to enable AD Authentication.
What exactly does that mean is that once a deployment time,
you tell us that I want the services in
the clusters to be integrated with AD so later for
users to authenticate using
their AD Identity that everything is taken care of in
terms integrating containers with Active Directory,
tokens, certificates and everything,
deploying all that stuff in a containerized environment,
it's something that it's new in the industry, right?
>> Sure.
>> So that's something that we were very hard to make it
happen and we have it available will their cluster.
>> Awesome.
>> Another thing that was very important and we
listened to customers' feedback when they were saying that
I want to make sure that the user identity that is being used to
login to a certain service is passed through
the entire stack because we know in big data cluster,
we have various layers of service,
and when a new user identity connects for example to SQL Server,
I want the same identity to be passed
through down to HDFS if needed so I
can audit and to track that activity of that user, right?
So that's something that is available
in big data clusters as well.
Again, certificates and rotational certificates
is happening automatically for you.
You don't have to do anything for that.
>> Okay. So that's great.
This sounds cool.
We sold all of this,
we made sure that your credentials flow from top to bottom,
we configure all of it,
but that's just security, right?
>> Yeah.
>> How about something else that's very
important like scalability,
making sure that when
something breaks something else is there to pick it up?
That's an important factor of database.
>> Yeah. So that's something
that was very important for us as well to make sure
that the mission critical services are available
in big data cluster like SQL Server Master,
HDFS NameNode, they're highly available.
>> Okay.
>> That's where we enabled
an experience where you can deploy and manage
all this aspects again in a very easy
and embedded within the control plane as well.
For example availability groups.
This is a flagship feature that was
available for SQL Server since 2012,
I think the least and
[inaudible] they know that they have multiple prerequisites,
they have to set up,
they have to set up database mirroring end points,
they need to set up the certificates.
There are multiple steps to even set up the cluster.
Once you tell us that you want HA and
big data clusters for SQL Server Master,
we take care of everything for you.
>> Wow. So we simplified it, right?
>> It's very easy for you to set up.
You don't have to think about,
did I use the right URL for the replicas or things like that?
You don't have to worry about these things.
>> Cool,so.
>> Guess what is the cluster technology that we use for that?
>> Well, you tell me.
>> None. So that's the beauty of Kubernetes.
So through a tight integration with Kubernetes and adding
the logic for monitoring and
orchestration tight into this control plane,
there is no need for additional cluster of technology
to put in the bigger cluster
to manage that aspect of SQL Server Master.
>> Okay.
>> Similarly for HDFS, right?
Other resources from the Hadoop stack,
they have to be highly available as well.
In this case, we use Zookeeper,
which is an open-source.
So well established cluster technology
to help with orchestration and
storing the metadata for the high availability of these services.
>> So you told us about mission critical,
you told us about security.
So you got my head spinning now.
There's a lot of stuff going on,
but how do I actually use this?
Do you have tools that
you can give to
me to make sure that I understand what's going on here?
>> Yes, don't worry about it.
So again, I was telling you that you can easily deploy, right?
>> All right.
>> The only thing that you need to do
as with anything in Kubernetes you just
have to declare your intention and describe
your target configuration and we take care of everything.
So one of the thing you want
to make sure you're using and you have
as a tool on your client machines ASI data.
You can do deployment, you can do configuration,
you can do monitoring even with ASI data as well as Azure data.
So if you want to go through a more guided or to see dashboards in
a more user-friendly way
and that's what I'm going to show you further,
you can make use of Azure Data Studio to
leverage some of these experiences that we're adding there.
>> Okay. Well, talking about experiences, can you show us?
>> Let's see how that looks like for monitoring.
If I want to see the status of the cluster,
are my services healthy or not.
>> There's a lot going on in the cluster.
So I need to know what's going.
>> There are many of them exactly and we have a new experience
in Azure Data Studio where you
can see the status of the cluster through the controller,
because again this is the brain of
your cluster and this is the source of truth.
>> Sure.
>> You can see here all of the services that are deployed.
There are health status.
If I want details,
I can go further to HDFS for example
and see what is the health there and so on.
So this is one of the things that we have new
in the upcoming releases for big data clusters.
All this experience again you can use ASI data for
as well with the BDC status,
and it's more intuitive to use a tool like this.
>> Sometimes. Yeah, makes sense. Cool.
>> Again, all this and how to deploy
all these services and
highly available configuration for security and so on,
you can find on our documentation page.
I put some pointers here for you to
leverage as a starting point for our documentation.
So either for deployment or for more resources on BDC,
workshops, samples that you can go
to these resources to make use of.
>> Cool. Well again,
thanks a lot for sharing.
I'm happy to see there's a lot of
administration and monitoring going on,
and very happy to see that we have
both a command line version of it to automate things again,
building graphs, building dashboards,
and then the interface, I like interface as well.
So I'm very happy to [inaudible].
So thanks a lot for being here and sharing this with others.
>> No problem, thank you.
>> Thank you for watching.
Please like and subscribe,
leave a comment and hope to see you next time. Thanks.
[MUSIC]