>> Did you know that you can deploy Azure SQL
anywhere with Azure Arc-enabled data services?
Learn more from Ben in this episode of Data Exposed, MVP Edition.
[MUSIC].
>> Hi, I'm Anna Hoffman.
Welcome to this episode of Data Exposed, MVP Edition.
Today, I'm joined by Ben.
Ben, thanks for joining us today again.
Can you start by telling us a little bit about yourself?
>> Thank you for having me again.
I wasn't sure if I was ever going to be invited again.
But then again, that was my best behavior last time.
MVP from Nuremberg, Germany,
mostly in the data warehousing and BI field.
Last time we spoke about big data clusters.
Well, now I guess I'm here for something new.
>> Yeah, we're excited to be learning something new.
It's this ever-evolving thing and I'm excited to
learn what you have to share with us today about Azure Arc.
Without too much preamble,
I thought I'd just ask you,
can you tell our viewers, and myself,
what is Azure Arc and how does it relate
to things we know like Azure SQL and SQL Server?
>> Azure Arc is this real cool thing.
It is bring your own datacenter,
but still enjoy the luxury of
Azure services in general, I should rather say so.
Azure Arc is not just for the data world,
but we're going to be focusing mainly on the data world.
The idea of Azure Arc is you go to any Cloud
with any infrastructure and any Kubernetes.
This can be Azure Kubernetes service,
you can run your own Kubernetes.
I'm basically running my own Arc datacenter in my basement,
which we're going to see in that emulator.
You can run it in any of the other public
or private Cloud, AWS, Google Cloud,
you name them, which means wherever you go, it doesn't matter.
It can be on-premise, it can be private, can be public.
It can be connected and disconnected.
We're also going to see that in the other slide.
There's data services, there's our Kubernetes services.
There's all kinds of
Azure services that are moving to other clouds.
Again, on-premise is a Cloud in that context.
The idea for the data world here is we get managed instances and
PostgreSQL Hyperscale into your own private estate.
still having all the luxuries that you would get
from the classic Azure SQL service world.
Classic, I mean, they're all brand new and
still you get to see them in the portal,
you get to manage them still on-prem,
you get manage backup,
you get manage updates, all that.
But all that without having to be locked into the Azure Cloud,
which is a big deal for many people that either
run multi-Cloud strategies or that just say,
"Well, I cannot move away from on-prem with
some of my data for performance rate."
You may just be sitting on an oil rig and just not have
the connectivity to everything to run
into the Azure Cloud non-stop.
What does that mean for the Azure Arc-enabled data services?
You start by just deploying your Kubernetes cluster.
I keep that rather gentle because like I said,
this can be Kubeadm, AKS,
EKS, all kinds of flavors.
On top of that, you deploy a so-called Azure Arc Data Controller.
This is like your main endpoint.
If we remember big data clusters,
we had the big data cluster controller.
This is the counterpart to the Azure Arc if you want.
One single point of entry,
and that's the only thing that you really have to deploy.
This will take care of everything else that's going on there.
This is happening again through the Azure Data CLI,
through AC data, which we are also using for big data clusters.
Azure Arc or Azure Arc-enabled data services,
I should rather say, that is the answer to that question,
the end of the path where people are wondering, "Well,
SQL and Linux, SQL and Kubernetes,
I see what it's doing.
Why?" This is why,
because now we get all this in those containers,
in those orchestrated containers.
Just like with big data clusters, again,
we can either use AC data
directly or we can use Azure Data Studio,
a tool that more and more people are
using for exactly those purposes: managing,
deploying, and developing their data solutions.
We got it connected to the Azure portal.
Again, same logic.
We will also be deploying our data instances, again,
like I said, Azure SQL Managed Instance
or Azure Database for PostgreSQL.
Those are the two offerings we would get
in Arc-enabled data services right now.
By the way, Arc-enabled data services are in public preview,
so we can still get monthly releases
with more and more interesting features coming.
Since when Azure Data Studio can even
automate all that through Git merges if you want,
you can have automated,
built and deployment pipelines for your data estate;
and you get the same Kibana and Grafana interfaces that
you already know from SQL on Kubernetes and big data clusters;
and we got the link back to the portal.
Like I said, depending on where you are,
you may have a direct connection to the Internet,
to the Azure Cloud or not.
That's why there's basically two ways of getting
those metrics and log files into the portal.
You can do the direct mode,
which means your data controller will constantly upload
your consumption data and
log files for analysis to the Azure portal;
or you can use the indirect mode.
Indirect mode basically means
that you're running a script every hour,
every minute, every week, every month,
that will just push that data to the portal so
your resources will show up there and are available for analysis.
Let's take a quick look at that in Azure Data Studio.
This is just PowerShell what I'm doing here,
but I decided just to run it in an ADS notebook,
so we've got it all in one place.
I already got my controller here in my basement deployed.
What I can do is I can see, well,
I got already three managed instances and one
on PostgreSQL instance deployed there.
I can manage all that right here in
Azure Data Studio but I can also
just run a PowerShell command where I can say,
"Show me all those instances."
Again, I can do that through code or through
the user interface depending on what I prefer
and I will see my three managed instances
showing up including their external endpoints.
Well, external from the Kubernetes perspective,
those are private IP addresses because I'm running that in
my own private basement datacenter.
If I want to create a new Managed Instance,
all I would need to do is run this command here,
azdata arc sql mi create.
If I would do this and just do go-titans 01.
I think Bob What he's
a big fan of the Titans so I'm doing this especially for him.
>> Huge fan.
>> Well, him and Buck, of course,
because Buck is the biggest sports fan that is out there.
Again, if I would be preferring
something with a graphical interface,
I could have gone here and say, "New Instance."
Basically doing the same thing so this would,
exactly the user agreement.
I could say, "Well, this will be
my go-titans too," and I could either
just script it to a notebook that I could
run later or deploy it right from here.
We're sticking to just running it right here in PowerShell.
What this will do is, in my Kubernetes cluster,
it will create another part, go-titans-01 - zero.
That is, by the way, super-interesting in Azure arc.
This will also be the platform which allows me to
run which is why it's saying, "Dash zero."
There could be a dash one, dash two if I had multiple instances.
This would be the way to bring SQL Server Availability Groups into
Kubernetes which is currently not supported by
the regular SQL Server, quote, unquote.
This is running, this is creating
my Instance and if I would go back here and refresh this,
we would also see this is being created.
We see this Instant as Creating
and once you've got your first Instance created,
all the consecutive one usually take
about a minute to a minute and a half.
There we go, go-titans-01 is ready and if I do a refresh here,
I would see the Instance just as well if I do
the azdata arc sql mi list here,
it would show up the Instance including its endpoint and including
its IP address and port so I could connect to it. Pretty cool.
>> Nice. This is pretty cool.
It's a good bit faster than installing SQL Server.
>> Slightly. For those trying this at home,
the first time will take a little bit longer because
it actually needs to download SQL and that's
again the beauty of Kubernetes once it's got all these images
downloaded to its local library if you want so.
Ninety seconds for a SQL server,
that's not just a regular SQL Server because
if we just go here and do a select,
add, add version on one of my Managed Instances,
what you will see is this is not just SQL Server or SQL Server on
Linux but what I have here running on my local premises,
I got my Microsoft Azure SQL Database Managed Instance Azure Arc.
This is a Managed Instance.
This would get automated updates,
this would get automated backups,
all that, if configured for that specific instance.
I could get all the local information
that or in my Grafana dashboard.
If I go here, where of course I did not deploy a certificate,
this would get me my SQL Server metrics.
I could go to all the different instances
that I got deployed here.
How many transactions, how many batch requests I could see,
what's actually happening on my host but I can also get that
back to the Azure portal and this
is where it gets super cool if you ask me.
Because that means in the end,
no matter where my databases are running,
no matter where my instances are running,
I get them all in one single place.
The first thing I'm doing is I'm exporting the LogFiles here.
This is something that you should schedule in
the real world so have
that run every two minutes, every two hours.
Unless you're in direct mode where like I said,
this would be happening automatically.
What this will do is it will collect
all the LogFiles and metrics from all these instances.
You can see, it already picked up
my go-titan so it's not like I have to configure anything,
it will just do it for every single Instance that I've deployed.
Then the next step, I will just upload those to the portal again
using azdata so what this
will do is will take the files that I just created,
the two JSON files.
Push them into my Log Analytics workspace in Azure and if I
go into my Azure portal and do a refresh here,
you will see there's
my local-mi-01 where I
could now see the metrics for
that specific instance right here in the Azure portal.
What happened to my CPU on that On-premise SQL Instance?
I see it in the portal.
If I go to the overview, I would also see,
well, it's running in Ben's basement.
This is my endpoint so
this is where it's all coming together and this
is the beauty of Azure Arc-enabled Data Services
or Azure Arc in general.
>> That's pretty cool because that means
wherever your SQL is running,
if you have it connected to the Azure portal,
you're going to be able to manage
and monitor them all from one place.
Am I understanding that correctly?
>> Correct. Without the worry
of running your own updates and all that.
All the beauty that you get from managed services
on your own infrastructure or somebody else's infrastructure.
It's like I said, it's bring your own data center if you want so.
>> Love it.
>> I think that is amazing because it really gives you
that flexibility no matter where that restriction is coming from.
>> Awesome, cool. Well, thanks, Ben.
This has been really interesting and for our viewers,
you've actually been pretty busy with
Azure Arc-enabled data services.
I think you might have something you
might want to announce or share with our viewers.
>> Indeed, my very good friend Anthony Enosantino and myself,
we wrote a quick book on
Azure Arc-enabled data services because
I'm known for writing book on
technology where you will know
for sure that the technology will change
five-times until the final product comes out.
This is based on the current public preview
but it was big fun writing that.
Trevis, the PM for
Arc-Enabled data service actually was nice enough
to write us a great forward for that book.
If you want to find out more about
Azure Arc-enabled data services, how to deploy them,
also more about the technical background,
make sure to check out our book,
I hope it's worth it and let us know what you think.
>> Awesome. Thanks, Ben, and I think for our viewers,
it's a great call to action.
You want to learn more about Azure Arc-enabled data services,
then definitely check out Ben and Anthony's book.
We'll put some links in the description.
Ben, again, thanks so much for joining me today,
and to our viewers, thanks for tuning in.
We hope to see you next time on Data Exposed.
[MUSIC]