Azure Data shared now supports the ability to share databases
in place from Azure data. Explorer clusters with no data
movement Azure data. Explorer is a fast fully managed data
analysis service for real. Time analysis of large volumes of
streaming data. Joanna is here to show us how in place sharing
works for Azure data. Explorer today on Azure Friday. Hey,
everyone, Donovan Brown here with another episode of Azure
Friday. I'm here with Joanna and she's going to show us about.
In place data share with Azure Data shared I get that right.
Yeah, that's right very cool. So what is in place data share so
festival Azure data? Share is a service that we GA just last
month at ignite so Azure data. Share allows you to do cross
tenant 3rd party data sharing with your customers and
partners. It's safe. It's flexible and it's secure so one
of the great things about data share is that you can actually
create a new data share shared with your customers and
partners. There's no code to write there's no infrastructure
to manage super easy to maintain and you also we've said the word
in place several times in place sharing as opposed to what other
type of sharing yeah. I'm glad you're out so we have a couple
of different types of sharing mode. So where in GA with
snapshot based sharing from Azure Synapse Azure. SQL
database and Azure data. Lake, an Azure Blob Storage. So you
can receive a copy of your data in your continuous tenant, but
now the exciting thing that we've now in preview with is the
ability to do in place sharing which is basically sharing via
Ray symbolically. Oh, I see and then is that in place now
supported across all the different data sources. You just
mentioned in snapshot or is it just a subset of those right
now, so right now with supporting in place sharing
using Azure daughter. Explorer, an we heard trainable more data
sources in the future but we're super excited about the ability
to do in place with Azure daughter, Explorer. OK got it.
So the cool thing about that is that you can do near real time
data sharing of your clusters in databases in Azure daughter,
Explorer. So if I've got so Azure data. Explorer great
technology for real time.
Streaming so if you're streaming through iot hub or Kafka,
streaming that's a really great way to gather telemetry logs
etc. So what we're actually offering today is the ability to
share that data that you're gathering with your consumers in
real time in the benefit. There is like with the snapshot. I
have to give you decide I'm going to take a snapshot and
however long it takes me to get you that data you're missing
that like there's the latency, there, but now it's like you
need the data. Here's the data and you're getting, it in real
time with no latency exactly got
it. Yeah, so some of these scenarios that we're seeing on
custom is we have a need for this requirement is a lot of
organizations may be getting Telemetry, daughter into their
database and then they may be sharing that with other
departments in their organization so rather than than
bearing all the costs of sharing that data in maintaining the
cluster. They can actually have a charge back model or load
balance. If they need to so for example, if you have data. We
are gathering and you're the HR
Department for example. And you you have finance and all these
other Department sharing that they can actually subscribe to
your data share and they could receive the data in near real
time, but they're going to pay for the compute interesting. Now
we just talked about a scenario where everyone's lives inside of
the same company but I would assume that you could use this
to also share with 3rd party companies as well. That's right
and that's kind of one of the really exciting features that
we're offering as well. So what we have is the ability to do
cross tenant data sharing so if you're a data provider and you
want to monetize your data or if you want to expand your business
model. You can actually share that data with your customers
and partners in near real time, so in a lot of organizations.
They share their data. Or they may have interesting data that
they monetize and they drop that to an FTP site or they may be
backing up a SQL Server database and shipping that to their
customers. So there's a lot of administrative overhead with
that standing up FTP infrastructure, having scheduled
backups. Also, the lag between when the data is received, and
when the consumer can get access to that data OK.
So you're going to be able to show us how to set all that
stuff up and help with networking right perfect. Yeah,
OK, so here I am with my Azure data. Share resource which I've
already deployed an I'm going to go ahead and click start sharing
your data so in this scenario. I'm basically gathering iot
sensor data from a number of thermostats. The deployed all
around the building and what I want to do is I want to share
this data in near real time with my data partners. So the owners
of the building. They want to know what the temperatures are
like on average throughout the different rooms in the building.
So I'm going to give this sharing name. It's going to be
cold thermostat data so this is the new exciting feature that in
place sharing capability. So I'm going to go ahead and create a
description so that my consumer knows what I'm actually sharing
with them. So I'm going to say thermostat data and then terms
of use. They can refer to a contract and they'll actually
need to actually accept these terms of use before they even
gain access to the data share and then here. I'm adding data
sets, so today, we're in preview with Azure Data Explorer.
So I'm going to add in the Azure data. Explorer and I'm going to
navigate to a cluster that I want to share data from so the
cool thing about this is that I can actually share any entire
Azure data. Explorer cluster if I want to or I can just share an
database. So it's really up to me. The level of granularity
that I want to share on so if I was to share on the cluster
level, then every time I add a new database to the cluster the
receiver, will actually automatically have that new
database actually similarly if I add a new table.
That the data consumer will actually see that new table pop
up pretty quickly. So just for the purpose of this demo. I'm
just going to add in this one table and I'm going to give this
data set a descriptive name thermostat data and then in this
next screen here. I can add recipients. So the cool thing
about data share is that you can share with multiple 3rd party
recipients outside of your organization. So I'm going to
add in myself. I'll add in Donovan as well, just for fun.
I can add more in if I choose 2 and then once I click create
this is when the data share gets created data sets get added an
then invitations get sent to my recipients, so at this point in
time. I'm essentially waiting for those recipients to accept
the invitation. Another really awesome feature of the product
is that you can actually monitor the state of an invitation so
right now. You can see that both parties that I invited still
have pending invitations and you'll actually see this change
too. Subscription once they've created a share
subscription to the dotted that I'm sharing with them.
And I can go there and I consume I can revoke them. If I needed
to since I know the status of them. Yeah, exactly right so if
you decide that you no longer want them to have access to your
Azure daughter, Explorer cluster
you just click revoke. So as the data consumers so I'm switching
hats to the data consumer now an I have this invitation that I've
received now if I'm a data consumer that's a net new to
Azure Super easy for me to get started with Azure. So I can
create an you free Azure subscription. I can also review
detailed instructions on how to actually configure the data
share that's being shared with me. So we may make it super easy
to use so now I'm going to switch over into the fabric AM.
Subscription so fabric Ham is the data consumer that's
actually going to be receiving this data from Maine.
So if I look at my data sharing invitations pain, I can see that
there is a pending invitation called thermostat data, an you
can see the company details. You can see the number of data sets
that are being shared with me and you can also refer to the
contract and if I agree to those terms of use then I can check I
agree if I don't then that won't actually work. So I won't be
able to accept the share into my
data Share Account. So here I'm actually selecting an existing
data Share Account to deploy this into but if I didn't
already have a data Share Account as the consumer. I can
easily just create a new one by saying create knew OK.
Sir I'm going to go ahead and accept and configure now, So
what that means is I've actually accepted that invitation. An
I've created a share subscription so there anything
left to do now is to map the data set so the way that I do
that is. I go into the data sets tab and then here. I actually
map to an Azure data. Explorer cluster which is in my
subscription. So the implications of that are pretty
cool because I'm actually going to run the compute.
In my environment, even though I'm querying the
providers tenant so that's a really cool capability in
customers. Love the fact that they don't actually have to
incur the cost of compute if they're sharing data.
So here I'm just going to select the Azure daughter.
Explorer cluster that I provision for the purpose of
this data share and then I'm just going to click map to
target.
And this is essentially taking the data that was shared with me
and making it look like it's a part of my data Explorer on my
side now that's right so to me, it's just another table. Another
database that I have access to that I can. Manipulate Yep got
it exactly right and then this mapping when it's done.
Do it that's when I get to indication on the other side
that this invitation has been accepted an now mapped to a
particular cluster. So so once I create a share subscription and
accept the invitation that invitation the provider, knows
that I've accepted the invitation. But once I configure
the mapping. I actually get access to actually start
querying the data so right now. You can see that the mapping has
succeeded so I'm actually ready to start querying the providers
data so for me to do that and you know, some of you might be
new to Azure Data Explorer. It's such a great service.
Particularly for iot daughter if you've got sensor data that
you're collecting Telemetry data log data. It's a great service
to essentially stream from different sources, so if I click
on this database. That's being shared with me. For me to
actually gain access. I first need to grab myself permission
to this database. So I just go into permissions and then I add
myself in so that's all. I need to do. To actually get this to
be able to start querying that database so I'm going to add in.
My alias.
So this is basically saying just because the company has access
to the data. Not everyone in my company has access to this data,
I get to now come in and choose selectively. This data was
shared with us, but I only want Donovan only want Joanna taxi
have access to that data that's what we're doing now. Yeah,
exactly right so right now. I have a live connection to the
providers Azure dot. RX plorer So what I'm going to show you
now is. So what I'm going to show you now is some thermostat
data. So you can see right now that I can query that data and I
can just get a sample of 10 rose, but where this gets
interesting is the fact that this is near real time data so
if I got my provider cluster.
An I can go into that cluster and let's just have a look at
it. So I'm back on the provider, subscription at this point in
time So what I want to do is I want to actually demonstrate
some live data. That's being inserted into my database. So
here on the provider cluster. I'm going to go into the
occupancy database, which is where my thermostat data resides
and here I'm going to so you can see I only have 3 tables right
now in this database and its
occupancy database. So what I'm going to do now is I'm going to
actually create a new table. So this is my this is my provider
and then over here on the right hand tab my fabric. Ham tenant
is my consumer and you can see just 3 tables right now, so I'm
back over to my provider and if I click run an create a table
and then I insert some data into
that table. What you will find is that if I then switch over
to my consumer tenant I'll be able to see that table as soon
as it's actually being created so it preserves the query so
if I then refresh.
You can see that new table is being created already now In
addition to that. I've also got some live data streaming into
this occupancy database so right now, you can see that the count
of the roses.
310,517. So we have data streaming into this database
every 30 seconds or so OK, so if I query. This data again. I
should see that number increments. So you can see that
the number of races incremented too. You know by about 4:00 or
so. So basically that Azure data. Explorer implies sharing
in a nutshell perfect so the ability for not only.
Internal teams but even external companies to be able to share
data with their companies and have complete control and I
really like the fact that the person who's consuming the data
is paid like paying for it. Not the person who is sharing it.
That's right and just on that point, so if you're in data
provider you actually will incur the initial dot IO rate costs.
But there is a caching policy. You can set on the data consumer
side where you can set aside in case you policies so that you
only read the data once on the provider side and then you never
really need to read it again
unless it's updated. So so there is that benefit as well perfect.
Thank you so much. We are learning all about Azure data
share here on Azure Friday.