>> Hey friends, did you know that you can now
parameterize the connections to
your data storage in Azure Data Factory?
Makes it much easier when
working with multiple data stores,
and to manage all those connections.
Gaurav Malhotra is here to show us how
it works today on Azure Friday.
Hey folks.
I'm Scott Hanselman, this is
another episode of Azure Friday.
We've been talking about the Azure Data Factory,
and I was complimenting you and your team about
the speed at which you were making improvements.
You're really doing everything you can
to smooth things out,
whether it be something small that adds up,
and changes someone's workflow fundamentally.
>> Yeah.
>> Now, you're adding a parameterized
connections for data storage.
>> Yes.
>> How's that going to make my life easier?
>> So, we have had feedback from our customers,
right, where they are, let's say,
connecting to 10 SQL Servers,
On-prem SQL Server or SQL Cloud Database,
and the only thing that's different
between those 10 SQL Servers is the server name,
because for the consistency reasons,
they're keeping the same username and password.
So, as before this release,
they had to create 10 separate connections to
10 separate SQL Servers in Data Factory.
But now what we're seeing is that you can
create a single link service,
and pattern retries that server name.
So that now from a manageability aspect,
you are just managing one link service in
your Data Factory rather than 10 separate ones.
Also if you are an ISP,
or a System Integrator who is
coding using our REST APIs or
using our PowerShell commandlets or using
ARM templates approach to create your pipelines,
you would just have one link service for
the pattern retries server name in there,
and at runtime you can provide values to that.
>> Interesting. It seems
like there's a theme here because in
another episode of Azure Friday that folks can checkout,
you talked about how the integration runtime would
ordinarily have to have one for each,
and now it's kind of normalized.
>> Yes.
>> You're doing some normalization across looking for
places where things are unnecessarily duplicated.
>> Yes. So, our goal is as
a team is to improve productivity for our customers,
and as we would see in our different releases,
we're very closely watching customer feedback,
and based on that seeing how
we can make them more productive,
and make things usable for them.
So, this is another example where
we have listened to customer feedback,
and we have released this new feature where you
can now parametrize your link services in Data Factory.
>> Can you show me what that looks like?
>> Yes. So, again this is Azure portal.
I have a DemoFactory created in it.
If I click "Author and Monitor",
it takes me to this
visual tools that we have for Data Factory.
>> Okay.
>> If I go into "Author" tab over here,
I have this Resource Explorer
where I can see all my pipelines,
and I have this connections tab.
So, if I click this "Connections" tab,
it opens up this,
and you can see that I have a bunch of
connections already created over here.
So, if I click the "Edit" for a particular connection,
and in this particular case,
you can see that now we allow you to parametrize,
and the way to know it is as you hover,
you can see that this "Add Dynamic Content" is coming.
>> So, all those places where someone
potentially could have hard-coded something?
>> Yes. So, you can see that it
is coming for your server name,
for our database name.
So, let's click on the one for database name, right?
So, it takes you to this site banner where now you
can parametrize your database name
for this particular SQL Server.
So, you can use the rich expression language
that we have for Data Factory.
Now, in this case again
usability is the key for our customers,
that we're trying to enable,
and you can see there if you pasted this,
it is telling you "It's invalid" because I still
haven't created a DB name parameter.
But I can do it right from here instead of going back,
and creating that parameter,
and coming back here.
If I click "Plus" new here,
and I say DBName,
and it is of type string,
and I don't want to give any default value.
So, if I click "Save".
If I wanted to give a default value I could have.
>> You could kind of hardcode without hardcoding?
>> Yes.
>> Given a default value, and
then override it if you needed to.
>> Yes. So, it has created this parameter,
and now you can see there it's all valid,
I'm not getting that error message.
>> Right.
>> I can say "Finish",
and not only this,
like I said you could parametrize,
as you can see your server name
in a similar fashion as well.
So, it's up to you what do you want to do,
and parametrize and all these values
will be passed at runtime.
So, if you had 10 SQL Servers at runtime,
lets assume you are maintaining a file,
and you could go lookup
against that file to get the list of those,
and you can pass it to this link server.
>> So, help me understand the scoping,
that when we create a variable and then
putting variables in air quotes like this.
Is it's scoped at the pipeline level?
Is it's scoped at anywhere within my entire factory?
How does it work? Do I have to think about scoping?
>> Yes. So, this particular parameter is
scope at the DR pipeline level, right?
There are different scopes that will enable you create
parameters at dataset level as well,
and you can pass values to
those parameters from your pipeline to dataset,
and then do your link service. Right?
>> Okay.
>> So, there are different scopes that we enable,
and there is a mechanism by which you can pass
parameters across these different scopes.
>> I see, okay.
>> Yeah. So, this is all
available today for customers to use.
We have our documentation
published on how you can get started with this,
and start using our visual tools, and parametrize.
We have the JSON.
If you want to use the JSON approach,
and do it using that,
you can do it using JSON as well.
>> Do you have a particular chip
or a strategy for someone who
might have recognized this as a problem,
and they've got a bunch of copies of data sources.
So, they should need to delete them all on mass,
and then add one, and then go back and add them manually?
Or should they used the JSON technique?
>> So, I would say they should just go
use our programmatic interface to go delete all of those,
and then parametrize the link service, and use that.
Because from a manageability aspect or in future,
if let's say a username,
password changes or if you have
a new server that gets added into the mix,
instead of adding a new link service,
all you need to say at
runtime pass this additional value,
because I have one new server to cater to.
>> Got it. Got it. Very cool. So, it's available now,
and people are just going to get it the
next time they log in.
>> Yes.
>> When they see the new portal.
>> Yes. Exactly.
>> Fantastic. All right,
lots of great improvements in the Azure Data Factory,
and if you just saw this one video,
I would encourage you to check out
all of our other great videos,
not just on Azure Friday because we
wrote over 500 videos now.
But also our great Azure Data Factory videos season.