>> Hey friends. Did you know that Azure API management has
a DevOps resource kit that helps you
automate the deployment of your APIs into API management?
Using that resource kit,
you can create an ARM template that enables you
to deploy your API in minutes.
In this, the first of two episodes,
Meow is going to cover the conceptual framework
behind it today on Azure Friday.
[MUSIC].
>> Hey folks, I'm Scott Hanselman and it's Azure Friday.
I'm here with Meow who's going to talk to me
about Azure API management.
You get a whole DevOps resource
pipeline management system around this.
You're really formalizing the way
that we deal with API management.
>> That's right. Because when we talk to our customers,
one of the most common questions that
come up today is that how do I build
CICD pipeline to automate
the deployment of my APIs into API management?
That's why we created
this resource kit to help with our customers.
So when you start thinking
about building a CICD pipeline for API management,
it will quickly run into a few challenges.
That's why I'm here to talk about it.
So what are the challenges?
Essentially, we know that organizations today use
multiple deployments environments such as Dev and Production.
So as an API developer,
once I have finished deploying and
developing the back end service off my API,
how do I automate the deployments of my API into API management?
That's the first challenge.
Second, once I have tested my API in a Dev environment,
how do I migrate all the configurations of
this API from a Dev instance into my production instance?
That's the second challenge. Some of
these configurations included things like in my API definition,
API management policies, authentication settings,
diagnostic settings and so forth.
So how do I make sure that nothing gets lost during the migration?
That's the second challenge.
Third, we know that an API management instance is often
a shared asset between multiple API teams and each
of this team is responsible for one or more APIs.
So if multiple teams are making
changes to a shared API management instance,
how do we make sure that they do not step on each other?
That's the third challenge.
Unfortunately, the answer to this challenge is that it depends,
there is no one size fits all solution.
There are multiple factors you need to consider here.
First of all, what's your team structure?
Do you have a single API team or do you have multiple API teams?
Also, what is your engineering culture?
Is it a low trust culture where you have
a central team who's responsible for everything or
is it a high trust culture where
each API team is responsible for deploying their APIs directly?
Also, you need to think about how to
implement a CICD pipeline using your existing tools.
Finally, obviously as an Azure service,
API management supports a variety of deployment options.
For example, there's the Azure Portal
which works for some customers who
have few APIs and who do not need to update those APIs very often.
But using the Portal is certainly not a automated approach.
So to automate your deployment,
you can use our management API or PowerShell commands.
It works for many customers but through some others,
it feels a little bit imperative.
We also have the classic Git deployment feature
which was shipped about four years ago.
It works for some customers but it also has its limitations.
For example, it does not support
RBAC and it does not support partial deployment.
If I just want to deploy a single API,
I still need to deploy all the files
in this Git repository together.
Also, the files in this repository uses
our custom JSON format which is not consistent was ARM.
So finally, there's Resource Manager templates aka ARM templates.
So if you're starting from scratch today,
this is our recommended deployment option for a few reasons.
First of all, it is consistent with
the rest of Azure and second it is
very easy to automate the deployment of ARM Templates and third,
ARM Templates, they are declarative and idempotent.
Also, if you modularize your templates properly,
it allows you to do partial deployment,
it gives you a lot of flexibility.
Finally, ARM natively supports RBAC.
So with that being said,
let's take a look at a suggested approach of implementing
a CICD pipeline to automate the deployment of
your APIs into API management using ARM Templates.
>> All right.
>> So in this example, let's say
my organization has two deployment environments,
production and development and each of
this environment has its own API management instance.
The production instance is managed by a central team called
the API publishers and API developers who work on APIs,
they only have access to
the development instance for testing their APIs.
So the key in this approach is to
keep all your configurations in ARM templates.
Also, you need to keep your templates in a source control system.
So in my example here, I'm going to use Git as an example
but you can use any source control system that you prefer.
If you look at here for my production API management instance,
I have a Git repository called the publisher repository which
contains the configurations of
this API management instance in ARM templates.
If you look inside the repository,
there are different types of templates.
First of all, I have a service template which contains
all the service level configurations of
this API management instance such as custom domain names,
number of scaling units,
managed identities and so forth.
Then I will have a bunch of shared templates for
shared resources in this API management instance such as products,
groups, loggers and so forth.
Then for each individual API,
I will have one or more API templates which contains
all the configurations for this API such as API definition,
authentication settings, diagnostic settings,
policies and so forth.
So each of these templates can be deployed individually.
So if I just want to deploy a single API,
I can just deploy the API template for this API.
But I can also use a master template which
links to everything if I want to deploy everything in one shot.
>> Is not a meta template,
a template of templates,
it refers to or it includes the previous, the other two?
>> Yes, it's a template that links
to all the other templates, yes.
Then for API developers,
they will fork and clone my publisher repository
into their developer repository and then they can work
on the templates for their APIs and
deploy their APIs into the Dev Instance for testing.
However, if we think about its approach,
there are two challenges here.
The first challenge is that we know that some customers,
they still prefer to use
the Azure portal to configure and publish their APIs.
So once they have configured their APIs,
how can we help them to extract
their configurations into ARM templates so they can
migrate these changes from
their Dev Instance to another environment?
So that's the first challenge.
So for that a challenge,
we have created a utility tool called Extractor which can
help our customers to extract
their configurations into ARM templates.
The second challenge here is that we know that API developers,
they often work with open API specification, aka swagger files.
So they may not be necessarily
familiar with the resource manager schema.
So if I'm an API developer and
I have open API specification for my API,
how can I easily create an ARM template
to help me to deploy
this open API specification into API management?
That's the second challenge.
So for that challenge, we have created
a second tool called the Creator which can take
an open API spec and optionally you can provide
the policies you want to apply it to this API in XML file.
So the tool will take the open API spec
and the policy file and convert them into
an API template so that you can deploy
your API into the API management instance.
>> So in that context,
the API definition expressed by
open API and the expression of the ARM template,
which is much more physical while one is more logical
doesn't contain enough context to go fully back and forth.
It's not an item potent experience.
So you have to add additional metadata during that point,
and that's the XML file to policy that goes along for the ride.
>> So the policy is for API management policy but we also
have a configuration of file which is
exactly what you're talking about for this tool.
So I will show in a minute.
>> Okay, great.
>> Cool. So once the API developers
have their API templates created,
once they have deployed
your APIs into the Dev Instance for testing,
right now they can submit
a pull request back to the publisher repository,
and this is when the API publishers can
validate the templates in this pull requests.
They can validate if the templates are safe and compliant,
and a lot of these validations can be automated.
After that, the pull requests will be merged,
the templates will be added to the publisher repository
and the changes can be published to the production instance.
So this is the conceptual framework.
So if we think about it,
there are multiple benefits here.
The first one is that it since everything is in ARM templates,
it is very easy for you to automate the deployment of your APIs.
Second, again, since everything is in ARM templates,
it is very easy for you to migrate
your changes from one environment to another environment.
Third, because different API teams,
different API developers will be a
working on different API templates,
different files, so it is less
likely for them to step on each other.
Finally, this is a very flexible approach.
You can easily adjust it to fit your own requirements.
For example, you can use different tools to implement this flow.
You can easily add
a different deployment environment here or instead of having
a central API publisher team you can also allow
each API development team to deploy their APIs to
the production directly with proper RBAC setup.
So it's a very flexible approach
and everything I've just talked about as well as the tools,
they're published on GitHub at
this link and the tools are open-sourced.
So you can also customize and extend them
if it doesn't fit your needs.
Also, community contribution is greatly appreciated.
>> Can I ask a couple quick questions?
>> Sure.
>> So one of the questions I have on the development side,
while I think things are quite clear on
the production side particularly around
the forking and then ultimately
pull requests going through to publisher repository.
But as I noticed that
the API developers are writing API definitions,
is the API definition the authoritative source?
If we use the term source code to
indicate if I could delete something and then build again.
I could theoretically delete some of the ARM templates
in the developer repository and recreate it again.
>> Correct.
>> Because the source is the combination of
the API definition plus additional policy data.
>> That's correct. If you delete something by mistake,
you can rerun the Creator to recreate all your templates.
>> Okay. So then I would not want to manually
ever edit ARM templates in any of the repositories.
The source code is on the right-hand side.
>> That's right.
>> That's very clean.
>> Yeah.
>> All right. Well, we're going to take a break here for part one.
This is the conceptual framework about
how we're going to do this and then in part two,
we'll put it into practice today on Azure Friday.
[MUSIC].