Coming up on the show, with orchestration and containers taking over a modern app management.
We're going to take a look at the role of kubernetes, now generally available.
For managing reliable and scalable cloud native apps using Linux and Windows containers.
We're going to look at the advantages to moving to container based modern approaches.
How kubernetes compares to existing IT tools and processes.
And what IT needs to do to take advantage of this and provide infrastructure for containers to run.
Microsoft Mechanics
I'm joined today by Brendan Burns,
the co-creator of kubernetes who leads the azure container services team at Microsoft, Welcome.
Great to be on the show, thanks for having me.
So kubernetes is really one of three different open-source orchestrators that
are supported by Azure container services.
It's just been released to general availability.
But why is it that we're seeing orchestrators and containers
taking over the way that people are thinking about delivering apps and services?
Well I think it's really about that handoff from developer to operator.
And there's a number of different reasons for that.
For the first it's fidelity.
So in the past as an operator you might be handed a package from a developer,
roll it out to the cloud and it would fail.
You would go back and the developer would say "well it worked on my machine".
But now with containers ,
all of the things that you need like libraries
and all the configuration files are moving into the container into that package.
So it's exactly the same thing that runs on the developers laptop
as runs in the cloud.
And along with that packaging up what we're seeing is
this decoupling of application and operating system.
So that you can have a developer worry about the application package itself,
while an operating system operator worries about the operating system itself.
And then finally orchestration provides this application oriented API
that makes it easier for people to focus on
rolling out, managing and other aspects of their application itself.
Without worrying necessarily about the details of the operating system,
So all of those things together make it easier
to reliably deploy an application to the cloud or to on-prem.
because of this abstraction layer
really there's no patching involved.
And you can also start doing things like swapping out the underlying foundation
with the operating systems and not affect your apps.
Yeah, there's really no patching at all in either the application or the operating system.
Each new version of the application
is deployed next to the old version independent of it.
And when you're done with the old version it's simply torn away.
And the same thing is true in the cloud with the virtual machines.
If I upgrade the operating system
I actually just create a whole new virtual machine and destroy the old one.
So I'm never doing any of this imperative upgrading or anything like that.
This is all pretty new to me coming from an ops management background.
And we're talking about a lot of management themes here.
I would have thought orchestration was really about provisioning their deployment time automation.
But it's really doing that end-to-end management.
Yeah, deployment is definitely a big part of it.
I think that's part of what brings people to container orchestration.
But container management has much more than that.
Because that application oriented API really enables the management to focus on
the health of your application, is it alive?
Is it doing the right things?
Taking steps to repair it.
And then also when you're doing a roll-out
understanding if that roll-out is successful or it needs to be rolled back.
So let me show you an example of what this might be like
by showing you a traditional application and then what it looks like containerized.
So here I have a simple nodeJS application.
I'm going to run it on my machine and there it's running on local host.
I'm going to go view that application;
I see there's this link here.
I wonder what happens when I click it.
And my system has died.
So let's go back and see what happens when I hit reload.
The server is no longer there
because that's crashed and I'm actually going to have to go restart it here.
Now let's take a look at how that's different
when you run it as a container.
All right so built the docker image
now I'm pushing it out to the registry so that I can run it in the cloud.
And now that it's pushed out
I'm going to run it in kubernetes.
So this one command line basically tells kubernetes "I want to run this container"
and there I've deployed my image.
Well the next thing I want to do is see and make sure that it's actually running.
So there it is, it's up and running.
And finally I want to expose it onto the internet.
So I'm going to expose it as a service with a load balancer in front of it out onto the internet.
Once that service is available,
I can actually go and find the IP address of that service.
I can cut and paste that IP address
and I can put it into the browser just like I did with my local address before.
And now once it's there, I can see the site again.
Now what happens when I go and click on that link that I wasn't supposed to go and click on?
You can see here it's died as before.
But when I go back and hit reload,
the application is still there up and running.
Kubernetes has noticed that the application has failed
and has actually restarted it.
Now if we go back to the command line and I list my containers again,
what you can see here is we're used to say "zero restarts",
it now says one.
Kubernetes actually understands that the application has failed,
restarted it and gives me that information to drive it into a monitoring system
or alerts that might fire to an operator to check it out.
So a great resilience to recover from failure
there in that case essentially.
So that was one container instance and it was able to spot that issue
and really recover from it easily.
But how was it able to do that?
Well again, because the container represents an application
and kubernetes is running that container.
It understands whether the container is running or not.
It's not like viewing an operating system where there may be a lot of different processes
and I don't know which ones are important which ones to pay attention to.
The container really saysthis is the one thing that I need to pay attention to
and if it dies please restart it.
But I want to show you a little bit something a little bit more complicated
which is, how would we actually do a rollout from one version to the next
of our software?
And do that in a safe and reliable way.
So look at this,
I have on one side I'm going to scale up my current deployment
so there are three replicas.
Of course if I want to do a reliable roll-out, I need more than one replica of my service.
Now that I have three replicas that you can see here,
two of our recently created
one was the one we created before.
Now I'm going to build version two of my image.
I've built the container.
Again I'm going to push it out to the container registry.
And once it's been pushed out to the cloud,
II can actually tell my deployment that I want to move to image number two.
In order to see this transformation from version one to version two in practice,
I'm going to go over here
and I'm going to run a little loop.
What this is going to do is load the version that's running of the software.
Now if there are any errors we'd expect to see
the errors in this in the output right there.
So we should expect to see a V2 pop up at some point, right?
You would expect to see a V2
but if you look at all the containers I just listed
version 2 is actually crashing.
I've actually rolled back version 2 back to version 1.
All the while
there have been no errors on the right hand side.
So none of our users have noticed anything.
So we were able to move successfully from version 1 to version 2
and now back to version one
without any sort of user impact.
That kind of reliability really means you can be much more agile in the way that you deploy software.
It's really cool at recover the user saw nothing.
It's really a lot faster than if it was a virtual machine.
You had to redeploy, reinitialize, maybe start up the app
app as the initial startup process.
All those things.
Now the the health check recovery here happened just in a couple of seconds.
And we saw how this is integrated as part of the normal rollout process,
but that's really great if you can detect something kind of immediately
out the gates.
But if you've got a memory leak or something that doesn't really
manifest itself for three or four or five minutes.
Will this help there as well?
Yeah you know anyone with significant Ops experience knows that
it's important to have the health checks
that validate and restart the application
and understand the application itself.
Each application is going to have different behaviors and different ways of failing .
So there's a lot of flexibility built into the Kubernetes health model.
And I want to show you that right now.
So here you can see that we are defining a liveness probe.
Now liveness indicates to kubernetes whether or not the application is running correctly.
We also have this notion of a readiness probe.
A readiness Pro basically indicates whether or not the application is ready to serve traffic.
So that's the difference between, should I be restarted?
And should I be added into a load balancer.
Now in order to customize that probe,
here I'm defining an HTTP request that I want to perform.
I'm also defining how often I want to do that health check.
In this case 10 seconds.
And finally, how many failures
count as an actual failure.
You may have transient problems if it fails five times in a row
that's a real failure.
Now here for the readiness probe
I'm defining the same kind of health check with a different path.
And the same timing and failure threshold parameter.
So a lot of flexibility really to have the
health check be tuned to the particular application that you're deploying.
All right so we saw V2
fail in terms of how we got deployed ou.
But, what if I had made all my bug fixes?
I've got my third container ready, my third image.
How would I go about deploying that and what does that look like?
Let's take a look again rolling from version 1
out to our newly fixed version 3 that we had from our developers.
So here we go, I'm going to build that third image.
And now again I'm going to push it out to the cloud.
So once the image is out there I'm going to be able to deploy it.
We'll wait for a second for that.
All right and now we're going to
do the same thing as we did before.
We're going to watch the servers to see what version they are serving.
Right now of course it's version 1.
Now I'm setting the image to be
version three.
So now I've done my image update.
I'm now in the process of doing a rolling upgrade.
It takes a bit of time for that third image to be downloaded from the image registry.
Now you've seen the first version 3 is coming through there.
And it's a mix of version 1 and version 3.
Each of the 3 replicas is upgraded.
And you can see that process happening with the containers that we're running here.
Eventually everything will say V3, right?
That's correct.
So eventually, here we are going to see that there's one last V1 there that I just saw.
And now it's version 3, version 3 all the way through.
So the deployment has actually finished successfully.
There have been no errors that any user saw.
And we've moved from version 1 of our software to version 3.
And the user is unaware.
Right it's important to point out the app just kept on running.
But a lot of what you're showing here is what a developer might do to orchestrate that service.
or that update to the service.
But what does IT actually need to do to get all of the underlying infrastructure running?
I mean really kubernetes or any other container Orchestrator
is just a program like anything else.
So setting up a cluster for containers is just like setting up a cluster for a database.
Or for any other kind of application.
That's if you are doing it on your own machines or your own private cloud.
But in Azure it's actually even easier.
So we can take a look at what it looks like to turn up a cluster in Azure.
Here we're actually just going to create a resource group.
That's sort of an organizational group that keeps things together on Azure.
We placed it into the east us region.
And then I'm going to create this container cluster
and tell it that I want the orchestrator type to be kubernetes.
It takes a few minutes for the cluster actually to spin up although the virtual machines
and configure everything properly.
But eventually once that's done, I have a working cluster.
I can get the credentials that allow me to access that cluster
securely just like a database password or anything like that.
Now that I have the credentials, I can just go
and via the kubernetes api I can see all of the machines
and the software that's been provisioned.
So it's really really easy.
Again, the cloud making it easier for people to provision and deploy their infrastructure.
Yeah, compared to what you have to do with physical infrastructure
on your own premises.
That's a huge time savings there.
Long term though, what happens if maybe you start hitting the app with load
and you've got to manage the underlying infrastructure?
So in that case actually kubernetes has an auto scaler.
So I will actually auto scale the deployment.
We'll go in and take a look at what we see.
Here's our deployment rounding it's got a single instance of the container.
The current CPU utilization is about 14%.
Now let's actually go and create a load test for that container.
We're actually going to use kubernetes and containers to run the load test also.
Again, the utilization isn't that high.
So we're going to actually need to turn up our load test.
So now we're going to scale that load test up to have 10 replicas.
Going back over to the the CPU monitoring for our service.
You can see now the CPU has spiked up.
It's over the 50 percent usage goal that we've set.
And so the auto scaler is actually kicked in
there are two replicas of our image now.
If we want to drive it even further,
we can actually go back to kubernetes.
And we can actually scale up our load test even more.
You can see here there are the two replicas.
And our 10 replicas of the load test.
Now we're scaling up to 20 replicas
And we can go back over to the CPU monitoring.
There's our little stair step while we were looking at it.
And eventually you can see the CPU utilization is up over our target of 50%.
So that when you wait for a minute,
the auto scaler will kick in detect that and create even another replica.
So within the bounds that I've set between one replicas and five replicas,
the auto scaler is going to tune you up and down in response to load that you see.
It's really great if I want to grow my apps out.
But what if I need to grow the underlying cluster nodes?
Right, so in Cloud and in Azure container service,
We can handle that as well.
So I'll show you manually scaling a cluster.
Here let's just take a look and
see nothing up my sleeves.
I have a cluster with three workers and one master.
Now I'm going to actually scale up so that I have for workers creating a new virtual machine.
It's the same one command just like creating the cluster to do the scaling.
Again it's creating a virtual machine so it takes a few minutes to happen in reality.
But now it's been scaled up.
And when I get the nodes
you can see here that I have a fourth worker node that can take new traffic
and new copies of my application.
It's been around for about 30 seconds.
In comparison to all the other machines in my cluster.
So since we're talking about all the physical nodes and underlying infrastructure
I bet a lot of people are wondering how I'd actually
have visibility into all of this stuff through my on-prem ops management tools.
Maybe OMS or other things.
Yeah so what's really great is you can plug in a wide variety of different monitoring solutions
into either Azure container service or kubernetes running on your own infrastructure.
Let's take a look at what OMS looks like in that world.
So first I'm going to actually just deploy the OMS agent.
Kubernetes actually has this concept of a Daemonset
that will deploy the agent by itself to each machine as it scales up and down.
So that's what I've done here.
It deploys it into a special system namespace.
So that my developers don't get confused and see the OMS agents.
Or maybe even terminate the OMS agents.
If we go into OMS itself,
You can see all of the nodes in my cluster.
All of the machines.
As well as all of the containers.
So I actually get this container oriented view into my monitoring.
There are those web containers as the most load test containers that we were playing around with earlier.
And as well as having CPU monitoring information,
I can also access logs and all of the other sorts of things that you'd expect out of OMS.
What are we doing for the windows world to
to bring that container and orchestration technology there?
Sure, so with Windows Server 2016
There was support for Windows Server containers.
And more recently with the announcement of general availability for kubernetes,
we brought Windows Server containers with kubernetes to preview.
So you can take a traditional .net app,
put it inside of the container,
get the fidelity of experience from laptop to cloud.
And then deploy it using kubernetes.
Using modern orchestration techniques.
And also deploy and manage hybrid environment.
So if part of your application is in Windows
and part of your application is in Linux,
you can use the same management, the same operation tools,
the same deployment tools to manage the complete application.
Some really powerful examples of seeing the tech in action
Kubernetes also manages and keeps everything up
and the uptime consistent throughout.
As a leader in the space,
how do you see the potential for this technology and what's next for you and the team?
I think the most exciting thing really is that the container Orchestrator is just the beginning.
I think it's kind of becoming the new virtual machine.
And we're going to see things that are built on top of that.
I'm excited about that ecosystem that's forming up
that allows people to move from ideas to reliable services in a short period of time.
Azure container service team is working with the upstream open source ecosystem
to expand and support for windows containers.
Both in Azure as well as on-prem.
And we're also integrating with Azure services.
So if you're running Azure container service
it really wants to look and feel exactly like any other Azure service
in terms of integration with key vault,
or monitoring we just saw.
I'm also excited to integrate with Visual Studio code
and all the great products that are coming out of the developers side of Microsoft.
And I think all of this is really going to enable container developers and operators
to deploy and manage their applications in a much more reliable way
with a dramatically less effort.
Really great seeing all this technology come together.
But where can people go to learn more about kubernetes and ACS, Linux and Windows containers?
Well you can learn more at the links below.
Or for training, we actually just recently announced the acquisition of Day S,
a startup that does a bunch of different kubernetes work
including training sessions and other social supports.
Really great seeing all these updates.
Thanks for coming on the show.
Keep watching Microsoft Mechanics for the latest tech updates across Microsoft.
We'll see you next time.
Microsoft Mechanics
www.microsoft.com/mechanics