>> [MUSIC] Guys. What are we doing today?
Today we need to actually set
up the build pipelines for the API server.
Because what I'd like to do is I'd like to get
the build pipeline in place, so that we can
then have the migration
in that build pipeline as part of the process.
Today, we're going to be working with Azure Pipelines.
We're going to click on "Create Container
Registry", and we're going to see what it does.
Again, there's a subscription,
we're going to need a resource group.
Quirk-dev-service is probably the resource group that we
would want this to be a part of it because it's with the services.
The registry name,
in this case,
I guess it would just be quirk-dev.
We'd be in West. Availability zones,
we're not going to worry about.
Azure Container Registry Service protects your data at rest.
Container Registry encrypts your images,
and other artifacts that are published.
Customer managed key. Yeah, that's fine.
We know that we can automate this.
Let's go ahead, and before we move anywhere forward,
go back to Azure DevOps,
because we have some automation to be done.
Now we can actually do
this automation in Quirk because this is a part of Quirk.
Remember, we have two projects.
We have our organizational project,
ALT F4 LLC and then we have our product projects.
The main thing about this that I want to point
out is if I go to Lucid App,
I can show you a quick little diagram of what all this means.
What this flowchart is going to show you guys is how I manage
infrastructure either security wise
as well as just where all the code goes.
At the end of the day, I really do want to be able to make it so
that I can automate as much as humanly possible.
But I also want to make it to where I can't just
let anybody just remove
repositories, or really important things need to
be abstracted away so that other things can use those things.
Here we go. We're going to say create
a registry per environment for services.
Priority is high, we need it.
Risk is low, I'm going to hit "Save and Close".
Let's create the container registry.
Again, with Azure DevOps,
I can also create branches.
I'm going to click "Create Branch" not on the Service,
but on Terraform because I want to work
with the Terraform repository.
We're going to call this feature/container -registry.
Create Container Registry, Main,
Terraform, we'll create branch.
Terraform, we'll do Git Fetch.
There's our branch checkout -b feature container_registry.
We're going to create a file called container_registry.tf.
Grab this, paste this in.
We want is we want
the local workspace, and that'll be the name of it,
the resource group in this one will be service,
service location with the standard.
I think we're good though. I use Terraform Cloud.
Once I push that up, and then I go to pull request,
I should see there's my feature Container Registry and
create pull request for it, sign it to myself.
It's already attached to the user story, which is what we want.
Hit "Create". Right now,
it should be triggering pipelines in Terraform Cloud.
Go to poll request, you can see both of them ran properly.
We're going to go ahead, and complete this.
We're going to squash this commit.
Now you can see here you see this in progress right here.
That means that if I go to workspaces, the planning,
I'm planning in depth now,
because basically going to rerun that plan one more time.
It's going to say, hey,
does this look good to you?
One to add one to change, which is fine.
What we want is the container registry created.
That's what we want, and we hit "Confirm and Apply".
I'm going to hit "Confirm" in a few moments we
should see the Container Registry pop up.
As a matter of fact, I should be able to actually get out of this.
There it is. It got applied, it was created.
If I go here, there it is.
It just took a moment. Cool. You can
see here Quirk Dev registry is now being created.
It's in the Quirk Dev service group,
which is in West US, and it's attached to
my ALT F4 LLC subscription.
Perfect. We needed this registry
so that we could set up the pipelines.
We're going to go to Create Pipeline,
going to create, click that API.
We're going to say an existing one,
and we're going to get it from the feature migrate,
and we're going to get it from the Azure Pipelines file
itself. We're going to hit "Continue".
Here it is right here. I just do "Run".
Here's my job, did it approve us? Let's edit the pipeline.
Edit. If we go back here,
leave, here, default, agents,
agent host, Azure pipelines,
agents, hosted agent,
agent host, default, agent.
Let's go into here. We're going to edit this,
we're going to change this, the fact of this.
Then the pool is going to be ALT F4 LLC.
Then I'm going to commit it feature migrate, fix,ci,
I use self hosted pool.
Get rid of this default.
Let's go to our pipelines. Look at that chat.
There we go. Permission granted. Click on the Job.
We got it. This is something else that we need.
We need tasks script.
This is a task called Docker installer.
Basically what we're going to do is we're going
to first say, Hey,
run the Docker installer because we want Docker to,
we need the CLI to install before we can actually do anything.
Then we want the Docker build, and push.
We're going to say Container Registry in
this case is going to go to quirk dev.
The Container Repository is going to be
API or say build, and push,
build contacts, add pipeline to metadata,
add base image metadata,
image metadata and add, and there you go.
Now we've got our second one.
Now, what we want to do from
here is we'll just leave this like this.
We're going to login to ACR and
this connection could be quirk dev.
The next thing we want to do is build and push.
We're going to call this API and the tags,
in this case will be, and I do it like this.
I can actually just do like boom,
and then F for now we'll just leave it like this. I'll delete it.
Feature migrate will default pass Docker login,
ACR login, quirk dev and tags will be build ID.
I'll have a Build ID, and it'll also get
tagged as the build branch source name,
CLI, Azure Docker CLI installer version.
Remember we're on this one,
ce, add here.
Now we've got the installer as well.
We'll get rid of this, and I'll write here, say task,
add, and copy, and then here do this and that.
Here's the pipeline. Look at that.
There's our new container,
click on this Docker installer, build, and push.
You'd see 718 megabytes.
It's not taking any CPU resources though.
It worked.
First try holy smokes.
I'm not going to lie. I was not expecting it to work that easily.
Now that that's done,
we were a little bit,
now we can genuinely just fly though.
We'll do env Docker file,
get rid of the the transpile here,
since these are all essentially going to be in the same space.
Here's our actual final base layer right here.
This is the base layer right here, chat.
We've got this, and then you can see this
is the final layer essentially,
node 14.17.3 transpile npmrc,
npm install, npm run script build,
and then rm-rf dot npmrc.
I need all of this as well,
and we'll do this, we'll paste this in, get rid of this.
This is going to be get lead all that.
You can just do this,
I will call it as service, and then the node env production,
workdir service it's going to do
the same thing here, package.json install.
Then we'll go here.
This isn't this, this is this,
so this should be just this now, there we go.
This should move this to here.
This should be, it should be that
or it should be this probably going to be that actually this.
The next thing we want to do is, is this node env.
We want this node env to be
our workspace environment like dev, whatever.
We don't want this to be something else.
What we're going to do now is we're actually going to go
back to pipelines at its variables.
Here we go, new variable.
This variable is going to be
workspace and we're going to call it dev,
this is not a secret,
and we're not going to let it be over edible.
Again, this is going to match technically the TF workspace.
This will match the Terraform workspace.
Terraform workspace equals quarks dev.
Cool. Now I should be able to do this.
What we're going to do is here,
we're going to call it ARG Terraform Workspace.
This ENV will be Terraform Workspace. See that?
I'm going to pass down from the CLI Terraform Workspace
and I'm going to use it as
a node environment so that when this runs,
if I have logs,
I could basically see quirk-dev in the logs or for the node ENV.
I am going to make this dev just
because I think it makes more sense for that.
I'm going to save it and there you go.
Now I have Terraform Workspace equals quirk-dev.
Now my node environment is different from production which is
fine but I'm still running
a production install here and
that's what I wanted to essentially achieve.
I just want to make it so that if I needed to do
that I could, node module.
You don't want node modules to be added ever.
Azure Pipelines, we don't need that.
Azure pipelines.yml.git, node modules.
Don't need git, don't need node modules, we need source.
We don't need Azure Pipelines.
We obviously don't need the Docker file itself.
Again, some of this is not
necessary but I'm just being explicit here.
Git ignore, that's fine.
NPMRC, that needs to be copied in which means I can't ignore it.
This one I can get rid of NVMRC, ormconfig.
I think this is pretty good chat.
We don't need git,
we don't need node modules,
we don't need Azure Pipelines,
we don't need a Dockerfile.
I think this is pretty much everything.
All I want to be able to do in our chat
is after I do an NPM install,
you copy and then
source destination just do the good old fashion, dot dot.
This is basically going to make it so that I have my installs
cached and it is going to
recopy my package.json on my shrink wrap.
It is going to redo that but it's fine.
It's just going to overwrite it. Basically what
should happen now is
it should grab node 14.17.3 alpine,
it should then do the install that we needed to,
set up the NPMRC,
do the install, run the actual script of build.
Now, actually you know what I could do for this,
instead of doing this we could call this source.
I could actually call this directory source and
rename it so that now instead of this being this
it could just be service/source/index.js
and we don't have really any kind of dist idea.
It's just, it exists for a moment and then
we rename it to the source essentially.
Quirk essentially existed as a big mono repo,
for a long time it was a mono repo.
One of the things we had to do is we had to
take each third service or what we're going to
have to do is take each service and
literally pull it out of the mono repo.
You'll see Api, Cron,
discordbot, these are all services.
But if you look here we have
the shared service and this isn't actually a service,
it is the only unique folder in
this whole thing because it is a shared library.
This shared library was something that is being used
across all of these services.
But because we're not using a mono repo,
one of the things we had to do was we had to
turn this into an Azure artifact.
If I actually take a step back and I go to Quirk,
then I go to Artifacts, Azure has this thing called feed.
Essentially what a feed allows you to do is set
up your own repository
or registry or whatever you need to
cache the packages that your app uses.
For example you'll see here that
my feed also has upstreams of NuGet,
Pypi, Maven Central.
Now, I don't need any of these,
so I can actually just go ahead and delete all three
of these and just keep NPM because that's all I need.
Now this will always grab from NPM,
cash it, save it,
and then I can run off of this.
Not only can I do that but I can
also publish my own packages to it.
You could see here I have Quirk service and this Quirk service is
literally the service package that was once called shared.
Now we're at the point where we've
got our databases, we've got our caches,
and since we've got everything we need and all we need
to do now is start deploying Containers.
Right now I'm literally in
the process of wrapping up the Docker file here.
The thing on the left, this file on the left is
the Azure Pipeline definition.
But essentially what this does is this tells Azure, hey,
on my feature migrate branch,
I want to use my pool which is my default.
I actually even showed you guys how to set up
a local Azure DevOps runner.
What you'll see here is that I actually have
a couple of different containers
running on my computer right now;
I have a YAML language server because I'm working with YAML,
I have a Docker language server
because I'm working with Docker files,
but then you'll also see a Docker agent here.
This Docker agent is literally,
if I go here, go to organization settings,
go to agent pools,
default, and then I go to agents altf4llc,
it's this service right here,
altf4llc, it's this literal agents.
I have my own local running agent
that I'm using with Azure DevOps.
For this branch, what happens is,
if we go back to our Docker agent and we look at the compose,
you're going to see that I have a volume here
called var run Docker sock,
which then mounts var run Docker sock.
You could see here that I'm actually mounting
my Docker socket on my current host into the Container.
The reason for that is because it
doesn't actually install Docker,
it just installs the CLI and then
runs that version against my Damon.
What's cool about Docker is Docker has all of its versions of
its API available to it in the endpoint itself.
For example, this 1709 version is going to call
a specific API version and it's okay
because all of those versions are available in the API itself.
I think we're good. Let's go ahead and see.
We'll say feature CI,
add Docker build tasks and push.
Now, once I push,
what should happen is I should be immediately going to
see top and see a new container
appear here in the next few moment.
If I go to Quirk and then I go to Pipelines,
we can see API is running.
Click on this, click on this.
There's our new container.
There you go, it's running other tasks as it's going through.
Now it's actually doing NPMCI to install all the packages.
There we go. All that in, we are good to go.
Six hundred and fifty-four packages installed and then we
copy everything and now we do NPM run script build.
You can see here, delete and then NPX TSC pretty.
Look at that, the build worked and log out.
This should finish log out,
post-job, finalize job, we are almost there.
We did it, there we go, job success,
one minute and 28 seconds. We just did it.
We just had our first Docker Container build.
That was it. It's right here. It worked.
It 100 percent worked.
We were able to install the CLI login to Docker,
authenticate our NPM,
actually build it, and install all the packages that
we needed it to and then log out which you can see here,
we were able to log out and then finish the job. Yeah, we did it.