[MUSIC].
>> Yeah, we're back. We got a lot of work to do.
Here's the fun part,
which is we need to start moving everything here.
I'm going to say, git clone services
and we'll say services migration.
Now I have a Services Migration folder.
What I need to literally do right now at least is figure out
how I can move the code in the API folder,
which is right here, over here basically.
We've got API of course, okay, cool.
Here's basically all the source code.
Let's move API, everything here.
We might have a little bit more automation than we expected.
What I'm thinking and we're going to have to
actually evenly use LucidCharts for this.
I guess the first thing we can start with is we have
a process and this process
right here, I know what it's going to be.
This is going to be a container.
In this Container it needs to run the API docker file.
But for now we'll just call it a Container and that
Container is also going to need to talk to a database.
We know that we have at least these two requirements here.
But before the Container can connect to Postgres,
it's going to need a way to access credentials.
We're going to need to put credential somewhere because honestly,
we're not going to be getting them from anywhere else but Azure.
What we're going to need to do is we're going
to need a container that goes out to Key Vault,
gets the specific credentials or whatever it needs to run
its migration and then actually access that migration.
On top of that,
we're also going to need a pipeline.
That pipeline is going to need to fire that trigger or
Container which then goes out to Key Vault which
then goes to Postgres go and run the migration.
This is from Azure DevOps whereas this is
going to Azure Cloud essentially.
What we need to do is we need a way in
our pipeline to trigger something that runs in
Azure Cloud and then that thing that runs in
Azure Cloud needs to get
credentials from Key Vault
so that it can figure out how to connect to the database.
Then later on the same pipeline will
then actually trigger a deployment
to another Container and
this pipeline after this migration is done,
this Container here will be run and then
when it runs it'll go out and it'll connect to
the Postgres database as well.
Let's look at Key Vault first,
let's go to Azure and let's go to Services and we'll go
to not Identity but Security Key Vaults, there we go.
Key Vaults are essentially ways to
storing secret data in the Cloud.
We have a resource group,
like always, a Key Vault name,
in this case it would be Quirk-dev, East, West,
whenever we do that and then we'd
have soft delete which is enabled.
There's a little bit more now that we also need to think
about and I've been putting this off for as long as I could.
Basically what I'm talking about
is we need to set up our networking.
Azure again has this thing where
basically the best approach to giving
services access internally is to
basically be giving them private endpoints.
What we're missing here is we're
missing this as well which is all of these.
Well, technically all of these needs to
exist on its own private subnet and then we
need to give this a little bit more room because this is
technically the services subnet.
Then we'll grab this because remember we
have our database subnet as well.
That means that we're going to have to create a services subnet.
We're going to go and say migrate API repository and
we're going to put this into play.
We'll create a branch feature,
create services subnet main,
create services subnet, awesome.
Now I should be able to go in a subnet,
there we go, there's my subnet.
This is going to be our services or service,
we'll call this service subnet to feature subnet,
add service subnet basically.
Now we have a subnet,
we're going to go to main git branch D,
here's where is the subnet.
We're planning, now we can actually go look at
the Container and let's just create
a Container real quick. Let's just see what it'll say.
It'll say resource group,
in this case I'm just going to leave that like that.
We want this to be private.
Virtual network Quirk dev,
subnet quirk dev service,
DLT is not available,
delegate it to other services.
That means that only Container Instances can join this subnet.
I like that, that's very nice.
The next thing I need to do is I need to create Key Vault.
Put this into active, that's myself.
Create a branch, create a Key Vault terraform,
and Key Vault,
all right, create a branch.
Awesome. Now I'm going to do
a [inaudible] git checkout SB feature create a Key Vault,
we'll create a Key Vault and then we'll go to
a resource group really quickly and we've
got a new resource group that we're going to
create and this one is going to be called service.
This is going to be the service resource group.
Cool. There is our resource group.
The Key Vault here will be local.workspace.
Now, this Key Vault is going to be for this whole workspace.
I'm going to have Key Vaults for multiple workspaces
and essentially that's how we're going to
keep these all isolated for now.
This will be called self again because it's for this environment.
After that we're going to say example,
this is going to be service.
I added my resource group and I added my Key Vault.
What I'm going to do now is I'm going to click on
Private Endpoint and you'll see there's no public access.
Right here, Private Endpoint here resource group,
dev redis name dev redis,
US West is where we want it.
Next, resource and the target sub-resource is redis cache.
If I go next, virtual network quirk dev,
and then there we go, private link, rediscache.windows.net.
Cool. It is in fact,
doing pretty much everything we need to.
We can set up a private endpoint now
or redis as well as our Key Vault.
We go board, create private link,
private DNS integration,
private DNS zone, and here it is.
Here's a privatelink.rediscache.windows.net and
this needs to go into the Redis resource group.
Let's go to quirk dev service.
Here we go, there's our new service
and there's our Key Vault and this is it.
Basically in the essentials it just shows
all the metadata of the actual service and stuff.
Let's go back here, access policy.
We've got another access policy here,
this is going to be network ACLs.
We're going to go here, bypass
equals Azure services, that's fine.
I'm okay with letting Azure services bypass default action.
We're going to say Allow and in
the IP rules or the subnet ID and we want
the Virtual subnet ID because we just want to make it so that
everything on the service subnet can access it as well.
Then this should be like that because it is
expecting an array and then what we
need is we need the virtual network,
not the network itself,
sorry, but the actual subnet.
We need the Azurerm.net.service.id.
Now what we're saying is this, hey,
we want to allow the owner to be
able to do everything he needs to but then we
also want the network to be able
to essentially connect only via this.
Now we'll go back here, boards service subnet,
create a branch, create services subnet,
that seems to be delayed but that's fine.
Feature, add service, ACL.
Add service subnet, create
branch and I'm going to do git checkout,
main pull batch and then git branch D
and in this case it's going to be create vault, create services.
That was it. Okay, cool. Now what I can do is I can do
git dash apply and nvim and then open this up.
Now you can see there's my network ACL right there.
What's about to happen?
We can see here that the Key Vault itself got
tags added to it because we
didn't add it with Terraform which is fine.
Same thing with the service group.
Now we're going to look at
the Key Vault and we can see here that the network ACLs,
virtual network sub IDs.
Boom. That's perfect.
Great. Let's approve this and complete.
Complete it and squash it.
We added that ACL.
Now what we want to do is we want to go
back to Workspaces and then
a couple of seconds we should see quirk-dev has a new run,
and sure enough, there it is.
Features and stories aren't on the board,
so let's go to backlog and then here's a bug.
How do I see bug? Maybe it's under Work Items.
It's under Work Items. I think I know how you would have to do it.
I think you'd have to create a user story and
then attach the bug to that user story.
For example, I think what I would
have to do is that I would have to
re-open the Create Services subnet user story like this.
I'd go in here and then I would say I would add Related Work,
Existing Item, Child,
and in this case it would be a bug. I hit "Okay".
Now that's attached to that because I had to
reopen this and now I need to fix the service endpoints.
Yeah, that's how that works. Now it's on the board.
Everything has to be underneath
either a user story. That makes sense.
Now, what I'm going to do is I'm going to say fix subnet,
add keyvault to service endpoints.
That's in its own branch. I'm going to push that up.
We're going to go here, create new pull requests,
going to delete that, do that.
Create. In the next few seconds I should
see those dry runs essentially. What do we need to do now?
We need to create the Redis private link,
but we also need to create the Key Vault private link.
Let's do the Redis one first since that one was there first.
Let's create a branch now and
we'll say feature/redis_private_link,
create Redis private link, create a branch.
Now what I want to do is main,
pull, fetch, awesome, branch d,
git branch, awesome,
git checkout -b, feature/Redis_private_link.
Now I can say endpoint and then private link service.
We can get rid of the private link service, we don't need that.
We're going to say rm private_link_service.
What this should do is,
this should essentially create a private endpoint for our service,
which is Redis cache under the
private-link-Redis-cache-windows.net using
the DNS zone privatelink.redis.cache.windows.net.
Cool. We're going to say feature Redis add private link.
Cool. In progress.
Now let's go to our workspaces.
Bam. There you go, quirk-dev,
you see it's planning already. Let's see here.
It looks like we were able to create the DNS zone,
but we weren't able to create the private endpoint,
so we had something happen.
I think I'm going to have to turn that delegation off.
Yeah, I think we might have to turn this off.
We can keep the service endpoints,
but the delegation I think is the problem.
We'll remove the delegation first.
We'll see if that does anything. Invalid subnet policies.
We're going to save and close,
we're going to click on this,
create branch, bug/invalid_subnet_policies,
create branch, git checkout main.
Now, what I'm going to do is that I'm just going to
simplify this subnet as much as I possibly can.
I'm going to push this up and we're going to
see if this does anything for us.
We got the network ACL stuff figured out.
Here we go. We've got an update to the service.
We're updating this now,
we're getting rid of the delegation.
Let's go back here. If I refresh,
cool the delegation's gone.
Look at that chat, it worked.
We got it. We got our private endpoint.
What did we do to essentially get ourselves a private endpoint?
Here's our Redis private endpoint.
We want to do the same thing with Key Vault.
We're going to say Key Vault private endpoint.
Now this is saying that this is in the Redis resource group,
which is not really what we want.
We want it to be in the resource group with wherever Key Vault is.
The Key Vault resource group is in the service resource group.
What we're going to do for now is put it in the same one.
We'll say service, and it's going to be in
the service subnet because that's where we want this to be.
The name for this is going to be key-vault like that,
and the private DNS zone will be key_vault.
We need to create a private DNS zone
so let's do that really quickly.
DNS zone, there's Redis,
do the same thing except for this is going to be key_vault.
We'll go back here, vault.
Instead of this being Azure Redis Cache,
what we need it to be is Azure Key Vault and then self ID,
because we want this to access to
the Key Vault for this environment, for these services.
We're going to hit "Configuration".
If we do this, we can see that this is actually
the configuration name is private link vault core.
Now, all we should need left is to do
the actual DNS zone like URL.
It's going to be privatelink.vaultcore.azure.net.
That should be good. Now I should be able to go to
Key Vault and then go to
networking and then go to private endpoints.
Bam, there's our private endpoint as well
as our firewall that only
allows it through a private endpoint
and selected networks. Sure enough, there you go.
That means that our Key Vault should be entirely locked down.
This part and this part is done.
That means now we actually need to
take a look at how we're going to run
this container in the Cloud as a migration.
We're going to click on this,
then we're going to close this.
The git checkout main, pull,
that git branch, and
then we're going to delete that one that existed.
Now let's go into our API repository again,
which is where we were working before and
we realized we needed some stuff.
If we go in here and I go to package.json,
it might just make sense to put this whole,
if I do git checkout, main,
I do pull,
and then fetch,
and then git branch -D. This is going to be Key Vault,
Key Value, whatever, policies.
We go up API. Then again,
if we go back to ORM Config,
if I can put this whole thing into Key Vault,
and I think I will. I forgot to do something.
Variables, I forgot to do this,
portal, principle,
the principle object ID for browsing in Azure portal.
New changes were pushed. View latest. There we go.
Yes, I had to add the variable. That was my bad.
Again, when we had to look and see what we needed,
this is essentially what we're building right now.
We're building a way for a pipeline in Azure DevOps to be able to
trigger a one-off API container
which will have the entry point of migrate.
That will then also make
sure that it'll have grabbed the data from
Key Vault so then it can connect to
the database and migrate the data.
Then after that, the pipeline will actually run
the service into deployment so that it can run when it needs to.
But that's basically what we're building now.
We got the Services subnet set up.
We got the database subnet set up.
We also got all of the security around everything set up.
Next stream, will basically be us working on this.
Love you guys. I appreciate you so much.
Thank you for being here.
Seriously, couldn't do without you guys. Have a good one.