Hello and welcome to another episode of This is My Architecture.
I'm Erin from AWS and we're joined by Alan from AQR.
-Hi Erin. -Welcome.
-Thank you. Thanks for having me. -Thanks for coming.
So, at AQR, what do you guys do?
We're a global investment management firm.
We have just under $200 billion assets under management.
-Oh, that's it? And so what do you do at AQR?
I run the cloud engineering groups. We do a lot of cloud enablement,
we're working on enabling the AWS platform for AQR.
Okay, and this actually looks like a representation of one of those enablements.
-That's right. -Can you tell us a little bit about this?
Sure, yeah, this is one of the compute environments that we manage,
one of the platforms that we build on AWS. It's a solution we call AWS PAAS.
-Platform As A Service. -Oh, okay.
It's essentially manage docker containers.
The docker containers build web applications that have load balancers
and extra cell certificates and things like this for the many different engineering divisions that we have at AQR.
So for your development team, is this kind of a platform in which they can deploy
whatever applications they're building. -That's right.
So, say I am building a new application and I want to be onboarded.
What happens? What do I have to do?
Like a lot of enterprises, we have ServiceNow.
So the starting point for all of this is you go to ServiceNow, you fill out a form,
click submit and some automation happens in the back end and you end up getting a Git repository
with a pull request in it and there's some things you need in there to host your AWS PAAS app.
All right, so after all that magic happens, I write my first PAAS at the app
and I want to see how it works. So I do a commit to my repository
and then what?
And then your Git repo is tied to a CI/CD pipeline and so the CI/CD pipeline
wakes up and it will do some building and pushing of things to ECR
and then it will talk to service catalog and launch your application onto the ECS platform.
Okay, so ECR hosts my docker container.
Service Catalog, what does Service Catalog do in this situation?
So for Service Catalog, it's really kind of like a front end for CloudFormation.
We actually have to touch a lot of resources when we launch a PAAS application
and so we use CloudFormation to do that.
And one of the things that we wanted to do is to tighten up the security on our CI/CD system
so we used Service Catalog to do that.
Our CI/CD server really just has the ability to launch PAAS apps.
It doesn't need to ACM or IAM roles or ECS or anything like that.
Service catalog sort of abstracts that away from the CI/CD server.
So is that why you chose Service Catalog? For security?
That's right. We wanted to make sure we didn't have this attack surface with all of our Jenkins systems
that are laying around doing things.
Oh yeah, that makes a lot of sense.
So, Service Catalog launches CloudFormation template,
that then deploys my app within this VPC?
That's right. So, Service Catalog itself
launches a CloudFormation stack. The CloudFormation stack will do things like
create the ACM certificate, it'll talk to the ECS service and launch the service,
it'll create a task definition, it'll create the listener rules on the ALB.
It does all of those things in a structured, organized way.
So, you've really enabled me as a developer to get up and running faster with my development
but how do you make sure that I'm also- making sure that I'm doing things in a secure way?
Like encryption? How do you enable me to do that?
Yeah, one of the things we relly like about ECS is the fact that we can run muliple containers in our test definition.
So the test definition itself has container definition.
It's actually a list of containers.
We have the ability to launch more than one container.
So the customer brings a docker file that we build and push into ECR,
and so we launch their container, but if they want authentication,
we can run a sidecar right next to their application.
So instead of them having to worry about getting SSL certificates up
and doing all the authentication for users and stuff like that, we can launch a little sidecar that does it all for them.
So within this sidecar, it uses that certificate created by ACM?
-That's right. -As a developer, do I need to worry about that?
Nope. Your application, you just bring it up on Port 80 or 8080
and you don't worry about SSL at all and then what we do is we have a little reverse proxy that runs on there.
The ALB brings it down to 443. Everything over the wire is always encrypted.
And then we just translate it to 8080 just before we put it into your container.
That makes it really easy to use encryption.
There are probably a lot of services that I need to speak to
that are maybe outside the VPC on premisis or something
that my application maybe needs to get data from.
How do you enable me as a developer to get access to that data securely
without having to handle user names and passwords and things like that?
Yeah, that's a great quesiton. That's a very common scenario
for a lot of development applications that we have and so what classically we would do is have to
put a secret in a secret store and the developer would have to worry about how to get it out of there
and thy'll have to get the thing.
So what we can do is run a sidecar. With another sidecar with the application
we call it the curb-row sidecar, and it's job is to go and fetch
the user information for your specific application, authenticate it to the KDC,
and then through a shared volume in docker it can inject the ticket cache
right into your application container. So your app is just authenticated.
Just connect to your database.
So I kind of get authentication for free and I don't have to worry about it as a developer?
-Absolutly. -So, within this platform, how many appiclations do you have running today?
We have about a little over 300 applications running across all the environments today.
And what's really great about it is we have a lot of Greenfield apps,
people are writing REST APIs, all different kinds of things in here
and we're growing it somewhere between 5 and 10% a month.
Oh wow, that's pretty great growth.
-So you're going to be scaling quite a bit. -Yeah, it's great.
Well thanks for sharing this great architectiure with us today.
And thank you for watching This is My Architecture.