Hi! Welcome to 'This is My Architecture'.
My name is Andrea and I'm here with Josh
from OnScale.
Hi, Josh, welcome to the show. Hi, Andrea.
So what do you guys do?
OnScale is a computer-aided engineering firm
and we provide access to high performance
computing resources in the cloud.
Excellent, so today we're going to talk about
Cloud HPC orchestration. Correct.
So what goes through that? What is that?
Sure, so, when an engineer wants to run a simulation
of a product before they manufacture a prototype,
they perform a series of regular simulations
of a type that have been going on for quite some time,
but these have been limited to firms that have
access to large on-prem data centers.
With AWS all firms can have access to that,
and that's what OnScale provides.
Wonderful. So it's running on AWS,
as you mentioned, and I see a number of the AWS services
here on the board, so let's look at one event.
I'm an engineer, I have a simulation
that I want to submit, what is the first thing that happens?
Well the first thing that happens is the customer will
send data to an S3 bucket.
They'll encrypt those data with a KMS key
and send it here, and then they will schedule
their job to run.
That scheduled job gets sent
off to EKS to spin up a right-sized
HPC node right here.
I see, so you have a Kubernetes cluster,
obviously for orchestration,
and the HPC nodes, walk me through that,
like how do you right-size, what instances you use. Sure.
So the customer will tell us how much
memory they require, or our estimators will,
and then, how many cores they need,
and how many different simulations they want to run,
they may run one job that's got 2000 cores
or 2000 jobs that have two cores each.
Something like that. I see.
So, depending on what they require,
we'll spin up the appropriate resources for them.
That takes, you know, between 3 to 5 minutes, right?
But since it can go from zero to hero,
instantly almost, our big concern is
how do we orchestrate all the services
so that they don't blow up. Exactly.
And for that, the real secret sauce of ours is,
we send all of the data that is streaming out
of our HPC nodes to a CloudWatch Log,
which can handle huge amounts of traffic. OK.
And it has alerts cooked into it that fires off data
that matches certain patterns to their appropriate
Lambdas for doing data processing.
OK. So, CloudWatch will start a
Lambda run to, say, alert to progress
or tell the customer that there's a preliminary result
available, so that they can look and see
if the simulation is right or not,
cancel or continue. I see.
And then
those Lambdas will fire off SNS messages
to cue yet more Lambdas... I see.
to do other things, like write data to DynamoDB,
send messages to Redis, for handling things like
WebSocket, to keep the customer informed
about what's happening. I see.
So, the HPC nodes themselves
connect to S3, but that's the only
service they connect to, other than Lambda,
to get a license and a key for S3.
After that, everything goes through CloudWatch
and then through Lambda, to handle the spikes
on traffic, which going from zero to 10,000 simulations
in a minute, causes. I see, so you use Lambda
in a very clever way to handle the spiky
workflows, also limitations.
So walk us through a scenario, I mean, I see
DynamoDB, right? Walk us through a situation
where DynamoDB will be triggered by Lambda. Sure.
So, the simulation starts, it's going to send a message
to CloudWatch that says "Simulation Running".
At that point, a Lambda will write database matches
with a UUID that describes the simulation
and a date, it says it started at this time.
And then, likewise, when it's over, it will say
"finished" and we'll write another row here,
and then, not surprisingly, we'll subtract the end time
from the start time and that's how long
the simulation ran. Does the engineer then
get any reporting, any insights into status?
How does that facilitate it? Absolutely, I mean, that's the beauty
of using CloudWatch and Lambda,
is that we can get out of these sequestered nodes,
which we keep well off the internet, right?
And the data is proxied through CloudWatch and Lambda,
so that the information that the customer desires,
which is, you know, is it going, are we progressing,
do we have... can I see your picture, of what's happening,
how's the signal, is it ringing down like I expect or not,
and then, finally when it's over, they'll find out
and they'll go to S3 and pull down the results.
OK, so the simulation completes,
then the output is back on S3? Correct, the HPC node will
deliver the results back into the same bucket
using the same key that the customer
already has. Oh, very interesting.
So it never goes to S3 until it's encrypted again
with the key that they downloaded to start
the whole process. I see, that's fantastic.
Walk us through the scale, I suspect
there's many simultaneous simulations happening. Sure, exactly.
How do you handle that? Yeah, the customer can
start one simulation, which is how
they're used to doing it, but in OnScale they can do
parametric computer-aided engineering,
which is, you know, similar to
parametric CAD, which they're all familiar with,
through innovations like SolidWorks,
but parametric computer-aided engineering,
they can do 10,000 simulations at the same time,
and pick out just the right combination
of lengths and sizes and materials
in order to get the perfect product.
Wonderful. This is a very interesting
architecture you've walked us through here, you know,
using Cloud HPC orchestration
and leveraging Lambdas to handle limitations
and other sort of spiky workloads
that is coming in from the engineers.
So thanks again for walking us through Thank you.
this architecture, and thank you
for watching 'This is My Architecture'.