(upbeat music)
- Hello, my name is Cormac Garvey.
I work as a HPC architect in the Azure Global HPC team.
Today, I'd like to spend a few minutes walking you
through one of the ways
to do a deployment of HPC end-to-end environment in Azure.
So what does a typical HPC environment consist of?
Well, first it consists of a network
which defines the address space we're working in.
It also has some storage component,
and, in the case of HPC,
typically that's a high-performance parallel file system,
which can handle the large IO requirements.
It also has a head node,
which allows you to log in to the HPC cluster
and schedule jobs to some type of job scheduler.
Also, in the cloud,
ideally, you would like to have some auto-scaling capability
that computation resources, compute, and GP
would scale up and scale down,
depending on the load on the queue.
In today's deployment,
we would like to use two tools to do this deployment.
The first is Azure HPC,
which is an open-source flexible
HPC deployment framework.
The second tool that we're going to utilize is CycleCloud,
which is an Azure HPC product.
And it's ideal for lift-and-shift scenarios
in which there is some sort of HPC scheduler component.
Also, CycleCloud has advanced auto-scaling
and cost-management capabilities.
So what are we deploying today?
First of all, we're deploying a VNet,
and the VNet will describe the address space.
The address space will be carved up into different subnets.
We're also going to deploy a jumpbox.
And the jumpbox is where the Azure HPC install scripts
are deployed from, are run from.
Also the jumpbox will act as an NFS server.
For the parallel file system,
we're going to deploy BeeGFS.
We're also going to deploy CycleCloud server
and leverage its excellent PBS integration
and it's auto-scaling capability.
Also note that in the case of the cycle server,
all of its computational resources will be deployed
into the compute subnet.
So when using Azure HPC,
there are a couple of prerequisites.
The first is,
you need to use a reasonably new version
of the Azure CLI.
Also, you'll need to have Git and jq installed
because we need those tools to clone the repository,
and jq is used to parse JSON files.
Now, you can do the deployment from a Linux workstation,
or, if you prefer, you can use the Azure Cloud Shell.
Now, in the case if you use the Azure Cloud Shell,
the prerequisites which I mentioned
are already automatically installed for you.
And also when you're doing any sort of deployment,
it's also important
to make sure you have sufficient quota.
If you don't have sufficient quota,
you can also
easily request a quota increase.
So, the first step is,
depending on whatever,
where you work from,
whether it's a Linux workstation or the Azure Cloud Shell,
is to
git clone the Azure HPC repository.
Once the repository is cloned,
the next step is to source the Azure HPC installed
at SH script.
This will set up the Azure HPC environment for you.
All that's required now at this stage
is to create a JSON config file,
which will describe the HPC deployment we want to do.
In today's deployment,
we're gonna work through one of the Azure HPC examples.
So in the examples,
there is a directory called cc_beegfs.
It's always best practice when using Azure HPC
to not deploy within the repository.
So, it's important to create a working directory
outside the repository.
So, first step, we create a working directory
outside the repository,
we'll copy two files from the cc_beegfs directory
to our working directory-
that is the init.sh script,
and the variables.json file.
All that's required now
is to go and edit some of the global variables
that are defined in the variables.json file.
So we need to set appropriate values
for the resource group,
location, a UUID,
and what scheduler we want to use.
In this case, we're going to use PBS.
So once that's done,
all that's required now is to actually execute the init.sh
in our current working directory.
This will effectively loop over all of the building blocks,
the HPC building blocks we want to use in our deployment.
It will reference the global variables that were set
in the variables.json file.
So, once we execute the init.sh,
it will create a number of files
in the current working directory.
The two files that we're primarily interested in
for the next steps
are the two configuration files generated,
prerequisites.json and config.json.
In this particular example,
there is an additional prerequisite.
We do need to have created
an Azure key vault,
and the cycle server administrative password
needs to be contained in the key vault.
This is best practice in Azure.
So, fortunately, the prerequisites.json file
does this for us.
So, we use the Azure HPC build command
passing in the --no-vnet option,
because at this stage,
we're not creating a VNet,
we're just creating the key vault.
And then we specify the configuration file we want to use
with the -c option.
In this case, prerequisites.json.
So once the prerequisites are all set up,
all we need to do at this stage
is to run the Azure HPC build command.
Now, when you run the Azure HPC build command,
it expects to see a config.json file
in the current working directory, which we have.
So the Azure HPC build
will read the config file,
the config.json file that we created.
It will create an ARM template.
The Azure Resource Manager will then read that template
and deploy the resources in parallel.
Once all the resources have been deployed,
the Azure HPC install scripts will execute
on the virtual machine and virtual machine scale sets
with the appropriate tagging.
When this is completed,
our deployment is done.
All that is required,
all that's needed at this stage
is you bring up a web browser,
go to the cycle server URL,
give it the administrative user and password to log in.
At this stage,
you should see the CycleCloud cluster
that we created.
In this case, it was named pbscycle.
All that we need to do at this stage
is actually start
the CycleCloud cluster.
So we hit the start button,
which effectively starts the CycleCloud Master.
Once the Master is up and running,
we're now ready to go ahead and log in
to the CycleCloud Master
and submit jobs to the PBS job scheduler.
The CycleCloud auto-scaling capability
will then spin up the appropriate computational resources
to run the jobs on the job queue.
When there are no more jobs in the job queue,
CycleCloud will automatically
delete computational resources
or scale down the resources to reduce costs.
Now, also note that the CycleCloud auto-scaling capability
can be customized
to whatever core requirements that you have.
When you're finished working with the HPC environment
that you just deployed,
there is a convenience command
to delete all of the resources that you deployed.
So, just execute the azhpc-destroy
in your current working directory.
That will go ahead and delete all of the resources
in your resource group.
Now, today we walked through a specific deployment
of a HPC end-to-end environment in Azure.
In Azure HPC under the examples directory,
there are many building blocks to choose from.
We can combine those building blocks
to create more complex HPC deployments very easily.
Also, note that in the Azure HPC apps directory,
there are over 30 commercial and open source
installation and run scripts
that you can choose from.
I really appreciate you spending some time with me today.
I hope this session was useful on your journey in Azure.
Thank you so much for your time.
(upbeat music)