>> One of the big challenges with
high performance computing is working with
terabytes to petabytes size datasets.
Avere vFXT brings a large datasets closer to
Azure Compute to limit latency and increase performance.
Scott [inaudible] is here to show us how it works today on Azure Friday.
[MUSIC]
>> Today on Azure Friday,
I have Scott [inaudible] who is with
our Azure Storage Team to talk to us about
the Avere vFXT which we just announced as generally available.
So, Scott can you talk to me about what
Avere will do to help with my compute workloads?
>> Sure Cameron. Thank you by the way for having me here today.
So, it all starts with
the ability to run what are called High Performance Compute jobs.
So, the idea behind high performance compute
is there's lots of different versions of HPC
like for example I can
run Hollywood scene rendering
and I'll talk about that in a second,
I can look at genomic analysis so I have got
these genes I've got to figure out how to
line them up and figure out where the variants are.
I mean if you really want to go crazy,
it's as I want to simulate a thermonuclear explosion.
So, there's labs that do that.
>> Okay.
>> So, these types of jobs are very particular
because they run hundreds or thousands of
virtual machines or machines
in datacenters and these machines will come up and they want
to read the same data over and over
again as they're processing what's
happening or running a simulation and they may read out
some results and then they need to read those results.
So, you've got this very intense large-scale compute activity
combined with a lot of
potential storage that needs to be there very quickly.
So, for example I was mentioning media and entertainment.
So, in Hollywood everything's based on,
have you ever seen just
say any random movie that has the animation,
maybe like Secret Life of Pets or something like that.
Well, everything that happens is a scene and
these scenes are consisting of thousands of frames of data.
So, when a dog moves from here to
here that's a frame and that's a frame and
they have to put these frames together and
piece them together and render
all of the assets that are in these frames so
for example a leaf might be an asset,
my shoe might be an asset so all of
these assets come together and create these frames of data
that then have to be built or rendered by like
tens of thousands of cores of compute.
They need to do it in a fairly quick fashion
so that they can build these movies.
One interesting thing is,
visual effects shops work on schedules and so
they work with somebody and they
say we're going to get it done in this much time,
it's going to cost this much money, and so on and so forth.
Well, things happen right in life happens so the director comes
back and says I don't want a dog there I want a cat okay?
>> Right.
>> Suddenly they have to rework it,
it bites into the schedule and there's a lot of pressure
on the shop to get things done even faster.
So, the ability to run as fast as they
possibly can each time they have to render
something is critical to their business.
Then the other aspect of it-
>> To do that they need more compute at times or they need to
have some agile access to
lots of compute to dial up or down those timelines.
>> That's right and this has been a historical problem.
So, in the days and it still happens I think to some extent,
a render shop they might lease
racks of servers and a truck will backup to
the datacenter and I'll put these racks
in and they hook them all up and they run these things
and that helps them get through
that burst problem that they had at that time.
But even that's very labor intensive.
You have to figure out how to do it,
the costs might be good but you have to make sure it's available.
So, if you're renting from X,
hopefully they have what you need.
That's been a real problem and so how to take care of
these peaks but also how to maybe do more work.
So, as a visual effect shop maybe I can do more scenes for
more movies with more assets
if I have more flexibility with how I do compute.
So, they're very interested in running compute jobs in Cloud.
But the type of compute jobs that you
have to run in Cloud whether we're talking
about Hollywood or we're talking about
health care like for genetics, I want to run in the Cloud.
It's all basically the same problem.
I want to run thousands of cores of compute and
I need to get very fast access to certain data.
The key is, that data may be here in my datacenter,
it may be in the Cloud,
it maybe in both places,
how do I do that? How do I-
>> That data is huge.
>> It can be quite large. So, a reference genome could be
hundreds of gigabytes in size or
could be very tiny in some cases
depending on what you're looking at.
The human genome is much larger than
a worm genome in some cases and so that
data has to be presented to all
of these compute nodes as fast as possible.
So, where Avere comes into the picture is we offer you
the ability to do what's called Bursting or Hybrid Cloud,
you can run the Avere system in Azure compute,
buy your new compute farm that you're building virtually,
and it can then present the data
to that compute farm in a fast way.
The way that it does that is it doesn't get every piece
of data that's on prem it only gets
what the clients have been requesting.
The chances are in your typical HPC job,
that the data itself is going to be requested
again and again and again and that's the key is it's there,
it's already in the cache layer,
in the Avere layer and it's fast.
>> Okay. Can you talk to me a little bit
more about that whether your data could be stored.
You mentioned you've got data on prem but I
see you also have support for Azure Blob.
>> Yes, absolutely. So, in the classic case,
we'll call it the classic case, right?
A lot of customers they have embraced the Cloud.
I think we're all done with to Cloud or not to Cloud thing.
Everybody is pretty much on board.
It's a question of how do they move into
the cloud in a way that everybody's happy
with and that they can do
things right away and it doesn't impede on their current business.
So, for example customer might have a lot of
network attached storage sitting in their datacenter.
It could be like petabytes and we call that
a data gravity issue where it's like
I've got petabytes of data down there,
how do I move that data up into the Cloud?
Azure certainly has some solutions
with data box and
some other things that can help you move that data,
but it's not always a technical problem.
A lot of times it's human factors issue
where it's like there may be regulatory issues,
there may just be politics concerned about
moving all the data up and over committing,
or it's just a time problem where it's like, yeah,
we know we need to move it,
but I don't need to move it right away.
I really would like to use that compute
Fargo and that's where you can say,
"Okay fine, I don't need to move all of my data.
I don't have to make that commitment today,
I can run Avere in the Cloud and get you that burst use case."
>> Right.
>> On the other hand, there are customers that they want to
move data into Blob because Blob is a great product,
it's a great service, it's low cost,
and it's highly available and all those nice things.
But one of the things that we've seen over time is they're very
concerned about moving their data up and
having to change their code.
So, like your typical HPC job is running and it's looking for
data at a path in a file well,
Blob is object-based and so I would,
as a data analyst if I wanted to move my job into the Cloud,
I'd have to re-architect my job to run it into the Cloud.
So, what Avere can do then is we can
basically leverage Blob as basically a NAS.
So, you can move a petabyte of
data or two petabytes of data into the Cloud,
you can move it behind the Avere,
and Avere provides like an NFS NAS for your HPC job in Azure.
>> So, it becomes [inaudible] transparent for removing
existing workloads up to
the Cloud without having to change your code?
>> That's right.
>> That's beautiful.
>> Yeah.
>> Okay.
As you move up to the Cloud or even anywhere machines go down.
Can you talk to me about what happens?
Are we protected as their high availability?
>> Machines never go down in the Cloud, Lara, come on.
I'm just kidding. This is another historic problem.
This is not just a cloud problem.
So, you have a visual effect shop,
they have a 10,000 server farm and they're ready to run a job, go.
Let's just say that you don't have an Avere in the mix
and all of a sudden the network link goes down to the storage,
then your job has just failed,
you've lost valuable time.
So, historically, Avere has always
been a highly available cluster.
So, you run multiple Avere appliances
in the classic on-premise case,
in the Cloud, it's the same thing.
So, you run a minimum of three
of your nodes sitting in Azure Compute,
and basically you'll use something like Round-robin DNS to
distribute the client connections to the vFXT cluster.
Then, if a node happens to fail on our side,
it's not a problem because what we'll do is we'll
basically move the IP address that's
bound to the NIC on the virtual machine that we're running
on to another one of the cluster members.
So, any clients that we're connecting to that IP address,
they just keep connecting and going.
>> Great. Well, do you want to show us how this works?
>> Sure. I'd be happy to.
So, this is like a cooking show where it's like, "Okay.
It's already done, those onions out of the oven."
But I did this before we ran
just because it takes more time than probably we have to run it.
But this is our dashboard and this is our historic dashboard,
you get this on the virtual FXT.
We also have physical FXT and it's the same thing.
So, it's the same experience both ways.
What you're actually seeing here is
a histogram of read throughput.
What I mean by that is I
basically and I can go over here very quickly and show you,
I have run 12 DS machines
with a high performance file testing mechanism,
so they all connected to
the Avere cluster and they're all asking for a bunch of data.
So, we were able to pump out around peaking up at
around 250 gigabytes per second throughput on my simple test.
If I back this up a little bit and we'll go back an hour,
you can see the whole test.
So, this was basically,
the test wrote some data into the cluster,
and by the way what you're seeing here is
an a Avere vFXT cluster sitting in Azure Compute on three nodes.
I'll go up here and show you that.
Here they are, "Avere-cluster one, two and three."
These VM test instances are all mounting that Avere cluster.
Basically, the test kicked off and it wrote
about a gig worth of data into the system,
and then all 12 of those nodes just jumped
on to the Avere and started
asking for a bunch of data and reading it.
So, you can see that you can get fairly reasonable throughput.
This is just an example test.
I mean, the types of scenarios that we've seen customers run are,
this is a three node Avere cluster,
we've seen customers run 12 nodes of Avere,
getting around at 1.7 gigabytes per second per node of
throughput and being able to scale to
tens of thousands of NFS connections across this.
So, you have to think of this as
a file performance tier
that is just invisible and when it's working-
>> It's a cache.
>> Exactly. It's a cache.
>> It's a cache.
>> When it's there and it's doing its job,
you're just happy and your jobs are running quickly.
>> Okay.
>> So, I can take you through a couple of
other data points about this.
So, I'm going to pivot over the slides
very quickly just to walk you through a couple of things,
and then I'll go over to the UI to show you what they are.
So, a couple of details and this is as of GA.
So, we are only clustered and typically you'll run three.
You can run as many as 24 odd number but that's
our tested production number instead of
a theoretical maximum number that nobody ever tested.
So, that's the max but our average
is about six to seven or eight nodes.
But you can run quite a lot of load
against a three node cluster or a six node cluster.
We have a customer that was doing a movie render,
they were using 6,500 cores on average all day,
every day for a couple of weeks and happily doing so.
As we said, the high availability
is important for a couple of reasons.
One reason is because,
as I said, if a node fails,
that can impact an HPC job and
nobody wants that impacts so you want the reads to continue,
so the clients are able to continue.
On the other hand, if you writing out data
from the HPC cluster through the Avere system,
you want to be very cautious about protecting the rights.
So, every write that's written through to the Avere cluster is
immediately cloned to another member of the cluster.
So, there's a election process and everybody is a partner.
Then, that means that in the event that the primary node
that that client was connected to happens to fail, the write,
we'll call it a dirty write,
is preserved and protected,
and then there's a new election and then
that's clone to another HA partner.
>> Classic architecture.
>> Some just basic details.
So, how do clients connect?
We have this thing called a Vserver and this is
the export point for the file system from a client perspective.
So, clients are connecting through this Vserver mechanism,
and if I go over to the UI,
I can show you down here.
We have a Vserver configured,
and you'll see there's a range of IP addresses on the Vserver.
That IP address is essentially what a client
would use to connect to the system.
>> Okay.
>> Then ultimately, I think the clients should not,
yeah, they're not connected right now.
But I can always get that to happen again.
We'll give that a minute to fire back up.
So, Vserver basically is
the place where clients will send their NFS requests
and the Vserver then
services that and sends it back to our caching layer,
and that's how clients interface with the system.
Then, there's this notion of a core filer.
So, what's nice about
our cache is not only does it lower latency and increased
performance but it provides a little bit of flexibility
in terms of accessing the data itself.
So, if I go over to the actual system,
I'm going to scroll up here for a second
and go into what's called The Core Filer.
What I can basically do is I can attach
in multiple sources of data.
So, if I have a net up sitting in my datacenter,
and it has an export, I can put that into as a core filer.
If I want to put a Blob container in for like scratch,
space, or I want to put reference genomes in that,
I can connect that as a core filer,
all into the same system and you can have anywhere reasonably from
10 to 20 core filers
connecting sources of data from various places.
This is what a core filer configuration looks like in our system.
This one here, MS Azure,
is the Blob Core Filer.
You see it has this notion of a cache policy,
that's a technical thing for a class.
We welcome anybody to go and look at
our documentation or our videos
and you can see how to configure that.
The key thing is you can see right here,
it has a one-hour right back delay.
What that does is if you're running
an HPC job and you have to
write out what are called intermediate results,
a lot of times the pipeline has to wait until
that's acknowledged before it can continue on.
This write back time allows the cluster to
merely act to clients and then move on,
and then within an hour, it'll flush that data to the backend.
>> Okay.
>> Then, just last couple things.
You can see I'm talking to a Blob container.
I had to find a unique name.
So, I always use my last name, it's very unique.
Then, basically uses an MSI Cloud credentials
to connect to that container and put that into the system.
You'll see I also have another Core Filer configured.
So, that was the Blob Core Filer and I just threw together
an Ubuntu NFS server sitting in
Azure Compute and enabled an export on that,
and you can see there is that particular system.
If I were to go to a client and I break my demo,
that thing is for edits.
This is the part they edit. In fact,
this is a client and I'll just give you
a quick show of what a customer would look at.
So, this "Mount Command" shows you
the various file systems that this particular client is aware of,
and you'll see I have three mount configurations.
Basically, these three you see 100012,
14, 13, those represent that Vserver that I was talking about.
>> Okay.
>> Those are different mount points that
this particular client sees and if I
change directory to one of those,
you'll see that there's the two core filers I was referring to.
So, the MS Azure is the one that's running the test that you
see in the background and that's under here.
You can see test run next 12.
>> That's where all the individual files that were brought in.
>> So, everything is getting pumped.
>> Okay.
>> Then on the other hand,
if I go to "nfsdir,"
this is the actual NFS file that I mentioned.
I have exported to that.
These are actually genome reference files,
and these were all sitting on that other NFS server.
So, this is a particular case where I have
a Blob container with lots of
files and I have an NFS server with other files,
and so you can see like an example for life sciences would be,
I put all my reference data in the Blob container
and have it sit up there at a nice low cost.
Then if I have to,
I can reach back and get the customer data for the processing of
that analysis but then the reference genomes are
already up in the Cloud and I can
have a nice low latency experience.
>> Okay. Well, where can we go and learn more about how to use
this and integrate this into our HPC workloads?
>> Great question. So, where we are as
of our GA is in the handy marketplace,
the Azure marketplace. Here it is.
You'll notice that there's
some information about plans and pricing.
Here's a very good data point for you.
So, we do run
elsewhere but primarily we're focused a lot on Azure these days.
But the nice thing is from a cost perspective,
the only thing you have to pay for with regards to
an Avere system are
the actual underlying virtual machines and disks.
There's no additional license cost to run the vFXT in the Cloud.
So, if you're interested in trying it against an HPC workload
and you're able to run whatever three
E32S v3s or something like that,
you don't have to worry about an additional license.
>> Fantastic. Thank you very much for joining us today and
sharing all the great news about Avere vFXT,
and we look forward to using this on our HPC workloads.
>> Laura, thank you very much.
>> All right. Today we've been learning all
about Avere vFXT on Azure Friday.
[MUSIC]