(light music)
- I'm Werner Vogels,
the Chief Technology Officer of Amazon.com
I'm excited to welcome Bill Vass,
VP of Engineering at AWS,
who's joining me to discuss how data movement capabilities
have evolved over the years.
Bill, thanks for joining us.
If I think of it there's multiple sides to storage engines,
on one hand, that's what you store.
The other hand is how do you get it into the storage engine?
- Yeah, so that's a great question.
And thanks for having me.
We always would prefer a customer to move data
over a network if they have it available to them.
And today you can have like DX connections
of hundreds of gigs.
And so, we have a lot of products
to make it easy to get the data in.
Of course, you can load it directly into Amazon S3.
We have transfer, which for FTP and things like that
and we have Kinesis, which lets you string things in
over the network for even things like cameras, for example.
You can click this in, and so there's lots of cameras.
This also works with Lidar and things like that.
And so, those can be streamed in.
And so, the network's a great way to get it in,
but a lot of our customers have way more data
than they can send over the network.
And so, they came to us and said,
"I wanna shut my data center down,"
or, "I've got this huge amount of data;
I'm collecting this scientific data out in the field."
or, "I'm doing movie shoots out in the field."
and things like that with 4K and now even 8K cameras.
And so, I need to move hundreds, potentially, of petabytes
and it would take years to do it over the network.
And now, we have a whole [array] of products.
And so, we have like the smallest one, which I have here,
which is [the AWS] Snowcone.
But there's a lot on this behind the scenes.
So, the E Ink label is there,
so you can never put the wrong label on it
because we wouldn't want you to put
your important data to the wrong place.
So, it knows where it's going.
It labels itself.
You might notice there's lots of anti-tamper on this,
but there's also three layers
of cryptography with three different keys,
one of which isn't on the device,
that encrypts the data three times
with three different keys.
And then we have it's larger brother,
which is the [AWS] Snowball products, the Snowball Edge.
And we were moving a lot of data centers
and a number of customers said,
"Hey, you know, why can't you just send me a truck full
of these at one time?"
And then that's where [AWS] Snowmobile came from.
So, we've kind of done the evolution,
but the interesting thing is some people might think
about this as, well, gee, that E Ink label is a gimmick.
But, it's actually a security feature.
If you remember, you and I are both old enough to remember
like loading things in with tapes and floppies, right?
And you'd do a backup with 40 floppies
and then you'd get like halfway through,
and like, 'was this number 27 or number 28?'
So the Snowballs, you don't need to worry about that.
They just know it and they just do it
and you don't need to worry about encryption.
They just know it and they just do it.
And so, this is secure enough for
the intelligence agencies to use.
It's secure enough for banks
and healthcare and all those things.
It meets HIPAA requirements and level six plus.
So, that's kind of the physical side of things,
but the network side of things,
we've invested a lot to make it easy to stream your data in
and also accelerate your data over the network.
- So, while in the original Snowballs
was purely a storage device. - Yep.
- These new devices, or the new generations,
have compute on them as well.
- Yeah. - It went from
doing Lambda functions,
over to actually be able
to run EC2 Instances on them.
So tell me why that evolution happened.
- Yeah, so we had a lot of customers,
a lot of data is collected in very rugged places.
You know, a movie shoot is happening now
in the middle of nowhere, we have drone operations,
satellite ground station processing, oil rigs,
scientific vessels, those kinds of things.
And they're rugged locations.
They can operate at, you know, minus 53 degrees
and positive 140 degrees.
So they can handle 200 G impact.
You can even airdrop them with a parachute,
they're raided for all that stuff.
- So when we introduced the Snowmobile
at really 2-3 years ago now at re:Invent
with the big truck running in.
Tell me a bit,
what kind of customers would use the snowmobile
- the really massive storage?
- Yeah. So, a great example is
that the biggest use case is satellite customers right now.
So, they have these huge archives
of imaging that are just, you know, hundreds of petabytes
or thousands of exabytes kinds of data.
And they're shutting their data centers down,
they're shutting down their ground stations,
and they want to move that data
into long-term archive in glacier and S3.
- So yeah, so data movement is, is really important there.
We've also got a number of things, including
more moving things to the edge, and giving customers
the ability to not necessarily have to go
to the centralized place, to upload data
but to make use of S3 acceleration, for example.
- Right
- So tell us a bit about the thinking there.
- Yeah. So, we kind of look at what we call the three laws.
So the law of physics and that's what a lot of what
we're talking about here, you know
you just can't go faster than the speed of light.
So if you need low latency, local processing
things like Wavelength and Outpost and Snowball
and IoT and RoboMaker and Kinesis video
all those things happen for you.
If you have the law of economics
you might be able to move some
of this data over a network as I described.
But if you have way too much data
to move over the network
it just gets too expensive
and it takes too much time.
And so, you're kind of dealing with
two of the laws there.
And then at sometimes there's the law of the land,
which is residency.
So customers who want to load, like that's why we have S3
for Outpost so they can keep their data right there
on the Outpost if they need to.
- Well, thanks Bill.
Thanks for talking to us here.
The amazing thing is that sometimes people ask me
"So are you ready to retire?"
(both laugh) And I thought, "What? No."
- What? Yeah.
So I'm really looking forward
to the next 15 years of engineering with you.
- Yeah, me too.
- Thank you
- It's great talking to you.
(light music)