(upbeat music)
>> We're back.
You're watching theCUBE's coverage of Red Hat Summit 2022
here in the Seaport in Boston.
I'm Dave Vellante with my co-host, Paul Gillin.
Justin Boitano is here.
He's the Vice President
of Enterprise and Edge Computing at NVIDIA.
Maybe you've heard of him.
And Tushar Katarki who's the Director of Product Management
at Red Hat.
Gentlemen, welcome to theCUBE, good to see you.
>> Thank you.
>> Great to be here, thanks
>> Justin, you are a keynote this morning.
You got interviewed and shared your thoughts on AI.
You encourage people to got to think bigger on AI.
I know it's kind of self-serving but why?
Why should we think bigger?
>> When you think of AI, I mean, it's a monumental change.
It's going to affect every industry.
And so when we think of AI, you step back,
you're challenging companies to build intelligence
and AI factories,
and factories that can produce intelligence.
And so it, you know, forces you to rethink
how you build data centers, how you build applications.
It's a very data centric process
where you're bringing in, you know,
an exponential amount of data.
You have to label that data.
You got to train a model.
You got to test the model to make sure that it's accurate
and delivers business value.
Then you push it into production,
it's going to generate more data,
and you kind of work through that cycle
over and over and over.
So, you know, just as Red Hat talks about, you know,
CI/CD of applications,
we're talking about CI/CD of the AI model itself, right?
So it becomes a continuous improvement of AI models
in production which is a big, big business transformation.
>> Yeah, Chris Wright was talking
about basically take your typical application development,
you know, pipeline, and life cycle,
and apply that type of thinking to AI.
I was saying those two worlds have to come together.
Actually, you know, the application stack and the data stack
including AI need to come together.
What's the role of Red Hat?
What's your sort of posture on AI?
Where do you fit with OpenShift?
>> Yeah, so we're really excited about AI.
I mean, a lot of our customers obviously are looking
to take that data and make meaning out of it using AI
is definitely a big important tool.
And OpenShift, and our approach to Open Hybrid Cloud
really forms a successful platform
to base all your AI journey on with the partners
such as NVIDIA whom we are working very closely with.
And so the idea really is as Justin was saying,
you know, the end to end,
when you think about life of a model, you've got data,
you mine that data, you create models,
you deploy it into production.
That whole thing, what we call CI/CD, as he was saying
DevOps, DevSecOps, and the hybrid cloud
that Red Hat has been talking about,
although with OpenShift as the center
forms a good basis for that.
>> So somebody said the other day, I'm going to ask you,
is INVIDIA a hardware company or a software company?
>> We are a company that people know for our hardware
but, you know, predominantly now we're a software company.
And that's what we were on stage talking about.
I mean, ultimately, a lot of these customers know
that they've got to embark on this journey to apply AI,
to transform their business with it.
It's such a big competitive advantage
going into, you know, the next decade.
And so the faster they get ahead of it,
the more they're going to win, right?
But some of them, they're just not really sure
how to get going.
And so a lot of this is we want to lower the barrier to entry.
We built this program, we call it Launchpad
to basically make it
so they get instant access to the servers,
the AI servers, with OpenShift,
with the MLOps tooling, with example applications.
And then we walk them through examples
like how do you build a chatbot?
How do you build a vision system for quality control?
How do you build a price recommendation model?
And they can do hands on labs and walk out of, you know,
Launchpad with all the software they need,
I'll say the blueprint for building their application.
They've got a way to have the software
and containers supported in production,
and they know the blueprint for the infrastructure
and operating that a scale with OpenShift.
So more and more, you know, to come back to your question
is we're focused on the software layers
and making that easy
to help, you know, either enterprises build their apps
or work with our ecosystem and developers
to buy, you know, solutions off the shelf.
>> On the harbor side though,
I mean, clearly NVIDIA has prospered on the backs of GPUs,
as the engines of AI development.
Is that how it's going to be for the foreseeable future?
Will GPUs continue to be core
to building and training AI models
or do you see something more specific to AI workloads?
>> Yeah, I mean, it's a good question.
So I think for the next decade, well, plus,
I mean not forever, we're going to always monetize hardware.
It's a big, you know, market opportunity.
I mean, Jensen talks about a $100 billion,
you know, market opportunity for NVIDIA just on hardware.
It's probably another a $100 billion opportunity
on the software.
So the reality is we're getting going on the software side,
so it's still kind of early days,
but that's, you know, a big area of growth for us
in the future and we're making big investments in that area.
On the hardware side, and in the data center, you know,
the reality is since Moore's law has ended,
acceleration is really the thing
that's going to advance all data centers.
So I think in the future, every server will have GPUs,
every server will have DPUs,
and we can talk a bit about what DPUs are.
And so there's really kind of three primary processors
that have to be there to form the foundation
of the enterprise data center in the future.
>> Did you bring up an interesting point about DPUs and MPUs,
and sort of the variations of GPUs that are coming about?
Do you see those different PU types
continuing to proliferate?
>> Oh, absolutely.
I mean, we've done a bunch of work with Red Hat,
and we've got a, I'll say a beta of OpenShift 4.10
that now supports DPUs as the,
I'll call it the control plane
like software defined networking offload in the data center.
So it takes all the software defined networking off of CPUs.
When everybody talks about, I'll call it software defined,
you know, networking and core data centers,
you can think of that as just a CPU tax up to this point.
So what's nice is it's all moving over to DPU
to, you know, offload and isolate it from the x86 cores.
It increases security of data center.
It improves the throughput of your data center.
And so, yeah, DPUs, we see everybody copying that model.
And, you know to give credit where credit is due,
I think, you know, companies like AWS, you know,
they bought Annapurna, they turned it into Nitro
which is the foundation of their data centers.
And everybody wants the, I'll call it democratized version
of that to run their data centers.
And so every financial institution
and bank around the world sees the value of this technology,
but running in their data centers.
>> Hey, everybody needs a Nitro.
I've written about it.
It's Annapurna acquisition, 350 million.
I mean, peanuts in the grand scheme of things.
It's interesting, you said Moore's law is dead.
You know, we have that conversation all the time.
Pat Gelsinger promised that Moore's law is alive and well.
But the interesting thing is when you look at the numbers,
that's, you know, Moore's law, we all know it,
doubling of the transistor densities every 18 to 24 months.
Let's say that, that promise that he made is true.
What I think the industry maybe doesn't appreciate,
I'm sure you do, being in NVIDIA,
when you combine what you were just saying,
the CPU, the GPU, Paul, the MPU, accelerators,
all the XPUs, you're talking about,
I mean, look at Apple with the M1,
I mean 6X in 15 months versus doubling every 18 to 24.
The A15 is probably averaging over the last five years,
a 110% performance improvement each year
versus the historical Moore's law which is 40%.
It's probably down to the low 30s now.
So it's a completely different world
that we're entering now.
And the new applications are going to be developed
on these capabilities.
It's just not your general purpose market anymore.
From an application development standpoint,
what does that mean to the world?
>> Yeah, I mean, yeah, it is a great point.
I mean, from an application, I mean first of all,
I mean, just talk about AI.
I mean, they are all very compute intensive.
They're data intensive.
And I mean to move data focus so much in to compute
and crunch those numbers.
I mean, I'd say you need all the PUs that you mentioned
in the world.
And also there are other concerns
that will augment that, right?
Like we want to, you know, security is so important
so we want to secure everything.
Cryptography is going to take off to new levels, you know,
that we are talking about, for example,
in the case of DPUs, we are talking about, you know,
can that be used to offload your encryption
and firewalling, and so on and so forth.
So I think there are a lot of opportunities
even from an application point of view
to take of this capacity.
So I'd say we've never run out
of the need for PUs if you will.
>> So is OpenShift the layer
that's going to simplify all that for the developer.
>> That's right.
You know, so one of the things that we worked with NVIDIA,
and in fact was we developed this concept
of an operator for GPUs,
but you can use that pattern for any of the PUs.
And so the idea really is that, how do you, yeah--
(all giggle)
>> That's a new term.
>> Yeah, it's a new term.
(all giggle)
>> XPUs.
>> XPUs, yeah.
And so that pattern becomes very easy for GPUs
or any other such accelerators to be easily added
as a capacity.
And for the Kubernetes scaler to understand
that there is that capacity
so that an application
which says that I want to run on a GPU
then it becomes very easy for it to run on that GPU.
And so that's the abstraction to your point
about how we are making that happen.
>> And to add to this.
So the operator model, it's this, you know,
open source model that does the orchestration.
So Kubernetes will say, oh, there's a GPU in that node,
let me run the operator,
and it installs our entire run time.
And our run time now, you know,
it's got a MIG configuration utility.
It's got the driver.
It's got, you know, telemetry and metering of the actual GPU
and the workload,
you know, along with a bunch of other components, right?
They get installed in that Kubernetes cluster.
So instead of somebody trying to chase down
all the little pieces and parts,
it just happens automatically in seconds.
We've extended the operator model
to DPUs and networking cards as well,
and we have all of those in the operator hub.
So for somebody that's running OpenShift
in their data centers, it's really simple to, you know,
turn on Node Feature Discovery, you point to the operators.
And when you see new accelerated nodes,
the entire run time is automatically installed for you.
So it really makes, you know, GPUs and our networking,
our advanced networking capabilities
really first class citizens in the data center.
>> So you can kind of connect the dots and see how NVIDIA
and the Red Hat partnership
are sort of aiming at the enterprise.
I mean, NVIDIA, obviously, they got the AI piece.
I always thought maybe 25% of the compute cycles
in the data center were wasted doing storage offloads
or networking offload, security.
I think Jensen says it's 30%,
probably a better number than I have.
But so now you're seeing a lot of new innovation
in new hardware devices that are attacking that
with alternative processors.
And then my question is, what about the edge?
Is that a blue field out at the edge?
What does that look like to NVIDIA
and where does OpenShift play?
>> Yeah, so when we talk about the edge,
we always going to start talking
about like which edge are we talking about
'cause it's everything outside the core data center.
I mean, some of the trends that we see
with regard to the edges
is, you know, when you get to the far edge,
it's single nodes.
You don't have the guards, gates, and guns protection
of the data center.
So you start having to worry about physical security
of the hardware.
So you can imagine there's really stringent requirements
on protecting the intellectual property
of the AI model itself.
You spend millions of dollars to build it.
If I push that out to an edge data center,
how do I make sure that that's fully protected?
And that's the area that we just announced
a new processor that we call Hopper H100.
It supports confidential computing
so that you can basically ensure that model
is always encrypted in system memory
across the bus, of the PCI bus to the GPU,
and it's run in a confidential way on the GPU.
So you're protecting your data
which is your model plus the data flowing through it,
you know, in transit, wallet stored, and then in use.
So that really adds to that edge security model.
>> I wanted to ask you about the cloud,
correct me if I'm wrong.
But it seems to me that that AI workloads have been slower
than most to make their way to the cloud.
There are a lot of concerns about data transfer capacity
and even cost.
Do you see that?
First of all, do you agree with that?
And secondly, is that going to change in the short-term?
>> Yeah, so I think there's different classes of problems.
So we'll take, there's some companies
where their data's generated in the cloud
and we see a ton of, I'll say, adoption of AI
by cloud service providers, right?
Recommendation engines, translation engines,
conversational AI services,
that all the clouds are building.
That's all, you know, our processors.
There's also problems that enterprises have
where now I'm trying to take
some of these automation capabilities
but I'm trying to create an intelligent factory
where I want to, you know, merge kind of AI
with the physical world.
And that really has to run at the edge
'cause there's too much data being generated by cameras
to bring that all the way back into the cloud.
So, you know, I think we're seeing mass adoption
in the cloud today.
I think at the edge a lot of businesses
are trying to understand how do I deploy that reliably
and securely and scale it.
So I do think, you know, there's different problems
that are going to run in different places,
and ultimately we want to help anybody apply AI
where the business is generating the data.
>> So obviously very memory intensive applications as well.
We've seen you, NVIDIA, architecturally kind of move away
from the traditional, you know, x86 approach,
take better advantage of memories
where obviously you have relationships with Arm.
So you've got a very diverse set of capabilities.
And then all these other components that come into use,
to just be a kind of x86 centric world.
And now it's all these other supporting components
to support these new applications and it's...
How should we think about the future?
>> Yeah, I mean, it's very exciting for sure, right?
Like, you know, the future,
the data is out there at the edge,
the data can be in the data center.
And so we are trying to weave a hybrid cloud footprint
that spans that.
I mean, you heard Paul come here, talk about it.
But, you know, we've talked about it for some time now.
And so the paradigm really that is,
that be it an application, and when I say application,
it could be even an AI model as a service.
It can think about that as an application.
How does an application span that entire paradigm
from the core to the edge and beyond
is where the future is.
And, of course, there's a lot of technical challenges,
you know, for us to get there.
And I think partnerships like this are going to help us
and our customers to get there.
So the world is very exciting.
You know, I'm very bullish on how this will play out, right?
>> Justin, we'll give you the last word, closing thoughts.
>> Well, you know, I think a lot of this
is like I said, it's how do we reduce the complexity
for enterprises to get started
which is why Launchpad is so fundamental.
It gives, you know, access to the entire stack instantly
with like hands on curated labs
for both IT and data scientists.
So they can, again, walk out with the blueprints
they need to set this up
and, you know, start on a successful AI journey.
>> Just a position,
is Launchpad more of a Sandbox, more of a school,
or more of an actual development environment.
>> Yeah, think of it as it's, again, it's really for trial,
like hands on labs to help people learn
all the foundational skills they need
to like build an AI practice and get it into production.
And again, it's like, you don't need to go champion
to your executive team
that you need access to expensive infrastructure
and, you know, and bring in Red Hat to set up OpenShift.
Everything's there for you so you can instantly get started.
Do kind of a pilot project and then use that to explain
to your executive team everything
that you need to then go do to get this into production
and drive business value for the company.
>> All right, great stuff, guys.
Thanks so much for coming to theCUBE.
>> Yeah, thanks. >> Thank you for having us.
>> All right, thank you for watching.
Keep it right there, Dave Vellante and Paul Gillin.
We'll be back right after this short break
at the Red Hat Summit 2022.
(upbeat music)