Welcome to this episode
of Bone Hunt
Discovering Dinosaurs with AI.
I'm Bob Chesebrough,
I'm a solution
engineer and technical evangelist
supporting oneAPI
AI here at Intel Corporation.
Today's episode of Bone
Hunt brings us to the OpenVINO Toolkit.
I worked with my colleagues Raymond Lo
and Zhou Wu to optimize the code we use
for map development
using OpenVINO Toolkit.
The classifier we optimized is great
because it works in the real world.
Not only that, but as engineers,
we want to come up with solutions
that work and run well,
not just on our own machines,
but on other people's machines as well.
We wanted to make sure that
someone beyond us
could utilize this work,
and that's what we call
a deployment challenge.
We wanted to deploy an AI solution
for more people to use.
When you deploy a solution,
you will often encounter problems
when deploying at scale.
For example, how many people can use it?
Or more importantly,
related to performance,
how can you ensure more people can use it
with a good experience?
When Raymond and
I looked at our solution,
we thought it might take about an hour
to crunch through a map generation
and the resolution of our initial map
was not the best.
We imagined a larger, more detailed
map would take longer to compute.
That's where OpenVINO came in
and provided
the speed ups necessary
to do more calculations
to generate a higher resolution map.
OpenVINO is an optimization tool
that Intel created
for Intel hardware,
particularly x86 architecture
at one point,
and now it also supports
different architectures
like GPUs, VPUs, and FPGA.
OpenVINO is like a funnel of
AI frameworks
like Caffe ONNX, TensorFlow and PyTorch.
With these frameworks,
we provide models
as input to OpenVINO
to optimize across
different Intel hardware.
When we say optimization
that may be very broad to you,
but we explicitly think about how,
for example,
we can reduce the model's size,
do quantization,
or think about different
architectures running coherent
work together.
We also have some interesting math
behind the scenes to trim the graph
and make it even more compact
so that you're not wasting resources.
At this point in our AI lifecycle,
I was thinking about this project
as an optimization problem.
Raymond and I spent about two weeks
getting our solution out
and accelerating it
to run on CPU and GPU simultaneously.
One of the things that is cool about this
is that we leverage
the Intel oneDNN library
via higher level frameworks
without troubling ourselves
with lower level details.
The way oneDNN works
is that when instruction
sets like XMX, AMX, and VNNI
for Xeon are available,
the library automatically takes advantage
of that underlying hardware instruction
so that we can do computations
effectively and efficiently.
For example,
VNNI is designed so that you can do
dot products fairly quickly,
which are common operations in AI.
Instead of doing three
different instructions
for the dot product,
we have a single instruction
greatly improving performance.
Remember,
OpenVINO is an open source,
easy to use tool
to get the best performance
from your hardware.
It is flexible
because it supports CPU, GPU and VPU.
When Raymond and I worked together,
we used my laptop to crunch
all the numbers for this use case.
The CPU was great,
but we wanted to go beyond that
and the best hardware
I had
was the graphics card next to the CPU.
With OpenVINO,
we could have simply used the CPU,
but combining CPU and GPU,
we saw a significant acceleration.
All of this
while keeping the power
envelope on just a laptop.
So I didn't need to buy a server
to crunch the numbers
and we easily beat our previous baseline.
For the future
what we see in GPUs is exciting,
with the Intel ARC,
Intel Datacenter GPU Flex Series
and the Intel Server GPUs now available.
OpenVINO has become
a very interesting tool
that has been built
over the last five years.
It supports multi-hardware platforms
as well as software
frameworks, and also leverages
some of the great work from oneDNN.
It is a valuable open source project
for everyone in the AI community.
That does it for this
overview of OpenVINO.
Remember that you can find links
to resources, tools and downloads
and where you can learn
more about everything
in this series in the description.
While you're here.
Like and share
this video
and subscribe
to the Intel Software YouTube channel.
Join me in the next episode
where we wrap up
Bone Hunt
Discovering Dinosaurs Using AI Adventure.
Let the
adventure begins!