Hello. My name is Alberto Villareal.
I'm a software applications engineer with Intel.
In this short video, I want to give you an introduction
to a new feature on both Intel Xeon Scalable processor, Max Series
and 4th Gen Intel Xeon Scalable processor
that is designed to accelerate deep learning, training and inference.
Intel Advanced Matrix Extensions or Intel
AMX is one of the new features in Xeon Scalable processors,
which expands the use of CPU's for artificial intelligence workloads.
In previous generations of Intel Xeon scalable processors,
Intel pioneered built in artificial intelligence acceleration
with a vector neural network instuctions, or VNNI,
as part of Intel AVX-512.
Now Intel adds hardware in the form of dedicated TILES
and a set of matrix multiply instructions, or TMUL,
to operate efficient matrix multiplication operations on those tiles.
Specifically, Intel AMX supports
INT8 and BF16 data types,
while AVX512 instructions continue to support
FP32 and FP64 data types.
Classical machine learning workloads, which are used
by most enterprises, benefit from existing AVX-512 instructions.
And Intel AMX speeds up matrix multiplication,
accelerating deep learning workloads, including recommender systems,
natural language processing, and image detection, among others.
By speeding up deep
learning training and inferencing, you can do much more with a CPU.
In this way, Intel Xeon Scalable processors will help you
maximize your Xeon investment.
Now let's talk a bit more about Intel Advanced Matrix Extensions.
Intel Xeon Max is a dedicated hardware block found on each core,
and it helps accelerate deep learning workloads.
It consists of two main pieces:
The first pieces are tiles, which are 2D register files 1 kilobyte in size.
The second piece is called TMUL,
or tile matrix multiply instructions that operate on the tiles.
What this boils down to is being able to store bigger chunks of data
in each core and then be able to compute larger
matrices in a single operation.
If you're a developer or a data scientist
or a systems administrator, or you might be asking what you need to do
to enable Intel AMX and start taking advantage of the new functionality.
The good news is that almost zero effort is needed to enable Intel
AMX on Intel Xeon Scalable processors.
As long as you're using industry standard operating systems
and so forth, tools like libraries and deep learning frameworks
INT8 A and BF16 operations in Intel AMX
are automatically accelerated in the oneDNN library.
The oneDNN library can also be used
by deep learning applications and framework developers interested
in improving the application performance on Intel processors.
Most important
for data scientists and developers is that industry
standard frameworks are already optimized with oneDNN,
so you just need to ensure you're using the latest versions.
Deep learning frameworks optimized to take advantage of Intel AMX
include TensorFlow, PyTorch and others.
You can also use other software toolkits in oneAPI
like the AI kit and OpenVINO to accelerate your artificial intelligence
pipeline from data preparation to training and inference.
And this brings me to the end of this presentation.
The main take away is a new feature in Intel Xeon Scalable [rocessors,
Intel AMX that helps you accelerate your deep learning, framing and inference.
And to benefit from these new feature, you can simply use toolkits
and frameworks that have been optimized to take advantage of Intel AMX.
In the description,
you will find the links to these optimized toolkits and frameworks
and other resources that you can use to make your artificial intelligence
applications run faster with near zero effort.
Thanks for watching.