Hey, my name is Sam Partee, Machine Learning Engineer at HPE and today
I am going to talk a little bit about what SmartSim can do for you. So what is
smart soon? Well SmartSim and open source library dedicated to
accelerating the convergence of AI analytics, and data science with HPC
simulation models. SmartSim can connect models written in Fortran,
C, C++, and Python to what we like to call the modern data science
stack in an online fashion. What this means is that you do not use the file
system to communicate data to and from those Fortran, C, or C++ -based
models online, meaning no file system interaction to analysis codes,
machine-learning codes and otherwise generally data science-like activities.
The SmartSim stack enables scientists to embed calls through machine-learning models,
execute and monitor those models straight from a Jupyter notebook, easily
communicate numerical data between those languages and the native array
format of that language, create, configure, and execute ensembles all
while bypassing the file system. It's a little bit of a high level architecture
of SmartSim. You can imagine this slide is a cluster supercomputer system
that we are going to be deploying things out onto using SmartSim. Well, the
first thing that SmartSim deploys is that orchestrator. The orchestrator is
an in-memory database. It's built on top of Redis and it's essentially
a large hatch map written in C that I like to explain to people who aren't
familiar with it as a giant Python dictionary that is globally distributed.
But this can be distributed out on through SmartSim out onto a cluster
supercomputer system into a shared- nothing cluster. Redis' module
system, through Redis AI, allow us to deploy ML run times like Pytorch,
Tensorflow and Onnx, which you can think of like Scikit-learn or Spark
inside that in-memory database so as that it can be called from any
of the SmartRedis clients and C, C++, or Fortran.
Those clients can be embedded into native [inaudible] or higher forms code,
commonly simulation workloads and connected through SmartSim
automatically to that in-memory database so that any data being
streamed with those clients can be immediately stored within that database
and consumed by any AI models that are stored within that database. As well,
scripts in the Torch Script API from Pytorch can be executed on data on CPU
or GPU on any data stored inside that in-memory database, SmartSim
launches both simulations and the same memory database such that they are
automatically connected at runtime.
SmartRedis, the client library also provides the user space object called
the dataset which groups in dimensional tensors together along with
associated metadata such that you can store and retrieve those groupings of
in dimensional tensors and associated metadata through a single key. Once
again, none of this goes over the file system. There are two components to SmartSim.
The first is the client library that I mentioned earlier, SmartRedis.
There is a SmartRedis implementations and Fortran, C, C++,
and Python which implement the Tensor and Dataset APIs that I will
talk about here in a little bit. They are built on top of the AI client
which implement the Redis AI API and are supported by two libraries
in the Redis open source ecosystem. The infrastructure library would actually
launch all of this infrastructure out onto supercomputers and cluster systems
it has a Python interface which allows users to deploy those in-memory
database technologies like Redis and multithreaded version KeyDB along with
those [MO] run times like Pytorch Tensorflow, an Onnx. Some of the
exciting new things that we have going on in those blue shaded squares are
reinforcement learning and HPO integrations through Ray, which SmartSim
will be able to launch out onto cluster systems for you, as well as a
new launcher, LSF. Speaking of launchers, SmartSim supports four
launchers today: Slurm, PBS, Local, and Cobalt, such that users never have to
write a batch file or, in general, interact with the scheduling or
workload manager of a system to be able to run their jobs. The Torch group API
that you see there provides online processing capabilities and it's [inaudible]
Trace Python that comes from the Pytorch ecosystem. Once again, all of
this can be executed inside this in-memory databases that are listed in the
storage layer above. This is the tensor and data set APIs that I mentioned
earlier for the SmartRedis client. It's important to note that these are
all available in Fortran, C, C++, and Python. The Tensor API, you can
see there, is simple put its semantics along with the model an script API, put
and get, set and get for models, and set and get for scripts, as well as well as
run models and scripts and those can be set from Python and called from any of
the other languages and vice versa. So if you would like, you can spin up your
simulation, load that AI model from Python and then call it directly from
Fortran. You can do this with any of those machine-learning runtimes that I
mentioned earlier, along with the Torch script API for online processing.
The dataset API that I mentioned earlier, which groups in-dimensional tensors and
associated metadata is the user space object that can be constructed with
the functions that you see there, adding tensors, metadata and strings stored in
the database with a single key through puts and gets and inspected once it is
in user space are retrieved through get, unpack, and the associated string and
metadata functions that you see there. All of the supported data types are
listed on the left for any of the data set or tensor API function. So it's a
fairly large stack but what can you actually do with SmartSim? What are
some examples? Well, the first example is online analysis. You can think of
this like streaming data analytics. We have some MPI + X simulation and
C/C++/Fortran and we use that simulation along with the SmartRedis
client embedded on each rank of that simulation to stream data to this
in-memory database, the orchestrator, that could be distributed on end nodes,
and while that's happening from a data science medium like Jupyter, are able to
pull out those fields in Python and immediately analyze them. So if you had
some simulation that needed to run for 10,000 [core] hours and you didn't want to
wait for all of the results to get there, in this way, you could
immediately analyze the results of those simulations and also consume
those results from a third-party application or other program.
The clients are very lightweight, the single static shared library that can
be embedded inside your code and called with just a few function calls that you
saw in the last screen. Once again, none of this goes over the file system.
Online processing is another example of what you can do with SmartSim. In this
case, we have a MPI + X simulation. Once again, it could be in any of those
languages that SmartRedis supports, and we are doing the same thing as we did in
the online analytics example, except for in this case, while we are streaming
that data, we are also processing it. So I have given a small code example on the
right here, which takes the singular value decomposition function out of the
[torch script] API and we put that calc_svd function inside the
database through a call to set function. Once set from Python, that function can
be called from Fortran, C, or C++ so that you could set that function
from Python, Call it inside your simulation, and have singular value
decomposition and torch script called on any of the arrays inside of your
simulation. Once again, this is the entire torch script API, so you don't have
to implement any of these functions, you just go look for them and then set
them in the database to be able to use them immediately from any of the
SmartRedis languages. The code example you see here includes code from both
the infrastructure and client libraries. The infrastructure code is highlighted
in red, which is the SmartSim library. And the blue code is the SmartRedis
client library. You can see here that we start up the database and then we
connect your client to that database. We put a tensor and we set that svd
function. We run that svd function on the tensor that we put into the
database and then we obtain the U, S, and V That you would expect as the result
of an svd function. Then we stop the database.
Online training is another example of what she can do a SmartSim. In this
case, while we are streaming the data from our MPI + X simulation, we are
also consuming that data with the TensorFlow training program.
This training paradigm could also be paralyzed use of Horovod or a
great DL Plugin and in this case, while you are training you can also be
check pointing that model and storing it inside that orchestrator such that
it's immediately ready for use in another program. So this creates
an entire training and inference loop, where you can create regular updates
to your model by passing data from a simulation, training a new model, and
then serializing that model and sending it to the in-memory database so that
they can be consumed once again through run model calls from any of the SmartRedis
languages. In this way, you have a really easy way to create multi-version
models and multi-version datasets all inside this in-memory database
without touching the file system. Lastly online inference. So this is one
of the primary capabilities of SmartSim. As I mentioned, models and Pytorch
Tensorflow or Onnx, like Scikit-learn or Spark or others, they can be written
in Python and then called from Fortran or any of the client languages.
The Python code for the models executes in a C-based runtime without the
interpreter, so you can be assured that it's performant it's that [trait] around time I
talked about earlier. The calls are evenly distributed. That calls to run
model or set model are evenly distributed among all shards of the
database cluster because it is a shared- nothing database cluster. They are
evenly distributed and they can be set on CPU or GPU, such that you can use
multiple GPUs, multiple CPUs, you can pick which specific device you can see
in the client.set_model called below, we specify that the CPU will be
running this particular model and this code example that I am showing on the right
once again shows how you can use the infrastructure library and
the client library together if you so wish but also it is important to note
that that blue outline code could also be in C, C++, or Fortran and
stored with inside that simulation such that you are actually performing
those calls from within an HPC code base. In this case we are showing
a simple serial example, where we are taking a Pytorch [inaudible] of cnn,
setting that inside the database, putting an input tensor inside of that
database and then running the model on that input tensor and obtaining the result.
In this way, you can see a full serial example of the use of SmartSim
and SmartRedis together. And lastly, here you can see the QR codes for a few
resources for SmartSim. One is our GitHub. Please feel free to check out
and star it, if you have some time and then also the QR code for our SmartSim
white paper that we did in collaboration with the National Center
of Atmospheric Research. As well as my email. Please feel free to reach out
with any questions and thank you for listening.