one of the most powerful ways to use rust is to dive into using things like pre-trained models
using GPU building high performance command line tools that you can do batch based machine learning
inference against what I'm going to show you is how simple it is to use GitHub code spaces with a
GPU enable that GPU build up the rust bindings for it and take a translation tool and build it out in
just a few minutes okay let's go and get started here I've got a repo no GE BJJ Rest by torch GPU
template that has all the things you need to get gpus cooking in code spaces and what I'm going to
take a look at here is the diagram of what we're going to build so to start with you have a Cuda
GPU enabled in code spaces and then I'm going to go through and use the rust bindings for hugging
face again rust is a high performance language that allows you to build very performant tools
and then we're going to put that together and use it to enable rest and GPU plus hugging phase to
to translate a song that's in Spanish to English and it actually happens in a very short amount
of time just a few seconds and we're also able to see that the GPU is being utilized so really this
is the dream scenario for many people that want to build batch based machine learning tools rust plus
a GPU plus hugging face is a great solution so let's go ahead and take a look at the the
crate real quick you can see here it's Rustburg ready to use NLP pipeline it takes advantage of
the pre-existing pytorch uh you know rust bindings here and we're able to actually use that so let's
go ahead and dive in I'm going to go ahead and go to this environment and launch a code space
all right once we go through here and launch this code space first I'm going to show you uh the code
and then we'll go through and run the code so to start with it's actually pretty straightforward to
interface with hugging face and a GPU a few things to point out when you first get started is that
it's a good idea to actually be able to monitor the GPU so how would you do that well one of the
ways that you want to take a look at the GPU is run Nvidia hyphen SMI Dash L1 right this shows us
if there's anything going on there's nothing going on with the GPU so when we do actually
do the inference that's actually the step that we're going to take the other thing that we'll
need to do is go over to a repo that I've set up here called translate and if I go ahead and I CD
into translate let's go ahead and walk through the code here so first up we have the cargo 2ml
here we have just three dependencies first I use those respir bindings second I use a command line
tool and then third I use this Library called anyhow so fairly straightforward and this is
the namespace that I'm going to work with again when I build out tools typically I like to build
out things with a lib and a main the main is the chameleon tool and the lib is where I do
all the work let's go ahead and take a look at how this work is actually processed the first thing I
do is I go through here and I say a library that uses hugging face to translate text here's where
I use Russ Bert and then I build a function that allows me to read a file and return back a string
right pretty straightforward something you do in python as well then I build another function that
reads a file and returns an array of lines of the file and you can see here that the reason
for this is that we want to pass an array which is like a list in Python into the translator now
the final piece is quite simple we just build a function and I expose it via Pub right here
so that the command line tool can get access to it and I say okay let's go ahead and build
this translator let's go ahead and say that the source language is going to be Spanish and the
destination language is going to be English we obviously could build a more sophisticated tool
that could accept different kinds of language models but just to make things simple we're
going to do this and then I go ahead and I create the model then I create a text here which is the
vector of strings and then I pass in the text to the model and what's nice about this again this
uses this function over here is that I'm able to actually print out what all the sentences are if
we go over to the main here very straightforward again a pattern I like to use over and over again
is I use this command line parser and I'm map the functions inside my library to sub commands I have
a translation command I have a print command to T it all together we just go through and we call it
right so this is a powerful powerful pattern to use and then if we want to look at the text here
inside we'll have some Lyrics these are the lyrics to a popular Spanish song from Mana all right how
do I get this thing running pretty easy all I got to do is invoke whatever is appearing
right here so I can go ahead and set cargo run dash dash dash help this is how I first kind of
Kick the tires here and get things cooking what what this will do is go through and invoke it
and show us what the sub commands are now the the part that I think is going to be fun is
let's go ahead and go back to the bash terminal here and let's actually go through and do Nvidia
SMI Dash L1 and this is going to show us whether our tool is able to utilize the GPU so let's go
ahead and run it and let's actually go through and see if I put in the path to the lyrics how long
it takes okay it's going through it's invoking we see in fact it is using the CPU the GPU and it ran
in just a few seconds we're able to translate a document so this really shows you the power
of why it's so important to use the hugging phase plus rust plus GPU combo you can build
batch based tools that really are going to run circles around interpreted languages like python