- Hi, my name is Haagen.
I'm a Solutions Architect
at the SageMaker Edge team.
SageMaker Edge supports different ways to deploy models,
to edge devices and production, including Greengrass to,
or using an old deployment mechanism.
But we also want to make it easy for you to get started
with SageMaker Edge and run your models on an edge device.
There are a couple of steps required before
you can start using our lightweight run time,
the SageMaker Edge Agent, that includes
creating certificates, setting up IoT credentials
for authenticating the edge device with AWS,
as well as downloading binary send certificates
to the device itself.
So we created the SageMaker Edge quick device setup tool
that simplifies an installation of an edge device
as well as required resources in the cloud.
The quick device setup runs on the edge device
and goes through the following steps for you.
It creates an S3 bucket to store data captures
from edge devices.
Then it creates a device fleet policy and device fleet role.
So that your device can interact with AWS services.
It creates and configures an IoT thing,
so that your device can use IoT services
to authenticate with AWS.
Then it creates a SageMaker Edge fleet and device
after creating all the cloud resources,
it downloads the created certificates,
as well as edge agent binaries onto the device.
Lastly, it creates a configuration file
for the edge agent on your device.
If you specify resources that already exist, for example,
the S3 bucket or device fleet,
then the quick device set up tool, will reuse those.
The installed edge agent package includes basic sample code
to communicate with the edge agent
and run inference on your device.
The package includes documentation as well.
Now let's have a look at how this fits
into the machine learning life cycle with edge devices.
The initial model can be created in SageMaker.
That model then gets optimized and packaged
for deployment on fleet of edge devices.
When the model is running on a device,
the edge agent runtime lets you capture data
from those models and send that data back into the cloud.
The uploaded data can be used to retrain the model
for better performance.
That updated model can then go through the same cycle again.
We are providing the quick device setup tool
and source from on github.
It has been implemented and go,
you can build the tool for a target device.
You can also download people binaries for each release.
The repository includes a README file with detailed
instructions on building and running the tool.
Please note that the quick device setup tool
is meant to be used in development and testing only.
One of the reasons for this is that it requires
a fairly permissive IAM policy,
so that can be create all needed cloud resources
for an edge device.
So let's have a quick look at the README file
and specifically the IAM policy.
You can see that the device will need access to IAM
to create the policy and role.
It requires access to IoT to create the IoT thing
that will be used by the device to authenticate against AWS.
It needs access to a suite
to create the bucket for captures.
Then it also needs access to SageMaker
to create the device and device fleet.
It also needs read access
to the SageMaker edge release store
to download the HHN binaries.
So let's run the quick device setup tool
on an actual device now.
I already copied the binary for the tool
onto my raspberrypi.
I also loaded some credentials so it can run.
Let's have a quick look at the command line. You will see,
you have to specify the account ID for the account
that you want the device to be registered against,
a local directory to install the edge agent to,
then an S3 bucket and folder prefix for data captures,
the name for the role that the device fleet should be using,
the name for the device fleet
and a name for the actual device.
So let's run the tool.
It will run through all the steps that are required
to set up your device.
It's going to take a couple of seconds.
At the very end you will see that it shows you
the configuration for the edge agent.
So you're ready to run the agent now.
Now that the device setup is complete.
We can start the edge agent.
Lets have a quick look at the command line.
We have to specify a socket that an application will use
to communicate with the edge agent as well as point
the edge agent to the configuration file
that the quick device set up created for us.
So lets start it.
Let's see some printouts and now it's listening
on its socket and we can run one of the simple applications.
The HHN package comes with two examples,
one with C plus plus, and the other one in Python.
Let's have a look at the Python example.
It comes up with a shell script that runs the whole example.
So let's have a quick look at that.
It first installed some Python requirements,
which are the grpc tools,
the api definition for the edge agents
provided in language independent proto file
that you need to compile for the language that you want
to build the application with.
So in this case, it's compiling it for Python.
Then it wants the actual example code,
which requires a couple of parameters,
the first one is the URL to the model.
Then the name of the model under which
you want to register it on the device,
the name of the image input tensor and the path to an image.
As you can see it requires an computer version model
on the device to run the example.
I downloaded mobilenet-v2.
So let's run the shell against that model.
So first the model URL which I have in my model path,
then the model name that I want to use is mobilenet-v2.
Their parameter is the input tensor name.
Just input one for this model,
and then a path to an actual image, which is this.
Let's run the example.
As I said, first installs the required packages
then compiles the model,
the API, and then it runs the
infers against the model that they provided.
Can see it loads the model then it lists the model
which shows some parameters of the model,
it runs prediction on the image,
and then it unloads the model again.
You probably noticed that the example of Python script
didn't output any meaningful information
about the prediction that it ran against the model.
There's a good reason for that.
The example script is mainly there to show you how
to communicate with the edge agent,
but not necessarily to create meaningful output
from an arbitrary computer version model.
Let's have a look at the predict image method,
it basically directly loads the image data
into the input tensor.
Each computer division model requires a specific
pre-processing on the image data
so that the model can make sense of the data,
as well as some post-processing on the output
to create human readable or understandable information,
you can easily modify the predict image function
to add that for the model that you want to run against
to create meaningful predictions for your testing.
So we just ran our example.
Now let's have a look at the SageMaker console
and have a look at our device here.
So we go into edge manager, edge devices,
and there is our pi-demo-device-001.
You can see it got registered today.
It got shortly thereafter the first heartbeat
we're running mobilenet-v2, version 1.0,
and we also running inference on device.
This concludes the demo. Here's some additional resources,
including a link to our documentation of supported platforms
and devices linked to the github repository
for the quick device setup tool,
as well as links to notebooks and workshops
that can help you get started with SageMaker Edge.
Thank you and please have a look
at our other demo videos as well.