What's up everyone? Today's episode is all about machine learning,
specifically computer vision.
If you're looking to bring computer vision to your application in the cloud,
stick around, this episode is for you.
Our guest for this episode, Mo, is based in the UK
and is the founder and CEO of Satis.AI.
SATIS.AI is a full stack artificial intelligence operating system for kitchens.
Mo has a PhD in formal methods, a branch of computer science
from the University of Manchester.
Let's hear from Mo.
On the technology side, we deployed a set of hardware to each kitchen,
which is going to be a couple of cameras with units
which are piping this picture initially to the cloud,
and eventually to some Edge GPU-powered device
for the videos to be interpreted and then produce events,
which we are gathering all together
and this digital twin state is getting evolved
through the events that are being piped to us
as a result of evaluating the picture or possible sensors
that we are going to install in the kitchen
as well as some sort of a structured data that is coming to us
like the order data, or equipment, IoT kind of feedback from the devices.
Mo wants to build an application
that helps line cooks place the right food items
into the correct customer's delivery bag.
I'll also share tips for you to build your app with computer vision using AWS.
Let's cook up a computer vision solution for Satis.AI.
I'm going to put an order in for a Manchester cheeseburger,
with Canary Wharf fries and a London Swirl.
The solution for Satis.AI will be connected to the restaurant's
point of sale system, which will capture my order.
Next, the line cook will get a new bag.
And tag it with the order number and then place that on a countertop.
In Mo's case, he doesn't just want to know
if the item is a hamburger or french fries,
which can be detected with existing computer vision solutions like
Amazon Rekognition,
he needs to know if it's the correct item off the menu.
Is it the signature Manchester burger with the Canary Wharf fries,
or is it the Liverpool double bacon burger with extra large Canary Wharf fries?
There we go.
This requires building a specific model with the right spice.
Is it oregano? No!
Pepper?
No!
Oh, wait that's it... sage!
Amazon SageMaker provides a single solution to train,
build and deploy your machine learning models at scale.
Now we're cooking.
Before you can run models, you need to bake them.
The first step is to gather your computer vision machine learning ingredients,
which are images of food from the menu.
You'll need to feed your model a lot of images.
The more images you provide,
the higher the likelihood you'll build an accurate model.
You'll also need to factor in lighting and different camera angles
to build a robust model.
Once you have your images, you'll need to prep them,
which means labelling them with descriptions.
Labelling images is very time-consuming.
I recommend using Amazon SageMaker Ground Truth
to offload labelling to your co-workers or to contractors.
So while you're at happy hour with your friends,
they're hitting the keyboard for you.
SageMaker Ground Truth is a fully managed data labelling service
that makes it easy to build accurate data sets for machine learning.
Once your training images are prepped with labels,
you can put them all into a large mixing bowl like Amazon S3.
S3 is your go-to AWS service for object storage.
You might need to try a few machine learning recipes
to get the right taste for our model.
Mo will evaluate models by having them predict labels
for a new set of images.
Then, he'll compare the predictive labels to the true ones.
The goal is to achieve a very high success rate of predictions.
Once you have a delicious model, you're ready to deploy it into production.
Voilà! Dinner is served.
Satis.AI will need to repeat the process for multiple models
to answer questions like,
"What item has been in place and to which bag?"
And, "which bag corresponds to an order?"
With this information, Satis.AI can use AWS Lambda functions
to keep track of orders in Amazon DynamoDB.
Lambda gives you serverless compute
and is even available on some devices for Edge computing.
DynamoDB is a NoSQL database that's great at capturing
the state of data in real time.
You can use DynamoDB Streams to keep a time ordered sequence
of item level modifications in a DynamoDB table.
If the wrong piece of food was put in a bag,
or gets picked up before the order is complete,
then DynamoDB Streams will trigger a Lambda function
to send a warning message to the restaurant's order management system
through Amazon Simple Notification Service.
Amazon Simple Notification Service is a web service
that makes it easy to send notifications from the cloud.
Now the next step is making sure that you are set up to deploy your model
in the real world.
For this application to be useful,
you'll have to run inference that's snappy
to make your customers happy.
This is crucial for any real time computer vision application.
For a use-case like Mo's that requires low latency and high bandwidth
you'll want to deploy your models on site by taking advantage of Edge computing.
The cameras have to constantly look at the order,
the bag and the item to understand the state.
This lends itself well to AWS IoT Greengrass.
Greengrass extends the cloud to Edge devices
so they can act locally on the data they generate
while talking back to the cloud.
With Greengrass, you can run Lambda functions
and machine learning models right from the device.
You're going to need camera sensors and Edge compute.
To do this, you have two options.
One, for models with multiple camera inputs
purchase a Computer Vision Developer Kit that is compatible with Greengrass
like the NVIDIA Jetson
and separate cameras that are compatible with this kit.
Two, for models that are fed from a single camera,
purchase an all in one Edge AI Camera.
This combines an image sensor and a compute module
for simplified deployment.
This is great for running one model per camera.
In both cases, we use hardware compatible with Greengrass.
Greengrass is compatible with Amazon SageMaker Neo
to optimize the model for the hardware that you're using.
SageMaker Neo is a feature in Amazon SageMaker that enables us
to take machine learning models trained in the cloud
and deploy optimized versions at the Edge.
You can find a list of Greengrass compatible hardware
in the AWS Partner Device Catalog.
When selecting a camera, consider the environment it'll be in,
the available bandwidth and the types of detections you'll need.
Most importantly, consider your budget since camera prices can range from
just under $10 for a Raspberry Pi Cam to one million dollars.
Pretty insane!
But it might help you get a better social media brunch picture.
Most solutions won't need a helicopter camera.
Though definitely consider setting aside a budget
for cameras with a high bit rate, frame rate and a wider angle lens.
Even the best models encounter unexpected situations.
Use Amazon Augmented AI to create a workflow
so that if your model identifies an image with a low confidence score,
it will be reviewed by a human.
Catching these Edge cases helps us make our model better over time.
That made me hungry. Now, it's time for a recap.
The first step of a computer vision machine learning project
is to get a lot of images of your subject matter.
Next, label them using Amazon SageMaker Ground Truth.
This will be your machine learning training data.
With this data, train and build models in Amazon SageMaker.
Next, select a hardware for deployment.
For simplicity, pick an all in one Edge AI camera
that is compatible with AWS IoT Greengrass
so you can run machine learning models right on the camera.
You'll thank me later.
For flexibility, purchase a Computer Vision Developer Kit
that is compatible with IoT Greengrass and cameras or sensors
that are compatible with that hardware.
Use SageMaker Neo to optimize and deploy your model to your device.
Build an application layer with DynamoDB and Simple Notification Service
that handles communication between your system and the customer system.
Finally, create a workflow with Amazon Augmented AI
so if an image generates a low confidence score,
it is sent to someone for review
and can be used to improve your model over time.
That's how you cook a delicious, organic,
vegan and low calorie computer vision app in AWS.
It doesn't matter when you start as long as you just get started.
Thanks for hanging out with me today. See you next time.