ANDREW TURNER: Hello.
Welcome to our session, Building the Next-Generation Edge
with Anthos.
In this session we'll highlight some relevant patterns
for software deployed at the edge on existing hardware
with Anthos Bare Metal.
I'm Andrew Turner, Solutions Architect with Google Cloud.
DAVID MAC MCDANIEL: Hi, I'm Mac, also known as David McDaniel.
I'm the Vice President of Cloud Services
at Cloudbakers and Qwinix.
And I'm honored to be the 26th Google Cloud Fellow.
ANDREW TURNER: We'll start today with the definition
of the customer edge, in addition to covering
some key features of Anthos.
Most of our time will be spent in the edge demo,
showcasing a sample application designed to run across a highly
distributed system.
The Cloud strategy is well defined.
Many organizations are exploring how
to modernize applications running
on existing in-place hardware.
We refer to these distributed edge
locations as the customer edge.
These customer edge locations could be inside
of a retail store, manufacturing facility, warehouse,
or a campus.
Anthos clusters on Bare Metal have enabled many organizations
to take advantage of existing enterprise infrastructure.
Hopefully you saw the exciting announcement
for Google Distributed Cloud.
Google Distributed Cloud introduces a portfolio
of managed hardware and software solutions
that extend Google Cloud's infrastructure and services out
to the edge.
Like Anthos Bare Metal, it's based and enabled on Anthos.
However, Google Distributed Cloud
includes features that are beyond the scope
of this session.
Instead, we'll be focusing on common software patterns
for applications running at the edge.
And with that, I'll turn it over to Mac
to highlight some of the key features of Anthos.
DAVID MAC MCDANIEL: Thank you, Andrew.
I'm going to talk about some of the really unique features
of Anthos, especially Anthos Configuration Management.
ACM allows you to apply a GitOps-style development model,
including all the approvals you currently
use in your standard Git-compliant repo
to Anthos configurations, no matter where they're
deployed across the world.
When you define your configuration
and approve it into your Git repo,
the Anthos Configuration Management operator
automatically sees that and applies those changes
to all selected clusters.
It also prevents manual, through a console or CLI,
changes that would be controlled by that configuration.
So it will roll those back immediately upon detecting.
Another really good feature is the service management
component.
This brings all sorts of telemetry
back to Google Cloud, things from infrastructure
to service-to-service level communications.
It enables you to create SLAs, SLOs, and SLIs,
and gives you that really single-pane-of-glass
performance monitoring and observability place within
Google Cloud.
I'll now hand it back over to Andrew for the edge demo.
ANDREW TURNER: Great.
Thank you, Mac.
We're working across multiple industries,
demonstrating how to solve business problems related
to distributed edge environments with Google Cloud
technologies--
industries such as retail, health care, and manufacturing.
And while the use cases vary across these industries,
many of the underlying patterns to solve these scenarios
are similar.
Introducing Abe's Burgers, a fictitious quick-serve
restaurant.
For the next few minutes, we'll showcase software patterns
enabling Abe's to run their applications across 5,000
locations using Anthos Bare Metal.
Let's cover some of these scenarios now.
Abe's has existing CCTV cameras and would like to leverage
these video feeds.
First, they would like to identify
abandoned and dirty tables.
In addition, they would like to leverage similar feeds
of drive-through for a better understanding
of the wait time per vehicle.
Abe's would like added features to measure quality control.
Through the use of IoT sensors, we
can detect the amount of time food
is sitting in the hot hold.
These IoT sensors produce a constant stream of data,
though.
So we help by defining application patterns which
take advantage of the data locally,
while providing aggregation and anomaly filtering,
rather than constant streaming to the Cloud.
The corporate office would like to leverage IoT
across the restaurant edges as well,
using anomaly detection to predict issues
with refrigeration units before it's too late.
Local real-time decisioning at the edge
is more important than ever.
With all the technologies available today,
consumers expect personalized experiences.
Imagine a menu board that highlights areas
based on a customer's previous interest.
Forecasting demand in real time is also critical.
There are many factors to consider,
such as the local traffic conditions
around a location, staffing augmentation,
and historical sales.
Evaluating these across each location would be cumbersome.
But through the use of Cloud technologies,
we can centralize the logic for evaluating the sales data,
calculating the expected demand, while pushing
these calculations up to the locations
for real-time decisions.
The customer experience is very important to Abe's Burgers--
understanding when the trash bins are full, for example.
Think back to a time when you were dining in a fast-food
restaurant, dodging the crowd to try
to get your trash into a receptacle,
only to notice that it's already overflowing.
Using sensor data and computer vision,
we can help avoid these scenarios.
And the decisions are not always binary.
For example, the trash may be 75% full,
but our forecasted demand model indicates that there's
going to be a lunch rush soon.
So perhaps taking out the trash now would be a better idea.
We won't have time to cover each of these scenarios in-depth.
Instead, we'll focus on the detection
of dirty tables, drive-through monitoring, and demand
forecasting.
This is the Abe's UI that we've created.
This runs locally on the restaurant Anthos cluster
and provides a single pane of glass for the restaurant staff.
Imagine combining data from the use cases that
were mentioned previously.
General managers and franchisees can better
understand the current conditions,
how they compare to a baseline, and forecast for the future.
These data points will also streamline
restaurant operations, helping the general managers prioritize
local workforce tasks, such as predictive food
preparation, dining room management, and staffing
augmentation.
Let's inspect some real-time data
streams, evaluating the historical trends in order
to predict demand, and notify the restaurant when demand
is higher than anticipated.
Since Abe's is fictitious, we don't have real-time data.
Instead, we created an emitter application,
which enables each cluster to produce realistic
looking sales and inventory events at large scale.
The output on the screen includes
events being produced by the emitter on store
number 1 Anthos cluster.
These raw events are crucial to daily operations
so we'll stream them to Google Cloud.
Using Pub/Sub we can collect these
across all 5,000 locations.
The JSON message on this screen should look familiar.
This is the same raw event coming from the Anthos cluster.
But before we can act on the data,
we need to store the raw event.
Raw events also need added context.
The data flow will assist windowing and aggregating data
as it streams through, while joining two additional data
sets.
For example, we will add context to each raw transaction
to understand which region and franchise the event belongs to.
Also, we'll be using data from the product catalog
to gather metadata about the item being sold.
Is the item part of a combo meal, kid's meal, or a dessert
item?
Data flow will also be beneficial by providing
windowing and aggregation of events.
Corporate dashboards, machine learning models,
and other applications will be able to take advantage
of this time series aggregation by minute, hour,
and at a daily log.
In addition, we'll group the aggregations
based on different categories.
For Abe's, that'll include product category, region,
and franchise.
These raw events will be stored in BigQuery for reference.
This table is important, as it records the event
as it enters into the system with no manipulation.
Aggregated data sets will also be persistent in BigQuery.
These tables are built from the raw event through data flow,
and could potentially be recreated at any point in time
through the raw events table.
BigQuery ML enables data scientists and data analysts
to build and operationalize ML models
directly inside of BigQuery and using simple SQL.
We built a simplistic version of demand forecasting
for Abe's Burgers using BigQuery ML.
While production models would likely be more complex,
this is meant to illustrate how simple it
is to leverage BigQuery ML for creation of models.
Here, we're building a linear model for demand forecasting.
This model takes into account historical sales patterns
per location and product to build demand forecasts
at a location and for a specific item
at a specific point in time.
Querying these models for future months
provides calculated answers to questions, such as,
how many combo meals will store 1,000 sell next month?
Abe's plans to use these calculations
throughout the daily operations.
So let's explore one option for getting these calculations out
of the Cloud and to the stores for real-time analytics.
Abe's already uses MySQL within their stores,
and they have a containerized version
of MySQL running on Anthos.
Leveraging our BigQuery model and scheduled queries,
we can calculate demand daily and store the results in Cloud
SQL within Google Cloud.
We'll then leverage our application
to update demand calculations at the edge.
Abe's restaurant demand application
has detected an increase in demand,
and we can see this reflected in the UI.
Do you remember the data aggregations
that we talked about earlier?
So Looker can leverage these in Tables and BigQuery,
providing franchise owners with visibility
and to real-time sales.
The corporate office is also excited to see
the updates close to real time as they
stream in from each restaurant location.
And here we have a Looker dashboard for Abe's.
You can see the real-time sales data, drill down on it,
or create alerts.
Franchise owners can also see real-time service indicators
across their organization.
And finally, we have a dashboard for the franchise owners
and corporate to visualize performance indicators
company-wide.
Next, Mac's going to show you some examples of leveraging
computer vision at the edge.
DAVID MAC MCDANIEL: Thanks, Andrew.
Now let's bring the drive-through monitoring
scenario to life.
Here's our in-restaurant, locally driven dashboard
showing that our refrigerators, coffee, and drink stations
are all good, and our tables are all clean.
Also, all actionable alerts have been resolved.
Let's focus on the top-right quadrant,
where we can see that our drive-through, curbside,
and inside wait times are all green.
As we can see here, there are only two cars
in the drive-through, with our wait time at only three
minutes--
very acceptable to receive fresh food.
We can also see that there is one guest waiting
at curbside delivery, and only a two-minute wait for guests
inside the restaurant.
As the drive-through queue lengthens,
we can see we're at a warning status with a six-minute wait
now.
Maybe time to start moving some staff from front of the house
to back.
But as typical with lunch rushes,
the line keeps building, and we're now at 13-minute wait.
This is all automatically monitored and calculated
using our computer vision, machine-learning models that
were trained in Google Cloud and deployed and executed locally.
The goal here is to forecast demand for both people and food
supplies, to enable the best experience for Abe's customers.
When this is done properly, this will lead to a positive ROI.
And the way we do this is, in this example,
are by taking video feeds from both the drive-through queue
and the indoor ordering queue, and using external Google Maps
traffic data.
Running the image through trained ML models
for detection, and then a separate model for forecasting
will result in the automatic capability of staying ahead
of demand and delays.
In this scenario, we're using two camera inputs,
the drive-through, or the parking-cam,
and the indoor security camera, security-cam.
Images from each of these cameras
are fed into the respective models
to produce an inference that indicates the predicted wait
times and the need to enable off-site order taking
to free staff locally to serve customers.
These two cameras' images are being fed through two models--
one using the free TensorFlow Hub people prediction model,
and the other being the Google Cloud AutoML-trained model
to count and provide timing on drive-through traffic.
Let's now go through the dining room cleanliness scenario
using some real-world data.
Here's our in-restaurant dashboard
showing no active alerts and everything in green status.
Here we see our quick-serve restaurant dining room
camera and our dashboard.
Our dashboard shows that we resolved all
the previous actionable status messages, and specifically,
that all tables are clean or in use.
Taking a deeper look at to how this actually works,
you'll first see the system recognizes that table 5 is
clean and not in use.
Once people are using the table, the system
recognizes the people at the table using our ML model,
and automatically updates the table status to in use.
The images are captured by the dining-cam pod
and run through the dirtytable-inference-API, which
in turn uses both the predict-people
and predict-clean-table models.
Once the dirtytable-inference-API
has both results, it stores these results
in the local database and sends any alerts
to the dashboard via a local queuing system.
We've used Google Cloud's AutoML service,
which is part of the Vertex AI suite of tools,
in order to create our dirty table model.
We simply uploaded labeled images of clean tables
in order to automatically train and test
the models used in Anthos at the edge.
You can see we actually have a pretty small number
of training images and a very good recall score.
Let's take a look at how we build and deploy the model
containers to the cluster.
At the bottom left we have a three-line Docker file
that builds our container with AutoML-generated model pb file.
On the right, we see our Kubernetes Deployment YAML.
That deploys our clean table model container and the service
exposing it to the other cluster resources on port 8501.
This enables us to call the model inferencing
function using standard HTTP protocol
and get the results immediately.
Running the models locally at the edge
reduces the inferencing latency significantly.
At the center, we see our Google Cloud Build YAML
that downloads the exported model directly from AutoML,
builds our container described in the Docker file,
pushes it to Google Container Registry,
and automatically deploys to our test
GKE cluster for model validation.
Once the testing has been completed,
we can trigger a Google Cloud deploy execution
that will deploy the containers to the Anthos Bare Metal
clusters at the edge.
For the people detection model, we used a publicly available
TensorFlow Hub pre-built model.
This significantly reduced the time and cost
of deploying and using models at the edge.
This made it very quick for us to get up and running
and start learning what, if anything,
we needed to tune in our models for better performance.
One of the biggest advantages to using Google Cloud's Vertex AI
Suite is that many of the steps that traditionally have been
manual are largely automated by Google Cloud,
once again, to save more money and increase the speed of model
development and deployment.
As we can see here on the right, these
are the basic steps involved using data
to create, train, test, validate, and deploy
a model to production.
The issues are that all the steps are somewhat manual,
that the skills needed to execute these steps are
expensive and hard to find, and you then
need additional skills for model deployment and application
coding.
When using Vertex AI's AutoML Vision service,
you don't have to actually create the model code yourself,
but rather, just select which type of model
you want to create, such as single, or multiple label
classification, or object detection.
Then you simply upload your training data,
which automatically starts a training run.
This training run automatically allocates all needed hardware
and software, which you only pay for what
you use, and automatically tunes your model based on the input
data.
As you are specifying the input to your model,
you also specify how much of your data
will be used to perform model testing, which again will
be done automatically for you.
Lastly, it's simple to download the automatically generated
saved pb model file to use within your application,
as we showed you in the previous slides.
That's it for our demo today.
And I'll hand it back to Andrew.
ANDREW TURNER: Thanks, Mac.
Sample code for deploying the scenarios we just
showed you with Abe's Burgers is readily available.
Feel free to follow along, browsing
the code for today's demo, and read more
about Anthos Bare Metal.
I want to thank you, Mac, for your partnership.
And I want to thank you, the audience, for tuning in.
We hope you enjoyed today's session.