DIVYA GUPTA: Hey, everyone.
I'm Divya Gupta, Idea Solutions architect at SAS here.
Today I will introduce to you SAS Event Stream
Processing, in short, ESP, and its integration
with Kubernetes.
Before jumping into the SAS ESP, let's
review the state of different companies in the cloud
computing domain.
Enterprises are now increasingly investing in cloud solutions.
77% of companies are already running their workload
in cloud environments.
Tech-dependent industries like manufacturing, telecom,
and others are going 100% cloud.
Containers and Kubernetes are widely used.
Data analytics and stream processing
are gaining popularity among the cloud users
as they provide an insight into the huge amount of data
coming from different sources.
Why move to cloud?
Or what motivates the companies to move to cloud?
Cloud environment provides elasticity, that is,
on the fly autoscaling ability.
It has geo-distributed data centers for quick
and 24 by 7 access, and access from anywhere.
Cost efficient pay-as-you -go model.
Flexibility on the choice of platforms, infrastructures,
storage, and computing resources.
Always available, reliable, and 100% fault tolerance.
Last, but not the least, failover and recovery systems
are quick to set and easy to manage.
Based on many benefits of cloud computing, containers,
and Kubernetes, along with the growing
need of data analytics and stream processing,
SAS offers that enterprises perform even stream processing,
analyze, and visualize their data in the cloud,
with fully orchestrated containers by Kubernetes.
With that, let me introduce you to SAS ESP with Kubernetes,
or simply, ESP-Kube.
It is a framework for controlled and automated deployment,
management, and autoscaling of ESP Server running
in container ports across the Kubernetes cluster
in a cloud environment.
Now here are some of its features
it provides horizontal and vertical runtime elasticity
with a grid optimization on consumption of computing
resources ESP-Kube offers of highly
available, reliable, and fault tolerant environment.
It allows enterprises to create multiuse, multitenant
deployments, and comes with a load balance sync
to provide scalability of ESP servers.
It has a secure web client and container support.
The framework also provides metadata management.
Let's check out the high-level architecture of ESP-Kubernetes.
ESP-Kube is based on Kubernetes, and leverages
many of its features.
For each tenant, ESP-Kube has an ESP operator
which allows starting, stopping, updating, deleting,
and monitoring of the ESP servers, containers
running in the cluster.
Then clients like ESP Studio, ESP Streamviewer, and even
Stream Manager also run in the containers in the same cluster,
and have seamless access to all the ESP servers.
It provides metadata management through persistent volume.
Ingress allows HTTP requests to the containers and access
to all the web using the browser.
User accounts and security are also managed.
ESP-Kube architecture allows access
to message buses, databases, and others
in shared spaces via Pub/Sub mechanism.
Let's have a look at the various complements and their roles
individually.
ESP Studio is used to create ESP models that
can be deployed and tested in the Kube pods running
ESP Server.
ESM manages various deployments.
With Streamviewer, you can visualize the data
by subscribing to the server pods directly.
ESP Operator orchestrates all the containers.
ESP Meter builds the processing on per event basis.
Finally, Filebrowser provides a file-managing interface
for easy management of files for web clients and containers.
Data analytics in health care is a way
to make hospitals more efficient.
It analyzes a vast amount of data
from various streaming devices and data sources,
and provides the medical facilities with relevant data
to accelerate their health care systems.
Let's look into our use case that is built on ESP-Kube.
SAS is working with a health care system which
provides asset management of the medical devices
in the hospitals--
ambulance tracking system to dispatch the ambulance
to a patient in need and prepare the onboarding of the patient;
staff management to provide work-life balance
and satisfaction; and patient tracking
to handle every single detail of the patient, from entry
to exit and post-hospital care.
However, they are struggling with a few problems.
In their current setup, they do not have any extra cloud
streaming analytics.
They have a high volume of data coming
from multiple sources, which causes data management problems
and involve resource-intensive data analytics.
This also brings in scalability and failover issues.
I would like to show you how SAS ESP-Kubernetes solution
addresses these challenges.
In the new architecture, we have the Kafka message bus
that collects a high volume of streaming
data from various edge devices to the cloud environment.
ESP Operator launches a set of stateless ESP server
pods that autoscale on the fly, depending on the workload
and resource utilization.
ESP models running in the server are
fine-tuned to perform intensive data analytics and complex
transformations.
We have integration with various databases
to handle the states and higher attention data.
ESP-Kube handles failover and recovery
of all components running the ESP-Kubernetes cluster.
Users can also use web clients to interact with the servers.
For example, ESM can be used to deploy the updated model to all
the running parts at once.
Edge-to-cloud data analytics using ESP-Kubernetes
is not limited to health care, but also
applicable to various industries,
such as retail, agriculture, telecommunication,
energy, smart cities, banking, to name a few.
Let me now show you the demo on ESP-Kubernetes, [? Server ?]
Studio, and Event Stream Manager.
This is our Kubernetes cluster, where
we have ESP Operator, Studio, ESM, and deployments running
under a tenant namespace.
Currently, we have three servers running in the cluster.
We also see them in the ESP Studio.
Let's now deploy a project under the Studio to our cluster.
ESP Studio allows the users to create, modify, update
the ESP model using drag and drop
windows from the left panel.
The right panel represents the corresponding editable XML
of the model.
We will now deploy and test our project.
Run test button loads and starts the project
in the cluster based on the specified deployment
settings for memory and CPU.
As soon as the ESP server project starts
in the Kubernetes cluster, you will start to see the server
logs at the bottom.
And the streaming data will start to flow.
ESP Servers tab now lists the project4
that we have just started.
Several properties provides the host path,
which is nothing but the path to our ESP server
pod in the Kubernetes cluster.
In our Kubernetes cluster, we see the same ESP server
in running mode.
We can also stop the server pod right from the Studio.
And you will notice that it is removed from the ESP Server
list, and also from the Kubernetes environment.
ESP Studio allows you to create, deploy, and test
your ESP models in a highly orchestrated Kubernetes
environment.
Let's now move to Event Stream Manager.
We again consider project4 for this demonstration.
Here I have a deployment called test
with the ESP model project4.
You can see the Status, Server, and other information
on this tab.
The Deployments tab lists all the deployments and the number
of projects running under it.
In our case, we have one running project instance.
ESM is a powerful solution that allows users to smoothly
deploy, update, and manage several ESP
servers at one button click.
Thanks everyone for joining me.
For more information, use cases, and customer stories,
please visit us at sas.com.