Hi, I'm Nirmit Desai from IBM.
I'm here to talk to you about Distributed AI.
Distributed AI is a paradigm of computing
that allows you to scale your data and AI applications
across distributed cloud environments.
Distributed cloud environments, as you may be familiar,
allow you to have a single pane of glass application lifecycle management
across public cloud, on-premises, and edge environments.
Now, as we look at the emergence of Distributed AI,
I want to take you through the journey of how we arrived there.
We start with the cloud-based AI,
we go to Edge AI, and then we talk about Distributed AI.
Also, I'm going to introduce to you
the challenges that the Distributed AI helps you address in your business.
Finally, all the capabilities that we are creating for enabling Distributed AI
are available for you to try freely at IBM API Hub.
See the link in the description.
Without further ado, let me take you through the journey of where we've been.
First, we're going to talk about cloud-based AI.
What happens here is you have a, let's take a concrete example,
so that concrete example is going to involve a plant.
It actually could be any location where you have your business operations
and you're making some local decisions.
On the other side of the picture, you have some kind of core location,
could be your enterprise data center, could be public cloud,
I'm just going to take an example here.
So let's say it's a public cloud,
and what you have is, in this public cloud, you have
some kind of Kubernetes service
with your data and AI middleware.
And then on top, you're deploying one or more applications.
These applications, when they are data and AI based,
you may be actually doing some kind of training for AI pipelines and you may be doing inferencing as well.
All right.
What happens on the business process side on your plant
is that, as the process takes place,
it generates a tremendous amount of data
and all that data is getting pushed to the core location
where the decisions are being made through the AI pipeline influence.
Those decisions are then communicated back to your plant
where it drives your downstream automation.
So clearly, because you're sending all the data over to core
and it could be a large amount of data, it could be sensitive data,
it could run into the challenges of connectivity, intermittent connectivity issues with your core location.
This has run into challenges.
This is why we're seeing the emergence of what I'm going to call "Edge AI".
So what happens in Edge AI?
So, in the case of Edge AI, you still have your client.
I'm going to draw a slightly bigger box here
because more is going to happen at the plant.
And you still have your core location.
Unlike before, where most of the decision making actually was happening in the core.
You're going to have the decision making happening here,
you're going to take advantage of distributed cloud environments
and distributed cloud platform capabilities
to make the application lifecycle from core to all your plant.
So what happens then is you actually have the container platform
with data and middleware deployed on it
and the application deployed right in your plant.
Your core still does what it did before,
except it is now taking care of deploying the application and taking care of its lifecycle.
So let's complete the picture here.
You have data and AI.
And then you also have the application deployed on it.
Unlike before, you're going to train your applications here,
deploy them through the distributed cloud platform,
the single pane of glass, that's important,
and these applications are going to make inferences here.
So if your business process is taking place,
or the business operations are taking place here,
what happens is that your application is generating decisions.
And they are driving your business process, and this process is then feeding data back
to your computing stack that is in the plant.
So what we have done here is we localize decision making.
We no longer have to continuously send data up to a core location
and wait for it to make a decision that then can automate our business process.
Of course, we still need to send some data over.
And we have to use that data to train these, or retrain, these AI pipelines
and redeploy them.
So, you made some progress when we switched from cloud-based AI to Distributed AI - or actually Edge AI, pardon me.
But when we try to deploy this pattern across a large number of locations
and across a large number of and large variety of applications,
we run into certain challenges.
And so we have then a need to address those challenges
with the capabilities we are describing as Distributed AI.
The pattern of Distributed AI is very similar to Edge AI,
but I'm going to replicate this and I'm going to move away
from the terminology of "edge" and "cloud" and "core"
to actually talk about what matters the most.
What matters the most is where is the data, and where does it need to be analyzed?
So, it is possible that you have a vast amount of data sitting in a public cloud,
but you want to consume the AI capabilities from another cloud.
In this case, the first cloud is what we call as a "spoke".
And this is where your data is.
On the other hand, the cloud where you have the AI capabilities and the application and analytics
is what we call the "hub".
And this is where your control plane is.
See how it allows us to talk about hub and spokes,
the hub and spoke do not really have a connotation of cloud or edge,
or whether this is a mobile vehicle
or this is a stationary datacentre, it doesn't really matter.
What matters is your data here.
Your control plane here.
You want to manage the deployment of applications from hub to spokes.
You also want to take control of the data and AI cycle from the hub.
So I'm going to, for the sake of completeness, complete this picture,
which looks not very different from what we have seen before,
so that I can take you through the challenges you're going to run into
when we try to scale such a stack to a large number of spokes
and a large number of applications.
OK, so let's say we have this application and you have you have your business process here.
The decisions are going down.
The data is coming back.
You are, of course, deploying through the distributed cloud environments,
and you're pushing some data over.
Just to complete that picture.
So what happens when you have a potentially large number of these spokes
on which you're trying to enable this AI application?
The first thing that comes to mind
is that because you are still collecting the data for training,
and you're pushing a large amount of training sets,
you know these models consume a large number of large amounts of data,
and you have a large number of applications in a large number of locations doing that,
you're going to run into a challenge we call as "data gravity".
It's just causing two main problems for you.
You're putting tremendous pressure on the resources in the hub to manage all that data,
and you're actually incurring costs in then having having to analyze the data, having to train that data,
not to also mention some of the network bandwidth limitations that may coming your way,
especially as you try to do this for a large number of applications.
So data gravity is a key challenge.
The next challenge I want to introduce to you
is the fact that each of these spokes maybe slightly different.
You're probably manufacturing a slightly different product mix at each of your plants,
or each of your retail stores are serving a slightly different demographics.
Because of that, one model that you're playing on, one pipeline that you're trained in your hub,
is not going to be fit for all your spokes.
So there is always going to be a challenge in dealing with that heterogeneity
and not having to do manual work.
So we'll get to how we address that in a second.
The third challenge is just the sheer scale.
We talked about scale, but it actually has two aspects.
One is just the number of spokes to deal with
and the computational complexity of doing that,
training so many models, deploying so many applications in so many locations.
The second part is the variety in applications and data.
Remember, in all these cases we have where we have looked at data,
the data could come in many different types.
So you have data types of, let's say, images.
It could be sounds.
It could be sensor information.
It could also be Lidar,
network information, and time sensitive information,
there is just a very wide variety of data modalities
that different applications that you're trying to deploy and manage would need to consume.
So that variety in applications and data
make it even harder for you to scale and accelerate deployments.
The last challenge I want to introduce to you is the challenge of resource constraints.
So, although the spokes and the hubs may have some resource,
it is also quite common for some of the spokes, such as plants and retail stores,
to have a finite amount and a small amount of resource.
So you have a resource budget that must be respected
as you deploy your data in the pipelines to them.
And that causes new challenges, right?
So resource constraint is a key challenge as well.
Now, we're very excited that in IBM,
we have addressed these challenges head on
and enable this Distributed AI that scales across distributed cloud environments,
across locations and applications.
So how do we address data gravity?
Well, the key thing to do in addressing data gravity
is to not collect all the data, but only the important data.
So intelligent data collection
is a key capability that we are going to bring to you.
And you can actually try it out, through API Hub as I mentioned earlier.
A lot of the data at the spokes is repetitive.
Some of that is noisy, so you don't want to necessarily collect all of it.
You only want to collect what's important,
and identifying what's important, especially when you have a large number of
locations and vast variety in the data modalities and applications,
is a challenging problem to solve.
The second part, about heterogeneity, basically means that
when you're deploying your AI pipelines and applications across different spokes,
you want to target them.
You want to adapt to those spokes.
So adapting.
And also then, after deployment, you want to monitor,
to make sure that they are performing well.
Adaptation and monitoring at each of the spoke locations
is critical in addressing the heterogeneity challenge.
And then in terms of the scale,
it simply means you need a greater amount of automation
in controlling your data and the AI lifecycle.
So, automation of data lifecycle basically is about policy-based decision making
to see what data should stay where,
when should it be purged, when should it be replicated where,
what policies in terms of data localization apply
so that you can respect those constraints as you take care of data lifecycle.
Automating that then lets you address
the large number of locations that we've been talking about.
Similarly, the AI lifecycle can also be automated.
So, starting from training the models, deploying them, monitoring them,
if the data or the environment drifts
then retraining them, collecting the right kind of samples through intelligent data collection,
and then using them for retraining the model.
Automating all that lifecycle is critical as well because
you may end up with hundreds, if not thousands,
of different AI models and pipelines that are automating various aspects of your business
as you start scaling this.
Lastly, on resource constraints,
what is essential is that we have some ability
to optimize the data and AI pipelines.
What this does is it does things like feature extraction,
a model compression, pruning, and some of those techniques.
It brings them to bear to make sure your resource budget is respected at all times during your pipeline execution.
So in summary, we have introduced to you
a new paradigm called Distributed AI,
and we have introduced to you some capabilities that actually bring it alive.
Distributed AI will allow you to scale applications to a large number of locations,
large number of spokes, and it allows you to scale across a wide variety of applications.
Thank you.
Thank you for watching this video.
If you're interested in more content like this, please hit like and subscribe to this channel.
Also, please check out the links in the description,
which will get you started on Distributed AI APIs on IBM API Hub.