My name's Kate Soule.
I'm a senior manager of business strategy at IBM Research.
And today I'm going to give a short overview of what are the different strategies
we can employ in order to improve the foundation model's trustworthiness
and efficiency for enterprise deployment.
At its core, you can break down a foundation model into a couple of key components;
the data, the architecture, and the training.
And for each one of these areas, there are different strategies and techniques that we can employ
in order to improve efficiency and model trustworthiness.
Let's start with the data.
When we talk about data, we're normally going to talk about it in regards to its quantity, quality, and degree of specialization.
On the quantity side, these models are trained on enormous amounts of unlabeled data in a unsupervised fashion.
It's this volume of data that they're trained on that gives them the super power
to be able to perform an immense number of different downstream tasks easily and efficiently.
But the more data that you're training your model on, you're going to drive up your compute costs when it comes time for training and inference.
There's been a lot of research actually going into it.
How much data do you need in order to train a foundation model as efficiently a foundation model, as efficiently as possible?
If you're trying to make it efficient for training, what's the least amount of data you need per parameter in your model?
So that's a way to measure model sizes, the number of parameters in it.
So it’s the least amount of data that you need per model parameter in order to get a degree of accuracy.
If you’re just focused on the training costs.
You need about ten words per parameter in your training dataset to make these models efficient.
If you're talking about the inference cost though, so think of this as like the fixed cost to train the model once
versus the the marginal cost to use the model over and over and over again.
You can improve the efficiency of your model by making your models far more data dense,
using as many as 100 plus words of data in your training dataset per parameter of your model.
The quality of your data is also going to be incredibly important when we talk about model trustworthiness.
It's just like machine learning, if you have a poor input quality of your data, you're going to have a poor quality model output,
if you have biased training data, you're going to have a biased model but unlike traditional machine learning, there's so much data to sort through.
It is really hard to verify the quality of your data.
And that's why it's important to be very careful of where the data is coming from.
Most of this data is scraped from the Internet
and there are dark corners of the Internet that should not be used when creating training data for these foundation models,
but also then employing hate and profanity filtering,
HAP filtering on this training dataset in order to extract and take out remove any hateful or harmful materials.
Finally, another strategy that you could take on the data side to improve the efficiency of your model is the degree of specialization.
Here the insight is coming from, if you have a question, let's say, about a medical issue,
would you rather ask the smartest person that you know or would you rather ask your doctor?
You would probably rather ask your doctor.
You want an expert's opinion.
And the same goes for foundation models.
We're seeing that smaller models that are specialized in a domain, that are experts,
that are trained on maybe a mixture of 50/50 specialized medical data or finance data.
And 50/50 general data can perform as well, if not better than a much larger general purpose foundation model with no degree of specialization
allowing us if we have tasks that are specialized to a domain to get away with much more lighter weight, smaller, efficient expert models.
So that's the data of the picture.
Now, let's talk about architecture.
On the architecture side, we think of architecture as a way to have a blueprint for how this data gets encoded into the model.
And there are different styles that have emerged that have different advantages.
One of the styles that have emerged is a decoder only model.
This is, for example, GPT3.
These models are really performative, they're really powerful, but they're also very dense,
whereas we have other models that have emerged, such as encoder decoder models that are much more lightweight and efficient.
In addition to the style of the architecture, there's the size.
And here I'm talking again about the parameters.
So you both want to make sure that your size of your model is related to the size of the training data that you have.
If you have too big of a model, too many parameters for the amount of training data you're going to over fit leading to trustworthiness issues.
And you're also going to have efficiency issues.
The bigger your model, the more compute costs it will take, both to train it and to run inference.
Finally, is the training, which is how I stitch all of this together, the data and the architecture with compute.
And you can break training down into a couple of different steps.
There's the pre training.
So I say pre-training to be very specific here.
These models, if you remember, are meant to be a starting point.
They're meant to be taken and then retrained in a process that's called tuning and used for different downstream tasks.
So pre-training refers to creating the first foundational model, the starting point,
and the pre-training is going to drive a huge percentage of the compute costs and the carbon footprint of foundation models.
These costs are going to be dictated by your architecture choices, your hardware choices, the inference stack choices,
all of which can be dictated and result in different carbon costs and compute costs.
After the initial pre training, there's a really interesting phase called the alignment phase,
which is when I take my pre-trained model, but it's not ready for prime time yet.
I need to polish it.
I need to align it closer to my values of how I want my model to behave, my values such as safety and trustworthiness.
There is an active area of research along alignments and how to do this effectively and as efficiently as possible.
Some techniques around alignment include things like reinforcement learning with human feedback, R.L.H.F., which is when I have a human actually sit in the loop.
They're evaluating model performances, scoring it, and then creating a reward function.
It's basically a game and you try and get the model to give responses that are going to have the highest scores from the human annotators
there are other methods that are more data driven, so everything in this picture so far has been under unsupervised learning.
There's been no labeled data.
We scraped a bunch of data from the Internet, we've trained it,
but now if we start to bring some supervised learning back in, we bring some labeled data that's specific for our task,
or that is labeled data that shows their domains.
You can do a process called tuning, which is where you actually go through and update some of the parameters
of this pre-trained model and ground it in that labeled data to be more effective.
There's a number of other techniques, things that will happen in the alignment phase, this is an active area of research,
of things like editing post-processing that you can also do and to improve your model's fairness once it's trained.
So now that we've talked about the different components and strategies,
let's talk about what IBM is doing in order to build efficient and trustworthy foundation models
now being made available through watsonx and infused into our product pipelines.
On the data side, IBM is building what we believe to be one of the largest enterprise datasets for foundation model training.
In addition to building a huge amount of data for training, we're taking a heavy emphasis on the quality of that data,
making sure that it's taken from reputable sources, and that every single data point that we collect and curate goes through legal reviews,
ensuring there's no ownership issues, copyright issues,
and then is red teamed in order to identify what potentially toxic information needs to be taken out to make the quality of the data as safe as possible.
And then finally, we're actively targeting different degrees of specialization, things like finance and cybersecurity,
so that we can train these expert models that are going to be more efficient for our enterprise needs.
On the architecture side, we're building a variety of different style models the encoder decoder models, decoder only models and more.
We're coming up also with net new architectures that have never been seen before.
Models that promise to have ultra efficient operations and have modular components that allow expert knowledge
to be infused in them that we think are going to drive a lot of value in the space.
And we're building models of a variety of sizes from smaller 3 billion size models to 20 billion and larger.
And then finally on the training side, what's really exciting here are some of the advancements coming from IBM Research, including a new technique called LiGO,
Learning to Grow, which is a modular training approach, allowing you to recycle models, reuse them, and thereby save significant carbon and compute costs.
When training these models, leading to some of our models being the some of the most sustainably trained foundation models available.
We've also done all of our training on Vela,
the supercomputer built by IBM Research that we've optimized across the stack to ensure that our training and inference is as efficient as possible.
And then IBM Research is working on a number of advanced alignment techniques,
including things like reinforcement learning with human feedback, but also going into advanced tuning approaches,
allowing models to follow strict rules in terms of what type of behaviors they can exhibit.
If you're interested in learning more, please check out the links below
both about the different innovations that are coming out from IBM Research on model safety and about our watsonx product portfolio.