[Music]
Patterns and relationships and vast amounts of data unlock entirely new possibilities.
Sometimes we learn about our past, or we discover something that helps us predict the future.
For some time, we’ve been collecting data without even knowing what might come of it.
And then the volume just becomes overwhelming.
And that’s when the relationship between data and AI gets really interesting.
Hello and welcome to AI Academy.
My name is Luv Aggarwal and I’m the Worldwide Sales Leader for IBM watsonx.
And my partner today is Edward Calvesbert, Vice President of Product Management for the IBM watsonx platform.
It’s great to reconnect with you here today in this amazing IBM research facility.
I know. This place is incredible.
This is an active working lab; and we’re in front of a prototype system designed for AI.
You and I both talk to a lot of clients and it seems like every conversation these days is about Generative AI or Gen AI.
Well, every conversation starts with AI, but it usually ends with data.
Because the truth is, there is no AI without data.
And data is the only sustainable source of competitive advantage in business.
Generative AI is changing how we think about data in a couple of ways.
First, Gen AI models can make much more effective use of unstructured data, which is the majority of all new data.
Gen AI can dive into large volumes of language data, primarily documents or software code, which is also a language,
...and spot patterns or make connections without much preparation or supervision.
It can see things that we likely wouldn’t see.
Yes.
Second, we can apply Gen AI to the data management problem itself.
And it can help us organize, refine and enrich data so that it’s higher quality and easier to consume.
Right.
So for example, if a client has different legacy applications that have been encoding data in different ways,
...like different formats or dates or people’s names and initials,
...or different columns and column headings and that sort of thing, Gen AI can help make sense of that.
And, the data is often scattered everywhere.
Gen AI can look at those siloed systems and begin to understand what the data is and how it could be related,
...so you can take advantage of all of that data holistically.
And, much more efficiently. Big savings in time and energy.
How effectively and efficiently you manage your data has real cost and business performance implications.
It becomes intellectual property and a point of competitive advantage.
So let’s talk more about Gen AI and data as a source of competitive advantage.
High-quality data is essential to helping enterprises use Gen AI to improve their business.
Almost every one of our customers is at least experimenting with Gen AI today.
Which means that everyone has access to essentially the same technology.
So it’s the customization or tuning of the large language models with enterprise data,
...and the infusion of Gen AI into new and existing enterprise applications,
...that drives productivity gains, improve business performance and competitive advantage.
So every company has data, but there’s really a spectrum of how different organizations take advantage of that data.
Some might be stuck with an architectural problem,
...like data being inaccessible or locked in silos across on-prem and cloud environments.
The data silos problem is pervasive.
And you can’t solve it by creating a new data silo in the cloud.
So we have different approaches to solving it,
...such as building a virtual data layer and querying data across multiple sources,
...or consolidating data onto one platform like a data lakehouse, which is open and cost effective.
And for other companies, what’s holding them back from unlocking the full value of their data is something more subtle.
It’s almost a psychological barrier.
They have to shift how they think about data and turn the business into data,
...so that they can then turn their data into a business.
Data monetization is nirvana when you can literally sell a version of your data,
...effectively a byproduct of your core business as a product itself.
To create those products, you need to ensure data quality, security and governance,
...so that it doesn’t become a business risk or regulatory exposure.
So let’s talk about those.
First, you need just good traditional data-quality practices.
Do the things you already know you should be doing.
You need to catalog or organization your existing data into a business glossary.
Now AI can assist with that work, but the more you have ready, the faster you can get value from AI.
And the financial incentives to do so are just going up and up.
Second, is having thoughtful data access policies.
Like with any user, you really need to define what information you want AI to be able to access.
Or said differently, which information you want to remove or redact in some way,
...like social security numbers or other personal identifiable information.
And the third aspect regarding governance is monitoring and enforcement.
So you don’t just set policies and call it a day. Right?
Ideally you set policies centrally and enforce them locally,
...while actively monitoring model inputs and outputs to ensure that the policies are effective;
...including how the output of the model is changing or drifting as it’s exposed to real-world interactions.
I don’t think we could empathize that enough.
Data and AI governance is absolutely critical.
Risk and uncertainty about data and model outputs are two of the biggest barriers to the adoption of AI today.
Everybody wants AI, but companies can’t risk having their own data or their clients’ data exposed.
I think it’s clear that quality-trusted data is essential to successfully implementing Gen AI in business.
But, a lot of our clients are still struggling with moving beyond a handful of single prototypes,
...to the next phase of customizing the models with their own data,
...and deploying them into production across the enterprise.
Okay, so let’s talk about how a company can customize Gen AI with their data,
...and integrate it with their enterprise applications and workflows.
There are two main ways to customize Gen AI with your own data, to make it work for you.
The first is by tuning the model with your data.
And the second is through retrieval-augmented generation, or as it's commonly referred to by its acronym, RAG.
Tuning a model involves instructing it or partially retraining it,
...with good examples from your enterprise data of how it should respond to certain prompts.
The model quickly learns from these examples and adapts to incorporate the language and structure of your business,
...so that it can become an integral part of your enterprise systems.
On the other hand, RAG doesn’t customize the model,
...but rather leverages a knowledge base, again, of your quality-enterprise data,
...to improve the accuracy and even limit the responses of the model to known facts.
Thereby, mitigating the new risk of hallucinations.
What about the data that’s used to train the models themselves?
Well, that’s a great question, considering we’re here at IBM Research where we train our foundation models.
As an example, our training system is built on a data lakehouse architecture.
And it breaks down the walls between data, AI and governance.
First, we source, catalog, filter and transform the data that is going to be used in the models.
Next, we use it to train, test and tune our large language models.
And finally, we govern that end-to-end lifecycle.
So we have the complete lineage of our datasets, data pipelines and AI pipelines,
...and we’re able to stand behind our models.
And that’s the sort of comprehensive approach that enterprises are going to want to build as a core capability.
So another way to think about a data lakehouse is like a commercial kitchen.
And I’ve used this analogy before, but if you think about a business like a restaurant,
...the ingredients for everything you make is the data.
Now it would be silly to send one truck to get carrots from a farm in Connecticut,
...another truck to get peas from a grower in California, and a third truck to get beets from Minnesota.
And then wait around while they bring it all back to the kitchen.
What are you making man?
I don’t know, I’m not a chef, but it’s proprietary.
The point is that this is how many businesses approach their data architecture today,
...when what you really need is a well-stocked pantry,
...where everything is already on hand and neatly organized in one place and labeled to be quickly accessible.
So ultimately, I believe the open data lakehouse architecture is the best way to achieve that well-stocked pantry.
It combines the flexibility, scalability and cost advantages of data lakes,
...with the performance and functionality of data warehouses.
So it’s the best of both worlds.
And as an industry, we’re really moving past the old monolithic architectures,
...to these more flexible, open and interoperable architectures,
...that let you choose the right tool, for the right job, at the right price.
So for example, we can use one query engine for data ingest in transformation,
...another for interactive queries, and yet another for embedding documents as vectors for RAG.
That sounds ideal.
And this flexibility also enables optimal price performance,
...which is an essential enterprise consideration when deploying this technology at scale.
I’m actually a huge fan of all the experimentation and innovation we are seeing with Gen AI in the market,
...in open source and across the ecosystem.
But in addition to choice of tooling and cost considerations,
...there is another important lesson to be learned from our experience over the last decade or so,
...scaling the adoption of AI.
A lot of good machine learning models never got deployed into production,
...because they were built by data scientists working on the sidelines of core enterprise IT.
And could not be properly assessed for risks, including regulatory compliance.
Data and AI governance is very hard to do after the fact,
...because so much of it involves managing and tracking the end-to-end lifecycle.
So, in order for all of that experimentation to some day make its way officially into production,
...companies need to implement data and AI governance frameworks, or platforms,
...from the very beginning, and not as an afterthought.
Right and that all makes sense Edward, but I see you sneaked in your beloved open source.
So what role do you think open source plays in the evolution of this technology and the market?
I think it plays a huge role of.
Simply put, open source technology and community-based innovation,
...delivers superior transparency, security and project oversight,
...which I think is critical at this early stage of maturity.
The transparency and security points are straightforward.
With open source technology you have more users and developers finding and fixing bugs and vulnerabilities.
But the point on project oversight is more subtle,
...and it’s about having a broad and diverse community of both vendors and users of the technology,
...driving its long-term roadmap.
Which ensures that it’s aligned with the interests of the broader community and not just those of a particular vendor.
The productivity gains of Gen AI are so massive that how and how rapidly enterprises implement it using their own data,
...could make the difference between the winners and losers in almost every industry.
Being well prepared is great,
...but the best way to start building a new capability is to just do it, with the proper guardrails, of course.
Absolutely.
Well, thank you Edward.
And for everyone else, thank you for watching.
Keep an eye on this space for more episodes of AI Academy,
...with expert perspectives and real talk about some of the most important issues in AI for business.
[Music]