An "open data lakehouse",
you've heard the term,
but what does it mean for modern data management?
Let's start with how we got here.
There are key challenges that have led to the open data lakehouse concept
including how the amount, types, and cost of managing data has exploded,
how data needs to be managed differently across environments,
and how consumption of data has changed with new use cases driven by AI.
And when we observe how enterprises are dealing with the explosion, management, and consumption of data,
we find most leverage multiple siloed and monolithic data warehouses and data lakes,
on-prem and in the cloud.
Warehouses typically offer high performance for processing terabytes of structured data
for business intelligence use cases
but can quickly become unsuitable and expensive
for data engineering, machine learning, and evolving AI workloads.
Price to performance is less than optimal by running these types of workloads in a data warehouse.
Data lakes offer lower cost solutions for big, unstructured data,
but can often become data swamps
and challenged with extracting insights in a highly performant and simplified way.
These data management challenges inform 3 critical needs for enterprises to scale AI:
First, is to modernize ineffective data architectures and reduce data warehousing costs
Second, is to seamlessly access and unify data across disparate hybrid cloud sources
and third, is to accelerate time to insight with built-in governance and automation.
The solution is a data lakehouse that helps you optimize for both cost and performance
by leveraging cost-effective repositories
combined with multiple fit-for-purpose engines
to power various analytics and AI use cases.
Watsonx.data gets to decisions you can trust in minutes,
for up to 50% less than the cost of a data warehouse.
So, how does it work?
Let s take a look at how watsonx.data is architected.
Watsonx.data starts with your data, wherever it resides.
It provides the foundation to connect to your existing data lakes and databases
and combine them with new data to develop and unlock new use cases and new insights.
Open data formats such as Parquet and Avro are all supported.
We then introduce a consistent and shared metadata layer
which organizes data in the Iceberg open table format.
This allows different engines to access and power all analytic workloads
while sharing a single copy of data.
What does this mean?
We eliminate data silos and duplication of data,
while reducing cost and governance risks.
The last layer brings multiple query engines.
This layer is at the heart of IBM's open data lakehouse.
Watsonx.data brings together the best of IBM
and the best of open source.
It is built on IBM s enhanced Presto engine
and leverages Spark with customized libraries
for fast, reliable, and efficient processing at scale.
You can also register existing data warehouse engines,
such as IBM Db2, IBM Netezza Performance Server,
or another data warehouse that supports Iceberg, to watsonx.data
What could make it better?
Watsonx.data can be deployed as containerized software
or as a fully managed service in multiple public clouds.
With flexible deployment options,
you can access all of your data and maximize workload coverage
across all hybrid-cloud environments.
Leveraging our multi-engine architecture, organizations now have the opportunity to
reduce the cost of their data warehouse by up to 50% through workload optimization.
Non-optimal, costly warehouse workloads can be shifted to lower cost storage,
compute, and fit-for-purpose query engines
that dynamically scale up and down.
Watsonx.data also helps organizations get value out of ineffective data lakes
by leveraging a new generation of SQL engines
designed to bring warehouse-like performance and functionality over big data.
The simplicity of watsonx.data allows users connect to any storage and analytics environment
in minutes and start generating trusted insights.
Finally, watsonx.data complements your existing data and AI investments
by integrating into your organizations' data fabric architecture.
The use of industry standards means
IBM can interoperate seamlessly across multiple data ecosystems.
Our solution integrates with both IBM and 3rd party data science,
business intelligence, and data integration tools.
For example,
watsonx.data integrates with IBM s governance capabilities
to enable responsible, transparent and explainable data and AI workflows across the enterprise.
IBM watsonx.data is the foundation
to help your organization scale analytics and accelerate the adoption of AI.
Thank you.
If you liked this video and want to see more like it,
please like & subscribe!
If you have any questions, please drop them in the comments below.