Businesses are increasingly adopting and deploying AI
to automate businesses and IT processes,
gain new insights through data analysis
and better engage with and serve customers.
Yet as AI adoption takes off across industries,
the biggest obstacle to widespread deployment is trust.
Trust, along with language and automation,
are the critical ingredients needed to scale AI for business.
First, it's important to consider how AI is built.
AI is managed across a lifecycle—
a sequence of steps from preparing and building models
to deploying and managing them out in the world.
These steps must all be carefully monitored,
and guardrails must be put in place.
At IBM, we call this process AI Governance,
which is designed to help businesses create policies,
assign decision rights
and ensure accountability.
With proper governance of AI,
we can prevent undesirable outcomes—
including models unintentionally harboring bias,
being trained with improper or unapproved information,
or even having unexpected shifts in performance.
One major challenge to date
is that the monitoring and documentation
of the AI model lifecycle is performed manually.
This requires substantial expertise
and can increase the risk of incorrect information.
Enter IBM Watson's AI FactSheets.
Born out of IBM Research
and built for a hybrid cloud world,
AI FactSheets will automatically capture
key information on a model's performance
and automatically create reports
to support transparency and compliance.
Drilling down deeper,
AI FactSheets are customizable for varying audiences—
external and internal.
and reporting views can be customized—
providing people like the business owner,
data scientist,
model validator
and model operator
unique reports with insights tailored to their specific needs.
Like nutrition labels for foods,
FactSheets would provide
important information about AI models,
such as its purpose,
performance,
data sets and more—
model facts that are key
to building consumer and enterprise trust
in AI services across the industry.
Creating a template allows organizations
to define the information collected on an AI model,
such as how the model was created,
tested, trained, deployed and evaluated.
It can also standardize what data can and can't be used.
What regulations—such as GDPR—
or company policies need to be accounted for,
and many other factors.
Automated Data Capture: Model Facts.
To date, documenting an AI model's performance
has required extensive amounts of time and resources,
often leading to quickly outdated and irrelevant reports.
Soon, organizations will be able to continuously
and automatically capture metadata—model facts—
across the entirety of the AI lifecycle.
Automated Reporting: FactSheet.
With automated data capture,
the FactSheet will provide real-time reporting
on the performance of the model,
along with custom metrics.
The FactSheet can be tailored to the needs
and preferences of different users and audiences,
enabling collaboration across varying levels
of technical expertise that otherwise wouldn't be possible.
As with all IBM Watson technology,
AI FactSheets will be open,
built to run anywhere,
and can help organizations manage risks
across any hybrid cloud environment.