>> On this special build edition of the AI Show,
join us as we hear from Joy Qiao, AI Architect,
and Mario Inchiosa, Principal Software Engineer at Microsoft
plus the Data Science Team at Ernst & Young, or EY.
Joy and Mario will chat with EY and discuss
how they're using Fairlearn in practice.
It's fascinating, make sure you take a look.
[MUSIC]
>> Hi, my name is Mario Inchiosa,
a Data Scientist here at Microsoft.
>> Hi, my name is Joy Qiao.
I'm an AI Solution Architect here at Microsoft.
We're members of the Azure AI
Platform Customer Engineering Team where we
help our customers and partners to
build AI products and services on Azure.
>> Today we're joined by Microsoft partner, EY,
to talk about how EY is using
Microsoft's machine-learning fairness toolkit, Fairlearn,
to assess the fairness of
their credit adjudication model and mitigate any fairness issues.
Fairlearn is an open source Python package and GitHub repo
created by Microsoft Research and the Azure AI Platform Team.
You can learn more about Fairlearn
by watching [inaudible] on-demand
build video entitled How to Test
Models for Fairness with Fairlearn Deep Dive.
Now we would like to welcome our EY partners to the conversation.
Mario, Jason, William, and Alex,
would you please introduce yourselves?
>> Hello. My name is Mario Schlener.
I'm a Toronto-based partner for EY and I lead
our risk management practice for EY in Canada nationally.
>> Hello, my name is Jason Wong working in Mario's team.
I'm currently in EY field,
AI use and machine use cases including model development,
validation and a model risk governance.
>> Hello, my name is William Chan.
I'm a Senior in Mario's team.
I work extensively in
financial risk modeling and model validation,
as well as advanced analytics machine learning modeling.
>> Hi, my name is Alex Miles.
I'm partner based out of Toronto and I am responsible for data,
AI, and analytics nationally in Canada.
>> Thank you. It's great to have you-all here.
To start off, Mario,
could you please tell us about EY trusted AI platform?
>> Our trusted AI platform has three key components that
we would like to cover as a solution offering for the industries.
One is part of the development cycle,
part of the audit and the assessment of
the AI algorithms that I'm using,
and the third component which is
the overall AI governance component.
All of these three components are part of
our AI trusted life cycle and we currently as EY,
we have invested in creating
a platform which provides an end-to-end solution.
>> Thanks Mario.
Now I'll turn it over to my teammate Joy
for the rest of the questions.
>> Thank you for that introduction.
Now can you describe
the specific use case you had
and where the considerations of fairness came in?
>> Sure. I'll take this one.
So through the collaboration with Microsoft Research Team,
their use cases was slacked
for model fairness detection and elimination
is the retail clients adjudication model
for a large Canadian bank.
This is classification model.
So the goal is actually we need to
differentiate good con and bad con,
depending on the likelihood of default.
So this is the fundamental model
utilized for the bank on-boarding adjudication strategies.
So they need a slack the good client to extend loan or mortgage.
So for model performance perspective, yes,
we will leverage lots of
advanced machine learning techniques
to improve the model performs,
such as AUC, as foreign score.
In the meanwhile, Fairlearn is also very important not only
for the assay consideration
and also for the relative hurry consideration,
and also banks reputational risk.
So in particular for this use cases,
we leverage the Microsoft Fairlearn tools to how to define
our fairness principles and how
to detect the unfairness issues in the model department process.
Also in the end,
we come up with the right approach to remediate
the unfairness issues throughout
model development and also post model developments,
the whole journey of the development process.
>> Thank you, Jason. That makes a lot of sense.
Now, could you walk us through how you used
Fairlearn to assess and mitigate the observed unfairness.
>> Absolutely. I will take this question.
So throughout modeling process,
UI and Microsoft research team have closely
collaborated to enhance the user experience
of the Fairlearn package.
So first step of solving
the problem of fairness is actually to know the problem.
So facilitated by dashboard function of the fair and package,
we were able to easily evaluate and visualize
the fairness of model output with regard to the protect feature.
In this case is the gender in terms
of demography parity and equalize the odds.
So knowing that fairness is indeed an issue with the model,
the question now becomes how we want to remediate this problem?
So Fairlearn package provides the flexibility of remediating
the unfairness both in training and post training.
The user-friendly Fairlearn API allows users to quickly create
a multiple remediated models and
visualize the trade-off between fairness and the model accuracy,
hence allowing a better informed business decision-making.
>> That's great. Thank you, William.
So now what do you see as the benefits of
using Fairlearn in this particular use case?
Also do you think it will help UI overall,
trusted AI platform and solution capabilities by
incorporating Fairlearn as part
of the amount of validation process?
>> Absolutely. I mean,
the package as it currently is set up,
it provides a readily available tool
with a variety of technical approaches.
They came in selected based on the use case at hand and
the unfair outcomes observed in the available data that we see.
This was actually one of the major focus
for the use case described by Jason and William
to be able to identify and visualize
easily the outcomes and accordingly to the outcomes,
we were able to interact with the pre-processed data or
post-processed data in a way that we can actually
remediate the unfairness in the overall model estimation.
>> Thank you, Mario. So as we're integrating
a Fairlearn as part of
our overall Azure Machine Learning Services.
How do you envision moving forward with this offering?
How will the toolkit be
incorporated in EY's machine learning life cycle?
>> Alex, I'll take that question.
So we see that first of all,
being able to explain models in terms of if they're
fair or biased or not
biased and explainability in terms of transparency,
here's the key differentiator in what we do.
So in terms of our service offering,
we see that incorporating Fairlearn is
a call it business as
usual aspect of how we will do modeling for any of our clients,
whether that is in
a regulatory environment or a non-regulatory environment.
We see generally speaking,
that while there is a lot of interests in
many open-source libraries and
the adoption of Python and the machine learning space,
we do see a tremendous gap around the machine learning life cycle,
the operations, the so-called machine learning DevOps,
the model management as well as tying
in all these aspects of fairness and bias detection.
We see that Microsoft is uniquely positioned to lead in this area.
Our plans for the future is to adopt this in leverages as part of
our regular model life cycle development
that we perform for our clients.
>> Thank you so much. That's been very insightful.
Thank you everyone for joining our conversation today.
Really appreciate the close collaboration
with EY for responsible AI.
>> Thank you.
>> Very welcome.
>> Thank you.
[MUSIC]