Hello! My name is Avinash Bhat, and I am a graduate student at McGill University, Canada.
I’m presenting our work on investigating the effectiveness of the current practices
in documenting ML models and introducing a potential tool to support documentation during
model development. A large number of pre-trained machine learning
models are hosted and shared publicly using GitHub or model hosting sites like HuggingFace.
However, most of these models are either poorly documented or not documented at all, resulting
in unanswered questions about their intended use cases, performance during adoption and
ethical and social impacts. Here’s an example of one such model that
has been downloaded about 8 million times last month from HuggingFace but only contains
a brief description, citation information and license.
These models are reused in various software systems, often leading to serious misuse-related
issues. To counter the issues, there have been notable
efforts toward documenting ML models among which, the proposal of Model Cards is the
most influential. Model Cards are short documents that accompany
the trained and hosted models and provide details about the context in which models
are intended to be used and other relevant information.
The model card proposal suggests nine sections when documenting ML models, including information
on the intended use and ethical consideration, along with technical aspects like the data
used for training, evaluation, and measurement of performance.
They are essential for promoting transparency and accountability in developing and deploying
machine learning models and ensuring that the models are used responsibly and ethically.
So far, Model Cards are increasingly being adopted by different parties. For example,
HuggingFace, the most popular ML model hosting site, has made model cards the default way
to access information about the hosted model. It further provides training materials and
web tools to support the model developers in writing documentation in the form of model
cards. However, it is unknown how the model cards proposal and corresponding efforts in
promoting its adoption have impacted the ML model documentation process in practice. To
reliably evaluate the conformation towards model cards in the wild, we converted each
aspect mentioned in the original model cards proposal to a yes/no question in our evaluation
rubric. We curated a dataset of 132 model cards from HuggingFace, GitHub and company
websites. Then, we used the rubric to analyze these model cards to investigate if publicly
available model cards contain different aspects of information recommended by the model card
proposal. Our analysis indicates that most of the model
documentation has an unbalanced coverage of concepts that are outlined in the model card
proposal. Only the information about the model type,
date and version, intended uses and target distribution are included in more than half
of the model documentation. Across the board, details about when the models
are NOT supposed to be used are only documented in 32% of the model cards.
Only 35% of the documentation includes a discussion on ethics.
Any discussion about the ethical issue mitigation or measurement process are included in only
10% of the documentation, which is alarmingly less.
Overall, we see two trends. most of the model documentation has an unbalanced coverage of
concepts outlined in the model card proposal. We also observe that the model cards provided
by the companies that trained the model and those in GitHub repositories tend to include
more information as outlined by the model card proposal when compared to model cards
on HuggingFace. To address the big gap observed between the
original model cards proposal and the model cards in the wild we design a tool called
DocML. ML Model development involves different stakeholders
such as domain experts, data scientists and software developers.
As a starting point, we focus on data scientists considering their critical role in model development.
DocML nudges the data scientists to consider and comply with the model cards proposal and
to support continuous assessment and management of the documentation quality during the model
development and not just towards the end. DocML is an extension to JupyterLab, the notebook
environment most data scientists use. When activated, the documentation panel opens alongside
the Jupyter notebook. The panel is interactive and lets the users
edit the suggested sections and their titles, input new sections and customize the order
with a configuration file. We maintain the model card content within the notebook to
avoid inconsistent and outdated information. To nudge the data scientists to follow the
proposal, we implemented three interactions. First, when the users hover the cursor on
the title, we show a concise description of the expected content within the section. Second,
explicit links to one or more high-quality model card examples are shown next to the
title. Third, when the users are exporting the model card for sharing, the tool will
perform a lightweight check and inform if any pre-defined sections are still empty,
to encourage the developers to complete any missing sections. To support cross-referencing
between model card and source code, we adopt the concept of traceability and let the users
link code cells to the interactive model card panel. We identify the data science stages
such as data cleaning, preprocessing, model training and evaluation for common libraries
like scikit-learn, numpy, pandas and matplotlib by creating a knowledge base. The knowledge
base is used to automatically inject trace links in the form of code comments which forms
a link between the panel and notebook. In case of any inconsistencies, the user can
add specific markdown tags and customize the links.
We designed a between-subject controlled experiment in a lab setting to evaluate the extension
with 16 participants. Participants were given notebooks with low-quality model cards and
were asked to develop and modify the documentation. We observed that the participants with access
to DocML considered the scope of model documentation more broadly and added more information about
the problem context and ethical considerations, which is often overlooked in current public
model documentation. When asking feedback on the existing features
of DocML participants mostly agreed or strongly agreed that those features are necessary.
Participants generally commented on the importance of having the model card template to structure
the documentation during the development. Participants also thought that the navigation
functions were effective and suggested having automated solutions to achieve similar functions.
The participants in the control group who did not use DocML expressed their frustration
when developing and maintaining the documentation for models. They hoped to have tooling support
that is available to software developers in the form of JavaDoc or Sphinx. They voiced
the need for similar features that are currently offered by DocML, such as documentation templates,
automated documentation extraction from notebook and code-documentation trace links.
In summary, in our investigation on the adoption of model cards, we discover that the
model cards are rare and often shallow, with mostly low quality and selective information.
Based on the insight that traceability and nudging towards good practises within an IDE,
can lead to better documentation, we build a documentation tool called DocML. In the
lab study, we observed that data scientists using DocML showed a deliberate consideration
of model development context and ethical concerns. Our work highlights the opportunities and
the requirement for better tooling in the space of model documentation.
Thank you!