hi everyone my name is ethan and i'm a
phd student at uc berkeley today i'll be
talking to you about b2 bridging code
and interactive visualizations in
computational notebooks
this work is done with joe at berkeley
and arvind mit
many data scientists use the notebook to
explore data
they could manipulate the data with code
they could visualize the data
and they could write down their thoughts
professor press
calls this literal computing after
canoes litter
programming where the the analyst is
able
to weave the narrative along with a live
computation environment
however this experience could become a
little lacking as i
explore the data further so here i have
analysis of wildfires in the united
states
and i have created a visualization of
the distribution of causes of fires
and as a next step i visualize the
distribution of fires by the state
and i notice that georgia has a
surprisingly high count
and i want to filter
the cause distribution by georgia
currently the only way i can do this is
to write out
code that says fires
state equals equals to georgia and i
have to
either create a new visualization or to
replace the old code and execute the
cell
this could be a little bit slow
especially compared to another commonly
used data analytics tool
interactive data visualizations this
work is an exploration to bridge
together the two mediums and it's not as
simple as
just programming a interactive
visualization into a notebook cell
we identify three gaps the first is a
layout gap we can see here that
the notebook is linear where
visualizations and code are interleaved
whereas in interactive visualizations
the visualization are a lot more compact
this is really important for coordinated
views if the
visualizations don't even fit into one
screen
then how it how is it possible for the
user to be able
to interact with visualizations second
is a semantic gap
if we look at the notebook to the left
we have already done a lot of work in
terms of specifying the static
visualizations
via query and visual encodings however
to create the interactive visualization
creates
a lot more work and
if we just look at the queries it's
actually not that hard for us to infer
what the notebook user may want when
they click on
ga and how they want to filter the
cost distribution they want to see all
the causes in ga and so there's a
missing opportunity
here where the interactive
visualizations need to be re-specified
despite the fact that the provenance
information has been provided in the
queries
another issue we see is that the
interactions are siloed
in the visual format and inaccessible
for further analysis in code
so here as the user painstakingly
specifies the interactions
and interacts with the visualization the
data and the code that derives these
filter results
are all siloed and inaccessible in code
the third and the last gap is a temporal
gap a notebook
is persistent all cell executions are
recorded by default and this is great
for reproducibility
however on the other hand interactions
are by nature transient
and the plus is that we can explore a
lot of different
options without creating clutter to
bridge these two mediums we really have
to create a mechanism to accommodate for
both modalities
so that the notebook user could make the
best use of both worlds
we believe that we made a good attempt
at bridging these gaps in b2 and i'm
going to show you a demo
to start the demo i'm going to import b2
from the library
and create an instance once b2 is
created it augments the traditional
notebook view with
a dashboard like area and some
additional buttons
so we're going to get started by looking
at some
data from wildfires in the united states
taken from kaggle
the data frame returned by b2 is the
same as any data frames you would expect
from libraries
you can inspect it you can apply
operators on them but what's more
interesting is that b2 here is aware
of this data frame and it actually
augments
the the pane to the right with the
columns of this data frame
v2 makes it super easy to look at
distributions of variables
you can just simply click on the value
you're interested in so for instance
here i'm interested in the distribution
of causes and you immediately see
a distribution of these cos
and what's special is that b2 does this
in a transparent manner that's not
locked in to the the ui
side you actually get to see the code
that was used to create this
visualization
which means that i can actually also
create visualization of my own
so for instance here i might be
interested in looking at a scatter plot
of the size and the discovery
so i can go ahead and execute this query
and a scot
scatter plot would show up and we can
make the interesting observation that
the large fires tend to happen around
noon time which makes sense
we can also continue to interact with v2
via the the shelves
and what's even more interesting is that
b2 actually makes these static
visualizations
interactive by automatically analyzing
the queries
so here if i'm interested in
how lightening fires has changed over
the
the years we can just simply click on
lightening and see the values change we
can
observe here that 1994 had a slightly
higher than
average and causes by lightening than
the
the years around it we can also make the
interesting observation that
a large proportion of these large fires
were actually caused by lightning as
well another
interesting thing that b2 does with
these transient interactions
is that it rarefies them into code
format so here
we can see that b2 recorded oh we made
the selection of lightning
we can continue making interactions and
as we make these interactions b2 will
continue to record
the selections we've made but in order
to conserve space it actually
folds them so that it doesn't bloat up
but
when we want to revisit past
interactions we can go ahead and unfold
it
and on comments previous state that we
want to go back to
so for instance right now campfire is
selected but if i uncomment arson and
execute the cell
you'll see that arson is now selected
and if we find the current state of the
visualization
interesting we can actually also take a
snapshot of them
so that they are persisted into
the notebook view that way even when i
hide b2
um the the cells are still persisted
and as we can see also that there are
cells with emoji colors annotated
this is our way of helping the developer
make sense of what cells are generated
by b2 and for what purposes
so the yellow ones here indicate that
they're for
creating visualizations and the blue
ones are the interactions and the orange
ones
are snapshots of
arts another
example i will show you is the ability
to
look at the visualization result from
the code side so
to cross the bridge from the visual to
the code
so i'm very interested in what is
happening with this
scatter plot here of this outlier values
i went ahead and brushed this region
and as we can see not much is showing up
let me get rid of the background data
and so here we see that
most of these big fires are caused by
lightning with some being miscellaneous
and equipment use
if i do fires.get filter data
i can actually just look at this
selection results right here
we can actually even make this
interactive by changing
marking this as reactive so what happens
now is that if we do
brush a different area this cell will
automatically execute
and so we've created us this super handy
little tool this way we're able to bring
the transients of interactivity
into the persistence of the notebook
by connecting the two states via this
api
we can actually do something even more
interesting with this setup
so instead of just seeing the data in a
table format i can
maybe do a visualization that's
difficult to do
right now with b2 in the dashboard so
i'm going to
copy paste some data i've prepared for
plotting the visualizations
um so without reactive
will be able to do a visualization of
all of the
fires in the united states but it might
be handy to
for instance be able to visualize these
heat map based on my
selections and so we can bring back
the reactive as was the get filtered
data
and then if we re-execute the cell we're
able to see that oh
like these big fires i just selected uh
happened in washington and somewhere in
alaska so let us
just remove this chart and
do some other explorations so if i click
on lightening i can see that it's skewed
towards the west coast at least with the
sample data i have here
and arson seems to be more skewed to the
east coast
now when we look at our notebook maybe
after some pauses
some of you might notice there's a lot
of interaction cells we can actually
easily overcome this
clutter by toggling them then the rest
of the notebook seems a lot
simpler however we can always bring them
back if we do need to get a sense of the
provenance it's like what did i explore
how did i get here and share this with
others as a
more complete narrative lastly i want to
show that not only could you access
the current selection by api call you
can actually directly access the code as
well
so here if we want to access the
computation
of the year distribution based on the
debris burning
values we can actually copy code to
clipboard
and if we create a new
cell we're going to be able to paste
that code in
and execute this code and also access
this value
so this way if you want to share the
result with someone without giving them
the whole notebook you can just copy
paste this
query results over hopefully by now your
comments that we've overcome the bulk of
these
gaps and are able to help the analysts
smoothly transition between coding and
interactions
to evaluate how b2 facilitate that
exploration
in the notebook context we conducted a
user study with seven participants
all with some experiences with data
science either through coursework or
their professional jobs
we started with a 20-minute tutorial
followed by two specific tasks and one
open-ended data exploration
on the berkeley police call for service
data
to give you a sense of how the users
engage with v2
here is a visualization of their actions
the x-axis represents time and the
y-axis represents
the different participants each tick
represents an action the user has taken
either on the notebook or on b2 you can
see that the darker blue
indicates user interactions with the
visualizations in the dashboard pane
so here we can see that the users
engaged extensively with these
interactions and they did so in a way
that was
interweaved with coding some users said
that
v2 spares the user of tedious commands
and really facilitates the eda process
others commented that the kind of
interactive visualization facilitated by
b2 is difficult to do with standard
libraries
another pattern that we've noticed is
that
users really engaged with b2 to cross
the bridge so to speak between the
coding land and the interaction land
some of them commented that it's often
the case that their colleagues wanted to
not only see the visualizations but the
underlying data and even
code and b2 made that really easy for
them
and another commented that the ability
for b2 to ratify their interactions into
a persistent form
was super helpful for them to remember
what was going on
if they left their task for a while for
instance for a coffee break or lunch
break
participants also pointed us towards
interesting new directions
one of them suggested that b2 could be
really useful for people who are just
starting out with programming and aren't
as familiar with all the libraries and
tools
of course there's also room for
improvements some of the
users still found the refined
interactions to clutter up their
notebook so
clearly there's still some design space
that's worth exploring
to deal with the trade-off between
reproducibility
and literate programming as well as this
ability to explore without creating
clutter
overall the participants found b2 to be
quite useful
and that concludes a quick tour of b2
thank you for your time and please refer
to the paper for more details