this is some really cool work that i've done with my collaborators at apple
this is our system called mage so let's talk about fluid moves between code and graphical
work in computational notebooks so this is a notebook today i'm going to load in some data
um i'm going to load in my favorite data set which is data set of pokemon
and we're going to look at what this data looks like in a typical notebook
in the level of interactivity that notebooks support today so i see this nice table and
this is really useful for seeing the table in the notebook so i can visualize my data
but you'll notice that the table itself is not interactive i can't do anything with it
now in recent years we've seen some really compelling examples of how we can push notebooks
to be more interactive and more deeply combine complex gui work so see we have for example b2
where we're able to do complex visualization work with interactive visualization dashboards
directly in the notebook alongside our code here we have some interactive tables that behave more
like spreadsheets we'd see in numbers or excel and all of these examples are really compelling
these today require a significant extension of the notebook and what we'd like to see is a more
general way that we can approach these style of interactions where we can fluidly work with our
data in both complex gui tools and in notebooks and this is the goal of this work is to push the
way that we com compute with data in general not just in notebooks but here we'll focus on
notebooks so our work is the mage api in which we've extended how notebooks work in general
to support this class of interaction where we can support a wide variety of complex interactive gui
tools that produce real work in the notebook so they auto generate code as well as changes in a
gui interface let's see what this means so once again i'm going to pull up a table of my pokemon
this time i'm going to use the mage api to summon a table now this table looks a lot like the table
we saw the first time but there's one key difference so as i scroll along looking at my
columns i can actually start to do real work in this table so i'm going to move the name of the
pokemon to the front of this table and when i do that it generates code to ensure that my work is
reproducible and my work can be permanent in the state of the notebook in both code and in this gui
representation so that's what mage is doing it's allowing fluid work back and forth between code
and gui let's show another example so this time i want to summon a plot for my pokemon
dataset i want to be able to look at different visualizations now this is a visualization gui
we've created based on voyager and vega light as i use the different operations i can make these
nice visualizations of for instance all my types of pokemon and it's automatically generating a
vagalite specification now as i select some data in my chart i can actually use it to produce more
code just through drag and drop or open another tool so here i'm gonna sort my psychic pokemon
and i can directly edit the code to change what i'm seeing in the table and my data set or change
it in the data set within the gui tool either way just works so this is what we'd like to see code
and gui alongside each other both ways simply work without any fuss or extra complication
so what we did to produce our api is we created the two tools you just saw the plot tool the
table tool as well as all these other tools um a lot more detail in the paper we created these
tools to test the boundaries of what our api needs to support and we also demonstrate these
tools directly to professional data scientists to get their feedback on this kind of interaction
now how mage works very simplified is that it handles the aspects of the user's programming
environment to provide a tool creator with what information they need to display
appropriate context information in the user's environment now as the user works they can summon
using the summon command the gui tool along with the data that the tool requires
and as they take action within a gui tool media's going to auto-generate code that corresponds to
the work they've done and this code is provided by code templates the tool creator uses
that mage fills in using aspects of the user's programming environment
now there's a lot more details in the paper of how this all breaks down and works so i'm really
hoping if you find this vision compelling that you'll check out our paper and hopefully we'll see
notebook and computations work more like this in the future thank you