Hello, I’m Andrew McNutt.
Every user interface design involves compromise, and the various available options in charting
are no exception Shelf builders, like Tableau, enable rapid
exploration, but can make constructing specific chart forms difficult
Chart choosers, like in Excel, allow easy creation of specific chart forms but at the
expense of flexibility
Textual programming, via tools like Vega-lite, can offer rich expression languages usually
at the cost of an increase in exploration difficulty
In this work we sought to understand how to combine the benefits of several common charting
modalities, while avoiding their deficits
To investigate this question we built a prototype visual analytics system called Ivy.
The root of the application looks a lot like the Vega-lite example gallery, and (therein
a chart chooser) However, if we click on any of these we’ll
find something instantly usable
In particular, ivy’s editor configured to that chart form
One of the main ways to interact with this chart is via a familiar shelf-builder-style
drag and drop GUI where pills, representing data columns, can be dragged
from left into the shelves in the center.
The configuration of which generates a chart on the right, which, in this case, is a Vega-lite-based
waffle-isotope.
Unlike most shelf-builders though, this GUI is generated by an end-user editable text
representation allowing for customization of the interface to taste and to task. Customizations
which can be saved and shared via Ivy’s gallery.
Each chart in Ivy is a Parameterized Declarative Template. These templates are special functions
which contain parameters, or arguments, and a function body, which, when combined, produce
a visualization
The typed arguments to this special function describe what values can be used, as well
as what GUI elements will be generated to represent them
Here we see several parameters controlling features such as color and size
The template body consists of parameterized json-based-visualization grammars (which currently
can be Vega-Lite as well as several others). Ivy’s language support is extensible, so
additional grammars can be added as needed
Backing this parameterization is a formal language specification, which you can find
out more about in the paper But what it boils down to is a standard JSON
enhanced with variables and conditionals
Let’s look at an example. Here’s a template for a bar chart, with the parameters in their
GUI representation
Variables are inserted into the text through string interpolation based on escape brackets.
Here we see the connection between the GUI element called yDim and the corresponding
code
Next let’s see conditionals. Conditions are simple control flow operators that, if
they evaluate to true, are replaced by their true branch, and if they evaluate to false,
are replaced with their false branch, That is, if there is one, or as here, the
containing key will be deleted.
To make it actually do something we give specific values to the parameters
by setting yDim to age and xDim to people
The conditionals are evaluated and variable values replaced as appropriate to generate
this Vega-lite spec
Which, finally, can be rendered, producing this chart
Just to briefly recap: templates are special functions that enable reusable charts
by enriching json-based declarative visualization grammars with variables and conditionals
While variables and conditionals might not seem like a lot, they can do a surprising
amount. To demonstrate this we took all of the examples in the Vega-Lite gallery and
transformed them into templates
When we conducted this exercise there were 166 unique examples.
Which we factored into 43 templates Yielding a 3.5x compression ratio, indicating
that, in addition to their other usability gains,
templates can act as self documentation for collections charts and improve a grammar’s
learnability
To further test the expressivity of our approach we also reproduced the shelf-builder polestar,
which is a simplified version of Tableau. We are able to successfully reproduce it as
a single Ivy template
In this work we examined the consequences of taking a meta-language approach to visual
analytics, which offers a number of useful features.
It is grammar agnostic, allowing use of any JSON-based grammar as may be needed by task
Exposing an underlying text representation enables opportunistic programming flows, wherein
users can leverage examples found online It is at least as good many alternatives,
as it can literally reproduce some of them and we’ve argued that we get the affordances
of each of the wrapped modalities while avoiding their shortcomings
This approach also has some limitations. It’s meta nature prevents contextual suggestions
based on the knowledge of the underlying grammar; which is a staple of modern “smart” visualization
systems, such as Lyra2. However, we believe this is addressable via improvements to our
grammar API In an approachability study (detailed in the
paper) users liked and were able to use the system effectively, but had a non-trivial
learning curve. We believe that this is not intrinsic to this form of multimodality, and
can be addressed though design refinements.
The system we described combines chart choosing, shelf building, and textual programming
However, this is not the first work to combine multiple modalities in visual analytics
In future work we’d like to see systems that combine not just visualization modalities
but also analytic contexts, such as spreadsheets and notebooks, as well as techniques like
programming by demonstration We are optimistic that the multi-modal and
meta-layer approach found in Ivy and the lessons learned therein will be a useful starting
point for future work in this direction.
Thank you for your time and attention. You can find out more about this work at this
address.