I am Andrew Head. I am presenting our paper, "Math Augmentation:
How Authors Enhance the Readability of Formulas using Novel Visual Design Practices."
Math notation is a pervasive language for describing mathematical ideas.
It is used across domains to express mathematical ideas formally,
from machine learning to data science, computer graphics, and human-computer interaction.
However, notation can be difficult to understand. Often, the meaning of a formula cannot be
understood from the notation alone. Consider this formula for a line. While this formula is familiar
to many of us with a background in algebra, for someone looking at it with fresh eyes,
it is not clear how this combination of cryptic symbols describes a line.
Could formulas like this formula be presented in a way that makes their meanings more transparent?
Perhaps they could. One approach is to add labels that explain the meanings of symbols.
For instance, labels could indicate that the formula’s purpose is to compute the “height”
of a point by taking the point’s "horizontal position,” multiplying it by something we
call the "slope," or a fixed multiplicative factor, and then adding a constant offset.
Another approach involves relating the formula to another representation of the same mathematical
idea using color. For instance, the formula could be colorized to relate it to this visualization,
conveying through color that the height of a point can be computed by taking a
constant vertical offset, then taking the horizontal position of the point
and scaling it to another vertical offset using “m” as a multiplier.
Both coloring and labeling are instances of what we call math augmentation,
the practice of embellishing math notation with visual design patterns to improve its readability.
The goal of this paper is to understand math augmentation,
and how future authoring tools could help authors create augmented notation.
Towards this goal, this paper addresses two research questions:
How do authors augment notation to improve its readability? And how can authoring tools
help authors augment notation in the ways that they wish to, more easily?
To answer these questions, we conducted two qualitative studies. First, to elicit
a design space of augmentations, we conducted a content analysis. 2 coders analyzed 281 formulas
from 47 mathematical documents. They analyzed dozens of dimensions describing various aspects
of each formula’s design. Second, to understand opportunities for future tools to help authors,
we interviewed 12 authors of mathematical documents containing augmented notation.
Authors were asked to describe recent experiences augmenting notation,
and to reflect on their motivations, processes, and the challenges they encountered.
Our first finding was a design space characterizing how notation
is augmented by authors. At a high-level, formulas were augmented in one of 4 ways.
They were embellished with embedded visualizations, like geometric drawings,
in place of expressions. Their text was styled with color, space, and font adjustments. They were
annotated with overlay and underlay marks, like labels. And they were imbued with interactivity.
Across these 4 categories of augmentation, there were 16 distinct kinds of augmentations.
The design space of some of the augmentations could be rather complex.
For instance, consider labels. In the documents we analyzed, labels could contain text,
visuals, and additional notation. Furthermore, sometimes labels applied
to overlapping expressions, explaining the same notation at multiple levels of
detail. Or they applied to other labels, rather than just mathematical expressions.
Nuanced design alternatives like these were present for many of the kinds of augmentations,
and we describe them at length in the paper.
Our second finding was a set of 11 design recommendations to motivate the design of future
markup languages, graphical editors, and automated tools to support the augmentation of notation.
These design recommendations were informed by observations from both studies.
Here, we describe a few of the most evocative recommendations.
First, tools should help authors color formulas and the content those formulas relate to.
Documents frequently made use of color to highlight expressions in formulas
and relate them to representations of mathematical ideas elsewhere in the document.
For instance, in this passage, the function "f(x)" is assigned a green color in the formula,
and the curve representing it in a nearby visualization is given the same color.
Authors often used color to relate expressions not just to visualizations,
but also passages of prose, code listings, and expressions in other formulas.
Second, tools should help authors enliven formulas with details-on-demand and animations. Authors
felt limited by the static media that they wrote in, believing they could better explain notation
with dynamic presentations. Authors proposed many visions of dynamic presentations of notation.
One vision was to expose explanations of expressions at multiple levels of resolution.
For instance, a formula could be augmented to initially show labels for just a few of the most
important high-level expressions. And then when a reader wanted to understand more, they could
click specific expressions that they wanted to understand to access finer-grained explanations.
Another vision was to show mathematical derivations as animations. For instance,
consider this derivation, shown in a conventional format—as a series of formulas, each one derived
from the last, with textual descriptions of how each formula was derived.
This one step of the derivation
could be shown by highlighting the position of gamma in the first formula, and then revealing
how it is distributed in a multiplication by sliding it into two new positions.
A third recommendation is to make markup languages for augmenting notation
easy to read and write. Markup code for augmented formulas could be messy.
Consider this relatively simple formula, consisting of 8 symbols and 4 colors. The messy
markup for this formula mixes together a small amount of code describing the underlying formula…
and quite a bit of code specifying the colors. This markup is difficult to read and modify.
A better design for a markup language might instead separate
the content markup that describes the formula from a presentation markup that describes the colors.
Combining the findings from the two studies, our paper makes two contributions. First, a design
space of augmentations that authors can apply to notation. And second, a detailed account of how
authors augment notation, and of the challenges that future tools can help authors overcome.
Because a talk of this length can convey only so much of the richness found in the studies,
we encourage you to learn more by reading the paper.
There, you can find details about all 16 kinds of augmentations,
11 design recommendations, and access the complete analysis results for each of the 281 formulas.