Hello!
Thank you for tuning in to watch my talk on techniques for flexible responsive visualization design.
My name is Jane Hoffswell and I am currently a PhD candidate at the University of Washington.
This work was done in collaboration with Wilmot Li and Zhicheng Liu from Adobe Research here in Seattle.
I am also really excited to share that this work received a CHI 2020 Best Paper Award,
so please go take a look at the paper after this talk!
And just so you know what we look like, this is us.
I am going to start this talk with a question for you:
think about the last news article you read, what device did you use?
Did you say your phone?
Because in general, mobile devices are now more common than desktop or laptop computers
for reading news articles.
While the text content of a news article can easily adapt via reflow,
it is non-trivial to create responsive visualizations.
Simply scaling the design is rarely good enough for complex visualization designs as the content
may become too small to read or interact with.
Instead, designs should change to appropriately adapt to the new form factor.
To really understand how people create responsive visualizations,
we surveyed the responsive visualization techniques used in 231 news articles from 12 sources.
We then interviewed five journalists from this corpus about their responsive visualization design practices.
Based on the survey and interviews, we identified four design guidelines and developed a new
system for responsive visualization design.
To demonstrate the utility of this approach, we recreated 7 responsive designs using our
system to emphasize a flexible design workflow.
For the survey, we collected examples from best-of lists for visualization designs and
galleries from individual sources.
Overall, we looked at 231 visualizations found in news articles from 12 sources, including
the New York Times, Reuters Graphics, and FiveThirtyEight.
For each visualization, we labeled the visualization type to ensure that the corpus covers a wide
spread of different designs.
We then used an open coding process to identify the changes between the desktop and mobile
visualization, in both the portrait and landscape orientations for the phone.
Take a look at this visualization from the New York Times article "The Places in the U.S.
Where Disaster Strikes Again and Again."
How many differences can you see?
Maybe pause the recording for a minute and take a look.
This article is also available at the link shown here: https://nyti.ms/38VBIzz
To answer my own question, I would say there are ten types of changes:
(1) The visualization has been resized for the smaller width of the phone.
(2) The title and subtitle have been repositioned to the top of the visualization.
(3) The text showing the disaster cost for the labeled bars has been removed.
(4) The labels have been shortened to remove the redundant word "Hurricane".
(5) The labels have also been repositioned into the chart area rather than the whitespace.
(6) A small tick mark has been added to connect the label to the corresponding bar.
(7) There are fewer gridlines (or tick marks) on the y-axis.
(8) And the remaining gridlines for the y-axis have been extended to fill the whole width on the phone,
(9) which includes repositioning the axis labels.
(10) Finally, the labels for the x-axis have been abbreviated.
As you can see, there are a lot of changes for this visualization.
But making such careful updates to the design can be difficult and time-consuming.
Based on these open coding labels, we labeled each technique based on the type of editing
action used and the visualization element to which it applies.
We then counted the number of techniques that we saw across visualizations in our corpus.
This grid shows the types of techniques used when adapting a desktop design to a mobile
design in the portrait orientation.
We also repeated this process to check for the changes to visualizations in the landscape orientation.
Based on the results of this survey, there are three key takeaways.
In general, we observed that more responsive techniques were used for the portrait orientation
than the landscape orientation of a phone (notice the more full grid on the left).
In other words, visualizations in the landscape orientation were less likely
to exhibit changes from the desktop design.
We also noticed that it was much more common to remove visualization elements from the
desktop version than to add new elements on a mobile version.
Finally, we noticed that most visualizations exhibited rather small shifts in the content
or layout, but a small subset of designs exhibited drastically different layouts on different devices.
The visualizations shown here are a few examples of the types of drastic changes we saw in our survey.
As the next step, we interviewed five journalists with prior experience
designing responsive visualizations for news articles.
In particular, there are three takeaways that I want to talk about now.
One interesting takeaway is that designers often exhibit a desktop first development approach.
As one participant stated: "by virtue of sort of sketching graphics on my laptop or on my
desktop screen, often the first iteration of something works best at those screen widths."
Since the default working environment is a desktop, the designs often reflect that default
rather than the more common expectation that visualizations will be viewed on mobile.
Another participant noted that "much of the programs we use are geared towards desktop
first or feel that way, anyway, so if all of them had a slight shift in default or in
tone I feel like that would also help us to think that way."
When it comes to actually creating the responsive design, participants did note some important
benefits of thinking about the mobile design early in the development process.
In particular, one participant explained that "There's a hierarchy of information, right?
So as you go down in the artboard size you make the decision about what information can be cut first."
Participants also noted that creating mobile designs encourages designers to "focus on what's essential"
and "it makes us more concise and it makes us get to the point quicker."
Finally, there was a user requirement for fine-grained control of the designs, which tended
to encourage participants to prefer developing separate designs over using dynamic resizing.
One participant explained that "dynamically positioning things like labels and annotations
at every possible screen width is very easy for that to go wrong and having a fixed number
of breakpoints tends to be a little bit less error-prone."
However, challenges may also arise from using these separate designs.
Another participant explained that "It’s annoying when you have to make changes to
three or four or five different artboards and that usually introduces mistakes...
so that's one of the reasons why the design for mobile comes later."
In order to avoid errors from having too many different designs, journalists would often
only implement the responsive design once the idea for the visualization was fully formed.
Based on the survey and interviews, we identified four key design guidelines for
a new visualization authoring system focused on responsive visualization design.
The system must be able to (1) show cross-device previews of changes and design decisions,
(2) enable simultaneous cross-device edits,
(3) facilitate device-specific customizations to the design,
and (4) support propagation of edits between designs for different devices.
I will talk through these techniques more with a demo of our system based on
the New York Times visualization showing the cost of natural disasters.
The main part of the system is the canvas, which shows a separate view for each visualization of interest.
In this case, the designer is working on a desktop, a portrait, and a landscape visualization.
On the left are the system panels which show details of the visualization design similar
to systems like Adobe's Data Illustrator.
The first panel is the views panel which defines the name and size for the different visualization versions.
The second panel lists the datasets and fields that can be used to construct the visualization.
Similar to other design systems, the third panel displays groups of visual elements
(such as the text marks) that act as layers in the visualization design.
Finally, the last panel contains information about the visual encoding decisions for the selected mark.
This panel updates based on which visual element is selected and users can also view the variation across designs.
For example, the user can see that some designs use a left alignment for the text whereas
others have been customized to use a right alignment.
As the user constructs the visualization, all three of these visualizations are updated simultaneously.
In this video, the user wants to produce a second text mark to show the cost of the natural
disaster in addition to the name.
By supporting simultaneous editing, the user can maintain consistency across designs and
therefore freely experiment with changes to the design.
However, notice that the current design is somewhat ineffective for the portrait visualization.
In particular, there is a large amount of whitespace that seems excessive for this use case
and severely narrows the width of the bars.
The designer may therefore want to customize this view to remove that whitespace.
To customize the visualization, the user can start by selecting only that design thereby
locking the other views to prevent changes.
Notice that once the visualization is selected, the other views are partially grayed out and frozen in the system.
Now, the user can modify the design by removing the recently added label to display the disaster cost
thus reducing the amount of text shown on the smaller view.
The user can then update the labels to remove the redundant work "Hurricane" thereby shortening
the text to make better use of the space.
Next, the user updates the alignment of the labels and can reposition the text into the
chart area to remove the whitespace via drag.
Notice that the visualization bars automatically update to fill the vacated space.
Finally, the user duplicates the text mark and changes it to a tick mark which connects
the labels to the bars of interest.
Throughout this process, the user changes have only applied to the portrait visualization,
you can see in the background that the desktop and landscape visualizations have not changed
at all during this process and remain frozen at the prior state.
The final goal of this system is to support propagation of edits, in other words,
to allow users to reapply changes from one design to another.
Let's skip ahead in this demo to see how that would look.
Notice that in the portrait and landscape visualizations, the title has been customized to a different position.
To propagate edits from one view to another, we can view the variation to the design in the attributes panel,
preview the impact of changes, and remove design decisions to synchronize the two visualizations.
We can repeat this functionality for both the title and subtitle to fully synchronize these two designs.
To demonstrate the utility of these techniques, we recreate 7 responsive designs using our system.
These examples were selected to show a range of visualization designs, responsive techniques,
and interactive capabilities.
To see the full video demos for several of these examples, please take a look at our
supplemental material, which can be found here: tinyurl.com/responsive-examples
The main takeaway of these reproductions is that responsive designs can now be developed
using a flexible workflow in contrast to the linear development process described by journalists.
In conclusion, we contribute a survey of responsive visualization techniques which shows that
designs employ minor modifications and generally remove content from mobile visualizations.
We also contribute interviews with five journalists about their design process that highlights
a linear, desktop-first development process driven by system defaults.
Based on these results, we contribute a system for responsive visualization design that emphasizes
simultaneous editing and customization, by foregrounding design variation.
And finally, we contribute reproductions of seven responsive visualization designs that
demonstrates a flexible design workflow in contrast to the linear development approach
described by journalists.
Thank you for listening!
Please take a look at our paper if you would like to learn more or check out the full video demos
in our supplemental material, which is available at the link shown here: tinyurl.com/responsive-examples