Hello, welcome to my talk. My name is Aoyu Wu from the Hong Kong University of Science and Technology.
Here I'm very excited to talk about our idea about visualization processing and analysis.
To give a background, data visualizations are becoming more and more popular on the web.
For instance, the covid visualization on the top right has been the most visited page in Financial Times.
And according to Google Trends, the search interest of chart has surpassed that of image globally since Feb 2020.
Given such exposion of data visualization online, we researchers want to know how should we respond technologically,
how should we manage and make use of them?
And besides techniques, we also wonder what are their values.
What new applications can we develop to make better use of them?
To answer those questions, we propose the idea of visualization processing and analysis,
defined as the use of automatic approaches for processing and analyzing visualization after they have been generalized and digitalized.
The key here is automatic. Visualizations are traditionally made for human consumption.
And we are here exploring how machines can also consume those visualizations
Following this idea, we have developed an initial pipeline that is very similar to image processing
We first convert visualization into digital representations.
Here we use the very popular Vega-Lite specification as an input in this paper.
More importantly, we explore what are the operations that can be made on those visualizations to create different applications.
So in summary we propose a formalism using mathematical operations to process visualizations.
We follow by a proof-of-concept implementation and multiple interesting example applications.
The idea to make use of existing visualization is not entirely new.
However, those ideas are very diverse and disjoint.
For example, there are studies about style transferring visualizations,
which can be described as combining a content visualization and a style visualization to create a final visualization.
There are also ideas to difference two visualization.
The GraphScape model for example model the differences between two visualizations as an editing action.
Besides those binary operations on two visualizations,
researchers have investigated how to analyze visualization collections by clustering analysis.
Based on reviewing the large body of related work we first propose a design space or visualization operations.
We divide the design space along two dimensions: operands and operator.
The operands are visualization primitives such as data and encoding.
We classify visualization operators into binary ones and advanced ones for more than two visualizations.
This figure shows a comprehensive list of visualization operators we developed.
We include three basic operators, union, difference and intersection.
For instance, the union operator covers many tasks studied in the literature such as style transfer and chart mosaicking.
In addition, we identified six advanced operators that concern more than two
visualizations including sorting, topological sorting, filtering, clustering, and aggregation
Based on the design space, we develop a proof of concept implementation based on the Vega-Lite specification.
I will skip the technical details here.
More importantly, we present multiple applications for visualization processing and analysis.
First, we discuss how to make inaccessible visualizations accessible.
A recent work in last year CHI conference has shown that our commonly used line charts are not accessible for people with intellectual disabilities,
whereas the dot-based chart is more accessible.
However, creating the dot based chart is very challenging since it's not a common chart and not well supported in many visualization software.
Our method can automatically make this change using a union operation.
As highlighted in the red rectangle, the code simply says that takes the data from the first chart and is encoding from the second chart
to create the final chart which is the accessible version of the initial chart.
Furthermore, we explore new opportunities for interacting with two visualizations.
For example, this video shows how two bar charts can be composed into group or stacked bar charts in a mobile AR environment.
And this operation in this video is simply done by using a union operation in one line code.
We also conduct a meta visualization analysis using intersection operator to perform clusterin analysis.
As shown below, our methods can better cluster visualization specifications according to the chart types.
For example, bar charts and histograms are projected nearby.
For more applications please refer to our paper.
To summarize, we propose visualization operator in terms of operands and operators, which is very compact and expressive.
As shown in the figure, we can use simple operations to describe complex and diverse tasks.
We start with binary operations on two visualizations which are compositional and modular.
We can extend binary ones to support more advanced tasks such as sorting and clustering.
However, our implementation is based on heuristic basic priorities.
For example, we cannot smartly decide which is the user-desirable ones in this scenario which needs more studies
And we currently only support Vega-Lite visualization specification which
needs to be extended to other grammar or direct manipulation of visualization images.
Before I end this talk, I would like to echo the problem at the beginning,
that is, how to respond to the massive resolution online?
This is critical because now more and more information is communicated through visualizations, so is misinformation.
I believe visualization processing and analyze measures are very potential solutions
to those problems to identify and find common data issues in those visualizations.
Besides, there are many new applications such as analyzing the design involvement of visual designs
For example, how can a scatterplot be extended to support more complex analytical tasks.
So I believe this work I present here today is just a a very tiny step towards the right research field
and there are many potential applications that can be built upon visualization processing and analysis.
With this, i would like to end my talk and thank you for watching my talk!