Flud: A Hybrid Crowd–Algorithm Approach for Visualizing Biological Networks
Many fields of science require meaningful and visually appealing layouts of networks (also known as graphs)
One such discipline is network biology, where scientists use networks to understand the chemical reactions and protein interactions that underlie processes that take place in the cell
In order to present and analyze these networks, researchers require aesthetic layouts of these
networks that clearly convey the relevant domain knowledge
One approach used by biologists to study their network is to layout out the nodes and edges
in such a way that it is easier to follow the downward information flow
We focus on such type of layouts in our work
While there are a lot of algorithms that can generate layouts within seconds
But the problem is that most of these algorithms are of general-purpose use and do not generate a biologically meaningful layout
For example, Here is a layout generated by a popular network layout algorithm that tries
to make all edges downward pointing
As you can see it is hard to follow the flow of information in this layout
Therefore, researchers often manually layout their networks after running these algorithms
However, this process is time-consuming and tedious and therefore doesn't scale well
And for computational biologists who generates 100s and thousands of these networks, it becomes
a challenge to generate aesthetic and meaningful visualization for their networks
In our work, we wonder if we can use crowdsourcing to solve this problem by leveraging spatial
reasoning skills of the crowd workers
Specifically, we consider a hybrid approach where we leverage crowd workers and
computational power of an algorithm to generate meaningful visualization
To visualize these networks we ask crowd workers to consider certain aesthetic criteria to
make sure that the drawing is readable
For example: remove edge crossing, keeping connected nodes close to each other but
disconnected nodes away from each other
Finally, keeping nodes away from the edges
However, keeping only aesthetic criteria in mind is not enough
We also need to make sure that drawings are biologically meaningful and
capture mechanistic information as well
Therefore we also ask crowd workers to increase the number of downward-pointing paths which
essentially capture the flow of signalling information from source nodes
(that are shown as green triangles here) to target nodes (shown as yellow rectangles)
So, overall, the crowd worker's goal is to create a drawing that adheres to these five
criteria which makes sure that the drawings are both aesthetic and biologically meaningful
To support the crowd worker in our hybrid approach, we built a crowd interface called Flud
Flud assists crowd workers who have no expertise in biology or networks to layout
these biological networks
Let me give a demo of the crowd interface in Flud
In this demo, you will basically see a user moving the nodes in the network shown on the
left so as to increase the total score which is shown in the blue panel on the right
You can ignore the text in the middle for the moment
What I want you to see is how the crowd worker is moving the nodes to create new downward-pointing paths
Also as the crowd worker moves the nodes, you will see that the system gives feedback
on whether the scores increased or decreased
Now let's take a closer look at how we convey the scoring information
Here is a screenshot of our scoresheet
As you can see we show the scores and priority values corresponding to each criterion so
that the users can understand how their designs are being evaluated
Additionally, we show the crowd workers visual cues like up/down arrow next to the score
to indicate if the scores went up or down after a move
Now, since we have five different criteria and each one of them poses a different challenge.
Therefore, in Flud, we have five different task modes where each mode allows the
crowd worker to focus on one criterion at a time
For example, in downward-pointing paths mode, we customize the graph visualization to show
downward-pointing edges in blue colours and upward-pointing edges in red colours
This customization allows the crowd workers to quickly identify the upward-pointing edges
as they focus on creating new downward-pointing paths
Now in a task mode, a crowd worker can stuck, and may not know where to start from.
To help with this situation we implemented clues
For example, here is a screenshot that shows the clue shown In downward-pointing paths mode
In this clue, we highlight a path that has at least one upward-pointing edge and asks
the crowd worker to reorient the edges to make it downward pointing
Similarly, we have modes and clues corresponding to the other four criteria
So now taking a step back, overall, as the crowd workers perform the task, we record
the highest-scoring layout and use it as the final output layout from the crowd worker
Next, we pass it on to the algorithmic component in our hybrid approach
For our algorithm, we choose a simulated annealing based layout algorithm
which also tries to optimize the total score used in Flud
Here are some sample layouts generated by Flud in comparison to layouts generated by an automated algorithm
Moving on to results!
After implementing the system, one of the questions we asked during the evaluation was
..
How do the layout scores achieved by hybrid or crowd-only approaches in Flud compare to
automated methods?
This figure shows the distribution of total scores achieved by different approaches
The total scores are shown on the y-axis and the approaches on the x-axis
As you can see Hybrid approach was able to perform considerably better than crowd-only
and automated methods, especially in networks with a large number of cycles
This is important because biological networks are known to have feedback loops
In the next part of our evaluation, we asked if these high scores translate to more effective layouts?
To answer this question we conducted a task effectiveness study where we asked the
participants to interpret the flow of information in the given network visualization
we recruited 7 participants with research backgrounds in computational biology and
asked them to perform 60 tasks each
What we found at the end of the study was that participants are significantly more accurate
with Flud layouts than algorithmic baselines
Similarly, when we looked at the time taken to complete a task, we found that Participants
were significantly faster with Flud layouts
We even found that they were more confident with Flud layouts
and felt that they were easier to understand in comparison to algorithmic baselines
So to summarize, in this work, we proposed and developed a hybrid crowd-powered system
to create high-quality visualizations of biological networks
One could imagine, a similar hybrid system could work for other types of layout design
tasks that require careful relative arrangement of entities in a two-dimensional space
Examples of such tasks include drawing integrated circuits, mapping social networks, and designing
interior spaces in houses and buildings
Flud is publicly available to citizen scientists at flud.graphspace.org
So if you found our work interesting and want to use the Flud to generate visualizations
for your networks or have any questions, please use these details to contact us
Thank you for your attention