Hello, I'm Ivar Troost, and today I'm happy to be presenting to you our latest
paper: Exploring Relations in Neuroscientific Literature using Augmented
Reality, A Design Study. Our work really starts here. As more papers are published,
the harder it becomes for scientists to maintain an overview. A case study: let's
say you're a neuroscientist and you have to figure out the relationship
between depression, a brain disorder and the amygdala, a brain region. In order
to do that, you would need to read through a lot of papers using this
document-centric workflow that a search engine will provide. You need to take
a look at the abstracts, figure out where the connections are, and then make up your
mind about what exactly that relationship is and questions like how important that
relationship is in the literature compared to other ones, something that is quite
hard to answer. Now, you could do that manually to some extent, trying to take
a look at depression in relation to a few brain regions. But the moment you try
to scale that up, you start to see the problem. We need computational methods
for that in order to figure out where relations are in connection to other
relations between, in this case, brain regions and brain diseases. And in that
way we can better support relation exploration in neuroscientific literature,
or in other words, to support distant reading, to not just read the papers
themselves, but also to be able to take in the overview and better understand
the whole picture of it in order to generate better hypotheses. Now, there
are techniques available to do this, but they quickly start to look like this
and it is not something you want neuroscientists to have to deal with every
day. We need an interface, a better interface to deal with that kind of work.
And in order to integrate with the current ways of working of neuroscientists,
which happens both on conventional computing devices, desktops, laptops
and also on their desk in the form of papers and books, we believe that
augmented reality can play a pivotal role in trying to offer that kind
of integration while also catering to the 3D nature of a lot of elements that
are part of working with anatomy of the brain. Now, there was a vision in 2019
and in 2020 we actually managed to implement a system which I'll be
presenting to you here today. What we are doing, in essence, is to take a problem
space, literature exploration, and apply another solution space - that of immersive
analytics. And both of these I will go into a bit more in-depth in our background
section. And after that, I will go through the methods that we used in order
to create this prototype. I'll talk a bit about the system design and also how
we then evaluated it, which has a particular focus on whether people would
like to adopt the system in their regular workflows. If you're interested
in the details of the system, I won't go too deep in this 10-minute presentation,
but I'd very much recommend you to take a look at our walkthrough which you can find
at the link here (which you can also find in our paper). Then, what is the input
of the work that we've delivered? Firstly, Literature-Based Discovery has been
pivotal in that respect. The goal of this particular field is to exploit already
known scientific knowledge to generate hitherto unknown but meaningful
connections. Now, what does that mean, this way of generating hypotheses?
An important aspect of that is the distributional hypothesis and that states
that if two concepts are often mentioned together in different contexts,
they probably are associated one way or another. And that kind of insight can
easily be harnessed through computational methods. Figuring out which words co-occur
in abstracts, and in that way trying to see which of these associations can be
strongest. In order to enable distant reading, we also used topic model
visualization, and in practice that means that you try to use a topic model
algorithm on a set of documents, in this case, diseases - and then try to reduce
their dimensionality to either two or three to visualize it and where the final
result shows us points in space where the distance between them relates
to the semantic distance between the topics [diseases]. And last, but certainly
not least, is the field of Immersive Analytics, which aims to create engaging
embodied analysis tools in virtual and augmented reality that try to go beyond
the 2D screen paradigm. There's been some work done on toolkits in quantitative data
analysis, but we really tried to here do is to create generalisable concepts that
allow analysis of relationships, categorical data and text. Methods-wise,
we're using a design-based research approach, working
from a constructivist/constructionist epistemological frame. Which means that
we really try to use this system as a way of thinking: moving objects in space
in order to create a better mental model of it. We worked in several design cycles,
of which we will be presenting cycle 3A at this point, focusing on the willingness
to adopt this kind of technology [of] practitioners. The system design: let's
talk about the data that went into it, design requirements we set up and also
the design itself. For the data we used the Brain Association Graph, which has
more than 100,000 co-occurrences of brain regions and brain diseases, including
the sentences that they derived from and statistics relating to them. For example,
Hippocampus and Alzheimer's disease could be related; appear certain amount of time
together in the literature; and we also have the probability of when the one
occurs, does the other one also occur. We also link this to other data sets,
Wikidata and MeSH (which is the medical subject headings, which is part
of PubMed). And also, we used a topic model based on LDA and t-SNE, provided
by Cunqing Huangfu - who has also gracefully helped us as part of the Linked
Brain Data Project. The design requirements are as follows: to support
open and closed discovery. So given any concept in the dataset, a user must be
able to see which other concepts are related to it. Um, to always highlight
rather than completely hide elements to enable always seeing the neighborhood
and the context. To augment and not replace: so not to try to replace all
the current systems that are already in place, like source management, but really
try to create bridges with current devices. And then making use
of the medium: not creating big global objects, but really try to think of things
in terms of local objects that can be stored in a physical space. There
are three main tasks that we wanted to support in this version of the system,
which is to be able to inspect a concept; so, ask what is it, and get a summary.
To contextualize: so to see where a single concept like depression relates to a class
of items like brain regions. And for that we can use the co-occurrence statistics.
And finally to also be able to explain why two things are connected. So,
concept-to-concept, where the sentences, the source sentences, and the papers
related to them can help the user explore that. So here you need representational
and visualization design, for which if you want to know the details, I'd really
recommend taking a look at our walkthrough. But, in big lines, wou see
that there are Resource Spheres that can be moved around by the user in space,
can be slotted into widgets, which allow certain data analysis tasks to be
performed, which can also be connected to data flows in order to allow filtering.
Then the evaluation. The big question we asked here is: are researchers waiting
for DatAR as a serious tool in their workflow? And in order to answer that
question, we use the Technology Acceptance Model as a lens, which focuses
on the perceived ease of use and usefulness, but perhaps even more
so on the attitude towards using and behavioural intention to use these systems
in their actual practice. We had twentytwo people participate in this, using
a convenience sample from diverse backgrounds - neuroscientific as well as
computer scientists and people from the social sciences and the humanities.
And we conducted a video survey in order to reach a larger target audience. We had
two categories of qualitative feedback. On the one hand the aspects of the system
that can be improved: color palette, finding an alternative for the display
of many data points (users often found it overwhelming). And also flaws inherent
to the AR platform: things we really need to take a closer look at. For example,
cognitive biases in 3D cloud visualizations. Is it really worth
it to do that? Um, and also that participants rated himself more likely
to use the system if on a 2D screen, which kind of foregoes the Immersive Analytic
aspect of it. Of course, that is hard to conclude based on the current evidence,
because they have not been able to immerse themselves fully; they saw a video survey
version of it. So that requires a bit more study. We also found that good work
practices in the current system are not quite catered to. So there's no connection
to reference managers - and that only showcases more that we need these bridges
between devices in order to make this kind of a paradigm work. We also found that
many people offered new directions for the system to take for future research,
for example, in early tertiary education it could be useful as a way to introduce
the anatomy of the brain, and it might also be [possible] to transfer this kind
of way of thinking about literature to social sciences and the humanities. Now,
in conclusion, we think there are real opportunities in pushing this Research
Sphere-Widget-Data Flow paradigm to its limits by expanding its scope and trying
to really kick the tires on it further. And we think that DatAR as a platform can
be quite powerful in speculating about the future use of AR in daily work practices.
Finally, I'd like to mention my co-authors, Ghazaleh, Lynda and Wolfgang,
who would love to continue discussing DatAR with you, and figure out perhaps how
we could collaborate in the future. So thanks a lot for listening and I hope
you have a wonderful rest of the conference. Thank you.