Hello everyone.
And thank you for attending the my presentation today.
I am glad to share with you the work that I've done during my PhD
together, with my supervisor James Eagan from University Télécom Paris,
Institut Polytechnique de Paris.
The work is a cross- device.
Affinity diagramming tool called ADQDA to demonstrate.
Revision for fluids and holistic qualitative data analysis
experience.
So in this paper, we are interested in a specific sense-making method
called Affinity Diagramming.
It is a spatial clustering technique where analysts manually move
data items around and group them into clusters based on their
similarity or relevance to a shared topic.
Many of us might be more familiar when it is applied to scenarios such
as brainstorming.
Well, affinity diagramming is also largely
applied to analyze qualitative materials such as those collected from
interviews, observations and surveys.
Affinity diagram is basically a method using space to think and many
different activities are often involved in such sense making
processes.
For example here, what we see are the screenshots that I took from a
documentary demonstrating the analysis process of a researcher who
is trying to make sense of her interview data and during the process,
she may examine original materials transferred the
data into diagrams and develops and compare multiple diagrams
in the same time.
We can see with paper,
she's able to appropriate the available spaces to contact these
different activities and also to expand her thinking space such as she
spread out the multiple diagrams on the walls and even the documents
to the ground.
So, both James and I have similar experience in using affinity diagram
to analyze qualitative data.
Yet.
We were always struggling to handle the analysis with current tools.
No matter with paper, or digital tools.
While paper,
though it's easy to be spread out.
Well, it is cumbersome and cognitive demanding to manage the large
amounts of data.
And it is also difficult for people to collaborate remotely and
digital tools,
on the other hand, provides the ability to facilitate data management
and also different kinds of collaboration.
Well, as current digital tools are not easy to be combined to
expands the analysis space or to support multiple different activities. To
better
understand this sense-making process and discover common pain point,
we further conducted informal interviews with three practitioners
two HCI researchers, and one Sociologist.
And during the interviews, our participants shared their affinity
diagrams with us and works through how they generated them.
And the pain points that they face.
Based on these interviews together with literature review and
reflections on our own experience,
we identified a design space for digital tools to support qualitative
data analysis.
This design space contains five dimensions.
Namely, the different analysis phases.
the conceptual methods that analysts might adopt, the multiple analyses
perspectives or lenses, as well as the
digital devices engaged, and the different collaboration styles.
So to deep into more details, in the First dimension, we characterize
the overall process into different analysis
phases, basically to demonstrate how people go from this messy
unstructured data to a more structured sense.
They often start from raw materials and they need to extract relevant
information from data items, assign
these data items into categories clusters, and they also often need to
form higher-level Concepts based on these categories.
And the analysis may start from any phase based on the strateges
take, for example, some analysts may perform open coding that is start
from the left part and moving
Right
while as others may perform closed coding, start from the right and
moving left and frequently
those two are actually mixed.
We also find out no matter the strategiies applied,
users
may iteratively flow between these different
phases.
For example, they may updates the categories or concepts as more data
items gets extracted.
So the key message of this dimension lies in these lines, these
transitions, it illustrates the iterative and explorative nature of
sense-making and it also highly resembles
the well-known sense-making model proposed by Pirolliand and Card.
Thus when designing such digital tools, it is important to consider
these transitions and instead of treating them as isolated faces.
So, the second dimension is that within this analytic ahses,
We further
find affinity diagramming is often conducted or coupled with another
method called coding.
As we see in the pictures, coding is a more structured form of
analysis, where analyst highlight relevant information directly in the
documents while reading, and they directly annotat them in the
margin
of the documents with keywords, labels often called codes.
So, encoding ideas can be more quickly recorded and since the
analysis artifacts are embedded in the documents, thus it maintains
its original context.
Well, as during diagramming, it actually frees the analyst
from the linear structure of the document so that they can explore the
space to test the different ideas.
And also the space can contain extra cognitive messages
so that both coding and diagramming are complimentary analysis
methods and they both together contribute to the cognitive
interpretation of the data.
And so, in this in our design space, we draw these two methods coding
and diagramming as the blue and orange components.
And we put the lines between them to emphasize that though they
they are different methods, but they actually should be treated as
conceptually identical things.
They both serve for a consistent analysis mental model.
And the third dimension is what we call analytic lenses is basically
we find that analyst too often need to view data from different
perspectives.
Thus generating multiple diagrams or multiple codes for a given data
sets we thus represent
these multiple perspectives as stacks of codes or stacks of diagrams
in the design space.
Moreover, we find that in an analysis, analyst to may adopt different
collaboration strategies, as their analytic goals, and their analysis
context evolve.
For example, we may have some analysts engage in same and analysis
phase.
While others in a different phase and they may use same or different
conceptual methods.
And they can work in the same location or from distributed work sites.
Furthermore, the last Dimension considered the interplay of different
kinds of digital devices within the web of users analysis activities.
Well, analyst may have access to different set of devices.
And these different devices might align well with some specific tasks,
due to their physical affordances, and computational properties, for
example, when
analyst works from home,
he may prefer to do coding with a laptop well as later when working in
a lab, he might continue the analysis with affinity diagramming
method on large touch display.
So these five dimensions are together constitute the design space for
qualitative data analysis and the design space shows how these
different analysis
phases, the alternative methods, these multiple diagrams and the
devices can fit together in the overall sense-making process and how
they enter to
and interplay with each other, this design space
Also helps us to better understand current rules.
If we use this design space as a lens, we find that many current tools
only support on some specific parts of the space.
For example, most tools only support one analysis method either coding
or diagramming.
And they often impose a specific analysis ordering or analysis
sequence.
And current tools
also tend to support specific collaboration styles or constrain on
some certain type of devices, as a result current tools tend to
partition
the overall analysis process into segmented peices, and it is
difficult for users to choose the method
they want or combine the spaces for the contextual needs.
So we envision cognative sense-making as a more fluid and
holistic process where we imagined users can arbitrary transition
between these multiple analyses phases and switch between coding and
diagramming as they see fit to their situated needs.
They may work alone or adapt to different collaboration styles and
they can appropriate any available digital devices for their tasks at
hand.
or to actually combine them when they need more space to think.
So in order to demonstrate our vision, we built a proof of concept
prototype, ADQDA, it is a collaborative cross-device system,
built on top of Webstrates using distributed web technologies, and
let me briefly demonstrate how it works, here,
we see what we see is user opens
a coding view in her laptop and the diagramming view on a larger
vertical display.
As she's writing the document, she highlights the relevant
test and as we can see, in the main time, the highlights
automatically turns to text snippets in the left region of the
diagram.
In the decoding view, she can further track the snippets and the
contacts and code them.
That tagging these text snippets with code.
And then she can transition from coding to diagramming where she can
directly drag a the already coded snippets around and cluster
them in space.
Moreover, multiple collaborators can work.
Same diagram by using a large wall-size display.
Well, remote users can also join the analysis in real-time.
And they can also work on multiple diagrams, for example, to examine
different perspective of the data and they can further generate a
radical diagrams to explore higher level Concepts.
And during the process users can request the context of a snippet
across different irrational, diagrams or check our participants code
in its original documents by using an extra device, like
phone.
So in order to facilitate these transitions ADQDA applied different
kinds of linking mechanisms, it automatically links associated
artifacts across the different views that we saw and it's also
synchronizes the updates as analysis going on.
ADQDA also maintains a bi-junction linking between the coding
artifacts and their equivalent diagramming artifacts.
So, these are the notes and highlights and tags and the same notes as
we see in the, in the videos, in order to facilitate a seamless
transition between those two different conceptual methods.
So through these links, no matter what the view
the user is using, they can always have the most up-to-date and
coherence analysis artifacts.
To summarize, in this paper,
we contribute to a design space for affinity diagramming tools for
qualitative data analysis, especially using an affinity diagramming
technique and a vision for a fluid and holistic qualitative data
analysis experience and together with the proof of concept prototype
ADQDA as a web-based system and also a series of demonstrative scenarios,
which you can refer to more information in the paper.
And thank you for your time, and I will like to take the chance to say
that I just finished my PhD and now I'm available on the job market.
So, please feel free to contact me, if you know any suitable opening
positions and thank you again for your attention, and I'm ready to
take questions.
Thank you.
That was wonderful presentation.
I would love to try out the tool at some point.
So thank you for that.
Do we have any questions in the audience or in the chat and just a
quick info for everyone because of the technical hiccup
we will extend the session for a couple of minutes to also ask some
questions about this talk.
Then, I will start with the first question.
So I was wondering since qualitative data analysis is inherently
messy.
And you said, so in your talk.
What are the, what were the challenges that you faced
when designing the system?
yeah, so
For example, one of the challenges we face is like, what we say, we
could figure out analyst to often transition from coding and
diagramming out like really.
I'll be there, I'll be arbitrary throughout the entire process.
So we were thinking how we can save the user the efforts to
synchronize the different set of artifacts.
And so basically make them
focus on their analysis instead of managing these messy artifacts and
through different activities.
While one challenge that we face is, although we treat coding and
diagramming as conceptually equivalent, but actually, they are not
identical things.
For example, coding text in the coding view, does not provide
necessary
spatial information.
to determine the codes or notes position in a diagram.
So we think about different solutions, such as apply some algorithms
to automatically positions notes, but then in our interviews and also
through our experience, we think that the spatial arrangement also
plays a very important part in the analysis like in the users mental
model.
We asked a question such as what I showed here, where to put a
highlight and you notes and how to put a coded snippet within a
cluster.
And we implement different kinds of solutions.
For example, like a messy pile, as we can see here, then it's as we
see in the video.
It's sort of maintain these messages, but
map the highlights in a space and then also we do this inbox and
outbox mechanism.
I have a short video.
So when the user is coding, the coating diagram and later are
actually when she transition to do diagramming and the previously
coded text are actually put in this in code box of this same
note
So it's also helps users to maintain the awareness as different things
going on through the multiple views may be at different
period of time.
That's great.
Thank you so much.
And I find it an interesting parallel to Jason's work, the inherent
meaning of space or locating something on the canvas.
So thank you so much for that.
Okay, then let's give a round of applause for all our
speakers.