Alright good morning everyone.
My name is Nathan Semertzidis.
I'm from exertion games lab.
From Monash University, Australia.
Today I'm going to tell you about something called Neo-Noumena, and how it augmented the
communication of emotion.
So what is Neo-Noumena?
Well to give some context, I'm currently working on my PhD titled: "Toward Understanding the
Design of Integrated Consciousness through Implicit Neural Interfaces.
So essentially everything that Pedro just talked about, I'm looking at how that interacts
with consciousness and how you can think of integration in terms of how integration can
be used to connect peoples consciousness's together, or how we can share conscious experiences.
To do that I have devised these three research questions, and i seek to answer these three
research questions through three different prototypes.
So these being.
How do we design neural interfaces to interact with subjective experiences of consciousness?
How do we design neural interfaces to augment the communication of subjective experiences
of consciousness.
And, how do we design neural interfaces to interpersonally integrated subjective experiences
of consciousness.
Now what Neo-Noumena is, is the prototype that I designed to answer this second question.
So rather than telling you what it is and how it works I'm going to show you with this
video.
People often find it difficult to communicate their emotions.
We believe that interactive technology can help with this.
We have designed a new system call Neo-Noumena.
Neo-Noumena uses brain-computer interfacing and artificial intelligence to read one's
emotions and present that to others in mixed reality through a head mounted display.
Both participants see fractals, representative of their emotions overlaid over their physical
environment.
Through studying how people used Neo-Noumena in their homes, we found that it's spatiotemporal,
representational, and social properties, allowed for participants to experience emotion as
something that is changeable, as objectively informative, and as a beyond natural form
of communication.
We intend that our work gives guidance towards a technological future of empathy and unity.
The design of Neo-Noumena was first inspired by literature in Neuroscience and psychology
discussing the mechanisms of exchange of information between humans.
The main takeaway that we got from here is that while language is our primary means of
information exchange, it has a lot of trade-off's that place limitations on how much information
can be conveyed while also being subject to misinterpretation.
So, meanwhile there's also evidence that humans do have the cognitive bandwidth to take in
additional information, we just don't use it.
So, we looked to research in neuroscience, emotion mapping, biomimicry see how we can
encode emotional information without language.
So this lead us to the use of fractals, colour, movement and proximity to encode emotional
information.
We then took the principles underlying neurofeedback, namely that representation of brain activity
can inform you about brain activity, and allow you to change it, ... and modified it to make
it interpersonal.
Hence the use of EEG.
And then this outward modification of neurofeedback was completed through the lens of Human-Computer
Integration, drawing specifically from Steve Mann's "Humanistic Intelligence", which informed
the adoption of AR to mediate the interpersonal communication of an "emotional reality".
So in practice this was achieved through taking brain activity through an EEG and then passing
it through machine learning algorithms to come to a classification of that persons concurrent
affective state.
And then using that with these classifications we drove the generation of fractal swarms
that followed and manifested themselves around that person for themselves and others to see.
So, to help us understand the resultant experience of using Neo-Noumena, we conducted an in-the-wild
study, in which the system was deployed to five pairs of participants, and during this
time participants used Neo-Noumena at will, for three days, within their own homes.
At the beginning and conclusion of their participation, we took measures of emotional competence,
you probably know this as emotional intelligence, and participants also kept diaries documenting
their time with Neo-Noumena, and lastly we interviewed participants at the conclusion
of their involvement in the study.
Now what this data told us is first of all, in looking at the questionnaire data, we found
a significant increase in interpersonal emotional regulation.
So people found it easier to regulate or to change the emotions of the people they were
interacting with.
And then through thematic analysis of the qualitative data we came to three themes.
So the first one is the theme of spatiotemporal actualisation.
So this theme of spatiotemporal actualisation concerns how emotions changed (or did not
change) over time, and how it was manifested within the participants world.
So the holographic manifestation of participants' emotion help participants to see their emotion
as a tangible thing, which lead them to explore different stimuli to better understand how
to regulate theirs and their partners emotion.
So a quote from this, one participant said "I wanted to see if smoking a small joint
would change how they reacted.
I think it increased my stress levels, which seemed to be reflected in the Noumena activity,
with them being far more active both in time and space afterwards".
I like this one because they say time and space specifically which was really interesting.
Umm, next is objective representation.
So this theme described how encoded and decoded their emotional information as representation
that allowed for emotion to be expressed as information, either about about themselves
or their partner.
So in expressing emotion as information allowed them to take in how they were feeling from
the third person perspective.
So this was a third person perspective both from the individual, and third person from
the group itself.
So emotion was experience as declarative, with participants describing the system being
a mirror into the mind or some kind of infallible intermediate between individuals.
An example of this is one participant said, uh they gave an example of them thinking,
" I think I'm feeling pretty good, let's put on the Neo-Noumena's for a little and see
if I'm actually feeling good, or if I'm just tricking myself that I'm doing good".
Lastly we had the theme of preternatural transmission.
So this concerned the connections between individuals, the system, and their surroundings,
and their social context, and the flow of information between these connections.
So through this theme, we came to an understanding that Neo-Noumena allowed participants to experience
the communication of emotion as some form of supernatural ability, which gave them sensory
insight and access to information which they otherwise could not have.
So this went beyond just accessing their partners emotional state, but participants also reported
the ability to make inferences about things in their context, based on their own or their
partners fractals, demonstrating an emergent sense of extended abilities of perception.
So one example of this is, one participant said it "enabled me to monitor seemingly drastic
changes in their emotions, even as they affirmed to me it was in fact okay for me to keep playing
music.
So they were playing guitar, their partner was like "oh yeah no it's totally fine", but
the fractals were betraying that was actually not the case.
There's another example where two participants were playing a card game, and they're saying
that "this other person wearing the hololens can see my reaction to the cards that I can
only see.
so it gave them more information about the card game.
So learning from this we devised a set of design stratagies in implementing these results
in your own emotion communication systems for designers building them in the future.
So we go extensively into this in the paper, I'm running out of time now so I'll just leave
it at that.
And then going beyond Neo-Noumena, so for my PhD I'm currently building my framework
of Integrated Consciousness, which this and the previous study I have conducted has informed.
The next step is a system I'm designing now called PsiNet, which is looking at how we
can interpersonally integrate consciousness not through some sort of symbolic representation
like in Neo-Noumena, but through direct neural stimulation through transcranial direct current
stimulation.
So yeah hopefully this paper gets into CHI next year and you all get to hear me talk
about that then.
But until then, thank you for listening, and there is anything I hope you take home after
this, is the notion that through technology one can access (and hopefully one day fully
experience) the subjective consciousness of another.
Thank you