Hello and welcome — this is Balazs and
today I will tell a story about how we
can connect people living with ALS
through the shared Joy of musical
co-creation
but before I get into that — I would like
to express my gratitude to my whole team
and friends — without you this bachelor
project wouldn't have been this fun and
successful
and next I want to thank all the
memorable work and thorough guidance for
our supervisors — who provided us with
great inspiration on the way — finally
kudos to the members of the Microsoft
Enable Group — who brought us this amazing
journey
now I want you to meet Alper who's a
truly special human being
Alper is from turkey and he used to play
the classical guitar as a virtuoso as he
puts it — “Music represents the color of my
spirit and when I make music it develops
my spiritual life”
but unfortunately Alper got diagnosed
with ALS in 1990 — later on as he started
to loose his ability to move his daughter
introduced him to different accessible
instruments such as a breath control
instrument and head mouse based
instrument
he still creates beautiful compositions
and uploads them to his YouTube channel
Alper is just a joy to be around his
energy and great advocacy work for the
ALS Community are very inspiring
so being inspired by Alper story we
realised that collaboration is such a
key aspect of music
and many people living with ALS or some
form of motor impairment who used to be
professional musicians they just can't
share the joy of musical co-creation
anymore
inspired by Alper's connection between
him and his family we also thought that
collaboration through music could be a
meaningful leisure activity to share
for PALS (People with ALS) with their loved ones or their
caretakers
we wanted to create a collaborative
accessible digital musical interface as
a universal way for connection for PALS (People Living with ALS)
not necessarily having any prior musical
expertise
with this twofold mission we gained
inspiration from the DataDuo a playful
collaborative synth — we extended its
collaboration metaphor — to a spatial
carousel where one person places beats on
the sequencer — while the other places
different effects on that beat — then the
carousel moves on — the roles are swapped
and they co-create music with the next
instrument again until they reach a full
circle and the whole composition
importantly the collaboration could
happen physically but remotely too
We were advised by the Enable Group to
focus on eye tracking and follow the
extreme constrained design paradigm — this
aligned well with eye-driven digital
musical instruments reviewed by Hornof et. al
which led us to design a radial UI
connecting eye-gaze-based interaction
constraints with quantised musical
sequencing
we can see the final 16-step radial
sequencer with turn-based collaborative
mechanics here. — after placing beats for
some time the carousel turns and then we
interact with the sliders — while the
partner places a beat — every target size
was chosen to be appropriate for eye
trackers and colors were chosen to
reduce eye fatigue — as much as possible
we implemented eye-gaze-based sliders sliders — you
can see them now — which worked
surprisingly well while testing them
the main body of the sliders providing a
convenient path to follow with eye-gaze
we also strive to keep the design
accessible for PALS — one main feature is
the euclidean sequencer — this provides
accessible interaction on two levels
corresponding to the previously mentioned
two goals of DuoRhythmo
one it helps users to make harmonic
beats with no musical knowledge by
spacing out beats equally on the
sequencer — we can see that now that it's
six speeds — and then, well now we activate
it to four — then we can increase it to
five — and then six equally spaced out
beats — and number two, it enables making
more complex sounds this way with fewer
interactions reducing eye fatigue for
PALS (People with ALS)
For the eye-fatigue issue we also embrace
customization which was highly suggested
by Frid et al. — as being a crucial in an
accessible digital musical interface (ADMI)
here we can see customisable dwell time
feedback for eye tracking
we further embodied customisation when
contacting the International Alliance of
ALS and MND Associations — We connected
with the community to further empathise
with our target group through several
interviews and sessions with PALS and
their caretakers — we increase support for
multiple input methods as we learned
that PALS prefer to use the regular
interaction methods until they can — then
as ALS progresses they shift to
accessible mice — and finally in the later
stages they rely more on constrained
input methods — so we came up with this
continuum of supported interaction
methods based on the progression of ALS
involving the community more in our
process we also wanted to evaluate the
prototype with PALS — this is usually
difficult because we are talking about a
particularly sensitive target group — some
folks might use a respirator — some might
use alternative communication devices (AAC)
but we could adopt many great methods
from the research community to do our
user testing with great precautions
we were also in the middle of a global
pandemic which just further complicated
these special conditions — so to get an
overview of the user experience of a
collaborative accessible digital musical
interface we choose a mixed methods
remote user experience evaluation — with
the system usability scale and the
Microsoft desirability toolkit
we had five PALS (People with ALS) to test out DuoRhythmo
from all around the world
co-creating music with one of the
members of the research group remotely
you can see Alper again here during his
testing session
we are aware that a small sample size of
only five PALS (People with ALS) to test is not too
comprehensive quantitatively — but using
the System Usability Scale (SUS) helped us to
follow an explanatory study and based on
the quantitative data use the
qualitative data to uncover deeper
tendencies
the Microsoft desirability toolkit
helped PALS to form a basis for their
opinion more easily and they could
elaborate on each card at their own pace
by an interview — you can see the
distribution of words on this slide — the
green bars for positive words and the
red bars for negative words
we process the qualitative data via
thematic analysis and identified six
main themes — I'm not going to get into
most of them in detail — I would just like
to highlight a couple of quotes from
some themes
“people don't have to think they are ill
or something you know? They can forget
their minds, and I don't have a problem
forgetting with my mind, but I know a lot
of people have trouble with it. So I
think it can help them”
“it's stimulating because the rhythm is
the basis of life I think and when I
play the rhythm and the rhythms around — I
feel some positive feelings that
stimulate me”
“from an accessibility perspective being
able to use eye-gaze I think it's huge
And my arms get tired easily so that's
why I use both mouse and eye gaze”
“you know I'm not a musician at first
glance the screen the interface can feel
like a lot — so I think it's a relative
thing — it really depends on the player
and then as people play along learn and
get up to speed about the app it may not
be complex anymore”
One key aspect we've decided to focus on
was designing a very accessible and
PALS centred remote testing procedure
we ensured that the experience wouldn't
be draining so we sent out a detailed
agenda before alongside a short tutorial
of the prototype then PALS could review
them and prepare for the testing session
we also had a couple of researchers
present during the testing session — if
any technical problems occur which they
actually did — as that is very likely to
happen with an uncontrolled setup — we
could resolve that quite quickly
We received praise from our test
participants on including a multitude of
accessible inputs — one of our
participants actually used her eye
tracker and the trackpad simultaneously
On the other hand we learned that
musical expertise did impact the user
experience as more experienced musicians
found more joy in expressing themselves
through DuoRhythmo — this became
apparent from their System Usability
Scale and thematic analysis themes
finally, they pointed out that having
CADMI (Collaborative Accessible Digital Musical Interface) was a very interesting way
of reaching out to each other — we are happy
that our extension of accessible digital
musical interfaces (ADMI) to the collaborative
domain was praised by our end
users in this initial study — so we want
to encourage the research community to
think about having CADMIs as an
extension of accessible digital musical
interfaces (ADMIs) and include collaboration as
a key component where loneliness and
depression induced by a specific
disability could be prevalent
but what about the future
of DuoRhythmo
since finishing this project's first
iteration which the paper actually
encompasses — the whole team felt that
involving the ALS community in the
design process was a really rewarding and
a very powerful experience for us
we wanted to give back more — we wanted to
extend DuoRhythmo from a research
prototype to a research product
so we just worked more to fix some
usability issues and added new features
based on user feedback — we also improved
the collaboration experience as
suggested by PALS — to reach more people
we need to guide DuoRhythmo the lab
so right now it is freely available to
download from the Microsoft store and
it's ready to be used by PALS (People with ALS) and
other motor impaired music living folks
and yeah one last thing, maybe you are
wondering why I talked about a
collaborative musical experience so much
and I haven't played any music yet?
well I would like to take the
opportunity now — I'm finishing my
presentation
we can listen to some pleasant beats in the background
so that was me — Balazs Ivanyi
presenting DuoRhythmo at CHI 2023
I'm ready to take your questions now
thank you so much