Hello, I am Jens Vetter, working at the Tangible Music Lab, University of Art and Design Linz.
I present the paper and three interactive tangible interfaces that are able to physically represent sound and music data.
But first: Why do we need interactive physical sound displays?
Well, during the last years a large number of tangible musical interfaces were developed
that change the way we create computer-music.
So instead of generic buttons, knobs and midi faders we are able to use tangible interaction elements,
tokens, embedded haptic feedback or even shape shifting interfaces.
Of course this development is very fascinating, especially for visually impaired musicians,
who are mostly excluded from common music technology,
which often relies on visual feedback elements like blinking LEDs
or graphical user interfaces of modern audio software with poorly embedded screen reader capabilities.
In the field of assistive technology previous attempts have been made
to support the use of the computer, such as the Moose, the HaptEQ or the Haptic Wave.
These interfaces are amazing and show the potential of tangible interaction.
But - for instance - when thinking about the difference between passive midi faders and motorized faders,
I wonder what could be achieved, if those tangible interfaces apart from custom shapes and haptic feedback
would provide actual physical feedback using elements, that in return could be physically manipulated again.
Over the course of one year and in dialogue with visually impaired musicians
three interfaces were developed, that embed interactive physical sound displays.
Also, an experimental software was developed to verify their functionality.
The software only serves for testing purposes.
It should be mentioned, that the interfaces are still in development, especially regarding their surface and safety measures.
The Tangible pins interface is based on eight interactive metal pins.
Each pin can either be set high or low by pressing the pin in either state or by setting it via software.
During the development process the interface was used as an 8-step sequencer.
Instead of using it to display steps of a sequence, it could be used to dynamically display other sound control data
such as, for instance, keys, chords or mute states.
The Tangible string interface enables the display of a two-dimensional shape using an interactive physical string.
During the development process the Tangible string interface was used in two ways:
firstly as an interactive physical envelope to visualize attack, decay, sustain and release.
Secondly it was used as an interactive parametric equalizer, allowing to move through the frequency bands.
The Tangible wheel interface is based on a motor-driven wheel and offers continuous changes of parameters,
for instance volume or pitch.
Its movement can be recorded and played back.
The interface design has already developed since the writing of the paper,
mainly regarding the type of the motor and the positioning of the wheel.
Since now it is using an stepper motor, it is possible to dynamically create force feedback points within the wheel.
In this way, cues or markers can be embedded that for instance
could be used to indicate the minimum and maximum volume settings of an instrument.
These interfaces are able to physically display and manipulate sound and music data.
They also allow to quickly load and display presets or the settings of different tracks in a project.
During informal meetings with visually impaired musicians applied use cases were discussed:
For instance using the interfaces in combination with an eurorack modular synthesizer,
or creating a DJ console for the Visually Impaired.
As a next step the interfaces will be implemented into the web-based music environment WELLE,
which is an earlier outcome of the Tangible Signals research project.
WELLE uses an accessible and textual interface to quickly compose and record music.
Once the interfaces are embedded Into the music environment,
a series of user tests will be conducted with sighted and visually impaired people.
Thanks for watching.