hello i am Li Su from the Institute of
Information Science
Academia Sinica, Taiwan it is my pleasure
to have a presentation in the ACM IUI
conference
the title of my presentation is
positioning left-hand
movement in violin performance a
system and user study of fingering
pattern generation
authored by chung ping chen, yee shin gen,
shiwei sun, and me. this paper is about
automatic fingering recommendation for
playing violin
organizing fingerings or say choosing
which fingers to press on which
positions and strings,
is a crucial step for playing the violin
proper fingering makes the hand
movements comfortable,
and also facilitates musical
expressiveness,
however arranging fingering is not an
intuitive task
since it is a highly subjective and
knowledge intensive process.
So we consider a computer-aided system
which provides
possible fingerings for violin players
here is the framework of the proposed
fingering generation system
as shown here, the proposed system is
constructed with three modules for
predicting string designation,
hand position, and fingering choice these
modules are neural networks in which the
inputs can be manipulated by users
according to the needs in the use of
strings and ease of finger
transition. Here we have a demo video for
our proof of concept user interface.
First we can load a music score which is
formatted in a standard music xml file.
The music xml file contains note
attributes
including pitch, duration, note type, and
others.
And our load function can automatically
gather these node attributes.
Press the button to open the file.
Here we choose the basic mode.
Then press the recommend button.
The string and fingerings are then
recommended and shown on the score.
The beginning is the d string from no
fingering to the first finger,
then the g string is recommended for the
c sharp note.
To train our model we compiled a new data
set
which contains 10 violin pieces and the
corresponding note by note annotations
by 10 professional musicians.
Given the data set violinists are
recruited to annotate the fingerings
according to their performance habits,
resulting in more than 200 kilo note
annotation in total.
The annotations specify the detailed
performance attributes of each note
such as pitch, onset, duration, note type,
string designation, hand position, and
finger.
We use bi-direction along short-term
memory networks which take a note
sequence and it outputs
probabilities for strings and hand
positions.
The string module outputs a sequence of
string designations
based on the predicted string the
position module outputs a sequence of
hand positions. Thereon the finger module
generates a sequence of finger choices
based on the outputs of the string and
the position modules.
We construct three generation modes
based on the output probability
distributions of the networks.
The three modes are basic, lowest, and
nearest.
For the basic mode the hand position
with the highest prediction probability
is generated for each note.
For the lowest mode the network outputs
the lowest and position among the hand
positions with top three prediction
probability values.
For the nearest mode the network outputs
the hand position which is in the top
three prediction probabilities and is
nearest to the generated hand position
of the previous note.
Here we show the generated fingering
samples of the three mode and a baseline
method.
We observe that the sample by the lowest
mode tends to switch between strings to
keep the left hand in a lower position
while the nearest mode mostly stays on
the A string with smooth transitions
between different hand positions.
Eight subjects are asked to assess each
sample using seven point scales
regarding the four
targets called ease of transition,
expression,
naturalness, and skill level. Results on
the first excerpt shows that the lowest
mode is easier for beginners than the
nearest mode.
However for the second excerpt we got no
significant differences between the
three modes.
This shows that the assessments of the
same mode could be diverse.
Comparing the proposed system with the
baseline method,
we find that the lowest mode of the
proposed system has the potential to
outperform the baseline.
Thank you for your attention