In this lecture, I'm going to present you
our ISWC Note paper "Audio-Based Onset Detection applied to Chewing Cycle Segmentation".
First, we explain the problem of chewing cycle segmentation and how it can be solved
using audio data from chewing activity. On the upper half of this slide, you can
see plots of chewing sequences, so audio from participants chewing from intake to deglutition.
Let us listen to two different food types. First potato chips, which is a louder food type.
Now baguette, which appears to be more quiet while chewing.
The goal of chewing cycle segmentation is to partition the chewing sequences into all
events of mandible closure and opening, so-called chewing cycles. On the lower half of the slide,
two chewing cycles are shown. We aimed at obtaining the segments of each of
these cycles using signal processing techniques on the chewing sequence.
We used rhythmic and onset structures within the chewing sequence to obtain chewing cycle segments.
Starting from the vibration signal of the chewing sequence, we window and transform the signal,
compute a novelty function, which is either an energy difference function,
a spectral flux novelty obtained from the spectrogram, or a predominant local pulse curve,
which measures tempo variations in musical pieces, and corresponding beat positions.
It turns out, that this principle can be transferred to chewing cycle segmentation.
After computing peaks of each curve, we obtain chewing cycle segments.
We compared and evaluated the three different novelty curves using a dataset of seven
participants who chewed 6 food types, three loud (potato chips, apple, and carrot) and
three quiet (baguette, beef jerky, and cheddar) with a total number of 9818 chewing cycles.
We recorded the data using a vibration sensor wired to a headphone jack, which was plugged into
an android smartphone and placed on the head to assure bone conduction. As an evaluation measure,
we used a 50% jitter allowance measure and recorded precision, recall, and F-measure.
On this slide, precision-recall curves obtained from all three techniques are shown.
Best performance is found in the top right corner of each plot.
On the upper left side the PR curve of spectral flux onset detection is displayed,
where one can see that this algorithm performs well especially for louder food types as it
works with differences in the spectral domain. On the upper right, the PLP beat tracking algorithm
is shown, where all food types perform with precision and recall of at least 0.7.
For comparison, the energy baseline algorithm is displayed in the lower two plots, where peak
picking was performed in an online fashion on the left side, and in an offline fashion on the right.
Still, both spectral flux and PLP beat tracking outperformed the energy baseline technique.
Finally, we evaluated PLP beat tracking and spectral flux onset detection with
a leave-one-participant-out cross-validation. For beat tracking, the range of possible tempi,
which is a parameter of the PLP computation is varied in the LOPO cross-validation.
The average F-measure over all participants is 0.83 for the PLP beat tracker.
For spectral flux, the relative threshold of the adaptive peak picking algorithm was varied.
The LOPO analysis resulted in an average F-measure of 0.76 here.
We analysed the applicability of two important algorithms from the field of music processing
for the task of chewing cycle segmentation. The algorithms spectral flux onset detection and PLP
beat tracking offer the possibility to segment audio chewing sequences into cycles. During LOPO
cross-validation, we obtained an F-measure of 0.83 for PLP beat tracking and 0.76 for spectral flux
onset detection on our dataset measured using a bone-conducting vibration sensor. In general,
our analysis shows that it is possible to segment chewing sequences into cycles by using the audio
signal only, which offers the possibility to analyse chewing with devices where only
microphones or other audio sensors are available. Spectral flux onset detection performs fair, and
particularly well for louder food types. PLP beat tracking offers a good performance, especially
when participants chew with a homogenous, and more or less constant chewing frequency.
Thank you for your attention!