We present to you MoCapaci, Posture and gesture detection in loose garments using textile
cables as capacitive antennas.
Body posture and gestures (BPG) can express how we feel and interact with other, show
our personality and it is also a key element in sign language. As a result, BPG recognition
has been among the earliest wearable sensing applications, and today many mature, commercial
applications for example for motion capture exist.
The popular options to tackle BPG recognition are IMU and stretch or pressure sensors. While
highly effective in many applications, most current systems share one limitation: they
require sensors to be firmly fixed to the body through tight garments or dedicated accessories,
such as bracelets and straps. Trending now is the Fiber optic modality where multiple
hardware has been proposed, but still pretty complex systems to deploy. A reliable and
simple method for BPG recognition with loose garments remains a largely open problem. In
this work we present a wearable system to detect body postures and gestures that does
not require sensors to be firmly fixed to the body or integrated into a tight-fitting
garment. The sensing system can be used in a loose piece of clothing such as a coat/blazer.
One potential use case is as a more sophisticated game controller for gesture-based games, such
as the Nintendo dance game Just Dance.
Here we use a really old approach, the theremin approach. The theremin is an electronic musical
instrument based on the frequency fluctuation of its antennas caused by the proximity with
a person’s hands. The human body could be modeled as a capacitor plate virtually connected
to the earth and, in conjunction with the theremin’s antennas (second plate), completes
a capacitor. Thus, human proximity changes the effective capacitance of the Clapp LC
oscillator in Figure. 2, affecting its frequency. A theremin has two antennas, one for volume
(loop antenna) and another for pitch control (rod antenna), emitting sub-MHz frequencies.
As the player moves inside the antennas’ range, volume and pitch can be controlled
by him/her hand’s position. We substituted the metal rod with soft wires and integrated
them inside clothing.In principle, any conductive wire/textile can be used as an antenna. Distinctive
aspects in our design are a discrete gesture dictionary and the antennas move with the
wearer’s body motion, consequentially changing the signal. This paper demonstrates a new
simple method for BPG recognition with a loose garment based on non-contact capacitive sensing
with off-the-shelf components. Specifically, we adapt the well-known theremin musical instrument
to the purpose of BPG recognition.
We present to you the MoCaBlazer, a loose-fitting garment prototype based on 4 channel textile
antennas. Using off the shelf components such as Interactive Wear textile cables and OpenTheremin
V3 Hardware. The placement of the antennas on the blazer did not modified the structure
of the tailor garment. And the frequency of the channels were set to be far away from
each other and avoid crosstalk.
To test in an initial general dictionary and expand it in our future work for specific
use-cases, we consider 20 body postures and gestures involving the upper body based on
generic everyday movements as shown in the figure. And to detect those ,14 volunteers
wore the MoCaBlazer for five sessions; each consists of four random repetitions per activity
resulting in 400 instances per volunteer. The participants were seven women, 24–64
years old, and 157–183 cm in height; seven men, of 25–34 years old, and 178–183 cm
in height. The experiment was conducted in an office and without any calibration per
user. And the signals of each posture were filtered by bandpass filter of 1-10 Hz and
normallized by the average of the start and end of the gesture.
A leave recording out and leave person out scheme were employed inside deep learning
methods, showing results up to 97 % for leave recording out or user dependent scenario and
up to 86 % for leave person out or user independent scenario. Hence, our model works well for newly collected
data from different people.The accuracy were similar between the deep learning models,
showing the strong pattern distinction in our signals. The first method 1D neural model
gives the best trade-off between performance, parameters, and training time as shown in
the Table. Even thought a one size blazer was used by
persons of different body size our design performance of 86 % signified strong recognition
even for the stranger case(user independent). Note that the experiment was in an office
with few metallic objects nearby, which is known to affect capacitive sensing. The impact
of common and subtle disturbances was reduced by normalization of the BPG instances.
In summary we propose and evaluate a novel method for posture and gesture recognition
sensing in a loose garment by using the MoCablazer based on off the shelf components and tested
in balanced dataset by gender, validating it with high performance. And as our study
has shown potential in loose garments, we would further pursue more elaborated garment
integration; including miniaturized sensing modules, more channels, stretchable antennas,
and sensor fusion with other modalities such as RFID to provide insitu re-calibration.
But, before we say goodbye, let‘s move the body with our MoCaBlazer.