My name is Hong Jia and today i would
like to introduce our paper SwingNet
ubiquitous fine-grained swing tracking
framework via stochastic neural
architecture search and adversarial
learning
swing tracking is one important aspect
that provides key information for many
sports assessments such as baseball golf
and tennis. There are various modalities
of sensors for sports analytics
and swing tracking could prevent injury
coaching and improving athletes'
performance and many other benefits
there are some previous work
that can be used for swing tracking
For example high fidelity cameras
and singular cameras. While vision-based
method usually suffers from occlusion
while the high-fidelity cameras
will be very pricey
and
lack of
mobility
Using multiple cameras
and using the multiple RF sensing
also lack of the mobility and simply
using the IMU sensor to do the
trajectory
tracking
is usually poor in
performance due to its accumulated
error
We
proposed to using the mobile depth
sensor and image sensor these two
sensor
modalities to do the sensor fusion
and we've done previous initial works
that using the ground truth for
example an opti-track to train the
framework to do the swing tracking
prediction. However the previous work
still lack of information
that whether proposed architecture is
the optimal and also lack of information
with the different domains for example
subjects and device and sports
So we propose swing that
the SwingNet includes three blocks that
is auto feature
extractor and regression layers and
also a domain discriminator. So auto
feature extractor trying to learns the
common feature
that
for different domains
and
regression layers trying to improve its
prediction accuracy while domain
discriminator trying to minimize the
different
domains among subject sports and device.
We want to bring the NAS to swing
tracking because NAS has been proven
very successful in many other
tasks for example image classification
and semantic segmentation.
So we basically using the
random graph generator to generate
certain layers. Here an example is
showing that using an Erdős–Rényi (ER) graph
with five nodes and eightedges can be mapped
into a CNN layer. Then similar like a
Residual and or Resnet
block.
So here's our feature extractor and
regression layers here they're basically
kind of three and one ER random graph
and for the domain discriminator we're
simply using the
DNN layers because during the
inference we do not using the domain
discriminator
This is the data collection
configuration.
We're using two
depth sensor to collect data. We also
consider the different
distance from the
of the deep sensor from the
subject and also the
orientation of the depth sensor
and here are the results
We should that in qualitative
figure on the
uh
upper left corner we see that proposed
framework SwingNet is very close to the
second
picture of the ground truth while this
in the fourth image the CNN-
LSTM
is compared with
our proposed framework in sub-optimal
For the quantitative result we show that
the SwingNet is outperformed CNN-LSTM
and simply use the depth sensor
we also show the domain adaptation
performance for a general model that
includes all subjects the general
accuracy of the SwingNet is around 3.8
centimeters
for leave while subject we show
accuracy overall is around 7.6
centimeters
for different depths
sensor distance and angles
we show that
they are not that impactful
but we observed that include more
subjects that
the performance is getting better
well
introduced with the domain adaptation
has slightly improved the
tracking accuracy or performance
from different devices
and here is the search architecture
within the configuration of the model
parameters
of the outputs
and
this is the energy cost
of the data collection using the depth
sensor in IMU and also the inference on
the mobile settings with three mobile
phones and we'll see that they can be
running efficient or correct data
efficient so that is suitable for the
mobile settings
we also show the outdoor experiment
we see that the golf and tennis can be
predicted
smoothly
thank you for your listening