welcome everyone today we introduce
approaching the real world supporting
activity recognition training with
virtualeye medera my name is jaquan and
this work is done with bing yao dr
gregory aba dr thomas plotz from georgia
tech you become cba lab and north system
university
last year we proposed imu tube system
which is a grand vision for creating
significantly large label data set for
variables
by using video data to generate large
virtually data set
in this work we extend the imutube with
adaptive video selection to generate
high quality large label dataset which
could improve deep learning models
significantly 0.5 to 0.75 for
challenging heart problem the weight
lifting
mainly in youtube motion information
from the video is extracted with 3d
motion tracking which is again converted
to virtual sensors or non-body locations
to collect high quality virtual sensory
data
high quality 3d motion tracking is
essential
however
there are a lot of challenges in
unconstrained online videos fast motions
complex camera movements and occlusions
to make iem youtube usable for
real-world videos to generate
high-quality large virtual data set we
propose to handle those challenges in
online video for motion tracking
in doing so
we propose adaptive video selection
the extender on youtube given a video
detects artifacts in the video segments
such as extreme background motions or
uncertain pose detection and remove
those noisy frames to prevent error
accumulation in motion tracking
and generate high quality virtual data
from the remainder of the video to train
the model
specifically we handle four different
kinds of challenges
non-human objects from the confusions of
post-detector are handled with noise
post filtering
various occlusions such as self
occlusions and occlusion by people or
objects are handled with occlusion
handling
motion blur which can generate spurious
poses for a fast human motion or handled
with foreground motion detection and
excessive motion blur from extreme
camera motions are detected with
background motion detection
to evaluate the effectiveness of the
extender imu tube
we
experimented with free weight exercise
classification specifically we
classified 13 number activities
as you see in these six examples
dumbbell exercises are very subtle in
differences in arm motions and poses
that challenge challenging in
classification
in our experiment we did use three
different benchmark data sets but in
this video we show results for myogen
dataset
which had 3.5 minutes of sensor
recording productivity with single risk
sensor
and where activities are performed in
the lab with control
for virtualmin data set we collected
videos from youtube where activities are
performed in the wild at various
occasions
and
100 times larger size of 4cm data was
collected compared to real-life dataset
which goes up to 3.2 hours
per activity with weekly labels where
searching keywords are used as target
growth labels with this large scale
virtual data set we show that we can
significantly improve deep learning
performance
here is experiment result in this
experiment with myogen data set we show
how changing virtual data size can
impact more the performance
we start with realignment data only
3.5 minutes
and we eventually add up to 3.2 hours
per activity which is 100 times larger
as a baseline we evaluated a random
forest with ucla model
as you can see that as we add virtual
data the classical machine learning
model can consistently benefit from more
data set
but when we use deep learning model the
improvement is substantial
data and hungry nature of the
convolution model which is standard and
very popular deep learning mode in heart
can largely benefit from large scale
dataset
which is 0.23 f1 score
and this shows that to train a complex
model you have to collect more videos
and extended item youtube can collect
high quality large-scale label data set
and that can train them learning models
more extensive experiments and analysis
are in the paper
in summary
we extended imu tube with adaptive video
selection to generate high quality large
virtual dataset
in doing so we are now ready to collect
large scale data beyond the single
resensor
and with multiple on-body locations for
complexity learning models and with our
scale virtual data set this is the right
time to design very sophisticated large
horror models to drive breakthroughs in
horror
and please download version data today
and have fun
thank you very much