hi i'm very happy to have this opportunity to present our work:
PACE, a passive acoustic localization system.
PACE is able to passively locate and identify valuable targets simultaneously based on
the footsteps they generate. Essentially a footstep contains rich acoustic components.
The one that propagates through solid median is called structural bone component
and we use it for ranging. The one that propagates through air is called the airborne component
and we use it for angle of arrival estimation and identification. Combining the ranging and angle
of arrival information, we thus can locate and track a target with just a single caustic beacon.
We now present more details on the acoustic physics behind PACE. Here we show a snapshot of a footstep
that contains structure-borne and air-borne acoustic signals. The speed of structure-borne component
travels around 3000 meters per second and air-borne one is only about 340 meters per second.
This sharp difference in propagation speed allows us to readily separate them. As shown
in this figure, we can extract the preceeding few examples as structure-borne signals and
use remainings as air-borne signals. The structure-borne component in solid medium exhibits the
acoustic dispersion properties where high frequency components travels faster than low frequency ones.
This acoustic dispersed phenomenon carries range information. Meanwhile using air-borne signals for
angle of arrival estimation via beamforming has become a well-investigated topic. Therefore, with
range and angle of arrival, we can thus locate and track a target. To identify our target behind each
footstep, we use air-borne signals that contains richer features. However, to implement the above
system entails two challenges. First, environmental heterogeneity can significantly aggravate the
range performance. As revealed by this equation, the propagation speed of each acoustic component
not only depends on the frequency, but also the properties of materials including
elasticity, stiffness, and thickness. This indicates that the structure-borne acoustic profiles
generated by the same person on a wood floor would be quite different from that on a concrete ground.
The same problem goes for airborne signals for identification as the generated footstep would
be quite different under high heel shoes and snicks. Second, a footstep is often of low volume
and poses significant challenges in angle of arrival estimation. They suffer from severe indoor
multipath and strong acoustic noise background, making existing solutions infeasible. To solve the
environment heterogeneity problem, we develop a neural network with adversarial training. It mainly
consists of three parts: a feature extractor, a range estimator, and a domain discriminator.
The feature extractor transforms an input footstep waveform into a lower-dimensional feature
vector. Using this low dimensional feature vector, the range estimator aims to infer an accurate
distance while the domain discriminator tries to identify different domains. The input to the domain
discriminator is a weighted feature vector, whose weights are extracted from the range estimator.
The ultimate goal of this network is to obtain a domain-independent feature vector so as to
achieve an accurate range estimation and cheat the domain discriminator to fail its task. It is this domain
independent feature vector, with only range specific features that enables a cross-domain
generalization. To tackle the second challenge, we propose a new spectral weighting function.
This function allows to obtain more sharper correlation pack with low volume inputs hence
notably improve beamforming performance for angle of arrival estimation. As we can see from these two
figures, with our spectral weighting, it can achieve sharper correlation peak and better beam pattern.
To identify a target, we utilize the Mel Frequency Cepstral Coefficients (MFCC) of air-borne signals.
As we can clearly see in this figure, the MFCC of
different persons exhibit noticeable distinctive features, thus facilitating user identification.
To exclude domain information such as different shoes or flow materials, we also use domain
adversarial training, we further enhance the identification accuracy with center loss.
This is a hardware platform that we used in our experiment. It is a circular arrow with
six microphones. We attach it on the ground tightly to capture footsteps.
Now let's check the performance. The green curve plot the arranging performance under different
environments with domain adaptation while the blue one is without. It is observable that
domain adversarial training can improve cross domain ranging performance. Meanwhile, when we
apply captured data from other environments that have never participated in the training process,
we can also achieve adequate ranging performance. As we can see from the figure, the red curve shows
the range of performance with new samples that have not participated in the training process. It
has similar performance with the green one. We now check the angle of rival estimation performance.
The left figure shows the angle of arrival spectrum when there are five footsteps simultaneously
captured. As we can see from the figure, the peaks which indicate the angle of arrival
are highly consistent with ground truth ones. In the right figure, we can see that the maximum angle
of arrival estimation arrows are within 10 degrees, even under concurrent five acoustic sources
This saline performance is accurate enough for our localization purpose.
We now see the identification performance. As we can see from the table, we can achieve a maximum of
96% identification accuracy. And during the experiment, we find that the adoption of center loss
can boost identification performance, by as far as 12 percent. We have also checked the tracking
performance under multiple persons. As revealed by the two figures, PACE can accurately track
multiple targets and do not mix their traces even there trajectories are overlapped thanks to its
ability to distinguish multiple targets. That's all for our presentation. Thanks for listening