Hello, my name is Wei, and today I will be joined by two of my coauthors for this presentation:
Tuochao and Jiayi.
It’s our honor today to present our research project titled VibroSense,
a system which is able to recognize home activities through a single point using Laser Doppler Vibrometry.
We believe that the smart homes of the future WILL be able to understand all human activities.
These include ELECTRICAL and WATER related activities, such as running a microwave, turning on a water faucet.
So WHY is it important to track home activities and appliances? What would this mean for homeowners?
For homeowners, a system that can recognize these activities can be integrated into current smart home interfaces, save energy, and reduce electrical leakage for a safer home.
Current solutions require instrumenting each individual device for activity detection.
So for example, to detect a laundry machine, shower, fridge and microwave, you would need 4 separate sensors or tags on each device.
Therefore our research question is can we monitor home activities throughout an ENTIRE house across several rooms and floors using just one SINGLE sensor?
Based on such question, vibration signal may be the potential solution for the following reasons.
Firstly the vibration signal can travel through the wall or floor in the entire house.
Secondly, different indoor activities have different vibration patterns and different travel path.
Hence, in theory, it is possible for us to use one sensor at one point of the wall and detect the vibration signal from different indoor activities.
However, in practice, the vibration signal traveling in the wall are usually very subtle.
Therefore, we used a laser Doppler vibrometer (LDV) to capture the subtle structural vibrations.
As shown in the image on the left, the LDV projects the laser beam at one point of the wall and detect its instantaneous vibration velocity,
whose resolution can arrive at half micro-meter per second.
To demonstrate our analysis, we utilized the laser module to collect the vibration of 18 kinds of indoor activities and visualize their spectrum as shown in the right figure.
Clearly, their spectrums all are visibly distinguishable.
So, we designed a deep learning pipeline to distinguish them.
Firstly, the vibration signal is segmented by the 2-second sliding window.
We then applied Wavelet denoising and short time Fourier Transformation.
Nextly, the 2D spectrum is inputted into a CNN model to classify the indoor activity.
To evaluate the performance of our system, we conducted experiments in five different houses.
These homes consisted of townhouses and family homes with 1 or 2 floors with different size.
In these homes, we selected 18 types of common electrical and water related indoor activities in our daily life, such as using the microwave, dishwasher, and fridge.
The evaluation of the system was divided into three parts: same-day testing, cross-day reset-up testing, and simulated online testing.
The same day testing was used to evaluate the system’s general ability to identify home activities.
The cross-day reset up testing was used to evaluate the performance on the scenario
Finally, the simulated online testing was used to evaluate the feasibility of applying VibroSense to real-world scenarios, by continuously tracking home activities over a 5 hours period
In the same-day testing, the average accuracy of the 5 houses was 96.6%.
After re-setting up the system, the average accuracy of the 5 houses was 89.4%.
Across all five houses, the average accuracy for simulated online testing was 90.99%.
As listed, we are showing the overall average accuracy, false positive, and false negative errors of the 487 samples of 18 activities collected in 25 hours.
To explore how the changes in angles and distance will affect the performance of the LDV, we conducted an extra experiment to test the LDV’s performance at 4 different angles and 3 distances from the wall in House 3.
The overall average accuracy was 92.2%.
Next, to examine the influence of LDV locations on performance, we selected 3 laser projected points in 3 different locations as shown on the left figure.
To test if our approach can further diversify activities into different stages, we divided toilet activity into two stages: flushing and replenishing.
The confusion matrix shown here visualizes that our system is able to recognize multiple stages of a single appliance with an average accuracy of 97.5%.
VibroSense recognizes 18 different types of activities by observing structural vibration patterns on a wall or ceiling using a single-point laser Doppler vibrometer.
That’s the end of our talk. Thank you!