Hello everyone I'm Bashima. I recently received my Ph.D. from UNC-Chapel Hill
and I'll be joining Worcester Polytechnic Institute as an assistant professor.
Today I'm here to present my work BreathTrack which I did during my internship at Samsung
Research America with my mentor Md Mahbubur Rahman. This pandemic has changed the world as we knew it.
It is difficult to make peace with the devastation this pandemic has brought and so many deaths around
us. But COVID-19 is not the only respiratory disease that takes so many lives. Every year
millions of people die from other respiratory diseases such as chronic obstructive pulmonary
disease and asthma. Researchers have shown that continuous and passive monitoring of breathing
biomarkers are essential for the management of respiratory diseases. Recent works have found
respiratory rate and shortness of breath to be significant symptoms for covid-19 early detection.
Traditional monitoring devices are costly and can only monitor severe patients admitted in hospitals
they cannot continuously monitor for detecting diseases early or predicting deterioration
in mild patients' health. Wearables such as chest bands and mobile spirometers are
uncomfortable for long-term wearing or require additional user effort, making them unsuitable
for continuous and passive monitoring. Now, if you look around you will see almost everyone
has a smartphone in their hand which makes it a cost-effective and suitable candidate
for continuous monitoring. Prior works have used smartphones for guided breathing exercises,
respiration rate estimation using motion sensors, and cough and sneeze detection.
Despite the commendable efforts and results, these works have drawbacks. Most of them require
an additional effort like force exhalation which has proven to deteriorate health condition. Though the
works of passive wheeze sensing are promising, wheeze is only present in severe patients and
cannot be heard for mild or moderate patients. Finally, an active effort is required to enable
the system that uses motion sensors which makes them unsuitable for long-term passive monitoring.
Microphones in the smartphones make a perfect candidate for passive monitoring and previous
works have shown commendable results with audio data where they focus on breathing exercises
and annotated data sets. However for early detection of respiratory diseases we need
to focus on regular uncontrolled breathing which will be passively monitored. Now for
regular uncontrolled breathing, it is hard to get labeled data sets. Because labeling inhalation and
exhalation phases in regular breathing is hard due to the inaudible part at the edge of each phase.
Thus we introduce breathtrack which takes audio breathing data as input and detects the
inhalation and exhalation phases as output. Here, the main challenge is how to develop a breathing
phase-detection model with unannotated data set. We use the knowledge from the motion domain from
the smartphone's IMU to train the acoustic model using teacher-student training method.
Now another question is now, do we need to annotate all this motion data. That
is also a tiresome and unscalable solution. To eliminate the burden of annotation we combine
deep learning with signal processing. We designed the teacher-student model, where during training
the input to the teacher model is the motion data which goes through a signal processing model,
unlike a deep neural network model in traditional teacher student training method, and then we train
the deep neural network of the student model that takes the audio data as input. During
inference or run time only the audio data and the student model is used to get the breathing phases.
Due to timing constraints in this presentation I am only showing the comparison of breathtrack with
breeze, a state-of-the-art algorithm that uses acoustic sensor for breathing phase detection
during breathing exercises. Now in breathing exercises, we know the duration of inhalation
and exhalation, there are intentional pauses between them, and we know that the user is
breathing in through their nose and out through their mouth. None of this is present when we're
working with regular breathing. Despite that we see that big track performs better than
breathing regular breathing because we are using the knowledge from the motion domain
that allows us to work good in the edge or inaudible sections. Now using these breathing phases
we can measure the biomarkers and we can use it for many other purposes. As an example, we show
that we can achieve up to 76 percent accuracy to detect different types of respiratory patients.
In summary, we propose a technique that can be very useful to monitor respiratory patients
including asthma COPD and even COVID-19 remotely using their own devices. Our technique does not
require additional effort or any burden of annotation of the training data. This is a stepping
stone towards passively monitoring respiratory patients using audio sensors embedded around us