Hello, everyone, and thank you for listening in today.
My name is Soujanya Chatterjee, and today I'll be talking about our work on assessing the
severity of pulmonary obstruction from respiration-phase based wheeze sensing using
mobile sensors.
This work has been done at the Digital Health Lab, Samsung Research America.
Obstructive pulmonary disease is a lung disorder characterized by narrowing of airways,
causing obstruction of airflow from the lungs.
Some of the common conditions of COPD, asthma, pneumonia and cystic fibrosis.
The primary causes of OPD are infectious respiratory viruses, such as the novel
Coronavirus, SARs, MERS, cigarette smoking and pollution.
More than fifteen million people in the USA are diagnosed with OPD and more than twelve
million of them remain undiagnosed, OPDs are the fourth leading cause of death in the US.
The ongoing coronavirus pandemic has caused the COVID-19 disease in more than four
million people across two hundred twenty seven countries.
More than 250000 people have deceased.
And unfortunately, the death toll is rising by the day.
So the major symptoms of respiratory diseases are shortness of breath, chest tightness,
cough, and Wheezing.
Diagnosis of these symptoms lead to the subsequent treatment and cure.
State of the art diagnosis include auscultation by stethoscope or diagnosis of lung
function by spirometry, both of which usually requires the presence of physicians or a
visit to the hospital or the clinic.
However, given the COVID-19 scenario, when the hospital staffs are overworked and the
hospitals are overcrowded and all the social distancing measures are in place.
Wouldn't it be nice if the patients can do their own pulmonary assessment anytime,
anywhere with their smartphones?
Hence our research question is the following.
Can we develop mobile sensor-based system as a convenient means of assessing the severity
of pulmonary obstruction?
Prior works on mobile based pulmonary assessment have been done to estimate lung function
from natural walk.
cough and forced exhalation sound.
These methods are reliable.
However, such activities pose a two-fold challenge in terms of usability and required
training. For instance, forced installation can be a risky maneuver for severe patients
and usually require the support of a trained personnel.
In our work, we developed a method that transforms the smartphone into a convenient tool
which patients can just hold on their chest to assess their pulmonary condition via the
sensing of the presence of symptomatic wheezing
Wheezing is one of the most important symptoms of pulmonary obstruction.
It is caused due to rapid breathing through the narrowed lung airways.
Wheezing may occur during inhalation or exhalation phase of the respiration cycle and is
usually longer than 250 milliseconds.
The severity of wheezing depends upon the location and extent of pulmonary obstruction,
obstruction in the upper airways and partial obstruction of airways results in
low-severity wheezes. While obstruction in the lower or central airways and acute
obstruction usually results in high-severity wheezing.
To develop our work first, we conducted a study to collect data from one hundred and
thirty one participants, which included ninety one COPD and asthma patients and 40
healthy subjects. Participants used Samsung Galaxy, Note 8 and Samsung Smartwatch and
wore zephyr chest-belt during the entire session of the study.
We collected continuous sensor data from each participants for 20 minutes when the
participants perform a set of tasks such as sit silent, supine silent, natural coughing,
spontaneous speech and reading.
We also collected spirometry data to obtain the standard measures of lung function.
We collected audio and accelerometer and gyroscope from the smartphone.
We also collected respiration data from the RIP sensor of the chest-belt.
The next task is to annotate pulmonary symptoms from audio data.
Annotation of 40 hours of data is challenging and intractable for a few experts and
health researchers. Hence, we utilized crowd-sourcing for this task.
We recruited twelve random annotators for this task.
We segmented the continuous sensor data into one minute chunks and the first phase the
annotators annotate the audio data just by listening to them.
Expert reviews revealed that the annotation accuracy for cough was seventy two point two
percent and for wheezing was thirty five point five percent.
To improve the annotation accuracy in phase two, we added Spectogram visualizations and
updated the instructions for annotation.
This improved accuracy to ninety five point three two percent for cough and forty six
point one five percent for wheezing.
To establish that the learning effect doesn't confound our finding, we employed a
separate group of four annotators to perform listening and visualization-based annotation
with updated instructions.
Finally, we obtained our annotation for occurrence and the severity of pulmonary events
with the accuracy of ninety two point two seven percent for cough and eighty six point
three eight percent for wheezing.
We use this annotations as ground truth levels to develop our model development.
Next, we describe our mWheeze system to detect the occurrence and severity of wheezing.
The system takes in audio and breathing data as input.
The first task is to segment the audio data based on the respiration cycles.
When the phone is held against the chest, we detect the respiration cycles from the
expansion and contraction of the chest using the smartphone, accelerometer and gyroscope.
And we detect respiration cycle from the chest mounted RIP sensor when the phone is held
at an arm's length. Next, we segment the audio data into nineteen thousand two hundred
twelve windows based on the inhale and exhale cycles.
Since we're interested in detecting pulmonary symptomatic sounds in the admission control
stage, we remove those audio windows which contain silence and which contains speech.
Finally, we are left with eight thousand four hundred ninety one windows, which contain
body-sounds and is transmitted to the next stage.
Next, we utilize the crowd-sourced annotations to label the body sound windows into the
occurrence of wheeze or non-wheeze where the non-wheeze is constituted to be cough throat
clearing and normal breathing sounds.
They also level the wheeze windows in terms of the severity levels as high and low
severity. Next to develop the wheeze detection model.
First we perform time and frequency analysis.
We observe that since wheezes are of musical and continuous nature, their presence
demonstrates a typical continuous near horizontal lines in the frequency spectogram.
On the other hand, cough and throat clearing are discontinuous sounds characterized by
short explosive bursts having relatively higher amplitudes.
Using the insights and observations from the time and frequency analysis to differentiate
between wheezing and other pulmonary sounds, we extract 32 frame level time and frequency
doing features, and we use eleven statistical functions to obtain three fifty two
aggregated level features.
Next, we train various supervised classification models and evaluated them using leave
one participant out cross-validation.
We observe that with a top 30 features, gradient boosted fee based model can detect
wheezing with a recall of eighty one point eight seven and an F1 of eight disowned by
nine six. A sample model output shows that this model can classify each respiration phase
based audio windows into wheeze and non-wheeze.
The detection of wheezing leads us to the wheeze characterization model, where the goal is
to characterize a breathing sound as low versus high severe wheeze.
We investigate the acoustic characteristics of low and high severity wheezing using the
power spectral density plots.
We observe that the power spectrum of high security wheezing are concentrated towards the
frequency band ranging from thousand to five thousand hertz.
And in contrast, the power spectrum of low severity wheezes are mostly concentrated
towards the lower frequency bands of hundred two thousand hertz.
Moreover, high severity wheezes usually have higher pitch and intensity as compared to
the low severity wheezing.
Using the insights from the above analysis for each wheeze sound window.
We compute the absolute and relative density from six different frequency ranges.
In total, we compute twelve power spectral density features.
We also compute 264 Chroma and MFCC-based window level features to capture the difference
between the intensity and pitch of high and low severity wheezes.
In total, we compute two-hundred seventy six window-level features.
Next, we train various supervised classification models and evaluate them using leave one
participant out cross-validation.
We observe that with the top twenty five features, gradient boosted tree based model can
characterize wheezing sound into high versus low-severity wheeze with the recall of
seventy four point six two.
And F1 of seventy nine point four seven, the final output of the characterization module
looks like the following where the wheezing sounds are classified in the low versus high
severity wheezes. Next, we show how we can use the mWheeze system to assess severity of
pulmonary obstruction.
Standard Spirometry test produces the following lung function measures.
FVC measures the total volume of air expelled.
FEV1 measures the total volume of air expelled in the first second of installation and
FEV1 FVC ratio measures the ratio, usually the ratio is used as a measure to estimate
pulmonary obstruction.
Our goal is to develop mWheez-based objective measure to detect the severity of pulmonary
obstruction and show the correlation with the standard measures.
To develop the mWheeze metric, first, we obtain respiration-phase based wheeze windows
from the mWheeze system. Next, we observe that a user may experience three types of
wheezes. First, wheezing may occur only during the inhalation phase termed as
inhalation-only wheeze. Second wheezing may occur only during the exhalation phase termed
as exhalation-only wheeze.
And third wheezing may occur during both inhalation and exhalation phase of respiration
cycle. We term them as respiratory-cycle wheezes.
Clinically, the presence of respiratory cycle wheezes indicates acute pulmonary
obstruction. We leverage this insight to develop the following with metric, which is the
probability of occurrence of respiratory cycle wheezes as a proportion of respiration
cycles containing respiratory-cycle wheezes and respiration cycles containing all the
three types of wheezes. Interestingly, we observe that the metric shows statistically
significant negative correlation with the FEv1 FVC ratio which shows that is in our data.
Severe patients having lower FEV1 FVC ratio experienced high likelihood of respiratory
cycle wheezes. Hence, we propose that this metric developed from the mWheeze system
demonstrates the feasibility of assessing pulmonary obstruction.
In summary, in this work, we developed mWheeze, a smartphone based method for assessing
the severity of pulmonary obstruction via wheeze sensing it is important to note that our
work is not to replace, but to complement the traditional methods of pulmonary
assessment. Our envisioned system is a significant step towards "Tele-health".
We hope that our work will be of high impact in the post pandemic world, where people can
use such systems to monitor the respiratory symptoms remotely when in self-isolation.
Our work has several limitations that present exciting opportunities for future research
in the HCI and health community.
First, in the future, mobile sensor based methods can be developed for the passive
sensing of relevant respiratory symptoms such as chest tightness, chronic cough and
shortness of breath. Second, our study was conducted in the controlled lab environment.
In-the-wild studies may introduce several new challenges, such as detecting symptoms in
noisy environments.
This would be of interest for future research.
Third, longitudinal studies can be designed for developing models to forecast or predict
pulmonary exacerbations under various contextual influences and fourth in the future
privacy preserve methods can be developed for passive assessment of lung function from
natural speech.
Thank you very much for listening to this.