Hi, we are a team from Carnegie Mellon University.
I am going to talk about our paper that aims to predict well-being
using short ecological momentary audio recordings.
Psychological well-being is a key component of one’s health.
Traditionally well-being is assessed using self-report
surveys which impose a burden on users and may lead to bias.
Recent approaches of predicting well-being based on audio data
are not suitable for everyday settings as they require professional equipment.
In this paper, we show that it is possible to get good quality audio
data for assessing mental well-being with daily recording devices like smartphones.
To collect the data, we created an Android app that sends out
EMA surveys every 30 minutes and end-of-day surveys.
In the EMA surveys, participants are asked for their recent social interaction
and to record audio.
In the end-of-day survey, we evaluate the participants’ end-of-day wellbeing score
based on self-reported loneliness, depression, and stress scores.
The well-being score will be the target output of our machine learning models.
We transcribe the audio files into transcripts and extract textual features from them.
We also extract audio features from audio files.
These features are aggregated into daily features
and will be the input of our machine learning models.
Using the surveys, we constructed 2 baseline measurements.
They are people’s reported social interactions during the day
and their mood at the end of the day.
Because of the number and the sparsity of the features, we need to perform a feature selection.
The first way is to eliminate low variance features.
The second step is to run a 10-fold Linear Regression and compare the text vectors' errors.
Finally, using linear regression coefficients
to determine feature importance is applied to textual features.
Different machine learning models with the optimal hyperparameters
found using grid search are trained and tested using 10-fold Cross-Validation.
We can see that the 2 baselines almost always have the worst performance.
We can conclude that the features extracted are effectively better than the 2 baselines.
In addition, it also proves that using audio recording and the automatically transcribed text
can indeed get rid of the need to fill out surveys.
However, the best feature has a high error rate of 14%.
It appears that using the audio data with daily recording devices
cannot yield good enough mental well-being assessments.
From the scatter plots, it is observed that all the orange high error data
points have negative well-being values.
Also, there are a lot more positive data points than negative ones.
In addition, the span of negative values is larger than the span of the positive values.
The small dataset is yet another concern.
These factors combined make it difficult for the model to learn the representation
of the negative well-being data points more effectively.
Lastly, the high error entries are mostly from the same participants
who usually only have negative entries.
It is possible that these participants are more sensitive to depression,
loneliness and stress and therefore exhibits somewhat different well-being representations.
We use permutation feature importance to determine the crucial well-being indicators.
The insights found are consistent with previous literature that
MFCC and affect-related measurements are valuable for analyzing human’s well-being.
We further prove that audio recordings can provide
as much and even more psychological information than self-reported surveys.
However, the less-desirable prediction results arise from the data collection mechanism.
Based on our experience, if any similar research should be conducted in the future,
we suggest that:
first, increase the size of the dataset and the number of negative data samples.
Next, we should provide more structured guidance on what people should record
and collect more continuous and instantaneous predictions of well-being.
Finally,
more testing on the generalizability of the models with other samples will be really helpful.
Thank you for your attention.
If you have any questions, please feel free to contact us.