- [Paul] Public Health, presented
by Munmun De Choudhury, but she doesn't take
any responsibility for the results, right?
She's happy to answer all the really
tough questions, though.
- Thank you Paul, and hi everyone.
I'm Munmun from Georgia Tech, but as Paul pointed out
I'm not the co-author and I'm presenting on behalf
of my friends and current collaborators,
Kiran Garimella from Aalto University,
Abdulrahman Alfayad from CMU Qatar,
and Ingmar Weber from Qatar Computing Research Institute.
Can you hear me?
Cool, do the title of this talk is Social Media
Image Analysis for Public Health, and it's a note.
So what I'm going to tell you is a brief overview
of the findings that the authors found in this paper.
So the ultimate goal of this work is to find out
whether social media data could help us understand
and track lifestyle diseases and health behaviors.
As you know, there's a rich portfolio of literature now
in SCI and related fields that looks at these types of data
as a way to get traction on health concerns
that are faced by populations.
So why social media, right?
It has many benefits and could be a way to get signals
about what people are going and how they are feeling
at different points in the day.
And this kind of data can be collected in real time
with very little intrusion from people.
This can complement and supplement survey techniques
which are typically used to identify these diseases,
and thus provide sort of more fine grained temporal
and spacial signal into these lifestyle disease
related experiences of people.
And of course, this kind of data can span
millions of individuals.
But most of the work that has happened in this space so far
has leveraged text.
So text is very valuable and these tweets
that you have up there kind of tells you
what people are eating.
So you can have things like I am eating a sandwich.
And sort of using these types of tweets,
you can get a sense of people's general dietary behaviors
and different health conditions that are related
to these type of eating habits.
So there is work, especially over the last two years
we have seen a couple of papers at least in this space,
Culotta looked at how these type of tweets
can be used to identify and assess a variety
of different county level health statistics.
And last year, Ahbar et al looked at analyzing
these food consumption related tweets to get an assessment
of obesity patterns that are prevalent in different parts
of the United States.
But one recent development is the use
of immaturing platforms like Instagram.
As you know, food is a phenomenon on Instagram
and people are constantly sharing things
about beautifully plated pictures of what they're eating,
like you see on the picture.
And if you do a simple search you would notice
that photos that are tagged on Instagram
as hashtag food, that's like 193 million posts
at that point in time when this was screenshotted.
And that's much higher than other generic hashtags
like travel, which are also very popular on Instagram.
But one interesting element of these pictures
is that they do have tags that people use
to describe these foods, but they're always
not very informative.
So here, I think the resolution is quite poor,
but some of the hashtags for this post are detox,
lunch, and a couple of others.
But they don't quite tell you what it is
that people are eating.
So as a result it can be difficult to get a handle
of the types of foods that you're eating
which eventually can tell you things
about lifestyle diseases and health behaviors
of people.
To tackle that, what the authors of this paper did
is very cleverly leveraged what are known
as machine-generated tags.
There are platforms like Imagga.com which,
if you upload an image would generate a set
of machine-generated tags which are automatically
identified based on an underlying deep supervised
learning framework which is a recent advance in the field
of computer vision.
And what the papers of this author do is leverage
those machine-generated tags as useful descriptors
of what people are eating.
So to give you an example, here for the same post
that we saw earlier, you can use machine-generated tags
to understand what people are eating and you see tags
like salad and vegetable which more accurately
describes the food in this bowl compared to the tags
that were used by the user on their own.
The other interesting part of these posts is that
they have geo-located information.
Actually on a map you can place all of these things
that people are eating, leveraging the visual
and the textual content of the pictures.
And this can give you a good sense of what people,
what health conditions people might be experiencing
in a spacial scale.
So actually what this paper does is it predicts
county level health statistics utilizing these types
of machine-generated and user contributed information
shared on social media.
Specifically, one of the goals of the authors
in this paper is to find out what is the benefit
of using these machine-generated tags.
The authors collected dataset of 200,000 Instagram
food pictures and focused on the 100 most populous
counties in the United States.
This is sort of a spacial distribution of the different
posts that were collected through this process.
Thereafter, they developed a supervised regression framework
in which the goal was to collect different county level
statistics of health behaviors utilizing
the Instagram data.
Dependent variables in the model where a variety
of different health statistics at the county level,
like obesity, drinking patterns, diabetes, and so forth.
So for the purposes of this presentation,
I'm going to talk about these three measures,
and then I'm going to, I would recommend you
to refer to the main paper for the other ones.
The predictive features of the regression model
were county demographics which are typical baseline features
that are used by agencies like CDC to get a handle
on these statistics of health.
And the authors also used the user-generated tags
and the machine-generated tags
and tested the combinations of these.
These are the performances that the authors got.
As you can see, generally, including the machine tags
improves the performance of the baseline model,
which is if you were to use demographics alone.
To go a little bit deeper, one of the better predictions
was for excessive drinking behaviors.
And as you can see, the machine-generated tags
include tags like alcohol, drinks, and beverage
which is a better descriptor of that picture
that you see up on the post and therefore you're getting
better traction at predicting excessive drinking behaviors
when you add these machine-generated vision descriptors.
So actually what this paper tells us is that there
is valuable signal to be gleaned from the visual attributes
of an image.
Usually, adding this type of information
improves the predictions that we could possibly get
by utilizing demographics or text-only information.
I think I'm out of time, and I can take a couple
of clarification questions if you have any,
but if you have more substantive questions
I recommend you to reach out to the authors
of this paper and the Twitter handles are up on the slide.
Thank you.
(applause)
- [Paul] Given our time constraint,
and that we don't have the authors here,
we'll just go right on.