Hello everyone, my name is Maria Montoya and today I am presenting the research paper titled:
"Inverse Foraging: Inferring User's Interest in Pervasive Displays".
This is a work jointly done with Antti Oulasvirta and Mario Di Francesco from Aalto University.
Let me start by explaining the motivation of this work.
This figure illustrates the scenario of a pervasive display solution
meant to show ads or information in a public area.
The main challenge here is that people tend to ignore displays if they find their content not interesting.
Indeed, research has addressed this issue by presenting methods to improve
user engagement with displays.
For instance: tracking devices have been employed to detect facial features and body posture
to get insights about user attention.
Moreover, mobile devices such as smartphones with Bluetooth have been used to collect user's
data and create user profiles, so as to make the displayed content more relevant.
Notice that these methods cannot quantify the user's interest but rather customize the content based on user input.
Motivated by these facts, this research address the following question:
How can we estimate user's interest in pervasive displays in a non-intrusive way?
In our case, we adopt a modeling approach that allow us to know the interests for each content
show on the display by using viewing time data.
As a result, this approach allows
to dynamically adapt the content shown on displays according to the audience.
I will explain more about this on the next slides.
Our approach focuses on displays that
show multiple content at the same time,
namely, tiled display layouts.
The main idea is to generate several layouts with a fixed number of content items and to measure the
viewing time of a user on each layout. Such data is then employed to fit the parameters
of a foraging model to find the interest values for each content shown on the screen.
In this work, we built two foraging models but here we focus on one of them, the tile forager.
Next, I will explain the principle behind models that relies on information foraging theory.
Such a theory characterizes users searching for information sources, choosing those that maximize
information gain, similar to how animals search for food.
We can see here the path followed by
a bird when foraging different patches in the environment to maximize the food intake
In a similar way, we can model a user perusing tiles in a display layout,
by looking at the different content items therein.
Models can be applied in different ways.
Normally we create a model and simulate user behavior
in the so-called forward modeling shown on the left.
Our inverse foraging approach, instead, employs real-world data
and fits model parameters to find the optimal values of interest.
In other words, instead of going from the model to the data we go from the data to the model.
We assessed the quality of the proposed foraging models by using synthetic data
as well as data collected in a user study.
Here we present the results obtained by the tile forager model
using real-world data.
The plot on the left shows the accuracy obtained based on the number of layouts
displayed to the user.
The results show that three layouts are already enough to obtain an accuracy
above 60%, while the maximum value of accuracy was 77% that was obtained for five layouts.
The plot on the right shows the correlations between observations and predictions
for eleven content items.
Each data point represents
the user's interest on a content item on a scale from 0 to 1.
Notice that in most cases the predictions are
very close to the ground truth values,
which demonstrates that the model adequately fits.
For further details about the results, please refer to our paper.
In addition, we implemented a prototype to observe user behavior when standing
in front of the screen.
The system leverages a Kinect to determine if a user is actually looking at the screen
based on facial features, for instance eyes and nose, and also their distance from the screen.
Similarly, it also counts the number of passers-by in the surroundings.
Such a system can be leveraged to adapt the content shown on the display based on its audience.
In summary, we presented a simple and practical approach to infer interest on tiled displays
by using foraging models.
The accuracy of the models is about 70% compared to the ground truth values,
and this method allows to dynamically adapt the content shown on the display.
As a future work, we are exploring the addition of other inputs to improve model accuracy
and also the use of machine learning techniques for content selection.
This brings me to the end of my presentation.
Thanks for your attention,
and now I will be happy to answer any questions you may have.