- I'm going to introduce a novel scrolling technique,
and the key idea of this technique
is that actually we used the model of human attention
to optimize the design of this technique.
So actually, it was first started
from our creative solution
to some existing problems of scrolling technique,
but after that, we used to derive the model
to that to understand the situation
and optimize the performance of that technique.
So it's called Spotlights,
and this is the final system,
and I'll explain.
So there are four chapters in this talk,
and first the motivation.
I'll explain about the motivation
to get to the initial idea of Spotlights,
and then how we optimized the design
based on the (mumbles) model of scroll,
and a short part is to explain
the implementation of the system,
and finally, the results from the three user studies
will be given.
So first, about the motivation and the idea.
The motivation was initially came from the fact
that we are not really able to read
or get any information from conventional
scrolling techniques when the scroll speed is fast.
So when you scroll more than two pages per second,
you are in trouble,
and there are many reasons.
First, when you scroll fast,
too many objects are now competing for the visual selection,
so you have to decide which one to look at,
and the motion blur from the fast movement
of text and figures makes the situation worse,
and finally the exposure time
for an object is too short,
and there is a cognitive phenomenon
called Attentional Blink
that the second of two targets
cannot be detected or identified
when it appears close in time.
So that's the problem of this conventional
scrolling technique.
So in this study,
we want to solve this each of problems
by using selected,
which selects important objects from the previous pages,
and those selected objects
will be remain static to avoid motion blur,
and finally, the exposure time of each object
will be prolonged or extended
so we can provide enough time to the user
to get the gist of important objects.
And this is the final form of Spotlights.
It uses selected and static overlays,
transparent overlays,
when you scroll down the document
so you can get enough time,
get enough, you know, performance for the visual selection,
but there is a question still remains here.
The first one is about which objects should be chosen
from the previous pages,
and how many objects do we need to choose,
and how long they should be extended
after the view changes.
So, to answer these questions,
the key idea is that we use this kind of model
based optimization to solve the problems,
and before starting this research,
actually our hypothesis was that
this kind of overlays, transparent overlays,
will not affect the performance
of conventional, normal scrolling technique,
but it will improve almost everything
after that two page per second,
because actually we are not getting
anything from current status of scrolling technique.
So actually there is a lot of room
too fill with Spotlights,
and from our user study,
the hypothesis was proved to be correct.
So I will explain about the optimization part.
So first, we imagined a specific context
of scroll with user.
Let's assume that we have N object,
important object, per page,
and the current scroll speed of user
is V, P pages per second,
and due to our overlaying technique,
the exposure time of each object
will be extended by the value of E,
and this here at the E value,
it represents the amount of temporal extension
of the users.
So we have more exposure time here,
and from this context of scrolling,
there is a time per object,
time allowed per object,
can be calculated like this,
which is adding the extended exposure time E
to the original exposure time of object
determined from the scroll speed.
And there is also space allowed per each of objects,
which can be also calculated like this.
So if you increase the exposure time
of each object,
there would be more objects in the current viewer,
so it decrease the space per object,
and from these two resources, time and space per object,
our attentional performance will be determined.
So, when we explore more this model of scroll,
we found that there is a fundamental trade-off
between temporal and spatial resources of objects.
When we increase the exposure time,
the temporal resource will increase,
but the spatial resource will be decreased.
I'm sorry.
So when we multiplied those two quantities,
we always get the same amount of resources,
which is displayed in the red line.
So actually we have trade-off.
If we increase the exposure time,
we have less space for the objects,
so we have to decide what the best exposure time.
And the basic idea here is that we are now
located at the one extreme of the scroll,
so the temporal resource is now too limited
when compared to the spatial resources.
So the basic idea of the optimization
is that it's to increase the E gradually
until there is no more gain from the attention,
and this optimization rule can be expressed
as a algorithm like this,
and based on this rule,
we conducted a calibration study
that tries to find the best exposure time
of overlays.
So in this calibration study,
the design is optimized from our user study,
so we are showing one or two important objects per page,
and 10 pages per second of scrolling speed,
which is the average between our typical
range of scroll,
and the initial value of exposure time
was set as 500 milliseconds,
which is to avoid the Attentional Blink problem.
And it is increased gradually until
we get no more gain from the attention,
and here the attentional performance of user
was measured from the eye gaze of user
on each of objects,
and we have conducted with four participants,
and two different types of objects,
graphical and textual objects,
and eye trackers measure the amount of eye gaze
on that user
on the objects.
So from the research we have found
that we need 1.5 seconds of exposure time
for textual objects,
and 1.25 seconds of exposure time for the figures,
and actually, there was no effect of number of objects
on the gaze performance,
which means that now,
only one object per page is fully using
the attention performance of user.
So, we are now ready for the questions
how many objects we have to choose
and how long should they be extended?
So I implemented the final system
based on this calibration study result.
I choose one object per page,
and I used 1.5 seconds of exposure time
for the textual,
and 1.25 seconds of exposure time for the figure objects,
and the system has a preprocessor,
which converts the PDF file to images,
and there is also an object parser,
which extracts the headings and figures
on each pages.
And during the scroll,
those selected objects are displayed
on the current viewer through the overlay controller.
And in the object parser,
I used a horizontal filter
to blur the everything in the document,
and I used blob detector to choose
each of lines, extract each of lines
from the document,
and based on height of that blob,
I can determine which one is heading,
and which one is figures,
and depending on the aspect ratio of the object,
we can also define which one is text
or which one is figure.
So from this parser,
I choose only one important object
from all pages.
Okay,
and this is the overlay controller,
and one thing I have
to note here is that the system
uses the same normal scroll technique
when the scroll speed is lower
than two pages per second.
So when you scroll slower,
it's just normal scroll technique,
but if you scroll faster than the two pages per second,
it becomes Spotlights,
and to avoid possible confliction
between overlays,
the selected objects are stacked
from the top of the pages.
And there's also a function called click-and-go,
and you can directly access the page
containing the overlay
by clicking on it,
so this helps you to
reduce the effort to go back to the important objects,
to read for the text.
And this is the final system.
So as you can see,
there is a Spotlights object,
and you can click to go there.
So if you find something interesting,
you can go there to read more.
So I have done three user studies.
Actually, from those studies,
we are trying to measure the performance
of Spotlights in many
broad perspectives,
like free recall,
or dynamics of the scroll,
or comprehension performance of the scroll,
and about the performance of look up tests,
and the important idea here is that
we didn't control the scroll speed of user
during the user study.
This means that we have maximized
the amount of ecological validity
for each of tests.
And however, we still limited the time
to be short,
so they should be
skim read the document
with time pressure.
And to our knowledge,
this is the first study to measure
the scrolling performance so broadly,
so maybe many people can cite this.
So in the first study,
we tested free recall performance
of participants.
We used six books with many pages,
like over 300 pages,
and only 90 seconds were given
to read the document,
and we have measured the self-reported
comprehension score,
and the dynamics of eye gaze of participants.
And from the result,
we have found that the participants
were express more confidence in their skim reading
when using Spotlights,
and from the kinematics of the scrolling pattern,
we have found that the participants
have scrolled faster with Spotlights,
and they also have used more backwards scrolling
that the normal scroll,
and this shows that actually
they have spotted more important objects
from Spotlights,
and they can go back to the object to read it for that.
So I want to emphasize this point
that Spotlights is a very highly dynamic technique,
rather than passive consumption of information,
that you can find important objects,
and you can go back to the pages,
and you can do many dynamic way of interaction.
And from the pattern of eye gaze,
it also shows that Spotlights
is much more active way of reading papers,
and in the second study,
we also tested with long documents,
but we gave more longer time,
about several minutes, to read it.
And from 16 participants,
we have tested two books
with different difficulty of level.
The first book was about the introduction
to the HCI, which is university level book,
and the other one is about engineering
at the level of high school,
and two dependent measures were used,
multiple choice comprehensions measurement,
and NASA-TLX workload.
The results show that especially
for the difficult questions,
Spotlights increased the score
when reading with HCI book.
So, Spotlights is good for reading difficult books,
and also for the NASA-TLX measures,
there was no different.
This means the read,
it does not increase any workload of reading
than the normal scroll.
And from the pattern of their scrolling,
I think this is the most interesting
part of this study,
but as you can see,
books that are predictable to users,
like engineering book,
which is much more easier
than HCI book,
they tend to rely on more normal scrolling,
so they read more active with Spotlights
when reading with difficult book
like HCI book.
That's why it increased the performance
especially for the HCI book,
and final user studies,
the look up test.
You have to find a figure
given by the experiment,
and I used 20 CHI full papers
connected in one PDF,
and this is the experimental screen.
So in the left side,
there is a document,
so you have to scroll down,
and there is right side,
the target figure you have to find
from the Spotlights, yeah.
So the result, twice better with the Spotlights.
It's much more easier to find figures using Spotlights.
So conclusion,
Spotlights is good for every measures
or similar to the conventional scroll technique,
and this is the first study to measure
the skim reading performance so broadly,
and I think this can be used
to different applications
like when Facebook or Twitter,
they want to expose more advertisement,
but when you scroll fast, actually,
you don't get any of the advertisements,
so it can be also used
in that kind of bad applications, yeah.
Thanks.
Any questions?
(audience applauding)
- [Michael] Michael, from Hong Kong Polytechnic University.
Thank you, very interesting work.
Could you comment a little bit,
what if your system
apply on small screen devices
such as tablet or cell phone
or even the smart watch?
How would it be useful
to such--
- Yeah, actually, that's a good question,
and you know, the scroll context is really different
from this kind of PDF viewer,
and small smart watch or smart phone,
so actually, we didn't test anything about
that kind of different devices,
but I think the scroll speed
is much slower is smaller devices.
Your finger movement ability
is much more limited.
In that case, maybe, the problem is less
than this PDF viewer.
So basically, we are assuming that we are reading
a very long document in a very short time,
so it's much more like for skim reading performance.
- [Michael] Alright, okay.
- (laughing) Thank you.
- [Demeter] Demeter from ETH Zurich,
very interesting work.
I was wondering,
do you have any comments on how
your approach could be applied to two-dimensional surfaces--
- Oh, yeah, that's a good question,
and, yeah, maybe can be also used in two-dimensional
or-three dimensional ways,
but you have to decide
what is the problem now?
So in this kind of one-dimensional scrolling,
the problem is really clear,
but if you have many other dimensions,
maybe the attentional blink problem
will be different from that kind of,
you know, yeah, so,
it's good, interesting future work.
- [Demeter] Thank you.
- [Jeffery] Jefferey Hall from
City University of Hong Kong.
I'm wondering in your user studies,
did you like give them introductions
or actually basic training
about your Spotlight features,
or you just let them to discover the feature--
- Yeah, actually, we let them practice our system,
and we also encouraged to use our system
when it's possible,
because, you know, it's not
a fair game with normal scroll.
They always use normal scroll,
but this is really unfamiliar to them,
so we tried to maximize the performance
of our system, yeah.
- [Jeffery] Thank you.
- [Woman] (speaking in a foreign language) from Google.
Very interesting work.
- Thank you.
- [Woman] My question was more about,
I can imagine that a Spotlight based
scrolling technique
could be helpful to the user,
or detrimental, depending on what you select as a spotlight,
so could you tell us a little more about this--
- Ah, it's very important question.
Actually, I'm now only selecting
based on the size of the blob
and the saliency of the shape,
but it can be also useful
if you can select the object
based on its context.
So, actually, it has much more future work,
many, many future work based on
that kind of direction.
Yeah, thank you.
- [Woman] And also, I'm eager to know
how this affects or changes the user's
natural scrolling behavior?
- Yeah, actually, we need to test
with more longer time of study
because I think it can,
yeah, it can completely change the way
how we perceive the information
from that document, yeah.
- [Woman] Very nice start.
- Thank you. (laughing)
- So let's thank our speaker
a final time.
(audience applauding)
And if you haven't had enough of eye gaze
and eye gaze applications,
there is another session in this room
after the break.