- Good afternoon, I'm Han-Jong Kim
from Industrial Design Department of KAIST.
I will present miniStudio, a Designer's Tool
for Prototyping Ubicomp Space with Interactive Miniatures.
My coauthors are Ju-Whan Kim and my adviser Tek-Jin Nam.
Due to the development of the information technology,
the subject matters of design are changing
from non-digital artifacts to digital and ubicomp devices.
Recently, the ubicomp devices are embedded
in a physical space and communicating with each others.
The space is consisting of various ubicomp devices
have become new subject matters for designers.
However, it is challenging to envisage user experience
in the ubicomp spaces.
The spaces involve complex interactions,
diverse forms and sizes,
and heavy resources to be prototyped
in the early stage of design.
Currently, designers utilize scenario-based
design method to deliver futuristic concept.
But, it is hard to consider detailed interactions
through acting on imaginary scenarios.
On the other side, some development toolkits
for ubicomp systems were proposing the HCI.
However, these are difficult for designers to use
without programming knowledge.
So, we aim to develop a toolkit for designers
without technical implementation skills
to build ubicomp space prototypes.
To deal with interaction prototyping for ubicomp space,
our research was inspired from proxemic interactions
by Greenberg et al.
They suggested a framework to concern spacial interactions
among people, devices and non-digital artifacts.
We applied four measurements of this framework
to deal with invisible and complex spatial interaction
for designers' prototyping.
As related works of our research,
several prototyping tools were proposed
with designer-friendly environments,
and augmented prototyping (mumbles)
only designed exploration with AR techniques.
But, these tools mainly focused
on a single interactive product,
and limited to support large and complex ubicomp system.
Miniature-based system could support
large-scale simulation,
such as urban planning or interior design.
But, the proposed system had a low flexibility
to cover free exploration for new design concept.
And some development toolkits enabled
to build real ubicomp system,
but it was difficult to use
designers without high level of implementation skills.
To support only design stage,
we developed a prototyping toolkit for designers
to explore our ubicomp space idea
with interactive miniatures and Photoshop.
Our prototyping system was integrated with Photoshop
as one of the well-known graphic editors.
It generates miniature figures,
and miniatures can be easily made
with only soft printed paper and Lego bricks.
And we applied augmented prototyping
to project digital images on physical miniature.
The images also shaded
where the special condition of miniature
can also be made in Photoshop.
After that, the color projector system
projects an image on the miniature,
and the image can be dynamically changed
by tangible manipulation.
miniStudio applies a mix of augmented prototyping,
miniature-based prototyping,
and integration of graphic software.
It enables rapid and low-cost prototyping
in a designer-friendly environment
for dynamic presentation on tangible miniature.
miniStudio consists of three functional components.
The Photoshop plug-in has tool panel and view panel.
And tool panel generates miniature figures
and defines events.
And view panel enables users to identify
the models and events in current workspace.
The camera-projector system tracks hidden AR markers,
marker stickers attached on the miniature model.
It projects digital images on the miniatures
that are generated in Photoshop.
Several tangible icons are used
for triggering and contouring events.
These are five types of events that miniStudio supports.
Location of the miniature model.
And distance between two miniature models.
Motion in the direction of left, right, up or down.
And orientation of multi-sided miniature model.
And custom events for non-spatial conditions.
We simplified and reconstituted four proxemic measurements
for our planar paper miniatures.
miniStudio supported four spatial relation events
based on proxemic interaction,
and custom event was added for non-spatial conditions.
We applied one-to-one mapping of one event and on image.
The event can be defined as naming layer group in Photoshop.
When the group is generated by miniStudio panel,
a layer inside the group
becomes a state of the miniature model
for also shaded in event.
The frame animation can be played
with certain frame duration,
with multiple image layers.
The Photoshop can be used for drawing a miniature surface
in various ways, such as sketch-like or realistic images.
This video shows an example walkthrough
to define different state of the miniature model.
The first step is building miniature model.
Our user can set height, size, name of the model
to generate model figure.
Then the generated figure has own identity number
for auditioning the marker sticker.
Then the user can complete the miniature model
after printing and assembling.
The second step is defining event and images.
When the user selects the event type and parameters,
the event layer group is generated.
The user can add the layer inside group
and generate images for defined events.
The next step is testing and modifying miniatures.
The user can test defined events and images
with common projector system.
And the user can easily edit images on Photoshop.
And multiple layers inside the event group
can be played as frame animation for dynamic representation.
The Photoshop plug-in was developed with Photoshop Script
for automated operations.
And tracking software for common projector system
was developed with Processing and NyAR Toolkit Library.
The software receives information of events and its images
exported from Photoshop plug-in.
And then, it organizes images according
to the spatial condition of miniature space.
The hidden AR marker were generate markers,
were generated by silk-screen printing
with white ink on a white reflective stickers.
This technique was cheaper than using special material,
such as infrared observing ink.
We used IR detection with hidden AR marker
for non-obtrusive integration
of miniature and projected images.
We conducted user study
to investigate the value of miniStudio,
and to identify the usability and areas for improvement.
We recruited six individual participants
and three pairs of graduate-level designers and researchers.
We gave them a design assignment
to design interactive bus shelter.
The session consisted of 30 minute tutorial
and one hour design session for prototyping.
A debriefing session with post-interview was followed.
Total nine interactive bus shelter prototypes were built,
and they could be categorized as bus information system,
communication space and personalized shopping space.
These are three examples of generated bus shelter ideas.
The one is bus reservation system with color indicator
instead of bus numbers.
And another example is a public display
that shows a map of visited countries
for communication between tourists and local citizens.
And online public shopping system
in the bus shelter was also demonstrated.
Participants used distance event most frequently,
and location event was used
as pre-state of the distance event.
Motion event was assigned to the bus
to interact with remote components in the bus shelter.
They used custom event
to represent detailed interaction step
for different time, such as day or night.
Orientation event was used
to represent different state of the miniature model
instead of actual orientation.
The participants regarded miniStudio was easy
to run and use due to the Photoshop.
After only 30 minute tutorial,
they generated their interactive prototypes
in about only one hour.
We observed the usage patterns
and participants actively refined
their prototypes after testing.
Most of all, Photoshop could support most rapid generation
and alteration of images.
We found that they used, (coughs) sorry,
they used augmented surfaces of miniatures
for aesthetic purpose.
They checked and compared the look and feel of the space,
and some participants said
that realistic human figure gave a sense of the scale
and it helped to decide the scale
and the size of the entire miniature space.
Moreover, the augmented surface were dynamic and realistic,
so they felt their prototype as more like high fidelity.
They were immersed in detail
in presentation of miniature space
to explore design objectives.
The last finding was that miniStudio helped
concretizing interactions of multiple components.
They could check holistic interactions
in macroscopic view of miniature space.
And really good physical miniatures
were testing defined events
also subjected to multiple miniatures.
Due to their interactivity,
playing with interactive miniatures
could also (mumbles) to develop interaction details.
Participants thought aloud or did role-playing
with their interactive miniature prototypes
for concretizing their interaction scenarios.
We found that there are further issues to be concerned.
The first one is imperfect sense of the scale
of the miniature space.
To reduce gap between view scale and miniature prototype,
magnified miniature view for immersive experience
or using larger mappers
and real-scale projection can be applied.
On the other side, miniStudio is regarded
as close to 2.5D.
Using multiple common projector systems
can complement this imperfect expression of 3D space.
The impact of miniStudio may be maximized
when it is combined with
or complements other design techniques.
As participants did, role-playing or video recording
is used to effectively concretize
and deliver ideas in the early stage of design.
And now miniStudio is integrated with Photoshop,
but other designer software can be considered
for effective event authoring and contents making.
In conclusion, we present designers
to for prototyping new proxemic interctions
in large-scale ubicomp space.
Our user study revealed
that the tools supported interactive prototyping
in the early stages of design.
And we expect this research
to inspire designers' prototyping
and future smart environment design.
And this is the end of the presentation.
And any question and feedbacks would be appreciated.
Thank you.
(audience applauding)
- [Announcer] Thank you very much for a lovely talk.
So, last opportunity before lunch
to ask some questions.
Come to the microphone,
say your name, where you're from.
And then, send us out
with a positive, constructive observation.
Aaron.
- [Aaron] Oops.
Hi, (coughs) I'm Aaron Quigley
from the University of St. Andrews.
Thank you for a very nice talk.
I was wondering,
do you think everything needs to be projected,
or do you think you could blend between some things
that are static, some things that are semi-dynamic,
and some things that are highly dynamic.
And is there any scope, do you think,
for actually incorporating other digital elements?
So I was looking at this
and thinking of putting in small screens
that could actually act in a more rich way.
So the projection, how important do you think it is
to have everything coming dynamically
from a single projection?
- This the big question.
Actually, the old miniatures do not have to be projected.
But, our prototyping system is focused
on interactive prototyping.
So, during the prototyping process,
the printed figure or printed image
can have a possibility to be changed.
So we concerned that at that point,
and also participants, actually designers,
participants, they liked the projected image
can be easily changed for focusing
on the look and feel of the space.
- [Aaron] But some of those shapes are constrained.
Like some of those body shapes
suggest one particular type of gender versus another.
So there are constraints already in there,
so it's not total freedom.
Like, I can't suddenly project in Chewbacca
that easily because I want my Chewbacca interface.
So I just question, where is the line, do you think,
between what's the most important thing
in this type of system to be dynamically-created,
projection-wise versus what could be static.
What's your expectation,
or what's your impression?
- So the--
- [Aaron] The most important thing that should be kept
as a projected dynamic.
- Most important thing is interactive prototyping.
So...
Actually we also considered in the tutorial session,
we let participants do some tasks.
So, actually the making of miniature figure
has taken about lower than five minutes.
So we think the regenerating the physical miniature
is reasonable in time.
And, actually our system is integrated with Photoshop,
so we focused more on the digital images
to be added to for changing,
for interactive prototyping.
- [Aaron] Okay, thank you very much.
- [Announcer] Okay, let's thank Han-Jong,
and all our speakers.
(audience applauding)