Hello, my name is Ken Nakagaki, I’m currently an assistant professor at the University of Chicago.
In this talk, I’m presenting (Dis)Appearables, which is one of my last research projects at the MIT Media Lab, where I completed Ph.D. last summer.
I’m presenting the work on behalf of my amazing collaborators at the MIT Media Lab and the University of Southern California.
The starting question of this research is quite simple. ‘How can we make physical objects Appear and Disappear?'
While making graphical objects appear and disappear on screens or VR headsets became easy, doing that for physical objects or tangible user interface system is still difficult.
While prior research have preliminary explored such an expression, either using motorized mechanism or inflatables, they have limited expression with the fixed mechanisms and configurations.
In our work we were greatly inspired by prior work in theatrical performance or HRI research to design appearing and disappearing expression by utilizing surrounding stages.
So, in our research we introduce a novel approach to design Actuated or Robotic Tangible user interfaces in conjunction with, ‘Stages’.
Stages are surrounding environment of the robotic hardware.
With stage, we open up a novel opportunity to make physical objects and user interface elements dynamically appear and disappear.
This allows a number of interaction design, physical expression and applications that greatly enrich our physical environment.
This paper is an extended research based on our previous paper HERMITS, where we have introduced the concept and approach of ‘Mechanical Shells’, using tabletop robotic hardware, toio.
While our (Dis)Appearables implementation is based on the same technical platform, this paper explores the remaining component of the system which is the ‘Interaction Stage’ — to augment the interaction and expression of the Actuated TUI system.
This is an example overall design of (Dis)Appearables.
By extending the stage of HERMITS to have front-stage and back-stage, now the actuated TUIs can transit between the front and back-stages,
.. so that they seem appearing and disappearing from users’ perspective.
As shown in this diagram, with the duality of front and back-stages, it opens up a whole new interaction design opportunity for the physical hardware system.
Let’s break this down into several components.
Firstly, the Transition Portals are the essential element in the design of (Dis)Apearables.
They are basically path ways for the robotic hardware to transit between front and back stages.
We have prototyped a variety of portal designs, such as this curtain,
actuated doors,
or even a lift for the robots to appear from the floor.
In our paper, we categorized the portals into 4 quadrant design space, where each of them can be utilized depending on the use case and applications.
These transition portals facilitate the expression of appear and disappear.
We also developed this design space for the rich expressions or illusions that our approach can convey.
For example, by combining the basic effect of Appear and Disappear with Graphics (like display or projector), the system can create expression of transition between pixel and physical.
OR, by combining multiple robots, to make robot disappear from one place and appear another robot at another place simultaneously, it can create illusion of teleportation.
These expressions are later demonstrate in the application section.
So that is the transition portals that facilitate appear and disappear effect.
Back-stage allows the system to intentionally hide certain processes, which may better be not seen or intervened by users for interaction design purposes.
Such process may include reconfiguring and assembling with other modules, as preliminary demonstrated in HERMITS,
or even digital fabrication machines themselves could be embedded in the backstage to fabricate different shapes.
With this robots can pick up from the 3D printer, and bring to the front stage for users.
So that is the core idea of (Dis)appearables, composed with Front-Stage and Back-Stage, connected with Transition Portals.
For the implementation, we have prototyped a stage design and control pipeline.
In the Plan and Design Stage, we have developed a GUI tool to design and iterate Stage model where they can define different properties of the stages.
The design is visible in 3D for users to view and simulate the stage, then the data is exported to be used for both fabrication and robot control on the stage.
Based on the stage data, users can fabricate the stage using different fabrication techniques.
For controlling and navigating multiple robots on the stage, we have employed MAPF: Multi-agent Path Finding to automatically generate paths from designated origin points to target points.
This is a technique and research well-developed in action planning for warehouse robots, and the detail can be found on our paper.
Based on our prototype, we demonstrate potential applications our approach enables.
First (Dis)Appearables can be used to enhance the interaction with data, where users can switch between GUI system or TUI system by allowing the robotic hardware to snap to graphical system on-demand.
Secondary, (Dis)Appearables gives novel opportunity for everyday affordance design.
For example, it can make your phone disappear, rather than turning off the notifications.
As our phones have become the most distracting thing in the world, such an radically new take on affordance design may be effective.
Accordingly, different types of user interface elements can appear and disappear, to design different types of physical affordances.
Next application is for remote communication or gaming.
Here you can see two users enjoying remote physical hockey, where a puck would disappear from one side and appear on the other side, to represent the continuity of remote space.
Lastly, (Dis)Appearables can also be utilized for playful and expressive application in physical gaming and storytelling.
In our paper, we introduced the concept and method of (Dis)Appearables by designin. g surrounding ‘stages’ for Actuated Tangible Interfaces.
We believe this research is literally a ‘Stage’ for many other researchers to explore or perform future research opportunities.
Such future directions include Stage control and design, user evaluation, and designing ‘stage’ for other actuated user interfaces.
Lastly, I would like to thank my co-authors and collaborators, of course including my advisor Professor Hiroshi Ishii.
Please feel free to reach out to me for any questions or inquiries. Thank you.