Hi, I am Kazuki Takashima from Tohoku University, Japan.
I am the second author of this paper, "ModularHMD: A Reconfigurable Mobile Head-Mounted Display Enabling Peripheral Interactions with the Real-World".
This work has been led by our former student Isamu Endo, and conducted with our colleagues, Maakito Innoe, Kazuyuki Fujita, Kiyoshi Kiyokawa and Yoshifumi Kitamura.
VR HMD is now expected to be used for mobile work contexts because it can easily create a personal and mobile workspace.
However, current mobile HMD uses full immersive cowls, which creates two types of boundary issues.
The first issue is that the HMD users cannot see physical objects and devices without removing headset. Such ad-hoc physical device interaction during VR experience is required for many of office scenarios.
Second issue is more about social context. Non-HMD users cannot see the face of HMD users, which makes communication between them quite difficult. Additionally, non-HMD user cannot quickly join the ongoing VR interaction with HMD users.
So, current HMD configuration has difficulties for ad-hoc real-world interactions.
To address these issues, many researchers have proposed solutions.
MDI uses transparency controllable LCD for its side and bottom parts of the HMD, allowing the user to directly see the real-world objects without removing the headset.
but it cannot support any ad-hoc collaboration with real-world.
FaceDisplay is designed to support collaboration with real-world partners by installing three touchscreens on it back.
This enables collaborative touch interactions between users beyond the HMD boundary.
But, it does not support ad-hoc physical interactions with objects or devices.
So, there is no coherent solution to support ad-hoc interactions with physical devises and partners.
To make HMD functions more flexible, we focused on modular mechanism.
Modular smartphone is an interesting concept where a smartphone device can be divided into several copies and one of them can be physically rented for partner.
This is a convenient way to quickly manage cross-device content sharing between owner and users.
We introduce this idea to flexibly control barrier of the HMD cowl and support collaborations with others.
So, we propose ModularHMD that allows the user to perform ad-hoc interactions with real-world objects and partners.
The main feature is installing three removable & reusable display modules at the side and bottom of the central screen.
As a basic function, side and bottom modules display extended virtual views, which will make a larger FOV experience.
Each module can be set to video see-through mode if the user wants to get situational awareness of the surroundings.
In case the HMD user requires physical interactions with the real-wprld, these module can be physically removed from the HMD and then reused as an interaction device.
So, we implemented a prototype using a stereoscopic mobile display as the central screen, three ipod touch for each module, and many of 3D printed parts.
It eventually has six lens in total. All lens and display arrangements were carefully designed to make a large FOV yet user’s eye movement range is still within the comfortable viewing angle.
The side and bottom modules are on the slider holders.
To remove the module, the user slides inner lens first, and then put the module out from the holder.
The user can wear ModularHMD just like a regular HMD and can easily see side module.
Using this prototype, we designed several interaction scenarios.
As the 1st scenario, we show its fundamental HMD’s capability. Here, the user wants to be immersed in a virtual world.
Here, side and bottom modules display the extended images of the frontal VR content, increasing user’s immersive feeling.
In the 2nd scenario, the user wants to take notes using a physical pen during VR experience. The user can see the note and pen by making the bottom module video-see-through mode.
The 3rd scenario also represents a case of increasing situational awareness. By using video-see-through mode at the side modules, the user can recognize a person nearby and estimate their intentions.
The 4th scenario is solving the boundary issue with physical devices, the user receives a chat message and wants to quickly reply.
The user can input the text by removing the bottom module and immediately reusing it as a touch keyboard.
The user can directedly see the keyboard and own fingers through the bottom opening slot, which supports accurate finger manipulations while keep looking at the VR content.
Once the typing is over, the user can quickly come beck to the original VR experience.
The last interaction scenario is collaboration between HMD and non-HMD users. This movie now shows the HMD users perspective.
A partner approaches to a HMD user who is now modeling a 3D content and they want to perform a quick collaborative content editing.
To do this, HMD user can share his current content quickly by physically lending the side module to the partner.
Here, partner can also join the interaction through the module’s touch screen, and his input can be also visualized in the VR view.
The HMD user can see the facial behavior of the partner through the side opening. If the collaboration is over, he can come back to the personal work by putting the module back in the slot.
Next, this video now explains partner’s perspective in the same scenario. The partner asks HMD user and gets a side module.
The partner can immediately see the content on the module and use its touch screen to give instructions.
Both user can exchange eye-contact and understand how they think each other while collaboration.
This interaction creates distributed cross-device workspaces and necessary face disclosure by a single set of modular actions.
We performed a user study to understand the fundamental benefits of our MouldarHMD. We here show some highlights.
First, we checked immersive VR experience, and found additional extended view were good but not significant for thier immersive experience.
We discussed the need for minimizing physical gap between the lenses.
Second, we checked ad-hoc text typing during VR experience. Our physical interaction via our modular actions offer high usability and significantly faster text typing.
We recommend this mode for mobile work contexts.
Third, we checked how the side opening slot helps their facial expression exchanges. We found that it is good for making eye contact, but not good enough to understand the facial expression.
We discussed possible extensions such as installing a small light to illuminate the HMD user’s face.
Last, we checked how MoudarHMD’s ad-hoc collaboration works. We confirmed that it provides high usability and smooth cross-device interactions.
In this work, we proposed ModularHMD, a new mobile HMD concept for ad-hoc physical interactions. And its unique benefits were confirmed by our prototyping and studies.
Here are limitations and our future work. Thank you very much.