Hello my name is Radiah Rivu and i am a PHD student at the Bundeswehr University in germany
Today, I want to present our work on Bi-3D: bi-manual pen and touch interaction
for 3D manipulation on tablets. This project has been a joint effort from
researchers at Aarhus University, Bundeswehr University, and the LMU.
Bi-3D is a novel interaction technique for 3D manipulation on tablets. It combines pen and touch
gestures to form 3D commands. This can be useful in many scenarios. For example, to sketch in 3D.
To perform drag and drop in 3D. To rapidly create new geometric shapes. Or for extrusion modelling
of wireframe models. In this paper we contribute the design, application, and evaluation of Bi-3D.
3d manipulation is a fundamental task to 3d design and development
This is involved in 3D model design, as well as content for head mounted VR and AR displays.
There are two families of techniques for the manipulation of 3d objects. Techniques based on
separation of degrees of freedom. And techniques based on integration of the degrees of freedom.
On tablets, the separation approach mostly means I am using a widget that provides
handles for each dimension. This allows precise manipulation of each dimension.
Integration is when I would control multiple dimensions at once. This can match more closely
to some of the higher dimensional tasks that 3D manipulation affords.
In the years before 2010, several research papers investigated integration techniques.
Many of the techniques employ multiple touch contacts. For example,
dragging both fingers translates it in screen space, and pinching moves it in the depth dimension.
However these are designed for large displays while tablets have little.
Also, in most applications, touch gestures are already employed for navigation. For instance
1 finger to rotate, 2 fingers to move in depth, and 3 fingers translate in screen space.
Meanwhile, a parallel line of research has made significant progress.
Over the last decade, research explored a variety of application areas for pen and touch interaction.
Both modalities can be combined in empowering and complementary ways. These works focused on 2D
graphical environments, and we extend the prior art with an investigation for 3D manipulation.
in this work, we introduce bi-3d. It is an interaction concept for tablets that can
be used in a variety of 3d manipulation tasks. At its core, it is based on how we intuitively
use the zooming gesture and a pen. Pinch is typically used to navigate the scene into the
depth dimension. The pen itself is used to drag and drop in 2d space. Bi-3D is the natural combination.
When moving an object in 2d space, a pinch gesture moves the object in the third dimension.
This enables an easy and fast way for 3 degrees of freedom manipulation.
As 3d manipulations are diverse, we explore the application
design space of the concept. In particular we see two distinct application areas:
Navigation plus manipulation, and manipulation times two. In the former, the zoom gesture will
perform the expected command, to navigate the canvas camera. In addition, the object held with
the pen moves with the camera. Thus the object’s position is manipulated when navigating the scene.
This is suitable for extending 2d apps like microsoft paint.
For example, a parallel pinch will directly extend sketching to 3d.
As an example, this allows one to quickly create 3d shapes like this dynamic spiral form.
It can also be useful in a straight line drawing mode for wireframe models.
For example here the user draws a larger shape through connecting several lines together. The user
can seamlessly transition between drawing 2d lines; and then drawing lines that project into depth.
The second type is useful for 3d object manipulations. Touch inputs directly affect
the object under the pen tip. But touch inputs do not affect any camera navigation.
Thus both hands manipulate the object that is in selection of the pen.
This can be useful for example to precisely drag and drop an object within a single pen stroke.
As well, to extrude and edit an existing shape.
Such as when rapidly designing the 3d outline of this stairs model. Here the user performs
several extrusion commands in various 3D directions to rapidly create a prototype.
We support further task combinations by moding through the amount of touches.
For example in this scenario, one finger input changes the size of an object’s face. A user can
successively apply this method in the creation of models, for example here for each part of a table.
In a user study, we have evaluated the base technique of 3d drag and drop in an object translation task.
Our goal was to understand the pro and cons across the range of degrees-of-fredom separation
and integration. We compare the proposed Bi-3D technique to a separation method
where a widget allows you to change each axis separately which is standard in many apps.
And to a fully integral method where users can control all three dimensions at the same time.
here the user points in mid air to a target and then presses a button to select it afterwards
the user can translate the object in 3d space. For this technique an optical tracking system was used
to precisely track the motion of the stylus above the screen. The results were promising. Users were
faster with our proposed technique than the widget. Thus our technique was faster than the standard
tool used in many apps. There was no difference to the mid air method. This indicates that our
proposed two handed method fares similar to fully integral mid air interactions. The analysis on
indicates that users indeed did employ more integral input with the two-handed approach
than the separable widget. Regarding user feedback, users reported that the widget is complicated
in 3d drag and drop, while users found it easier with Bi-3D and the mid air method.
Before we summarise, we want to point out a few limitations. First of all, many of our proposed
techniques need users to coordinate their two hands. While this is easier in simple tasks like
drag and drop, it is more difficult for tasks that combine rotate, scale, and translate tasks.
Also our application examples are mostly based on a menu that switches between different operations.
Ideally, users have one input language that can cover most of the tasks.
Lastly the Z manipulation by pinch to zoom allows precise input,
but is not as precise and perceivable as using the pen for manipulation in screen space. For example,
in the video you can see that a uniform star was intended but the result was somehow skewed.
In this work, we introduced Bi-3D as a concept for dynamic bimanual interactions in tablets to
facilitate the operation of 3d manipulation tasks. This opens a rich design space for techniques
to operate a variety of 3d manipulation tasks. We showed that this can significantly improve
user performance for the fundamental 3d drag and drop task. Interesting future directions include
exploring mid air techniques above the tablet, and integration into augmented reality environments.
We thank everyone for their attention and are happy to discuss further questions.