Hello, My name is Yujiro Okuya, working at Limsi-CNRS.
In this presentation, I will be talking about
how the use of a wall-sized display affects user collaboration for a design exploration task.
Large-format displays can visualize a number of data-set or a full-scale of data
that facilitates multi-users discussions or gives users an immersive experience.
These large displays are used in a variety of applications.
Industrial design review is one of the applications which often uses such large displays.
For example, large 2D screens can replace a traditional design discussion place,
where a multi-disciplinary team compares many design alternatives.
Or virtual reality systems allow users to review the usability of the car cockpit in a 3D environment,
instead of using a physical mockup.
In these workspaces, users can interact with digital mockups to assess the product design.
However, they cannot generate new designs and explore them.
Digital mockups are built by engineers with CAD software before the design review.
From our interview with researchers in an automotive company,
they prepare different design alternatives from CAD data.
Then project members review prepared design alternatives within the large interactive system.
Engineers then apply post-modifications on the CAD data
and prepare new alternatives based on annotations.
As experts cannot directly apply their ideas, miscommunication could occur,
which increases a number of iterations and development time.
Our previous study proposed a method to reduce this back-and-forth process
by merging these design and discussion steps.
In this previous work, users can directly modify the CAD data during the product review.
Here, a user is reviewing a rear-view mirror of a car in a 3D environment.
He can generate a new set of designs and explore them without asking engineers.
However, although the 3D environment is valuable for a realistic simulation,
a large number of project members cannot compare many design alternatives at once,
like a traditional design meeting practice.
Our current research question is
how the system can support design comparison practice for collaborative design reviews?
Based on the previous study, we designed a system
in which users can generate and explore design alternatives, by taking advantage of a wall-sized display.
With ShapeCompare, users can generate new designs with simple touch interaction and explore design history.
I will now introduce a set of interactions with ShapeCompare
then present the study we investigated how ShapeCompare affects users' collaboration.
ShapeCompare visualizes each design alternative on each screen on the wall-sized display.
At first, users can select the shape they want to modify by simply touching on a shape.
When it's selected, the background color of the selected shape turns green,
and the selected shape appears at the center columns on the above row.
Thus design alternatives below are the design history that users accumulate during the design exploration.
After the shape selection, the CAD part selection widget appears,
allowing users to select the specific parameter of the CAD model.
Let me explain how the selection widget works.
We designed this widget based on the usability study
to enable non-CAD experts to perform CAD data modifications.
The CAD data is defined with various internal parameters and constraints.
In our backend system, the CAD server loads the CAD data at the beginning of the session.
Then the CAD server links each 3D part of the mirror and parameters tied with the part.
For example, the curvature of the left-top corner and the width of the mirror are
defined on the corresponding part.
Based on this mapping, users can select the parameter by touching the window linked to each part.
Each window shows two shapes that are defined by minimum or maximum parameter values,
which helps users understand the shape modification without the parameter information.
When the user selects one of the windows,
the system sends the information of the selected part and design alternative to the CAD server.
For the moment, a number of shapes and parameter offset between each alternative are static values.
Then the CAD server generates design alternatives using the CAD engine.
The generated designs are displayed in the top row.
The shapes defined with lower or higher parameters are distributed from the central column.
There are some multi-touch interactions on ShapeCompare.
With a three-finger sliding gesture, users can scroll whole shapes to explore design history.
Users can rotate all displayed shapes with a bimanual touch-based interaction widget.
This widget was developed by Louvet and others to manipulate the 3D object on the wall-sized display.
They can also send the specific design to another display
where they can check the design in a realistic environment.
They can hold the shape down with a two-finger tap to export the design.
We conducted a user study to investigate the effect of ShapeCompare.
Particularly we investigated
how multiple design alternatives affect the collaboration between participants.
In this experiment, paired participants explore a design of the rear-view mirror
with negotiations on several design criteria.
Since it was difficult to invite real experts involved in an actual industrial design process,
we controlled users' expertise by giving individual design criteria.
They are strictly forbidden to share their own design criteria.
Therefore they need to negotiate to find the design that satisfies all design criteria both users.
The first specialist is like a designer who focuses on shape properties.
The criteria for the first specialist are the aspect ratio and asymmetricity of the shape.
And the second specialist is like an ergonomist who focuses on reflections of the mirror.
The second specialists pay attention to specific objects reflected on the mirror to avoid or to be displayed inside.
Also, they have to care for the size of the mirror surface.
Parameters of aspect ratio, asymmetric balance, and size of the surface are displayed above each mirror,
so participants can use the parameters to check each criterion on the shape.
During the task, we gave them a sheet of paper,
which describes a specific parameter range that they have to meet,
and specific objects that specialist 2 has to avoid or display in the mirror.
The experiment is a within-subject design with two factors.
The first factor is "Visualization."
We compared ShapeCompare to a single shape visualization called ShapeSlide,
which I explain later.
The second factor is "Shape."
Participants tested two tasks with different design criteria,
the one criteria guides them to reach a small shape, and the other criteria guide them to a larger shape.
We recruited 24 participants for this experiment.
The ShapeSlide is a technique that displays only one shape at a time.
With ShapeSlide, users can change the shape displayed at the center of the screen with a sliding gesture.
Also, they can go back to previous designs with an "Undo" button.
Originally, ShapeSlide is designed for a standard touch screen or a desktop computer.
For this experiment, we implemented ShapeSlide on the wall-sized display
to fairly compare the effect of shape visualizations without the bias due to the device difference.
We collected four types of data in this experiment.
The first one is Task completion time.
The second one is a number of shape selections,
which is counted when participants select or generate design alternatives.
The third one is questionnaires that assess a cognitive load of interaction techniques.
Lastly, we analyzed participants' conversations and gestures during the task.
We grouped their behaviors into specific categories.
There are three main categories.
The first one is deictic instruction.
This is counted when participants used deictic gestures to show something on the screen to partner.
We also divided into two sub-groups: pointing only one specific shape or range of shapes.
The second one is the design expression.
This is counted when participants expressed the design that they want to make
either verbally or with gestures.
The third one is the magnitude.
This is counted when participants quantified the sized of the modification to the partner.
Particularly, this event happened to the person watching at the partner who is modifying the shape.
Here I quickly show you the video of one of the participants to explain the video analysis.
For instance, she often points a specific shape to ask her partner about the design.
Magnitude is counted when the participants tell or ask their partners
how much larger or smaller they want to modify the part.
With ShapeCompare, participants often specify the magnitude with direction
since the design of the shape changes on the horizontal lines.
Expression words are counted when the participants want to explain the design that they want to make.
Participants use deictic instruction also for specifying the range of shapes
to show their constraints and limit the partner's choice.
Sometimes participants can also express their design idea using a gesture on the displayed shape.
We counted it as the expression gesture.
We formulated four hypotheses.
With ShapeCompare, participants can find the right design faster,
and participants can find the design with fewer selections.
Then participants use more deictic instructions to communicate.
Overall, participants prefer ShapeCopmare for communication.
As a result, participants finished the task significantly faster in ShapeCompare than ShapeSlide,
which reduced the task duration from about 23%.
It supports the first hypothesis.
In terms of results in a number of selections, there was no significant difference between the condition.
It does not support the second hypothesis.
This result indicates that participants explored more designs in a shorter period of time in ShapeCompare.
Results of the video analysis show that participants used more deictic instructions with ShapeCompare
than with ShapeSlide.
It supports the third hypothesis.
On the contrary, they used less Shape-related expression and Magnitude with ShapeCompare.
As a result of questionnaires, ShapeCompare was generally preferred
and perceived as more helpful for communication in comparison to ShapeSlide.
That supports the fourth hypothesis.
But how ShapeCompare helped their collaboration?
With ShapeCompare, the participants often used displayed shapes as references
for their communication.
They conveyed their design idea and limitations to their partner
by pointing a shape or specifying the range of shapes.
Also, they used the shape to ask for their partner's design intent.
It led to the increase of deictic instructions and the decrease of verbal
or gesture descriptions of their design idea with ShapeCompare.
In other words, the use of design alternatives could minimize communication costs
and increase common ground in communication.
It is helpful to bridge the gaps in communication,
especially between distinct experts who do not have common vocabularies.
While participants preferred ShapeCompare for communication,
some of them preferred ShapeSlide for interaction.
They liked changing the shape by swiping gestures without moving.
Also, it was helpful for them to notice a slight difference between shapes.
The current system is a research prototype, and there is a lot of space to explore and improve functionality.
For example, the system should support the customization of design alternatives.
Also, the system should reduce the computation speed of design alternatives.
To summarize, I presented ShapeCompare, which supports design adjustment of the industrial design
on a wall-sized display.
ShapeCompare allows users to modify native CAD data, which is especially valuable for non-CAD experts.
The results of user study showed that the visualization of multiple design alternatives
could increase common ground between different experts and help them to reach a consensus.
The proposed approach could also be applied in many other contexts,
such as generative design or physical simulations, for which many alternatives can be generated
by intuitively varying parameters.
Thank you for listening.
Feel free to send me emails if you have any questions.
I’d happy to answer them.