Hi all, Thank you for your interest in this work.
My name is Chunggi Lee from UNIST, South Korea and today I am going to talk about GUIComp
This work is a collaboration between researchers at UNIST and IBM Research.
Thanks for all of the co-authors and my advisor, Sungahn Ko.
As there are many apps with similar functions, graphical user interfaces can play an important role in attracting users.
Here, we have two example online shopping mall main pages. The right one is more clearer design and it has
consistent layout.
It can attract more users than the left one.
However, it is not easy to make good guis because of many design factors.
For example, we need to simplify much information on the small size of screen.
GUI designs need to be visually pleasing and creative to give positive first impressions to design users.
One suggestion for producing high quality gui designs is to perform iterative design process,
which is a cyclic process of designing, analyzing, and refining in the design.
While the idea of iterative design is simple and clear, doing the process can be hard if designers do
do not have any knowledge in design or have little design experience
For example, the novices in design can be easily overwhelmed by a blank canvas and may hesitate to rearrange their interface elements.
They may have difficulties in conceptualizing designs and harmonizing their design concepts and could
make simple mistakes during their design.
As such, there is a need to help these novices in design and GUIComp is the first attempt to help the design
novices in the GUI design domain.
Our work starts from a semi-structure interview, where we interviewed novice users to find challenges that they may face during GUI.
Before we started the interview, we asked them to create a GUI design for a shopping mall with an existing tool.
After codifying the interview, we find three major difficulties.
From the Grounded theory, we identified three difficulties.
First, novices are not good at conceptualizing designs, so they have difficulties in starting their design.
Second, they hesitate to fix their design, as they do not know which design elements are not harmonized.
Lastly, they do not know which part the viewers will see, so that they cannot decide which design parts.
From the interview, we derive three requirements for a system to help the novices.
Frist, we need to resolve difficulties on the most up-to-date design in progress. This means, the feedback for users also needs
to be updated to user canvas in less than a second.
Second, we should ensure that D1-D3 are met non-intrusively.
Third, the tool can be attached to a web-based prototyping tool for generality.
GUIComp is a gui prototyping assistance tool for helping novice users perform iterative design process.
The tool is a web-based add-on tool, works in real-time, and provide multi-faceted feedback.
Left figure is Kakao Oven. It contains thumbnail panel, canvas, and element panel.
Thumbnail panel which is A1 shows GUI pages that users created.
A2 is the canvas where users can prototype GUI designs by dragging and dropping gui elements from A3, the element.
The element panel has pre-designed buttons, texts, and images.
This is GUIComp and there are three panels for evaluation, recommendation, and visual importance feedback.
The evaluation panel provides the quality of GUI,
using visual complexity metrics such as element balance and alignment.
GUIComp recommends high quality example designs based on what is being designed in the user canvas.
Users also can filter the recommended designs based on categories and quality scores and acquire color information from examples.
The attention panel shows which parts or elements the viewers will focus.
Let's see how our GUIComp works.
As users make any medication on the canvas, the mutiation observation javascript library detects the modification.
Then, the GUIComp sends the modification to server to process type and attributes of the modification elements
Let's see how each panel works to produce feedback in the next few slides.
In this slide, we created html codes to have structures of images for recommendation.
First, we used the rico dataset. It has about 70,000 mobile app screenshots from 9,700 google play store apps,
but we only used 6,683 screenshots from 2,448 apps after manually filtering.
We then decomposed the images into several elements have structures of each image.
The structures are used to search examples for recommendations.
To reduce the search space for recommendation,
we used auto-encoder to compress images to latent features.
There are two parts in the auto-encoder which are encoder and decoder.
The encoder compresses the image representation to small latent features,
while the decoder decompresses the latent feature to the image representation.
The loss function is used to minimize the difference original image and reconstructed image.
We used KNN to find examples that are similar to what users is creating.
As users modify the design in the canvas, the recommendation panel calculates distances of the current design on the canvas
and the designs in the dataset and selects k neighbor images as shown in this figure.
We used the mobile GUI evaluation metric, proposed by Riegler and Holzmann, which allows real-time processing.
The inputs are html codes and the design on the user canvas.
We slightly modified the original metrics scale and their names to help users intuitively understand.
For example, we changed the name of element smallness to element size and adjusted the 0.5 is the best score.
This is an example metric for calculating element size.
It calculates average elements' size and screen space and tell users whether the elements sizes are appropriate or not.
In the attention panel, we provide the user design on the canvas and used the FCN model to solve the segmentation.
The model compresses image features, conducts the convolution operation with the features, and
reconstructs the compressed feature for the deconvolution operation to make image segmentation.
There are several fcn models for different deconvolution operation points.
For example, the fcn-32 model reconstructs features from 7 x 7 size features
while fcn-16 model reconstructs the features from 14 x 14 size features.
The fcn-16 model is more sophisticated than fcn-32 model,
so we used FCN-16 model to have better results.
We have three research questions.
First, Does GUIComp help users make the design closer to an acceptable design to general users than Kakao Oven?
This means, the novice users can reduce the basic mistakes during the design process.
Second, What UI/UX benefits does GUIComp provide to users?
The last one is how do users use GUIComs's feedback to overcome the difficulties during prototyping?
It means does the GUIComp perform iterative design process?
To answer the questions, We recruited 30 novice users and gave two tasks.
The experiment was a between-subject study, where users were asked to design two user interfaces.
The control group used kakao oven only, while the experiment group used kakao oven and GUIComp.
We used an eye-tracker for gaze analysis.
After the experiment, the participants were asked to fill out post-experiment survey,
that includes efficiency, fulfillment, and effectiveness of GUIComp.
We also measured satisfaction and preference scores.
To investigate the quality of the produced GUI designs,
we conducted an design evaluation session on the Amazon Mechanical Turk.
In session 1, we recruited 26 MTurk workers for evaluating perceived design quality at 0-10 scale
without rubric which is general users' subjective assessment scores.
In the session 2, we recruited 21 other MTurk workers for evaluating design quality
with rubric which gives minimum guidance to online workers.
The rubric was assessment criteria from the evaluation panel.
The results indicate that GUIComp users produce designs that are more acceptable to general users
regardless of the task type corresponding research question 1.
This figure shows dot and whisker represent mean and 95% confidence interval, respectively.
The blue and orange mean kakao oven and guicomp, respectively.
The result was GUIComp help users make more acceptable designs in terms of perceived quality .
There was no significant difference in the amount of time spent for design.
The second result showed that using GUIComp is enjoyable, satisfactory, and affordable in terms of usability.
We also find that there were significant difference
in the satisfaction, efficient, fun, fulfilling, effective, and comfortable scores.
Lastly, users employed multi-faceted feedback for overcoming difficulties in the iterative design process.
Gray, red, green and blue represent gaze of users in kakao oven, evaluation, recommendation, and attention, respectively.
We think the interaction between recommendation and
kakao oven is prototyping, kakao oven and either evaluation or attention panel is testing.
Refining is the opposite direction of testing.
From this gaze analysis, we see the many iterative design process.
Our first finding was that users frequently used recommendation panel in the early stage until the middle stage.
Also, users mainly used evaluation panel in the later design process.
Lastly, the attention panel used in the several stages.
Lastly, the users enhance the design quality after viewing the evaluation panel.
Representative iterative design process was P3 found related on example and decided to follow the example below area.
P6 also shows an interesting design pattern.
After she was inspired by an example, she decided to use the sales promotion image such as 50% off.
Once she put the image, then she checked the attention panel if viewers will focus on the promotion image.
Here are our lessons learned from our experiment.
First, we need to give additional feedback for 'why' for justification.
For example, when providing which parts negatively affect the design quality or
scores can help users better accept the feedback.
Another example is that P14 questioned why her overlapping design decreases the overall score,
although she thinks the overlapping design looks better to her.
Lastly, users want to lead their design using creativity and would have appreciated with some automated corrections.
We designed, implemented and evaluated GUIComp to assist users who have no and little design experience.
GUIComp provides three types of feedback which are dynamically updated as users continuously make changes in their design.
The experiment results indicate that GUIComp allows users
to create more acceptable mobile GUI designs with more fun, fulfilling, and satisfactory experiences.
This is end of my page. Thank you for listening.