Hello, I'm Shunan Guo
Welcome to my talk
Today I’m going to present an intelligent graphic design system
for automatically generating advertising posters.
This research was a joint effort of several other members in the
intelligent big data visualization lab of Tongji University
Advertising posters are a form of information presentation
that combines images and overlaid text
They are commonly seen on the streets, metros, and webpages.
Designing and creating an attractive advertising poster can take a lot of efforts
Take this pizza poster for an example,
the designers are usually given a product image
as a starting point.
Then they need to search for design elements
that matches with the product image to complete their design,
such as the background image and the embellishments.
They also need to determine the sizes and the positions
of the embellishments and texts.
Note that this workflow is never a one-way path,
designers need to move back and forth
to iteratively make decisions between design elements
which makes this design process even more tedious
Moreover, designers are usually required to create
multiple design variants for the same product
such as posters in different styles and sizes
to cope with different posting environments
To make this design process easier for designers
we developed an intelligent design system called Vinci
for automatically generating advertising posters.
The system takes a user uploaded image
and a series of text inputs to generate
a series of advertising posters in a ranked list by design quality
In the middle is the poster editing panel,
which allows the designer to
make tweaks of a selected poster design
It supports a variety of common editing functions
and also provides recommendation of
alternative design elements
that are similar to the user selected one
Here we can see the user selects an embellishment
and in the left there is a list of embellishments
that are recommended for replacement
After finishing poster editing
the user can click on the “update” button
and there is an back-end editing-feedback mechanism
that will automatically analyze the editing operations to learn
user’s design preferences and update the generated results.
Finally, the user can choose the satisfied posters
and download them by clicking the “download” button.
To train the generation model
our first step is to characterize the design of posters
by constructing a four-dimensional design space of posters
that includes the key design elements
Next, we try to factorize the design behavior of designers
by constructing a design sequence on the design element
After doing so, we shall have a library of design elements
decomposed from our training design posters
and a collection of design sequences
representing poster designs
The design sequences are then sent to a Seq-2-Seq VAE model
for training and generating new design sequences
After that, the design sequences are reconstructed
back to the design elements to form a real poster.
To evaluate the design quality of the generated poster
we also trained a estimator for rating the posters
based on ground-truth ratings from real designers,
and finally the highly ranked posters will be
sent back to the user as output
To evaluate the performance of Vinci
we conducted three user experiments to evaluate
the performance of the model
and an interview with designers to collect subjective feedback
In our first experiment,
we asked participants to compare designs that are
generated randomly and those are generated from our model.
We found that participants have a significant preference
on the model-generated design
and in particular they like the consistent color scheme
and design context between the
selected elements with the product image.
We also found participants generally agrees
with the judgement of the estimator
and have a strong preference towards high-rate designs
because they see less element overlap.
In our second experiment, we test the overall
design quality of posters by asking the participants to rate
designer-created posters,
posters generated by a commercial state-of-art tool
developed by Alibaba group called Luban
and the posters generated by Vinci
and asked them whether they think
the posters are generated by human or machine
Without any doubt
posters from real designers received highest ratings
However, we found significantly larger percentage of posters
are perceived as human-created and in Vinci
comparing to Luban
while at the same time they also received higher ratings
This indicates that, our system outperforms the state-of-art tool.
In the third experiment, we test the performance of
the online editing feedback mechanism
by asking the participants to change the orientation of a poster,
which usually requires tedious element moving and resizing
The study result shows that this feature can significantly
accelerate the poster editing process
by memorizing users editing preference
If you are interested in our work,
please check out our paper for more details
Thank you for your attention,
and I'm happy to take any questions.