Using parameters to control the appearance of some sort of visual output is an extremely
common editing paradigm.
When editing photos, you use a set of standardized parameters (such as exposure and contrast),
shown here in Lightroom on the right, to change the appearance of the photo.
When making a particle system in Unity, you use a much more complex, but still finite,
set of parameters to define the look of a particle system
And many modern content creation tools allow experts to define their own custom parameter
spaces to show to users. The parameters used for this cloth material are not the same ones
that would be used for a brick material
All of these applications share a common parameterized design editing method. Conceptually, the parameters
define a high-dimensional design space where every point in that space maps to a visual output.
As with other types of design, creating a design in a parameterized space is an iterative process.
For example, when creating a “futuristic” font, solutions that appear to be going in
the right direction early in the process may actually be dead ends, as the design goals
get continuously updated during the process. Many of these goal updates are often driven
by personal preferences.
While the design process can seem somewhat random, it is often thought of occurring in
three phases, shown here with some 2D representations of a high-dimensional space: preliminary design
exploring all possibilities in the design space, refinement, taking a design that has
desirable properties and generating alternatives, and detail making
small tweaks to finalize a design. A real-world design process often requires fluidly switching
between phases due to changing design goals.
In order to best support this design process in parameterized spaces, we build the design
adjectives framework according to three principles:
Provide support for guided exploration based on personal intent, allowing users to create
variations of current designs and easily explore the design space
Maintain access to low-level controls, as the detail phase is a fundamental part of the process.
And ensure that all actions taken in the interface resolve at interactive rates to support fast ideation.
Design adjectives is a software framework that enables interactive model-guided exploration
of parameterized design spaces. The framework is built around a model of intent, called
a design adjective, that can represent a design goal based on a small number of user-labeled
examples. Other components interact with the model by sampling from it, arranging the returned
samples, providing initial input to the adjective, and allowing for detail design with access
to low-level parameter selection tools at any time.
We present a domain-agnostic implementation of the design adjectives framework that uses
Gaussian Process Regression to represent design adjectives. This implementation works with
any bounded parameterized design domain.
Before describing the framework and our implementation,
let’s take a look at what kind of interactions this framework enables.
In this toy example, I am using our domain-agnostic design adjectives interface to create a yellow
brick texture. I first assign high scores to yellow brick textures and low scores to
any other color. I then sample from the newly defined “yellow brick” adjective, which
returns a gallery of designs to browse through. I then assign scores to some of the returned
designs to refine the design adjective, and repeat the sampling process.
Note that the sliders can be used at any time, and update their highlighting when the adjective is updated.
Next, let’s take a look at the framework components that enable this kind of interaction.
The design adjectives framework is built around a design adjective. A design adjective represents
a design concept from user-scored input examples. It outputs a numeric score representing how
well an arbitrary input matches the current adjective’s design goal. Adjectives are
trained and updated in real-time.
Navigating through the design space is done by sampling designs from the design adjective.
Users can move through the space relative to the score of the current design by using
simple operators, such as “similarly scored designs”
Designs returned by the sampler are arranged in a design gallery to let users browse through
and score designs for adjective refinement.
The components described so far have been working with an adjective that already exists.
A newly created adjective has no scored examples to sample from, so we need a way to generate
initial samples. This is handled by a bootstrapper, which generates such designs.
Access to the original parameter space is preserved, as designers often want to fine-tune
aspects of their design. Per-parameter selection tools, such as slider highlighting, can be
added to aid this fine-tuning process
Our domain-agnostic implementation provides implementations of each of these components.
We use Gaussian Process Regression to implement a design adjective.
Gaussian Process Regression is a machine learning technique often used to learn preference functions
in low-data environments. This has been used with success in prior work in computer graphics
when learning preference functions from small amounts of user data.
Given an input, Gaussian processes return a mean and variance value. In this implementation,
we discard the variance and treat the mean value as the preference score. We can input
any design in the space and get a preference value from the adjective.
Note that the preference score assigned to an example is not unique. Multiple examples
will have similar scores. This is great for us, as we want to be able to show designers
a lot of similarly scored things to help refine their design idea.
Next, we need to generate design suggestions from our design adjective. We do so with a
guided rejection sampler.
The idea with this guided sampler is that we know which points in the space are already
known to be good, so if we start with that example and randomize some of the other parameters,
we’ll be more likely to find things that the user will be interested in. Conceptually,
this is similar to taking an existing design and modifying it to explore alternatives as
part of the refinement process.
Here’s what that looks like. First, we’ll select a random high-scored example.
We need at least one of these to run the sampler, so the user needs to provide at least one
high-scored example to get started.
We’ll take some of the parameters in the selected example, and randomize them using
a uniform distribution.
We’ll then take the new parameter configuration and use the current adjective to score the result.
We then accept or reject the sample according to user-specified criteria.
If accepted, the example is added to the set of designs displayed
to the user, and if rejected, the process repeats from the top.
There are a couple details of the sampling algorithm to make sure returned designs are
diverse and get generated in a reasonable amount of time, please see the paper for additional details.
To provide user-directed control over the returned samples, we allow the user to change
the acceptance criteria. Different acceptance criteria enable different methods of traveling
through the design space. The system provides four criteria: Towards, away, nearby, and axis.
Please see the paper for details about these criteria.
When samples get returned from the sampler, we arrange them in a design gallery.
We display a simple gallery of samples sorted from best adjective score to worst adjective
score. From this interface, users can browse through the results visually, and easily provide
feedback to the model to update its definition.
Everything I’ve talked about so far related to the adjective assumes that we have defined
an adjective to work with. At the start, the definition is empty as there are no labeled
examples. We need a way to initialize the adjective with some labeled data, and we use
the framework’s bootstrapper component to do this.
For generic domains, we find that a simple uniform random sampler is sufficient. If we
were working with a specific domain, we may be able to provide better preliminary samplers.
Finally, we provide a number of tools to help designers manipulate individual parameters
during the detail design phase.
One such tool is parameter highlighting. We highlight sliders to indicate the most impactful
parameters in a manner similar to Koyama’s prior work that does the same with a crowdsourced
preference function. We exchange the crowdsourced function with the design adjective model.
Our other parameter selection tools are described in our paper.
We evaluate the design adjectives system through a series of user studies.
We ran two sets of studies to evaluate the design adjectives framework: two case studies
with professionals, and a user study where novice to intermediate designers performed
simple design tasks.
In the professional case studies, a graphic designer and a material designer used the
tool for unstructured design tasks in their respective domains. Each designer spent 8
hours with the design adjectives interface. Some of the fonts created by the graphic designer
are shown here.
The professionals provided feedback through interviews. They found that the framework
was extremely helpful for design concepting, and felt that the system sufficiently captured
the intent of their design concepts.
The novice designers were given three 10-minute design tasks. The first two tasks used the
adjectives interface, and the final repeated the first task with a per-parameter baseline
interface. Participants filled out surveys measuring how they felt about the design task
they just completed, and interface logs were recorded. The tasks were simple directed design
tasks. For example, users were asked to make the font shown here look more “futuristic".
From this study’s survey results and interface logs, we found that users preferred design
adjectives for performing exploratory design, that adjectives were perceived to sufficiently
represent design concepts, and that the design adjectives interface was more enjoyable
to use for these tasks.
Please see the paper for complete evaluation details.
In summary,
We presented the design adjectives framework for building parameterized design interfaces
based on user-guided exploration, and presented a domain-agnostic implementation that uses
Gaussian Process Regression to represent design adjectives. We hope that this framework makes
it easier to build and customize principled interfaces for parameterized design.
Thank you for watching! You can find additional information and code about the project at
the listed URLs, and can contact me, Evan, at the email listed.