Welcome to this talk on KnitGIST,
A Programming Synthesis Toolkit for Generating Functional Machine Knitted Textures
I'm Megan Hofmann, and my co authors are Jennifer Mankoff and Scott Hudson.
We contribute to the burgeoning field of algorithmic machine knitting,
which is largely based on the work of McCann et al presented at SIGGRAPH 2016,
where they develop the compiler for a machine level language for
industrial knitting machines like the one pictured here.
This can be combined with higher level programming languages
such as those presented by Kasper et al and, myself, Hofmann et al at UIST 2019.
This serves as the back end to this work.
Past research has largely focused on creating knitted shapes,
such as knitting a Stanford bunny or
creating soft actuated objects like this lamp shade.
But we contribute to the area of knit textures,
which determine the mechanical and aesthetic properties of knit fabrics.
Knit textures can curl up on themselves,
or shrink and stretch in horizontal and vertical directions.
These textures can be created using KnitSpeak, a programming language for knitted textures
based off of hand instructions like those shown here.
This presents an opportunity to generate complex textures tailored to a designer's purpose, such as:
designing swatches with target physical properties,
creating handed patterns with target aesthetic properties,
and combining different properties to create functional and attractive objects.
This work builds on the stochastic program optimization technique
defined by Schkufza et al, which is a super optimization programming synthesis technique
that searches a space of non-looped programs, such as KnitSpeak,
and is effective and non-convex, discrete search spaces.
We present KnitGIST: a generative instantiation synthesis toolkit for knitting,
which produces complex textures based off higher level goals defined by a programmer.
It applies concepts from program synthesis, and
it uses an extensible API which is amenable to a variety of domains.
The framework can implement a range of optimization techniques,
such as hill climbing methodologies and genetic algorithms,
by mixing and matching functions from our extensible library.
It's currently a programming interface
but it is structured to support constructing interactive tools in future work.
For the rest of this talk will present
an overview of the framework and the optimization loop at its center.
We'll follow this up with a description of heuristic maps,
which are the core contribution of this work.
Heuristic maps are composed of objectives and tactics.
We will then provide examples of how those are implemented.
Finally, we'll provide a demonstration of how you program
a heuristic map in order to create a higher level object.
The KnitGIST framework consists of
a library of synthesizers, objectives and tactics
which are combined by the programmer in a weighted mapping to create a heuristic map.
And a synthesizer which generates random textures based off of
width and height parameters defined by the programmer.
We start the loop with the N-best population of textures generated randomly by the synthesizer.
We score each of the candidates by their objectives,
and then select a candidate with a bias towards the one that is
currently performing the best. This is to ensure that we don't
necessarily jump into local maxima with each iteration.
If that candidate meets the programmers requirements,
then we're done and we can knit our object.
Otherwise, we choose a tactic that will modify the texture in a small way
and then apply that tactic to the candidate texture.
This will create a new texture which we then feed into the population.
The objective function is defined by a
heuristic map which is a weighted mapping of objectives,
which measure how well a given texture performs on a given metric,
to tactics, which are small operations on a texture
which produce a new texture.
Objectives are weighted based off of their value to the designer.
So in this scenario, the "objective 3 style"
is weighted more highly than the objectives of symmetry and curl.
Tactics are weighted by how likely they are to improve a given objective score.
So in this scenario, the mirror stitch tactic is more likely
to improve the curl objective than the flip stitch tactic.
You can map tactics to multiple objectives.
Notationally, a heuristic map is defined by a set of objectives, O,
each of which has a score, s(o), that is between 0 and 1 on a given texture.
1 being matching the objective and
0 being not matching the objective.
Each objective has a weight defined by the programmer, Beta_o.
And we also have a set of tactics, T, and a weighting alpha_(o to t)
which is between each objective and its tactics.
In order to select tactics, we first have an objective function
F(O), which is simply a weighted sum of
the weights of each objective times it's score.
We define the value of a tactic for improving an objective as
the proportion of the weight of that objective to that tactic
over the sum of all of the weights of that objective to any tactic.
We also define the importance of improving any particular objective as the
weight of that objective modified by
the difference between one, the maximum score, and the current score,
divided by the sum of all the weights of any particular objective
times its current score.
We combine these values to create an expected improvement score or an EIS score
for any particular tactic over the whole set of objectives.
This is simply the sum over every objective of the value of the tactic times the importance of the objective.
Conceptually, a greater EIS score implies
a tactic is more likely to greatly improve the objective function.
We select tactics with a bias towards those that have the greater EIS scores
but leave a small probability of selecting a low value tactic,
which widens our search space.
In order to create these heuristic maps,
we provide an extensible KnitGIST objective library,
which consists of functional objectives and aesthetic objectives.
The functional objectives are curl, shrinkage, stretch, and opacity.
The aesthetic objectives are style, symmetry, and imagery.
I will describe the curl and style objectives.
But first, a brief introduction to knitting. Consider a yellow loop of yarn,
if you were to pull on either end of the yarn, the loop would unravel.
But if we pull the yellow loop through a green loop
and pull on either ends of the green loop, then the green loop will not unravel.
In this scenario, the green loop is a parent loop
to the yellow loop, which is a child.
We can pull a loop from back to front, which creates a knit stitch,
or we can pull a loop from front to back, which creates a purl stitch,
each of which have an intrinsic positive or negative curl.
If we align these stitches across an entire piece of fabric,
we can create a texture that curls in on itself
or shrinks and stretches horizontally or vertically as shown in these photos.
We can decrease and increase the number of loops we're working with by
pulling multiple loops through each other
or adding a new loop onto the set. When we do this,
it creates small eyelets in the fabric which are used to create lace patterns.
We can cross stitches over one another,
which across a whole fabric creates this winding cabled pattern.
Given this information, we can estimate the curl of a single loop,
l presented in yellow, based off of its parent loops, P presented in green.
The orientation between the loops is defined as
negative one if they are front to back and one if they are back to front.
and loops pulled back to front curl forward.
The curl of a loop is just the average of its orientation with each of its parents.
But we must also consider the effect of
child loops, m presented in red, on the fabric.
And so the curl of a loop l is also defined with a
recursive operation of the curl of its child divided by a decay factor, sigma.
If we want to estimate the curl across a whole texture,
we simply take the average curl of all of the loops on the bottom row.
Knitters tend to classify textures into knit purl, lace, and cable patterns.
So we took a sample of over 1500 textures from an online database of knitting patterns
to hand-categorize based off of swatch photos.
Then we tokenized them to determine what stitches were used.
Using a C4.5 decision tree algorithm, we were able to get an accuracy of
95.6% with a Kappa value of .86
We also provide an extensible library of tactics, which slightly modify a texture to create a new texture.
We have: flipped stitch orientation tactics,
lean stitch left and right tactics, replace stitches tactics,
mirror stitches on the axis of symmetry tactics,
and swap stitches between two textures tactics.
I will present the flip and lean operations.
Flip simply means to take a stitch that
is pulled through back to front and flip it to be front to back.
We can also lean stitches such as decreases, which tend to lean left to right,
or lean cable stitches, which tend to be cross over in a left toright pattern.
In the paper, we present three demonstrations for forming heuristic maps
to create a variety of textures with physical and aesthetic properties.
I will focus on the final one, a lampshade.
So if we want to create a lampshade,
we consider a series of objectives.
First, we want it to curl up on itself on the top and bottom regions
so that it can grip the lampshade.
We also want to add low opacity stripes which let a little bit of light through.
We want to maximize the elasticity of the fabric
so that it can wrap it around the frame.
Additionally, we add cabled stripes to increase the of complexity the texture.
Using domain knowledge from knitting, we know that we can use the
flip stitch orientation tactic to affect the curl and max elasticity objectives
because this may align the stitch orientations.
We can add increases and decreases to create eyelets which create a lace pattern.
Additionally, we can add in cable stitches
and a particular type of increase decrease pair
which appears like a cable to create a cabled pattern.
The resulting lamp has: gray edges that curl in and cling to the frame,
blue stripes that let light shine through,
and green cables that add an attractive texture to it.
It's worth noting that this is a rectangular swatch,
but because it is so elastic it is able to stretch around
the conical shape of the lampshade.
We also note that this is a hand knit sample that we could have machine knitted.
Unfortunately, we did not have access to our industrial knitting machine during the COVID-19 pandemic.
In summary,
programmers can map valuable objectives to tactics which takes small steps through the search space.
This is built on top of a heuristic map which allows us to
take small steps with a bias towards improving the texture,
and allows us to widely search the space by adding in some randomness.
KnitGIST produces textures that match high-level functional goals
and complex heuristic maps can be used to generate
functional and attractive whole objects.
The system is broken up in such a way that, in future work,
we will be able to create an interactive generative design tool for whole garments.
Thank you so much for listening. If you have any questions, reach out at hofmann.megan@gmail.com