So as you said, this is a talk KnitPicking Textures--
Programming and Modifying Complex Knitted Textures
for Hand and Machine Knitting.
I'm Megan Hofman, and I'm presenting
this on behalf of my co-authors Lea Albaugh, Ticha Sethapakdi,
Jessica Hodgins, Scott Hudson, James McCann, and Jennifer
Mankoff.
And I'm going to put on my excessively chair.
Hat on real quick.
If anyone wants to see captions for this talk or any other
talks in this room, you can find them at this URL,
tinyurl.com/UISTSOUTH.
All right.
So you might have guessed, this is a talk about knitting.
And this might be what you picture
when you think of knitting.
But instead, we're mostly going to be talking
about this kind of knitting.
This is an industrial knitting machine
that we have in the textiles lab at Carnegie Mellon,
and industrial knitting is a very robust and mature
technology.
In fact, everyone in this room is probably
wearing something that was made on
an industrial knitting machine very similar to this one.
So you might be wearing a sweater, though it's
a little warm in New Orleans.
Better odds are that your wearing socks,
those are probably knitted by a machine.
And it looks something like, there you go.
Something like this.
These are pieces of technology that are used on factory floors
across the world.
And as a result, the interfaces for them are a little bit poor.
They're not really designed for kind of rapid prototyping,
or thinking through interesting fabrication problems.
They're designed to create one product, which
you make a million times with a lot of upfront effort.
And what gets me really excited about industrial
knitting or knitting machines is that they
can produce all of these really interesting textures
with all of these really cool physical properties.
So they can be stretchy.
They have all these kind of diagonal features to them.
They could be lazy.
And as my co-author, Lea Albaugh showed us at Cairo
earlier this year, if you start to play
with the physical properties of these different textures,
you can create all sorts of interesting physical objects.
So you can create something like this lamp
or this very attractive sweater, which will pop up.
There we go.
Love that one.
So let's say you want to get started
with your own industrial knitting machine,
while you're going to have to learn a piece of software
like this.
And if you're expecting me to explain
what's going on in this picture, I have to tell you,
I have no idea.
There might only be one person in this room who
could figure it out, and that's my co-author.
But this is what the industrial tools look like.
But there have some been some recent advances in the area.
So back in 2016 at SIGGRAPH, people from our team
started to work on a compiler for 3D knitting.
And they said, there are basically four basic operations
that control these machines.
And you can treat those operations
as kind of the assembly language of knitting.
And once you have an assembly language,
you can start to build algorithms
on top of that to deal with things like shapes.
So you can take a 3D model this famous Stanford Bunny,
and then produce something that will
produce the skin of that bunny out of a knitted object.
But there really hasn't been that much focus on texture,
because these textures change the physical properties
of knitted objects.
And as a result, the shape is going
to change if what you're producing.
Earlier this year at SIGGRAPH Asia,
there has been some work on applying textures
on top of these.
But they're still kind of treating them
as two separate problems.
Adding the texture on top of the shape later.
But there are lots of people who understand
how knitted textures work.
They're called hand knitters.
And they do this on a fairly regular basis,
so how do they understand knitted textures?
We'll they understand them in very much the same way,
as person, or as a computer mind.
They understand as a series of instructions
that they write out in this language right here called
Knitspeak.
There's a few different dialects of this,
and every knitter has their favorite.
And then often, they'll kind of visualize them
with charging tools like the chart shown here.
And then the representative sample
to understand how it's going to look
before they go through the process of hand knitting.
So we've looked at this very pragmatic language,
and we said, well, let's build a compiler.
So we wrote a compiler that can compile
two different versions of Knitspeak that
are very similar.
One is The Essential Stitch Collection
by Stanfield and Griffiths, and the second
is Stitch Maps, which is an online open repository by JC
Briar.
And we have compiled today, and measured 300 different samples.
Here are a few of my favorites.
But if you're thinking about this as a compiler,
you have to compile down to some kind of representations.
And in order to understand that representation,
I'm going to have to teach you a little bit about knitting.
This is a loop of yarn.
And if you were to pull on the two
tails of this yellow loop of yarn,
you no longer have a loop of yarn.
You just have a strand.
But if you pull that loop of yarn
through another loop of yarn, that bottom green loop of yarn
now.
If you pulled on those tails, it would be safely
you'd keep that loop.
So we call this the loop to loop constraint
that all loops pulled through other loops
stabilize the first loop.
And you can start to play with this.
You can pull them from the back to the front,
which is called a knit.
Or you can pull them from the front to the back.
We call that a purl.
And so now we've got two different pieces of texture
here.
And you can pull more than one loop
through a single loop, which creates a decrease that
decreases the number of loops you have available to you,
the yellow ones.
Or you can add a loop that you're
going to have no parent to, and that we
call that an increase or a yarn over,
which increases the number of loops that you have in a row.
And if you pair these decreases and it increases together,
you get kind of like a hole in the fabric.
And that's how we make lace.
And then you can also cross these loops one
over the others, so you have kind
of a three-dimensional feature here.
We call this cabling or traveling stitches
colloquially.
And this gives a very interesting kind
of springy effect to the fabric.
And if you're thinking yourself now
about these different kind of nodes connected
to other nodes that sounds like a graph,
then you know where I'm going next.
We call these KnitGraph, a time ordered
collection of loops pulled through other loops.
So you can think of the green edges
here as the yarn of the fabric connecting all the loops that
are made one after another.
And then the purple edges are connecting them
where you're pulling one loop through the other.
And as long as you have four properties,
but primarily that loop to loop constraint meant.
You're going to produce a graph that would result
in a viable knitted object.
All right.
So now we've got a compiler.
We have something that we can compile down to,
and then we have actually two different output products
where we can create instructions for an industrial milling
machine.
Or you can create hand instructions,
so that someone can knit these different samples by hand.
But we don't really want to stop that just
representing a single texture.
We want to say, well, what if I want
to change the size of that texture,
or shape it in some way?
Or what if I want to combine it with two other structures?
And to support that, we have two different optimization
algorithms.
I'm going to start with KnitCarving.
And KnitCarving is for sizing individual textures.
You can think of a texture is kind of like a tile
that you repeat across a rectangular grid.
So in this case, there's four repetition
of this lace structure.
And if I want to say, like, cut into these textures
to reduce them.
Reduce the width in this case.
Then, I'm going to have to think about that loop
to loop constraint, and which loops
can I remove before the thing unravels.
So let's say, I want to narrow this by just cutting off
this edge right here.
Well, the naive result is going co-produce this hole
where this fabric is starting to unravel as I touch it.
So instead, let's think about this
as a cropping problem or some lighting
an image cropping problem.
So if I have this representation here
with all of the different stitches,
and I decide that each of those stitches or loops
has a cost to remove.
If a loop is dependent on it, then its cost is infinite.
The whole thing's going to fall apart if I remove it.
But if it doesn't, then its cost is based off
of its rarity in the sample.
So the number of loops is a fraction of the total loops
that make up similar stitches.
So if I've got a lot of knits, then knits
have a very low-cost to remove.
And if I've got very few purls, then purls
have a very high-cost to remove.
And then once we've assigned this cost each of these loops,
we can find the minimum path using dynamic programming
to kind of cut this nice diagonal.
Or a Chevron pattern out of the texture,
and get something very similar.
And when we do this on knitted results,
we get something like this, where
the one on the bottom left is almost
indiscernible from the original texture.
But I guarantee you, it is one stitch narrower.
And if I start to play with these samples,
and narrow selectively starting at the bottom only narrowing
a little bit, and the narrowing as I go up more and more.
You can start to shape or change the shape
of the different knitted objects, which
is how you get something like the hat I'm
wearing and pictured here.
But we want to work with more than one texture at a time.
In this case, I actually used two different textures.
One at the little rib at the bottom,
and then one at the top.
But if I want to do that in a more complex way,
maybe to create this panel here, we
get this problem where that gets very lumpy.
That's because of all those cool physical properties
and knitted textures are different
across those textures.
And if you put two things that have different cuff stretches
to them, they'll bunch up at the boundaries.
And see you get something lumpy and not quite square.
So how do we solve this problem?
Well, we start first sort the thing
about the number of loops at the boundaries
of these different patches.
These different colored sections of the fabric right here.
And we think about trying to balance those loop counts,
so that we reduce the error at those boundaries.
The first source of error is sizing
error, which is the idea that the number of loops
is correlated to the width of the patch fabric.
And so you have this idea of gauge,
the number of loops per inch.
And then, you subtract that from the actual width
that you're going to get, or the target with you're
going to get.
And that is representative of the error
that you're going to get from sizing.
The next error that we get is texture error.
So if I have to narrow this texture by less than a tile,
I'm going to have to use KnitCarving,
which will introduce some error into the texture.
And so the number of removals or that delta there is going to be
can increase the error of this fabric.
And then finally, we can actually add edges
in the graph at the boundaries.
So we can have the number of loops
of the top or bottom of this yellow patch
be different than the number loops across it
by having increases and decreases.
But those increases and decreases
are going to create a little islets
that you can see, as so that's going to produce
an error in the fabric.
And so here's the end result that we
get this much more uniform, and flat sheet of fabric.
But you can even start to see where that joining error
is coming in with those little islets.
And this actually works on almost every texture
in our database.
We can see that this works pretty well
across different kind of cables and different laces.
In fact, those earlier samples, which are pretty basic
textures have the most divergent stretch.
And so they're going to have the greatest effect.
So some future work to think about in this space.
Ideally, we wouldn't be working with a programming
language for a user interface.
And so we need to use this as kind of a jumping
off point to build more graphical interfaces that
might be more approachable for different types of designers.
And then we further need a combined texture shaping,
and physical simulations that we can really crack
this problem of the shape.
And its relationship to the physical properties,
and the textures that it's built out of.
And once we've done that, and while we're doing that,
we can start to work on thinking about using tools
like industrial knitting to produce more interesting
soft functional objects.
The kind of takeaways I want to leave with this talk
are that machine knitting requires a design pipeline
to support texture and shape.
We've kind of started to approach that pipeline here,
but I think there's a lot of room to grow here.
And handling instructions are actually
a super valuable resource for knitting designs.
There's lots of work that's already
been done in this space outside of the industrial space.
And so we can really rely on that boundary of knowledge
to study knitting as a field.
And then finally, sizing, shaping, and combining data
textures poses new opportunities to design soft materials.
So once we have kind of cracked this pipeline,
we've built all of these tools.
It's going to be much more accessible to start
to play with us for things that are,
maybe a little more interesting than garments.
Thank you so much.
Are there any questions?
Thanks very much for that excellent presentation.
Also very cool that you're wearing your own results
from the paper.
So we have a couple minutes for questions.
Any questions from the audience?
So can you talk a bit about things outside of fabrics
that you might think this texture and shape relationship
might be important for?
You mean outside of the knitted space?
So I think this is very-- this work is very specific to kind
of the knitted properties.
And I know, I had to kind of gloss over it for a time
that we go into detail about those physical properties
in the paper.
I think outside of knitting, this
is a challenge for other types of kind of textiles like woven,
materials, or other types of ways of producing fabrics.
I know, some of the papers later looking at 3D printing,
and I'm sure they will attract some of the same issues.
I'm not quite sure how the concepts that we've taken here
will translate necessarily.
But focusing on this idea that you
can create these kind of like microstructures in this case,
and then apply them to change the physical properties
is a space where optimization.
And different approaches are really
applicable across the field.
Whoever has the microphone.
There we go.
Really interesting work.
I really think is cool.
I have a question though.
How do you envision the abstraction between moving away
from interfaces very much programmatic
and like assembly into a more graphical interface?
How do you envision that interface
looking in terms of like, how a designer would actually use it?
So there's actually a lot that I think
we can start off with learning from kind of how handlers have
hacked together charging tools.
The stitch maps repository actually
has a tracking tool built into it that we looked at.
And I think that kind of representation
is maybe a good place to start.
The challenge here is that, if you draw something
that you have as an intention for a texture,
and then you want to go back to this programmatic
representation.
That's clearly a little bit more difficult than going
from the program to the representation
or to the visual representation.
I'm looking into some work on using program synthesis
approaches to try, and solve that problem.
But I think there's a lot of challenges there
that are unclear at this point.
I think another approach that might be appropriate
is to think about how do people kind of paint
on what they want, or what they expect from the fabrics.
I wanted to have cables.
I wanted to be lacy.
I wanted to stretch, and then use optimization approaches
to learn from things like our data set.
And generate something as a result of that.
And then work in kind of a mixed initiative approach,
until the user gets something that they really want.
Big talk.
Thank you so much.
I had a related question about moving into a graphic user
interface.
And it relates to who you're thinking
about designing this for.
Because it seems like audiences who are already experienced
knitters think about things algorithmically
and programmatically are used to dealing with something
that is a programming representation,
or very similar to that.
Versus audiences or maybe new to designing for knitting,
wouldn't be familiar with that space?
So it seems like having a programming
based interface might be very appropriate for people
who are experienced knitters.
Can you talk a little bit about, how
you think about that trade-off?
I think it's hard, because there are
lots of people who can read these kind of knitting
instructions.
But I think it's much harder to find
people who are used to writing them,
or would like to write them.
We haven't really done any of the groundwork
to analyze how these communities produce things,
so we could look at groups like rivalry
to see what knitters would want.
And I think in some ways it's very
different than industrial knitters.
So industrial knitters use a completely different pipeline
to define these textures than hand knitters.
I think the graphics approach is probably better,
because we're still in that space
where we're ultimately producing something that
is very visual or physical.
And so people are going to have an easier time understanding
that, if they're looking at what they're getting.
Rather than going kind of through this very laborious
processes.
And I'm going to program this out,
and then I'm going to make it.
Or visualize it in some way, and then I
realize that's not quite right.
And then I go back and forth, and that process
is very tedious.
I can say from even just making the demos in this paper.
Great.
Thank you very much.
Great.
Question period.
And let's thank--