Hello.
I'm Artem.
And I'll talk about SensorSnaps.
And basically, we're integrating our wireless sensor notes
inside the snap fasteners for textile interfaces.
And this work was mostly done at Google with Alex Olwal,
and also with the help of Tomas Vega from MIT Media Lab.
So first I'll show a little video of the sensor snap.
So they're low power wireless sensor nodes.
They seamlessly integrate into caps
of a fabric's snap fasteners.
SensorSnaps provide a new technique
to quickly, intuitively, augment any location on the clothing,
with sensing capabilities.
SensorSnap securely attach and detach
from ubiquitous commercial snap fasteners
and use inertial measurement units.
SensorSnap can detect tap gestures as well as track body
movements.
So first, I want to start off with the high level
motivation for this work.
So most of our interactions are done
through a touch screen of a mobile phone,
and this is not ideal in many situations.
So a lot of researchers have been
looking into alternative wearable seamless interfaces,
and especially the advances in electronics, miniaturization,
and low power integration.
For example, like the chip on the golf ball
is the smallest arm microprocessor.
This allows us to create explore new interfaces,
and that's what we are doing in this work.
We're integrating electronics into small sensor nodes.
So specifically, we are looking into snap fasteners which
are commonly used in clothing and accessories,
such as on the jean jacket, the winter coats,
as well as handbags and many other applications.
And they are very ubiquitous.
So you can actually buy a kit of them
on Amazon for like $10, which includes a box as well
as all the tools you need.
And you can get them in metal, plastic, different materials.
And you can actually attach them at home
in the same way they can be attached in the factory
as well.
So this is a very scalable fastener,
a very scalable invention.
So they are-- usually they have four different parts.
The cap and the socket--
they sandwich a piece of fabric together.
Usually they are pressed.
Using a crimp tool, they're attached.
And on the other side, you have post and a stud,
which also sandwich another piece of fabric.
And the socket and the post, they
snap together to secure the two pieces of fabric
with the interference fit.
And in our case, we only modified
the cap of the snap fastener.
And this is the part that faces the outside.
And the other parts remain commercial off-the-shelf
products.
So we created the SensorSnaps, the miniature sensor nodes,
into the cap of the snap fastener.
And they contain a wireless communication, microcontroller,
as well as sensing capabilities.
And they are tethered with a battery as well.
So our design requirements is we want
them to integrate with the existing
clothing and fasteners, and to be ready to wear, ready to use.
They should be easy to attach and detach
to allow fast prototyping without the use
of complex machinery.
And they should work on unmodified garments.
They also should be low power so that the battery can last
for at least a day or more.
They should be small and lightweight,
so that they're unobtrusive to wear.
And also, they don't stand out during normal wear,
everyday wear.
So let's look at some previous work first.
One of the common ways to attach electronics to fabric
is using conductive thread and embroidery or hand sewing,
like islands of electronics together.
And this was done in Arduino Lilypad,
which is a very popular kit for making conductive fabrics.
The disadvantage of this technique
is that you need to modify the fabric so it doesn't seamlessly
integrate into the fabric.
Also, it requires some skill to add it.
And it always stands out because it's not
part of of the fabric itself.
There's other approaches such as knitting and weaving, usually
using industrial machines that can be programmed
like in the previous talk.
So one of the ways to add the conductive fiber and conductive
fabric, make this--
one of the good examples is Google Jacquard,
which added capacitive touch pattern on the sleeve
of a Levi's jacket.
And the snap fasteners were actually
used as the connectors in this case.
But the one difference from our work, this
requires modifying the garment itself
on a very industrial machine, which usually people
don't have access to unless it's on a factory scale.
There are other approach is using straps and Velcro.
And this is a very standard approach.
So you can just strap devices.
So usually it stands out.
It's not very seamless.
But it's a very quick way to prototype.
And we haven't seen any electronics
being integrated directly into the fasteners or the buttons.
We've seen the snap fasteners--
the metal ones used as conductors.
The connectors, for example, is an antenna in this case.
So looking at the previous work, usually the previous work
required modifying the garments themselves
to add electronic and sensing capabilities.
In our case, our device works with unmodified garments.
So this is one of the key differences.
So I'll talk a little bit about the electronics.
So we have an AR-52 microcontroller, which
has Bluetooth and ARM cORE.
We also have a 20-pin port, which gives us access
to all the pins so we can add more shields.
We can also use it for charging and debugging.
And you can see on the bottom, that's a break out board.
On the other side of the board, we have an IMU, 9-axis gyro
accelerometer magnetometer.
It's a very powerful IMU.
And the whole board is just 12 millimeters in diameter.
It's four layer PCB.
On the mechanical side, we created the enclosure
using SLA 3D printing.
And we have 12 milliampere battery,
which is probably the smallest battery that we could find
on the market, which still allows for the Bluetooth's
capabilities as well as higher power usage.
And the weight of our device is only a 2.4 grams.
In comparison to the metal fasteners,
they're about 1 gram, so it's pretty
similar in terms of weight.
So the way we came up-- we designed the mechanism
to attach them to the fabric with the commercial snap
fasteners.
And we embedded a hex nut inside the SensorSnap
so that we can screw it directly with--
just using a screw we can attach it.
The commercial snap fasteners employ crimp tool.
But we didn't use it because it applies a lot of force
to the fastener.
But we have electronics and 3D printed enclosure just cracks.
And this is one example of attaching this.
So you can see here, first we open up
a hole with the cap over the plastic cap.
Then we put a screw into the socket.
And then we push a screw through the other side of the fabric
here.
And then we just screw on the SensorSnap after that.
And this is pretty much all we need to do to attach it.
And this takes about 70 seconds.
The commercial one takes 60 seconds.
And the other side you would do is you would do normally.
It is like the commercial one using a crimp tool.
We just crimp a little piece of plastic
to make a permanent connection.
And then you can see, it's a robust connection
that you can open and close as just
a normal commercial snap fastener.
And so with the capabilities, we have gestures.
The main gestures are done with the IMU.
So it's the tapping and rotation gestures.
So the rotation gives us granular absolute angle.
And then tapping is just kind of like a button interface.
And we can also create a network with multiple SensorSnaps.
We can have a mesh network.
For example, this is tapping multiple snaps.
And, yeah.
So we can add more sophisticated capabilities.
So the rotation gesture.
We look into y-axis of the gyroscope.
And this is our growth signal over time, from the y-axis.
Basically when we rotate the SensorSnap left and right,
we get the signal.
And then we integrate it.
We get a very--
pretty accurate angle.
So this is from--
on y-axis, that's 30 to minus 30 degrees.
So I'm rotating it left minus 30, then right minus 30,
and then over multiple times.
And we actually don't experience any gyroscope drift
in this case.
So the accelerometer gestures we--
the tapping is done with a z-axis.
We only look at the z-axis of the accelerometer,
because the accelerometer would be too sensitive if we use
other axes for false positives.
So this is the raw signal with one of the taps.
You can see there's a dip there.
So we take the derivative of the raw signal.
So it's done on board in real time.
And then we set the threshold which corresponds to the taps.
And once again, here is the gestures being performed,
the rotation and the tapping.
They also have a capacitive touch sensing.
And this, you can see on the picture,
we added an electrode under the lid.
And the capacitive touch allows us
to have more granular control over the touch gestures.
For example, we can distinguish short and long presses.
But mainly, we're using it as a low power
wake up of the device.
Because the tapping with the IMU is very sensitive to motion.
And this is not sensitive, so you don't have
a lot of false positives here.
Only when you touch it, it will trigger.
So that goes into our interaction modes.
So in a standby mode, we only look at the capacitive touch.
So the capacitive touch is periodically sensored.
And then the processor goes to sleep.
Then the sensor goes to sleep.
So it's digicycle.
Very low, low power.
If detects the touch, it goes into the gesture mode,
where it again, sends a gyroscope, an accelerometer.
It turns on the IMU.
And the Bluetooth sample interface is about 20 Hertz,
so we get real time data.
And this is the gesture that I showed.
And then we have the motion mode,
where the rotation matrix is extracted from the IMU.
And this is very power intensive.
So we get about 4 to 15 hours in a standby mode,
depending on how often the capacitive touch
sensor is sampled.
And we get about 40 minutes in the gesture mode.
And here is some of the examples.
So we have a music player where we connect the SensorSnap
to an Android phone, and we made an app for-- a music app.
So you can hear.
We use a tapping gesture to start, pause.
We can rotate to get them to change
the volume, left and right.
And also we do double tap, and then it
goes into a different mode where we can rewind and fast forward.
So you can see we can add these kind of capabilities anywhere
on the body without much modification
to the clothing itself.
And this is ready to wear.
As well, we also have motion sensing.
So this is using the rotation matrices.
So you can see there is a digital avatar
of my arm, which kind of repeats the motions that I do.
And this shows also the capability of meshing.
So you can have multiple of those buttons working together
to provide more functionality.
So in the future, we want to improve the charging.
So we want to use wireless charging,
so you can waterproof and make the enclosure more robust.
We want to further miniaturize the size.
So for example, if we could make it
like those buttons on the shirts, like very small.
But it's pretty challenging-- the ones with four holes.
It's still hard to do.
We also want to optimize the power.
Mainly, we could use a lower power IMU, which
will save us a lot of energy.
And we can create new software so the users can create
custom interfaces more easily.
There's multiple buttons.
Right now it's just--
we only have pre-programmed interactions.
And we also want to look into other applications.
Can this be-- for example, this is used as an earring.
Can we use it as jewellery, accessories, kind
of a universal platform for little sensor nodes
on the body.
So in conclusion, we made a wireless sensor node
inside a snap fasteners.
They are quick to attach and detach.
And they are compatible with commercial products,
and don't require garment modification.
So they can actually be employed both in mass manufacturing,
as well as in the hobby settings.
The current cost to manufacture was about $10,
but we think this could be much cheaper if it
is done on the mass scale.
And the visual application that utilize the IMU and capacitive
sensing.
Thank you.
And we'll show a demo today at 6:00 PM, if you want to see it.
Right.
Thank you for that excellent presentation.
So many practical applications of this work.
So we have time for one very quick question.
Hi.
I was wondering if you could speculate on making this thing
battery-less, or how you envision people having
to change batteries over time.
I mean, there's definitely RFID technology.
You could make it battery-less, but it's also--
the battery technology and the power consumption
is pretty good now, that we can actually
have batteries on those things because the battery allows
us to have a lot--
we can have a lot more functionality with a battery.
So I think the future of this kind of stuff
is with the battery.
Is not without them.
Because it just is a power problem, right now.
So the batteryless stuff is very limited
and especially for interaction, for something that's wearable.
When you touch it, you want it to respond immediately.
And with batteryless you don't usually
have that, because it needs to harvest energy,
then it's very periodically activated.
But it's an interesting direction
to look into as well for some areas, especially a sensor.
Thank you.
Let's thank Artem once more.