Cool.
Thanks for the introduction.
OK, so hello everyone.
My name's Jun Gong.
So I'm a fifth PHD student from our XDiscovery
Lab at Dartmouth College.
Today, I'm going to talk about tests
with evil contextual interactions
on interactive fabrics with inductive sensing.
This work was in collaboration with Yu, Lei, Teddy,
and my advisor Xing-Dong Yang
So before I talk about motivation and ready work,
first of all, I just want to quickly show you
what is Tessutivo and how Tessutivo works.
So Tessutivo is a contact based inductive sensing technology
for contextual interactions on interactive fabrics.
It uses six by six spiral coils made of conductive threads
to recognize conductive objects when you place
these objects on the fabric.
Just like this demo video shows, the conductive objects
that are commonly found in people's daily life,
such as quarters, pop cans and a USB drive can be recognized.
I will talk more details about implementations
and applications later in this talk,
but before that, let's start with the motivation and ready
work.
So, nowadays, fabrics make up an essential part
of our daily life since they are generally lightweight
and highly flexible.
They have been widely applied a variety of applications
like clothing and fashion; furniture and decoration,
such as sofa, cushion, and tablecloths; bags and toys.
Since objects that are already made or covered
by soft and lightweight fabrics are we wearing our daily life,
plenty of existing work is trying
to add some interactivities into these fabrics.
For example, projector cord from Google
uses a new type of conductive yarn
that can be over into the textiles
to enable touch based on capacitive sensing.
The yarns and textiles can be produced
by the standard textile manufacturing process.
And a small sleeve can also sense both surface
and information gestures in real time.
It expands the gesture vocabulary
using advanced information gestures, such as twist,
fold, and stretch.
A more recent work study proposed a newly designed yarn
comprised of metallic thread with a resistive coating.
it enables textural pressure-sensing--
a simple structure.
So far, current techniques of interactive fabrics--
many focus on sensing the touch or deformation off the fabrics.
So last year at UIST we presented Indutivo.
It uses inductive sensing to recognize conductive objects,
and their individual movements will place these objects
against the sensor.
So this year, we bring these sensing technology
to the fabrics and provide new opportunities
for contextual interactions or interactive fabrics.
So before I dive in, first of all,
let me briefly explain how Tessutivo works.
So our sensing system consists of an LC tank,
including an inductor, or say the sensing coil,
and a capacitor.
The LC tank is connected to an inductive sensor
that can track the induction variance of the LC tank.
When a conductive object comes within the proximity
of the coil, a circulating current
induces an electromagnetic field coupling between the sensor
coil and a conductive object.
This coupling will induce another circulating
current called current on the surface
of the conductive object.
In turn, the induced anti-current
will generate its own electromagnetic field coupling,
which opposes the original field generated by the inductor.
Objects in different receptivity, size, and shape
will have different coupling effects,
which can be shown in the form of a slight shift
of the inductance of the LC tank.
And this is what we use to distinguish
different conductive objects.
To enable greater sensing resolution
in two dimensional space, we build a dense grid of coils
for both sensing the shapes of objects contact area,
as well as detecting objects movements on the fabric
surface.
So that's for the sense in principle.
Now, so let me talk about the coil fabrication.
So unlike in Tessutivo, just printing the sensing coils
on a rigid subject using copper wires,
one of the biggest challenges of Tessutivo
is how to fabricate multi-layer sensing coils on the fabrics.
So first of all, to pick a proper material
to make these sensing coils, instead of the copper wires,
we used the conductive threads, which
can be easily stitched on a fabric to spiral the coils.
Among what is available on the market,
we picked and tested four candidate options
that are highly conductive and relatively thin.
So based on our initial test, we finally
choose LIBERATOR 40 made of silver-plated fiber
for our current implementation since it
is the most conductive thread among four options.
And its conductivity is about three ohms per meter.
To stage the conductive threads into the fabrics,
we use a home sewing machine.
The patterns of the coils, like shape, size, and spacing,
can be designed using graphics adding in software
and sent to this machine.
So here is a video showing you how it works.
To improve the sensitivity, we also
design a multi-layer structure of the sensing coils.
Just like this image shows, the green line
stands for the non-conductive thread,
while the blue line stands for the conductive thread.
We carefully tune the tension of the top non-conductive thread
to ensure that the conductive sway on the bottom only floated
on the surface of the subject without penetrating it.
So then, we can put two layers together
without causing any short circuits.
Just like this figure shows, the coils
are well-aligned and facing outwards.
In total, our sizing coil have four layers.
We stitched conductive thread in the middle two
layers, which are connected using twist space.
And a coil layer is then sandwiched between two
isolation layers to our wired coils
to be shorted by the conductive objects.
So beside the coil fabrication, the key
to the success of using inductive sensing
is the design of the sensing coil
because it affects the sensitivity, sensing range,
and the recognition accuracy.
The first thing we need to consider is the coil shape.
In principle, the coil can be made into any shape.
But the most common ones are square, circle, hexagon
and octagon.
The circular coil has the best quality factor
and lowers the series' resistance, allowing
the largest resistance in distance.
A square shape has the largest resistance
in area per core unit in a 2D space.
So in this work, in order to maximize the sensing area
and to keep a small sensing distance
to avoid false positives, we choose square shape.
And second, we also need to consider the coil size.
Our goal was to design the coil to be the smallest in size
without violating the inductance requirement
because a small in a dense grid of coils
enables a greater sensing resolution in a 2D space
for both sensing the shape of the object contact area,
as well as detecting an object's movement on the fabric surface.
So last but not least, coil inductance
is also an important factor in our design process.
In principle, with lower inductance
can increase the sensitivity of the system.
Due to its high limit, I won't go
into the details about how we derive
a formula for calculating the inductance of the coils made
of conductive threads and how it helps to decide the coil shape
parameters, like number of turns.
If you're interested, please refer
to our paper for more details.
So here's our prototype.
Our prototype contains a grid of 6 by 6 square shaped coils.
Our customized sensing board uses
multi-plasters to control the columns and rows of the coils.
All sensor readings were sent to a laptop for data processing
via Bluetooth.
So next, I'm going to discuss how
to use this prototype to perform real time object recognition.
So while a conductive object is placed anywhere inside
of the sensor, the sensor reports a 6
by 6 array of inductance values, one for each coil.
And we call it two dimensional inductions footprint.
It describes the object's material, like receptivity,
and a low resolution geometry information
of the objects contact area.
Based on our observation and initial tests,
we extracted 81 features that are in variance
to the locations and orientations of the contact
area of the object.
The features can be divided into two categories-- shape-related
features, like number of covered pixels,
and the material-related features
like mean and the medium.
After the future vector is packed,
we fit it into a random forest model
since it has been found to be accurate, robust, scaleable and
efficient in many applications.
And we use the output from the model
as our prediction results.
So we put the objects that can be
detected into four categories.
First of all, small conductive objects,
whose contact area is smaller than the sensing area,
like this candy box.
So in the following demo video I'm going to show right away,
the detected item name will be shown on the left,
and a heat-map image of the inductance footprint
will be shown on the right.
And the sensing area is marked by the pencil.
So here's a demo for the small conductive objects.
We included more than 10 different conductive objects
in this category to encompass a broad range
of different properties like sizes, material, and shape.
So the next category is large conductive objects
whose contact area is larger than the sensing region.
Most of them are electronic devices
with built-in metallic components.
As you can see as shown here, our system
can recognize the Apple Pen with the Surface Pen,
since they are different in electronics.
Front and the back of the same device can also be identified.
For some of the conductive objects,
we also try to attach a strip of copper tape
to create a unique inductions footprint so
that the same object can be used for a different purpose
just like this video shows.
So to be enable non-conductive objects sensing,
we instrument them using copper tape
with different patterns that can be distinguished by the sensor.
As you can see here, the system can successfully
identify different books with different combative patterns.
And a heat map image reflects the cover type patterns.
Besides the object recognition, Tessutivo team
can also support object manipulation.
Here are some examples using a quarter, candy box,
and a iPhone.
So to validate the system's accuracy and robustness,
we ran a temporary spin religiously.
Each participant was asked to place
27 objects on the sensor in random orientations
and locations inside the sensing area.
Each object was repeated for five times
in a randomized order.
And participants were asked to place the tested objects
on the seat of sofa instrumented with our prototype,
we deliberately designed this procedure
to evaluate how the collected object model would create
a more realistic setting.
So we pre-collected data from one volunteer
three days before the study and a trained model
for later real time evaluation.
Note that this volunteer was not recruited again in the study.
And it turns out that our system can achieve
an overall real time cost-efficient
accuracy of 93.9 percent, which is very promising considering
no producer calibration, no user training,
and a considerable time separation
between the experiment, and when the training
data were collected.
So that's for the evaluation.
Next, I'm going to talk about several conceptual demo
applications.
We envision five time more applications
on the tablecloths, in your pocket,
and backpack to showcase the possibilities of Tessutivo.
So the first application is a hydrogen tracker placing
[INAUDIBLE] mark on a tablecloth starts the timer.
And a reminder is sent to the user's phone
if the mark stays at desk longer than a preset time period.
And a tablecloth in a meeting room
can also sense the laptops on it between pride your submitting.
With this information, a users phone
is set to a do not disturb mode automatically.
When dinner is ready, the tablecloth on a dining table
can sense the utensils, and the baking tray.
Your delicious meal is ready downstairs.
And to remind the child on a different floor
to come to the kitchen and eat.
So the tablecloth can also be combined with a backpack.
A nearby floor lamp is switched on
or a user grabs his Kindle from a table.
And after the user is finished reading and a
puts their Kindle into the backpack,
the lamp turns off automatically.
A pocket that is instrumented with Tessutivo
can detect if a user's phone has slipped out
of a pocket when they have gotten up and then left
on the sofa.
OK, that's for the conceptual demo applications.
Next, I will discuss the future work.
First of all, with our current implementation,
the threads are stitched tightly inside a small area
of the coil, which has made the substrate harder than it
was before the instrumentation.
We believe it will be interesting
for the future research to consider the trade off
between the size of coil and how well
we can preserve the softness of the fabric substrate.
And second, since the readings can
be affected if the coil is deformed
and which may introduce false detections.
But on the other hand, we might also
be able to use this property to sense the fabric deformation.
And, finally, in our current implementation,
objects are required to be in contact with the sensor.
So it would be interesting to explore the situations where
the objects are within a short distance range from the center,
just like keys in the pocket.
So with that, I would like to conclude our work with three
take home messages.
So we present the technique using
inductive sensing for contextual interactions
on interactive fabrics.
Our technique can precisely recognize conductive objects.
And we believe Tessutivo holds the potential
for seamless contextual sensing using interactive fabrics.
So that's all I want to share about for today.
So a short ad before the questions.
I'm on the job market this year and looking
for research positions in industry and academia.
Thank you.
Thanks for the great presentation.
Very creative use of Faraday's law of inductance.
So we have time for one or two questions from the audience.
Hi.
Thanks for the interesting presentation.
I have one question.
So if the fabric does not allow the deformation,
what is the benefit of using the fabric for this technique?
Sorry, can you say that again?
So if the fabric doesn't allow any deformation,
what is the benefit of using that fabric for this technique?
Yeah.
So you are asking if the--
So is there any potential application
which utilized the deformation or stretchable
properties of the fabric?
So right now, we do not use any deformation properties
of the fabric.
So we just place the objects on the sensing coils
and it senses the objects on it.
It would be super interesting for us
to exploit the deformation properties of these fabrics
because the coil inductance will change
based on this deformation of the fabrics.
OK.
[INAUDIBLE] to you and have a great talk.
With higher frequency signal, a inductive sensor
can be used to sense finger touch.
Do you have any early experience about that?
Yes, yes so, basically, in our paper
we also present that this sensing coil can also
sense the finger touch.
But here we cannot run a formal investigation, I mean,
I ask participants to tap on the sensors.
But yes.
It is going to work on sensors for the finger touch and even
multi-finger touch.
Yeah.
I think we have to move on due to time,
but I'm sure the authors will be available
after the presentations.
And let's thank Jun once more.
Thank you.