Hello! My name is Denisa McDonald and I am a PhD student in computer science at Drexel University.
Today, I'm presenting our work: KNITTED SENSORS: DESIGNS AND NOVEL APPROACHES
FOR REAL-TIME, REAL-WORLD SENSING
This work extends the capabilities of knitted sensors which rely on one conductive
carbon-coated yarn and 2 external connections. The sensor depicted in Figure 1 is produced in
one piece using digital knitting, with minimal post-processing. It's very thin,
stretchable and behaves like fabric. Figure 2 illustrates the scalability of this process.
Wires can be connected at the two yarn endpoints.
The fact that there are only two connection points, makes both
manufacturing and system integration easier. Additionally, it reduces the risk of breaking
contact at connection points, making the sensor more usable and robust.
However, the minimalism of the system, also means that there is reduced information output from it,
shifting the burden of complexity to more refined sensing and computational models.
In this work, we discuss moving this sensor technology closer to the real world.
Contributions include: Building a recognition
model to identify precise location of touch on an existing knitted sensor
Building a real-time signal processing and acquisition circuit
Conducting user studies to evaluate the model and investigate its robustness in the real world,
and Introducing
several design prototypes of knitted sensors constructed through the same knitting process.
We propose a system which is composed of several connected components:
The knitted textile sensor with 36 locations of touch, along the same conductive yarn.
A real-time sensing microprocessor. And a recognition model. It has a feature
constructor step, and a neural network which identifies location of touch.
The recognition model, once trained for high-accuracy,
can, in the future, be embedded into a dedicated microprocessor or handheld device,
and connect to the sensing microprocessor to identify location of touch in real time.
In the next slides we are going to be looking at these components individually.
The microprocessor inputs a signal into the system and processes its output through Bode analysis.
There are 96 frequencies used, outside of the range of electromagnetic interference or EMI.
The plots on the top, show the system response in the absence of EMI. The input
signal is depicted in black, while the outputs from either yarn endpoint in red and blue.
The bottom plots illustrate data captured in the presence of
EMI. From the first plot we can see that the output is distorted during EMI exposure.
However, the second one shows that the EMI frequency range can be discarded,
and the signal can be reconstructed based only on the desired input frequency range.
So, each sensor touch event consists of 250 temporally-related measurements of the
voltage gain values captured as explained Statistics are computed on the frequency
values for each step, for a more stable representation
Additionally, to retain the most relevant information only, we subtract a baseline
representation of the system, which is its state with no touch occurring.
The next component of this system is the neural network architecture. It is composed of three
layers of Long Short-Term Memory cells, which have been widely used for sequential data analysis,
including time-series signal data, since they capture temporal dependencies.
The input into the network is a touch event, represented as the sequence of statistics on
the gain values computed above, and the output is one of the 36 locations on the knitted sensor.
This model in order to properly recognize location of touch,
needs to be trained with user data, which we collect through our user study described next.
In order to move toward real-time, real-world knitted sensors,
we conducted a series of user studies. The first study enables the creation
of a robust location detection model The second, demonstrates its generalizability
with new users The third study
investigates its robustness to potential sources of distortion– divided into two smaller studies:
In one, the sensor was exposed to electromagnetic radiation
And, in the other, the sensor was stretched while the data was being collected
Here are the results of those studies, together with their classification matrices:
The data from the first study, which was used for training through 5-fold cross-validation,
produced an accuracy of 58% for location detection. Training happened with 386
samples per class, which is a limited number for deep learning algorithms to perform well.
Through the model evaluation study, we obtained an accuracy of 66%
Our robustness tests report result of 80% accuracy for EMI exposure, and 37% for stretching
Chance accuracy is 3%, since there are 36 classes; These results are subject-independent,
in order to remove the need for individual calibration. In the future, training should
include a variety of conditions, and more data, for results to improve.
Using the same manufacturing process, we can produce a variety of knitted design forms
that rely on one conductive yarn, combined with regular yarns, and two external connections.
The sensors illustrated here are some examples, and include a piano keyboard, a volume controller,
media control buttons, as well as larger pads of different designs.
These sensors can be connected to the same signal processing hardware and rely on the
same neural network architecture, but each model needs to be trained with new user data
that is specific to the sensor geometry. The technical contributions and studies
conducted in this work helped us lay the foundation to advancing towards viable,
robust knitted textile touch interfaces. For more information, please refer to our
paper, or contact us. Thanks for listening!