So hi, everyone.
So I'm going to present skin-on interfaces,
a bio-driven approach for artificial skin design
to cover interactive devices.
So this work has been done with [FRENCH]..
We are researchers from Télécom Paris, [INAUDIBLE] University,
and University of Bristol.
So I will start by saying the obvious.
We are surrounded by devices, and we use them
to access information a lot.
But most importantly, we use them
to communicate with others.
And they are pretty good for that, actually.
We interact with them through touch thanks
to their input surfaces so that they can detect touch position.
However, this kind of input surfaces is somehow limited.
So it's rigid, it's smooth.
It doesn't have texture.
In the opposite, if you take the human skin, like the human hand
here, the skin is a sensing organ we are familiar with.
It can sense also touch location, and multi-touch,
but it has also some depth and some texture.
And most importantly, it can sense complex and expressive
gestures, like pinching, or grabbing, or stroking.
So on the one hand, the skin is a great interface
that we are familiar with, and that is really expressive.
And on the other hand, the devices are everywhere,
and they are great for mediated communication,
but their input capabilities are somehow limited.
So in this project, we wanted to use
the expressivity of the skin and to put it
on the existing devices.
So that's why we developed and created what
we call skin-on interfaces.
Skin-on interfaces are surfaces or artificial skin that
looks and feel like skin, and that
are capable to detect expressive gestures, like we only
do usually on human, like stroking or tickling.
So they can be used on a touchpad,
or they can also be used on a mobile device.
The question I will answer in this presentation
is how we went from the human skin
to the design of such interfaces.
For that, we follow what we call a bio-driven approach.
So this here, it's a cross section of skin.
So starting from that, we try to systematically characterize
the properties of the skin, and try to reproduce it
with the interactive system.
So we can see it's composed of many years.
You have the epidermis on top, the dermis, where we have
all the nerves and the sensing.
And also the hypodermis layer, which is more like the fat
layer we have for the fat people, like I'm--
So in this project, we try to reproduce each layer.
And that correspond to also the visual and tactile aspect
of the skin, the sensing layer, and what
is kinesthetic replication, and similarly.
So in this presentation, I will explain how we reproduce
and how we created this interface.
I will talk about the user studies we conducted,
the fabrication method, and also the use cases.
So I will start right now with the user studies.
So we had three main questions that motivated this work,
and three main research question.
So to explore which factors impact
the creation of realistic skin, first, we
had to see how the visual aspects of the interface
impact the perception.
Second, how the surface texture of such interface
might impact the comfort when we touch it.
And finally, how the thickness impact the comfort, and also
the expressivity that we can do.
So to answer this question, we did some small-sized samples
in silicone.
So as for the color, we picked different pigmentation.
As a baseline, some white and black colors,
which are something like a device.
We also picked something like skin pigmentation,
beige and brown, and also something more vegetal,
or just something-- different color.
As for the show face texture, we picked a gradient
from something without any texture, so raw silicone,
to something really granular.
So in between, we have something similar
to subtle fold and subtle wrinkles
that we have in the skin.
And finally, for thickness we picked several thicknesses,
from something really thin, like the thickness
we have in the feet, to something
more like we have in the belly.
So with all those samples, we ran a user study every time
with 16 different participants.
And we asked them, how does it feel?
How does it look?
Is it comfortable to perform gesture on it?
Does it looks like human skin, and is
it expressive, and so on.
So the goal of this step was really
to inform the design of an artificial skin.
So results suggested first that the skin-like colors,
pigmentation, were looking more interactive.
So that was something a bit unexpected,
because it's an interactive system,
so we were expecting something like black or white
was looking more interactive.
Also, participants say that the wrinkles and the surface
texture is really important for the tactile perception,
and also the comfort of interaction.
And a thickness of 5 millimeter to 1 centimeter
were also something quite comfortable
to perform a gesture on it, and so it
was a good compromise between something
expressive and comfortable.
So now that we know the best factors
to reproduce artificial skin with silicone,
the next challenge was the fabrication.
So we analyzed in the literature, the [INAUDIBLE]
literature, which gesture are frequently
used to perform interaction on surfaces.
So obviously, we have everything related
to the interface control, traditional pointing
and multi-touch.
But also, we're interested in the gesture
of emotional communication, like tickling, stroking,
or slapping.
So with all those gestures in mind,
we tried to select the best sensing technique.
So we choose to use the mutual capacitance sensing technique,
because this technique allows to sense with a good accuracy,
and can sense single touch, multi-touch,
and with different resolutions.
So we had that in mind, and then the next step
for us was to actually build it.
So I looked at many HDI papers, and I tried to reproduce it.
And I totally failed for most of them.
So I tried with connective silicone, and connective
fabric, and it was too difficult for me to make,
because I'm not an expert, and also, I
didn't add all the material and [INAUDIBLE] available.
So I decided to do something much more simple,
and something really do-it-yourself
that could be reproduced easily by HDI researchers that
don't have any skills.
So like me.
So now I will explain how I did to design and build
the skin-on interfaces in a really simple way.
So first, I start by mixing some silicone, which
is normal silicone, with some pigments.
Then I pour it on top of a texture layer.
So the blue thing here is just a texture
that looks like human skin.
So I spray it in an even thickness of something
around 1 millimeter, because that's the thickness
of the epidermis of the human.
So I try to approach that.
And then to build the electrodes,
I used something commercially available
is the conductive stretchable wire, which
is really great, because that's good conductivity.
And so I sew it--
I position it in a grid pattern, basically.
So here, I have a grid to help me,
but I can do it manually also.
So the spacing between the electrode
have an impact on the tactile acuity.
Like for something like that, I have around,
I think, 3 millimeters of tactile acuity.
So then after doing this step, I just put a mold
and pour some silicone, which this one is
the viscous silicone that feels like human fat.
Finally, I connect the electrodes,
and I have the working interface.
So the electrodes are connected to
tiny mutual-capacitance-sensing hardware.
So that's a tiny open hardware [INAUDIBLE] that we developed,
and where we can connect a grid of 12-by-21 electrodes,
so we can scale it and connect any electrode on that,
and it's talking via I2C2 Arduino.
So really easy to do, and it's available.
And then, on the software side, we are getting the data.
And we have this kind of data.
So we have the touch and multi-touch,
and we also have some sense of depth.
So here, we retrieve the data at a rate of 30 frame
rate per second, which is not really high,
but it's enough for us to apply after some detection.
It can also be--
this interface can also be placed behind a phone,
and then it's a skin phone case.
How cool is that?
[CHUCKLES]
Here, it's-- so there was a working interface behind
the phone.
So to detect the touch gesture, the simple
and the complex gesture, I used a simple--
there is nothing special.
I used a computer vision.
So here, I will give you two examples.
Here are two different types of gestures.
We got the data.
We scale it, we blur it, and then we applied blob tracking.
And you can see, even in the bottom, that the grabbing--
you can clearly see the hand grabbing the interface.
So based on that, it's easy after to create
different algorithms.
So now we'll talk a bit more about the use cases
and what is possible to do with some human skin on top
of devices.
So for that, we developed three different prototypes.
So keep in mind that here, we have
the grid of electrodes behind that is doing the sensing.
So obviously, it can be used as a--
we can use traditional and multi-touch gestures.
It can be used as a touchpad.
But also because it has some thickness,
we can really sense depth.
And in this example, we have a joystick with a z-axis,
and we have really like a three-dimensional joystick.
It can be used also on the back of the devices,
and we can perform back-of-device interaction,
which has been really explored in HDI.
But here, there is a twist.
There is a twist.
It's that we can pinch the phone.
We can pinch a phone and create [INAUDIBLE] on the go.
So also, most importantly, that was the motivation of our work,
it was the mediated communication.
Because skin, obviously, we can use it as a skin proxy.
So here, we send an avatar.
It can represent the arm of the avatar.
And if I'm not happy, I can pinch the avatar,
and the avatar reacts.
Or another example is when we communicate with text messages,
the touch channel is not used at all.
But in real life, we use, actually,
touch to communicate with people.
So in this example, we augmented a text application
with some natural gestures, like tickling.
And the tickling of the phone is then
sensed remotely to another user.
So to conclude a bit and to sum up these interfaces,
so we have this chunk of skin here
that we put on top of a device.
We worked a bit on the pigmentation
to have something realistic.
We work also on the texture, a texture
that impacts the tactile sensation in the [INAUDIBLE]..
And the interface is capable of some strength
and some thickness, actually much more
than the traditional touchpad.
And of course, we have the sensing layer.
But there are some other aspect, other factor
that we left aside, like the sweat of the skin.
It's for future work.
We want to investigate how we can put sweat
on such interface.
Also, the hair, the human hair, because that's important.
Actually, the presentation after me,
you're going to work with hair.
And also, the color change and the blush,
because that's something we do and we use to communicate.
And if you look at this picture, you
can see it's slightly different-- like the touchpad
we developed and the skin, the human skin is more realistic.
So we ask ourselves, can we go further?
So that's a work from last year.
I know some of you remember.
So I presented MobiLimb, which is an anthropomorphic device
that can actually perform output on the user wrist.
And it's anthropomorphic.
It's realistic.
And it's anthropomorphic both in terms of shape,
and also capabilities.
So I dive a bit more into the anthropomorphic aspect,
and I created other interfaces that have really
like a realistic look.
And I changed a bit the colors, the surface texture,
which is reinforced.
Also, I play with the volume of the interface.
And it's now an interface that looks really
like a piece of skin and that has
the same actual capabilities.
So you might think it's creepy, but you're not the only one.
And actually, I've seen online everybody thinking it's creepy,
so it's OK.
I'm aware of that.
But I think I think it's great, because this project explores
a possible future of what can be interaction.
It questions also the shape of the device we have,
and how we interact with them.
And also, this work opens up the discussion.
Does a device need a cover that feels like human skin?
That's an interesting question.
I don't know if they need it, but at least I
try to explore what's happening if [INAUDIBLE]..
And thank you for it.
I'm [INAUDIBLE] will have a demo,
so we'll be able to touch the interface.
[APPLAUSE]
So I think we have time for maybe one question,
and the next author might come up.
Hi.
Thank you.
I'm [INAUDIBLE] from USC.
I was wondering if you have thought about adding
force sensor to this.
Force sensor?
Yes.
So actually, so with the mutual capacitance technique,
and the--
how do you say, yeah, what we use,
we can sense somehow forces into a sense of pressure.
But we could--
I wanted to do that, actually, to put
force sensor under the fat layer to have
like a really sense of force.
But we can infer force, but not actually sense the real force
right now.
Any other questions?
Hi.
So you were looking into maybe sweat, and hair, and texture.
I didn't see temperature.
Did you guys--
Temperature.
Yeah, I didn't put it in the slide,
but temperature is one of the most important factors,
and that's of use that we want to enter.
Actually, that's funny, because when I was doing the prototype,
sometimes it was heating up, and it was heating of the skin.
And I was like, hmm, that's going to really stick.
And this is actually like a side effect.
But it's cool.
Yeah.
The first creepy thing is like cold, dead skin.
[CHUCKLING]
Also, I have another remark.
That if you want to go to the airport,
don't put it in your pocket, the skin,
because you might have problems with the security,
and I know what I'm talking about.
Yeah.
[CHUCKLING]
OK.
Let's thank [INAUDIBLE] again.
[APPLAUSE]