- We've had a pleasant stay in Brighton so far.
I'm Sven Kratz from the Harman Future Experience Team
and my co-author is Tony Dunnigan from FX Palo.
We're both based in California.
And I'm happy to talk to you today about ThermoTouch,
a new scalable hardware design for thermal displays.
So, what we set out to do is build an interactive display
with a 2D adjustable grid of thermal pixels.
That means we can add certain pixel locations,
provide hot or cold feedback for the user
when he's touching the display.
And we wanted it to be scalable, so
enable a large surface area, more than a square meter,
and also enable a large number of thermal pixels,
so, in the hundreds or thousands.
Why did we think this is interesting?
So, you know, you have museums, you know,
increasingly using large touch displays,
and I think it might be interesting to add a further
output channel in the form of thermal feedback because,
for example, in technical resumes there are lots of
processes which you can show that involve temperature
change, for example a jet engine
or a nuclear power plant.
A bit on a different track, you could imagine desktop AR
games and adding, you know, additional feedback that
the user can feel on the desktop.
We also looked at office spaces so you could imagine
augmented work desks with ambient notifications
via thermal feedback, and all that requires
fairly large thermal displays, which we didn't have
at the time, and which we still don't have.
There's quite a bit of related work in this field, so
one track of work goes into reproducing
material sensations through temperature change.
For example, Richter2012.
There's been a few works on using thermal feedback to
display information, both in fixed locations and Wilson
also looked at using this for mobile devices.
Harvey studied how different ambient conditions influence
perception of thermal feedback and one prototype which was,
it's actually pretty close to this work, is called
Thermo-Pict "Neo" by Takada, and he built a grid of
Peltier elements and also experimented with
temperature, so thermochromatic ink displays, so
you can see here when the temperature changes
then the color of this ink changes.
So there was a few issues we wanted to address
with existing techniques, so many prototypes were
single-pixel, so we wanted a grid of pixels.
And another thing is, most of them are based
on the Peltier element as a transducer and
it was actually pretty useful component
because it provides both heating and cooling
depending on the polarity of the voltage you apply to it.
It does need a complex driver circuit, so you need a
H-Bridge circuit, as you can see here in the bottom right,
which requires at least four power transistors.
At least for the cooling, it has a relatively high
power consumption, and it's per unit it's quite expensive,
actually, to buy.
And you only have discreet modules, so imagine building
a thousand pixel display from discreet modules.
So, these are kind of impediments to scalability, so
we wanted to think about a new concept, or
adapting existing technologies to enable scalability.
So, for 2D thermal grid, we looked at...
So what's the most simplest technique for heating?
It could be electric resistance and liquid cooling
is going to be an established technique
for cooling stuff down.
So the way it works is you have a reservoir with coolant,
which gets pumped through some kind of a chilling mechanism
and goes to the place which you want to cool down,
and then the warm coolant flows back to the reservoir.
So it's actually a pretty simple system used for
internal combustion engines up to nuclear power plants,
but the principles are always the same.
This is another view of the prototypes.
So, we have a PCB mounted on top of the manifold,
which gets cooled by coolant in the bottom,
and with inflow and outflow ports.
Here on the right you can see the arrangement,
we chose for this.
This came together as an initial prototype, so
here you have the complete assembly
of the PCB and the manifold.
It's all bolted together with screws and we have
some gaskets to make it watertight.
What's missing is
we put a layer of thermally conductive silicone on top
to enable top projection, so that we have a visual part.
So, in the paper, we go into some detail about the
PCB design, so it's composed of an area of basically
spirals of copper trace and in the paper we describe
how to calculate the length to find the correct resistance.
In the middle is the switching circuit, so it's actually
a very simple circuit using one power transistor,
so that's a quarter that we would need for Peltier element.
Here on the right, as you can see,
the finished prototype of the PCB.
So, when this all comes together, it looks
somewhat like this.
So, here on the left, you can see the prototype PCB
put together, as you can see, on the bottom of this table
is a lot of ancillary equipment.
The pump and chiller and reservoir and lots of plumbing,
so it was actually quite fun to put together.
Might be streamlined in the future.
Yeah, so it actually does, it's actually cold
at the surface, so on the left you can see it
in a cold state and on the right is a thermal image
of an activated heater cell,
so you can see the hot spot there.
Once we've assembled this device, we need to
know its characteristics.
Can it kind of come close to a Peltier element?
So that's kind of the goal.
So, first we looked at what input power do we put in and
what's the output temperature
on the surface of the device.
We chose a temperature range, for evaluation, between
five degrees Celsius and 70 degrees, so obviously
in an interactive scenario you wouldn't want to use or
to touch a 70 degrees hot surface, but this is just
for knowing the limits of
what you can do with this device.
Then we find a linear relationship between the power
you put in and the temperature which is displayed,
which is great for easy control.
We also looked at maximum and minimum heating rates,
so, sorry, maximum cooling rate and maximum heating rate,
so the maximum cooling rate is about three degrees
Celsius per second.
Maximum heating rate is quite a bit higher.
It's 21.4 degrees Celsius per second.
And so how does this look in comparison to a kind of
run of the mill Peltier element, which you can get
on several electronics hardware prototyping stores.
That's the CP40236.
We mounted it on a one inch aluminum block to act as
a heat sync because, you know, you need to get rid
of excess hear from the Peltier element.
We also found it has a linear relationship between
input power and either cooling or heating.
And so what is interesting for us is the energy consumption.
So we can see for cooling...
And so for heating, it's actually more efficient than
ThermoTouch, because obviously ThermoTouch is working
against the cooling effect of the water,
although for cooling, ThermoTouch would basically
just turn off the device, and here the Peltier has to
actively work so it is slightly less efficient.
Cooling rates...
Cooling and heating rates compared.
So for cooling, they're about the same, so Peltier would
have a slight advantage.
Heat and for heating up, ThermoTouch is much faster.
So, that was an interesting result.
And, you know, what can you do with this?
So, we implemented a test application
just to see if, once we've put the device together,
what it can do.
So, we thought about a thermal feedback system for
video playback, so this enables
an additional output channel for video, so you can,
if you touch the device at a certain point,
you can feel the temperature.
I think this could, for once, be used for educational
applications so you can feel
the components of thermal processes and also you get
a different experience of
videos recorded with a thermal camera.
And there are two ways of creating a thermal video.
So, one variance was with a key frame editor we developed.
So, basically it's a web-based system and the user can
enter, for a given point in time, can enter the
thermal output characteristics of the board, save that
in a key frame list, and then basically have the video
annotated with thermal information and the system will
then automatically transition between these key frames
according to what the user entered.
So I have another variance, which it is easy to apply
to video from thermal cameras.
So, basically, it takes in the raw video feed from
a thermal camera and converts this to thermal output
on thermal touch, and this is just an algorithmic approach.
More details in the paper, I won't go into that theory.
One key component which is missing is how to actually
change temperatures from one value to another.
We chose open loop control for simplicity, and because we
have this linear mapping function, so we actually know
which power to set for a given temperature.
However, it's very slow to just switch from...
So, if it's switching from 20 to 30 degrees Celsius,
just switching to the different voltages, which
correspond to the output temperature.
Actually, what we want is to change the temperature
at the maximum rate.
So what we did is find out what the maximum heating
and cooling rate and use full power for heating and
zero power for cooling and then we can determine
the time interval required for full power or zero power
to effect the temperature change as fast as possible.
So, basically, results on these, kind of,
power profile functions you see on the right.
This is kind of a small demo...
(narrated demo video)
You can see the thermal output on the top left.
(demo video narrator)- ...10 seconds, go for main engine
start, seven, six, five, three engines up and burning,
three, two, one, and lift off of Space Shuttle Discovery!
- It starts to warm up.
(demo video narrator)- ...to the Moon, Mars, and beyond,
and the vehicle has cleared the tower.
(upbeat electronic music)
- You see it's really red-hot.
(demo video narrator)- all systems remain go for Discovery.
Altitude now nine miles, six miles...
- This is the automated variant, so it is just directly
taken from the thermal video and
output as a thermal feedback.
(upbeat electronic music)
So let's discuss...
So, at the beginning I mentioned we can
also use closed-loop thermal control, which might be better,
but more complex, so you would have to add a temperature
sensor on each thermal pixel, but it might help compensate
for fluctuations in the coolant temperature and
make it more robust in general.
I still owe you, actually, a large-scale prototype.
I claim this is good for large-scale prototypes,
so this still needs to be realized.
So, to discuss the liquid cooling system, there's obviously
a low, constant power demand, so you have to keep
the chiller unit running all the time.
However, you don't need to put much power into this,
because the coolant itself has lots of thermal capacity,
and so peaks in thermal demand are bridged, actually,
by the capacity of the coolant.
And this approach can also be used for
Peltier-based systems, so it's not exclusive
to resistive heating.
And obviously the cooling system will have to scale
with the size of the thermal display.
What we did see is...
Another problem is the heating efficiency, so you can
see it's working against the coolant, so
we actually have to put in more heat energy than necessary.
However, there are some work-arounds around this, so you
do not need to heat the entire board all the time.
You could use a camera and see where the user's touching,
and then just heat the area surrounding the touch and just
leave the rest of the PCB switched off.
But still, I believe that large-scale resistive
heating grids can be cheaper than lots of Peltier elements.
For one, they can be printed, so you can print the manifold
with a 3D printer.
You can print the PCB on different substrates and make this
really cheap and very large, and you
only need one quarter of the resistors.
So, with that, I'd like to thank you for your attention,
and I'd like to give a shout-out to Tony, who's been
very great at doing all of the visual assets for this work.
So, thank you very much.
(applause)
(woman)- Are there any questions?
(man)- Thank you.
Can you talk about the actual numbers of how much...
How much does it take to build the prototype?
I mean, how much money would it be that you would need to
build such a type of prototype?
- Well, it depends if it's a one-off or, you know, if you
go into mass production, but I don't know the exact number.
I would say the ballpark...
So, the most expensive part was the power supply,
which costs 300 dollars, so it's a high-power power supply,
but for any large system you would need
a strong power supply, so maybe 1,000 dollars
went into the hardware.
(man)- Could you maybe improve the cooling performance,
because that's where it is going a bit lower, just like
they do with computers, like, underneath the PCB,
have some aluminum ridges, have a faster flow, so that you
have more surface and more flow across the surface?
- Yes, that's a great suggestion, because this PCB
was just a normal PCB from PCB Fab,
so there's no modification.
So, you could have more, like, a thicker copper layer,
solder on cooling fins, try to put it on a different
substrate, not this fiberglass, because I think that
insulates also a little bit.
So, yeah, great suggestion.
(woman)- I have a question about, since I don't
know anything about thermal displays, can you leave
something on, can you leave the heat on in a certain
spot for long periods of time?
- Yes, I think, so you just...
You apply...
The system uses pulse width modulation to apply heat,
so, its basically a dynamic, we see a dynamic voltage
setting, which you can apply to each thermal cell.
If you don't leave it at max power, then you can
leave it on indefinitely.
Max power would burn out the cell at some point.
(woman)- When you were showing the demos, I was immediately
thinking this might be a very interesting way to serve
visually impaired people using touch displays.
- Right.
(woman)- So, basically, heating where a button is, or
things like that, so has anyone ever
looked at anything like that?
- Not to my knowledge right now, but certainly could.
(woman)- Are there any other questions?
Okay, lets thank the speaker.
Thank you very much.
- Thank you!
(applause)