So thank you for introducing us.
I'd like to talk about a scale module [INAUDIBLE]..
So we made a small black box type toolkit
to exploit a force-based interaction.
Here's a summary of our project--
summary video of our project.
We made 3D force sensors, packaging
into the small black box, then we can detect the touch
position based on the force.
In addition to that, we can capture the 3D position
on top of it.
This is a technical high rate [INAUDIBLE] revisited.
Then we have a three feature touch detection
then this object detection.
In addition to that, we can also capture the human motions
on the single platform.
Then this is a summary of my project.
Then here's the motivation of my work.
The human activity, that's the common interest in HDI field.
So human activity is everywhere.
To augment the experience in a human life, the capturing
or measuring or estimating the human activity
is really important.
And to achieve the activity recognition,
so we have a bunch of different type of sensors.
Like from a vision based module, capacity based,
and the acoustic based.
Actually, I couldn't list out all of the sensors,
but we have many types of sensors.
Then in this year, 2019, I'd love
to revisit the force-based sensors for the activity
recognition purpose.
The reasoning is human activity can be characterized
by false information.
So we defined this kind of interaction
as a force-based interaction.
The interaction is a conveyed-- the information is
conveyed by the force vectors.
So you can imagine our hand, when
you are drawing some letters.
So between the hand and surface there must be 1 gram each,
then 1 millimeter resolution force happens.
When we put the object on your desk,
the desk has a force based on the gravity of the Earth.
Then, in addition to that, the more larger scale
between the body and the surface be
different if you have a force informations.
So then my motivation is how you can capture all the information
on a single platform.
Then to capture this wide range of information,
so researchers have been using these sensors.
This is a loadcel.
The loadcel say it is a combination
of the metal bar and the deformation gauge-- deformation
sensor.
Then if you can replace the metal bar part
so that we can adaptively change the load maximum, the load
tolerance, and the precision of the sensor itself.
Based on these senors, many researchers working
on the individual properties.
For example, the most recent the one from the 2016 [INAUDIBLE]
Conference.
So the researchers working on the 3D touch detection
on top of that over the load sensing.
Then table-top activity inference or the body tracking
is another very traditional target
of the research of the load sensing result.
Then we combined the unified or all the activity application
purposes into the one platform.
That's the part of our contribution.
Now here's a list of our scale research
that we made a concept of a load sensitive modules,
which is very adaptive very versatile module
for the force-based base interaction.
Then, in addition to that, we had a big technical invention.
We made a 3D touch detection version
without prior information of shape geometry.
Then we also made packaging this concept into one small boxes.
Then we made a GUI and the module hardware for the novice
uses.
Then also, we conducted and re-evaluated our system
with having the workshop with 20 or more designers
and the cooperative designers and engineers.
Then here's our design space of a scale system.
So I'd love to visit the one by one.
For the first part, we made a toolkit, the scale toolkit.
The most important part of our project is it's modularity.
So the user doesn't have to screw it
after the modules in the every time.
You can just place these small modules beneath the surface
or beneath the object.
So you can put it this basically everywhere, wherever you want.
Then we combined from the module sensors--
we combine all that in the data into the total force.
That you can see the green line, that's the force the vector.
That's the most important component of our research.
Then this is a detail about our module hardware
and the architectures.
So that we have a bunch of modules,
then we connected it to the one microcontroller.
That's a Teensy base that's already
in a compressible microcontroller.
And then we connected it to the host computer
and we had a bunch of modules like touch detection,
object detection, the motion detection.
Then the users can easily make their own applications
on the open frameworks platform that the C++ library
for the interactive GUI making.
Then for the right part, so that we made this hardware.
So that inside it--
so instead of the traditional loadcel with the 1D axis,
just the Z axis.
We utilize the XYZ, the 3D axis loadcel.
So this is the core point--
the technical point about our project.
Then we classified a force-based interaction
into the three categories.
Between the hand and the surface,
that's a touching direction.
Between the object and the surface,
that's the object detection.
Then the body and the surface, that's the motion detection.
Then the [INAUDIBLE],, I developed [INAUDIBLE] one
by one.
Then for the first, the touch detection.
Then in the table-top, the interactive surface committee,
the 2D touch detection is really a traditional field,
so there's nothing new.
We implemented that, but nothing we invented in that 2D thing.
However, we invented a 3D touch detection
without shape information.
Then this is a video of our demonstration.
Then when you touch the tripod--
you can touch the tripod on the three block modules.
That's a scale toolkit.
Then just touching onto the surface,
you can get to [INAUDIBLE] shape information of the tripod.
Then when you touch on the surface,
you can easily make a--
you can assign the [INAUDIBLE] touch button onto any object
to visit the prior shape information.
Then here I'd like show once more the technical detail.
So in a previous method, with a target geometry,
once you can capture these force vectors,
you get easily calculate the intersection of the surface
and the force vectors.
However, without having the target geometry,
how can calculate the intersection--
there's no surface information, geometry information.
So there is a one dimensional uncertainty.
Then to solve that, we utilize the hand information.
So we had a hand, but when we touching something,
so we couldn't completely freeze.
There is a slight unsteadiness there when we touch the object.
Then utilizing this, sampling the multiple force vectors,
I guess you can see the slight intersection of all
the green lines.
So we capture-- we estimate that the point as a 3D touch vector.
Then this works very well.
Then this is a little bit mathematics about it.
For the single force vectors, we can represent
it's equation of the line.
Then when we capture a bunch of it.
If you have N-samples, then we can make a matrix.
Then what we can do is just solving it
with a [INAUDIBLE] matrix method.
Then we introduce a generalized [INAUDIBLE]
enough regularization instead of the standard Tikonov
Regularization.
If you are interested in some mathematics,
you can read it in our paper.
Then we implemented primitive applications
on top of the platforms.
In the workspace, we can make the self really smart.
We put to the modules, building the gray surface, then there
the lead shelf and the inside contents, that's
a totally as it is.
We didn't modify anything.
So by just putting the modules beneath the surface,
we can capture the usage or statistics
on how the user uses the tools and the consumption of the two
components.
The photo of the post office--
in the post office there is a weight measurement.
Then we can augment the measurement stage
with the positioning sensor method
on top of all of scale modules.
But the one sacrifice of the [INAUDIBLE] this 3D sensing
capability in sacrifice of the [INAUDIBLE]..
Now for the next object detection.
Then object detection is demonstrated here.
When you put the object on the surface,
we can get the rate and each position.
Then, the system already knows the weight of the object,
so we can keep tracking the location movement of the object
on top of it.
Naturally, we cited the basement or foundation of our project
is from the paper from 2006.
So there is a [INAUDIBLE] user interface.
This is a title of the paper.
So that they utilize the everyday object
as the user interface.
The concept is a weight as an ID.
So we introduce the weight as ID concept to the our project
then combined with the weight based identification database
manipulation.
So we implemented the object status tracking.
Then here is an example application
of a future scenario of our project.
Then if we augment all the shelves
under desks with scale modules, so work space
already knows the movement of the tools.
Now, if you lost your car key in your house,
then the system already knows that, OK,
that your key is a 100 gram.
Then that 100 gram object is in your kitchen.
So, like this.
Then for the final functionality,
so we implement the motion pattern detection.
So we augmented the small square.
You can imagine that's a part of the floor.
So that system-- that floor can sensitive of the 6 DoF vector.
That six means that three--
they are the upper part--
that's the 3D axis, the force vectors.
Then for the bottom part, that's a torque information,
location information.
So you can see there, so now I'm just
stretching along the z-axis.
And you can see that this black lines.
So that we can utilize this information
to that some recognition tasks.
Then we implemented a really primitive preliminary--
the motion recognition pipeline on top of it.
We extracted the 6 DoF key feature
as a set of the standard deviations of the force
and torque information.
Then we classified it into the four different categories.
Then this is the summary of the functionalities.
Then to achieve the touch object motion infomat functionalities,
we did a samples over time.
That's a key concept, the rate as ID.
Then 6D motion vector.
That's the technical features of my project.
Then here's a one page result of the technical variation.
Then we conducted a technical variation.
The horizontal accuracy.
That's a very nice kind of three.
The motion recognition, since we did a really simple one using
support vector machine to classify the four things so
that we achieve the 90% thing.
Then for the vertical accuracy for the 3D touch detection.
So the current three, it's a slightly [INAUDIBLE]..
The 7 centimeter areas around the 50 centimeters height.
So I'd love improve these things for the future applications.
Then we conducted a workshop to explore
the our application field over the load sensitive modules.
Now thanks to the collaboration with the Panasonic
Beta the product company in California.
So we conducted the workshop to explore the applications base.
That's the first purpose.
For the second purpose, evaluation
of the utility of our toolkit.
Then we had a longer ideation session.
Then we had a--
in collaboration with the cooperative designers,
so that we made a bunch of ideas.
Then we classified it into the six categories.
So I guess this is a really nice foundation
of the force-based interaction.
The key findings from these ideation that we really
need a touch detection systems.
Then, I guess, I our toolkit with a 3D detection technique
could be contributing for the force-based interaction method.
Then we also had a hands-on prototyping session.
Then after generating all the many ideas,
we categorized 20 engineers and the designers
into the four groups.
Then they individually, as a group,
so they explored application space
on top of the scale model kit.
Then one interesting thing is one group
explored-- you can see the top right
figure that some groups explored floor based ideas for the yoga
lesson.
Then the other groups can augment stuffed animals
or stuff the characters to make them interactive.
Then this is a comment from the workshop.
Then that this is the future challenges.
Then I'd love to emphasize on the response things.
So this [INAUDIBLE] convinced to be the 3D detection techniques.
So we'd love to improve the speed of that project
to the future.
Then this is the summary of our project.
Now we are scaling up the project to the future.
Thank you.
[APPLAUSE]
Thank you, Taka.
Do we have some questions?
Yes.
Back row?
Very interesting.
Thank you.
The basic input devices, how many numbers do they give you?
Because if it's three numbers, I don't see how
you got five numbers out of it.
So it's giving you more than one pressure number, right?
Yeah, right.
OK, thank you for that question.
I'm sure you saw the minimum requirement of our module
is actually three.
So to support the one surface, we really need three points.
Then we can accommodate three or more modules on top of it.
Now, after, you know, putting the [INAUDIBLE] modules,
we can combine all the information
into the one total [INAUDIBLE].
Yes, a question right here.
Great talk.
Thank you.
I might be misunderstanding, but is it
possible for the system to estimate
the shape of the object?
For example, if you put a square on the table--
not square, sorry.
Yeah, any kind of shape.
And stroke the surface or just keep
on touching, then is it possible for the system to tell,
well, this is a puppy or anything else?
Yeah.
Yeah, thank you so much.
So actually, that possibly that could be the future next
project-- our next, next project.
So now we're really passionately working on top of this project.
So just touching on the object so that we can roughly
estimate the object's shape.
So that maybe just this swiping on the
like this microphone-- really swiping on this microphone.
So there's some reducted or reduced information
of the geometry and our human vectors.
Then I'd like to--
how do you say, make the shape information from that things.
Yeah, that's a really challenging thing
that I'm very passionate about.
Thank you.
I have time for one more very, very quick one
in the back there.
Thanks for the great talk.
Can you explain about the initial setup process?
Like, if you have those sensors, do you
need to like register that these tendencies were positioned
to those sensors and then you're able to sense
the touch and other interactions?
Sorry, say again, what's the point?
Sorry, I missed the point.
I was wondering about the initial setup
when you have those sensors and you have the environment.
Interesting.
So actually, currently, we need to set to the initial point
of the module.
So we need to manually set the module one,
the relative positioning things.
So for the next generation, I'd rather
put the relative position sensor that's built in the things.
So then the more easily deployable modules [INAUDIBLE]..
OK, thank you very much.