Hello, my name is Romain and I am here to present our work dextrEMS
in collaboration with Shanyuan-teng, Pengyu Li, Yujie Tao,
and my advisor, Pedro Lopes, at the University of Chicago.
Let me first show you a short clip on the dextrEMS in action.
On the right side, we have a person with hearing impairment spelling out “What's your major?” in American sign language.
The person on the left does not understand sign language, and uses their smartphone to translate the question.
Then types in their response.
dextrEMS interprets the message and starts spelling out the letters by actuating the user's fingers.
We can see here the letter “H” being spelled out
then "C"
and finally "I".
Traditionally, the way to actuate the hand is through an exoskeleton.
In order to produce force feedback,
exoskeletons need enough force to push back against the fingers
meaning their actuators tend to be big and heavy.
dextrEMS on the other hand uses a lightweight exoskeleton
that not only lock finger joints on demand
but utilizes muscles as the main method of locomotion,
using electrical muscle stimulation
Now let’s break down on how dextrEMS is actuating the fingers to render the letter “I”.
First, the exoskeleton locks the pinky finger with its mechanical brakes.
If we zoom in closer, we can notice a ratchet mechanism being engaged.
This will lock the finger in place.
Then dextrEMS moves the other fingers through EMS with electrodes attached to the forearm.
Notice that the index, middle and ring fingers contracted but not the pinky.
This is because the pinky was locked by the mechanical brakes.
Finally, to achieve a stable pose, the remaining fingers are also locked.
But wait, lets rewind for a second
you’re probably thinking, “why can't we just do this with EMS alone?”
Well, let’s try to find out!
We are going to spell the letter K in sign language by using only EMS
The letter K can be spelled by flexing the ring and the pinky fingers
like such
and keeping the index straight and the middle finger around 45 degrees
Now fully flexing the ring and pinky fingers is easy with EMS.
However, posing the middle finger at 45 degrees is very challenging.
To achieve such precise actuation, researchers move the muscles on both directions,
the pushing side and the pulling side,
trying to balance out the forces to stop the fingers at a precise location.
Electrodes are placed both on the flexors and extensors
The fingers are actuated until they reach their target position.
Notice here that the index moved from its initial position even though we did not actuate it.
This unwanted movement is caused by EMS running through adjacent muscles
due to the way they are layered and bundled around the forearm.
When the ring and pinky fingers are fully flexed, we just stop EMS.
As for the middle finger, it is actuated until it reaches its target angle.
If the finger overshoots, the opposing muscle, the extensor, is actuated to compensate for the errors.
The same applies when the finger undershoots with the flexor.
However here, as seen in the video, keeping finger at the correct angle is very hard and causes unwanted oscillations.
This is because finger muscles are very sensitive to electrical stimulation.
Now let’s look at how dextrEMS achieves such a pose.
First, the exoskeleton locks the index to prevent it from moving.
Then we apply EMS on all four fingers.
Because the index is already locked into place, the remaining fingers are able to move with EMS actuation.
This solves the first issue we had with EMS-only, where applying EMS on one finger can affect others.
Once these fingers reach their target angle, the exoskeleton locks them into place to retain a stable pose.
Unlike the previous implementation,
we don’t need to balance the forces since we are using mechanical locks
and we can simply just turn off EMS.
This means there is no oscillation and therefore solves our second problem with EMS-only,
where we saw oscillations.
Now let’s see both implementations side-by-side one more time.
We implemented dextrEMS as wearable device combining EMS with a brake exoskeleton.
We want to utilize the precision of an exoskeleton and add EMS, a low-profile lightweight actuator.
The whole exoskeleton weights in at 68g including the battery
which is significantly lighter than traditional exoskeletons,
such as this one,
weighing close to one kg, and that excludes the battery.
Our design includes 3D printed linkages, a custom PCB,
a lipo battery,
bluetooth,
and 8 mechanical brakes, 4 at the PIP joints and 4 at the MCP joints.
As usual, electrodes are placed on the forearms and connected to an external muscle stimulator.
Now let’s take a closer look at this joint to understand how the mechanical brakes work.
It is based on a simple one direction ratchet system where we have
a ratchet connected at one end of the finger,
a pawl connected at the other end,
and a small DC motor in charge of jamming the pawl into the ratchet.
When the pawl is disengaged, the joint can just move freely.
But when the pawl is engaged into the ratchet, the joint is locked into place and cannot flex any farther.
DextrEMS can also lock in the opposite direction.
To do so, the device first engages the brake at the joint.
When it moves in the opposite direction, the brake mechanism will not lock but rather “slip” on the ratchet.
Then EMS is applied to the extensor
When the position is reached, we just turn off the EMS.
The hand will naturally want to recoil back its resting position,
but since the locks are already engaged the joint will just be held in place.
To better understand dextrEMS, we conducted 2 technical evaluations.
In our first one, we evaluated how each joints are independent to each other and compared it to the traditional EMS-only.
We found out that dextrEMS doubles the independence of EMS in average.
In our second evaluation, we measured and compared the amount of oscillation and the precision,
and found out that dextrEMS had similar precision to EMS-only but significantly improved the unwanted oscillations.
However, some oscillations remain due to the mechanical slack of the exoskeleton.
All that is great, but what can we actually do with dextrEMS?
As we already saw with fingerspelling,
this un-pre-cedented precision enables applications previously not possible with EMS alone.
For instance, dextrEMS is great for VR applications that require force feedback on individual fingers.
In this Whack-a-mole game, our device can render the mole's shape,
just like a traditional exoskeleton glove would by locking the joints when a collision is detected.
But when the mole fights back by raising its right hand,
the user’s index gets pushed upwards by our device.
dextrEMS is also very useful to teach skilled movements,
for instance learning how to play the piano with individual finger movements.
Or even learning how to play the guitar.
dextrEMS can shape the hand into the position of a chord.
Then EMS is applied to push the fingers onto the fretboard.
From there the brakes are engaged to prevent the fingers from moving.
Finally, the user can strum the strings with the right hand and play an E minor chord for the first time!
dextrEMS follows a long line of research in our lab.
We are trying to integrate devices with our body and not rely on heavy external exoskeletons
Instead of adding more electronics that externally induce sensations,
We add only the minimum components to trick the user’s biology into internally inducing the sensation
As you can see here, we are not the only ones thinking about EMS as a very promising alternative to mechanical force feedback
Unfortunately, all these systems offer very coarse actuation.
Typically just coarsely actuating large muscle groups like legs, arms or wrists,
nothing with the precision needed for high skilled finger manipulations
My colleague, Akifumi Takahashi, recently tackled the same challenges of EMS at CHI21.
By placing the electrodes at the back of the hand, he actuated the fingers independently.
However, the fingers could only move in the extension direction and only at the MCP.
It also does not solve the oscillation problem.
DextrEMS, on the other hand, takes this a step further by actuating fingers in both flexion and extension
and also on the MCP and PIP joints
To conclude, we increased the dexterity of traditional electrical muscle stimulation by proposing and engineering dextrEMS,
an EMS-based haptic device combined with lightweight mechanical brakes attached on each finger joint.
While we have shown dextrEMS on the hands,
which is considered as one of the hardest limbs to move with EMS,
we can also expand the same system onto the larger body parts, such as arms or legs,
allowing us to eliminate the need for large and heavy actuators.
We made dextrEMS opensource which can be found at the link shown on the screen.
We hope that our device inspires many and we are excited to see what comes next.
Thank you for listening to our talk.
Please stop by our demo session.
And check out our paper for more details.