Hello attendees of CHI’22, i am Keng Wei and this is my teammate Joseph
Today, we’re presenting our submission for the student game competition: ESsense. It is a DIY electrostatic sensor system for simple, contactless, motion-based inputs in games
When most people think of the word “sensor”, they think of complex electronics that are capable of reading all sorts of different metrics
Things like temperature, light, sound, rotation, and movement. And indeed, these are all intricate devices that are difficult to build and repair on one’s own
Well, what we’d like to showcase today is a simple DIY electrostatic sensor or ES sensor. The sensors can be made out of easily obtained materials
Conductive materials like copper tape or aluminium foil, specific dielectric plastics such as PTFE, and a microcontroller such as an Arduino
When there is relative motion between a dielectric material and a conductor without contact,
the free electric charges within the conductor are disturbed, inducing current within the conductor, which can be detected and processed by a microcontroller
The induced current changes with distance and speed between the two materials
In our local climate, the small induced current is sometimes too miniscule to detect. This is mainly due to our high relative humidity,
which reduces electrostatic interactions between all objects. So to help amplify the input current received by the arduino, we had to place
This PCB in between the ES sensor and the microcontroller. Though, in many climates that aren’t as wet as Singapore, this component will not be necessary
With some code and calibration, we managed to turn this system into a motion sensor, where the dielectric is held by a player and moved, with the microcontroller detecting motion through the stationary conductor
The Arduino can then send a keyboard input to a connected computer
This sort of detection only enables momentary activation, almost like a “contactless button” of sorts
The simplicity of components, as well as our limited time, resources and coding ability, did not allow for detailed motion controls like in video game controllers
Nonetheless, there are still a number of possibilities enabled by this system which we’ll show later
When used together, ES sensors can detect time differences between the activation of separate sensors
In this example, we connected two conductor pads to different analogue channels on the microcontroller
Depending on the order of activation of the sensors, the microcontroller is able to distinguish the direction of travel over the two pads
More refined detection is possible if the sensors are configured around some of their limitations, which we cover in our submission paper
Time for our sample demos! For our submission, we showcase the system working in the contexts we have here on screen
Motion-based controls are most intuitively used as a solution for embodied controls in video games,
allowing players to mimic in-game actions, for greater immersion and more memorable gameplay experiences
This is the approach we adopted for every example we are about to show. So first up is…
Lumberjack+, our example of new controls for existing games
This is a simple web game from Telegram, a popular messaging platform
Its original inputs featured either left and right screen taps or left and right presses of the keyboard
Using our directional ESsense sensor system we showed earlier, we made a tree stump that houses 2 conductor pads, while the model axe houses some PTFE
After processing the directional inputs from the player’s swings, the connected Arduino sends the corresponding key press to the computer,
providing the relevant input. Next up we have…
Levitation Charm, our example of new contexts for existing games
This is a recontextualised version of Flappy Bird, a popular smartphone game that has, unfortunately, since been removed from app stores for good
Instead of tapping a screen to flap a bird’s wings, the player instead flicks a magic wand Harry Potter style to levitate a feather and avoid obstacles
And finally, we have ConducTENG, our example of a brand-new experience made possible with the ESsense system
Built using a playback speed function in p5.js, this game plays a music track in accordance with the tempo you swing the baton at
Players can try to match the original tempo of their song of choice, or simply mess with the song’s speed just for fun
It’s quite hard to truly visualise what a gameplay experience with it would be like,
so i encourage you to check out our trailer and overview videos to get a better sense of how this game works
We have a number of other possible interactions that can be built using this system in our overview video and submission paper
With ESsense, we hoped to create a medium for those interested in DIY to build their own controllers with simple motion-activated gestures,
one that didn’t require specialised hardware, and fixed configurations that had to work around said hardware
ESsense does have its limitations as a result of its flexibility,
but it does work well to support controller innovation in the DIY community, especially with the new possibilities provided by motion-based inputs
And with that, thank you for viewing our presentation! We hope you enjoyed it :)