Hi I'm David Ramsay and I'm going to be presenting the work that Patrick Chwalek and I have done on
the Captivates smart eyeglasses platform under the supervision of Joe Paradiso at the MIT Media Lab.
So Patrick and I are interested in psychophysiology-- that's using
measurements of subtle changes in physiology to estimate something about someone's cognitive
experience-- and there's quite a bit of work in this area, but a lot of it happens in labs,
so unfortunately these lab settings typically use pretty contrived, unrealistic stimuli to
elicit a physiological response; they isolate and identify only one stimulus and how that affects
your physiology, when in the real world lots of different things might affect that physiology;
and they typically average out variation across different people and really to take what we've
learned about these causal links in the lab and understand how representative and useful they are
and to really look at gestalt experiences as well we need to build a measurement platform that can
collect physiological measurements across real, naturalistic contexts and we want that platform
to be product-like enough that it can last all day long and be taken with you everywhere
and also attractive enough that it doesn't alter the behavior and the experiences that we care
about measuring. We also want this thing to be open source so that we can allow for transparent,
auditable, large-scale data collection because we believe that that kind of data-- and making
that data available-- is the best way to move the field of psychophysiology forward, much like what
we saw happen with machine vision many years ago. With all that in mind we created the Captivates
platform which you see here. This is the open-source platform that's a result of
a lot of hard work; it sports two really large batteries in either arm for all-day measurement,
quite complicated electronics that run all through the face of the glasses and are embedded in the
nose piece and through the hinges, and custom plastics and lenses, and it's a very easy to
extend platform. Here are a few pictures that show the hinge electronics and the side PCBs.
The system sports position sensors that you can see here that work with five base stations so
we can easily track large groups of people in a space; it measures temple temperature and
head motion; it also measures nose temperature, and on the left you can see a sensor that is
used to measure blink intensity and frequency. We also think we can derive a heart rate from
these different sensors. So these are a lot of really high quality physiological
measurements that tend to be understudied but have really strong empirical value. It also has
touch control on the side, and several LEDs mounted for peripheral and external cueing,
and of course we've built software infrastructure to go with the hardware to monitor the signals
coming off of the glasses both using bluetooth-- which you can see on the right as an app--
and using the open thread network stack on the left. So of course we wanted to make
sure that our platform is doing everything that we set out for it to do and the first
of those things that we did was a quick user study to see how well we did aesthetically,
and you can see from these plots that we actually made huge improvements in terms of
social acceptability over our first prototype-- which is in the top center-- and we're on par or
exceeding several other actual commercial smart glasses platforms in terms of our aesthetics.
We compared thermopile measurements and IMU measurements after calibrating the thermopiles,
and we did that using thermal cameras and derived motion measurements from the video.
These sort of gut checks corroborated our sensing modalities, although our sensors are actually
higher resolution than these modalities can account for. We ran quite a thorough
blink detection comparison using a simple blink detection algorithm on our raw data; so you can
see that here on the top is someone whose data did really well with our algorithm-- on the bottom is
someone who performed a little bit less well-- you can see the raw sort of signals in green
and the pulses are where we've predicted blinks and then in sort of a faint blue behind that you
can see the hand annotated blinks. Originally we had planned to compare our blank data against
the OpenFace video classifier-- which you can see here on the top left-- but it actually turned
out that the open face algorithm performed pretty poorly on the results we collected,
and that's because we told people that they could just act naturally and do their work naturally,
and it turns out they were frequently looking down at their phone or doing some work or
moving their heads and OpenFace really struggled with a lot of these natural kinds of movements.
So it actually forced us to go through and hand label thousands of blinks per user and
use that to quantify and compare our results, and it turns out-- as you can see-- that we did quite
favorably compared to this sort of top-of-the-line video-based blink classification method.
So with that, if you're interested to learn more, please feel free to reach out to me or check
our website-- we're excited to introduce this platform to the community and hope that it can
enable a lot of really great data collection in very interesting contexts. Thank you.