- Give it 30 seconds.
- Are we turning it off and turning it on again?
- We are.
- Great.
A while back I made a video about motion amplification
a video processing technique that amplifies
tiny movements in a video.
And look, here's the original video.
The human eye can't detect any motion
but it's possible to take those tiny motions
and amplify them so they can be detected by human eyes.
You can also amplify changes in color
until it's possible to see someone's heartbeat.
Link to that video in the description.
But when my friend Andrew Steele saw the video,
he said, "I've got an idea.
Can I strap electrodes to your chest and film it?"
So he explained the idea further.
And you know, smart watches,
they have a sensor on the back
and flashing lights and stuff.
Well, what the sensor sees when those lights
are flashing is incredible.
And Andrew wanted to see if we could recreate
what a smartwatch does when those lights are flashing
just using a camera pointed at my face
and some software that he wrote on his computer.
The most common light you'll see
on the back of a smartwatch is green
and that helps to measure your pulse
but you may also see a red light.
In fact, if your watch has a red light on the back,
it has another light that you can't see.
It has an invisible light on top of that.
And the red light and the invisible light together
help to measure the level of oxygen in your blood.
But let's look at the pulse measurement first
because that's the one we're trying to recreate here.
- I'm gonna be filming you.
I've got a nice bright direct light source.
My wife's a doctor.
This is her stethoscope.
And what we've done is, highly technical,
I've just shoved a microphone
and that's gonna allow us
to listen to the sound of your heart.
- Okay. - And also under here,
I believe, we've got a heart rate monitor.
Because you're measuring the electrical activity directly.
- Yeah. - Like it's gonna be
much better than the rubbish I get off this camera.
- So this is to see whether we're any good.
But first things first,
I need to check whether I've
got this in the right place.
Do you know what? (heart beats)
I think I've nailed it.
- I'm gonna start recording
from that chest strap. - Yeah.
- I'm gonna set my camera going.
- Yeah. - And we're gonna see
if we can detect your heart rate using just your face.
- Okay. - Try and hold still.
All right, that'll do.
There is your face.
It's going through and it's processing this video.
So it's measuring the color of all the different pixels
on your face.
- Your software is trying to find the ups
and downs in those red, green, and blue channels.
And then it's
amplifying them. - [Andrew] Exactly.
- [Steve] And laying it on top, okay.
- So you can imagine this is actually
quite medically useful.
Imagine you've got a prematurely born baby.
It was nice if you could just remotely monitor them
by filming them basically.
- And obviously if the premature baby has a beard like this,
you would shave it.
Ooh, this is cool.
These individual frames that you generated
using your Python code will then turn into a video.
- [Andrew] You look a bit unwell in this.
- I do, yeah. Yeah.
I mean I look a bit unwell in this.
- These are the raw RGB values.
So I've subtracted that slowly moving background.
I've smoothed it out a little bit and then.
- [Steve] Look at that.
That's great, isn't it?
- [Andrew] What really interested me,
I thought one of the reasons they use green lights
is 'cause the green signal is the strongest.
And I've read this in papers.
And as you can see that probably is true.
- [Steve] Yeah. - The green peaks
and troughs do tend to be a bit bigger
and smaller, but at the same time,
there's actually quite a lot of data there in that red
and blue channel as well.
- A green LED is cheaper than a white LED.
- But the main reason is actually about how deep
these various different colors go into your skin.
Red's got a very long wavelength.
That means it travels really deep into your arm for example.
And so what that means is that you are,
you know, potentially getting distracting signals
from other places basically.
And blue light, by contrast,
it's got a very short wavelength.
And that means it doesn't go very deep into the skin at all.
So it might not get deep enough to see the blood.
Whereas green is this sort of nice compromise.
It goes in deep enough to see the blood,
but not so deep it's basically just detecting
muscles or something.
So if you had like really sensitive eyes,
you'd just be walking around
and people's face should be getting brighter
and darker and brighter and darker.
- Sounds like what it's like to be Superman.
That's weird. - It must be.
- He never talks about it.
- He can probably hear it as well.
(Steve laughs)
What a nightmare.
Because of that stethoscope, remember that,
we've got the sound of the heartbeat as well.
So we've got that.
That act as a gold standard
and prove that I've not just totally faked this.
So here we go. You ready?
(heart beats)
- [Steve] Look at that (heart beats)
- [Andrew] And what you can see.
Oh, and then you get a little twitch,
flash green. - [Steve] Yeah.
(heart beats)
It's very twitchy obviously.
(heart beats)
- [Andrew] Yeah, it goes woo-dump, woo-dump.
- Yeah.
That's not bad, is it?
The heart rate that you can see on my face
seems to match up with the audio of my heart
that you can hear.
You can see why this reminded Andrew
over the motion amplification video.
The difference is we are using the whole face
to work out what the pulse is
and then converting that pulse data into
a color overlay so we can see what's going on visually.
We also took an ECG using that strap around my chest
as an extra test to make sure
that this thing was really working.
- It's got two little contacts on your chest here and here.
- Yeah.
- And it's measuring the voltage across that
is basically the voltage, you know,
from one side of your heart to the other.
- And when you made me lick them,
you weren't just being weird.
- No. Cause it's really important
for the electrical contacts.
- Okay, sure it is.
- [Andrew] Absolutely beautiful.
- [Steve] Yeah. - It's so clean.
But these massive big spikes,
then we can just overlay this
lovely, clean, beautiful ECG trace.
- Yeah. - With that RGB video data.
So what you can see is the ECG.
Well, basically I can believe those peaks line up,
can't you? - Yeah.
That's not bad, is it?
I reckon we nailed that.
Here's a question though.
We were using a constant white light to illuminate my face
but watches use a green light that is flashing.
Why is the light flashing?
- They want to flash the green light on.
They take a measurement with their little
one pixel sensor. - Yeah.
- And then when the green light's off,
they can then measure the ambient light.
So the surrounding light.
Let's imagine a worst case scenario, you're in a disco.
Disco light's are flashing on and off.
They don't want that to distract their measurement.
- [Steve] Yeah. - So they need to subtract
that flashing disco light
and therefore by turning off that green LED,
they can subtract the constant ambient light around you.
So the other thing these watches can do, right,
is they can measure your breathing rate
using your heart rate measurement.
So when you breathe in. - [Steve] Yeah.
- [Andrew] Your lungs get bigger obviously.
- [Steve] Yeah. - [Andrew] And that means
the space inside your chest for your heart
gets a little bit smaller.
- No. - So if it wants to
pump the same volume of blood
around your body per minute,
it's gonna have to beat a little bit faster.
And what you can do
is you can watch for the changes in heart rate over time.
And that allows you then to extract the breathing rate.
- So that's what the green light is for.
It's your pulse and as it turns out,
your breathing rate as well.
But what about the red light?
Well, you may have seen one of these before.
It's a pulse oximeter, a device that attaches to your finger
and it will measure how much oxygen there is in your blood,
how saturated your blood is with oxygen.
And your smartwatch is doing the same thing.
They both do it with the help of a red light.
The reason they use a red light is because
the more oxygenated your blood is the more red it is.
So it shines a red light onto your body
and there's a sensor that detects how much
of that red light is reflected back.
The only difference with these things is
that the sensor is on the opposite side of the light source.
So it's detecting how much light passes through your finger
as opposed to how much is reflected off your wrist.
And because the amount of blood in your wrist goes up
and down a little as your heartbeats,
you'll expect to get a signal
from the sensor that looks a bit like this.
So the higher the line is
the more oxygenated your blood is
and the lower the line is
the less oxygenated your blood is.
But here's the problem.
That line will also go up
and down depending on how thick your veins are,
how much ambient light there is around you,
how thick your skin is,
how close the sensor is to one of your veins.
So there's actually no way to look at this line
and know how much oxygen there is in your blood.
So how do we fix that?
Well, the trick is to use a second light
that's a different color.
So different, in fact, that you can't see it.
It's infrared.
That's the invisible light I was telling you about
at the start.
This footage is from an infrared camera.
So the flashing light you can see here
you can't see with your own eyes.
And here's that infrared light on the pulse oximeter.
You remember I said that blood gets more red
the more oxygen there is in it.
Well, the opposite is true for infrared.
The more oxygenated your blood is
the less infrared it becomes.
So if you were to plot what the light sensor sees
in infrared, it would be up here for deoxygenated blood
and down here for oxygenated blood.
So if you were to look at both readings together,
this is what oxygenated blood would look like.
And then as you slowly remove oxygen from the blood
one line would go down
and the other would go up until they cross over.
And so now you don't have to worry about,
you know, ambient light,
where exactly is the sensor pointing on the wrist,
all that sort of stuff.
Because look, as these external factors
like ambient light vary,
well, these two lines move up and down
but they move up and down together.
The relative height of the two lines stays the same.
So that's the thing you measure
and that's the thing that can tell you
how much oxygen there is in your blood.
How clever is that?
Now because blood is typically
90 something percent oxygenated
and because oxygenated blood strongly absorbs infrared light
with an infrared torch and an infrared camera,
we can see what your smartwatch sees.
So this is what your smartwatch sees
when it looks at your wrist.
Of course, it's a one pixel image that your smartwatch sees.
We've got many more pixels here
so we can pick out the detail of the veins and arteries.
How cool is that?
But even freakier than that,
look what happens when I shine it into my head.
- [Andrew] That's absolutely terrifying.
- Me and Andrew had quite a wide ranging conversation
in the end.
So I've taken all that extra footage
and turned it into a bonus video for Patreon's,
link to my Patreon page in the description.
There's also a load of outtakes
at the very end of this video, so stick around for that.
I've recommended Andrew Steele's videos before.
If you haven't checked out his channel yet,
this is the perfect opportunity to do it.
His follow up video to this one about
empirically testing whether these different watch brands
are any good at all, is brilliant.
The link is in the description
and it's also in the card there.
Guess what's coming in the post.
- [Child] What?
- More KiwiCo.
(kids cheer)
One of the greatest challenges of parenting
could be summed up in a Venn diagram.
Here are all the things that kids enjoy and want to do
and here are all the things that are good for kids
that nourish them and enrich them.
And the overlap between the two is razor thin.
It feels so good to find something in that overlap,
which is why I've been getting KiwiCo for so long.
KiwiCo is a subscription service
where you get a STEM project in the post once a month.
Everything you need for the project is there in the box.
And there are nine different subscription lines
for every possible age group.
This is from the Kiwi subscription line
which is for five to eight year olds.
It's actually got two projects in it.
One is this disappearing coin box.
My son loves magic tricks,
figuring out how they work, and explaining them.
So this one is perfect for him.
This one is a game that you can play with
a visor that has a mirror in it.
So you're doing the puzzle that's on the ground
but you're looking straight ahead through a mirror.
And it sparked all these conversations about
how your brain communicates with your limbs
because you tried to move something up
but it moves down because
you're looking through this mirror.
Because we've been getting it for so long,
I feel like I can talk about
the long term effect it's had on my kids.
Like there's such tinkerers now.
We just got this dancing singing Rudolph
out of storage for Christmas.
And my daughter didn't want it to sing,
she just wanted it to dance and for its nose to flash.
So she said, can we take it apart
and figure out where the voice box is?
So we unscrewed everything
and we saw the bit that makes it dance,
the wires that go to the flashing nose,
and we found the wires that go to the speaker.
We yanked them out, put it all back together again,
and everything worked except for the speaker.
And that's just how my kids look at the world now.
They see something
and they start thinking about how does it work.
The offer on this one is really good.
If you go to kiwico.com/stevemould,
you'll get your first month for free.
You can also buy individual boxes one off as gifts.
I've done that before.
Best on clever.
The link is also in the description.
So sort out your holiday shopping today.
- Now I'm listening to your abdomen sounds.
- I'm hungry, okay.
It's mansplaining my heart.
- Not that sort of doctor.
- No, no. You've got PhD. - I've got PhD.
- Oh, show off.
I'll get emails from people sometimes saying professor.
- Professor Mould.
- Yeah. - Totally.
- Thanks.
- Professor of YouTube.
(Steve laughs)
- Could you make the text smaller though
because it's almost like-
- [Andrew] Okay, I mean, it's a waste of space, isn't it?
It's gonna be beautiful, isn't it?
I mean, look at those colors.
- [Steve] It's just, I mean, it looks horrendous.
- [Andrew] Look smurf.
- [Steve] Yeah.
Showing your code in a video is just
asking for code review in the comments.
- [Andrew] Oh, it's absolutely,
I mean, what is all this commented at, nonsense.
- [Steve] And what are you doing?
- [Andrew] Why have I, I mean,
why have I written all these functions that I never use
at the top of the file?
- [Steve] Oh, it's self-documenting it.
- [Andrew] It certainly is.
Yeah, yeah, totally.
So this is a picture of-
- It's not a picture, it's a graph.
I know you're a nerd, but that's not a picture.
- That's true. It is a graph.
And what we're looking at here.
Yeah, so- - You run
at a very steady pace.
- I mean, I guess most people do.
I don't know if I'm- - I don't know.
- Am I the Terminator?
- Yeah, it does.
I'm a bit freaked out by-
- It's remarkable. - How robotic you are.
- This wasn't a perfectly controlled experiment.
- Yeah. - You had a
scary guy just out of shot.
- There was. I dunno why,
well, it's 'cause while you were filming I suppose, so yeah.
You had to be there, didn't you?
- I had to be there. - Yeah.
- Not much I could do.
- Yeah. It was very intimidating.