- [Linus] This might look like a GPU.
Smell like a GPU.
Sticking in a PCI Express 16x slot like a GPU
and even talk like a GPU.
(muttering gibberish)
But I assure you,
its intended purpose is very different
and to some potentially terrifying.
It's dubbed the ASUS IOT AI Accelerator PCIe Card
or CRL-G116U-P3DF for short.
And it's designed for artificial intelligence computation.
Speaking of intelligence, you'd be intelligent to know
I'm going to tell you about our sponsor Honey.
Honey is the free to use shopping tool
that helps search for the best promo codes
on tons of your favorite sites.
Get it today at joinhoney.com/LTT
(upbeat music)
Historically, we've mostly shied away
from covering AI stuff.
And there's a couple of good reasons for it.
One is we're not AI or machine learning developers.
So finding practical use cases that we are able to set up
and demo for you guys can be a little tricky
and second, a lot of those use cases are extremely technical
behind the scenes kinds of things, at least for now.
So hardware like this,
isn't too applicable to end users like me and you,
that isn't till today.
While doing research for and testing
the home automation setup at my new house,
get subscribed by the way,
you don't want to miss any future content
around revamping that place.
We stumbled into a bit of a roadblock
regarding presence detection or the ability for the systems
in the house to be aware of whether or not
anyone is actually home.
Presence detection provides us with a number of benefits.
For example, we can turn off lights
and set the HVAC to be more energy efficient
when there's nobody at home.
And this is relatively simple to implement
on a whole house level,
but therein lies the challenge.
We put a lot of effort into making my house a lot more
granularly controllable, practically every room has its own
independent HVAC and lighting,
which is awesome for personal preferences.
Like the kids might want their rooms cooler
than my wife and I do for example or vice versa,
but it can also be used to improve efficiency
and save costs.
If no one's in the giant rec room downstairs,
what's the point of heating and cooling it.
I mean surely that can't be that hard to automate.
Well as we found out, it can be really freaking hard.
One idea we had was to install Bluetooth low energy beacons
in every room and read people's presence
through their phones.
The only problem with that is that people have this tendency
when they're at home to put their phone on a charger
and oh, I don't know, walk around their house.
So that won't work.
Another idea was to use motion sensors,
but those kind of crapped the bed
as soon as you decide to take a nap on the couch,
and then you wake up in a sauna of a room in the summer,
which sounds like a pretty bad time to me.
So this is where our AI card comes into play.
This is an ASUS designed product,
but all of the important AI bits
are actually made by Google of all people
under the brand Coral.
We've got both of them linked down in the description below.
Let's take this thing apart.
This specific card has 16 onboard Edge TPUs
as Google calls them.
And the TPU stands for Tensor Processing Unit.
In the most simple terms,
these TPU's are hardware processors
that are physically designed and optimized
to run a specific application.
You may have heard of something called an ASIC
as something that's really good at mining
a specific cryptocurrency.
Well, these are the same idea except that
instead of mining Bitcoin they are for running
Machine Learning Inference Calculations.
So what we're looking at here is essentially
just a PCI Express 16x interface here
going you can actually see the traces
going right into this PCI Express switch,
which appears to be just splitting out these lanes
into an M.2 2x interface for each one of these
little Dual TPU's here,
I was expecting something less
kind of modular and science fair project-y.
Now each of the Edge TPU of which there are two
on these M.2 cards can do around 4 trillion operations
per second with a power draw of only 2 watts.
So 2 watts times 16 TPU is brings us
to 32 watts for the card.
You add in your PCIe switch and fans
and that brings you to 52 watts,
which is well within the 75 watts
that this PCI Express slot could power.
But for some reason it still requires
a six pin power connector.
And it's got this gigantic copper fin cooler on it.
Now you're probably thinking,
wow, that's really cool Linus.
But most of this sounds like the boring technical stuff
that you were talking about earlier.
But if you were following closely,
while we were researching better ways to detect presence
in my new house,
we stumbled upon some software called Frigate,
that uses cameras plus this hardware
to really efficiently detect presence.
It's really freaking cool.
Let's take it for a test drive.
I got to put this back together.
And we're at this test bench is running
the latest version of unRAID in it
and it's going to stand in for my home nest.
Now you might notice there is also a GPU installed in it.
So we need that as a video output
because this is a non-APU Ryzen Processor,
but it's also useful for offloading
the camera video decode process from the CPU.
Now in my home deployment,
I'll probably just use the CPU because
I'm going to have 24 epic cores
that are otherwise going to go unused.
But for now, this is the better solution.
Now, once unRAID's booted up,
you can see that we have just a single array disc
with no redundancy.
That's not what I'd recommend as a production deployment,
but it's good enough for this video.
Now we just pop over to the community apps tab
and installed apps.
So we've got our Coral accelerator module drivers.
We don't need this if we're using a USB version
of Google's AI processor,
we only need it for the PCI Express version.
And then we've also got our NVIDIA drivers,
which are for our GPU.
Now it should be noted that
you can use your CPU for Frigate,
but performance is not good.
Next, we install Frigate.
Now we can't just launch it.
We've got to actually configure Frigate,
otherwise it's going to have no idea what devices
it's supposed to be using.
So step one, we're going to copy
the GPU UUID from our NVIDIA driver and paste that there.
Then for our TPU mapping,
we've mapped it to our Apex_0 device.
So that is only one of the 16 TPU's that are on that card
because we're only going to have a couple of cameras.
We're not going to bother mapping all of them,
but we did map five of them.
So that should be enough for our demo for now.
Wow this is a great password, Jake. I like it.
♪ Are you serious? ♪
Don't get overwhelmed.
Most of this is included in the template,
but there are a couple of things that we had to tweak.
So one is we had to set up our camera,
so you can see we pulled our camera
from the writer's den
and we've got our network password right here.
We're just using the one camera for now.
We actually didn't take all five of our NPUs.
We've just got four of them configured here,
again this is total overkill for what we're doing.
And then we've also got this right here.
So we are limiting our frames per second
for the camera feed that we're using to 5 FPS.
That is all we're going to need in 99% of situations
to have functional detection
and adding more processing to this
would serve no real purpose.
One other critical thing back up at the top here
is that you can plug this into an MQTT server,
which is basically its way of communicating back
to your other home automation devices.
So this MQTT server 10.20.0.71
is running on our office home assistant instance.
Hey, 5 FPS of green.
I can see why you said we didn't need more FPS.
There's not a whole lot of movement there.
♪ It's peanut butter demo time. ♪
♪ Peanut butter demo time. ♪
♪ Well, yeah. ♪
- [Jake] Okay, That one camera is set up.
We have another one too,
but let's just...
- [Linus] Okay, hi there.
- [Jake] Oh, wait, hold on. I screwed up.
- [Linus] Oh! All right.
Do we not have the detection on?
- [Jake] We do but I, I didn't turn it on,
like, the bounding boxes.
- Bounding boxes are just little boxes that show us
what the software's detecting
and the degree of certainty that it has
for what that object is.
Humans aren't the only thing that it can detect.
But they're kind of the only thing that we care about
for home automation presence detection.
- You're telling me we don't have to detect
how many apples you have in your fruit bowl.
Oh yeah. Oh yeah.
Oh, you're, you're there buddy.
Oh, it's kind of detecting your feet.
- [Linus] Does it know who I am?
- [Jake] That part we have not done yet.
- What if It only sees like this much of me?
- [Jake] Yeah it still thinks you're there.
- [Linus] What if my legs show?
- [Jake] Oh, it lost you.
- [Linus] It lost me? What about a hand?
- [Jake] Oh yeah, it sees you again.
- Oh, really?
Okay, what if it's just so an elbow?
- [Jake] Oh yeah, it's detecting you.
- [Linus] What? Really?
- [Jake] Yeah.
- [Linus] Okay. Can it can it see me?
- [Jake] It's confused right now.
- [Linus] It's a little confused?
What about if there's just like a casual,
like hand there?
- [Jake] It's kind of freaking out.
So sometimes it actually sees two people
when he's kind of obscured but that's totally fine.
- Oh like, it thinks I might be like two people
having a (beep) behind the curtain.
- Yeah sure.
But it's fine because all we need to know
is if there's anyone, right?
- Right.
- Yeah. And (beep) aside.
- You don't want the heat to cut out during your (beep).
- [Jake] Okay, all right. - [Linus] You will get cold.
Man! You can do so much cool stuff with this,
but wait, there's more.
- What if you have a common area,
like a kitchen or a rec room
and you want to be able to apply personalized preferences
when you're present, like say have your Spotify music
with your specific playlist play when you're in the room
and not when Yvonne's in the room, right?
- Oh, I never even thought of that.
- You could walk around the house
and have Spotify playing here.
Oh, I'm over here now.
And now there's my Spotify over here.
- Oh my God. That would be so cool.
- [Jake] Turns out someone had that exact thought
and created Double Take,
which is a piece of software
that allows you to determine the face.
This is more creepy.
- Yeah.
- But it's all local.
- I mean we could just configure this to not log
any of this (beep), right?
- Yeah.
- Did a Linus appear?
- [Jake] Oh, Wow! Yeah.
- [Linus] Nice.
Now Double Take isn't available
on the unRAID community app store, at least not yet.
So we need to manually set up
the docker container ourselves.
Ben wants it set up.
We need to configure it to listen to Frigate over MQTT.
It should be noted that Double Take itself
doesn't do facial recognition.
It's just an API and interface to make hooking into
and training facial recognition software easier.
So the first thing we need to do
is grab some pictures of me,
which are going to be training data
for our facial recognition.
So I guess I need to go over there, right?
- We could go over there or we can just upload
some photos we already have.
Double Take currently supports DeepStack,
CompreFace and Facebox.
They're all machine learning based
facial recognition models.
DeepStack is on the community app store
so that's the easiest to set up,
but I tried it out and it wasn't really that great
with the limited training data that I had.
CompreFace, on the other hand,
it requires Docker Compose,
I have to run into VM, but it's scary good.
I put a couple of photos of Linus
and secretly kind of had him walk by without telling him.
And it was like, Linus.
And then I walked by and it was like, Jake
with like 10 photos of each of us.
- All right, let's try it.
Why do you have so many pictures of me on your hard drive?
(both laughing)
- Well you make my drive hard.
- Why CPU when I can see you pee?
- Why talk to you when I can leave the zoom?
How do I use a Mac, do you know?
- No! These stills are horrible.
- [Jake] It's Scary.
- [Linus] Good Lord.
- [Jake] It's you without a beard.
(Linus singing)
Oh, wow. Yeah, yeah, yeah.
- Oh my God, 95.8%?
How can it be that sure on unbearded Linus training?
- There was only one that it is actually confident about.
- [Linus] I mean 94% seems pretty damn confident to me.
- [Jake] You know what we can do is take all of these
and say train off of those data.
Now you wanna walk over there again?
- Yeah, for sure.
- [Jake] Okay. That's plenty.
There's so many Linus' now.
Linus, Linus, Linus, Linus.
Like there's a couple of times that it thinks it's James.
I think like once you train a good model with CompreFace,
I'd probably say that like, if it only accepted as,
as that person, unless the confidence was like 99%,
some of these other models,
Madison, also made with people's photos
with them wearing a mask.
So like it literally has the eyes.
Can you go back?
Run over there and just...
- All right, I'm going, I'm going, I'm going.
- See if it makes some more mistakes.
- I'm going to pull up a chair, Linus style.
- [Jake] It's very confident about Linus now.
(Linus struggling)
- Who else would behave like this?
- [Jake] I don't think that's what it's looking for but...
- Man so you could just kind of wander
around your house for 10 minutes,
find anything that's wrong, boom, train it.
- Part of it is also going to be specific cameras
because you know, each camera looks a little different.
- One thing I asked Jake about and he said,
it doesn't have this functionality for now is,
can it morph over time and can it continue to self-train?
- The thing with facial recognition
is like the more data you feed it,
the more uncertain it becomes.
- All of this leaves us with some questions.
Do you even need this thing?
Could a bog standard CPU do the same calculations?
The answer is yes, it could, but way slower.
To put it in perspective,
your average Quad Core CPU would probably only be able
to handle a few frames per second, on a Quad Core.
But because these Edge TPUs are so optimized
for running these specific calculations,
they are stupid fast at them.
A single Coral Edge TPU can handle around
a hundred frames per second of people detection in Frigate
and we've got 16 of them.
So if my rough math is correct,
that's around 1600 frames per second.
We might actually be pushing that
once we've got all the cameras in the new place.
- [Jake] But we're only taking only 5 FPS for me.
- Still, it's a lot of cameras.
- [Jake] I don't know 5 FPS.
- Yeah, five. Okay, yeah, man that's crazy.
So a single one of these TPUs could probably
handle 15 to 20 cameras at 5 FPS.
Another question this raises is,
are we worried about the security implications?
Fortunately, since we have all of our own hardware
and all of this is open source and locally hosted,
none of the data goes outside of my house.
So I'm actually pretty happy
with the level of security and safety.
As for being worried about AI taking over,
no one's asking that question.
So it's in here anyway.
The answer is no!
Leaving only one question then, who's our sponsor?
FreshBooks.
FreshBooks is the simple accounting solution
that is designed specifically with you in mind,
the small business owner.
It features built-in automation
that allows you to spend less time tracking projects
and more time growing your business.
So whether you're a trades person,
creative agency or a YouTuber,
you can choose the plan that's right for you
with FreshBooks.
They have an Award-winning Toronto-based support team
who are always happy to help you if you need it.
And you don't have to take my word for it.
You can try FreshBooks for free for 30 days today
with no credit card required
at freshbooks.com/Linus.
If you enjoyed this video,
maybe go check out the video where we built the server
that is going to be running all this stuff.
We're going to have to get a little bit creative
with PCI Express expansion on it,
but that's the machine that's going to somehow run
this full AI card.
- [Jake] Yeah, yeah. - So good luck everybody.
Are we going to do an external PCIe box?
We should totally do it.
- [Jake] Yeah, we should totally do it.
- Yeah, that'll be a video.
Get subscribed.
- [Jake] We have water cooler and everything.