what's going on everybody and welcome to
another facial recognition video in this
video we're going to be talking about is
doing facial recognition on video and
then kind of expanding our art logic and
code a little bit from there so just to
do something a little more interesting
so where we left off we were basically
just doing this on images and we had
this known faces door and then unknown
faces derp and unknown faces is where
the images were that we wanted to do
facial recognition on and see if hey do
we know this face so now we want to use
video instead of the unknown faces
directory so what I'm going to go ahead
and do is just comment that out and we
actually aren't going to need the
unknown faces directory anymore because
really it's going to be our video file
or video source that is going to be the
unknown faces so the next thing we want
to do is actually load in that video so
I'm going to say video equals
whoops I'm going to say equals CB 2 dots
video capture and then here you'll
specify either kind of an index number
for the video so it's either video 0 1 2
and and so on in my case that'll be a
video too otherwise you also could could
put in a file name here as well and
later on we are going to use a video
file but for now I just want to use a
video feed since I think a lot of people
were interested in that as well plus
might as well show both so ok so now we
have our video feed we're still going to
load in our known faces that's fine and
then the only other thing that really
changes is here so instead of saying for
file name we're actually going to put
this in a while true loop and then we
don't need file name anymore we don't
have the thing that's going to replace
image now is going to be reading for
mark capture so instead we're gonna say
actually it's gonna be ret in an image
and then we're going to and that's just
like return image and then it's video
dots what is it is it capture I think I
always forget this video
okay so so video read now we have our
image and then we do allocations and
coatings and in this case we're actually
already using OpenCV to get the image so
we don't need to do this conversion
anymore because it's already in BGR and
in theory you might need to do the
conversion before the encoding is I
suppose or even locations but it seems
to me through playing around it the face
rec works no matter what if you're using
BGR or RGB so I don't know how big of an
impact color is playing if anything it
might even in the back end be converting
things to grayscale for all I know if
you know let me know I actually do not
know what they're doing at like I said I
won't be surprised if they convert to
grayscale at some point okay so then we
show the image weight key we no longer
need to wait key and so we want to do is
have this running kind of indefinitely
so instead what we're gonna say is we're
going to throw in that kind of if C v2
dot weight key one and then this is the
code four and zero lowercase X capital F
F equals or Q basically if the user
presses the Q key we want to break and
then because I'm on a boom - I guess
destroy window just is what does not
work on a boon - so so one thing that
Daniel was saying was one way around
this is you can just try and accept so
you can try cv to destroy window accept
pass and can't just be done there cuz I
guess for some reason I'm going to that
this function just does not work so very
interesting okay with that I think I
think that's everything I've got my
trusty webcam here and let's see if that
works let's pull this up here we go and
we're gonna say it's Python I've renamed
this file to be video face rec example
dot pi let's go ahead and run that it's
gonna load in our known faces which
really is just my face and as you can
see that works decently well obviously
as the face starts to come up
little bit more you start to like lose
the the face but not bad okay so let's
go ahead and press Q and that should be
everything you want for video you know
from a video either a video file or a
video feed the same thing is going to
work you'll have your known faces you'll
look but to me that's a little boring
because that was like three lines of
code so now what I want to do is take an
orwellian turn and I'm going to say from
our video either video capture device or
a video file because I don't really have
lots of people passing by and say what I
want to do is I want to start labeling
giving people IDs so imagine you running
like an office building or a school or
something like that where you know you
have a bunch of people that are commonly
coming through that area so it's very
common to see common people but it's
very uncommon to see uncommon people and
then there's other places where it's a
little more rare to see common people so
think about like any place where
traveling so if it's not an employee or
something like that see like let's say
you're at an airport seeing common
people that have no good reason to be
there commonly that's a weird thing
right same thing if you depending on
your office building right there's some
places where you might have a bunch of
employees that you expect to see
commonly but then if there's some new
face that has no reason to be there
commonly you've got a problem so these
are kind of typical security issues that
are faced by you know businesses and and
stuff like that and even government
buildings kind of the same thing so what
can you do well using face rec we can
actually do this quite simply we can
find out either we can find out both of
those questions who are the common
people who are the uncommon people and
any time an uncommon person shows up for
example we can find out like instantly
so so yeah so what we're going to do now
is rather than using the video capture
because I'm the only person in the room
right now um we're going to use a video
file that I created so I will do my best
to share this file if I forget someone
just remind me but it's basically just
me holding up my cell phone and swapping
through
photos of various individuals and
actually I think I don't know how many
total we have made five we've got Obama
Trump Elon Musk and I think maybe
there's one with I thought there was a
Joe Biden one zone
yeah there's Joe Biden anyway you could
do this with an infinite number of
photos if you wanted this is just a
quick example I just made it purely for
debugging purposes so now what we're
gonna do
mmm is rather than the loading in the
webcam I'm gonna load in that video so
face face was it face rec video video
mp4 so that's all you have to do because
that video file is local to it to what
we're I'm working the next thing is we
want to actually rather than doing
really anything else what we're trying
to do is assign assign IDs to two people
so we don't really care about names
anymore we're gonna be working on IDs
later if you wanted to actually name
these individuals you just have a
dictionary that map's IDs to names but
that's not really what I'm going for at
the moment so instead what we're gonna
say is we're gonna yeah we're gonna load
in known faces and known names the other
thing we're gonna do is because we're
working in IDs so known faces right now
has names but soon it's gonna have IDs
so instead what I'm gonna do is I'm
actually gonna I'll make a copy of known
faces just in case and then I'm actually
going to delete everything in known
faces so right now we're starting clean
slate but later this will have IDs in it
so minimize this and inside those IDs
rather than having images now we could
save an image so when you grab the
location of the face you could do its
ROI I can't remember its region of intro
I think it's region it whatever it is in
CV to its ROI you could grab that and
then save that in the directory it that
way later if you are doing like some
debugging or something you could see you
could see like who is this person
actually like you show me a face of this
person and then later you could compare
that that face to a known database of
people so again if it's an office
building or something you've got
headshots of all the individuals for
their IDs or whatever the case you could
very quickly label all those individuals
when necessary so we don't need the
image anymore for now though
so we're gonna do that the encoding and
instead what we're going to do now is
we're gonna save the encoding because
again we don't care about the image so
much so having the image then encoding
the image just kind of makes the up
start time to take a while so instead
we're going to say it's encoding rather
than equaling that we're actually going
to say encoding equals and we're gonna
import pickle I don't think we have yet
yeah and we're gonna do is pickle or
encoding equals pickle about load and
then we're going to load the open of
file name open file name and in fact it
should be for name know establish your
file name and notice that Lister
shouldn't be I guess we would actually
really see for name okay so for file
name and then we'll say is name file
name so we're gonna open the file name
I'm pausing on that for a second because
I I'm pretty sure I've used just open
file name and that worked which is odd
to me that pickle would do that
successfully anyway I'll have to come
back to that well that's how pretty sure
it worked and that doesn't make any
sense anyway so we want to open that
file oh and we want to or B so that will
be our encoding object and instead I'm
gonna comment this one out as well
okay so now we have the encoding and
what we want to do is they known faces
we haven't had the append the encoding
and then here no names we append the
name which again in this case is going
to be an ID then what we want to do is
right now we're starting to clean sit
slate but eventually we are probably
going to start with at least some IDs so
what we're gonna say now is we're gonna
say if Len know
names greater than zero so if we have
more than you know if you don't one or
more what do we want so what we're gonna
say the next ID that we're going to sign
is equal to max' known names because
suppose just be a list of ID's plus 1
and then max of I can't really decide if
I encoding name this might also need to
be an int now I'm thinking about it I
think we'll do that what's even get away
with that and then else next ID will
start at 0 cool so now what we're gonna
do is we load in yep all this stuff
comes in we're iterating and then
basically the only thing that really is
now needs to change is if true we made a
match if we didn't right we're gonna say
else what we need to really know is you
know what's the next ID so we're gonna
say match equals string next ID and then
we want to say next ID plus equals 1 and
then like all this stuff is stuff we
actually just want to do regardless
right so we're actually gonna untap over
all those things and i think the rest of
this remains so I'm trying to think here
if we screwed anything up but I think
the easiest way to find out is to just
run it so let's see what happens surely
we get at least an error mm-hmm huge
okay so where's our CB to tie em show
here is so and so we'll just well just
make this empty empty quotes this is the
video title by the way and we were
putting filename there because we wanted
to show for the unknown faces file name
but we don't want to do that anymore but
don't have unknown faces I'll see what
happens this time okay so this time we
are getting constantly a new ID every
single time for that face so let's go
ahead
and what we what we did not do is what
we need to do is append right we need to
do way more things here so the other
thing we I don't think we've saved any
so we have we forgot a lot of things I
don't know why I thought that was ever
gonna work okay so we do the next ID but
then what we need to do is known names
we're gonna do dot append match not next
ID otherwise we continue raising that up
so I just need a lot more coffee this
morning no names and then known
underscore faces dot append in this case
for face so this would be face encoding
so we'll save those two things and then
what we want to do is save now that
we've got this new encoding and it's
only when we have a new encoding later
you could save more and I'll talk a
little bit about some of the logic I
thought of to like add more faces but
for now we're just gonna save the first
new face encoding so we're gonna say
known faces append face encoding which
also might be kind of sloppy you almost
might want to wait a little bit because
like the first face it finds its like a
sliding face in this example for example
and that's probably not the best face
for encoding but anyway known faces
append face encoding now we're gonna do
is oh s dot make Durer and again I'm
using f strings
I did ask so people had a good point
about the backslash but does not i'm
always thought the forward slash still
worked in python for me on windows like
if I was giving forward slash file names
I'll have to go back to Windows and see
if I'm incorrect there but anyway if
you're on Windows I guess use the
backslash I don't know and actually it
would be the double backslash okay known
faces derp /os not make derp and then we
have a new index which is going to be
the match so we make that directory and
now what we want to do is actually save
save that encoding so that would be
pickle dot dump and then
we will say it's like this is a poor
example of why I like kite so so now
it's not there for me there we go
I always forget the order like do you
dump the filename first and because I'm
pretty sure in numpy you save the file
name then the object I could I could be
wrong on that as well
but I always forget which one comes
first and it's just super useful to have
kite here to tell me no it's the object
first so yet again another shout-out to
kite the best autocomplete ever so
object file so too busy being enamored
with now so we're gonna dump the face
encoding and then the file name is going
to or the file rather is going to be
open and again we're gonna just do an F
string for known faces match and in this
case I think what I would say for the
file name you could just say match you
really could like you could say match
dot PKL and then what we'll say is WB
but I think what I want to do is prepare
us for the possibility later of saving
multiple encoding so what I'm going to
do instead we're gonna import time and
then we're gonna come down here match
and I'm gonna say - the int value of
time def time cool so a really long
filename but basically all we're doing
is we're just saving the match and the
time of when so the match is encoding
and when was that and that's it so okay
so we've got all that pickle dump so now
we have saved that we've attended it and
I think we're ready let's try again
we can either review our code or just
run it and fix the errors as they come
okay this time okay so Obama is ID zero
no real surprise there let's wait a
moment okay we've got a new person new
face ID one it'll be interesting to see
what happens as we have like two people
together and so on luckily we're still
constantly my biggest fear was the first
time we see like his face
it's as is the thing it's like look how
fast boom so now you can see even here
Obama is mostly zero but sometimes he's
getting a 5 Trump is mostly 3 now but
he's also a 4 this is clearly Obama
again idea zero and then try to think
who else worked okay we're still gonna
Biden coming in and then we still have a
mosque coming in so most of these are
pretty good faces so I guess they're
getting a good thing so so as we're kind
of waiting on this one a couple of
things I was thinking about was first
off a face okay so here we are
I think there's our first seeing of Elon
Musk's Oh Elon is id9 and always id9
hopefully I've been seeing it change yet
but we could actually have some output
or something and then now Obama
apparently has a new okay so figured
once I camp once it comes into focus is
like okay it's a zero and then Biden is
an 11 apparently but every time it goes
out of focus
Obama's like new people that's
interesting hold on let me yeah let's
close this and I forget what our
tolerance was what do we said that is
0.5 let's try to sit let's let's do
let's give it a little more we're gonna
say 0.6 so make it a little more
tolerant at recognize because look at
this we've got 17 IDs let's do let's
close that so one option we have is you
could you can either increase or
decrease the tolerance so let's let me
let me explain let me run this while I
explain so one option we have is to have
a pretty open tolerance so let's just
see how much this changes how many IDs
we wind up get
you see if this actually is popping up
at risk of pulling up OBS and an
infinite thing cool
so okay so so so far we just have the
208 you can't even see it the two IDs
now can you see it let's see yeah so we
just have the two ideas it seems to be
doing a little better at least so that's
good but we actually got a few new ones
now those are just like moments in time
so one of the things I was thinking of
is the face detection aspect of fate of
the face recognition kind of package the
face detection is really good so what I
was thinking about was could we use some
logic to say hey this is this is this
face like as this face is moving around
the face cannot change right like just
logically the face can't change okay so
we should be able to logically assume
that this is always ID zeroes face okay
it will never flop around
so especially as you're building this
database and anyway now we're still only
on five I think by now we actually were
a lot deeper so this seems to be better
to make the tolerance we'll give us a
little more tolerance so anyway one
option we have is you could actually set
the tolerance quite low and then have
knowledge that these IDs should not be
changing as time goes on and then every
time you see a new face or a new version
of the face also just as note with the
tolerance at 0.6 we can see we're having
really no trouble identifying either of
these people as that well Biden bounced
around a little bit but it did a better
job and same thing here so I think
before we had 17 total IDs now we're
nine we just got another new one let's
see if we end up with 10 doesn't look
like it okay cool so that's pretty
interesting so now as you gotten as you
get a new face you would be able to
recognize hey this is a new face or not
so basically every time you hit this
this else that's a new face so if you
wanted to do some sort of handling for
an uncommon face you could same thing
here if you wanted to handle for her
head we're seeing this face a lot you
could so that's all
so pretty interesting fairly over well
in I think of a few bad news cases but
yeah so pretty cool and like I said I
think you could add a little bit more
logic here and be way more accurate so
another idea I had too was like you know
with you you could have tolerance well
you first of all you could have two
tolerances right you could have a
tolerance for you could have like two
versions and then anytime there's a
disagreement you could have a little
more logic trying to figure out like
what is this face cuz chances are the
same person is likely to have a few IDs
in your system so maybe later you could
run through those or from time to time
kind of merge certain IDs because you
could probably figure out especially if
they were all together in the same frame
and that was just a moving face
chances are hey that's the same face
right so then what you could do is you
could merge those two so anytime you've
got IDs bouncing around like we've just
seen what you could do is you could
through logic you could say hey that was
always the same face which one of these
faces is the more common face and then
like hey let's make that the main ID but
what we'll do is we'll add one more face
encoding to that face so we could just
merge the two together so I think using
some pretty simple logic if maybe if you
guys are interested in that part three
could be me doing that and so as we see
a new face but yet it's on the exact
same frame that could that face could
not have changed hey here's a new
encoding for this face because
apparently that was a hard to recognize
example so hey let's throw that one in
there and then hopefully over time make
this much more robust so it would only
get better over time so anyway there's
ideas like that and there's a few others
that I had floating around in my head to
make this much much better but you get
the idea so okay I think that's
everything if you've got questions
comments concerns whatever feel free to
leave them below in this video is
sponsored by me if you guys are
interested in learning about neural
networks at a very low level from
scratch in Python learning how they
actually work rather than in plop
employing or applying whichever one you
want to go with on the script that's
fine how to actually how these neural
networks are actually working rather
than doing like cookie
models and pretty much just solving
things that have already been solved for
you chances are if you've ever tried to
work on a real-world problem you've seen
that real-world problems are much more
challenging than like eminence or
classifying dogs and cats or you know
whatever and if you want to kind of
level up your knowledge of how neural
networks work and how to actually make
something work on a problem that hasn't
yet been solved definitely worth
checking out the neural networks from
scratch book you can go to neural
networks from scratch which is n n FS
dot i/o and you can pre-order the book
now and eventually the book will be out
and you can buy it if you do pre-order
it you get access to the draft which is
live right now and you can comment and
ask questions in line with the actual
document itself and currently at the
time of my recording this hopefully over
time there will be much more information
there it's about I think 180 pages or so
we've got everything from building the
model to training back propagation and
all that so it's off to a really good
start and I'm really happy seeing all
the community engagement in the document
and stuff like that so anyway check that
out if you're interested
otherwise I will see you guys in another
video