Ok, so here's another important aspect
about image processing that we left out
in the previous video.
(clears throat)
So what did we do?
We had, an image of a frog
we processed every single pixel and we
copied that pixel to our processing display window and maybe we manipulated it while
copying it, we made it brighter we changed it's color, we did all sorts of things.
This is, I would say, pixel point processing.
What we are doing is there's a one to one
relationship, for this pixel
change this pixel
for the next pixel, change this pixel
for the next pixel, change the next pixel.
However, most sophisticated image processing
algorithms that you might find in
not to keep using photoshop as an example
photoshop or a ** image processing.
actually process goups of pixels
for example,
let's say this is my frog
pixel
my frog image
and this here is my processing window,
and let's say this is the pixel I want to
set here in my processing window.
What if inorder to set this pixel I actually
used a whole bunch of neighbouring pixels
around that pixel, so that pixel
is over here
I don't know what am drawing here exactly umm but I actually took all of the
pixels in five by five matrix around it and this is the roo.. the color of this pixel
is a function of all of these pixels combined.
Ahh aah, a classic image processing
algorithm that you might
have used before is a blur.
What if you take a pic.. an image
and blur it making it fuzzier,
less sharp, less in focus.
Well actually if you average all of the pixels
of an area
to set a particular pixel, that's a
blur algorithm.
It is making each pixel
more like it's neighbours
almost ahh bluring it, averaging it,
smoothing it between pixels.
So this is a actually an algorithm
and sharpening it is the opposite
taking each pixel and making it more
different than it's neighbours.
Umm another example of an algorithm is finding the edges
in an image. So if I have a silhouette of a person
(chuckles)
this looks nothing like a person
and I don't know..
some weird ghost alien thing
and I wanna find the edges of this person
well let's say this person was wearing
a black shirt,
much like the.. my black shirt here
how would I find the edges?
Well, if this is
if you're looking at an image of me
the pixels that are very different
the white pixels and these black pixels
a pixel that's very different
from it's neighbours is an edge right?
(finger taps his microphone twice)
this pixel over here is very similar to
all it's neighbours.
This pixel right here is quite different
than it's neighbours it must be
on an edge.
So, finding the edges of an image
has to do with looking for pixels that
are very different from their neighbours.
Let's actually see if we can,
implement this really really quickly
in our frog example.
(taps to switch cameras)
So if I come over here, to the frog
and if I look at this froooooog
and what where we doing here?
We were ahh kinda doing this threshold
thing, that's where we last were.
Well, let me say, let's aah
take all this out and
let's take all this out.
Here is a particular pixel location,
what if I want to look at this pixel
relative to the pixel to it's right
or to it's left, or above or below.
Well, what will that pixel be this is
location1 another location might be
(x+1) times y times width
and if you look at this what is
the difference here?
Now am saying, let me look at pixel x + y
xy and let me look at pixel x+1, y
right? Here ahh if am over here and then
am in some image
if this is pixel x - oh you can't see that.
If this is pixel x, y then this right here
right next to it is pixel x+1, y
so ofcourse a more sophisticated algorithm
migh look at x+1 and x-1 and y+1 and y-1
x+2 and y+2 aloot of pixels
in the neigboring area but just to
start really simply let's look at this pixel
and the one next to it.
(taps to switch camera)
So if I'm looking at this pixel and
the one next to it, I can look at both
of the brightness of both of those pixels.
b1 equals frog.pixels[loc1];
b2 equals frog.pixels[loc2];
Aaah those cameras are always shutting off
am gonna fix that in the middle
of this video look I turned black
for a second and am back
and am just going to fix this one
you'll have to bear with me, there's
nothing I can do about this right now.
ohhh come back (taps twice)
Ok, am back I'l edit all that out later
ok so now uhh I could get a
difference which is maybe am just gonna
say the absolute value of (b1-b2)
and let's set the pixel of the window
equal to that difference
noow, oops loc1
so loc1 is kind of my core original
location so I wanna set that individual
pixel but I wanna look at it as a function
the value of that pixel as a function of
two pixels that are next to each other
let's run this and by the way
we got this sad, very difficult, really
traumatic array out of bounds exception
this error is something that will happen
to you as soon as you start doing any
image processing work. What happened?
I tried to look at, investigate,
think about access a picel that doesn't
exist, because at some point I was at a
pixel that was all the way at the edge
of my image and I tried to look at
pixel one more to the edge
which doesn't exist.
So there's alot of strategies around this
problem for right now am just going to
do something which is kind of simple
which is say let me just look at all the
pixels all the way up to one before the edge
and that will kind of like deal with this
problem but you can think of other things
you could have pixels wrap around
there's lots of other ways you could
approach this problem
but now we can see
you know this isn't the perfect
demonstration but I've kinda really got
something here the the frog by the way
was had a lot of similar colors so am
getting ummm umm
actually am not exactly getting
what I expected I guess the frog actually
didn't have lots of similar colors it's
very different cuz we're getting we're
getting a very bright color everywhere
there. I guess there's a big difference
of all the pixels inside the frog
and the background is very
noisy so there's a couple of things
we're noticing here; one - this isn't
such a great algorithm am only looking
at the pixels to the left coming to the right and to the left if I looked at the
pixels above and below and diagonally
I might be able to uhh determine
alot more about the image
the other thing is - this particular
image has alot of noise in it
so I'm am there's so much noise that
am not really getting accurate edges
maybe blurring the image first and then
looking at it looking to the edges might help, but in the end am actually
visually able to see something here
but this wasn't the exact demonstration
I was hoping for but uhh
to.. look at this, maybe try to
oohhh I know what the problem is
huuuuuh (sighs deeply)
am gonna have to re-record this video
but what did I just do here? I did the thing that I did originally
which is just I looked at the arbitrary difference between the giant integer value
you're probably all just watching this like yelling at the screen like you're
doing it wrong, what I wanted to do was
look at the brightness of each one of
this pixels just forgot like I wanna
compare the brightness of each one of
these pixels. This is gonna give us
something much much more umm
oh yeah and it's very hard to see here
but actually this really worked.
I'm really getting the edges I'm really
getting the edges cause the edges of
the frog that's where the pixels are most
different and infact if I want
to be a bit more thoughtful about this
I might just come up with some threshold
like if the pixel values are at all
different then,
draw black otherwise draw white
and now if I run this, we can see
aah I'm really starting to see the
contours of this frog. I think this could
be a better demonstration. So here
we're seeing just a really basic
algorithm for looking at the comparison
between two pixels and finding the edges
now again this was a very simplistic
approach most of the time what you wanna
do wooah haha all that work and this
camera is not even on umm most of
the time what you wanna do is look
at a whole selection of pixels and in fact
this is something that you might not
be aware of this is right there bring up
photoshop for a second if I go to
photoshop and I've got my frog loaded
in here and I go to filter ahh ahh ah
other custom what I dont..
what your actually seeing here
is an and this is hard for you to see
is this are the weights of a pixel
If I wanna make a pixel on the center
much more important than the ones in
the area, that's a sharpening.
I want the pixel to be more different
than it's neighbours I actually ahh
repr.. ahh oops I created very quickly a
blur and you can see here
this this would be a blur, what if
I take all the pixels in a huge neighbour
space, add them all together
divide them by 25 as an average
and you can see here that this
umm frog is now blurred am gonna uncheck
the preview let me kind of zoom in here
umm you can see ahh over here
look over here, that frog is blurred
that frog is not blurred so this
particular algorithm is blurring that
image by adding all the pixels together
and infact umm you know am not
gonna go through the code for this
in detail but we can look at this
right here so if I look at this particular
example this this particular example you can
see here if I zoom in to it that the
pixels inside the rectangle are being
blurred and the pixels outside are not
you can see kinda all the arm above me as
I move the rectangle over am kinda
blurring out that pixel those pixels
if I go back to the code, and I put in
these weights this is actually
sharpening, again I want the pixel
on the center to substract the pixels
around it to be more different than
the pixels around it and if I run this
this is a bit more obvious, we can see
that this is me sharpening that image
now there's alot of noise is happening
here am very zoomed in but you can
see how this is changing the
quality of the pixels so comparing the
pixels and its neighbours this idea of
looking at a pixel and looking at the
ones that are right of it or above or
below is an important powerful
technique in image processing
and we can just you know this video
can go on for hours and hours and
look at every image processing
algorithm that photoshop uses
that anyone uses and we can see this
technique in alot of them. I guess
what am saying here is see what you can
do with this idea.
What if you umm what if you ahh umm try
to recreate this edge detection algorithm
but actually use more than just the two
pixels that are to the right and left
what if you use the pixels above or down
what if you actually were able to
interactly select whether you're looking
at the comparison of vertical pixels
or horizontal pixels there's lots of
possibilities you might explore and try
here and actually something that I
didn't do which I think will be
really interesting to see which I
would like to maybe add to another
video at some point and I.. think of it
as what happens
when we're processing the pixels
this pixel goes here this pixel goes here
what if I start to move the pixel around?
what if I take, what if every time I look
at a pixel I actually offset put it in
some like random location near where
it originally was this will actually give
me almost llike a watery effect because
the pixels start to like look like
they're dispersing or not. This this is an
idea that I'd love to demonstrate
so I think this video was already quite
loong so am not gonna do it here umm
but you can think of this as an excercise
and I'll try to add one in at some point
too that does this.
So here are the things that are missing
umm am gonna upload these so if you're
watching these in the first pass
this is where it's gonna stop
but what I wanna get to in the next set
of videos, number one I wanna look at
moving the pixels around which will
also be relevant to the idea of what
happens if we just sort of like forget
about this way of doing things entirely
it's like this one to one pixel idea
what if I just have a system where am
drawing you know patterns and shapes
to the screen maybe I draw grid of
rectangles or I draw random circles
everywhere or I draw swirls swirly spirals
all over the screen what if just use an
image essentially a database of color
so what if I could like look up colors
doing form a drawing algorithm
create a painting for an image
that's one thing I'm gonna look at
and once we've done that, the next piece
that I wanna look at is what happens
when this images aren't static
what if I'm getting images from a camera
that are changing live real time images or
images from a movie file.
So those are the I guess there's like
three more three/four more videos that
need to come to kinda finish off
all of these concepts.
Ok so send me your feedback, umm I'll
be happy to add more things, delete
things, redo things or just go away not
come back. (laughs) Ok so have good..
enjoy your morning, evening, night.
I don't know when you're watching this and you're just like eating some cereal
at home that's... something you might be doing and
enjoy that cereal (laughs) talk to you later.
Subtitles by the Amara.org community