I haven't talked about texture access in fragment shaders. The short answer is,
you can access textures in fragment shaders. The theory of sampling and
filtering is the same as you've already learned in previous lessons. Most of
the rest is just the syntax of setting up the texture access. The key bit is
that in the shader you use the texture 2D function. This gives back the color
from the texture which you then use as you wish. Search the Three.js code base
for this keyword and you'll find more than 40 shaders that use texture access.
The main file for material shading is WebGLShaders.js, and it's worth your time
to give it a look if you want to become better with shaders. The other function
to look for is textureCube for cube maps, which takes a 3D direction as an
input. Many of the rest of the shaders perform image processing, where the
pixels of the image are used to form another image. Let's take a concrete
example. You want to convert a color image to grayscale. The formula is
luminance equals this much red, this much green, and this much blue. This
formula is for linearized colors, and luminance is intensity, the grayscale.
There's also another formula for luma, for when the inputs are not gamma
corrected. It's surprising to me that in both formulas how important green is
and how little blue matters. Whichever formula you use, you might think you
could simply add it as a final step in a fragment shader. However, transparency
again is a problem. You want to apply this formula as a post-process. That is,
you want to convert to grayscale after the whole scene is rendered. But by then
it's too late. The image has already been sent to the screen. Well, in fact,
you can send the image off-screen and have it become a texture. This is called,
simply, an offscreen texture, pixel buffer, or render target. This texture is
not typically not a powers of two texture, it's not normally 512 by 512, or
anything that you'd use to make a mipmap chain. Our goal is to perform image
processing on this so-called texture. We want to read a pixel and convert it to
gray. You'd think there'd be some mode that could just take one texel and spit
it out to a new pixel, but the GPU is optimized for drawing triangles and
accessing textures. The way we use this texture is by drawing a single
rectangle with UV coordinates that exactly fill the screen. You'll hear this
called a screen-filling quad. The UV values are then used in the fragment
shader to precisely grab the single texel that corresponds to our final output
pixel on the screen, one for one. This sounds inefficient, and in a certain
sense it is. But this process is fast enough that often a considerable number
of post-processing passes can be done in each frame. In Three.js, the Effect
Composer class let's you create and chain different passes together with just a
few lines of code. For our grayscale post-process, the vertex shader is along
these lines. This is almost as simple as a vertex shader can get. It copies
over the UV value and projects the screen-filling quadrilateral to clip
coordinates. The whole point of rendering the rectangle is to force the
fragment shader to be evaluated at every pixel. The fragment shader code is
also quite simple. The texture is accessed by the screen location, essentially.
So each textel will be associated with its corresponding output pixel. We use
the color of this textel to get the grey scale color, in this case using the
luma equation. This color is then saved to the fragment's output color and
we're done. But we don't have to stop there. Multiple post-process passes can
be done, and it's often necessary or even more efficient. For example, a blur
can be done in two passes, a horizontal blur and then a vertical blur. Doing so
results in fewer texture look-ups than a single pass. Multiple passes can be
quite memory efficient as the input texture from one pass can become the output
texture for the next, and vice versa. This process is called ping ponging, as
the flow of data goes back and forth. Here the horizontal blur uses the left
image as input, the right for output. The vertical blur uses the right as
input, left as output. Say we now want to convert to gray scale, we'd add
another pass and reverse the direction again. Here's an example of grayscale in
action. The critical idea here is that we can do all sorts of things in this
fragment shader. We can sample nearby texels and blend them together to blur
the image, detect edges, or 100 other operations. We can chain together each
post-process to feed the next. This demo by Felix Turner shows some of the many
effects you can achieve. Three.js, in fact, has lots of post-processing
operations in its library, and it's easy to add your own. I highly recommend
looking at the additional course materials for links to Felix's tutorial and
other worthwhile resources.