Let me come back to one piece of this code that needs a little more explanation,
and that's this kernel call.
Here's the name of the kernel call, square,
and we call it with these launch parameters with these arguments.
There's a lot going on in this call so I need to explain details that we didn't need in our example
that are necessary as we move forward.
What I told you is that we were launching one block of 64 threads, and that's absolutely true.
But let me explain what's happening under the hood in a little more detail.
When you launch a kernel, you specify both the number of blocks and the number of threads per block.
In our example, we only had one block of 64 threads.
But we want to run bigger problems than this.
What you need to know about the hardware right now is two things.
One, it is capable of running many blocks at the same time.
Two, each block has a maximum number of threads that it can support.
Newer GPUs can support 1024 threads per block; older GPUs can only support 512.
When you have lots of work to do, you'll divide that work into any number of blocks
each of which had no more than 512 or possibly 1,024 threads.
If we wanted to launch 128 threads and square the values in each of them instead of 64 threads,
we could change this call to square of 1,128.
If we wanted to launch 1280 threads instead we could call square of 10,128,
launching ten blocks of 128 threads each.
Or square 5,256, we're launching five blocks of 256 threads each.
But we can't call square 1,1280 because that's too many threads per block.
You should pick the breakdown of threads and blocks that makes the most sense for your problem.
As we saw before, each thread that we launch knows its index within that block.
And you won't be surprised to hear that each thread also knows the index of its block as well.
We'll see how we access this information in a moment.
Now how do these kernels actually map into threads and blocks?
Well, when we square 10,128, we're going to launch 10 thread blocks of 128 threads each.
When we square 5,256, we'll launch 5 consecutive thread blocks of 256 threads each.
These diagrams are one dimensional, they only progress in one dimension, the x dimension.
That's fine if your problem is one dimensional.
But many problems are 2 or 3 dimensional.
You'll be doing image processing in the homeworks, for instance,
and that's very definitely a two dimensional problem.
It makes sense that CUDA would natively support not only one dimensional layouts of blocks and threads,
like we're showing here, but also 2 and 3 dimension as well.
For instance, perhaps we like to process this 128 x 128 pixel image.
We'd like to launch one thread per pixel.
We might choose, for instance, to launch these 128 x 128 threads
as 128 blocks in the y-dimension where each one of those blocks
is a 128 by 1 block of threads in the x-dimension.
Or we might instead choose to launch an 8 x 8 grid of blocks where each block is 16 threads by 16 threads.
So CUDA supports 1, 2, or 3 dimensional thread blocks.
We can also arrange thread blocks into 1, 2, or 3 dimensional grids.
Now, let's return to how we launch kernels.
We put two parameters in the triple chevrons.
The first is the dimensionality of the grid of blocks,
and the second is the dimensionality of the threads within a block.
We can specify up to three dimensions for each.
But if we don't specify a dimension, it defaults to one.
More generally, you can specify a 3 dimensional configuration for grid of blocks
or block of threads with this dim3 struct which you initialize as dim3 x,y,z.
Recall that, again, if we don't specify y or z, they default to 1.
When we say dim3 of w, that means the same thing as dim3 of w,1,1,
and you can actually abbreviate this with simply the integer w.
When we specified square of 1,64, that was equivalent to square of dim3 1,1,1, dim3 64,1,1.