Welcome back Last video we talked
about unified shared memory The fact that a programmer
can virtually see a one unified memory space across the host and other devices
like GPU or FPGA makes memory accesses much easier.
And it is great that SYCL is executing kernels asynchronously and in a non-blocking manner.
But it is also means that race conditions are possible..
And you as a programmer have to make sure data coherency is kept.
SYCL has several ways to control such execution dependencies.. Let me show you in a minute
A simple example. And all the code snippets could
be found in the video description.. So I have a Q, attached to my GPU.
And data, defined as unified shared memory, is an array of size N,
filled with 1s, And I’m using malloc-shared,
so the memory is shared between the host and the device mapped to the Q, not any device..
And now.. Kernel A is filling all the entries of data with 2.
This kernel uses parallel-for, so basically every iteration, every work-item is independent..
It will be launched and executed asynchronously, on the device, on the GPU
and it is not blocking, so the host will continue its work..
The host will continue to Kernel B,
I’m updating each of the entries to be 3 And again N work-items will be launched on the GPU
And Kernel C is updating data to 4.
All of these kernels will be running asynchronously, and will compete for the data..
There is no guarantee what will be the execution order..
And hence no way to know the final results as they could be indeterminant.
We must of course free up the data at the end of the program..
But as it is written here, the host might free up the data
while the work-items have not yet finished all and are still working on the GPU…
Be careful. This is a race condition.
So a barrier before we free up the data is required.
We can add a wait command separately like I did here or at the end of the last kernel…
But what about the data races?? This code is far from being “Thread safe”
One way is to add a barrier to each of the kernels,,
So we can add wait() commands to each kernel And now we can make sure that a kernel will
be launched only after the previous one has finished all its work-items..
This will block or synchronize the execution on the host.
A different way is to use an in-order Q.
A wait command will synch the execution on the host,
An in-order Q, does not require the host interaction, but the device will manage the
Q work-items executions in order Might be more efficient..
So the device will make sure, that the kernels are performed internally on the device in-order
Getting the data back to the host requires another synchronization, on the host,
To make sure we do not print or free the data before all kernels completed..
Moving to in-order execution of the kernels might slow the execution down,
sometimes without a real need. So, there is a more precise,
and elegant way to define kernels’ dependencies to each other
I’ll better just show it by an example.. Again a Q to the GPU,
And array of data I define here a SYCL event and I call it K1,
And another event I call K2..
And another one K3
And I can define that K2 event, is dependent on K1 event.
And K3 is dependent on K2. And so on, for any flow graph,
as complex as could be. So now, the graph can
executed asynchronously, in a non blocking manner
Unless I define the dependency.. Please note that at the end,
I’m not waiting for a Q but waiting on an event, here waiting on Event K3..
All of these synchronization examples are basically doing the same thing
and should give similar performance. Feel free to try them all out and compare,
Link to the code bellow.. And yes, lots of resources to learn from
Books, webinars, tutorials Here are some links, enjoy