Hello, everyone.
Today, in this video, we will look
at some of the basics of parallel computing
and how we can implement them using MATLAB.
Before moving forward, make sure that you
have used other ways to speed up your code in MATLAB.
To do that, check the how to speed up MATLAB code video
link in the resources section.
To use the features of pilot computing,
check if you have the parallel computing toolbox
by using the command v e r on a local MATLAB command window.
Then you can see all the tool boxes
available with the parallel computing toolbox.
Why do we use parallel computing?
To solve complex big data problems by distributing data.
To save time by distributing tasks to workers
and executing them simultaneously.
And to take advantage of additional compute resources
by scaling up to the clusters and cloud.
To address these, we have different methods in MATLAB.
You can speed up your iterative code using parallel computing
and parfor or parfeval.
You can scale up your computation
using parallel computing with functions
that support distributed arrays and with big data processing
features, such as tall, datastore, and mapreduce.
You can use gupArray to speed up your calculation on an NVIDIA
GPU in your computer.
You can use batch to offload your calculations
on your desktop or to computer clusters or cloud computing
facilities.
Lastly, you can also scale up to clusters and clouds.
If your computing task is too big or too slow
for a local computer, you can offload your calculation
to a cluster on site or in the cloud using
MATLAB Parallel Server.
Based on your problems and compute,
we can decide which method to use.
But in today's video, we will be digging deeper into speeding up
the code using parfor.
To illustrate it, I will be using an example
of integer factorization.
The idea behind this example is a problem
which is computationally intensive
and where the complexity of the factorization
increases with the magnitude of the number.
Here, we create a vector of prime numbers
in 64-bit precision and multiply pairs of prime numbers
randomly to obtain large composite numbers.
We also create an array to store the results
of each factorization.
We then use a for loop to factor each composite number
and measure the time that the computation takes.
Here, we observe that a for loop takes around 43 seconds.
The iterations in the previous for loop are independent.
And so now, to enable parallel computing,
we use a path for loop to distribute iterations
to multiple workers.
What do we mean by worker?
Parallel worker is a MATLAB computation
engine that runs in the background on your machine,
or the cluster.
A collection of workers is referred
to as a pool of workers.
Now let's run parfor loop.
When we use parfor and we have parallel computing toolbox,
MATLAB automatically starts a parallel pool of workers.
The parallel pool takes some time to start.
The default [INAUDIBLE] profile is local.
WIth this profile enabled, MATLAB
creates as many workers as physical cores
on your machine for the parallel pool.
Here the number of workers are four.
You can control the parallel behavior
using the parallel preferences.
Go to Home tab.
Select parallel and parallel preferences.
Now run the code and measure the overall computation time.
As you can see here, a parfor loop takes 20 seconds.
Because the workload is distributed
across several workers, the computation time
is lower than a previous for loop.
Now to measure the speed up with the number of workers,
we run the same code several times,
limiting the maximum number of workers.
For this, we first define the number of workers
for each run up to the number of workers in the pool
and create an area to store the result of each test.
Next, we use a loop to iterate through the maximum number
of workers and run the previous code.
To limit the number of workers, we
use the second input argument of parfor.
Calculate the speed up by computing
the ratio between the computation
time of a single worker and the computation
time of each maximum number of workers.
To visualize how the computation scale up
with the number of workers, we plot the speed up
against the number of workers.
Observe how the speed up increases
with the number of workers.
With one, with two, and then with four.
There are many factors that can contribute
to the speed up be achieved.
Some common factors that contribute our difference
in the execution time for the iterations,
access to processor memory cache,
and overhead of distributing work.
Also note, that for any problem, there
is always a point at which using additional workers
stops providing additional benefit.
So it's always good to try scaling with a few workers
before trying to scale with a large number of workers.
To understand more about when to use and when not to use parfor
and some troubleshooting techniques
check out these links in the resources section.
Thank you.