Multiprocessing is the use of two or more central processing units within a
single computer system. The term also refers to the ability of a system to
support more than one processor and/or the ability to allocate tasks between
them. There are many variations on this basic theme, and the definition of
multiprocessing can vary with context, mostly as a function of how CPUs are
defined. According to some on-line dictionaries,
a multiprocessor is a computer system having two or more processing units each
sharing main memory and peripherals, in order to simultaneously process
programs. A 2009 textbook defined multiprocessor system similarly, but
noting that the processors may share "some or all of the system’s memory and
I/O facilities"; it also gave tightly coupled system as a synonymous term.
At the operating system level, multiprocessing is sometimes used to
refer to the execution of multiple concurrent processes in a system, with
each process running on a separate CPU or core, as opposed to a single process
at any one instant. When used with this definition, multiprocessing is sometimes
contrasted with multitasking, which may use just a single processor but switch
it in time slices between tasks. Multiprocessing however means true
parallel execution of multiple processes using more than one processor.
Multiprocessing doesn't necessarily mean that a single process or task uses more
than one processor simultaneously; the term parallel processing is generally
used to denote that scenario. Other authors prefer to refer to the operating
system techniques as multiprogramming and reserve the term multiprocessing for
the hardware aspect of having more than one processor. The remainder of this
article discusses multiprocessing only in this hardware sense.
In Flynn's taxonomy, multiprocessors as defined above are MIMD machines. As they
are normally construed to be tightly coupled, multiprocessors are not the
entire class of MIMD machines, which also contains message passing
multicomputer systems. Pre-history
According to a 1985 article in Byte, possibly the first expression of the
idea of multiprocessing is found in the 1842 words of Luigi Federico Menabrea,
which said about Charles Babbage's analytical engine: "the machine can be
brought into play so as to give several results at the same time, which will
greatly abridge the whole amount of the processes."
Key topics = Processor symmetry=
In a multiprocessing system, all CPUs may be equal, or some may be reserved
for special purposes. A combination of hardware and operating system software
design considerations determine the symmetry in a given system. For example,
hardware or software considerations may require that only one particular CPU
respond to all hardware interrupts, whereas all other work in the system may
be distributed equally among CPUs; or execution of kernel-mode code may be
restricted to only one particular CPU, whereas user-mode code may be executed
in any combination of processors. Multiprocessing systems are often easier
to design if such restrictions are imposed, but they tend to be less
efficient than systems in which all CPUs are utilized.
Systems that treat all CPUs equally are called symmetric multiprocessing
systems. In systems where all CPUs are not equal, system resources may be
divided in a number of ways, including asymmetric multiprocessing, non-uniform
memory access multiprocessing, and clustered multiprocessing.
= Instruction and data streams= In multiprocessing, the processors can
be used to execute a single sequence of instructions in multiple contexts,
multiple sequences of instructions in a single context, or multiple sequences of
instructions in multiple contexts. = Processor coupling=
Tightly coupled multiprocessor system Tightly coupled multiprocessor systems
contain multiple CPUs that are connected at the bus level. These CPUs may have
access to a central shared memory, or may participate in a memory hierarchy
with both local and shared memory(NUMA). The IBM p690 Regatta is an example of a
high end SMP system. Intel Xeon processors dominated the multiprocessor
market for business PCs and were the only major x86 option until the release
of AMD's Opteron range of processors in 2004. Both ranges of processors had
their own onboard cache but provided access to shared memory; the Xeon
processors via a common pipe and the Opteron processors via independent
pathways to the system RAM. Chip multiprocessors, also known as
multi-core computing, involves more than one processor placed on a single chip
and can be thought of the most extreme form of tightly coupled multiprocessing.
Mainframe systems with multiple processors are often tightly coupled.
Loosely coupled multiprocessor system Loosely coupled multiprocessor systems
are based on multiple standalone single or dual processor commodity computers
interconnected via a high speed communication system. A Linux Beowulf
cluster is an example of a loosely coupled system.
Tightly coupled systems perform better and are physically smaller than loosely
coupled systems, but have historically required greater initial investments and
may depreciate rapidly; nodes in a loosely coupled system are usually
inexpensive commodity computers and can be recycled as independent machines upon
retirement from the cluster. Power consumption is also a
consideration. Tightly coupled systems tend to be much more energy efficient
than clusters. This is because considerable economy can be realized by
designing components to work together from the beginning in tightly coupled
systems, whereas loosely coupled systems use components that were not necessarily
intended specifically for use in such systems.
Loosely coupled systems have the ability to run different operating systems or OS
versions on different systems. = Multiprocessor Communication
Architecture= Message passing
Separate address space for each processor.
processors communicate via message passing.
processors provide local message queue memories.
focus attention on costly non-local operations.
Shared memory Processors communicate with shared
address space Processors communicate by memory
read/write Easy on small-scale machines
Lower latency SMP or NUMA architecture
Flynn's taxonomy = SISD multiprocessing=
In a single-instruction stream, single-data stream computer one
processor sequentially processes instructions, each instruction processes
one data item. One example is the "von Neumann" architecture with RISC.
= SIMD multiprocessing= In a single-instruction stream, multiple
data stream computer one processor handles a stream of instructions, each
one of which can perform calculations in parallel on multiple data locations.
SIMD multiprocessing is well suited to parallel or vector processing, in which
a very large set of data can be divided into parts that are individually
subjected to identical but independent operations. A single instruction stream
directs the operation of multiple processing units to perform the same
manipulations simultaneously on potentially large amounts of data.
For certain types of computing applications, this type of architecture
can produce enormous increases in performance, in terms of the elapsed
time required to complete a given task. However, a drawback to this architecture
is that a large part of the system falls idle when programs or system tasks are
executed that cannot be divided into units that can be processed in parallel.
Additionally, programs must be carefully and specially written to take maximum
advantage of the architecture, and often special optimizing compilers designed to
produce code specifically for this environment must be used. Some compilers
in this category provide special constructs or extensions to allow
programmers to directly specify operations to be performed in parallel.
SIMD multiprocessing finds wide use in certain domains such as computer
simulation, but is of little use in general-purpose desktop and business
computing environments. = MISD multiprocessing=
MISD multiprocessing offers mainly the advantage of redundancy, since multiple
processing units perform the same tasks on the same data, reducing the chances
of incorrect results if one of the units fails. MISD architectures may involve
comparisons between processing units to detect failures. Apart from the
redundant and fail-safe character of this type of multiprocessing, it has few
advantages, and it is very expensive. It does not improve performance. It can be
implemented in a way that is transparent to software. It is used in array
processors and is implemented in fault tolerant machines.
Another example of MISD is pipelined image processing where every image pixel
is piped through several hardware units performing several steps of image
transformation. = MIMD multiprocessing=
MIMD multiprocessing architecture is suitable for a wide variety of tasks in
which completely independent and parallel execution of instructions
touching different sets of data can be put to productive use. For this reason,
and because it is easy to implement, MIMD predominates in multiprocessing.
Processing is divided into multiple threads, each with its own hardware
processor state, within a single software-defined process or within
multiple processes. Insofar as a system has multiple threads awaiting dispatch,
this architecture makes good use of hardware resources.
MIMD does raise issues of deadlock and resource contention, however, since
threads may collide in their access to resources in an unpredictable way that
is difficult to manage efficiently. MIMD requires special coding in the operating
system of a computer but does not require application changes unless the
programs themselves use multiple threads. Both system and user software
may need to use software constructs such as semaphores to prevent one thread from
interfering with another if they should happen to cross paths in referencing the
same data. This gating or locking process increases code complexity,
lowers performance, and greatly increases the amount of testing
required, although not usually enough to negate the advantages of
multiprocessing. Similar conflicts can arise at the
hardware level between processors, and must usually be resolved in hardware, or
with a combination of software and hardware.
See also Symmetric multiprocessing
Asymmetric multiprocessing Multi-core processor
BMDFM – Binary Modular Dataflow Machine, a SMP MIMD runtime environment
Software lockout OpenHMPP
References