In computer science and mathematical optimization, a metaheuristic is a
higher-level procedure or heuristic designed to find, generate, or select a
heuristic that may provide a sufficiently good solution to an
optimization problem, especially with incomplete or imperfect information or
limited computation capacity. Metaheuristics sample a set of solutions
which is too large to be completely sampled. Metaheuristics may make few
assumptions about the optimization problem being solved, and so they may be
usable for a variety of problems. Compared to optimization algorithms and
iterative methods, metaheuristics do not guarantee that a globally optimal
solution can be found on some class of problems. Many metaheuristics implement
some form of stochastic optimization, so that the solution found is dependent on
the set of random variables generated. By searching over a large set of
feasible solutions, metaheuristics can often find good solutions with less
computational effort than optimization algorithms, iterative methods, or simple
heuristics. As such, they are useful approaches for optimization problems.
Several books and survey papers have been published on the subject.
Most literature on metaheuristics is experimental in nature, describing
empirical results based on computer experiments with the algorithms. But
some formal theoretical results are also available, often on convergence and the
possibility of finding the global optimum. Many metaheuristic methods have
been published with claims of novelty and practical efficacy. Unfortunately,
many of the publications have been of poor quality; flaws include vagueness,
lack of conceptual elaboration, poor experiments, and ignorance of previous
literature. The field also features high-quality research.
Properties These are properties that characterize
most metaheuristics: Metaheuristics are strategies that guide
the search process. The goal is to efficiently explore the
search space in order to find near–optimal solutions.
Techniques which constitute metaheuristic algorithms range from
simple local search procedures to complex learning processes.
Metaheuristic algorithms are approximate and usually non-deterministic.
Metaheuristics are not problem-specific. Classification
There are a wide variety of metaheuristics and a number of
properties along which to classify them. = Local search vs. Global search =
One approach is to characterize the type of search strategy. One type of search
strategy is an improvement on simple local search algorithms. A well known
local search algorithm is the hill climbing method which is used to find
local optimums. However, hill climbing does not guarantee finding global
optimum solutions. Many metaheuristic ideas were proposed
to improve local search heuristic in order to find better solutions. Such
metaheuristics include simulated annealing, tabu search, iterated local
search, variable neighborhood search, and GRASP. These metaheuristics can both
be classified as local search-based or global search metaheuristics.
Other global search metaheuristic that are not local search-based are usually
population-based metaheuristics. Such metaheuristics include ant colony
optimization, evolutionary computation, particle swarm optimization, and genetic
algorithms. = Single-solution vs. Population-based =
Another classification dimension is single solution vs population-based
searches. Single solution approaches focus on modifying and improving a
single candidate solution; single solution metaheuristics include
simulated annealing, iterated local search, variable neighborhood search,
and guided local search. Population-based approaches maintain and
improve multiple candidate solutions, often using population characteristics
to guide the search; population based metaheuristics include evolutionary
computation, genetic algorithms, and particle swarm optimization. Another
category of metaheuristics is Swarm intelligence which is a collective
behavior of decentralized, self-organized agents in a population or
swarm. Ant colony optimization, particle swarm optimization, social cognitive
optimization, and artificial bee colony algorithms are examples of this
category. = Hybridization and memetic algorithms =
A hybrid metaheuristic is one which combines a metaheuristic with other
optimization approaches, such as algorithms from mathematical
programming, constraint programming, and machine learning. Both components of a
hybrid metaheuristic may run concurrently and exchange information to
guide the search. On the other hand, Memetic algorithms
represent the synergy of evolutionary or any population-based approach with
separate individual learning or local improvement procedures for problem
search. An example of memetic algorithm is the use of a local search algorithm
instead of a basic mutation operator in evolutionary algorithms.
= Parallel metaheuristics = A parallel metaheuristic is one which
uses the techniques of parallel programming to run multiple
metaheuristic searches in parallel; these may range from simple distributed
schemes to concurrent search runs that interact to improve the overall
solution. = Nature-inspired metaheuristics =
A very active area of research is the design of nature-inspired
metaheuristics. Many recent metaheuristics, especially evolutionary
computation-based algorithms, are inspired by natural systems. Such
metaheuristics include Ant colony optimization, particle swarm
optimization, cuckoo search, and artificial bee colony to cite a few.
Many nature-inspired meta-heuristics mediate between local and global search
and are therefore instances of Dual-phase_evolution,
Applications Metaheuristics are used for
combinatorial optimization in which an optimal solution is sought over a
discrete search-space. An example problem is the travelling salesman
problem where the search-space of candidate solutions grows faster than
exponentially as the size of the problem increases, which makes an exhaustive
search for the optimal solution infeasible. Additionally,
multidimensional combinatorial problems, including most design problems in
engineering such as form-finding and behavior-finding, suffer from the curse
of dimensionality, which also makes them infeasible for exhaustive search or
analytical methods. Popular metaheuristics for combinatorial
problems include simulated annealing by Kirkpatrick et al., genetic algorithms
by Holland et al., scatter search and tabu search by Glover. Literature review
on metaheuristic optimization, suggested that it was Fred Glover who coined the
word metaheuristics Contributions
Many different metaheuristics are in existence and new variants are
continually being proposed. Some of the most significant contributions to the
field are: 1952: Robbins and Monro work on
stochastic optimization methods. 1954: Barricelli carry out the first
simulations of the evolution process and use them on general optimization
problems. 1963: Rastrigin proposes random search.
1965: Matyas proposes random optimization.
1965: Nelder and Mead propose a simplex heuristic, which was shown by Powell to
converge to non-stationary points on some problems.
1966: Fogel et al. propose evolutionary programming.
1970: Hastings proposes the Metropolis-Hastings algorithm.
1970: Cavicchio proposes adaptation of control parameters for an optimizer.
1970: Kernighan and Lin propose a graph partitioning method, related to
variable-depth search and prohibition-based search.
1975: Holland proposes the genetic algorithm.
1977: Glover proposes Scatter Search. 1978: Mercer and Sampson propose a
metaplan for tuning an optimizer's parameters by using another optimizer.
1980: Smith describes genetic programming.
1983: Kirkpatrick et al. propose simulated annealing.
1986: Glover proposes tabu search, first mention of the term metaheuristic.
1989: Moscato proposes memetic algorithms.
1992: Dorigo introduces Ant colony optimization in his Phd Thesis.
1995: Wolpert and Macready prove the no free lunch theorems.
See also Stochastic search
Meta-optimization Matheuristics
Hyper-heuristics Swarm intelligence
Genetic algorithms Simulated annealing
References External links
EU/ME forum for researchers in the field.
MetaHeur - Excel application to metaheuristic methods