today I want to talk about someone I
think has had a great effect on age
based modeling and
very effect on me since he was my
advisor which is John Holland and John
unfortunately passed away last summer
but he has really left a big impact down
the field
so let's go back to where it started
right so John was inspired by heads
ideas about neurons and Rosen let's work
on the perceptron and these ideas where
ideas about how you could think of the
brain as a computational entity and how
it could kind of compute right and so
Rosenblatt built the perceptron model
which was actually the foundational
model for neural networks right and HEVs
1949 organizational behavior can let
that groundwork for thinking about how
neurons might be used to structure a
computational brain right and based upon
those ideas Collins realize you know
these are interesting because they're
actually adaptive models right they're
showing how not only something can take
inputs and create output how they can
change over time to adapt to the system
around them and that's the real big
point I'm gonna there's a lot of great
things John Howard did but this is the
point I want to bring out today I want
to talk a lot about
adaptation and agents which is something
I think Collins really was a big
contributor in so basically pond has
thoughts along this line so you decided
to think about what about the
evolutionary adaptive process right and
so in 1975 you published the book
adaptation or chill and artificial
systems which essentially laid the
groundwork for the genetic algorithm
right the jack algorithm which is
something I'll actually show you an
example of in the next video lecture
essentially takes a population of
solutions to a problem
evaluates that and then creates a new
population using evolutionary principles
in much the ways same way evolution
evolves the population of a biological
system right
but Holland was you know as who's
working on this to doing some great work
that's really kind of founded the field
that Jake algorithms in many sense he
was also influenced by our sir Samuels
work checkers so our so say my let's
start doing some work on developing a
computer algorithm to play checkers and
in that algorithm he basically used the
notion of a notion of reinforcement
learning actually something that's still
used today in some cases of things like
where you have these deep neural nets
that are playing games against each
other in order to solve problems but how
I wanted to think about how could you
use
Howard Samuels work in combination with
the genetic algorithms to create systems
that could adapt to playing complex
games right to create adaptive agents so
while I was thinking about this in the
early
1980s 1985 mid 90s he became involved in
the Santa Fe Institute which is where we
are taking this course right now right
and in September of 1987 he presents
some of his ideas about adaptive agents
at a meeting at the Santa Fe Institute
of the economy is a complex adaptive
system and partially is response and
response that you started working with
people like Brian Arthur Blake LeBaron
Richard Palmer and Paul Taylor to create
the Santa Fe artificial stock market
right which was one of the first
attempts to kind of embody everything
that happens in a stock market within an
artificial simulation of that stock
market right and in August of
1986
partially as a response to some of the
some of the problems they were seeing
John Reed who was CEO The Seeker met
with at Santa Fe with Kent Aero Brian
Arthur and John Allen along with some
other s5 researchers to think about new
ways to model economics and in fact that
led to swarm right which was one of the
first Asia based modeling languages
right and there was really because the
all these great thinkers where I asked
if I realized that we needed to start
thinking about different ways to model
economic systems in order to move
forward following on the tails of that
in 1981 John Miller and John Holland
published the paper called artificial
adapt a should Adam debt at artificial
and adaptive agents and economic theory
which in many ways was one of the first
papers to use the word agent in the
current sense when you need it and a lot
of this work was actually inspired and
you know by that by the person who I'm
picturing here which is Herbert Simon
Herbert Simon's work really talked about
I know shifts that I called bounded
rationality and the idea was that a
lot of traditional neoclassical
economics is built upon this idea that
we have these agents are these entities
that can make infinite decisions about
the world around them right and in
Herman - you know that's not the case
that that so true the matter is people
have a small amount of time they can to
make a decision about something and that
timeframe is rather short and so they
really really need to think about
boundedly rational solutions rather than
these these overall perfect solutions
that might exist right and so
while this was happening
John was obviously wasn't at the Santa
Fe the whole time he was also spending
time back at Michigan which is where his
home department was and she helped found
a group of researchers at Michigan known
as the BOC group which originally
composed of Arthur Burks that we've
talked about previously Robert Axelrod
Michael Coleman John found and Robert
Axelrod you may really have heard of at
some point along the way really
developed the notion of the tit for tat
and the prisoner's the lemon studies
that really went on right and those are
many ways also agent-based models right
you have two agents playing a game of
cooperation or and affection against
each other and Michael Cohen goes on to
write papers on which some of his
colleagues on the garbage can model of
organizational choice which is in effect
a in nature based model and
my client had had a chance to chat about
that before he passed away and he said
that the time he would have used the age
of based modeling as a framework to
build it it's just that they didn't have
that term at the time right this room
also inspired two prominent graduate
students who continued and continued to
work in the space right Melanie Mitchell
who teaches actually another course here
at the SFI on being mature up sort of
complexity and Melanie you know built
upon some of the work that John and the
and the other group members were doing
to really think about added adaptive
agents in the cellular automata space it
she actually huge dick algorithms to
show that you can evolve cellular
automata solutions to really horror CA
problems right and then you could solve
these problems fairly well among many
other things that Melanie's done but in
this particular case thing with the
space are adaptive agents that's a big
aspect of it and it's definitely for us
use the notion of adaptation and agents
to start to think about artificial
immune systems so these are systems they
construct within a computer in order to
protect that computer against viruses
and other defects in the software and
she's also going on to think about it in
the case of building software that can
automatically repair itself to defects
in bugs that might be in those systems
now following the panelists work I think
really in the area in which Palin really
hit his stride in terms of adaptations
is that that is dividing to develop the
idea of the classifier system and in the
classifier system howland really kind of
realizes that what we you have is you
have this ability to chain together
rules right so you have one rule they
can look at the world you would create
signal and then you can have another
rule that activates because of that
signal not because of the world around
you you have another rule that can chain
upon that right and these they can take
multiple inputs so maybe the second rule
actually looks at multiple different
rules right and if you created this
ability to classify things and entities
on the set of a whole bunch of rules and
he developed what was called the the
bucket brigade algorithm to pass back
kind of the appropriate incentives for
each of those rules based upon when they
fire right working on this led to the
idea of a world where never-ending
novelty could evolve and he really
wanted to develop this idea if you could
have agents who are moving through a
world potentially using classifiers
systems to process and develop resources
in the world and he created a set of
Suites of mom it's not just one model
this mita models called the echo models
and these models were described in a
Stanislaw ulam lecture but another name
you have heard from a previous video
lecture after s fi which eventually
became a book called hidden order right
and so this kind of gives you an idea of
like where so how does this all have to
do well a lot of these ideas around
adaptive agents continue to persist in
the work we're doing the base bombing
today right so if you look back to the
original idea about evolving solutions
right we still see genetic out of those
as a big component right that these
systems are involved in there if you
look back to some of the earliest work
of economic modeling right the notions
of bounded rationality
and inductive reasoning or something
that you know was brought out really
from the attitude adaptive agent weren't
done in the eighties right if you look
at the back route than the work that was
done there right there all these ideas
about how to use agent-based models
adaptive agents and specific to solve
different policies here around this so
in many ways Holland's kind of work on
adaptive agents over the course of his
lifetime really is a legacy is with us
today in the HIV spiraling world