The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare
continue to offer high quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: So what I'm going to do in this course
is discuss mostly ideas that are already in the book called
"The Emotion Machine."
I'm sorry.
I used that title, and the older book
called it "The Society of Mind."
The books are not quite the same.
They overlap a bit in material, but they're
sort of complementary.
I like the old one better, because the chapters are all
one page long.
And they're moderately independent.
So if you don't like one, you can skip it.
The new book is much denser, and it has a smaller number
of long chapters.
And I think it's--
over the years, I got lots of reactions
from young people in high school, for example.
Almost all of whom liked "The Society of Mind,"
and found it easy to read, and seem to understand it.
There are lots of criticisms by older people who
maybe some of them found it harder to put
so many fragments together.
Who knows?
But most of this class, most of the things I'd like to say
are in those books.
So it's really like a big seminar,
and my hope is that everyone who comes to this class
would have a couple of questions that they'd like to discuss.
And if I can't answer them, maybe some others of you can.
So I like to think of this as a super seminar,
and normally, I don't prepare lectures.
And I just start off asking if there are any questions.
And if there are not, I get really pissed off.
But anyway, I'm going to start with a series of slides.
So why do we need machines?
And partly, there are a lot of problems.
Unlike most species or kinds of animals,
humans have only been around a few million years.
And they're very clever compared to other animals,
but it's not clear how long they will last.
And when we go, we might take all the others with us.
So there are a whole set of serious problems
that are arising, because there are so many humans.
And here's just a little list of things.
There's a better list in a book by the Astronomer Royal Martin
Rees of England.
Anybody know the title?
AUDIENCE: "Our Final Hour."
PROFESSOR: Yes, "Our Final Hour."
It's a slightly scary title.
And when I was a teenager, World War II came to an end
with the dropping of two atomic--
oh, this is getting terrible.
--two atomic bombs on Japan.
And I didn't believe the first one was real,
because it was in Hiroshima.
So I assumed that the US had somehow
made a big underwater tanker with 20,000 tons of TNT,
and some few grams of radium or something,
and blown it up in the harbor.
And first, it flew an airplane over,
dropping some little thing.
And this was to fool the Japanese into thinking
that we have an atomic bomb.
But when they did it, again, over Nagasaki,
that wasn't feasible.
And when I was in grade school,
Sometimes, if I said something very bright,
I would hear a teacher saying, maybe
he's another J. Robert Oppenheimer.
Because that was the name of a scientist who
had been head of the Manhattan Project, and he was,
I think, three or four years earlier in grade school
than I was.
And I thought it was very strange for a person
to have a first name just being a letter rather than a name.
Many years later when I was at Princeton in graduate school,
I met the Robert Oppenheimer, and that was a great pleasure.
And in fact, he took me to lunch with a couple
of other people I admired, namely, Girdle and Einstein,
which was very exciting.
Except I couldn't understand Einstein,
because I wasn't used to people with a strong German accent.
But I understood Girdle just fine.
And after that lunch was over, I went and spent about a year
learning about Turing machines, and trying to prove theorems
about them, and so forth.
So anyway, in the course of these talks,
we'll run across a few of these people.
And here's a big list of the people
that I'm mostly indebted to for the ideas
in the society of mind and the emotion machine.
The ones in blue are people I've actually met.
It would be nice to have met Aristotle,
because no one really knows much about him.
But you really should read, just skim through some of that,
and you'll find that this is a really smart guy.
We don't know if he wrote this stuff,
or if it were compiled by his students,
like a lot of Feynman's writing is and Von Neumann's writing is
edited from notes by their students.
Anyway, the astonishing thing about Aristotle
is that he seems to be slightly more
imaginative than most cognitive scientists you'll
run into in the present day.
It would have been nice to know Spinoza,
and Kant, and the others also.
Freud wrote 30 or 40 books.
Did he fall off this list?
There he is.
I just made this list the other day,
and I was looking up these people
to find their birthdays and stuff.
Yes?
AUDIENCE: Why are there no Eastern philosophers
in history?
PROFESSOR: Because they're religious as far as I can see.
AUDIENCE: Religious?
PROFESSOR: Well, who would you-- would you say, Buddha?
AUDIENCE: No, I mean, just Eastern thinkers thought that.
PROFESSOR: Name one.
Maybe I've never heard of them.
AUDIENCE: Confucius.
PROFESSOR: Who?
AUDIENCE: Confucius.
PROFESSOR: Confucius?
AUDIENCE: Or [INAUDIBLE] great thinkers from China.
PROFESSOR: Well, I only know of them
through aphorisms, single proverbs,
but I don't know that Confucius had a theory of thinking.
You think he did?
AUDIENCE: There are a lot differences of thoughts,
and I think they probably do have [INAUDIBLE]..
PROFESSOR: Well, I've looked at Buddhist theories,
and they're--
I don't think they would get a C plus.
And one problem is that there are cultures-- there's
something about Greek culture.
Because it had science.
It had experiments.
Somebody has a theory, and they say, like Epimenides Lucretius.
Somewhere in the society mind, I think
I quoted Lucretius about translucent objects.
And he says, they have the particular appearance,
because the rays of light bounce many times
before they get to the surface.
So you can't tell where they started.
And I don't find in eastern philosophy theories
that say, here's what, I think, and here's a reason why.
I've looked at Buddhist stuff, and it's strange lists
of psychological principles.
Every one of which looks pretty wrong,
and they make nice two dimensional diagrams.
But no evidence for any of them, so I
don't know whether to take it seriously.
AUDIENCE: I think knowledge is from observation.
I think you're right that in some them probably
didn't really test it, because a lot of the ideology cannot be
tested.
On the other hand, there are scientists--
PROFESSOR: But what can't be tested?
AUDIENCE: I mean, some of the ideologies probably.
PROFESSOR: If they can't be tested,
why should one look at it twice?
AUDIENCE: Test it in terms of something logically.
I don't know.
Like culture, can you test culture?
PROFESSOR: OK, I think this is a serious argument.
It seems to me that science began a little bit in China,
a little bit in India.
In the Arabic world, they got up to the middle of high school
algebra, but then--
AUDIENCE: That's the foundation.
PROFESSOR: What?
AUDIENCE: That's the foundation.
PROFESSOR: Well, but it wasn't as
good as Archimedes, who got to the beginning of calculus.
So if you look at most cultures, they never
got to the critical point of getting theories, doing
experiments, discussing them, and then throwing them out.
And so if you look at Buddhist philosophy,
it's 2,500 years old.
If you look at Greek physics, yes, Archimedes almost
got calculus, and he got lots of nice principles.
And Buddha mentions, at some point,
if you want to weigh an elephant, put him in a boat.
And then take the elephant out and put rocks in,
until the boat sinks to the same level.
So there, you see a good idea.
But if you look at the history of the culture,
if people still say, this thousand year old stuff
is good, then you should say, no, it's not.
AUDIENCE: By the way, same story about the elephants.
There's like a story in Chinese history that has the same.
PROFESSOR: Sure.
AUDIENCE: I mean, maybe there's no one person that [INAUDIBLE]..
PROFESSOR: No, but the question is, why did it stop?
Why did it stop?
Ancient wisdom is generally not very good,
and we shouldn't respect it for too long.
And that's--
AUDIENCE: [INAUDIBLE] past where everybody's
standing on the giant's shoulders, right?
PROFESSOR: No, we got rid of alchemy.
We got rid of--
what do you call it?
What's caloric?
You jump off their shoulder.
You don't stay on them, so it's good to know history.
But if the history doesn't get anywhere,
then you don't want to admire it too much.
Because you have to ask, why did it stop?
What went wrong?
And usually, it went wrong, because barbarians
came in and--
well, you know what happened to Archimedes.
Some Roman killed him.
Anyways--
AUDIENCE: [INAUDIBLE].
I'm sorry.
PROFESSOR: No, it's a good question.
Why didn't science happen a million years ago?
Because humans are five million years old, so
what took it so long?
AUDIENCE: [INAUDIBLE]
PROFESSOR: No, it's more--
AUDIENCE: [INAUDIBLE]
PROFESSOR: Sure, OK.
Do you have a theory of why science
didn't develop for so long?
In most cultures, it might be religion,
which is a sort of science that doesn't use evidence,
and in fact, kills people who try to get it.
So there are systematic reasons why
most cultures failed, and maybe somebody has written it.
Is there a book on why science disappeared, except once?
It's rather remarkable.
Isn't it?
After all, the idea, if somebody says something,
and somebody else says, OK, let's
do an experiment to see if that's right,
you don't have to be very bright.
So how come it didn't happen all the time everywhere?
Here he is.
AUDIENCE: I don't know the answer to that,
but I know Paul Davies has sort of an anecdote about that.
Or he's exactly speculating, even
in Europe when it did happen was a fluke.
And he gives the example of suppose an asteroid or a comet
crashed in Paris in--
I forget what year he gives.
--1150, or 1200, or something.
Then what?
Whether it's science [INAUDIBLE] as a thought problem.
PROFESSOR: History is full of flukes.
I'm trying to remember who wrote that nice book
about the plague, some woman.
And she mentions that this was spread
by rats and fleas or something.
And 30% or 40% of the population of many countries in Europe
died, and the next generation had a lot of furniture.
The standard of living went way up,
so anyway, here's a list of disasters.
Oh, come on.
And Martin Rees is the royal astronomer
and has that book about the last hour or whatever.
I'm making another longer list.
But he has lots of obvious disasters,
like some high school student looks up
the genetic sequence for smallpox virus
has been published, and now, you can write a list of nucleotides
and send it somewhere.
And they'll make it for about $0.50 or $1 per nucleotide.
So for a couple of hundred dollars,
you can make a virus or a few hundred.
So one possibility is that some high school student
makes some smallpox only gets it wrong, and it kills everyone.
So there are lots of disasters like that,
and no one knows what to do about that.
Because the DNA synthesis machinery
is becoming less and less expensive, and probably
the average rich private high school could afford one.
So there are lots of other things that could happen.
But one particular one is this graph, which I just made up.
An interesting fact is that since 1950
when the first antibiotics started to appear,
as I mentioned, I was a kid in the 1940s.
And penicillin had just hit the stands,
and there wasn't much of it.
And there was a researcher who lived a few blocks from us
whose dog had cancer.
So its father--
I don't know what you call the owner of a dog.
--sneaked some penicillin out of the lab
and gave it to the dog, who died anyway.
But he said, well, nobody's tried penicillin on cancer yet.
Maybe it will work.
And a lot of people were mad at him,
because he probably cost some human its life.
But he said, he might have saved a billion humans
their lives, so ethics.
Ethicists are people who give reasons not to do things,
and I'm not saying they're wrong.
But it's a funny job.
Anyway, since that sort of thing happened
and medicine began to advance, people
have been living one year longer every 12.
So it's 60 years since 1950, so that's five of those six.
So they're living six or seven years
longer now than they were when I was born.
And somebody mentioned that curve
stopped the last few years for other reasons,
but anyway, if you extrapolated, you'll
find that the lifespan is going to keep increasing.
How much we don't know--
another problem is that you might discover
enough about genetics to get rid of most
of the serious diseases.
Maybe just 20 or 30 genes are responsible for most deaths
right now.
And if you could fix those, which we can't do yet,
there's no way to change a gene in a person.
Because invading all the cells is a pretty massive
intervention, but we'll get around that.
And then it might be that people suddenly
start living 200 or 300 years.
Now at some point, the population has to slow down.
So you can only reach equilibrium
with one child per family and probably less than that.
So all the work has to be done by 200 or 300-year-olds,
and let's hope they're good and healthy.
So anyway, I think it's very important
that we get smart robots, because we're going
to have to stem the population.
And I hope people will live longer and blah, blah, blah.
So these robots have to be smart enough to replace most people,
and how do you make something smart?
Well, artificial intelligence is the field whose goal with
has been to make machines that do things
that we regard as smart, or intelligent,
or whatever you want to call it.
And the idea of seriously making machines smart
has roots that go back to a few pioneers,
like Leibniz, who wrote about automata
and that sort of thing.
But the idea of a general purpose computer
didn't appear till the 1930s and '40s in some sense.
The first form of the general purpose computer
appears really in the 1920s and '30
with the work of a mathematician,
Emil Post at NYU, who I happen to never meet.
But we had some friends in common,
and he had the idea of production rules.
And basically, rule based systems
and prove various theorems about them.
Then Kurt Girdle showed that, if you had something,
like a computer or a procedure that
had the right kinds of rules, it could
compute all sorts of things.
But there were some things it couldn't compute,
unsolvable problems, and that became an exciting branch
of mathematics.
And the star thinker in that field
was Alan Turing, who invented a very simple kind
of universal general purpose computer.
Instead of a random access memory,
it just had a tape, which it could write on, and read,
and change symbols.
And it would go back and forth.
And if it's in state x at, say, symbol y,
it will print symbol z over the x and move
to the left or right and just a bunch
of rules like that, where it was enough to make
a universal computer.
So from about 1936, it was sort of
clear to a large mathematical community
that these were great things.
And a couple of general purpose light computers,
very simple ones, were built in the 1930s and more
in the 1940s.
And in the 1950s, big companies started
to make big computers, which were rooms full of equipment.
But as you know, most programs could only
do some particular thing, and none of them were very smart.
Whereas a human can handle lots of kinds of situations.
And if you have one that you've never seen before,
there's a good chance you'll think of a new way
to deal with that and so forth.
So how do you make a machine that doesn't get stuck almost
all the time?
And I like to use the word resourcefulness.
Although, I left an R out of that one.
Is there a shorter word?
So here's a good example.
My favorite example of a situation
where a person is born, more or less,
with a dozen different ways of dealing with something.
And the problem that I imagine that you're dealing with
is this.
My favorite example is I'm thirsty,
so I see that glass of water.
And I do that and get it.
Actually, I am.
On the other hand, if I were here,
I would never in a whole lifetime do this.
You never walk out a window by mistake.
We're incredibly reliable, so how do I know how far it is?
And that slide shows you 12 different ways that your vision
system--
that's only your vision system.
--has to measure distances.
So gradients, if things are sort of blurry,
then they must be pretty far away.
That's sort of on a foggy day outside.
Here's a situation.
If you assume those are both chairs of the same size,
and you know that this chair is about twice as far away
as that, although, you don't--
well, and how far away they are pretty
much by the absolute size.
If you have two eyes that work well,
then, if something is less than 30 feet away,
you can make a pretty good estimate of its distance
by focusing both eyes on some feature.
And your brain can tell how far apart your eyes are looking,
so there's 12 different things.
It's more than you need.
Lots of people are missing half of those.
Lots of people have very poor vision in one eye.
Some people cannot fuse stereo images,
even though both eyes have 20/20 vision.
And in some cases, nobody knows why they can't do that.
I think I once took a test for being a pilot,
and they wanted to be sure you could do stereo vision, which
seem very strange.
Because if you're an airplane, and you're
less than 30 or 40 feet away from something, it's too--
you could use stereo.
But it's too late.
Anyway, that's interesting.
See if you can think of an example, where a person has
even more 12 of these.
But it's pretty amazing, isn't it?
It's more redundancy.
This is too hard to read, but somehow, I
found it in Aristotle essay the idea that you should represent
things in multiple ways.
You might describe a house.
One person might describe a house
as a shelter against destruction by wind, rain, and heat.
Another might describe it as a construction of stones, bricks,
and timbers.
But a third possible description would say,
it was in that form in that material with that purpose.
So you see there's two different descriptions.
One is the functional description.
It's a shelter.
The second one is a structural description, how it's made.
And Aristotle says, which is the better description?
And he dismisses the material one or the functional one
is not rather the person who combines
both in a single statement.
And then I found a paragraph by Fineman, who says,
every theoretical physicist who is any good
knows six or seven different ways
to represent exactly the same physics.
And you know that they're all equivalent,
but you keep them all in your head hoping
that they will give you different ideas for guessing
I should put more dots.
Anyway, that whole argument is to say
that the interesting thing about people
is that they have so many ways to do things,
and perceive things, and think of things.
And in some cases, we even know that there
are different parts of the brain that
are involved in one aspect or another
of constructing those different representations
or descriptions.
If you look at one of my favorite books,
it weighs about 20 pounds.
It's the book on the nervous system by Kandel and Schwartz.
And the index to that book is quite a lot of pages long,
and it mentions 400 different structures in the brain.
So the brain is not like the--
well, I shouldn't make fun of the liver.
Because for all I know, the liver
has 400 different many processes for doing things.
But the brain has distinguishable areas
that seem to perform several hundred different functions.
And with a microscope, at first, they all
look pretty much the same.
But if you look closely, you see slightly different patterns
of how the most layers of the cortex of the brain, most
parts of it have six layers, and each has a population
of different kinds of cells.
There are a lot of cross connections up and down
and sideways to other.
They're arranged in columns of between 400
and the 1,000 cells, and you have a couple of million
of those.
And there are lots of differences
between the columns in different areas,
and we know some of the functions.
In most cases, we don't know much about how any of them
actually work with the main exception of vision,
where the functions of the cells in the visual cortex
are fairly well understood at low levels.
So we know how that part of the brain
finds the edges and boundaries of different areas,
and textures, and regions of the visual field.
But we do not know even a little bit
about how the brain recognizes something
as a chair, and an overhead projector, and a CRT screen,
and that sort of thing.
The kind of question that I got interested in was, how can you
have a system, which has a very large number of different kinds
of computers?
Each of which by itself might be relatively simple or might not,
I suppose.
And how could you put them together
into a larger system, which could do things, like learn
language, and prove theorems, and convince
people to do things that they would never
have dreamed of doing five minutes earlier,
and stuff like that?
Now the first sort of things I was interested
in was, in fact, how to simulate simple kinds of nerve cells.
Because in the 1950s, there was about almost 100 years,
really more like 50 years of science discovering things
about neurons and nerve cells, the axons, and dendrites
that they use to communicate with other neurons.
So if you go back to 1890, you find a few anatomists
discovering some of the functions or connections
of neurons in the brain.
And you find a few experimental physicists.
There was no oscilloscope yet, but there were very high gain
galvanometers, which could detect pulses
going along a nerve fiber.
And by 1900, it was pretty clear that part
of the activity in a nerve cell was chemical and part
was electrical.
And by 1920 or '30 with the cathode ray tube
appearing mostly because of television,
but it became possible to do a lot of neurophysiology
by sticking needles in brains.
The vacuum tube appears around 1900,
and you can make amplifiers that can see millivolts
and then microvolts.
So in the beginning of the 20th century,
there was lots of progress.
By 1950, we knew a lot about the nervous system,
but we still don't know much about how you
learn something in the brain.
It's quite clear that the things called synapses are involved.
The connections between two neurons
become better at conducting nerve impulses
under some conditions, but no one
knows how higher level knowledge is represented in the brain
yet.
And the Society of Mind book had a lot of theories about that.
And in particular, there was a theory
called k line's, knowledge lines, or something
that came partly from me and partly
from a couple of other researchers named
David Waltz and Jordan Pollock.
That's a sort of nice theory of how neural networks might
remember higher level concepts.
And for some reason, although that kind of work
is from around 1980, which is 30 years ago,
it has not hit the neuroscience community.
So if you look at the emotion machine book or the society
minded in Amazon, you might run across a review
by a neurologist named Richard Restak, who
says that Minsky makes up a lot of concepts, like K-lines,
and micronemes, and stuff like that, that nobody's ever
heard of.
And there's no evidence for them,
and he ignores the possibility that it
isn't the nerve cells in the brain that are important.
But the supporting tissues called
glia, which hold the neurons up and feed them.
And he goes on for a couple of insane paragraphs.
It's very interesting, because it doesn't
occur to him that you can't look for something,
until you have the idea of it.
So here is this 30-year-old idea of K-lines, and go and ask
your favorite neurologist, neuroscientist what it is.
And he said, oh, I think that's some AI thing,
but where's the evidence for it?
What do you suppose is my reaction to that?
Who is supposed to get the evidence?
So it seems to me that there's a strange field in neuroscience,
which is that it doesn't want new ideas,
unless you've proved them.
So I try to have conversations with them,
but get somewhat tired of it.
Anyway, in this course, I'm taking the opposite approach,
which is that we don't want a theory of thinking.
We want a lot of them, because probably, psychology
is not like physics.
What's the most wonderful thing about physics?
The most wonderful thing is that they have unified theories.
There wasn't much of a unified theory, until Newton,
and he got these three wonderful laws.
One was the gravitational idea that bodies attract each other
with a force that's the inverse square
of the distance between them.
Another is that kinetic energy is conserved.
I forget with the third one is.
Oh, equal reaction is equal and opposite.
If two things collide, they transfer equal amount
of momentum to both.
There was a little problem up to Newton's time.
Galileo got some of those ideas, and my impression
from reading him is that he has a dim idea that there
are two things around.
There's kinetic energy, which is MV--
oops, momentum is MV.
And there's kinetic energy, which is MV squared,
and he doesn't have the clear idea that there
are two different things here.
And you can't blame him.
I would think-- you wouldn't think that two quantities would
combine in two different ways to make two
important different concepts.
Well, that got clear to Newton somehow,
and Galileo is a bit muddled.
Although, he gets almost all the consequences of those things
right, but he doesn't get the orbits and things to come out.
Anyway, what's happened in artificial intelligence,
like most fields, is that people said, well,
let's try to understand thinking and psychology.
And let's use physics as our model, so what we want
is to get a very small number of universal laws.
And a lot of psychologists struggled around to do that,
and then they gradually separated.
So that there were some psychologists,
like Bill Estes, who worked out some very nice
mathematical rules for reinforcement based learning,
got a simple rule.
If you designed an experiment right,
it predicted pretty well how many trials
it would take a rat, or a pigeon, or a dog,
or whatever to learn a certain thing from trial and error.
And Este's got a set of four or five rules, which
looked like Newton's laws.
And if you designed your experiment very carefully
and shielded the animal from noise and everything else,
which is what a physicist would do for a physics experiment,
the reinforcement theories got some pretty good models
of how to make a machine learn.
But they weren't good enough.
So here's a whole list of things that
happened in the early years of cognitive psychology
when people were trying to make theories of thinking,
and they were imitating the physicists.
By physics envy to borrow a term of Freud,
the idea is, can you find a few simple rules
that will apply to very broad classes
of psychological phenomena?
And this led to various kinds of projects.
Lots of neural network, and reinforcement,
and statistical based methods led
to learning machines that were pretty good at learning
in some kinds of situations.
And they're becoming very popular, but I don't like them.
Because, if you have a lot of variables, like 50 or 100,
then to use a probabilistic analysis,
you have to think of all combinations
of those variables.
Because if two of them are combined in something,
like a exclusive or a manner, you know,
I just put the light pen in a pocket.
It's either in the left pocket or a right pocket.
It can't be both.
That's an x or.
That will cause a lot of trouble to a learning machine.
And if there are a hundreds variables,
there's no way you could decide which
of the two to the 100th Boolean combinations of those variables
you should think about.
So lots of statistical learning systems
are good for lots of applications.
But they just won't cut it to solve hard problems, where
the hypothesis is a little bit complicated
and has seven or eight variables with complicated interactions.
Most statistical learning people assume
that, if you get a lot of partial ones,
then you can look for combinations of ones
that have high correlations with the result. Then
you can start combining them, and things
get better and better.
However, mathematically, if an effect you're looking for
depends on the exclusive or of several variables,
there's no way to approach that by successive approximations.
If any one of the variables is missing,
there won't be any correlation of the phenomenon
with the others.
Anyway, that's a long story, but I think
it's worth complaining about.
Because almost all young people who
start working on artificial intelligence
look around and say, what's popular?
Statistical learning, so I'll do that.
That's exactly the way to kill yourself scientifically.
You don't want to get the most popular thing.
You want to say, what are my really good at that's
different?
And what are the chances that would provide another thing?
End of long speech.
Another problem in the last 30 years--
and as you'll see during my lectures,
I think a lot of wonderful things
happened between 1950 when the idea of AI
first got articulated in the 1950s.
And then the 20 years after that from 1960 to 1980,
a lot of early experiments-- and I'll show you some of them.
--looked very promising.
In fact, they may be-- here we go.
1961, Jim Slagle was a young graduate student here at MIT.
He was blind.
He had gotten some retinal degeneration thing
in his first or second year of high school.
He was told that he would lose all his vision,
and there was no treatment or hope.
So he learned Braille while he could still see.
And when he got to MIT, he was completely blind,
but there was a nice big parking lot in technology square.
And he would ride a bicycle.
And people, like Sussman, and Winston, and whoever
was around, would yell at him, telling him where
the next obstacle would be.
Jim got better and better at that,
and nothing would stop him.
And he decided he would write a program that-- oh, I
wrote a program that would take any formula
and find its derivative.
It was really easy, because there
were just about five rules.
Like if there's a product UV, then
you compute U times the derivative of V plus V
times U of DV plus DVU.
So I wrote a 20 line list program
that did all the algebraic expressions,
and what it would do is put Ds in the right place.
And then it would go back through the expression again.
Wherever it saw a D, it would do the derivative of the thing
after that and nothing to it.
So Slagle said, well, I'll do integrals.
And we all said, well, that's very hard.
Nobody knows how to do it.
And in fact, in Providence at the home of the American
Mathematical Society, there is a big library
called the Bateman Manuscript Project,
which has been collecting all known integrals for 100 years.
And when anybody finds a new integral
that they can integrate in closed form,
they send the formulas to the Bateman Manuscript Project,
and some hackers there develop ways to index it.
So if you had an integral, and you
didn't know how to integrate it, you could look it up.
And that was pretty big.
I should say that Slagle succeeded
in writing a program that managed
to do all of the kinds of integrals
that one usually found on the first year calculus course
at MIT and got an A in those.
He couldn't do word problems.
And the uncanny thing is that, if it
was a problem that usually took a MIT student five
or 10 minutes, Slagle's program would take five or 10 minutes.
It's running on an IBM 701 with a 20 millisecond cycle time.
It's incredibly slow.
You can type almost that fast and 16 K of words of memory.
So there's no significance whatever
to this accident of time.
It would now take a microsecond or so.
It would be 1,000 million times faster than a student.
Quite remarkable.
I don't have a slide.
Joel Moses, then Slagle went and graduated.
Joel Moses was another student, who is--
is he provost now or what?
He got tired of it.
A terrific student, and he set up
a project called maxima for project Max Symbolic Algebra
and got several people all over the country working
on integration.
And at some point, a couple of them, Bobby Caviness and--
forget the other one--
found a procedure that could, in fact, integrate everything--
every algebraic expression that has a--
can be integrated in closed form.
I forget the couple of constraints on it.
And that became a widely used system.
It ultimately got replaced by Steven Wolfram's Mathematica.
But Maxima was sort of the world-class symbolic
mathematician for quite a few years.
And Moses mentioned to me he had read Slagle's program thesis.
And it took him a couple of weeks to understand
the two pages of--
or three pages of--
Lisp that Slagle had written.
Because being blind, Slagle had tried
to get the thing into as compact a form as possible.
But that's symbolic.
It's too easy.
It was in a more ambitious one, which was, three years later,
Dan Bobrow, who is now a vice president doing something
at Xerox--
and it solved problems like this.
The gas consumption of my car is 15 miles per gallon.
The distance between Boston and New York is 250 miles.
What is the number of gallons used
on a trip between Boston and New York?
And it chomps away and solves that.
It has about 100 rules.
It doesn't really know what any of those words mean.
But it thinks that the word "is" is equals.
The distance between-- doesn't care
what Boston and New York is.
It has a format thing which says the distance between two
things.
And it never bothers to--
you see, because the phrase "Boston and New York"
occurs twice in the example, it just
replaces that by some symbol.
It was fairly remarkable.
And generally, if you had an algebra problem,
and you told it to Bobrow, Bobrow could type something in,
and it would solve it.
If you typed it in, it probably wouldn't.
But it was-- it had more than half a chance, or less--
about half a chance.
So it was pretty good.
And if you look at an out-of-print book I wrote
called--
I compiled, called--
Semantic Information Processing, most of Bobrow's program
is in that.
So that's 1964.
I'll skip Winograd, which is, perhaps,
the most interesting program.
This was a program where you could talk to a robot that--
I don't have a good picture in this slide.
But they're a bunch of blocks of different colors.
They're all cubes in the--
or rectangular blocks.
And you can say, which is the largest block
on top of the big blue block?
And it would answer you.
And you could say, put the large red block
on top of the small green block, and it would do that.
And Winograd's program was, of course, a symbolic one.
We actually built a robot.
And I guess we built it second.
Our friends at Stanford built a robot.
And they imported Winograd's program.
And they had the robot actually performing these operations
that you told it to do by typing.
And it was pretty exciting.
My favorite program in that period
was this one, because it's so psychological.
This is called a geometrical analogy test.
And it's on some IQ tests.
A is to B as C is to which of the following five?
And Evans wrote a set of rules which were pretty good at this.
Did as well as 16-year-olds.
And it picks this one.
And if you ask it why, it says something like,
I don't have a reason.
It moves the largest object down or something like that,
makes up different reasons.
So you see, in some sense, we're going backwards in age.
Because we're going from calculus, to algebra,
to simple analogies.
Oh, there it is.
That's one where the largest object moves down.
I don't know why I have two of them.
These are for another lecture.
OK.
So that was a period in which we picked problems
that people considered hard, because they were mathematical.
But when you think about it more,
you see, well, those math things are just procedures.
And it's once you know what Laplace, and Gauss,
and those mathematicians--
Newton and people-- did, you can write down
systematic procedures for integrating,
or for solving simultaneous algebraic constraint
equations, or things like that.
And so there's very little to it.
So in some sense, if you look at the--
what you're doing in math in high school,
in education, you're going from hard to easy.
It's just that people aren't--
most people aren't very good at obeying really simple rules,
because it's so hideously boring or something.
So we gradually started to ask, well,
why can't we make machines understand
everyday things, and the things that everyone regards
as common sense, and people can do so you
don't need machines to do them?
And one of my favorite examples is, why can you pull something
with a string but not push?
And there's been a lot of publicity recently about
that interesting program written at a group at IBM
called Watson, which is good at finding facts
about sportspeople, and celebrities, and politics,
and so forth.
But there's no way it could understand why you could push--
pull something with a string but not push.
And I don't know of any program that has that concept
or way of dealing with it.
So that's what I got interested in.
And starting around the--
maybe the middle 1970s or late 1970s, several of us
started to stop doing the easy stuff
and try to make theories of how you
would do the kinds of things that people
are uniquely good at.
I don't know if animals--
well, I don't know.
I'm sure a monkey wouldn't try to push anything with a string.
Maybe it does it very quickly, and you don't notice.
And one aspect of commonsense thinking
is going right back to that idea of vision having
a dozen different systems, is that, whatever
a person normally is doing, they are probably representing it
in several different ways.
And here's an actual scene of two kids named Julie and Henry
who are playing with blocks.
It's pretty hard to see those blocks.
And you can think that Julie is thinking seven thoughts.
I'd like to see a longer list.
Maybe a good essay would be to take a few examples
and say, what are the most common micro-worlds?
See physical, social, emotional, mental, instrumental--
whatever that is--
visual, tactile, spatial.
She's thinking all these things.
What if I pulled out that bottom block?
You can't see the tower very well.
Should I help him or knock is tower down?
How would he react?
I forgot where I left the arch-shaped block.
That was real.
It's somewhere over here.
But I don't think we could--
maybe it's that.
I don't know.
I remember, when it happened, she
mentioned that she reached around,
and it wasn't where she thought it was.
So commonsense thinking involves this--
in most cases, I think, several representations.
I don't know if it's as many as seven, or maybe 20, or what.
But that's the kind of thing we want to know how to do.
OK, I think I'll stop, and we'll discuss things.
But in the next lecture, I'll talk about a model
of how I think thinking works.
What's the difference between us and our ancestors?
We know we have a larger brain.
But if you think about it, if you took the brain that
you already had and say--
trying to remember the name of the little monkey that
looks like a squirrel, jumps around in trees.
Anybody know what--
AUDIENCE: Spider monkey?
PROFESSOR: What?
AUDIENCE: Spider monkey?
PROFESSOR: Maybe.
It's a squirrel-like thing.
You wouldn't know it was a monkey
till you took a close look.
AUDIENCE: Lemur, maybe?
PROFESSOR: Maybe.
Lemur?
I don't-- I forget.
I'll have to-- anyway, if you just made the brain bigger,
then the poor animal would be slower, and heavier,
and would need more food, and take longer to reproduce.
The joke about difficulty to give birth--
I don't know if any animal has the problem that humans have.
A lot of people die and so on.
So how did we evolve new ways to think and so forth?
And my first book, The Society of Mind,
had this theory that maybe we evolved
in a series of higher and higher levels,
or management structures, built on the earlier ones.
And this particular picture suggests
that I got this idea from Sigmund Freud's early theories.
There's been a lot of Freud bashing recently.
You can look on the web.
I forget the authors.
But there are a couple of books saying that he made up
all his data, and there's no evidence
that he ever cured anyone, and that he
lied about all the data mentioned in his 30 or 40
books, and so forth.
AUDIENCE: Also [INAUDIBLE].
PROFESSOR: Yes, right.
But the funny part is that if you
look at his first major book--
1895-- called The Interpretation of Dreams,
it sort of outlines his theory that most of thinking
is unconscious, and it's processes
you can't get access to.
And it has a little bit about sex,
but that's not a major feature.
And it's just full of great ideas
that the cognitive psychologists finally
began to get in the 1960s again and never give credit to Freud.
So he may well have made up his data.
But if you have a very good theory
and nobody will listen to you, what can you do?
His friend Rudolf Fliess listened to him.
And there was another paper on how
the neurons might be involved in thinking, which was also
written around 1895, but never got published till 1950 by--
forget who-- called "Project for a scientific psychology."
And it's full of ideas that, if they had been published,
might have changed everything.
Because-- anyway, what's on your mind?
Who has-- what would you like to hear about?
Or who has another theory?
AUDIENCE: I've got a question.
PROFESSOR: Great.
AUDIENCE: So earlier, you talked a little bit
about how we don't really see the neuroscience, all
of these things like K-lines, et cetera.
Do you think it's because they're just
really hard to find, or no-one's actually looking for them?
PROFESSOR: Well, Restak's review says,
he uses vague, ill-defined terms like K-line, and microneme,
and a couple of others, and frame, and so forth.
They're very well-defined.
They're defined better--
I mean, when he talks about neurotransmitters,
it's as though he thinks that chemical
has some real significance.
Any chemical would have the same function
as any other one provided there's
another receptor that causes something
to happen in the cell membrane.
So you don't want to regard acetylcholine or epinephrine
as having a mental significance.
It's just a-- it's just another pulse, but very low-resolution.
And yes, a neurochemical might affect all the neurons
a little bit, and raise the average amount of activity
of some big population of cells, and reduce the average activity
of some others.
But that's nothing like thinking.
That's like saying, in order to understand how a car works--
what's the most insulting thing I could say?
Or to understand how a computer works,
you have to understand the arsenic, and phosphorus,
and/or--
what's the other one?
You have to understand these atoms that are--
what?
AUDIENCE: Germanium.
PROFESSOR: Yeah, well, that's the matrix.
So there are these one part in a million impurities.
And that's what's important about a computer,
isn't it, the fact that the transistor has gain
and so forth.
Well, no, the trouble with the computer is the transistors.
That's why practically every transistor in a computer
is mated to another one in opposite phase
to form a flip-flop whose properties are
exactly the same, except one in a quadrillion times.
In other words, everything chemical about a computer
is irrelevant.
And I suspect that almost everything
chemical about the brain is unimportant except that it
causes--
it helps to make the columns in the cortex, which
are complicated arrangements of several hundred cells,
work reliably.
Whereas the neuroscientist is looking
for the secret in the sodium.
When a neuron fires, the important thing is that that
lets the sodium in and the potassium out or vise-versa--
I forget which-- at 500 millivolts--
really quite a colossal event.
But it has no significant.
It's only when it's attached to a flip-flop, or to something
like a K-line, which has an encoder
and decoder of a digital sort every few microns of its length
that you get something functional.
So the trouble is, the poor neuroscientists started out
with too much knowledge about the wrong thing.
The chemistry of the neuron firing
is very interesting, and complicated, and cute.
And in the case of the electric eel,
you know what happened there.
The neuron synapse, it got rid of the next neuron.
And it just-- in the electric eel,
you have a bunch of synapses, or motor end plates
they're called, in series.
So instead of half a volt, if you have 300 of those,
you get 150 volts.
I think the electric shock that a electric eel can
give you is about 300 volts.
And this can cause you to drown promptly
if you are in the wrong wave when
it happens to bump into you.
I don't know why I'm rambling this way.
You're welcome to study neuroscience.
But please try to help them instead of learn from them.
[LAUGHTER]
Yeah?
They just don't know what a K-line is.
And that's a paper that's been widely read.
It's published in 1980, and Restak says ill-defined.
And I guess he couldn't understand it.
Yep?
Yeah?
AUDIENCE: Why is there no trying to make
the neuroscientists trying to find this in the human mind?
Why don't we just, as computer scientists, program the K-lines
and try to [INAUDIBLE]?
This is the human mind, and we can reproduce it.
Why is not-- is that not widespread into the computer
scientist field?
PROFESSOR: Well, there are--
I'm surprised how little has been done.
There's-- Mike Travers has a thesis, Tony Hearn.
There are three master's theses on K-lines.
They sort of got them to work to solve some simple problems.
But I'd go further.
I've never met a neuroscientist who
knows the pioneering work of Newell and Simon
in the late 1950s.
So there's something wrong with that community.
They're just ignorant.
They're proud of it.
Oh, well.
I spent some time learning neuroscience when I was--
I once had a great stroke of luck.
When I was a--
I guess I was a junior at Harvard.
And there was a great new biology building
that was just constructed.
You probably know, it's a great, big thing
with two rhinoceroses.
What are those-- what are those two huge animals?
So this building was just finished and half occupied,
because it was made with a future.
So I wandered over there, and I met
a professor named John Welsh.
And I said, I'd like to learn neurology.
And he said, great, well, I have an extra lab.
Why don't you-- why don't you study the crayfish claw?
I said, great.
So he gave me this lab, which had
four rooms, and a dark room, and a lot of equipment,
and nobody there.
And he had worked on crayfish.
So there was somebody who went, every week, up to Walden Pond
or somewhere, and caught crayfish, and bring them back.
And I was a radio amateur hacker at the time.
So I was good at electronics.
So I got my crayfish.
And Welsh showed me how to--
the great thing about this preparation
is you can take the crayfish, and if you--
claw-- and if you hold it just right, it goes, snap.
It comes off.
Grows another one-- takes a couple of years.
And then there's this white thing hanging out,
which is the nerve.
And it turns out it's six nerves, one big one
and a few little ones.
And if you keep it in Ringer's solution, whatever that is,
it can live for several days.
So I got a lot of switches, and little inductors, and things,
and made a gadget, and mounted this thing
with six wires going to these nerves.
And then I programmed it to reach down and pick up
a pencil like that and wave it around.
Well, that's obviously completely trivial.
And all the neuroscientists came around, and gasped, and said,
that's incredible.
How did you do that?
[LAUGHTER]
They had never thought of putting the thing back together
and making it work.
Anyway, it was--
I'm always reminding myself that I'm the luckiest person
in the world.
Because every time I wanted to do something,
I just happened to find the right person.
And they'd give me a lab.
I got an idea for a microscope.
And it was this great professor, Purcell,
who got the Nobel Prize after a while.
And he said, that sounds like it would work.
Why don't you take this lab?
It was in the Jefferson.
Anyway-- yeah?
AUDIENCE: I think part of the reason
that you don't see experimental neuroscience on things
like K-lines is that neurons are long and thin.
So if you want to do an experiment to actually measure
a real neural network, you have to trace structures
with, roughly, maybe tens of nanometer resolution.
But you need to trace them over what
might be a couple, or even tens, of millimeters to start to--
and you need to do this for thousands and thousands
of neurons before you could get to the point of seeing
something like a K-line and understanding it.
So it's just a massive data acquisition and processing
problem.
PROFESSOR: Oh, but they're doing that.
AUDIENCE: They're starting to try to.
PROFESSOR: But they don't have--
they don't know what to look for.
Maybe you don't have to do so much.
Maybe you just have to do a few sections here and there
and say, well, look, there were 400 of these here.
Now there's only 200.
It looks like this is the same kind.
Maybe you don't have to do the whole brain.
AUDIENCE: No, but I mean even getting a single neuron is big.
Because it might get down to--
you need to be looking at electron micrographs of brains
that are sliced at about 30-millimeter-- sorry, excuse
me, 30-nanometer slices.
So even just having a single person reconstruct
a single neuron takes--
might take weeks.
PROFESSOR: Well, I don't know.
Maybe a bundle of K-lines is half a millimeter thick.
AUDIENCE: Oh, so If you actually do some larger-scale structure
to start off looking at, yeah.
PROFESSOR: Why not?
I just think they have no idea what to look for.
I could give you 20 of those in five minutes,
but nobody's listening.
AUDIENCE: So I guess you need to know
what it looks like before you can look for it at that scale.
PROFESSOR: What scale?
AUDIENCE: I don't know.
I mean, they know what neurons look like.
So you know--
PROFESSOR: Yeah
AUDIENCE: You know what to look for if you're
saying a neural net level.
PROFESSOR: I'm saying you may only have
to look at the white matter.
AUDIENCE: Oh, all right.
PROFESSOR: Ignore the neurons.
Because the point of K-lines is, where do these go?
And what goes into them and out?
I don't know.
It's just this idea, let's map the whole brain,
100 billion things.
And then people like Restak says, oh,
and there's 1,000 supporting cells for each neuron.
He's just glorying in the obscurity of it
rather than trying to contribute something.
Anyway, if you run into him, give him my regards.
[LAUGHTER]
I really wonder how somebody can write something like that.
Yes?
AUDIENCE: Excuse my ignorance, but what is a K-line?
PROFESSOR: The idea is that--
suppose one part of the brain is doing something,
and it's in some particular state that's very important,
like--
I don't know, that--
like I've just seen a glass of water.
Then another part of the brain would
like to know there's a glass of water in the environment.
And I've been looking for one.
So I should try to take over and do something about that.
Now at the moment, there is no theory
of what happens in different parts
of the brain for a simple thing like that to happen, no theory
at all, except they use the word "association."
Or they talk about, what are the purposeful neurons?
Goal-- forget.
OK, so my theory is that there are
a bunch of things which are massive collections of nerve
fibers, maybe a few hundred or a few thousand.
And when the visual system sees an apple,
it turns on 50 of those wires.
And when it sees a pear, it turns on a different 100 or 50
of those wires.
But about 20 of them are the same, so forth.
In other words, it's like the edge of a punched card.
Have you ever seen a card-based retrieval system?
If you have a book that has--
suppose it's about physics, and biology, and Sumatra.
And a typical 5 by 8 card has 80 holes in the top edge.
So what do you do is, if it's Sumatra,
you punch eight of these holes at random, particular set.
They're assigned to Sumatra.
And then if it's--
I forget what my first two examples were.
But you punch eight or 10 holes for each of the other two
words.
So now there are 24 punches.
Only probably four or five of them are duplicates.
So you're punching about 20 holes.
And now, if something is looking for the cards that have--
were punched for those three things,
even if there are 30 or 40 other holes punched in the card,
you stick your 20 wires through the whole deck and lift it up.
And only cards fall out that had those three categories punched
for.
So you see, even though you had 80 holes,
you could punch combinations of up to a million
different categories into that.
And if you have to put a bunch of wires through,
you'll get all of the ones that were punched for those cate--
the categories you're looking for.
And you might get three or four other cards
that will come down also.
Because all of the eight holes were punched for some category
by accident.
Do you get the picture?
I'll send you a reference.
It was invented by a--
in the 19-- early 1940s by a Cambridge scientist
here named Calvin Mooers and was widely
used in libraries for information retrieval
until computers came along.
But anyway, that's the sort of thing
that you could look for in a brain
if you had the concept in your head of Zastocoding.
But I've never met a neuroscientist
who ever heard of such a thing.
So you have this whole community which
doesn't have a set of very clear ideas about different ways
that knowledge or symbols could be
represented in neural activity.
So good luck to them when they get their big map.
They'll still have to say, what do I do with 100 billion
of these interconnections?
Yeah?
AUDIENCE: What are your thoughts about the current artificial
intelligence research at MIT, such as Winston's genesis
project?
PROFESSOR: I think Winston is just about one
of the best ones in the whole world.
I don't know any other projects that
are trying to do things on that higher
level of commonsense knowledge.
He's just lost a lot of funding.
So one problem is, how do you support a project like that?
Have you followed it?
I don't know if there's a recent summary of what they're doing.
AUDIENCE: [INAUDIBLE]
PROFESSOR: We used to write a big--
a new book every year called the progress report.
The nice thing is that we never wrote--
we had a very good form of support from ARPA, or DARPA,
which was, every year, we'd--
every year, we'd tell them what we had done.
They didn't-- they didn't want to hear what we wanted to do.
And things have turned the opposite.
So what would happen is, every year, we'd say,
we did these great things, and we might do some more.
Went on for about 20 years.
And it was-- and then it fell apart.
One thing-- it's a nice story--
there was a great liberal senator, Mike Mansfield.
And unfortunately, he got the idea
that the defense department was getting
too big and influential.
So he got Congress to pass a law that ARPA shouldn't be allowed
to support anything that didn't have
direct military application.
And Congress went for this.
And all of a sudden, a lot of research
disappeared, basic research.
It didn't bother us much.
Because we made up applications and said,
well, this will make a military robot
that will go out and do something bad.
I don't remember ever writing anything at all.
Because-- but anyway, around 1980,
the funding for that sort of thing
just dried up because of this political accident.
It was just an accident that ARPA,
mainly through the Office of Naval Research,
was funding basic research.
And that was a bit of history.
If you look back at the year 1900 or so,
you see people like Einstein making these nice theories.
But Einstein wasn't a very abstract mathematician.
So he had a mathematician named Herman Weyl polishing
his tensors and things for him.
And Herman Weyl's son, Joe, was at the Office of Naval Research
in my early time.
And that office had spent a lot of secret money
getting scientists out of Europe while Hitler was marching
around and sending them to places like Princeton
and other forms of heaven in the--
in Cambridge.
And again, one of the reasons I was lucky is that I was here.
And all these-- you know, if you had a mathematical question,
you could find the best mathematician in the world
down the block somewhere.
And Joe Weyl was partly responsible for that.
And the ONR was piping all that money to us for work
on early AI.
So it was a very sad thing of the--
maybe the most influential liberal in the US government
actually ruined everything by accident.
ARPA changed its name to DARPA.
It was Advanced Research Projects Agency.
And it had to call itself Defense Advanced Research
Projects Agency.
Yeah?
AUDIENCE: My question is, do you think
the achievement of artificial intelligence is inevitable?
Or is there an obstacle that we're just never going
to be able to overcome?
PROFESSOR: Well, Christianity wiped out science.
That might happen tomorrow.
Only choose your religion.
AUDIENCE: Even favorable circumstances.
PROFESSOR: It's a hard problem.
The number of people working on advanced ideas in AI
has gotten smaller and smaller as the--
f right now, the--
around 1980, rule-based systems became popular.
And there were lots of things to do.
Right now, statistical-based inference systems
are becoming popular.
And as I said, these things are tremendously useful.
But the problem is, if you have a statistical system,
the important part is guessing what
are the plausible hypotheses and then making up the--
then finding out how many instances of that
are correlated with such and such.
So it's a nice idea.
But the hard problem is the abstract, symbolic problem of,
what sets of variables are worth considering at all when
there are a lot of them?
So to me, the most exciting projects
are the kind that Winston is developing for reasoning
about real-life situations.
And the one that Henry Lieberman--
would you stand up, Henry?
Lieberman runs a world-class group
that's working on commonsense knowledge
and informal reasoning.
And it seems to me that that's the critical thing that all
the other systems will need.
In the meantime, there are people
working on logical inference, which
has the same problem that statistical inference has--
namely, how do you guess which combinations of variables
are worth thinking about?
Then it seems to me that the statistics isn't so important.
In fact, there's a great researcher
named Douglas Lenat in Austin, Texas
who once made an interesting AI system that
was good at making predictions and guessing explanations
for things.
And it was sort of like a probabilistic system.
It had a lot of hypotheses.
And every time one of them was useful in solving a problem,
it moved it up one on the list.
So Lenat's thing never used any numbers.
It didn't say, this is successful 0.73 of the time,
and now it's successful 0.7364825 of the time.
What it would do is, if something was useful,
it would move it up past another hypothesis.
Every now and then, it would put a new one in.
Well, if you're doing-- if you're
trying to solve the problem, what do you need to know?
You want to know, what's the most use--
what's the most likely to be useful one, and try that.
You don't care how likely it is to be useful as long as it's
the most, right?
I mean, if it's one in a million, maybe you should say,
I'm getting out of here.
I don't-- I shouldn't be working in this field at all,
or get a better problem.
But Lenat's thing did rather wonderfully
at making theories by just changing
the ranking of the hypotheses that
was considered-- no numbers.
It did something very cute.
He gave it examples of arithmetic.
And it actually-- it was a rather long effort.
And it actually learned to do some arithmetic.
And it invented the idea of division
and the idea of prime number, which was some number that
wasn't divisible by anything.
It decided that 9 was a prime--
didn't do much harm.
And it crept along.
And it got better and better.
And it invented modular arithmetic by accident
at some point.
And it's a PhD thesis.
A lot of people didn't believe this PhD thesis,
because Lenat lost the program tape.
[LAUGHTER]
So he was under some cloud of suspicion
for people thinking he might have faked it.
But who cares?
Anyway, I think there's a lesson there,
which is that, let's start with something that works.
And then, if it's really good, then
hire a mathematician who might be able to optimize it
a little.
But the important thing was the order.
And a good statistical one might waste a lot of time.
Because here's this one that's 0.78.
And here's this one that's 0.56.
And it's the next one down.
And you get a lot of experience.
And it goes up to 0.57 and 0.58.
And it never-- you know, might be a long time
before it gets past the other one,
because you're doing arithmetic.
Whereas in Lenat's, it would just pop up past the other one.
Then it would get tried right away.
And if it were no good, it would get knocked down again.
Who cares?
So it's a real question of--
I don't know.
Mathematics is great.
And I love it.
And a lot of you do.
But there should be a name for when it's actually slowing you
down and wasting your time because there's
a better way that's not formal.
Yeah?
AUDIENCE: Isn't there a saying there
are people who know the price of everything
and the value of nothing.
PROFESSOR: That's very nice.
Yeah?
AUDIENCE: I know you're also a musician.
So I have a music-related question.
What do you think is the role of music?
Like, why do all cultures have it?
PROFESSOR: I have a paper about it.
Oh, OK.
I've been trying to revise it, actually.
But it's a strange question, because there is music
everywhere.
On the other hand, I have several friends
who are amusical.
And so I have this theory that music
is a way of teaching you to represent things
in an orderly fashion and stuff like that.
Well, I have three of my colleagues who aren't musical,
but they dance.
So they may-- it may be that--
I don't know the answer.
It's interesting.
The theory-- the first theory in my paper
is that when you have a lot of complicated things happening,
then the only way to learn is to represent things that happen,
and then look at the differences between things
that are similar, and then try to explain
the differences, right?
I mean, what else is there?
Maybe there's something else.
So in order to become intelligent and understand
things, you have to be able to compare things.
And to me, the most important feature of what's called music
is that it's divided into measures--
bah, bah, bah, bah, bah, bah, bah, bah, bah, bup, bah, bah.
And measures are the same number of beats, or whatever they are.
And so now you can say da, da, da, da, da, da, da, da, da, da,
da, da, da.
What's the different?
You changed the eighth notes in the second one, the last four
eighth notes--
no, the two before last-- to a quarter note.
So you're taking things that were in different times,
and you're superimposing those times.
And now you can see the difference.
And the reason you can see the difference
is that you have things called measures.
And the measures have things called beats.
And so things get locked into very good frames.
Now there's some Indian music which
has 14 measures for a phrase.
And some of the measures go seven and five.
And I can make no sense of that stuff whatever.
And I've tried fairly hard, but not very.
So I don't understand how Indians can think.
Any of you can handle Indian music?
AUDIENCE: I guess, just to add on what you said about this,
my favorite quote from your paper
on music, mind, and meaning is the one about what good is
music, about how kids play with blocks to learn about space,
and people play with music to learn about time.
And I think, in that sense, both music and dance
are different ways that people can arrange things in time.
And in a sense, improvisatory and improvisatory movement
are both ways of--
different blocks, if you will, in time as opposed to space.
PROFESSOR: Mm-hmm.
Yeah, my friends who seem amusical, they probably--
maybe there's something different about their cochlea.
Or maybe they have absolute pitch
in some sense, which is a bad thing to have.
Because if you're listening to a piece composed
by a composer who doesn't have absolute pitch,
then you're reading all sorts of things
into the music that shouldn't be there.
And if you're-- and the opposite would be true.
I read music criticism sometimes.
And maybe the reviewer says, and after the second and third
movement, he finally returns to the initial key
of E flat major--
what a relief.
Well, I once had absolute pitch for a couple of weeks,
because I ran a tuning fork in my room for a month.
And I didn't like it.
[LAUGHTER]
Because you can't listen to Bach anymore.
Oh, well.
It's a good question, why do people like music.
And I don't know any other paper like mine.
If you ever find one, I'd like to see it.
Because if you go to a big library,
there are thousands of books about music.
And if you open one, it's mostly Berlioz complaining
that somebody wouldn't give him enough money
to hire a big enough chorus.
But I've found very few books about music itself.
Yes?
AUDIENCE: It's not about music anymore.
Is that all right?
OK.
Do you think that having a body is a necessary component
of having a mind?
Could you do just as well just as a--
you know, sort of a simulated creature?
PROFESSOR: Oh, sure, you could--
AUDIENCE: And have all the things?
PROFESSOR: Simulation-- I think a mind that's not in a world
wouldn't have much to do.
It would have to invent the world.
And I don't see why it couldn't.
But you might have to give it a start, like the idea of three
or four dimensions.
But can't you-- what happens if you sit back and just
think for a while?
You wouldn't know if your body had disappeared for--
would you?
There also is a strange idea about existence and--
why do you think there's a world?
One of the things that bugs me is people
say, well, who created it?
And that can't make any sense.
Because this is just a possible world.
Suppose there are a whole lot of possible worlds,
and there's one real one.
How could you ever--
how could you possibly know which one you're in?
And then you could say, well, didn't someone have to make it?
And what's the next thing you'd ask?
Well, who made the maker?
So the body-mind thing--
seems to me that once you have a computer,
it can be its own world.
It just can sit.
The program can spend half the time
simulating a world and half the time
thinking about what it's like to be in it.
Yeah?
AUDIENCE: Do you have a current theory of existence?
PROFESSOR: Yes, it's an empty concept.
It's all right to say this bottle exists,
because that's saying this bottle is in this universe.
But what would it mean to say the universe exists?
The universe is in the universe?
So there's something wrong with thinking about--
so there are only possible worlds.
There's no-- it doesn't make any sense to pick one of them
out and say that's the real one.
Yeah?
AUDIENCE: So it's existence is relative?
PROFESSOR: Yeah, you don't say, this is the world I'm in.
But you shouldn't say--
that doesn't mean it exists.
Like, 2 is in the set of even numbers.
But what's the set of even numbers in?
It doesn't stop anywhere.
Yeah?
Lots of worlds.
AUDIENCE: So is mathematics [INAUDIBLE] worlds.
But physics, it only explains the current world?
Or I don't know how you compare these two subjects.
PROFESSOR: Well, you can't tell.
Because five minutes from now, everything might change.
So nothing ever explains anything.
You just have to take what you've got
and make the best of it.
Yeah?
AUDIENCE: So this relationship means systems knowledge
in artificial intelligence?
PROFESSOR: Which knowledge?
AUDIENCE: Systems, basically, in general.
PROFESSOR: Well, there are people
who talk about systems theory, but I'm not sure
that it's well-defined.
AUDIENCE: Right, exactly.
PROFESSOR: Artificial intelligence
means, to me, making a system that is resourceful and doesn't
get stuck.
And so if you have a system--
and also, it's an--
how do you put it?
Some definitions are not stationary,
like what's popular.
Popular is what's popular now.
There isn't any such thing as popular music
in terms of the music.
So I know there were--
there was once a little department
called systems analysis at Tufts, which
had a couple of rather good philosophers
try to make general theory of everything.
And they were writing nice little papers.
And it got a--
it moved along.
But then there was this Senator McCarthy
you've probably heard of.
And he announced that he had evidence that the--
one of the principal investigators
had slept with his wife before they were married.
Well, Tufts was very frightened at this
and abolished that department.
And Bill Schutz went to California, and started.
Esalen, and had a good time for the next 50 years--
more stories.
Yeah?
AUDIENCE: Kind of as a extension of the body-mind question,
it seems to me we-- as humans, we--
learn a lot from just interacting
with the environment.
Like language, we hear it being spoken.
We speak it.
We see things.
We touch things.
But as far as I know, a lot of the efforts
in artificial intelligence so far
have been confined to the computer that does not go out
into the real world, interact.
Doesn't [INAUDIBLE] to see, learn new think.
PROFESSOR: Well, here's the problem--
I look over at Carnegie Mellon, and there
are some nice projects.
And the most popular one is robot soccer.
And here are these little robots kicking a ball around.
They're Sony-- what are they called?
AUDIENCE: AIBOs.
PROFESSOR: Yes, the Sony AIBOs.
Sony stopped making the AIBOs.
But it respected Carnegie, and it
made a little stash, secret stash, of AIBOs
to send to Carnegie when the present ones break.
But my impression of AI projects that have robots is that they
do less, less, less than projects that don't.
The reason is, if you have a robot like ASIMO, made by Sony?
No.
AUDIENCE: Honda.
PROFESSOR: Honda.
ASIMO can get in the back seat of a car with some effort,
and usually falls over.
However, if you simulate a stick figure
in a computer getting into a stick figure of a car,
then you can make it learn to do that and get better and better.
And so all AI projects without robots
are way ahead of all AI projects with robots.
And the profound reason is that robots are usually expensive,
and they're always being fixed.
So if you have five students, and the robot is being fixed--
I don't know what they're doing--
but they have to wait.
Whereas if you have a stick figure robot,
then you can just run it on this,
although it might be a little slower than your mainframe--
probably not.
Yeah?
AUDIENCE: So then back to the idea of the mind and the body,
here's a theory that I just thought of--
the idea of the body as a seeming abstract, basically
a mechanism for input-output.
It's a set of sensors from which our brains can get information
about the world, and instead of actuators,
in which we can display our state.
So in that light, it's almost as if our brains are really
independent on our body itself.
It can adapt to any sort of body if we
happen to hook it up that way.
And it just so happens that we've
been hooked up to this body since birth,
that we have such good mental models of how to use this body.
And I guess an example of--
from experiments that support this theory might be how,
when people have limbs amputated,
it takes them a while to forget that they have the limb.
Because their mental model still exists.
The mental models don't go away overnight.
And also, I guess, they train monkeys to control robot arms
with their brains.
PROFESSOR: Sure.
Well, but it just seems to me that a large amount
of our brain is involved with highly evolved locomotion
mechanisms.
And as I said, when you're sitting back with your eyes
closed in a chair thinking about something,
then it's not clear how much of that machinery is important.
But it might be that--
I have a strange paper on--
I don't know if it's--
trying to remember its name.
It's called-- do you think I can actually get a--
I can't remember the name of the title.
Oh, I give up.
The idea is that maybe--
in the older theories of psychology,
everything is learned by experience in the real world--
so conditioning, and reinforcement, and so forth.
In this theory I call internal grounding, I make a conjecture.
Suppose the brain has a little piece of nerve tissue which
consists of a few neurons arranged to make not
a flip-flop, but a--
what would you call a three- or four-flop, a flip-flop
with three or four states?
Let's say three.
So when you put a certain input, it goes from--
couldn't find the chalk.
So here are three states.
And here's a certain input that means if you're in that state,
you go to this.
And if you pop that input again, it does this.
And if you, say, go counterclockwise, it goes.
So three of them get you back where you were.
But if I go this, this, and that,
that would mean to go like this, this, and back.
So this would be--
that means that's equivalent to just going one.
Get the idea?
In other words, imagine that there's
a little world inside your brain, which is very small
and only has three states.
And you have actions that you can perform on it.
And you have an inner eye which can
see which of the three points of that triangle you're on.
Then you could learn, by experience,
that if you go left, left, left, you're back where you were.
But if you go left, right, left, right,
you're back where you are.
And if you go left, left, right, that's like going one left.
In other words, you could imagine
a brain that starts out-- before it connects itself
to the real world, it starts by having
the top level of the brain connected
to a little internal world which just has three or four states.
And you get very good at manipulating that.
You add more sensory systems to the outer world.
And you get to get--
learn ways to get around in the real world.
So I call that the internal grounding hypothesis.
And my suggestion is, maybe somewhere in the human brain,
there's a little structure that's
somewhat like that, which is used
by the frontal part of the cortex
to make very abstract ideas.
You understand?
The more abstract an idea is, the simpler, and more stupid,
and elementary it is.
Abstract doesn't mean hard.
Abstract means stupid.
Real things like this are infinitely complicated.
So we might have--
and I wouldn't dare suggest this to a neuroscientist.
There might be some little brain center somewhere
near the frontal cortex that allows the frontal cortex
to do some predicting, and planning, and induction
about a very simple--
a few simple, finite-state arrangements.
Who knows?
Would you look for it?
Well, if you were a neuroscientist,
you could say, oh, that's completely
different from anything I ever heard.
Let's look for it.
And if you're wrong, you have wasted a year.
And if you're right, then you become the new Ramon y Cajal
or someone.
Who's the best new--
who's the currently best neuroscientist?
Maybe it's late.
One more question.
One last question.
This is Cynthia Solomon, who's one of the great developers
of the Logo language.
Yay.
Yes?
AUDIENCE: So maybe it's a bad question for the end.
I will ask anyway.
What do you think about theories such as Rodney Brooks' theories
that can speak of no central [INAUDIBLE]??
PROFESSOR: Completely weird.
Obviously, those theories have nothing
to do with human thinking.
But they're very good for making stupid robots.
And the vacuum cleaner is one of the great achievements
of the century.
However, his projects-- what was it called?
Cog-- disappeared without a trace.
That theory was so wrong that it got a national award.
And it corrupted AI research in Japan for several years.
I can't understand--
Brooks became popular because he said,
maybe the important things about thinking
is that there's no internal representation.
You're just reacting to situations.
And you have a big library of how to react to each situation.
Well, David Hume had that idea.
And he was a popular philosopher for hundreds of years.
But it went nowhere, and it's gone, and so is Rod.
However, he is one of the great robot designers.
And he may be instrumental in fixing
the great Japanese nuclear meltdown.
Because they're shipping some of his robots out there.
The problem is, can it open the door?
So far, no robot can open the door
even though it's not locked.
[APPLAUSE]
Thank you.