I'm Tomaso Poggio and you have heard already
from Jim and Daniela about science and about engineering.
Science of intelligence meant as science of natural things.
You know, it's chemistry, physiology, neuroscience.
And engineering as the building artifacts,
understanding how they work.
This includes, of course, for instance microelectronics,
and even mathematics.
I consider it as part of engineering.
Now why science and engineering?
I'll be very brief.
Just look at two recent success stories in AI
that I could argue these are the main two in recent years.
One is AlphaGo winning against human champions at Go.
This is a system developed by DeepMind,
which is part of Google.
And the other one is autonomous driving
and Mobileye is a leading company, now part of Intel,
to develop as a leader in visual-based autonomous
driving.
So the point I want to make here is that these two success
stories are based on two key algorithms
in modern artificial intelligence, modern machine
learning.
One is reinforcement learning and the other one
is deep learning.
And both of them come from neuroscience.
If you look at reinforcement learning,
it came from essentially studying
the psychology of animal learning with Pavlov and then
Donald Hebb and so on.
The first person who realized the relevance
of this work in psychology for artificial system
was probably Marvin Minsky, who in the '50s
developed this system called SNARC
that was able to solve a maze like a mouse
does by trial and error.
And it was made up of neurons and one neuron
is shown there was implemented by several vacuum tubes.
And the current MIT experts on reinforcement learning--
I know one of them, Leslie Kaelbling, is in the audience.
The other one is Demitri Bertsekas.
Deep learning was restorated by Jim DiCarlo.
It started with work by David Hubel and Torsten Weisel
at Harvard.
They recorded from individual neurons
in the visual cortex of monkeys and cats.
They suggested a system that was made up
of several layers of neurons, each one
with local connectivity.
And then there were models by Fukushima and others, our labs
and Jim's labs, and modern deep learning networks like ResNets
have exactly the same layer architecture
with local connectivity.
Now, of course, at MIT we have been acting
on this idea for some time.
And the history of this is about eight years ago,
Rafael Reif, which was then the provost of MIT,
challenged the BCS department to come up
with a bold vision for the future.
Mriganka Sur, who was the chairman at the department,
he put together Josh Tenenbaum and myself in an effort
that we called the Intelligence Initiative, supported
by the dean of science, which reached
across the various schools of the Institute
and funded seed projects in the science, engineering,
and social and economic impact of intelligence.
There are several of the speakers
who are part of this and more people in the audience.
Now, through NSF funding, the Intelligence Initiative
became more focused and morphed into the Center
for Brains, Minds, and Machines.
I think it's appropriate to think of CBMM as part
of the core of IQ in the same way
that DeepMind is part of Alphabet.
In fact, there is a very interesting, important link
between CBMM and IQ.
David Siegel was the very first member
of the advisory committee of CBMM five years ago.
For CBMM, the problem of intelligence
is one of the great problems in science.
Personally, I think the problem of intelligence
is the greatest problem in science,
greater than problems such as the origin of life
or the origin of the universe.
We could argue about it.
And and so our goal at CBMM is to develop the science
and engineering of intelligence, to understand
how the brain creates in the mind,
how to make intelligent machines.
And clearly, the combination of science and engineering in this
is very blurred.
But the fact that we have projects
that are science projects on intelligence,
engineering projects in intelligence,
and projects that requires both science and engineering,
I think this is the unique feature of MIT
and difficult to replicate anywhere else.
Jim DiCarlo showed earlier--
and you'll hear more from Josh Tenenbaum--
showed several moonshot projects.
CBMM is involved in one.
It's the one about visual intelligence,
about seen understanding.
That project is really trying to understand
and simulate our unique ability that none of the deep learning
systems can achieve so far, which is the ability
to answer any question you can be asked
about the environment you perceive
around you and without any pre-training.
Here for instance, this is a picture
of a dinner after a CBMM workshop
in a nice place in Europe.
And you can see, understand what is going on,
kind of guess about discussion at the different tables.
You know immediately how you'd have
to navigate if you want to reach the only empty table
at the extreme left.
You have these illusions of seeing everything around you
and understanding what's going on.
It's an illusion which is really computed
by various parts of our brain, which are more
than just the ventral pathway.
The input from the eye to visual cortex involves other areas.
That's what we are trying to sort out
with cognitive neuroscience techniques
and try to simulate this in computers.
And this also, after trying to answer the question of what
is behind in terms of the brain, what
is behind this perceptual illusion
of perceiving everything with high resolution,
understanding everything around you,
the perceptual equivalent of cogito ergo sum.
So of course, this project of seen understanding,
of visual intelligence is just one
of the several projects you need to attack in order to solve
the problem of intelligence.
And so MIT IQ is the way forward.
Now, while pushing beyond deep learning,
since everybody seems to think, and I agree,
that intelligence is much more than deep learning,
we have also been developing a theory of deep networks.
Because it is actually an interesting irony,
is that the excitement about deep learning has--
the fact that nobody really understand why deep networks
work as well as they do.
And so let me tell you some of this--
the question we are trying to answer in order
to develop a theory that explain when deep networks work
and when they don't work.
And there are these three questions
that were listed before, which are really an approximation
theory, an optimisation theory, and in machine learning.
The first one is the one on power of approximation.
How powerful are deep networks relative to networks
with one hidden layers, shallow networks,
in representing a task, a function, an input
to output mapping?
There was, an order is out in the '80s that said,
any network even with one layer of nonlinear units
can approximate any continuous function on a bounded interval.
And this seems the end of the story.
But it's interesting that this has
what is called curse of dimensionality.
The number of units you need to do this approximation
depends exponentially on the accuracy epsilon
you want to achieve.
And the exponent is the dimensionality
of the function, how many inputs the function has.
And so it's epsilon, 1 over epsilon is 10, 10% error.
1 10 to the D can be very large, if they
say it's few thousand inputs like pixels in an image.
So this is a really serious problem,
this curse of dimensionality.
It turns out that for certain classes of function
that essentially represent task that you
can think of like a decomposable, where you can
speak of things that are made up of parts, which are themselves
parts in larger objects.
Like, say, text.
Text is made up of paragraphs.
Paragraphs are made up of sentences.
Sentences are made up of words.
Words are made up of letters.
For compositional functions, turns out the deep networks,
but not shallow one, don't have the curse of dimensionality.
In And number of parameters depends
not exponentially but linearly in the dimensionality
D. So this is an advantage for special classes
of tasks for deep networks.
And this brings up another question about,
which is almost philosophical, whether the fact that the word
appear to us pretty much decomposable, as I said,
text, speech, even what we see.
There are objects and parts and scenes.
Whether this is due to physics, is
because the word is made this way, or our brains
because we see it this way.
I leave this question open for the question part
and go to finish because I'm overtime.
Just to mention that we have good answer to also
the other puzzles, and so even deep learning
is a neat way to have a deep--
no pun intended-- mathematical theory.
Now, of course the theory's important to satisfy
our basic curiosity for scientific understanding
and for opening the way to better
engineering, all of which is what IQ is about.
Thank you.
[APPLAUSE]