Do you see a problem where
the commercial applications of this
and the money going into it
is a gold rush, even making
the Internet gold rush seem modest?
Would that possibly
drain people out of academia,
who are doing the important work,
or do you see
that happening somewhat?
Unfortunately, there’s a leak
from academia to industry.
But actually,
there’s a bigger concern for me.
Whether they’re in industry
or in academia,
I do worry that a lot of people
feel a bit hopeless,
in the way that
there’s really strong messages
dominating the field,
which is that scale is all you need,
and GPT-5, 6, 7,
will be even more amazing.
There’s maybe nothing
one can do about it.
There’s a bit too much
currently shifted towards
the prompt engineering
as the main research focus.
I genuinely worry about that,
everybody doing the same thing,
can that be good?
I do hope that people explore
what happens with the bigger scale
out of curiosity,
but the fact that there’s so much emphasis,
and all the companies, major companies,
now they feel like they need to
catch up with ChatGPT.
I hear from many friends that there’s a lot
of this internal refocus, reprioritization,
which is totally understandable,
but if this is a global phenomenon,
that’s not healthy at all.
We need to put more research effort
around safeguarding AI
and building alternative methods
that are more compute efficient,
and therefore also less carbon footprint.
We need to bring math
and maybe even physics people,
but certainly math people.
I feel lucky that I was a mathematician
and then did computer science,
because these models
are very mathematical.
Just being a programmer
isn’t really the training you need
for this stuff.
And currently, brute force at scale
is the way to go,
but there may be an alternative,
where sometimes these smaller models,
the specialized models
do learn on a lot more specialized data,
and the data is actually the key.
And that data can be not just more data,
but it’s better data, high quality data.
Sometimes the data that was
really designed to teach you that particular
mathematical concept, for example.
When you think about humans also,
nobody learns very well
just by reading random Web data.
We tend to learn better when
there’s a great textbook and tutorial.
Similarly, I do think that
this is about how to transfer knowledge
or information in the most efficient way.
That’s another reason, for me,
why I believe that the smaller model
or modest-size model could have a major edge.
But that requires innovation about how
to get that information, alternatively.
What are some of the ways
you’re most enthused about
that AI can help us improve the world?
My wishful thought is AI
to really better understand humans more
than humans ourselves do.
I think that’s fundamentally a reason why
there’s a lot of conflict.
There’s a lot of disagreement,
and I’m hoping that we can use AI
as a tool to better reflect about ourselves,
and then be able to
communicate to each other better,
and coexist together more peacefully.
I completely agree with that.
It’s kind of scary,
that we seem to be more polarized.
Other technologies gave us
hydrogen bombs and bioterrorist pathogens.
It’s just a dream,
because the AI is not there yet,
but if it could help us understand each other
and maybe reverse this trend
towards polarization,
that would be an incredible favor
to the world.
A lot of people worry about AI safety,
that it doesn’t take over the world,
but at the same time,
maybe it can improve and reduce conflict,
and improve understanding.
That’s worth working on.
Subscribe to ”Unconfuse Me” wherever you listen to podcasts.