A question that has inspired
computer scientists for decades
is about, can we build
machines that can program
themselves?
Now to answer that question, in
its essence, we
really need two aspects.
First one is intelligence.
In other words, I would say
artificial intelligence, AI.
The second ingredient needed is
for machines to be able to
understand their own language.
I would say code understanding.
Code understanding.
To resort in a certain area
that we are going to dove deeper
into today, called AI
for code.
Now, significant progress
has been made over the last decade
in artificial intelligence itself.
If I were to look at what were the
major foundational
pillars that resulted
in those breakthrough innovations,
which is percolating through our
society today, they
were...data,
algorithms, and
very powerful computer
hardware.
When massive amount of data,
combined with breakthrough
innovation now in algorithms,
combined
with very fast computing
hardware resulted
in tremendously
powerful and large AI
models.
We were able to understand
seamlessly human language
as we speak, human language
among each other and understand
each other. Machines are now able
to understand us as well, which
resulted in machines to be
able to understand
the perceptual world around us, the
visual world around us.
For us to be able to build
self-driving cars and
for machines to be able to
understand the textual documents
that we write as well.
If I were to look at
one pillar, which was most
important among that, I
would point out that will be
data.
In fact, even among data, a
particular data source has had
a pivotal role to play in this.
That data source was called
ImageNet.
It i s said that there
is no AI without data.
And in fact, I would say there
wouldn't have been the latest
incarnation of AI without
ImageNet. This
was a dataset that had 14
million images
and twenty two thousand
classifications.
This resulted in breakthroughs in
algorithms that we are
reaping the benefit of in
other modalities as well, like
natural language processing and
beyond.
And we believe, that
the breakthroughs in natural
language processing can not
only help us understand human
language, but they actually can
help us understand machine
language as well in terms
of machine language understanding,
machine language reasoning, machine
language explainability and
so on.
So the question arises, "what
is needed for AI for code?"
And same three pillars
which were foundational for progress
in AI, will be needed
for AI for codd
progress as well.
And then the second question arises,
"what is our ImageNet
moment?" And in fact,
very recently we announced
the ImageNet moment for
AI for code called
"CodeNet."
CodeNet has,
just like ImagineNet, 14
million code samples in
55 different
programing languages.
And to top it off, five
hundred million, or half
a billion lines of code.
It's
a massively large, first-of-its-kind
dataset, which is available
to researchers
and developers alike in open
to be able to make massive
progress in algorithms
for AI for code
to bring together these three
pillars so that we can accomplish
tasks like code
language translation.
Code debugging.
Developers spend most of their
time not just writing
code, but most of the time is spent
debugging code.
For machine to be able to generate
new code, the nirvana is
imagine the scenario
where rather than, you know, people
just leaning on keyboards and typing
these programs to be able to build
applications that we utilize
every day for us to be able to talk
to machines and they actually
generate the code automatically.
So, code generation from
natural language.
Code performance improvement.
My code doesn't work as well.
Can you make recommendations to
improve my codes performance?
Code memory improvement,
my code doesn't scale as well.
Can you make recommendations
so that my code scales much
better?
And finally,
for us to be able to do code review?
Point out all the flaws in my code.
It suggest little
functionality improvements and so
on. Now to able to accomplish
all of these tasks, which are
critical to building software
systems which are now part
of every aspect of our society,
since software has eaten the world
already.
We need to think about it in a
very organized way.
And for that, we have organized
ourselves in building a stack
called AI for Code Stack.
AI for Code Stack
is comprised of multiple layers,
the first player, as I said, there
is no AI without data.
So the first layer is actually
the data layer itself.
When we talk about data for AI
for Code, we're not just talking
about source code.
We actually are talking about
source code.
We are talking about configuration
files that are able to run that code
and deploy that code.
We are talking about ingesting
data sources.
We're just like developers
are on some of
these social media forums
to be able to debunk
their problems, to be able to get
help for their problems that they
are experiencing on a daily basis.
Forums like Stack Overflow,
like Quora.
If I were to have a hope
of addressing these problems
that I just outlined, it need to
understand the knowledge which
has been gathered over decades in
solving those problems as well
by humans.
So source code is part
of that data. Configuration
files are part of the data.
And finally, forums are
part of the data and many other data
sources.
The next layer is what I describe
as ingestion layer.
To be able to ingest all these
diversity of data sources and
to be able to build
a representation which can
actually correlate among that them
to be able to so that we can reason
upon them as well.
This is the data ingestion layer.
Which results in, or
which leads me to the next layer,
which we call intermediate
representation layer.
Also, known as IR layer.
Now, IR layer, think about,
think of this as I did
a multitude of data sources, I
correlate among them, to
this is, think of it as graph
representation, just like we did
with many other dependencies
in our world with graphs, think
about it like those graph
representations.
So this is actually comprised
of graphs.
Now right above it
is a layer, think of this
as graph algorithms
layer,
with something called embeddings.
Embeddings are critical to
converting the intermediate
representation into numbers
because computers can only deal with
numbers, so the embeddings
and graph algorithms allow us
to convert the representations
in IR into numbers
so that algorithms can work on them.
And finally, we
have the representation, a
layer of AI
algorithms now.
Also
known as graph neural
networks is one kind of techniques
in that, and there are many other
techniques which as
I speak, researchers are working
on to be able to build more and more
powerful AI for code techniques.
And above that, are
applications that we are building.
And to build those applications,
there are four major capabilities
needed.
For us to be able to understand
code.
For us to be able to retrieve
and search code.
For us to be able to generate
new code, just like the example
I gave of natural language
and code getting generated.
And for us to be able to test
and verify code.
And these four capabilities,
these four major capabilities
can be combined
to build applications for
the real world, like
my code has security
flaws.
Can AI help me understand
and identify those security flaws
automatically and fix them?
So what we will call AI
driven vulnerability analysis.
Another key applications,
AI to be able to modernize my
legacy infrastructure or legacy
software systems.
Now there were languages
that were invented decades back
like COBOL, which
need to be modernized because the
skills for understanding and
modernizing those software systems
have really gone away.
But the need hasn't gone away at
all. It's actually called the
famous 100 billion dollar problem
because they are the more than 100
billion lines of COBOL code
that exists, and we need to be able
to modernize them into some of
the more recent languages,
like Java and others.
It takes 50 cents a line to be able
to modernize it.
There lies the problem 50 to 100
billion dollars and a massive
time crunch in which we need to
modernize them, AI to
the rescue.
AI for modernizing
legacy systems.
AI to be able to
test and debug my
system.
AI to be able to generate
new code, and
build my applications.
This stack, which
we call AI for Code Stack,
and
the progress that we can make
in connecting data
through algorithms.
And finally, to compute hardware and
connect all of these three together
to give rise to massive
innovation, which will
result in answering the question
that computer scientists have
pondered for for decades,
can we build machines that can
program themselves?
And I think we are closer to that
reality than ever before,
and I'm looking forward to the
progress in this area.
Thank you.