Elon just claimed that Tesla will ship a fully functional humanoid robot by next year.
Naturally the press thinks it’s impossible. But could he actually do it?
Let’s look at what’s possible now. Boston Dynamics already has a robot that can do parkour
and ten years ago they built a robot that could run 28 miles per hour. The Tesla Bot will only go
5 miles per hour, so it doesn’t seem impossible. In fact, if Elon wanted to release a robot that
matched the capabilities of Boston Dynamics next year, he could. And I’m being 100% serious.
Boston Dynamics is only worth one billion dollars, but Elon is worth
over a hundred and eighty billion dollars, so he could just buy Boston Dynamics, paint the Atlas
robot black, and start producing them next year. Obviously Elon isn’t going to do that, but the
fact that he could should make you pause before you say that Tesla Bot will not ship next year.
And even if Elon misses his target ship date by years,
does it really matter? Two things are clearly true: building the Tesla bot doesn’t break the
laws of physics, so it’s possible. And building such an advanced piece of technology will require
lots of smart people to work on the project, and that’s why Tesla is recruiting so aggressively.
Elon is known for always announcing unrealistic timelines and then missing them,
sometimes by years. But I think this is a lot less important than most people seem to think.
Everyone agrees that Steve Jobs was one of the most impactful technologists of the last hundred
years. But he missed deadlines all the time too. In 1983, he said he wanted to build a
networked computer the size of a book and would ship it by the end of the decade:
“Apple’s strategy is really simple. What we want to do is we want to put an incredibly
great computer in a book that you can carry around with you and learn how to
use in 20 minutes. That’s what we want to do and we want to do it this decade,
And we really want to do it with a radio link in it so you don’t have to hook up
to anything and you’re in communication with all of these larger databases and other computers.”
Well, the first iPad didn’t ship until 2010, so Jobs missed his prediction by a full 20 years,
but it didn’t matter at all. In the time between 1983 and 2010,
Apple shipped dozens of incredible products that moved the entire industry forward.
Looking back now, no one remembers that Jobs missed that deadline! Just that when he finally
did ship the iPad, it was a great product. That Steve Jobs clip comes from a talk he
gave in Aspen and his goal with the talk was to recruit talented engineers to come work for Apple.
Elon is doing the exact same thing with AI Day. He’s highlighting all the incredible advances that
Tesla has made in the field of self-driving and teasing what is going to come next.
And it’s the advances in self-driving that we should really be focusing on,
because the stuff that Tesla is doing right now is absolutely incredible.
The entire company has basically transformed over the past two decades. What started as a hardware
company focused on building cars has transformed into a software company focused on building self
driving technology. And now, the AI models that power Tesla’s self-driving cars have become so
complex, that Tesla has gone a layer deeper and become a hardware company that makes computer
chips. It’s really incredible stuff. During Tesla’s AI day, Elon’s team
explained how they are tackling hard problems across three key domains.
The first is the core neural network architecture that powers autopilot. It’s a common misperception
that self-driving cars use a single AI system to process camera data and then control the car.
There are a few companies that are trying to do that type of “end-to-end” self-driving,
like Comma AI, but Tesla actually breaks the problem down into smaller parts.
AI is only used to understand the environment around the car,
the actual decision making about where to drive within that environment is developed using more
traditional software development techniques. Even though Tesla could use machine learning
to plan out the route taken by the car, they don’t, and Elon had a great explanation for why:
“As long as you’ve got surround video neural nets to understand what’s going on and can
convert those surround video into vector space, then you basically have a video game. If
you’re like in Grand Theft Auto, you can make the cars drive around without crashing. You don’t have
to have a neural net for control and planning." See, programmers have been able to design basic
systems for cars in video games for years that react to environments and avoid collisions.
That part is actually not that hard. The hard part is understanding the real world
and all of it’s incredible variations as if it was as simple as a video game.
Elon mentioned the importance of “surround video” and “vector space” because that’s where Tesla took
a big step forward. Fortunately, those two phrases sound more complicated than they actually are.
Surround video just means stitching all of the different cameras around the car into one big
image. Years ago, when Tesla implemented the first version of autopilot, the car only had one camera,
so this wasn’t an issue. The only job of the autopilot was to keep you in the lane
and stop you from running into a car directly in front of you. But now, Tesla cars ship with
lots of cameras placed all around the car. This creates a ton more data for the machine
learning algorithm to process, and up until recently, the Tesla systems were dealing
with each camera feed individually. This caused some problems though,
like when a big truck appeared on multiple cameras, it became very difficult to understand
exactly where that truck was in the real world. Allowing the system to see all the cameras at
once as “surround video” also made it possible to translate the two dimensional images into three
dimensional representations. That’s what Elon is talking about when he says “vector space”. The new
autopilot system now takes all of the camera data and creates a fully 3d representation of the data,
just like a video game. And this has some powerful implications, especially on the
second key Tesla innovation: training data. Every machine learning system needs data and
there are three main ways that companies generally get data. The most reliable data is hand-labeled
by a human, but that’s very expensive. Using a separate system to automatically label data
is much cheaper, but it can be prone to error. Lastly, you can create an infinite
amount of data using simulated environments. AI experts have debated which kind of data is best
for years, but Tesla just said “yes, and” to all three kinds. They hired a thousand data labelers,
built an automated system for labeling the data that the humans couldn’t get to in time, and then
also developed a remarkably realistic simulation environment for generating even more data.
All of these systems help teach AutoPilot about various situations it might see in the real world.
And they all support each other and help fill gaps in the data. For example,
the simulated environment is great for creating bizarre scenarios like a group of people running
down the middle of a highway. It’s unlikely to happen in the real world, but if it does happen,
you’ll want your Tesla to know what to do. And all of this is aided by that move to “vector
space” that Elon mentioned. Human labelers can now manually identify a traffic cone in a 3D
environment that looks like a video game and the raw images from each camera that saw that
traffic cone will be updated automatically. This three pronged approach shows that the
Autopilot team isn’t turning away any good ideas. They’ll use every
tool at their disposal to build a reliable system. But doing so creates new problems.
Tesla now has hundreds of thousands of cars sending in real world video feeds every single
day, and most of those videos get labeled with either by humans or automated systems,
so the AI team at Tesla now has an immense amount of data to work with.
They train their models multiple times per month, which means running all of that data through a
massive supercomputer in order to recalculate the entire neural network. This is an incredibly
computationally complex task and it has to be done on a special type of computer chip.
Traditionally, the training of neural networks has been done using graphics processing units,
or GPUs, and there’s really only one major player in the GPU space at the moment:
Nvidia. The problem is that Nvidia has increased their prices to the point where it no longer makes
sense for big companies like Tesla. But there is another option. In 2016,
Google released the Tensor Processing Unit, or TPU. It’s a chip specifically designed for
training these machine learning models. It’s a good option for many companies,
but it wouldn’t work for Tesla because they are in direct competition with Google.
See, you can buy an Nvidia graphics card and basically do whatever you want with it.
But Google’s TPUs are kept behind lock and key in Google’s Cloud datacenters,
and in order to use them, you have to agree to their terms of service. The thing is, one of the
clauses in that agreement states that you promise not to use their systems to train self-driving
cars. So that option is off the table for Tesla. But Elon saw this coming and had the Tesla team
start building their own “D1” chips that would serve as the basis for a new supercomputer called
Dojo that could be used to train autopilot. They still haven’t deployed this system fully, but for
anyone who was paying close attention, this was by far the biggest announcement of the day.
The scale of everything they are doing is absolutely insane. A single D1 chip is one
thousand times more powerful than a brand new PS5. And Tesla has connected one million of these chips
together to create their compute center. The complete system has 1.1 exaflop of computing
power, which is so massive, it’s likely the most powerful supercomputer ever created.
The combination of improvements to the neural network, training data,
and underlying hardware are going to compound to help Tesla finally get across the finish line
and deliver full self-driving. But Elon obviously isn’t stopping there.
After over an hour of technical demonstrations, Elon decided to have some fun and spent 5 minutes
talking about the Tesla bot. He’s clearly having fun with it, but I do think that he
really believes that Tesla will be able to transfer a lot of the work they have
done in self-driving to the humanoid form. But there’s a big question, why try to make
it look so much like a person? Science fiction authors have been writing about humanoid robots
for decades, but building something with all the flexibility of the human body is incredibly
difficult. It’s much easier to build a robot vacuum than a robot that vacuums after all.
I think the humanoid form could actually wind up being an intermediate step on the path to
even more useful robots. Kind of like how Apple tried to make everyone look analogue
in the first versions of the iPhone. Consider the calculator app. In the 1980s,
running calculations on a computer required keying in values on a command line, the device
looked nothing like a real-world calculator. But when Apple released the iPhone, they opted
for a “skeuomorphic” design that mimicked physical calculators. Only after a few years
did Apple realize that users didn’t need this familiarity, and opted for a more minimal design.
The same could be true for robotics. We could move from iRobot vacuum cleaners, to Tesla bots,
and then to something entirely new. But I’m excited to see where it goes.
And remember, even if Elon delivered the Tesla bot in the year 2040, he would still be faster
at shipping than Steve Jobs, and that would be pretty incredible. Thanks for watching.