Yes, of course I’ll talk about this. Just a few days ago, OpenAI released Sora,
their text to video AI that took the world by storm. But Sora was not the only thing
that was released that day. DeepMind’s AI assistant, Gemini 1.5 also appeared,
and it offers something that interestingly, not even the great GPT-4 can do.
Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
And the story of Gemini 1.5 is a story of context windows. What is a context window? Well, you see,
we are living in the age of smart AI assistants. They help you with assignments, homework,
it is a personalized teacher, a programmer, your assistant for daily planning, and so much more.
However, there is something still missing. Imagine that you are in a conversation with someone,
and the context window means how long of a conversation this person can remember. If
you have an assistant, you expect them to remember what you just talked about
minutes ago, right? Well, let’s see together if that is really the case.
GPT-4 had an 8 thousand token context window. So it remembers at most a few
pages of a book. By the end of the book, it doesn’t remember who even wrote it or
how it started. And let’s not even talk about movies, that is way too much data.
Then, a few months later, GPT-4 Turbo appeared. We jump from 8k to 128k. Oh my. That is an entire
book. We can talk about a book now. Fantastic. Now hold on to your papers Fellow Scholars,
because from 128 thousand, the new Gemini Pro offers a one to ten million token window.
Wait what? That is insanity. Now let’s see what that means in practice.
Level 1. Now, little AI, here is a 400-ish page transcript of the Apollo 11, the American
space flight that landed on the Moon. Now find three of the funnier moments. Not a problem,
and now a multimodal question. What is this? Of course, you Fellow Scholars already know,
but does the AI know? Oh my, yes, one small step for man, one giant leap for mankind.
And believe it or not, this was the easiest task it was given.
Level 2. Now let’s give it a huge codebase with up to a 100,000 lines of code. Now sift through it,
and show me parts of it that have to do with character animation. Not a problem.
We can even run these animations and now it can go from engineer to teacher
and explain to us how it works exactly. And then, back to the AI engineer again,
and code up a little slider to control the animation. I absolutely love this. How many
people are there who are great teachers and engineers at the same time? Very few. Loving it.
And now, the boss level. Level 3. Check this out. Little AI, forget text, that’s too easy,
let’s step it up and watch a movie together. And, look at that! We can ask questions about a
44-minute movie, and it knows exactly the moment we are asking about. But it gets
better. I did not think it could do this, but it did. Now here is a crude drawing,
what is this and when did it happen? Yup, that’s the scene. This is incredible.
And according to Google, it is going to be released soon. And here comes the kicker.
Oh! This is just the pro, not the ultra version. Remember, the pro version is the
one that is free for everyone to use. The ultra will likely be even better.
Now wait a minute. We are experienced Fellow Scholars here so we have questions.
We found out recently that ChatGPT remembers. What does that mean? Well,
have a look. This is Claude 2.1, a competing chatbot volt Anthropic,
and it had a context window that was twice as big as GPT-4 Turbo. However, wait. Look at this.
Context length is one variable, but being able to actually recall all
this information is also important. And when asking questions about the document,
it matters where in this document the information was. For instance, if it was early on, it might
be forgotten by the time it reads until the end. But ChatGPT remembered much more reliably.
So, our question is clear as day - what is the price to be paid for this million token
context window? Is Gemini going to be as red as Claude 2 here? Let’s see…and…whoa! That is
insanity. It is not perfect, it will forget a few tiny details, but my goodness, look at that
improvement. And all this barely a few days after the initial release of Gemini. Bravo.
And, yes, you’re seeing correctly. We have a paper. Oh yes. So a deeper look is coming soon,
subscribe and hit the bell icon to not miss out on it.
Now the judge is always going to be us, Fellow Scholars, the users. As soon as it is out,
let’s run our own experiments and find out how well it works in practice. But,
just think about it. Given this pace of progress, we will soon have essentially
almost infinite tokens for a context window. This is likely not even years away, but a year away,
possibly just months away. And then, it will know not just one film, it will know the entire
history of films better than Tarantino. Well, maybe not as well as Tarantino, but who knows.
And these smart AI assistants will be lifelong partners that remember every single thing you
talked to them about. They remember and can recall your bloodwork from a decade ago and
have the knowledge of a good doctor to connect the dots of what happened since. They will be
a lifelong teacher who is infinitely patient and gets smarter and smarter every month. An
assistant that has access to almost the entirety of the knowledge base of humans.
And just imagine all the good we could do if we had this assistant right at our fingertips,
and all this for free, or for very little. What a time to be alive!
And yes, a closer look into Sora is coming in a few days. I can’t wait to show this to you!