We are living in the age of AI where amazing smart assistants like ChatGPT
are popping up almost every day. They can do amazing things like being a
personalized teacher and helping you with assignments, homework,
they can write computer code, they can be your assistant for daily planning,
you name it. But Google DeepMind’s Gemini 1.5 Pro had something up the sleeve that others didn’t.
We are going to talk about that, and have a look at how you Fellow Scholars are already using it
for lifting weights, yes, lifting weights with an AI, and so much more. We will talk about what
the trouble is with this technique, and if there are solutions. And as I promised, we will look
at the paper too and find out how it learned a language that is almost impossible to learn.
Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
So, what is Gemini 1.5 Pro’s trick? The trick is that it remembers. What does that mean?
It means that it has something called a long context window,
which makes it remember long passages. It can read a whole book, and then,
we can talk to the book. It can look at a huge codebase and remember most if not all
of it. And finally, this context window is so long, you can even chuck a movie into it,
make a crude drawing of a scene, and it will tell you where exactly it was shown in the movie.
This truly feels like an AI assistant that came back from the future. You Fellow Scholars are
already using it for amazing things, like having it look at lectures and writing lecture notes,
ask it to look at your weightlifting session and summarizing all the exercises you’ve done
and get this, also the number of sets and reps you made too. Wow.
Or, it can look at your bookshelf and immediately give you a list of books that you have. And you
can give it prompts that are even a thousand times longer than this video. Goodness.
If you wish to ask a question about
a 1000-page legal correspondence and find an obscure detail, easy.
And it can recall often every single detail really. And hold on to your papers Fellow
Scholars, because it gets even better! That was just the 1 million token window,
whereas in the paper, they note that it can do up to 10 million. 10 movies
if you will. This is going to be a movie historian soon. So cool!
But it’s not all sunshine and happiness. We have a problem. What is the problem? Well, formally,
we say that the transformer neural network’s self-attention mechanism has a quadratic
computational and memory complexity. Okay, but what does all this mean? This means trouble. Big
trouble. Look. It takes a while. If you have one movie in there, it takes a minute. But,
if you would have 10 movies, that would take 10 minutes, right? That is reasonable. Yes,
that would be reasonable, but unfortunately, that is not the case.
The quadratic complexity means that the 10 times bigger query might take not 10 times longer,
but a 100 times longer! Yes, it may take up to 1.5 hours. Ouch! That is not practical. And
it gets even worse. This is a property that is inherent to the structure of transformer
neural networks. That is the heart of the majority of AI assistants out there. So,
we won’t get these amazing AI assistants from the future? Is everything lost?
Well, not so fast. Just based on the fact that Google DeepMind put
this out there in the wild for testing, a solution is hopefully possible and might
already be on the way. If it appears, I’ll be here to tell you all about it,
make sure to subscribe and hit the bell icon to not miss out on it.
Now, a few really cool tidbits from the paper that I loved. When we go from 1
million tokens to 10 million, let’s say from one movie to 10 movies. How does the
accuracy degrade? It cannot be a 100% accurate, right? Well, it’s not a 100%
accurate. It is only 99.7% accurate. Even the incredible GPT-4 Turbo cannot do that. Wow.
And here is another absolutely incredible insight from the paper. It learned Kalamang. If that
language does not sound familiar to you, don’t despair, it is an endangered language that has
fewer than 200 speakers across the world. And after being given a book about this language,
it learned how to translate to it at least as well as one of the active speakers.
My goodness! It can immortalize culture and languages. I am getting goosebumps.
Truly an AI assistant from the future. What a time to be alive!
And just imagine what we will be able to do just two more papers down the line!
Wait, what is this? This is not Gemini, this is Gemma. A small,
open model version of Gemini that you can try right now. Note that this is much smaller size
and context length, don’t expect the million token thing from it. Not even close. However,
it builds on a similar architecture to Gemini, and you can likely even run it
on the phone in your pocket. The link is in the video description.
So, what is the verdict on Gemini Pro 1.5? I have been lucky to have had access to it for
a while now, and the fact that it remembers so much is incredible. That is the good part. Now,
issues and limitations. You heard about the quadratic complexity, that is one. Two,
if we are using the long context window to ask one thing, that is one needle in a big haystack,
where it performs nearly perfectly. But, if we are looking for 50 or a 100 needles,
we see a bit of a degradation in its accuracy. Still quite good, though. Three, in my personal
opinion, for my crazy calculations that I like to do, GPT-4 Turbo still has more horsepower,
and Claude seems a great deal better at coding. Interesting how these major large
language models seem to be so differentiated. Each of them are good at different things so
take that into consideration whenever you try them. But I hope this gives you the warm and
tingly feeling that this is an incredible world that we live in today. Goodness. I love my job.