That distinction between performance and learning that Bob mentioned, I think, is really important.
I mean, we like to think we know how to learn, how we learn best, in what situations we learn
best, and that we know when we've learned something. But as we saw in episode three,
we have no real insight into the basis of behavior. It's really difficult to predict
what we'll learn and whether we'll be able to retrieve this stuff in the future.
I mean, that's exactly right. But who better to provide an indication of how much we learn
than ourselves, right? Again, as you said, we saw this in episode three. We are not the
best people to determine whether we understand something or not.
We keep coming back, to this idea of fluency or ease of processing. System one is responding,
and system two is picking up on how easy it was for system one to do that. That's exactly
what's happening here with learning. If something is easy, if something goes down easily, we
misinterpret that ease, that simplicity, with knowledge, with comprehension, with understanding,
and that's a real problem. It's paradoxical in a sense because people use that information
to gauge how much they know about a particular concept.
I have a really good example of this. It was back quite a few years ago. I was teaching
an advanced statistics course for honors students, and it was compulsory. All the students were
required to take this course, a fourth-year honors level course called Advanced Multivariate
Statistics. Students were understandably very worried, very anxious about taking this difficult
topic, and it was difficult. I worked hard. I really worked hard to try to do everything
I could to make it easier for the students to understand this material. But at that time,
I didn't really—I hadn't really read much in this area, and so I didn't understand the
importance of retrieval and the importance of desirable difficulties.
At that time, I gave—I prepared this lecture. It was a very difficult lecture on eigenvalues
and eigenvectors, and I did everything I could to make it easier on the students. For example,
I represented it in a whole bunch of different ways, so you have—you can represent this
in matrix notation and algebraically, geometrically. I had diagrams. I was essentially pre-processing,
digesting all of this stuff for the students, so all they had to do is just sit back and kind of watch it unfold.
Now they did that, and I had one student who came up to me after the lecture. She said,
"Jason, you know, I completely understand. I get it. It feels so much better now that
you've presented it to me. I understand everything about eigenvectors and eigenvalues." I thought
to myself, "Well, there's no way. There's no way that you could've—I struggled!"
It took you weeks to put this stuff together.
Exactly. Preparing it, you have to sit down and work at it, do a few problems in and try
to explain it to yourself, and she didn't do that. Sure enough, when the exam came around,
she didn't do that well. She clearly didn't understand it nearly as well as she thought
she did. Yes, the same thing happens to me when I'm watching a documentary on a topic
that I don't know much about. When I'm watching a beautiful BBC documentary by Brian Cox on
the wonders of the universe, he's talking about a complex topic like entropy, and he
goes on and he's using these beautiful examples with sand and talking about time and how it
relates to basic physics and how we perceive time, it just—it looks fantastic. I'm like,
"Wow, I've never understood this so well." I guarantee, if somebody paused that TV show
and asked me one thing about entropy, I would not be able to tell them at all. I was fooled
by that feeling of fluency. It felt really good, but I actually didn't understand the
material as well as I thought.
That's really important: to have these checks. That's exactly what this sort of retrieval
does when you're actually forced to generate entropy. If you pause the film and said, "What
is entropy," and forced you to say, "Well, it's, uh—yes, I really have no idea exactly
what it is, but it went down easily," then I think we're getting closer.
There are a bunch of strategies that we can use, a bunch of these desirable difficulties.
Yes, they are hard, but—one is retrieval, retrieval practice. Another is spacing, putting it time and time again.
Another one that we didn't talk about is this idea of interleaving. What you can do, if
you're studying, study statistics for a bit, followed by biology, do a bit of English and
present these things in kind of an interleaved format, so you're not just studying statistics
in a five-hour block. You're kind of spreading it across a bit. You're getting spacing in
there as well, but it's this idea interleaving, of doing one thing for while and switching
to another task. That is also a desirable difficulty.
There are a few others that you can do to make it just a little—restructuring your
notes in a particular way, organizing it by theme instead of by topic. Textbook producers
do this all the time, right? They present it in exactly that the most logical format.
They highlight the important points for you. They're doing exactly what I did in my stats
course. They're pre-digesting all of this information for the students so they don't
have to—so they can just let it wash over them. They mistake that fluency with understanding, and that's a real problem.
Students in this course should probably be careful as well because, I mean, we're not
professionals but we tried as hard as we can to make these videos look as good as they
can. We tried really hard to make the online experience as best as it can possibly be.
But be careful not to be fooled by that feeling of fluency. Instead, pause the video. Spend
some time working, actively working, recalling this information and going through that struggle
to make sure that you really understand it and that you're really going to be able to
retrieve this stuff accurately for months and years to come.
Well, we've done a few things. In Think101, we're not just producing this stuff for people
to kind of absorb and then let it ...
Wash over them.
Have them wash over it, exactly. We're actually practicing what it is that we're preaching here.
So in the next segment, we're going to talk about exactly some of the things that we've
done to take advantage of these desirable difficulties to try to help the learners in
Think101 retain the content for a much longer period of time.