>>presenter: Good afternoon, everyone. Let's get started. It's an honor introducing, hosting
Khan Academy today, here at Google. We have three wonderful guests here. Sal, who's the
founder, and Shantanu, who is the President and COO, and Marcia, who is a software engineer
of Khan Academy. I learned about Khan Academy about a year ago. I wasn't sure how, just,
it also comes with a lot of comments, also like people appreciating. I wasn't sure just
how this video is going to be theoretical as far as personal education is concerned.
So I tried it myself and I guess the first thing that I noticed was that it almost seems
that he was having more fun teaching me than I was having fun listening to him. [audience
laughter] He is so passionate about teaching. And I intentionally chose a non-technical
topic, which I always hated, like history. And I watched through all the videos and it
was just like glass and ice. At a very certain level, I realized why it works. With that,
I will actually give it to Chris Uhlik. He will formally introduce Sal. He has been working
with Khan Academy over the past year and he has actually helped them get a $50,000 grant
from Microsoft Research and he also helped them get two million dollars from Google,
as part of the 10 to the 100th Project. So, with that, I'll give it to Chris.
[applause]
>>Chris Uhlik: Hey, so, I have four children, ages eight through fifteen, and we homeschool
them. My wife is very interested in education technology. She has a degree in education
technology and it's illegal to experiment on humans, so she made her own.
[laughter]
And we've been basically educating our kids using 100 percent online materials. Every
kid's got a computer; they spend all day at the computers and then they go off to physical
education classes outside the home and stuff like that. So, she's a great student of what
kind of education technologies are out there in the world. We buy everything; she researches
everything well. Just over two years ago, she sent me a link to this Khan Academy. She
said, "Hey, check this out. This one's pretty cool and itís free." And I started watching
a few videos. I watched the banking crisis explanation videos. And I was like, "This
stuff is really, really good." And I had been thinking about what are a few of the really
big, high impact projects we can do in the world, like fixing carbon dioxide emissions,
educating the people in the world. These are some of the huge problems where you can really
totally transform the future. And I firmly believe that doing this for free, so that
everybody in the world has access to education materials, by taking advantage of the leverage
that technology offers, the ability to take one lecture given by one guy and then have
a 100,000 people watch it over the next few months. And Sal's done 18 hundred little capsule
lectures and they have a typical number of views--around a hundred thousand each--40
thousand here, a million there, give or take. So, he's reaching, right now, about two million
people a day.
>>Salman Khan: A month, unique a month.
>>Chris: Unique a month. The University of California has .2 million, right? So, he's
reaching ten times the University of California with his own efforts, a few people helping
him, and a free video hosting service, right? So that's the power of technologies like YouTube
and AppEngine to let people build incredibly impactful, valuable things for free. So, a
year ago, I was working on this education project. It wasn't Microsoft Research that
gave him the 50K, by the way. It was Google Research.
[laughter]
And I also helped the Google 10 to the 100 team decide and figure out how to give them
one fifth of the total prize, which is two million dollars. And I think he's gotten some
significant backing from the Gates Foundation and from John and Ann Doerr, and a few others.
So, he's able now to really commit full-time and actually hire people and expand and he's
getting some serious publicity help. That publicity has caused him to grow a factor
of 50 in the past year. He was getting something like, 40,000 views a month a year ago; he's
getting two million a month right now. And that's just incredible and it's probably going
to continue accelerating at that kind of pace for a while. You can look forward to most
people on the planet having seen a Khan Academy video in the next couple of years. That's
impact. So, we're gonna start the talk. There's a Dory page at go/khanacademy. There's gonna
be a lot of kind of question and answer period at the end. There's a microphone in the middle.
Go up and ask your questions at the end. Don't interject and I'd like to introduce the Linus
Torvalds of education.
>> Salman Khan: [laughs]
[audience laughter]
He's gonna transform the operating system of the education future. Salman Khan.
[laughter]
>>Salman Khan: Thank you, thank you.
[applause]
Actually ñ I don't want to be -- how many of you all saw the Ted talk that came out
a week or so ago? Ok, I don't wanna bore y'all, but some of y'all haven't seen it, but there
will be a little bit of an overlap. And since you don't know the structure of this, I'm
gonna do a quick overview and feel free to interrupt me at any time, or ask a question.
And then we'll just open up to Q&A cause that'll be more fun. We'll learn things. But just
to start off, as Chris mentioned, Khan Academy started off me making videos. So, I'll show
yíall a little bit of a video montage to see a feel for what the videos are like, and
we'll show you what else we're up to.
[pause]
[video plays]
>>Salman Khan:(speaking on video) So, the hypotenuse is going to be five. This animal's
fossils are only found in this area of South America; nice, clean band here and this part
of Africa. We can integrate over the surface, and the notation usually is a capitol sigma.
National Assembly that create The Committee of Public Safety, which sounds like a very
nice committee. Notice, this is an aldehyde and it's an alcohol. Start differentiating
into effector and memory cells. A galaxy. Hey, there's another galaxy. Oh, look, there's
another galaxy. And for dollars is their 30 million plus the 20 million dollars from the
American manufacturer. If this does not blow your mind, then you have no emotion.
[laughter]
[end video clip]
>>Salman Khan: So, that's a feel for what we're doing. And as Chris mentioned, we are
now -- there's now on the order of 2200 videos. I made five this morning. And they're--
[laughter]
ten minutes, you know. And they now cover everything from basic arithmetic all the way
to vector calculus and the French Revolution, and all the rest. And we're reaching -- actually
a month ago, we were reaching a million unique students a month. Now, weíre reaching two
million unique students a month just with the latest buzz. So that's kind of where we
are now, but there's a lot more that we're up to. Yíall have very generously given us
two million dollars, so I view this as a progress report.
[laughter]
But before I go into that, I'll talk a little bit about how I got started. And as yíall
know, I was a hedge fund analyst five years ago in Boston. Then, we moved the firm out
here in Northern California. It was actually a two-person hedge fund and my boss -- his
wife became a professor at Stanford, so he moved to Palo Alto. And I was tutoring my
cousins remotely in New Orleans and started working with one cousin, then another cousin,
then another cousin. Before I knew it, I had this cohort of cousins all over the country
and I was looking for a way to scale myself up. And one of my buddies [Sulfaka Ramzana]--I
should give him credit--I was literally hanging out at his house, showing him how I was tutoring
my cousins and all that, and he's like, "You know, why don't you put some of your lectures
on YouTube?" And I said, "Oh, that's silly. YouTube's for dogs on skateboards. It's not
for--
[laughter]
It's not for a serious learning. But once I got over the idea that it wasn't my idea,
I--
[laughter]
said, "I'll take a shot at it." And I remember the first video, I think, and you can go there,
literally, it's like November 6th, 2006. It was greatest common divisor, least common
divisor. One of those. I wanted to teach my cousins fractions. I was like, "Oh what do
they have to know? And the negative numbers." And I put in like, 20 or 30 videos and this
has turned into a bit of a one-liner, but it's true. The very first feedback my cousins
gave me were that they preferred me on YouTube than in person. And so, I felt like this was
something to do. So I kept making the videos. I started getting feedback from people all
over the country, saying how it helped them. "Hey, this helped me on my exam. I passed
the exam." But some of them were like, "Hey, I was gonna drop out of high school until
these videos". Or "This motivates me to go to college and become an engineer." Or "This
is the only reason why I can, now that I'm retiring from the military, I can feel comfortable
going back to the community college." So, I was excited. So I kept going. And along
the way, the site grew and then the other thing that happened--actually this is what
most people don't realize-- sometimes, I switch around the story because it sounds better
when I started the videos. But I actually started on the software side. If you rewind
before I even wrote that first piece ñ I recorded that first video, when I was tutoring
my cousins, I would just point them to random websites. I was like, "Hey, I just found some
website run by this university and there's ten good problems on fractions. Why don't
you do those problems and we'll go over them tomorrow?" And the next day, I'd say, "Hey,
now you do the problems?" She'd be like, "Yeah, yeah, I did them." I was like, "How many did
you get right?" She said, "Yeah, I think I got them all right." I was like, "Oh, when
did you do the problems?" "Oh, yeah, I did 'em like at night."
[laughter]
It wasn't that ñ it wasn't that informative. So I was like, "Oh, I'm gonna write my own
problems." And then she would run out of problems. There's these little worksheets that you see
all over the web, like ten problems and then you're done. So I wrote these really primitive
JavaScript generate problems, as many as you want, in adding fractions, or adding negative
numbers, or multiplying fractions, or whatever. And so that was -- the premise was, "I'll
give you as many problems as you need until you get ten in a row" and I'll show you what
they look like in a second, "and you'll get hints for them." And before I even made the
first video, I thought that was the solution. That was like, "this is all someone needs
to do math." Because on the exercises you have hints and all of that. But then, once
the videos took off, it was just a better use of my time to make five videos a day rather
than one module every five days. So, that kind of ñ it got orphaned a little bit. And
actually, one thing that happened is, I left it out there and I had this 50 dollar a month
webhosting and at some point, there were too many people using that software, so I just
turned it off -- which is probably a bad idea if you're ever starting a business. [laughter]
It's probably a signal that you shouldn't. Maybe spend a hundred dollars on your webhosting.
So when the opportunity -- once Khan Academy started growing, we had this viewership and
it seemed like there was an organization that we could start here. The question was like,
"How do you take what we're doing to the next level?" And that's what ñ [pause] And that's
what I wanna show you right here. So, these are the exercises and what I started with
my cousin was a much more primitive version, but this is the same, actually some of the
same basic code. It's been fancied up a good bit now. But the general principle is, it'll
give you as many--this is subtraction one--it'll give you as many problems as you need until
you get ten in a row. And it's a very simple--it got cut off here-- but you can have the videos
here, there's hints, you can see it draws a number line for you. The Khan Academy videos
can be pumped in, and itís a very simple idea; you do it until you get ten in a row,
but itís--at least in our minds--completely different than what happens right now in a
traditional school. In a traditional school, lecture, homework, lecture, homework, lecture,
homework, lecture, homework, then snapshot exam. And regardless of whether you get 80
percent, 90 percent, or 95 percent, the whole class moves on. Even if you fail an exam,
actually, the whole class will move to the next topic and what was that five, twenty,
or thirty percent you didn't know? It was probably something that you actually probably
need for the next topic, or definitely something you'll need in a few years. And there's no
-- people just ignore it. They place a label on your head; that some kind of value judgment
on whether you're smart or not and then everyone just proceeds down with those gaps. And so,
what we're saying is, "No, we're gonna have them do the opposite." Instead on penalizing
you for failure and not expecting mastery, we wanna do the opposite. We wanna allow you
to experiment. If you're learning ñ if you're learning to ride a bicycle, you would just
sit on that bicycle as long as it takes to actually learn how to ride the bicycle. And
so, that's what we wanna do here. We wanna allow you to experiment, we wanna allow you
to fail, but you're not going to move on to more advanced topics until you actually, until
you actually master the topic. So this is -- that is the subtraction one module. This,
right here, is trigonometry. [pause] This, right here, is shifting and reflecting functions.
And then this is how all of the exercises are structured. So literally, this node right
up there is single-digit addition. It's literally five plus seven, or one plus one. And once
you get ten in a row there, it'll move you up to one digit and a double digit addition,
and then single digit subtraction and then we call it very basic multiplication. Once
you get ten in a row there, it keeps moving you down what we call this "knowledge map"
and if you keep going down the knowledge map, you start getting into some more advanced
arithmetic; some pre-algebra here. Go further down and it starts getting into algebra, a
little bit of trigonometry, geometry, and precalculus. And so, the general idea is that
we want this graph to eventually cover everything. Actually, the funding that yíall have given
us, roughly half the funding is to translate the videos into ten languages, but the other
half of the funding is to build out this graph structure to cover on the order of about two
hundred or three hundred modules. So, literally is all of K through calculus mathematics.
But the goal, and it's already pretty good coverage for K through nine, or K through
ten mathematics, but the goal is right now, it's about 107 modules. The K through calculus
is probably gonna be on the order of about 300 modules; totally very comprehensive. But
there's no reason why you can't expand from this into logic, into computer programming,
into genetics, into probability, into finance, into accounting, into grammar and logic. You
can just keep building off of this knowledge map, so it really covers everything that can
be quizzed in this type of a form factor. [pause] And so, this is just some more of
it right over here. And so the paradigm that we're doing -- when I started this, I assumed
it would be a kind of a nice to have -- a supplement for people. Even when we started,
when we formally became an organization, we didn't assume that it would be adopted in
schools. But then, one thing that happened was that teachers started adopting it on their
own; just the videos. We started getting letters from teachers saying, "Hey, you've already
given the lecture, so we're using those to flip the classroom. So instead of me giving
the lecture ñ me the teacher -- , I'm assigning your lectures as homework. And then what used
to be homework, I'm having the students do in the class." And it's a very simple concept,
but it really changes what a class is all about then. Now, all of a sudden, instead
of you literally have 30 people completely silent and completely passive, most of them
zoned out, a teacher having to give this one size fits all lecture, even a great teacher,
they're losing probably two-thirds of the class, now that happens at home. You don't
have to be embarrassed to rewind and look at something that you might have missed, or
fast-forward if you're bored, or pause something ten times; you don't have to interrupt the
whole class. And now, when you actually go to class, you actually have all of your peers,
you actually have the teacher to actually help you out. And it's interesting. One thing,
and I had mentioned this is in the Ted Talk, "that'll work for motivated students, but
what about the students who aren't going to do that?" And I was like, "Well, if you're
not gonna do anything at home, period, it's still better that you're doing the exercises
in the classroom, because that's, frankly, where you're gonna get most of the learning
in and if you didn't do it, watch the video in the classroom, too. And so the paradigm
is, where you really learn stuff and where you're really getting your head around something,
you want other people to be around you. When you're actually trying to solve the problem.
But when you're trying to listen to a lecture, you don't want people around you. You don't
want your peer to say, "Oh my God, look how stupid Sal is. I didn't have to pause that.
He's reviewing stuff from 30 -- " You don't want that around. So you want the lecture
to be intimate, but you want the actual classroom experience to be social. So what we're doing
in classrooms now is taking that to the next level. Los Altos, right here, they came to
us, they came to us actually about four months ago. Shantanu and myself, we had a meeting
with their school board and said, "If you just had carte blanche in a classroom, how
would you run the classroom?" And we said, "Well, we would let every student work at
their own pace on those exercises on that knowledge map we just showed you, and the
teacher would walk in every day and just get a dashboard that looks something like this."
And this is actually the live dashboard now from Los Altos, just so you know how the story
ends. "And the teacher would let -- everyone would work at their own pace and only intervene
on the kids who were stuck." And they said, "Oh, that's a very radical approach." But
then, the next day they said, "Oh, we'll do it." So, Los Altos, in November, they started
with two fifth grade classrooms and two seventh grade classrooms, and this became ñ I would
say this became 50 to 60 percent of their math curriculum, with the other forty percent
being project-based learning. So they completely gutted the traditional part of the curriculum;
no more lectures, no more traditional homework, no more of the lecture-homework-lecture-homework
cycle. Now, the students walk into class, the teacher looks at a dashboard like this.
Each row, here, is a student. I've blanked them out for privacy. Each column, here, is
one of those concepts you saw on that knowledge map, and green means the students already
got ten in a row, blue means that they're working on it, but no need to worry, and red
means that the student's stuck. So there could be a kid working on third grade math in a
fifth grade classroom, and there is. That's cool, as long as he or she isn't stuck. There
could be a kid working on trigonometry in the fifth grade classroom, and that actually
is happening. And it's actually entertaining because a reporter who asked the girl, she
was literally doing trigonometry and she's a fifth grader and she's like, "Do you think
this is fifth grade math?" And she's like, "No. I think it's sixth grade."
[laughter]
But that's cool. They should both be working on those respective concepts because that's
where they are. But, the teacher just works on whoever is stuck, or even better, get one
of the kids who are already proficient in that module, who've already gotten ten in
a row, let them be the first line of attack and actually tutor their peers. And we all
know when you actually tutor someone, there's a whole other level of learning that's going
to happen when you do that. And the teacher only intervened -- and so, every moment of
a teacher's time is actually spent on being a human; being ñ kind of interacting with
the classroom. I've mentioned this in other talks is that whenever people talk about technology,
they assume its like, "OK, it's probably good for economics. It's probably good to get more
bang for your buck, but it's probably gonna dehumanize the classroom." And the one thing
we keep saying is, "No, itís the complete opposite. When you do something like this
in a traditional classroom, only about five percent of a teacher's time is actually being
a human being, actually sitting down next to someone and tutoring them, mentoring them,
or working with them. Now, a hundred percent of their time is." So, when you're using technology,
you're actually humanizing the classroom by, if you believe those numbers and I actually
think the five percent was generous, by a factor of 20. And so, we don't want, even
when I worked with my cousin, Nadia, and even when I had the data on ---, I would ask her,
"Did you get this? Did you understand this?" And she's like, "Oh, yeah, I got it. I got
it." There's these very painful discussions that you have if you ever tried tutoring anybody
on what they do get and what they don't get. And so, we wanted to avoid the teacher having
to do those really painful, awkward questions. And so, we're trying to arm the teachers with
as much data as possible. So this is like ñ this is one of the views. A student can
view this about him or herself, or a teacher or parent can see this about any of their
students. And this is just one of the reports; we have a bunch of them. But this is literally
just the activity for the student over the last 30 days. You could see it over the last
day, you could see it by hour, you can see it over the last year. But each of these bars,
these are on a certain day. The light blue is the time spent on exercises. The navy blue
is the time spent on videos. These are the highest achievements achieved that day. And
if the student or the teacher, they scroll over any of this, they actually see exactly
the student spent 38 minutes on exercises; they got that many energy points. These are
their achievements. The videos would be there if they watched videos that day, and we're
actually even tracking when they pause the videos, when do they repeat the videos. We're
gonna start using that. One, for the teacher to know what happened, but also even for us
to know how can we, how can we improve our videos. [pause] One report that teachers ask
us is -- they said, "Look, it looks like some of the kids are really focused and they're
moving ahead, while some kids are jumping around a lot. Can we have a report on focus?"
And we said, "Sure." So, this is, this is the focus report. The outer circle is literally
a pie chart of what exercises students spent time on. This inner circle is where they spent
on videos. And, once again, if you scroll over any of it, you get more detail on exactly
what the student was up to. [pause] This right here just shows how granular the data is.
This is a student who is doing the probability one module. So, a teacher can go and say,
"OK, this is the first problem; they got it right. That's why it's blue. They probably
guessed because they spent like, four seconds on it. And they watched the video. Then they
got the next one wrong, a couple right, maybe guessed. Wrong, wrong. It looks like whenever
they actually made an effort on the problem, they got it wrong and then they guessed a
few right.
[laughter]
Then they used a hint. And if you click, if a teacher actually, if a student, later, when
they see this on their report and they click on any of these, they see the exact problem
that they didn't get right. And then, you can eventually see-- and this was so funny--it
got clipped off here, but you can see the student eventually getting ten in a row. And
it's fun; you can almost see them thinking. There's some deep thought going on the 99th
problem, where the student said, "No, I'm gonna get my streak now." And so, he or she
literally got this problem right, and then you can actually see that they get faster
and faster and then they get ten in row. So, you can actually see the learning process.
[pause] And most people ñ when you say "self-paced learning" sounds like a good idea, but the
one thing that we're seeing over and over again. And this goes back to ñ I -- there's
literally in one of the fifth grade classrooms, in Los Altos, there's a few kids doing third
grade math and there's a few kids doing even calculus. And there's a huge temptation when
you see that. "Oh. Gifted kids, remedial kids. But it's cool. They can be in the same classroom,
but those kids are gifted, those kids are remedial. Those kids are not gonna be at Google,
these kids are. I mean that's the -- we all do that. When you see that, you just think
that. We're so used to labeling people. But one thing we're seeing over and over again,
we've seen it at Los Altos, we're seeing it in other summer camps we've run, is that some
subset of those kids--five days into it--there's this spread. This is just days into the pilot.
This is the modules completed. Some days into it, there's a group that clusters here and
you can see that group here and there's a group that clusters here. These are the gifted
students in a traditional system; these would be the slow students. But you always see these
students, once they spend extra time on some module, and they get some core weakness out
of the way, and you don't know who they are ñ you really don't ñ you have no idea; they
just rocket forward. Everything else just starts to click and so, in a matter of six
weeks, we've seen this flipping multiple times of the students that you used to think were
slow or remedial; they just had some basic core weakness. And once you actually give
them a change to get it out of the way, they just rocket ahead. I mean, this kid right
here, literally, I think he's our number one or number two kid in the class, and starting
off, he was one of the worst. So you see this over and over and over again. [pause] So that's
the general idea of what we're up to. We can address a lot more of it in the Q&A, but our
goal is literally to be a free classroom for the world; give a world-class education to
anyone, anywhere. And I want to be careful. We don't think we're doing that yet. We think
we're solving parts of the problem. We have the videos, we're doing the exercises, we're
doing the data, we're doing the analytics, we can kind of become a kind of operating
system for a classroom, but we think there's a lot more and that's why itís fun to talk
to yíall because to some degree, we can even brainstorm on what that would mean. But, I
think it's completely possible that you can have a reality where, in five or ten years,
ideally you will have a classroom where you will have a human being actually helping you
out. But there's a reality where even if you don't have that, you can get a pretty darn
good education. So I'll open it up to questions. Anyone who wants to walk up to the microphone,
and if yíall, I guess there's that website where people have ranked questions. [pause]
I could ñ I'll take this one.
>>audience #1: Hi. Thank you for coming. I wanted to ask a question about if the model
for the future is gonna be one like Wikipedia, where you're gonna open up the creation of
the content, the videos, and the exercises? And if so, how would you maintain the quality
in those types of things?
>>Salman Khan: Yeah. That's a great question. The question is, will we open it up to kind
of be a Wikipedia of education as opposed to -- and so, we're open to opening it up
somewhat, but I think for video-based lectures, if we just look at the video side, I think
the reason why I think that kind of crowd sourcing doesn't work is because it's one,
your quality is gonna be all over the place. And even if the quality is good, it would
actually be very jarring. If I came and give you -- imagine if you're in a math class and
I came and gave the first lecture, and then the next day you have another good lecturer,
but you're like, "Who is this guy?" And then the next day you have another lecturer. So
there's actually this huge level of trust that you have to build with the lecturer for
you to be able to invest in more and more watching. So, that's why we resist really
opening it up. We do suspect there's a whole set of things that other people know that
I could never know, or that would teach better than me. Or maybe they'd teach the same thing
that I'm teaching, but in a different way that responds to other people. And so we are
open to people. We're telling people, "Hey, make 30 videos, 40 videos, 50 videos on a
certain topic." One, you should do it on your own, cause you don't need us to broadcast
them, but if it really does feel the same, it's consistent with our brand, we'd be open
to it.
>>audience #1: Thank you.
>>Salman Khan: Yeah. I'll take--
>>presenter: [ ]
>>Salman Khan: Top down. OK. I'll read this first one. "With such busy schedules and varying
priorities, how are parents/students supposed to identify online learning centers like Khan
Academy that are legitimate academic institutes and not just money-making schemes created
to take advantage of the online trend?" One, we're not-for-profit, so we're definitely
not a money-making scheme, unless I got particularly creative with the--
[laughter]
But we're not a money-making scheme. And actually, that was, that was one of the main reasons
for being a not-for-profit. I mean, there's a lot of people who talk about double bottom
line organizations which is an oxymoron. Bottom line is meant to be literally the bottom line.
Like that's the thing that you care about. And you can't say I care about two things
that are usually in opposition with each other. So, if I was genuine, and I hope I am, about
-- no, the number one goal is not to become rich, not to do whatever. The number one goal
is to provide an education for people; make it a not-for-profit. And there's other realities
that it actually has a better chance of becoming a lasting institution then. If it's for-profit,
it'll be acquired by McGraw Hill at some point and I'd buy a nice house, but that would be
the end of it. But here, there actually is a potential to last potentially beyond me.
I'll take a question.
>>audience #2: Yes. So one of the problems I have is -- we run a training for our Google
partners -- it's technical training. We have a classroom-based and it's like your model.
We focus on getting the learning exercises; walking around, helping them, get through
them covering the theory. And we have another group of you who are ñ that can't come here
for various reasons.
>>Salman Khan: Yeah. Yeah.
>>audience #2: I have problems getting them to actually do the homework because that's
where the learning, watch the tutorials, but the homework pieces seems kind of ñ it drops
off. People are not that interested. What kinds of things can you recommend?
>>Salman Khan: You know ñ I think ñ so -- to get people to do the homework, I mean -- the
best way is just to track it. I mean, that gives you information on whether they are
or aren't doing the homework.
>>audience #2: They're not submitting it, so I know--
>>Salman Khan: Yeah, yeah. So, they're not submitting it, so they're not doing it.
>>audience #2: And we have some carrots. We have certifications and that's a big carrot
that we're seeing people completing the content. But are there other, because people ñ there
are competing events they have in their life.
>>Salman Khan: I think -- it's interesting we had the gut sense that gamefying a lot
of it, giving badges and points, and actually I can even show you we have some of that over
here. We have some of these ñ we have some of these badges right over here. So, gamefying,
we thought it would be kind of a nice to have, like it would make it kind of cool, but itís
amazing what's itís done. Oh -- it's not on the screen now.
>>audience #2: Have you seen a lot more people completing the content with the badges?
>>Salman Khan: Yeah. It's great just changing the amount of points you give for certain
achievements or the wording has actually changed like thousands of people's behaviors overnight.
Oh, yeah. There you go. Those are the badges. So we just have a ton of these badges that
we've set up and I was saying earlier at lunch, in Los Altos, these are fifth graders. So,
lemme show you there's a black hole badge that right now, the black hole badge is -- we
have it defined. It's the most legendary badge on the Khan Academy. It's actually impossible
to get right now.
[laughter]
Because we said no. That has to be -- once we have three hundred modules, then you can
only, if you get like, two hundred, we don't even have that, so itís actually impossible
to get now. But the fifth graders became so obsessed with getting--fifth gradersó with
getting a black hole badge that some subset of them went into our code, so itís an OpenSource
project,
[laughter]
went into our code and figured out they had to get 150, they told us, "Oh, we figured
out we have to get 150," and I'm like, "How do you?" "The code."
[laughter]
So, itís amazing what, I mean, yíall should be giving offers to those kids. I don't know
what they're --
[laughter]
But itís amazing what this type of thing does to motivate, especially kids. I don't
know if it'll do it for employees.
[laughter]
>>audience #2: Thank you.
>>Salman Khan: Yeah. Those, exactly. And I'll take one from the -- I'll just read it. Where's
the? [pause] Oh, there's not a, oh, these are always the same. OK. "So you tend to focus
on math, science, and economic concepts. What do you think it would take to make this format
work for grammar, writing, or composition?" So grammar, I think, is one that's very doable.
I've tried to contact people who have good--I won't name the companies--but there's a company
out there that I think has a really good, you know -- when I studied for the GMAT, I
used their book, and I was like, "I finally learned grammar."
[laughter]
And it might not be obvious, hearing me speak now, but I just wanna find good materials.
But, I think, grammar actually is perfect for both the videos and for the exercises,
because it is actually logical and very mathematical to a large degree. And you can diagram things
out. I think, one way or the other, writing or composition, those will be interesting.
I think writing and composition are much more activity-based. You can't just have someone
give you a lecture on what it means. Although, it would be probably ñ it would be interesting
to have someone give a lecture. Maybe itís me, maybe it's someone else. On what -- like,
this is well-written, this is badly written, and why it is. But the more useful thing is
to actually to write things and maybe get some type of peer review. So, it probably
won't fit directly into that knowledge map structure, but I think there probably, there
is something we could do to leverage the two million students using Khan Academy right
now, where you write an essay -- maybe we have essay competitions, or people write essays
and then they peer-review it and the best ones surface. I think there's something that
we could do, but we don't -- we haven't figured that out yet. I'll take one right here.
>>audience #3: So I'm a huge fan. I actually watched most of your linear algebra lectures
over the winter break, and I've been trying to tell everyone I know--
>>Salman Khan: You deserve a badge.
[laughter]
>>audience #3: I thought about it; different things to get badges and showing them to everyone.
So I've been telling everyone I know about Khan Academy, but I was brainstorming with
other engineers here about how we can do -- what can we do to help? As you mentioned, other
people giving lectures isn't necessarily helpful because they might not be good lectures. So,
what can other individuals, or Google, do to accelerate this process other than just
giving money?
>>Salman Khan: Yeah. Oh you could give more money.
[laughter}
>>audience #3: Well, I mean--
>>Salman Khan: No, no, actually, genuinely, geniuinely and you're right. I mean, we view
the ñ actually the videos are kind of what get people into our fold, but then the real
value is the exercises and the other. And I think, eventually, the communities are gonna
be super valuable. So, I don't know the policies of volunteering code for other projects and
things, but those modules, they're very stand-alone. We're speccing them out and if you say, "Hey!
I could write a cool or I could write a whole set of modules on grammar." We'd love that.
Work with us; we're literally based on Castro Street, so we're not too far away. But if
there's more sophisticated things, people at Google, even outside of the funding, have
helped us move forward with some of the translations and things like that. So, there's interesting
things out there, AdGoogle, either products that could directly benefit some of what I've
talked about, or you're like, "Hey, I think it'd be fun to crank out a couple of those
modules or have some neat ideas for them", we'd love to have your help and we are hiring,
if any of yíall are looking for a sabbatical.
[laughter]
I don't know what the policy is on things like that. I'll --
>>audience #3: I had one more quick question, 'cause--
>>Salman Khan: Oh yeah, yeah, sure.
>>audience #3: How much planning do you do for your videos?
>>Salman Khan: How much what?
>>audience #3: How much pre-planning do you do?
>>Salman Khan: How much pre-planning do I do before the videos? So, if I'm doing like
an algebra worked example, I literally will like, put it up there and just do the problem.
Like the first -- when you see me do it, it's usually the first time I'm looking at the
problem, unless something disastrous happens. If I'm doing, like I did some videos recently
on China. Actually, there's one of the ones on the montage on how China pegs its currency
to the US dollar; that one is something I'm kind of familiar with from my hedge fund life,
so on my walk to work, I'll think about it and then when I sit down I'll just do it.
So, if I'm doing something that I'm really not that familiar with, where I really have
to get myself re-immersed, like organic chemistry, that I'll spend like two weeks literally just
immersing myself in the field, and getting the ñ kind of living and breathing organic
chemistry. And then I'll make the video. I tell people that I try to prepare my brain,
but I don't try to script the lectures, because then they come off unnatural, or scripted.
[laughs] I'll alternate. How can we be sure that great programs like this don't widen
the gap between the haves and have nots? Those who need the most help are often unlikely
to have a home computer or a high speed connection. So, that's actually a really good point, and
I suspect, it's not something that I'd like to admit, but I suspect that right now we
maybe are; that right now it is people, I think Khan Academy's penetration is much higher
in Google households than it is in random households across the country. And I think
itís definitely much higher than households where there isn't a parent with a college
education. So I think right now we are, probably in the short term, unfortunately doing that.
I think over the long term, though, what it does is it--right now when everyone says,
"Oh, we should get one laptop per child or we should put fiber to this school or that
school." There's not a tangible reason to do it. Like people -- there's a feel-good
reason that maybe it'll help. It'll probably help kids have the access, but I think once
something like a Khan Academy exists and people see that ñ we better get our act together
-- this will actually be the catalyst for getting fiber into the classroom, the catalyst
for getting laptops in front of kids. Then it'll actually close the gap because the delivery
cost is actually zero. One thing that we're saying is -- Los Altos is not a needy neighborhood,
but they were quick to adopt it because they're flexible and we were really impressed with
how they run. There's some private schools that want to adopt it really fast and there's
no bureaucracy or nothing, so they're adopting it. But what we suspect is going to happen
is they're setting the best practice and then, hopefully, around the rest of the country
or around the world, all sorts of upper middle class, middle class, lower middle class, will
say, "Wait, if they're doing that at Los Altos, or they're doing it at that fancy private
school, how come our kids are not doing it? And on top of everything, it's free. So, there's
no reason not to do it." So I think in the near term, there might be a little bit of
that, but hopefully over the long term, it actually serves as a catalyst to close, to
actually motivate the technology gap.
>>audience #4: Hi. My name is Anne, and I should admit that both of my parents are math
teachers and I was a math teacher before I came to Google. So, it's really exciting to
see, like, I've certainly seen a lot of times with my family being skeptical of various
ways of technology trying to replace, but it's really exciting because it's very much
correlated with the types of challenges. And my mom is especially into self-paced learning,
which is really time intensive and people-intensive so it's a really amazing way to let that happen
and let the teachers focus on the most important place where they can add the most value. I
was wondering how you've seen teachers incorporate this in terms of when they want to do deeper
problem solving, or group work, and how does that, how does that classroom experience integrate
and --
>>Salman Khan: Yeah, so the question is about how does this integrate with what teachers
are trying to do with a deeper experience and we haven't, I get a lot of anecdotal stuff
on emails and stuff and I said, even before we started doing this pilot with Los Altos,
there were teachers who are flipping the classroom, lectures at home, or home and in the classroom,
and then the teacher can work with them. And what we're seeing in Los Altos is, and even
those teachers, is because you're freeing up all of this time where this lecture doesn't
have to happen in the classroom, it depends on the teacher, but it frees up a ton of time
so they can do the project-based learning. There's so much debate: project-based learning
versus the traditional doing a lot of exercises and seeing a lot of problems is learning.
And what's cool about this is you can do both. They're both valuable. It's actually not an
either/or proposition. And even a lot of people said, "Do you teach new math or old math?"
I do both. You know, why not? There's no reason ñ there's no limitation for our classroom,
so you can actually do every possible way of looking at something.
>>audience #4: Awesome. Thanks again. It's really amazing.
>>Salman Khan: Yeah. Oh, thanks, thanks. I'll take one here. Students are now truly--I'm
doing it like the videos, I don't know how bad these questions are gonna be ñ they might
put me on the spot--
[laughter]
"Students are now truly empowered to work through course material at their own pace.
How do you create meaningful cohort groupings in this context? Age--as opposed to skill--seems
to be inefficient from the perspective of maximizing learning." So, I completely agree
with that. So this, even in the Los Altos classroom, you have--it's a fifth grade classroom;
they're all ten or eleven years old--and there you have that mix. Our idea would actually
be a classroom of all age groups. Literally, in the same room, you have fifth graders,
or you have five year-olds and you have 18 year-olds, and there's no kind of preconceived
notion that if you're 13 and you're learning this, you're stupid. It's just the pace that
everyone learns at. And what you get there, I actually think everyone in the mix will
actually act a lot more mature when they're around older and younger students. And then
you get the true benefit of the learning across generations. So, I completely ñ I completely
agree with that; that it's a good comment, I guess.
[audience member claps]
Yeah, yeah, yeah, yeah. I'll take--
>>audience #5: [ ]
>>Salman Khan: Oh, very good. Thank you. Go ahead.
>>audience #6: Great. After all these very wonderful questions, I have a really pragmatic
one.
>>Salman Khan: Yeah, yeah.
>>audience #6: Which is, your lectures, for example, on the French Revolution and the
California High School Exit Exam, use imagery and problems from other sources. How do you
dance around copyright?
>>Salman Khan: How do I dance around copyright? So I have a very loose interpretation of "fair
use".
[laughter]
No, no, no. Well, that's actually true. But the -- so, all of this, like the French Revolution
stuff, I do look on Wikipedia, I look for stuff that's in the public domain. Most of
those paintings and stuff are in the public domain. The copyrights expired. Stuff like
the California or the Massachusettsís Test for Teacher Licensing, I mean, these tests
are written by government bodies and I assume they want teachers to pass. And so, I've taken--on
things like that, if it's a government sponsored-- I also have stuff for like the IIT Joint Entrance
Exam, I have some of their problems. These are issued by the Indian government. I didn't
ask their permission. But I assume, I mean, they can sue me. It would be an interesting
PR issue for them.
[laughter]
But I ñ I ñ I ñ one -- I actually do think it falls under fair use, but even -- I would
be very surprised if they were upset. I wouldn't take something from McGraw Hill and do it,
is a simple answer, or I'd ask their permission first.
>>audience #6: Thanks.
>>Salman Khan: And I am careful about the Wikipedia creative commons and all that other
stuff, too. Yeah. Simple. So, next one." What tools and techniques are you using to interest
students who are traditionally not self-motivated or those requiring a different learning style
than a standard video provides?" So, traditionally not self-motivated. Actually, the gaming is
a huge thing for the traditionally not self-motivated. And actually, I think once -- a lot of people
are just not motivated because they have these gaps in their knowledge that the traditional
model can't address and people are just talking past each other. And if you allow them to
remediate, without feeling embarrassed, and they can do it in a game framework and they
get points and badges, that, actually, I think solves a lot, a lot of the actual motivation
problem. The other ñ the other -- the learning styles, what are you doing requiring different
learning style that a standard video provides? So there's a couple of things. I mean, there's
the video, but I think the exercises are at least as important and that's what we do more
for the peers to be able to interact with each other. That's as least as important.
But the other thing is--this isn't Khan Academy related--but they've actually done a lot of
studies, like double-blind control studies and the learning styles is kind of a myth;
that there's actually a good instruction and bad instruction. Everyone likes to think that
"I'm an auditory learner, I'm a visual learner," but the reality is ñ no -- there's some things
you learn, you're learning art history or geometry, you're a visual. If you're learning
a foreign language, you're auditory, or maybe a little bit of visual. If you're learning
certain things, you like to see it written. And when they actually did double-blind, this
is easy -- you can go Google it. There's a bunch of styles; I was just reading this last
week, that there's actually no evidence that to teach people who are self-identified, or
somehow identified auditory learners in an auditory way, actually is better than not.
But with that said is ñ with that said, I think there are multiple ways to teach the
same thing and so, I'll make videos. I think in the future, we'll probably have other hopefully
good lecturers who will take a slightly different take on things. Even some topics, I take five
different takes on it, so people see it from a bunch of different angles. Maybe at different
skill levels. Even now, there are some videos that are very primary, a little bit more mechanical,
but some go a lot deeper and go into the proofs and go into the actual intuition. So I think
itís just making sure that there's enough material out there that it responds to every
type of learner. I'lló[pause]
>>audience #7: The way, the implementation, that you've done is great, but the ideas aren't
really new.
>>Salman Khan: Nope.
>>audience #7: They're 30, 40, 50, 200 years old.
>>Salman Khan: Yep.
>>audience #7: So, I was wondering how much, how much you've been able to tap into the
small bit of this kind of learning that's been done in the past, talking about like
Sudbury Schools--
>>Salman Khan: Yeah.
>>audience #7: or some of the very radical educational groups, un-schoolers? How much
has been able to flow back into the things you're doing in Khan Academy?
>>Salman Khan: Yeah. So the general concepts here are not at all new and actually, I was
recently looking, I mean people -- this whole notion of actually mastering something before
you move to another concept, it's called mastery-based learning and people have been talking about
it for 80 years and have actually done studies on it, like 30 years ago. The two definitive,
in all of the education research, the two definitive studies, or at least the only two
that I've found that are just super definitive, are one, mastery-based learning. Every trial
they did, and they did this 30 years ago with like worksheets and people walking around,
but it was essentially the same model had unambiguous success. I mean, these kids were
learning at several fold the pace of traditional students, with much better retention and much
better mastery of things. And then the other thing, there's actually a good bit of research,
30 years old, definitive, never refuted, is that when people watch a lecture, they zone
out after 15 minutes. And they zone out for about five minutes and then when they zone
back in, they're only able to zone in for about nine minutes. And then they zone out
for another five minutes and then when they zone back in, they can only zone in for about
six minutes. And this process goes until, by the end of a 90 minute lecture, they can
only zone in for like three or four minutes. And I think we've all experienced that. Hopefully,
none of yíall are right now.
[laughter]
But these are not ñ but to the question's point, these are not new ideas. Even when
radio came out, people said, "This is gonna revolutionize education; get the best lecturers,
put them on radio." TV came out; same idea. VCRs came out; same idea. I think what makes
it different this time is it is so easy to get at. I mean, it is literally on demand,
and you have all of the data, all of the analytics. And I think there is something about the actual
craftsmanship. This wasn't built by a bureaucracy at the Department of Education, or a bureaucracy
at a publishing company. This was built by people who are literally looking at the learner
and saying, "Well, what does the learner want? And what's useful for them?" In terms of actually
leveraging some of that stuff, I think we can leverage the spirit, I think we can leverage
a lot of the models. And it gives us confidence that look, there's actually a huge amount
of research, it's just nothing exists because it wasn't practical until now. And I think
that's probably the level that we can leverage a lot of these other models. I'll take one
over here. "Your program demonstrates the role that technology can play to make learning
efficient and effective, yet the adoption rate for technology in the public education
system is lagging. What hurdles must we overcome to get these tools in the classroom?" So I
think the technology in the public education system is lagging for good reason. I don't
know -- even when I was in school, in the 80s, we had computers in the classroom and
we would go into the computer lab and we would learn how to insert a diskette. Or, really
silly stuff, like how to boot the computer. There was no actually using the technology
in the classroom. And I think that goes all the way now where people have iPad's but they're
still not really doing something constructive. So, I think a lot of the skepticism was actually
warranted. And hopefully, Khan Academy will be a catalyst for saying, "OK, finally we
have something that is actually something to do with the technology in the classroom."
And so, I'm hoping it's actually going to be a catalyst for solving that problem. Go.
>>audience #8: I mean this is a very disruptive approach, things like flipping the classroom
and so on. Are there any stakeholders where you've encountered strong resistance? I don't
know, parents, principals, teachers, or somebody?
>>Salman Khan: Yeah, so have you encountered any resistance? It's strange, but no. One,
because from the get-go, you know the way -- it's been a very organic process. So, never,
five years ago, even now, we're not saying, "We are going to lobby the Department of Education,
or this state body, or this school district, and we're going to try to sell our product
to them." If we did that, we would face a lot of resistance. What we're doing is we're
literally putting stuff out there. We have mainly the student in mind and forward-thinking
schools and institutions and school districts are coming to us and so, they've been, they've
been pretty amazing about doing it. I think one interesting thing that'll happen, I think
for almost everyone in this room, if you were to be a school teacher, this would be a more
fun way to teach. You get to go into a room, you don't have to prepare.
[laughter]
And, and you have -- you get to be a mentor for these students. I think what could be
a little scary for some teachers is that you're a fifth grade teacher, but there's some kids
doing calculus in that room. And what we've seen with the Los Altos students, it's actually
pretty profound what they've been doing. They walk up to those kids, cause some of those
kids go red on like, the chain rule, right?
[laughter]
And so, the teacher's like, "I have to intervene."
[laughter]
She'll walk up to that student, she's like, "You know what? I haven't seen this since
college and I forgot how to do it. Let's learn about this together." And it takes a lot of
self-confidence for a teacher to admit that they aren't the know-it-all at the front of
the classroom, but they are just maybe a more mature, someone who's seen more of the world,
but they're at the same level mathematically right now but they can learn it together.
So I think that's the paradigm shift that will have to go on, but I think it becomes
much more pleasant as a teacher to be able to do that. And the kid immediately responds
to the teacher. The kid immediately respects the teacher more when he or she actually communicates
in that kind of way.
>>audience #7: And parents seem to be OK with it, also?
>>Salman Khan: Yeah, with the Los Altos, we had no experience actually working in a school
system. The Los Altos principals and school board, they set up some parent meetings, and
the parents ñ they were all -- so that they felt comfortable with it. They gave everyone
the option to opt out; no one opted out. And now, they're getting a lot of push to go,
as you can imagine. We're gonna continue to multiple classrooms. You can imagine there
are fifth graders where some of the kids aren't doing the chain rule and their parents are
getting a little paranoid.
[laughter]
That little gap is occurring. I'll ask a--." The modern economy demands an increasingly
educated workforce, yet higher education continues to become more and more unaffordable. How
is this situation rectified? Can 'college' be disrupted?" So, the simple answer is yes.
[laughter]
And actually, I think this is something, I mean, I don't know if I should go on the record,
on video, but I said this at lunch, so no harm saying it now. Actually, I think Google
could play a big role in that. Imagine--and the details have yet to be worked out--but
imagine where all of the core academic stuff Khan Academy takes care of. If you wanna learn
electric circuits, if you wanna learn big-O notation, if you wanna learn capital asset
pricing models, Khan Academy can do those academic things. So, imagine if there was
a university in Mountain Viewówe'll put it in downtown Castro Street; it has a nice vibe
and all of that-- and you literally, I mean, we could even leverage Khan Academy. We know
all of the best students are gonna be using Khan Academy already. I think a lot of them
are and so, we don't have a, if you started an institution, you don't have this kind of
recognition problem. But what you do is the students, you say, "Look, instead of going
to MIT or Stanford or Harvard, why don't you come to this new university--whatever you
call it; Silicon Valley, whatever; some name-- and the focus is you spend one year at Google,
or six months at Google, six months at Apple, six months working on a project for Kleiner
Perkins, six month writing an iPhone app, six months working at Facebook. And you do
a series of projects; those projects are essentially your transcript. We have seminars. We'll even
set up a little bit of a dorm for you so you have the nice college experience and you can
walk into each otherís classrooms, each other's bedrooms in the middle of the night and chat
about things and all the rest. And instead of you paying any tuition, we're gonna pay
you. Each of those companies, instead of, I mean, all of a sudden, when I graduated
from college, all of a sudden all of these--Google didn't exist then--but Google-like companies
were like, "Oh, here's a smart guy and from a good school. I wanna hire him." And literally
four years ago, I had probably 90 percent of the same skills and I couldn't get a job
at a local supermarket. I'm serious. Like, I literally applied to a supermarket and I
couldn't get a job. And it was just the process of going to one of these schools that was
just like a big signal. So, on one side, actually you would get these really smart 17 or 18-year
olds immediately doing useful things and we know from recent companies that started, that
there's some very innovative things that 18 or 19 year olds can do, and the company's
benefit. But even more important, these people are actually getting tangible skills, but
they're still getting the academic component because a lot of that can be done self-paced,
it can be done in a seminar format, and frankly, the faculty then, it becomes all of you guys.
And the management at Google and Apple and the VCs in the areas and the entrepreneurs
in the areas, and I think if you have an institution like that, and all of a sudden, people not
only do they not have to pay for school, they can actually get paid to go to school and
they finish and they're probably gonna get a better salary and better job placement than
pretty much any other university in the world. I think just that example would pretty much
disrupt education; disrupt higher education. So, we'll have to work out the details.
[laughter]
>>audience #8: Hi. So, the mastiff actually looks really great and I'm just really excited
to see this kind of thing. My question was how it extends to sciences, where sometimes
there really isn't a correct answer--
>>Salman Khan: Yeah.
>>audience #8: or, at least if you just write a bunch of word problems, physics word problems
aren't really physics. There's something more deep going on there and I was wondering how
you would, if you had any thoughts on how that extends.
>>Salman Khan: Yes. How do we extend that stuff? Maybe itís not as concrete, multiple-choice,
or free answer, right answer all the time. Well, what I think there is a lot that is
that, is that kind of builds a scaffold, so you would do the traditional kinematics and
Newtonian mechanics and the traditional problem sets, you can do in this type of a form factor.
And then it can be all self-paced. I think to get to the next level of real, deep understanding,
there's a couple things that we could do. We could more simulation-based learning, where
as you go through that knowledge map, at some points these simulations unlock, or maybe
they always unlock and you have to figure out some things just by observing the simulations.
I think a really interesting thing -- one thing that people -- we talk about providing
education, but one thing we don't do is we don't credential people right now; we don't,
in an authenticable kind of way. So, maybe oral exams is the right answer. Like, you
have badges and right now, you're a novice physicist, or whatever, first level physics
person, on the Khan Academy. If you want to get to the middle level or the ninja level
or whatever we end up calling it, you have to get five other ninja level people to have
a Skype oral exam with you. And only then, if four of the five, or five out of five,
say "He's ready to be a physics ninja," that's too cheesy of a name, but then he will move
up. There's actually a possibility here to actually do the best of, I mean, the gold
standard in evaluation is the oral exam. But it has to be well done. But I think there
is something here where we could go to that level.
>>audience #8: Thank you.
>>Salman Khan: I'll take, "Currently, most of the content is focused on learner-centric
approach. Are there plans to incorporate content for instructors/facilitators- more softer
skills? Example: Presentation skills, facilitation skills, adult learning theories, etc." We
will. I mean, we will do, I don't know if it'll just be for instructors/facilitators,
but, I don't know -- this might ruin the Khan Academy, but I've been tempted to give dating
advice on the --
[laughter]
My son's two years old right now and the theory is, when he's 17, he's more likely to listen
to 34 year old Sal than 50 year old Sal. And so, if I just record them now, it'll be like
a nice time shift of--
[laughter]
I'm kind of serious about -- but--
[laughter]
we're open to it. I mean, we don't wanna do something that's really cheesy, like "Make
sure to eat five different groups of fruits and vegetables." I mean, some people have
lobbied for videos like that. We want people where any Khan Academy video you watch, there's
kind of line an "aha" moment, like "Yes. I understand something about the world that
I'm not getting from other sources." We would want to do it so it's actually meaningful,
but we're not opposed to doing softer skills type of things. We'll see how it works. We
don't know.
>>audience #9: So, it's clear to me how this stuff can work really well at the elementary
level, but I'm more worried about what it might do to the curriculum at the higher levels.
I have helped my kids learn this stuff all the way up and by the time you get to Euclidian
geometry, a good problem might take half an hour to an hour to solve. The answer might
be half a page proof and there might be ten different proofs that are equally valid. I've
actually had some specific problems that I've worked through with my son. So how do you
cope with the fact that you're limited to this form factor that you described to us?
>>Salman Khan: Yeah. I think some of that can be tackled--this form factor--not all
of it. I mean, if we designed the modules right you could actually probably do fairly
deep, I mean -- it would be hard to be able to have someone actually write the proof and
then do a proof check. I mean, even people have attempted--
>>audience #10: [ ]
>>Salman Khan: OK, so it's done. Computers can do it.
>>audience #9: Not as a practical matter.
>>Salman Khan: I mean seriously -- I think we're gonna try to do, we're gonna try to
tackle as much as you can do with that form factor, but what we're hoping is if you get
a lot of the blocking/tackling done this way, it frees up a lot of time so that when there
actually is human interaction, and a human interaction might be with a parent, it might
be with a teacher. We have these Meetups happening and these Meetups might turn into spontaneous
tutoring sessions, where people can work through hard, more difficult problems, or group problems.
I think some of that stuff could, I mean, we're even entertaining, I mean, now we have
enough students where we're even entertaining Khan Academy math competitions. It's the next
level where it's like human written, unique problems and that maybe teams in different
areas can try to solve it. Maybe they can even collaborate. We haven't thought about
it deeply yet, but I think it's doable. It won't just be a hundred percent automated,
though.
>>audience #9: Right. And in a much shorter question that perhaps has the same answer,
I coached Math Olympiad for a number of years and one of the things I really enjoyed was
when the students came up with a solution that was not the one that I was gonna suggest.
You kind of cut the feedback loop here. Is that a problem?
>>Salman Khan: Yeah, I think on Math Olympiad type questions, I think that's going to be
a completely different-- it won't be this. It will be, so we are playing with the notion
of having a question repository, human-written questions. Although, right now, we're thinking
it would still be more the right answer type of things, not proof-based. But you could
imagine, actually, that would get closer to the peer-to-peer writing type of thing. So,
there's a question, people submit solutions and then if you have a critical mass of people
on the system, they can evaluate it--kind of a peer review-- and they can say, "Whose
proof was the coolest?" or something like that. I don't know. So, it won't be automated
in the instruction but because of the Web and the number of people you have, kind of
a critical mass, you could probably get a good peer-to-peer thing going, I think.
>>audience #9: All right. So, one last try at the first question.
>>Salman Khan: Yeah, yeah, yeah.
>>audience #9: Because you didn't really answer what scares me most. I'm worried that this
will be so successful that it's gonna dumb down the curricula. It's going to be easy
enough, maybe it's gonna be so much easier for the school to do stuff in this way, that
things that aren't amenable to automation will simply vanish from the curriculum. Is
that a legitimate worry?
>>Salman Khan: I don't think so. I hope the first part about your worry is true; that
it's wildly successful.
[laughter]
But I think the answer is that right now what you have in schools, you have great teachers
and you have not so great teachers and you have things in-between. What happens is, the
kids who have the not so great teachers, they're still gonna be able to do well on the SAT.
They're still gonna be able to do well on the AP Calculus. They're not gonna be math
Olympiads, but they're gonna get the blocking and tackling, so that they still have the
same opportunities and all that. And the students who have the good teachers, the great teachers,
those teachers are gonna see this as "Yeah, this is taking all the blocking and tackling."
Like, if you're tutoring kids, you can say, "Look, you need to learn how to add fractions.
I know you can do that on your own, especially if you're gonna get to the Math Olympiad.
You can do that part on your own. Khan Academy has your back, but then with the time with
me, we're gonna go to the next level." I think, across the board, hopefully it'll have an
upward effect on the level of discourse. I mean, it's like a lot of people are gonna
have their gaps out of the way, so just everything, hopefully, will be more intelligent in the
classroom. Hopefully I'll take -- and anyone should feel free to leave, I'll hang out here
and answer a few more questions over the next, I think we're out of time, but, I'll do another
ten minutes of this and then we can just whatever. OK? "Do you have any ideas about how to engage
parents in their child's development?" The easiest way to do that, any parents, is actually
you should log on to Khan Academy right now, and you should start doing the math problems.
Although, I suspect, many of you already know a lot of this material. But we've seen over
and over again, that that fundamentally changes the conversation that happens at the dinner
table; when parents are also doing the same mathematics as their kids. I'll take one.
>>audience #11: Is there a chance that kids are just kind of like figuring out the trick
and sort of getting through the thing, but there's no real retention? Like ñ do you
do anything?
>>Salman Khan: No, no. Not only is there a chance, they are doing that. The kids are,
the fifth graders especially, it's amazing how good they are in gaming things, at figuring
out "Oh, that can't be the choice, that can be the choice." And so, it's a mixed blessing.
One is, we are watching them and we're saying, "Oh, they're just gaming that module." So,
we either have to make that module less gameable, make it free answer, or do something to it
so that it can't be gamed. Or, we have to add other modules to make sure that they're
really learning the concept. In terms of retention, we actually do have a thing that after X days,
if you haven't done a module or another module that covers that module, intrinsically covers
that same concept, it asks you to review it, so you get extra points to go back and review.
So, the model is, the ideal school would be, you're walking down the hallway in seventh
grade and all of a sudden, your fifth grade math teacher comes up to you and asks you
a fifth grade math question. And so, that's what we're trying to do. It is gaming, but
it is actually interesting because we actually saw that with the chain rule module. A couple
of fifth graders just figured out mechanically how to, even that's not easy though, but they
figured out just the pattern. But it was still cool. Well, one, I'd argue a lot of twelfth
graders are also doing that, but it was cool once they figured out the pattern. They started
to teach each other. So we actually even have this one view, you can see the cascade. Who's
the first student in the class and you can see the cascade of the other students right
after that, after one person gets it. But, with that said, is we did identify that as
-- we have to add more modules to make sure people really are understanding the concept
as opposed to just learning the mechanics.
>>presenter: So, if you just take one more question and after that, we have to basically
wrap up for the video also. But, as he said, he'll be hanging around and you guys can come
over and basically chat [ ].
>>Salman Khan: OK. I'll take it right here.
>>audience #12: So, as you said, the Khan Academy is definitely turning things around,
and how, in the classroom with this kind of self-instructed learning, how do you try to
reapply the framework of a grading system or a testing system to give the same kind
of progress feedback that people are expecting from education?
>>Salman Khan: Yeah, so the question, how do you do the same type of assessment, or
how can we start labeling kids if we're--. I think it's a good problem to have. Maybe
we shouldn't and I think the ideal is you take -- I actually do think it is good that
we do have these standardized tests, the SAT and the AP test, but the ideal is you don't
take them when you're 17, like everyone lockstep takes them when you're 17; you take them when
you're ready for it. So, there's some fifth graders and they're doing the Khan Academy.
When we have the 300 modules and we're like, "Look, statistically, it looks like if you
go and take the AP Calculus test right now", we can even give them a distribution. "We
think there's an 80 percent chance you're gonna get a five, a four or a five or something.
Go take it." And if there's a 17 year old who we think ñ "there's an 80 percent chance
that you're not gonna get higher than a three, don't take it yet. Just keep working. Wait
until you're 20, wait until you're 25, whatever. If your goal is to learn calculus". One thing
I'm hoping we can do, in a non-touchy feely way, cause a lot of people, when people say
"no grades," it sounds very touchy feely. It's the opposite. We actually want people
to master; everyone should be an A+ student. Everyone should have a hundred percent mastery
if they really wanna get to some future level. I mean, I can't tell how many, even when I've
interviewed, even in my past life, people who've gone through algebra, trigonometry,
geometry, calculus, blah, blah, blah, and they still don't know when to apply algebra.
What was the whole point of this non-mastery based learning? It was just a big jumping
through hoops and showing that you know how to do homework type of exercise. So, yeah.
I think it'll be a good problem that if there's no grades, and if there's literally just,
"We say you know this." And if you don't know this at A or B level, if we say you know it,
you know it at an A level. Cool.
>>audience #11: Thank you.
>>Salman Khan: Thank you.
[applause]