My name is Barbara Cockrill.
I'm the Director of Faculty Development at HMS,
and I'm going to talk to you just
in this little 5-minute slot about some work we've
done on optimizing preparation for the flipped classroom using
efficiency metrics.
And I really want to give a shout out
to Henrike Besche, who's the first person up here, who's
really done most of this work, but unfortunately she
can't be here today.
So what we have done at HMS in 2015, we launched the--
completely blew up our curriculum
and started over again.
And in med school, for those of you who are unfamiliar,
our students are spending the first 14
months in the classroom.
And that's all been done in a flipped classroom pedagogy.
And then they spend years two, three, and four
on the wards in the hospital.
So I'm focusing on what they do in the first 14 months.
And we used a flipped classroom pedagogy
that we call case-based collaborative learning.
So I know that I don't need to talk
to this group about exactly what flipped classroom is,
but there's two things that you need
to know about the way we've done it that is
important to this project.
Number one, like most, or many, students get their prep work
the night before.
And then they also, before they come into class,
have to do a quiz.
And that's a multiple-choice quiz that we track.
The second point that is different than a lot
is that our students come in, and they
work in groups of four.
So they are assigned to groups of four.
They come in, and in that group of four,
they work through the prep work from the last night before.
And then they also apply it to new problems.
So something came from it that we learned is number one--
we were all experienced faculty, who
had been preparing and teaching Harvard Medical
students for quite a few years.
And we had been assigning the same amount of prep,
kind of, that we'd always been assigning.
But what we found was that our students actually
in these groups of four, they wanted
to be prepared for themselves, but there was also
a lot of peer pressure.
Because part of the contract here is you
work with this group of four, and you learn from each other.
And there's time in the classroom to do that.
So if you show up unprepared, it's not just on you,
but it's on--
it ruins the morning for the rest of the group.
So what we learned was our students were doing the prep,
and it was way too much.
So we'd been assigning what we thought
was appropriate amount of prep, but our problem
was that this was taking forever.
It was too much.
It was too hard.
And so what we wanted to try to do
is figure out how do we most efficiently
assign that prep, because the students are now doing it,
to optimize learning in the classroom.
So we used a simple approach.
A student-- a learner-centered approach,
where we asked the students about their prep.
And we were asking them not, is this too long,
but, is this efficient and effective preparation
for your learning?
And then we compared it to the scores
that they did on their readiness assessment
quizzes ahead of time.
And what we were trying to work toward
is what we are calling an efficiency graph.
So on this axis-- and I'm going to show you
some data next-- is the level of difficulty as perceived
by the learner.
And then on this axis is the successful learning,
like how well they scored on that quiz.
So we want things to be in this area,
like it was effective learning, and they did well on the quiz,
or maybe it was really hard, but they still
did well on the quiz.
We wanted to avoid this down here where things were just
super hard, and it wasn't effective.
So this helped to--
one of the things is many of our courses
are run by a number of different content experts.
So you'll have 90 sessions in the class,
and five sessions are the genetics content expert.
The other five are the biochemistry content expert.
So it was really important for us
to try to identify that expert blind spot
so that faculty weren't giving things that--
you know how this goes.
It's so easy.
Why is this so hard for the student?
But we forget how difficult it can be for the novice learner.
So here I'm going to show you how our data--
and this is primarily for faculty development
and determination of best practices.
So remember, difficulty here and quiz score here.
This is a summary of one course, which
has 10 different disciplines within that course.
Each one of these arrows tells you
where they ended up for all of their content.
What we did was we picked apart each one of those disciplines.
So it's not surprising to us that anatomy is up here.
It's hard.
It's typically the hardest one.
Any med students in the room?
It's very hard.
But we picked it apart and looked
at each of the 90 sessions and identified
some that were up here, some that were here,
some that were here, some that were here.
What we were then planning to do is
take this back to the faculty, have them look at it,
try to understand it.
And we're going to look at this group
up here to identify best practices.
So what allows these sessions to be very effective learning.
And by the way, it wasn't just the length of time.
Some of these sessions up here in the best area
were the parts that took the students the longest.
So it wasn't just that they were working.
It was effective learning material.
And then the things that show up down here
are the ones that we want to target and figure out
what it is about those that--
what can we change to make it more effective.
So our future goals--
in progress, we're scaling up.
We're evaluating alignment between faculty and students.
Faculty can't tell.
They think it's easy, and the students think it's hard.
So we're not very good at identifying that.
We're engaging faculty in conversations
about desirable level of difficulties
across all courses.
So hitting it in that good area up there.
And then our next steps will be to identify best practices
and improve instructional design, figuring out
why they're up in that top-left area,
and then to automate this data collection so all faculty have
access to it.
So you can go in and look and see
where your session falls in.
So thank you to HILT. I think I'm a little over time.
But what HILT did for us is helped fund, really,
the statistics and people helping
with all of the computer programming to put this up.
And without them, we couldn't have done it.
So thank you.