Our project is centered around these experimental design chalk
talks that we've incorporated in our introductory molecular
biology class for graduate students
at Harvard Medical School.
And we've looked at the effect of these chalk
talks in a few different ways, both
looking at video recordings of students
in their first and second times doing these chalk
talks during the course, as well as
written answers to experimental design questions
throughout the course.
We've also been looking at and thinking
about how this affects students confidence or self-efficacy
with the skills or science communication
and experimental design and how that changes from the beginning
to the end of the semester.
We've also been developing some resources
for training students.
Training students in how to do experimental design
in chalk talks.
So a chalk talk is literally talking with chalk.
And so what it is, it's very different from a type
of presentation that we see a lot with our trainees
is that we do a good job of teaching presentations using
PowerPoint in the background.
But in this case, they don't have PowerPoint, right?
So they actually have to do an unscripted talk.
Well, they're scripted.
But they do it in an unstructured way at the board.
And they walk through an experimental protocol.
What's different and what you see
with that presentation in general
is that it's built to be interrupted.
And a lot of times, the conversation
goes where the audience wants you to go,
and thinking about how you craft a narrative while actually
interacting with the audience.
It's used at multiple different stages.
We know that faculty are typically
asked to do it as part of their job interview.
Along with a seminar, they have to do a chalk talk.
A lot of trainees don't know that.
But we've also seen instances at multiple places
where some postdocs are asked to do it
as part of their interview.
We know that graduate students use it in their prequalifying
exams for many of our programs.
So we're training them for something
that they'll see in the second year.
And I've actually heard of instances
now where graduate students are being
interviewed for graduate school using a chalk talk.
One of the things we found working with our students
is we talk about experimental design a lot,
that that's what you get out of graduate school.
But we actually don't do a lot of formal training
and experimental design.
That's a deficiency that we saw.
And we tried to address that in our life sciences
graduate course by actually trying to intervene, teach them
skills in experimental design, teach them skills in science
communication, and then force students to actually put that
into practice through the types of assessments that we use.
So when we talk about teaching experimental design
and how to do a chalk talk, we do them together.
And we've developed a series of resources including videos.
So we have sample questions that we show students.
This is what the assessment will look like in our course.
Then we have videos that we've developed
a good version of what a response would look like
and a great version of what a response would
look like, and actually put those on display
for the students along with an annotated guide that tells
them, this is what a good, this is what a great chalk
talk looks like, and this is what
gets you to a high score using a novel rubric that we've
developed.
So the students get sample question, sample presentations,
and a rubric with an annotated guide that tells you
how to excel at that rubric.
And the last piece that we're finishing up with our project
is actually trying to make animated movies
to explain this, so that actually accompanies it too.
We're at the point now, I think one
of the nice outcomes from this whole project
is that we've lined up a pretty robust set of papers
that we're developing as outcomes from the project.
And we're actually spanning back,
we're at a point where we're analyzing data for 10 years.
And so this chalk talk was actually
introduced in this class in 2009, the very first time
that it was done.
And then 2010, it was modified.
And we've gradually modified it to the point where
it's been pretty consistent for the last four years
what it looks like.
But the idea was simply we had a standard molecular biology
class.
The person who was very involved at the time
was Johanna Gutlerner.
She was a curriculum fellow in the course.
She preceded me as the curriculum fellow.
And I preceded Madhvi as the curriculum fellow.
She thought we should find a way working
with the faculty, the faculty and the class
also, to make the class more interactive
and actually to focus on skills, which at the time,
was their idea for life sciences graduate education, right?
Everybody said you come to graduate school
to learn all the mechanisms and the details.
And they were talking about trying to introduce skills.
Fast forward 10 years, now that's what everybody
is talking about in grad ed that we need to focus on skills,
that it's not just an undergraduate thing.
So it's taken on a life, I think, a value in that respect.
So we worked it into the course as an assessment.
And I think the driving force behind it
was watching the student feedback
on the course evaluations, which told us that students liked it.
And what we learned was to use the feedback in a way
to continuously address the problems they
had with the assessment to the point
where it's consistently been one of the strongest
positives in the class.
One of the outcomes from our project
is all the data analysis we've done on course evals,
consistently positive.
And another thing that we've seen
is when we ask students years after they've
left the class, what was the lasting learning
outcome from the course?
It's actually the exercise and being
able to apply that skill in multiple different settings.
Really enjoyed working with the students.
And I think for me, they weren't just workers.
They were also collaborators in the project.
And I think it was really important to involve them
in that way.
And I think also just the breadth
of things we were able to look at related
to experimental design chalk talks and the different types
of outcomes, I think that was really
valuable to have that breadth.
And I think that the grant really
enabled us to have a broader view of it
instead of just looking at an individual part.
One of the things that's very nice about the Spark grants
in particular is that when you need a relatively robust amount
of money, but not an immense amount of money,
like a NSF grant or NAH grant, it's
enough money to get a good amount of projects done
and get research done.
And it really helped with us being able to engage students.
So this project really benefited from the fact
that we have six students who are helping with the project.
And I mean, one of the projects literally entailed
transcribing 500 rubrics for multiple years, right?
And two students did that.
And so that type of work is not something
that it's really easy to justify for a big grant.
But it's a starting point to actually get the work done.
And what we've been able to do is do that, right?
So a lot of that engaging students
who have different expertise like biostatisticians
to help us with analysis.
So I think the student engagement part of it
actually has been a really important outcome
of having the funding.