CHRISTOPHER POTTS: Hello, everyone.
Welcome to part 2 in our series on presenting your research.
We're going to be talking about writing papers in our fields.
To start, let's look at the outline of a typical NLP paper.
By and large, these are either four or eight page papers
in a two-column format that you get from the style sheets.
Those links don't include the references.
And there are a bunch of conventions
for how the papers are typically organized.
So you have your title in abstract on page one.
Usually, you have an intro section
that kind of fits on that first page, maybe a little bit
onto the second.
In place two, you discuss the related work
or prior literature or background
that's needed to kind of contextualize
the work that you're doing.
Then there's typically a data section, followed
by a section on a model or this could
be thought of as the kind of core proposal
section of the paper.
Then there'll be some methods related to the experiments.
The reporting of the results of the experiments.
And then some analysis of what the experimental results mean.
And then possibly a short conclusion.
It's not set in stone that you have
to follow these conventions, but if you do follow them,
I think it will be easier on your readers
and also easier on you as a writer
because you can kind of slot your ideas
into this familiar format.
Let's look at those sections in a little bit of detail.
So starting with the intro, the ideal intro, to my mind,
really tells the full story of your paper at a high level.
We don't need all the details, but it is very helpful
to know from beginning to end what the paper accomplishes.
And good intros provide all of that information
and really tell the reader precisely what they'll
learn as they go through the rest of the paper.
In place two as I said, is the discussion
of background material or related work
or prior literature.
And this is an opportunity for you to contextualize your work,
and provide insights into the major themes
of the literature as a whole.
What you should really be thinking about doing
is using each paper or each theme
that you identify as a chance to kind of contextualize
your ideas, and especially articulate
what's special about the contribution
that you're making.
So this kind of sets the stage for the reader.
The data section, this could vary a lot.
This could be very detailed if you're offering a new dataset,
or using a dataset in some unfamiliar way the community is
not used to.
But of course, if you're just adopting some data
off the shelf, then this section might be pretty short.
Then you get to the heart of your proposal, your model.
You want to flesh out your own approach
and really help us understand your core contribution.
Then we turn to supporting your ideas
with some experimental evidence.
You report the methods, your experimental approach,
including descriptions of the metrics.
And again, that will be long or short
depending on whether the metrics are familiar or unfamiliar.
You want to describe your baseline models and anything
else that's relevant to kind of understanding precisely
what's going to happen in your experiments.
I will say that for details about hyperparameters
and optimization choices and so forth, you can probably
move those to an appendix, unless they're
really central to the argument.
What you want to offer here are kind of the crucial pieces that
will help the reader understand precisely what you
did for your experiments.
Then we get our results.
This could be a no-nonsense report of what happened.
It's probably mainly going to be supported
by figures and tables that report a summary of your core
findings according to your data models and metrics.
And then things open up a bit.
You have an analysis section.
I think this is really important.
You should articulate for the reader what your results mean,
what they don't mean, where they can
be improved, where their limits are, and so forth, right?
These sections vary a lot depending on the nature
of the paper and the findings.
But I think they're always important
and they can be very rewarding.
It is intimidating because this is awfully open ended.
But I'm hoping that the previous unit on analysis methods
in our field offer some really good general purpose tools
and techniques for doing rich analyses of this sort
and really helping us understand precisely what you
accomplished.
Now this is, as I said, is not set in stone,
and different projects will call for different variants on it.
And one really prominent variant that you see
is that, if you have multiple experiments
with multiple datasets you might want
to repeat that method's results analysis rhythm across all
of your experiments to give them kind
of separate sections in your paper.
But again, it really depends on what
you think the most natural way to express your ideas is.
These things aren't set in stone,
they're just conventions that help us
as readers and as authors.
Then finally, you have a conclusion.
This is probably a quick summary of what the paper did,
and then an opportunity for you to chart out
future directions that you or others might pursue.
So it's a chance to be more outward looking and expansive.
Let me close the screen cast with some general advice
on scientific writing that I think
can be helpful kind of in the background
as you think about expressing your ideas.
First, I just want to review this really nice piece
from Stuart Shieber, where he advocates
for what he calls the "rational reconstruction"
approach to scientific writing.
And to build up to that, he offers two contrasting styles
that you might think about.
The first is what he calls the continental style.
This is in which one states the solution with as
little introduction or motivation as possible,
sometimes not even saying what the problem was.
He says, "Readers will have no clue as to whether you're
right or not without incredible efforts
in close reading of the paper.
But at least, they'll think you're a genius."
At the other end of the extreme, you
have what he calls the historical style.
And this is a whole history of false starts, wrong attempts,
near misses, redefinitions of the problem.
This is a kind of genuine history
of maybe the struggles that you endured
as you built up to the final product for your paper.
And Shieber says, "This is much better
than the continental style, because a careful reader can
probably follow the line of reasoning
that the author went through and use this as motivation.
But the reader will probably think
you're a bit addle-headed."
We don't need to hear about every dead end
and every false start, what we would like, rather,
is what Shieber calls the rational reconstruction.
You don't present the actual history that you went through,
but rather, an idealized history that perfectly motivates
each step in the solution.
You might selectively choose models
that you abandoned as a way of helping the other reader
understand how you built toward your actual core set of methods
and findings and results.
So it's going to be a kind of streamlined version
of that historical style.
The goal in pursuing the rational reconstruction style
is not to convince the reader that you're
brilliant, or addle-headed for that matter,
but that your solution is trivial.
Shieber says it takes a certain strength of character
to take that as one's goal.
Right, the goal of writing a really excellent paper
is that the reader comes away thinking,
that was clear and obvious, and even I could have done it.
That's an act of genuine communication.
And it does take a strength of character.
But in the end, this is what we should all be striving for,
this kind of really clear and open communication.
This is also a nice document from David Goss.
He has some hints on mathematical style.
There's a bunch of low level details
in there, especially related to presenting very formal work.
The piece that I wanted to pull out
is just "Have mercy on the reader."
This is, again, recalling the rational reconstruction
approach that Shieber advocated for, where you're really
thinking about what it's like to be a reader encountering
the ideas for the first time and genuinely
trying to understand what you accomplished.
You have to really think about what
it's like to be in that position in order
to have a successful and clear paper.
I also really like this piece from the novelist Cormac
McCarthy which was published in Nature.
It's full of great advice.
The one piece that I wanted to highlight is this.
Quote-unquote, McCarthy says, "Decide
on your paper's theme and two or three
points you want every reader to remember.
This theme and these points from the central thread
that runs through your piece.
The words, sentences, paragraphs,
and sections are the needlework that holds it together.
If something isn't needed to help the reader understand
the main theme, omit it."
This is helpful to me because I think
it not only results in a better paper,
but it will also be easier for you to write your paper.
Because the themes you choose will determine
what to include and exclude, and resolve a lot of low level
questions about your narrative.
And conversely, I've often found that when I'm really struggling
to write a paper, it's because I haven't figured out what
these core themes are and I'm kind of casting about,
unsure of what's relevant and what's irrelevant.
And if you step back and really figure out
what you're trying to communicate,
then the act of writing kind of all falls into place.
And then the final bit of advice that I wanted to offer,
which I'm going to return to when we talk about presenting
work at conferences.
This comes from Patrick Blackburn.
It's about talks, but it really extends
to any kind of communication and science,
is fundamental insight, he asks.
"Where do good talks, and I think good papers,
where do they come from?"
And he says, "Honesty."
A good talk or a good paper should never
stray far from simple, honest communication.
And you can hear this in the way that we talk
about evaluating your work.
That fundamentally for us, we're looking
for papers that offer open, clear, honest communication
about what happened, and what it means.
And that's really a fundamental value.
And I think it's inspiring to think
about this as your kind of guiding light
when you report your scientific results to the community.