[MUSIC PLAYING]
DAVID MALAN: This is CS50.
[MUSIC PLAYING]
DAVID MALAN: Hello, world.
This is the CS50 podcast.
My name is David Malan.
BRIAN YU: And my name is Brian Yu.
And today, we thought we'd discuss academic honesty in CS50.
And so every year in CS50, we always have some number of cases
of academic dishonesty where some number of students
submit work that isn't their own, either by copying homework from a friend
or by looking something up online and using a solution they
find online as part of their solution.
And so this is something that CS50 has had to deal with for years
now in terms of how best to address this type of situation,
and how best to prevent academic dishonesty in general.
DAVID MALAN: Indeed this was-- when I first took over the course
myself back in 2007, it was really an end of semester process.
After the teaching Fellows would evaluate student's work
and provide feedback throughout the semester,
I would finally, all too often by semester end,
carve out some time in order to then cross compare all of the submissions
from that semester looking for statistically unlikely similarities
between students work.
Indeed, what a student might sometimes unfortunately do
is copy the work of another student, lean too heavily
on some resource online, copying more than a reasonable number
of lines of code.
And so by cross comparing all submissions with software
itself, do we then notice which lines of code
are in both student A student B's work, and then conclude ultimately,
that statistically this was unlikely to happen.
BRIAN YU: Now, how exactly do you draw those conclusions.
Because I'm thinking about a programming language like C,
there are only so many parts of the language.
Their for loops and their conditions.
And probably everyone's solutions to similar problems
probably have these sorts of elements.
So what exactly do you look for in this process?
DAVID MALAN: Yeah, it's quite fair.
If we relied on this kind of cross comparison
for programs like Hello, World, everyone would appear
to have written exactly the same code.
But as soon as we get into CS50's second and third weeks
where the programs they write in C tend to get a little longer,
there does end up being more opportunity for creativity,
for different stylized actions by students.
And so students code does start to drift.
Even though at the end of the day the solutions
might still be using for loops and while loops and conditions and so forth,
students might format their code slightly differently.
They might write slightly different comments.
And so what tends to happen over time, as the programs exceed
maybe 10, 20, 30 lines of code, is there enough variation?
And indeed, unfortunately, what we often notice
is not even necessarily that the code is identical, because as you know,
that in and of itself might just be a coincidence.
Especially, when nowadays we have 800 students,
it is absolutely going to be the case that two students write,
by chance, very similar code.
But unfortunately, the kinds of things we tend to notice
is when students have the same typographical errors,
or they use precisely the same variable names,
or they make precisely the same mistake in precisely the same location.
And at that point, our instincts start to kick in
and we look at code like this and start to realize,
while this may have happened by chance, on scale
the odds that had happened in this line and in this line
and in this line between two students code is
just more likely than not better explained by some deliberate act.
BRIAN YU: So at Harvard at least, when there are cases of academic dishonesty,
they're usually referred to some administrative body, which
now is called the Honor Council here at Harvard.
And I think you've pointed out and a couple other people
have pointed out that CS50, though it is the largest course that the university,
does refer far more people to the Honor Council like any other class on campus.
Do you think that has to do with something about computer science
or introduction to computer science?
Or why do you think that might be?
DAVID MALAN: No, I don't.
And that's certainly an unfortunate distinction that we've long had,
say for, one or two years where there are issues in other departments.
No, I don't think that computer science students are any less honest
than their classmates in other fields.
I don't think students in CS50 or any less honest than students
in other computer science courses.
I think it really boils down to one, you and I and educators in computer science
are perhaps somewhat uniquely positioned with tools--
with software tools via which to detect it.
And in a large introductory course like CS50,
I think it's important not only out of fairness to those students
who are behaving honestly throughout the term, but also because one of our goals
should be in this course, to teach students
the ethical application of computer science.
That we should be holding students to those same expectations as
are prescribed in great detail in the courses syllabus.
And so I think it's really a function of our one, looking for it.
And to two, through on it that really ends up explaining the large numbers.
BRIAN YU: Yeah, so I'm looking here at the data from past years in CS50,
and it does seem that there's also a fair amount of fluctuation
in terms of what percentage of students in the course end
up being referred to the Honor Council.
Like, in 2009 for example, it looks like nobody
was referred to the Honor Council.
And in other years like 2010, 2012, there's like 1% or 2% of students.
But in other years like 2015, it's up to 5%, 2016 is up to 10%.
What do you think accounts for that fluctuation
because that's a pretty big difference between one year and another?
DAVID MALAN: Yeah, there really has been as you say, from 0% to 10%
depending on the year.
I think it's a few things.
Part of it I think is just a function of how much time
I or we put into the process.
I think the year in 2009 when there were 0%, I did look for worrisome instances
at that particular year, but admittedly in retrospect, I probably
spent less time that year than the subsequent year.
Because the subsequent year it went up to 2%.
With that said, it might have been by chance,
just a group of students who exhibited this pattern of behavior
with far less frequency than others.
So I think that's certainly possible as well.
But I think the uptick in more recent years for instance, 10% in 2016
and roughly 4% or 5% then, which is where
we've been rather in equilibrium the past few years,
I think is also a function of just how much time we invest in it.
So back in 2008, and for a few years there after,
it was only me who is engaged in this process.
I would run the software by myself.
I would look at students submissions side by side.
And I would ultimately decide which to refer forward
to Harvard's Honor Council.
And then ultimately, document all those cases.
But in more recent years have we involved more of CS50s senior staff
in the process.
The upside of which is that we can now one, analyze the submissions roughly
on a week to week basis.
The upside of which is that we can provide the Honor Council
with the tails far more quickly.
Students themselves, while though, never a pleasant
process at least no sooner rather than later, rather
than getting to the entire end of the semester
and then realizing just how many or how often they cross some line.
But two, the fact that we have multiple human eyes on it
means that we do allocate more time week to week
on each of the individual submissions and the crossways comparisons thereof.
The upside though of those multiple humans,
we now have two or three of us who ultimately vote on whether or not
a case should move forward to the Honor Council is that I at least,
and hopefully all of us, have much more comfort in sending a case to the Honor
Council because not one pair of eyes, but two or three
have all adjudicated it to be a clear indication of a line
having been crossed.
BRIAN YU: Can you tell me a little more about that process?
You've talked about now that there are now
a couple of eyes that are all looking at the submissions,
but you've also talked about software being involved too.
So what is the interplay there between the role that software
plays in trying to detect this sort of thing and the role
that people play in trying to detect academic dishonesty?
DAVID MALAN: Yeah, I should first emphasize
that it is not software that is ultimately
disciplining students or referring them to Harvard's Honor Council.
It is rather just a tool that we use as a first pass.
Given that we have some, nowadays, 800 students, each of whom
are submitting 10 homework problems over the course of the semester.
This is a big O of-- n squared problem times 10 or so.
So it's a huge number of comparisons that need to be made,
and it just wouldn't be practically done by hand or by eye alone.
So what we do is run software that literally cross compares
every submission against every other submission
sometimes, within the current year or even, based on our archives,
against recent prior years as well which explodes the problem even more.
And what we get out of that software based process
is a list from top to bottom of pairs of submissions that the software considers
worrisome least similar.
And then we, the humans, typically go through the top 50 or the top 100
matches on that list and use our human eyes and our own experience
and our instincts to decide, ah, this just happened by chance
or, oh, as you said, this is a relatively short program like Hello,
World or Mario.
This is just bound to happen at that point in the semester.
But certainly as the problems get more sophisticated
and the code gets longer is it more clear to multiple humans that, hmm,
looks like something's awry here, especially when it is again,
the same variable names or the same comments or worse,
the same comments with typographical or grammatical errors
in exactly the same place, odds are that's much more
likely to indicate copy paste than it is two students independently
in their own rooms, on their own laptops literally writing
in the same place the same errors.
BRIAN YU: Makes sense.
And it's also interesting that depending on the type of software
that you use, in the same way that a compiler can take a C program
and figure out what is the structure of the program
and compare the structure of a program to another,
that these sorts of comparison programs can do the same thing.
They can take two pieces of code, and even
if they might use slightly different variable names,
can still look at the structure of the program as a whole
and try and compare them against each other
to do some more sophisticated comparisons.
DAVID MALAN: Yeah, and thanks to some of CS50s team members, Chad
and [? Yella ?] and Kareem, we now have our own tools, Compare50,
which automates this process for us.
And you can perhaps, given your experience in the space,
speak a little more perhaps to the algorithmics underneath the hood?
BRIAN YU: Yeah, it is really Chad and [? Yella ?]
and Kareem that were doing a lot of the work there.
But algorithmically, it's sort of an interesting challenge
to figure out how to do these sorts of comparisons.
Because even though it might seem like a computer
is obviously going to be able to do it faster
than people are going to be able to do it,
it's still a lot of work even for a computer.
Especially, if you consider like 800 students in the class being compared
against all of the other students, plus all of the students who have ever taken
CS50 before, not only for one problem, but for all of the problems
in the course.
That's a lot of work for any computer to do.
And so there is a lot of interesting algorithmic efficiencies
that have been put into the software in order
to make it work a little bit better.
Trying to take advantage of things you actually learn about in CS50.
Things like hashing in order to store data inside of a hash table
so you can very quickly look up whether or not
you've seen a particular pattern of characters in a file before.
Those sort of data structures all come into play if you start to think about,
how do you try and solve this problem in a way that's efficient?
DAVID MALAN: Yeah.
And besides software, certainly our own policies have evolved over time.
So you know for instance, that in a few weeks time,
we'll be presenting at a computer science education conference called
CSEIT a recent paper that a few of us worked on
based on our experience with issues of academic dishonesty
over the past few years.
And it's perhaps worth noting that software aside,
I think one of the more noteworthy policy changes
we introduced some years ago was CS50s so-called Regret Clause.
Which was just a single sentence that we added to the courses
syllabus that encourage students to come forward
if within 72 hours of submitting some work,
they realized that, oh, they had indeed crossed some line.
They had copied unduly from some resource online.
They had copied some portion of code from a classmate
or otherwise, somehow other across the line
that was prescribed in the course of syllabus as being not reasonable.
And what we committed to doing in writing in the courses
syllabus was there would still be penalty
and there would still be consequence, but it
would be limited for instance, to our zeroing the problem or the problem
set that the student had submitted.
And we committed not to escalating the matter to Harvard's Honor Council.
The hope was that we could actually turn what had historically
been purely punitive processes whereby we detect some transgression,
we refer it to the Honor Council, and there
after the student is penalized in some way, the most extreme outcome of which
might actually be required time off from Harvard University itself.
We wanted to create a window of opportunity
where students after some sleep, some thought, some reflection,
can actually own up to a mistake.
Because for so many years, so many of our cases
were truly involving students who at 2:00 AM 3:00 AM 4:00 AM are
under very little sleep, under significant amount of stress,
and with a deadline not only in CS50, but perhaps some other course looming,
made some poor decision to take the quick way out
to just copy and paste someone else's work and submit it on their own.
And even if they've decided or realized a day or two later, wow,
really didn't mean to do that.
Really shouldn't have done that, we had never described
a well-documented process for how they should handle that
and how they could own up.
And so this Regret Clause was meant to help ideally chip away
at the total number of cases we were seeing.
But ultimately, help students meet us halfway
so that it becomes more of a teachable moment
if you will and not just punitive.
BRIAN YU: So I remember when I first took CS50 in fall 2015 it was,
I remember seeing the Regret Clause in the syllabus.
And I remember being a little surprised.
Because it wasn't something I had seen before.
It's not something that many other classes do.
Not really anything that I was familiar with.
So I'm curious about where the policy came from?
Was it inspired by any other policy?
Or where did you start to find your way to this idea?
And what was the process like for bringing this into the course?
DAVID MALAN: Yeah, it was really inspired
by having, for almost 10 years, watched the number of cases
come through CS50 and watching the circumstances that ultimately explain
them.
Again, these late night poor decisions under a great stress.
And it just felt like we, the teachers of the course, should be doing
or could be doing a more proactive job at trying to tackle this problem.
And not just looking to detect it, but looking to teach students how to one,
ideally avoid it altogether.
But two, even if they do cross some line how to address the situation then.
And yet, it was not with great ease that we rolled this out.
There were absolutely some sensitivities on campus among administrators,
among the universities Honor Council, who had long standing processes when
it came to issues of academic dishonesty, not only for CS50,
but all courses at Harvard.
The upside of course, is that by having a central body, Harvard's Honor
Council, adjudicate all of these cases, you have uniform processes.
You hopefully have more equitable outcomes overall.
And there was great concern initially in some circles
that we were now doing something more on our own internally.
And so it only debuted after quite a few conversations with Harvard's Honor
Council and administration so that we can ultimately
get folks comfortable with what, at the time, was an experiment,
but now is an ongoing six year policy for us at least.
BRIAN YU: All right so now six years in, policy
has been around for a little while.
Do you feel like it's done what you expected it to do?
How does it compare to what your original goals and objectives were
for what the policy would do for the class and for students?
DAVID MALAN: Yeah, so we hoped that it would actually
chip away at the total number of cases that we
were referring to Harvard's Honor Council,
but it did not in fact, do that.
Interestingly enough, the number of cases
we have referred to the Honor Council since have been roughly the same
or even higher in some years than prior to the Regret Clauses introduction.
We had the wonderfully successfully and nontrivial number
of students avail themselves of this clause.
Most years so in the court clauses first year,
2014, we had 19 students come forward under this clause,
reach out to me in the courses hedge, generally by way of an email first.
After which we would then schedule time to chat with me.
And I would chat with these 19 students one on one
and better understand what had happened and what had they done.
Better understand what circumstances had led
to them having made whatever decision it was we were then discussing.
And then ultimately, explicitly tell them, all right,
let's consider the matter behind us.
After zeroing the particular work in question
to reassure them that this was indeed the end of that process.
But the beginning, hopefully, of a healthier approach to future problems
sets.
And we would then encourage them to--
and discuss with them ways for better managed managing their time,
better managing their stress.
In some cases, too, it came to light that there
were extenuating circumstances.
Students struggling with issues at home, with their family,
with relationships, with other courses, issues of mental health.
And so what was a pleasant revelation to us
was that we were able more proactively than had been possible in the past
to connect students with support resources on campus,
whether academic in the case of tutoring,
or perhaps health in the way of mental health.
So that too seemed to be a positive outcome and the experience
that we were able to connect up to 19 students
that first year with other resources on campus.
And there after it fluctuated.
In 2015, we had 26 students.
In 2016, we had seven students.
Then it went back up in 2017 to 18 students.
And I think this variation is partly just a function
of messaging on our part, on my part.
How much time we spend in lectures and in emails
during the semester reminding students of the policy's availability.
I also suspect that there's some ebb and flow based on the current--
the given year.
If more students in this class know that a student in the previous year
might have invoked this clause there just might be broader awareness of it.
But it's been a good number of students, I think every semester.
However, the fact that we didn't see a downturn in the number of cases
we referred too was also a surprise.
In fact, in the first year of the Regret Clauses existence,
it turned out that most, if not all of the students
that invoke the Regret Clause did not even appear on our radar
when we ran our software based cross comparisons of their work.
Which suggested that had they not come forward,
we actually would not have noticed and they would not
have been connected ideally with these resources.
And so that too was a bit of a surprise.
These students invoking the Regret Clause
dare say composed a different demographic of students
that we hadn't yet previously identified.
Students who had indeed crossed some lines in many cases,
but that had not been connected with or been
offered some teachable moment that might actually help them course correct.
And I should note too, that of the 19 students, 26 students, and so forth,
not all of them it had indeed crossed some lines.
In several cases each year, were students unnecessarily worried.
And so I would simply reassure them and thank them for coming forward,
but not to worry, you've navigated the waters properly.
BRIAN YU: Yeah, it's really interesting that now
by reaching this other demographic, you've
been able to have these sorts of chats that otherwise may not
have been able to happen and connect them with other kinds of resources.
I'm curious as to what are the kinds of advice you
give to students that find difficulty with time management and stress?
Because I think this is not a unique problem to CS50
that and other computer science classes are just in school in general
or even outside of school.
Like, time management, stress, managing these things and making good decisions
is--
it's challenging.
And something that I'm sure many students and other people face.
DAVID MALAN: Yeah, absolutely.
To be honest, it's fairly straightforward things.
It's things that we even put in the courses syllabus or FAQs often.
For instance, in a programming class like ours, start early.
You have nearly seven days from start to finish for each programming assignment.
And the key to avoiding a lot of the stress
is to just start early, so that when you do invariably hit a wall
or encounter some bug that you just can't quite see, you can go to sleep,
you can go for a run, you can take a shower.
You can take a break from it and come back to it
some hours or even a couple of days later and have that perspective.
I mean even I found in the real world that I do not produce good code when I,
myself am under stress.
It's no fun.
It doesn't yield correct results.
And so really helping students realize that, it is a relatively simple fix.
They just really need to take charge and commit themselves to that.
Besides that, it's often a matter of referring students and reminding
them of the many resources that the course offers on campus, whether it's
the courses lectures, or sections, or office hours, or notes or tutorials,
or any number of online and in-person resources.
And just reminding themselves that you need to meet the course halfway
and take advantage of these resources.
And it's no surprise that you are struggling
if you're not availing yourself of at least some of these resources.
BRIAN YU: Yeah, actually it's always incredible to me
when on our problems at forums, we always ask students like, on what day
did you start the problems set?
And so many students respond like the day of the deadline or the day
before the deadline for a project that we wrote with the expectation
that it will take students a week to complete it.
And students are trying to do it like day of or day before.
It always amazes me the number of cases where that ends up happening.
DAVID MALAN: Yeah, so I think the more we can send that message even before we
get to the point of a student having regret
clause this conversation, the better.
I should note though too, that another surprise effect of the regret clause
was not even that we-- or the number of cases we referred didn't go down,
but rather at least in at least one year they went significantly up.
In 2016, and as you noted, is when we had 10% of the courses student body.
So this is 10% of the students taking CS50 referred to the courses--
to the university's Honor Council.
But to be honest that too was in part.
And I think our numbers since have been partly a reflection of our feeling
that when we do detect what appears to be a straightforward
case of academic dishonesty, plagiarism of some sort, duplication of code,
these days, I think I personally am even more comfortable referring the case
than I was in years past because we have given students an opportunity
to meet us halfway and reach out.
And indeed, as you know, in every one of the courses
problem sets this year on the form via which they submitted their work,
we asked them to check a checkbox to acknowledge their understanding
of the clauses availability.
And so at that point, if we are not only reminding students
each week that it's available and they are not thereafter
taking advantage of it, it seems quite reasonable,
I think, for the course to move forward with the more
traditional punitive process involving the Honor
Council to investigate whether indeed the line had been crossed.
BRIAN YU: I'm curious.
So we often talk now about like the line being crossed
and what it means to cross the line.
I'm curious about how you see this in the context of programming assignments
in particular. like if you're writing an essay
and you copy a sentence, that seems like very clearly copying.
But in the case of code if you copy a line of code
you see from Stack Overflow for example, if you're looking up like,
how do I solve this particular problem, and you incorporate a line of code,
that that might not be crossing a line.
So how do you think about where the line is in the context of a programming
assignment?
And how to teach that kind of thing?
DAVID MALAN: Yeah, it's a really good question.
And it's a common question, because I think
there's a perception among folks both in the software
world and non-software world that this notion of academic dishonesty
in a programming class itself is incompatible with the idea
of programming.
And I do very much disagree with that.
The lines that we prescribe to students, both in broad strokes
and in very precise bullets in the courses syllabus, essentially
try to teach students to be reasonable so to speak.
And what might that mean?
Well, early in the semester in CS50, we of course,
have students in C, and later in Python implement Mario's Pyramid.
So a sort of pyramid-like structure just using some ASCII art to paint that
picture.
And it involves ultimately like a couple of for loops.
It would be unreasonable for students to go off and Google or look
on Stack Overflow for something like, how print Mario's Pyramid.
That would be a search for the outright solution to the problem.
And surely it is not our intent to assess you
on your ability to Google a solution like that as opposed
to crafting it yourself.
However, it would be very reasonable for instance, to Google something
like, how write nested for loops in C. Or how print spaces in C.
Because it's actually not obvious to students one, how you can actually
have two loops and one nested inside of the other
using different counting variables.
And two, how to print would appear to be blank spaces on the screen,
not quite appreciating that it's actually just the SPACEBAR.
So I think it's very reasonable for students
and it is allowed in the course syllabus to look for short snippets
so to speak of code.
Where a snippet itself is one line, few lines,
but it is not the essence of the problem.
And so indeed when we do find that students
have crossed the line, what has happened is
we notice some curiosity about their code.
It's maybe very similar to another student's code
or it suggests a technique that we haven't
taught in the class or some syntax that's not consistent with what we
know students have seen in the class.
And so we ourselves might Google certain key phrases or portions of code
or comments that we see in their code.
And sure enough, it too often leads us to the very same GitHub repository
or Reddit post, wherein someone else has posted exactly that same code
that the student has copy pasted.
And so there too, the kinds of cases we are referring
are not the many, many, many students code
who very reasonably use these kinds of digital resources.
But the ones who use these resources, and then take
shortcuts to submission as by just copying and pasting many lines of code
that they see.
BRIAN YU: So other than the Regret Clause now,
which we've talked about for a little while,
have there been any other things you've thought about doing
or things you have done to the course in terms
of thinking about how to either address academic dishonesty when it happens
or to try to prevent it beforehand?
DAVID MALAN: We have.
So couple of years ago, we introduced the courses Brink Clauses, so to speak.
Which was a couple of sentences inspired by a colleague of ours at Princeton,
Chris Moretti, who gave us some really inspiring language that
encouraged students in the courses syllabus to write us late at night
just as they felt themselves being on the brink of making a poor decision.
That is to say, even when you and I and most of the courses staff
might be asleep and a student might be working late at night on their work,
it would be reasonable to assume that they
could get a response to a request for an extension for instance.
And so with this brink clause prescribed was
a mechanism for students to send that note to say, listen,
I really feel like I'm in a bad place.
And I worry I'm about to make a poor decision as by copying and pasting
too many lines of code online.
I'd like to discuss this tomorrow and indeed that's
what the syllabus asked them to do.
Go to sleep, don't submit your work.
We'll figure it out in the morning.
And just writing students to write us and meet us halfway
under that sort of duress was the intent of the clause.
Unfortunately, when it was invoked some number of times
that first year, based on the wording of the emails,
based on the conversations we had with students,
it really devolved into a backdoor to just extensions.
We did not believe, ironically, that most
of the students who were invoking this clause
were actually on the brink of doing something academic dishonest.
They were simply on the brink of not meeting the deadline.
And so we ended up removing the clause from the courses syllabus, .
Ultimately
But I'm glad we did try it, but this was one example of a measure that, at least
for us, in our context, in our implementation failed.
But I do think more compelling has been what we introduced a few years ago
in the spirit of the Regret Clause, but whereby we actually
initiate the conversations.
So it's not infrequently been the case that when
we've crossed compared so many students submissions that there's
a few cases that seem a little worrisome,
but it definitely doesn't seem like it's over the line.
We certainly wouldn't refer them to the Honor Council on that basis.
But we realized that this then would is an opportunity for us to maybe
go chat with those students now and say, hey, listen, you appeared on our radar.
We think it's because of the similarities between your code
and maybe some other students.
And we would leave the other student anonymously out of it.
But we would then ask the student, how did you get your code to this point?
Walk us through the process and let's figure out
how you came so close to what we worried was crossing a line,
so that you can just avoid it moving forward.
And so these interventional conversations,
as we describe them internally, I hope has actually
gone a long way to just helping students navigate the waters.
Even if they don't cross those lines, they at least now
are being more conscious and thoughtful about what it is they're doing.
BRIAN YU: And what do you usually gather from those sort
of interventional chats?
Like what sort of actions you find that students are taking?
Does is seem like there's some teachable moment there
that you're helping students with?
DAVID MALAN: I think so because not infrequently would it
be the case that two students were indeed working reasonably
on the homework assignment together.
But they were perhaps asking each other a few too many questions about code.
It wasn't necessarily entirely in pseudocode or in English,
their conversations.
And maybe one was being shown the other's code,
which is allowed within some circumstances per the syllabus.
But maybe a little too frequently.
And so as such, their work was just sort of over time,
converging to become one in the same.
And so given that we would have these chats within a week of them having done
that, it was usually pretty obvious to students like, oh, let's
not do that again.
And recalibrate their approach.
BRIAN YU: So it seems like all in all, CS50
has tried a lot with the Regret Clause, with the Brink Clause,
with these interventional chats that you've had with students.
A lot that CS50 has done with regards to the issue of academic dishonesty
and trying to create teachable moments out of that.
And trying to work within the university and with students
on how to improve that situation.
What do you think are the lessons to be taken away for other courses?
What can other classes do, either in computer science
or outside of computer science that they can do based on the lessons
that you and the course overall has learned
from these years of working with these issues of academic dishonesty?
DAVID MALAN: I think one takeaway has been just clarity.
Our policy in the courses syllabus is not short, but it is detailed.
And that's the result of a lot of situations
having arisen over the years, a lot of conversations
having happened over the years.
And so I am glad that we do documents so clearly for students,
where the lines are and what our expectations of students are.
Toward that end too, I think it has been a good thing that we've introduced
these interventional conversations.
Even if a course is not as involved in the mechanics of the process as we are,
they're not necessarily running software across compare your submission.
But when something does appear on the radar,
if a teaching fellow or teaching assistant does
notice some curiosity in the student's code, it's dissimilar to their code
last week or it's a little too similar to another student's, I
think just being comfortable reaching out proactively to those students,
not to impugn them, but rather to say, listen, we have some concerns.
We don't feel you've crossed a line, but we'd
like to better understand what you've done and how you did this.
So that we can steer you in the right direction moving forward.
That too seems a very straightforward, healthy and teachable opportunity.
And as for the Regret Clause, I certainly
think it's worth trying in other classes.
I think it certainly is completely reasonable
that a course, whether ours or anyone else's,
just clearly defines what steps students should take when they find themselves
in certain situations.
And prior to the forgot clause it was ill-defined.
What should a student do if they make a poor decision, especially late at night
and then they do actually regret it the next day or some number of hours later?
There was no well-defined process.
And while technically, there was nothing stopping a student
from coming forward and turning themselves in,
I can certainly appreciate the trepidation
that a student might have with taking that on not knowing
what the outcome might be.
Especially, if they assume it might even be time off from the University itself.
So I think the fact that we've sort of clarified
how to conduct oneself before you get to that point,
after you get to that point, and after we have detected as much,
is just only fair to students in the class.
BRIAN YU: I think there are a lot of very useful lessons
there in terms of what classes can start to do about this sort of issue.
Certainly, if any of you are interested in learning more about this,
we've actually written a paper, the two of us
along with Doug Lloyd on CS50s teams about economic honesty in CS50.
So we can provide a link to that if you're
interested in reading more about the policy and about the Regret Clause
and about other interventions that CS50 is made on these sorts of issues.
DAVID MALAN: Indeed.
The title is Teaching Academic Honesty In CS50.
If you want to Google something like that.
And if you're more interested in the software side of things and the cost
comparison of submissions, if you go to github.com/cs50/compare50 you'll be
able to play around with the open source software there as well.
BRIAN YU: Certainly, if you have any feedback about today's podcast
or suggestions for future podcast ideas, you can always
reach us at cs50.harvard.edu.
DAVID MALAN: This was CS50.