It's Tipbox talks and we're here at the cancer and metabolism 2018 event by Abcam to talk to people
about reproducibility in science
Let's go
have you heard of the reproducibility crisis? Yes
I haven't
So I think that it's a major concern especially because nowadays that science is gonna be internationalized
even more there's gonna be this network worldwide and network so it's critical
to be able to reproduce of course yes okay yes the whole thing of doing
research makes some types of findings more difficult to reproduce than others
I've heard some stories and things that went wrong in terms of reproducing
So yeah I know a little bit about it
do you have any experiences of your own of trying to reproduce data or
struggling to reproduce data yeah there was a paper only one paper published on
a particular phosphorylation site that required a specific antibody and we
tried to reproduce it pretty important we noticed that several other labs in
the world were not able to reproduce our results because they used different
antibodies of just didn't do the optimization of the protocol that was
needed so nature did a quick survey 1,500 people and showed the 50% of
scientists failed to reproduce their own experiments and 70% have failed to reproduce
a published experiment what would you say was most likely the biggest cause of this lack of
reproducibility because there are a range of options but if you had to put something in the
top two, what do you think that would be? Some types of
experiments are more amenable to this but I think that people need to design
experiments that build in kind of durability type of analysis like how how
reproducible is this in different systems and they often we we try to cook
the systems that we're finding it this an area really study something and that
it's important to get down to a molecular mechanism you need to kind of
strip it down to its essence I think it would be it would be difficult to
pinpoint it on reagents because the reproducibility crisis as such shows up
in so many different yeah with using so many different regions I think
definitely having more detailed methods in papers would improve that a lot of me
from sound to regions
also sometimes just technical inexperience or technical errors we need
a platform to share definitely having more detailed methods in papers would
improve that but I also think sighs this whole slew of contributing factors one
of which is our refusal as a field to accept negative data getting published
and so I think if you're already skewing what kind of results are showing it's it
makes sense that we're less able to reproduce them because maybe they're
just a smaller fraction of what's really happening database probably with
standardized protocol would be able to publish more give more space about
anybody but it would say if it was for two hours or 24 hours
I think that's a good thing if it's in the Supplemental
I think I think as much as that education in this period early on in
graduate school and maybe even undergraduate curriculum but especially
first-year graduate students need to be taking classes on reproducibility and
rigor and despondent how to experiment how did you know what what is a
reproduction of an experiment what is that what is the envy which biological
replicate versus the technical replicate and all these things that you know just
just like the core the core values that we all as P eyes kind of take for
granted we should be directing our students towards this they should learn
this and should just be part of their that you need to validate your reagents
you know you need to reproduce experiments and then three times and so
on we kind of take for granted that they should know this and don't always just
like like oh this one worked and I'm going to show this one and they're three
experiments only one of them worked when we should like sharing footballs it
should be just like a recipe do exactly the things things the same way using the
it's a big question with where do you start but changing the incentive
structure in publications I think I don't know at which level that has to
start but yeah I think something has said I shouldn't feel my ways yeah yeah
so we are back at Abcam HQ I think we had some brilliant conversations yeah
absolutely there was a really good push towards
the need for education lots mentioned that as a young PhD student you just
kind of thrown in the deep end and validation is really something you ever
cover in detail and also it was interesting to see how much people
It was interesting to how much people wanted to see changes in journals so people really wanted the materials and
methods sections of journals to be changed to allow for better
reproducibility and experiments and that dude that wanted protocol
platform yes that we can share a detailed protocol without having to go
back to the reference who reference the reference you reference the reference
you know the find the actual source material
talk about what vendors need to be doing whether it's knock out validation
or testing multiplications or references or mass spec or whatever but that
feeling that vendors need to be doing a lot more before the antibody even
reaches them but not just an antibiotic reagents in general
yeah yeah tune in next time for more typica talks and awesome commentary from
Bill and Vix. See you!