Michael Cherry: Okay, good morning.
I’m Mike Cherry from Stanford University and my group does the data coordination center
for ENCODE Project.
Unfortunately it’s not -- it’s covering too much today, so I won’t give you an interactive
demo of this, but hopefully I’ll point you to enough information that you’ll be able
to do that on your own.
We hope that we’ve done enough as far as creating tutorials and such that you’ll
be able to do that.
Today -- so what I’m talking about right now is a little bit -- so is the data accessibility
and how to -- in the processing that we do at the DCC and introduce the standards that
the data is put through.
You will hear a little bit later, after me, actually not that long from now, about how
to interpret some of the data as well as the analysis that comes at the very end.
Okay, so the goal of what I’m telling you today is so that you’ll learn, at least
in concept, how to download data, where to find the data, information about the features
of our portal, the understanding of why we create metadata, maybe not what the metadata
is all about, as well as a real cursory introduction to the processing pipelines that we have created.
Okay, so as you’ve seen the ENCODE is the collection of assays of characteristics of
the genome.
These assays put together are used to create -- associated together are used to define
where promoters and enhancers and such might be.
The assays themselves are just measuring various characteristics of the genome, so this is
accessibility of the DNA and where particular proteins are found.
So I want to focus my talk on this particular issue [unintelligible] that came up in this
particular issue about the reproducibility of research.
So of course this is a big problem and we’re all sort of -- sort of -- want to make sure
that the data we provide is very standardized and reproducible -- reproducibility.
Nature came up with -- it’s a collection of articles that have come out over the past
couple of years; there is a URL at the bottom, you probably can’t read it, but I think
if you Google that very top line you’ll find this issue.
So some of the big worries, concerns that are pointed out here, is that the antibodies,
the problems [unintelligible] about this -- you need quality control and particular measures
of confidence; you want to have -- make sure that there’s appropriate replication designed
into the experiment; the data needs to be accessible and not hidden away somewhere,
and that there needs to be really a standardized, unique identifiers for the data sets themselves.
Okay, so I’ll start in on the accessibility of the data.
It’s been mentioned a couple of times, the www.ENCODEproject.org is the main portal for
the site.
You can basically find everything that the project has; it’s a very, very transparent
project; everything that we have is available from this website, this will include documentation
of our standards, publications -- as Mike had already mentioned.
If you go to this website now, really quickly, before I move on, you’ll see that there’s
mainly four menus at the top; you can get at the data that we provide, you can understand
what we’re doing because of the documentation we provide, lists of publications, and in
particular this -- the one on the farthest right there, the menu that’s “help.”
So we’ve been putting together large sets of tutorials and guides; we certainly need
to develop that more but I think you can get into the project pretty quickly by going there.
In particular, as Wang mentioned, that there was a users’ meeting last summer where there
was a three day meeting with lots of tutorials and presentations.
You’ll find videos of those there, so they’re not your common little five minute tutorials,
it’s actually the whole talk, but you’ll hear from Jill and the others in an hour -- format
of what they actually have done in a much more rich way than we’ll get in a short
presentation today.
So I encourage you to go there and we’ll also have the videos and everything from this
meeting, too, to go up there and of course there’s always a search bar for getting
at information and if you really just want to get in there quickly, don’t know where
to start: there’s a “quick help” down at the bottom.
Okay, so the main look of the page, if you go in and click on -- from data, go to “assays,”
you’ll get this page, this is the starting view, you’ll see all of the experiments
that have been provided, that have been collected by us -- I did miss one little thing up here,
sorry: the total.
So to date we have over 5,000 data sets, experiments that have been available, so that’s more
than 5,000 files because of the replicates and such; this includes a large number of
bio samples, so the biological material that has been included and of course there’s
hundreds of terabytes of files that are available for download from our Amazon [unintelligible].
As of last week we’ve included the genetics roadmap metadata into this site.
It was quite a project to synchronize the metadata on these, so we’ve taken the roadmap
metadata and incorporated it into our site and had to do a lot of work to map it up to
the standards that we use for ENCODE itself and so that allows me to remember to say that
here, at the very bottom of this page you’ll see -- you’ll have the ability to switch
between projects, so you can -- you’ll find at the very bottom a “projects” heading
and you can click there to see just the roadmap experiments; by default you come in and it’s
the ENCODE experiments.
Okay, so this is called “faceting search,” which is basically a way of filtering.
So we have large amounts of metadata, very rich, hundreds of fields that are available
for each experiment and what we’ve done on the site is create these facets, sort of
pre-computed filters that you can do, so if you click on these numbers there, you know,
like on a shopping site, you reduce the total number of matches as you go forward and so
in this example here, we’ve clicked it down to a smaller set so there’s only six experiments.
At this point or any point along the way you can click on the “download” button; it
doesn’t actually download the files immediately; it gives you the URLs that you can use to
retrieve those files because you may have many hundreds of thousands you don’t actually
-- not hundreds of thousands, tens -- but you probably don’t necessarily want to download
those files to your Mac or whatever; with the URLs you can take that and you download
from any command line.
We also give you a short summary of the metadata associated with those files.
There’s also a “visualize” button and you can use track hubs to the Stanford browser
to actually display these particular experimental results on the [unintelligible] browser as
well.
Okay, so that’s the biggest feature of the ENCODE portal as far as getting at the data.
It’s normally where we want to spend a lot of time on an interactive session, but I encourage
you to explore that and hopefully the tutorials are there.
If you need assistance from the DCC in any particular way, there’s a -- on the site
there’s a help desk email; I certainly encourage you all to basically tell us anything, if
it works, doesn’t work, how can we help you.
Okay, as you dig deeper into the individual experiments, you’ll see that we’ve accessioned
basically everything, so that -- everything about the experiments have been accessioned
and so in this case it’s an antibody -- has been used in an experiment, so the antibody
itself has a record, the bio samples have records and these are all accessioned.
So we’ve accessioned things by categories; it’s basically a shortcut so we can sort
of remember these license plate numbers.
So it’s “ENCODE” at the beginning, then there’s like “FF” for “files,” there’s
numbers and then letters, so all accession numbers look in this way so it’s easy to
find; like a particular aspect of this is that you can share the accession number with
somebody else; the accession number may go to a file, it also is associated with the
metadata for that file, so you really only have to give somebody the accession number
and then they can retrieve whatever you want them to.
You can also share a list of accession numbers and that allows them to download the same
files that you have, so you don’t need to, you know, share some complicated file name,
directory path, such as that, you just need the accession number itself.
So I’ve talked a little bit about a few of these already.
We need to have high-standards for the information that comes in, we want to be able to have
replicates of -- designed into the experiment itself and part of the standards is to create
metadata and this is really the amazing part of the ENCODE Project, is that the laboratories
have committed to sharing the information about their experiments.
Okay, so those of you in the wet lab, you know that you write down in your lab books,
you know, how you made the solution, where you bought your products, how you did the
extraction of your DNA, you know, what speed you ran the centrifuge at, all this kind of
information.
Normally when you read a paper you won’t find this information, right?
They’ll say, “Oh, we did it with a method that we previously used, published here.”
You go to that paper and it says, “Oh, we used this method that was mentioned on this
website,” but it references a supplemental site and it was done with changes not published.
Okay, right?
So how are you supposed to reproduce that experiment?
Read their minds, call them up; the student has already left, the lab notebooks been lost,
right?
So there’s a commitment from the ENCODE production labs that they provide us all of
this information; it’s a requirement, okay?
So this is incredible, right, I mean -- so it’s just part of the process of ENCODE,
is that all this metadata is transmitted to us and is made available, stored in the database;
you can retrieve it all.
Built into the standardization that’s done is that the experiments, the particular assays
-- the experiments have to match a particular standardized methods approach and this will
include, for many of them, replicates.
So in this particular experiment you see that the same bio sample -- two different libraries
were made from that bio sample and each one of those libraries was sequenced independently
and at the bottom there, what’s showing is, there’s an accession number for each
of the FASTQ files associated with those replicates, so you can download the replicates of everything,
again -- so the transparency of sharing all the data together.
Again, I have to really congratulate all the labs for their commitment to doing this; it’s
a lot of extra work, there are standards as far as the quality of the data, the [unintelligible]
of the data, of course they’re providing all of the metadata -- it’s typically something
that most labs wouldn’t want to do; here it’s just the requirement of the project.
So the whole point of ENCODE for us is to create the highest-quality standard of results,
of metadata, of all the processing; so the point is, is you really can’t get better
data than what you can get out of ENCODE, okay, it’s just not going to be there.
It’s really -- we often get asked, “Can we incorporate our data into the ENCODE portal,”
and the issue is the level of these standards.
When people say they want to do that, I know they don’t approach us, they approach the
labs typically and the labs will tell them what they have to do and I don’t think to
date we’ve gotten more than maybe one person that says, “Oh yeah, I can do that.”
It’s a really high bar that’s put on the ENCODE labs.
Okay, and so moving forward with some of the other aspects of this: we want to have control
of the confidence and reliability of the samples and part of that goes to the pipelines that
are created.
Okay, so again, like you’re -- if you’re on the dry side or not, you might meet in
an experiment that the data was processed in a particular way, using methods published
by such-and-such with changes here, “We substituted this version of that,” and all
this, right?
So you oftentimes don’t really know what the processing was involved, or they say “processed”
and they give you some name for a pipeline of things which you don’t actually know
what goes on inside that.
So avoid this sort of pipeline black box -- sort of point -- as part of ENCODE -- so we want
to have everything completely transparent, so the pipelines have been defined and are
run on each type of sample from each data type that’s submitted to the project by
my group and to do this, each one of the data-producing centers has come together with the others
in creating the same type of data, to define a pipeline to be used.
Now, this may not necessarily be the best, trendiest pipeline that’s out there, but
[unintelligible] the pipeline analysis itself is research, so -- but what has been agreed
upon is a very high-quality, standard pipeline to be used and it’s used for everything
of that particular data type.
So this is -- avoid the problem where you want to compare data between different labs;
you would have to actually understand how they process the data, very specifically,
to know is it really appropriate to compare the data one for one.
So all data from ENCODE of a particular data type can be compared because it’s all been
run the same way and has the same standards for production; so everything is really the
same and so if you want to understand the pipeline, you can look at a page that describes
the pipeline itself.
Each one of the boxes tells you a little information about the staffing, which software has been
used, and of course with -- you have metadata for the sample, for the files, you also have
the metadata for the processing of the files.
So this describes if, on a particular experiment, do you want to look at the results?
You can see the processing that was done on that file here and so, in this case, you see
these triplicate, duplicate boxes, one on top of the other: the replicates involved
in here.
So each replicate was processed and then joined at the end, so each replicate’s processing
is also shown here and if you -- what it’s showing in the upper right there is, if you
click on one of the boxes, you get more detail about what happened underneath that; you’ll
see what versions of programs that were used and sort of more detail flowing into there.
We track all of this; this is really rich metadata that’s provided and as with the
data coming in and flowing out of the pipelines, if integrative analysis has been done on these
files, we have that data coming -- that we have the results of those analyses coming
back into the portal and the method that is associated with what was done in the computational
labs is also included.
So -- very last thing here is the antibodies; if any of you have worked with antibodies,
you know it’s kind of a little bit of magic how they’re made; it’s a huge problem
because the reproducibility of antibodies from one body to the next is -- can be quite
variable and so ENCODE, again, standardizing everything to really the extreme level, requires
a lot of extra validation work to go into that.
So what you’ll see for a particular antibody here -- that it’s been determined to work
-- there’s the western blot provided, okay, so how many times do you look at a paper and
they’re talking about an antibody and they show you their western blot, why they think
that antibody works?
The ENCODE labs do that for you.
So this is a huge amount of extra work, lot of grumbling because it is so much extra work,
but it’s a requirement of the project.
You know, you -- I don’t know what the exact numbers are, but it’s a huge number of antibodies
that just never pass; it’s really a small number; they really have to process lots of
them to get the few that work, okay.
So I hope you’ll see that because of the standards and the transparency and the agreement
between the labs and both the wet labs and the dry labs, that ENCODE really provides
you the best possible data that you can -- you can get anywhere and so then that allows you
to compare your results with that, knowing that, you know, if you find some variation
between your data and the ENCODE data, you can explore it to the metadata and find out
what might be happening there.
We’ve hopefully reduced the variables associated with all this and then the last two things
are -- so as far as it’s an advanced workshop and some of the things you’ll hear a little
bit later, you want to start processing data coming out of ENCODE -- we have created programmatic
access into the portal.
This is described a little bit in the help section, under the tutorials and the “help”
there.
There is an API which allows you to get at all of the metadata; basically everything
in the database is transparent.
The facets are only a small amount of that metadata and that’s -- the facets were created
for user interface, usability for people, but you can -- with a computer you can get
at everything.
There are examples of Python scripts that you can use and get into these files and so
you can develop -- if you’re a programmer or can twist somebody’s arm that is a programmer,
you can really get at all of this information in ENCODE, pulling it back as you want and
the little example here: if you looked at the slides online when they’re available
there, it’s basically -- you just add a little tag at the end saying, “Give me this
information in JSON format.”
So the -- everything that you see on the page that’s -- the information that’s used
to build the page is actually available as a JSON format, so when you do a search and
you see a long list of experiments there, you can get at this from the JSON objects
themselves.
I really -- I skimmed over the way that we standardize information, but -- that we use
ontologies and so using the ENCODE standards for the metadata has allowed us to incorporate
information from other projects because we’re using ontologies.
So for example the Uberon ontology, which talks about anatomy, cell line ontologies,
assay ontologies and such, are all -- are used and that gives a standard way of going
between experiments as well as projects and we’re starting to incorporate other types
of information as well, as Mike and Louise [spelled phonetically] tell us we should.
So -- and then this is all coming out of the port, okay?
And so the last thing is the group, really got a great group of people at Stanford there.
The second row are the wranglers and the assistant curators that help all this information; they,
you know, in a sense have to touch all of the data coming out of the labs; they work
with the labs to sort of help them remember that particular fields of metadata need to
be incorporated.
There’s a lot of effort really going into matching the design of the experiments so
that the types of replicates, how the metadata is described -- and then the bottom level
we have the folks that build the pipelines, maintain the software we use within the group
for analyzing the data, just pushing things back and forth.
Two great people have created our website, really fantastic pair there, as well as the
fellows that do the website itself, the U.I. system managers, and then the secretary for
the group and I thank you very much.
[applause]
[end of transcript]