Hey everyone, in this video I wanted to do a simple introduction on how to download
bioinformatics data and in particular I'm going to do an example for single-cell RNA seq data.
I'm going to walkthrough how you could download the FASTQ files, which is the raw data from
a publicly available dataset. This is kind of like the first step of some kind of bioinformatics
project that you might have to do. The first thing would be get the data. For this video, I'm going
to download publicly available single-cell RNA seq data from this paper in a nature journal. And this
paper has data available for healthy controls and patients with multiple myeloma, which is a type
of blood cancer. This data is available online. When I go to the paper, there's a section called
'Data Availability' that explains how you can find the data. When I click that section,
this paragraph gives me a link to ArrayExpress, which is the website where the data is hosted.
When I click this link, it says that ArrayExpress is soon going to be a website called BioStudies.
This is how the website looks for me now when I click to go to ArrayExpress and it says in
the future it might look something more like this. Either way, you can get the data on either
version of the website and it has the data on the righthand side in this blue box but for me,
when I want to download the data, I want to know the detailed information about where
each of these files are coming from. I'm actually going to download the data from this section here,
which says detailed sample information and links to the data. I'm going to click 'View Table' here.
When I click that, it gives me a detailed table for the different samples and for each sample
it gives me more information about what the data actually contains. It will tell me, for example,
in the column 'Disease' if that sample FASTQ file comes from a healthy control or from a cancer
patient. And then also importantly it will tell me the phenotypes of the cells. This has a couple
different cell types for the different samples. But this also contains at the end the links
to the data. There are two possible ways that you might want to analyze single-cell RNA seq
data. Maybe you want to download the raw data, in other words, the FASTQ files to a local machine,
in which case you can just use the graphic user interface to click and download the data to a
local machine. More likely, you're probably going to want to download the data to a remote server
and then do pre-processing or analysis on that remote server. I'm going to show in this video
mainly how you would download the data to a remote server. I'm currently recording on a mac and I'm
going to open a mac terminal window and basically talk about how you would download the data through
a terminal. Usually, a remote server is going to be a linux server but I'll also talk about
how you can download the data on a mac if you for some reason are analyzing the data on a mac.
Basically what you want to do is you want to create a folder that's going to have
all the FASTQ files per one subject. For me, I have already created this folder on
my desktop folder. I'm going to cd desktop and I had created a folder called 'sample' that's
going to be used in this video. If you haven't created that directory, you can create a directory
using mkdir and then the name of the folder. Since I already have created that folder,
I'm just going to cd into it, which means just to change into that directory.
And right now there's nothing in the directory and I want to download the files into this folder. For
sample one, there are four files for the same subject, which is an older female. When I look
at the assay names, we have two lanes of data and then three FASTQ files per lane and then we have a
S1 and an S3. It looks like this subject has cells from two different time periods.
Let's say I just want to analyze data from sample one. I need to download the three FASTQ files for
lane one and for lane two. This folder is going to have six files total. These three FASTQ columns,
when I hover over the little download icon, this one refers to the read one or R1 file;
this one refers to the R2 or read 2 file; and this one refers to the index read file or I1
file. There are these three files for lane 1 and then these three files for lane 2. For each of
these six files, I'm going to right click on the icon and I'm going to click 'Copy link'. That's
going to give me the link to download the data. And then going back to my terminal, let's say that
I'm on a linux server that uses bash. The command to download the data from the internet is going
to be wget and then I'm going to click control or command paste to paste the--or control V,
whatever, to paste the link to the FTP website that has the data. That is the only command that
you have to do and now that file from the internet is being remotely downloaded to my folder.
And it says it's going to take about three minutes to download. This file is about 1.5 gigabytes
large. Okay, so that took about three minutes to finish running and then when it's done, I'm
just going to check again what the contents of my folder is with ls and we see that now we have one
file downloaded into that folder. And since I was actually downloading it into my mac local machine,
I can also just check using the GUI. If I go to the sample folder, we see that I have this file
and it's about 1.5 gigabytes large. That's pretty much how you would download the data assuming you
have the wget command in linux. If for some reason you have to do this type of download on a mac,
the mac does not come with wget command automatically. You can--there are two options. You
can get wget like I already did, which would be through--possibly through homebrew. You would
download something called homebrew and then once you have that, you would do 'brew install wget' to
get the wget command on a mac. There's also a mac command that you can use called curl,
which you can also use to download files from the internet. It's kind of the mac equivalent
to wget or an alternative to wget. If you wanted to just use the mac command without having to
install wget, if you're using a terminal on a mac, then you could do--the command would be 'curl',
then you're going to again paste the url, then you're going to do dash o, which stands for
output, and then give a name to the file. I'm just going to name the file this. And you would run
that command. Since I already downloaded this file, let me try putting I1 instead.
I'm going to say to download the index file. I'm going to click enter and again it's just
downloading to that same sample folder that I already have the read one file in. Okay so that
second file took another two minutes to download and then when it's done I'm just going to type in
'ls' to check the contents of my sample folder and we see that we now have two files in that folder.
The first example I showed how to download the data from the internet to a folder either on a
remote server or just through the terminal using wget. That's a linux command but you
can also run it on a mac if you have it installed already. Or you can use curl if you're on a mac
and that was the second example that I showed. And so once you have the three files per subject,
if you have the R1, R2, and I1 files, then you can use those FASTQ files as input--so that's
raw data from single-cell RNA seq analysis that you can use as input to the command 'cellranger
count' and then that would do your read alignment step. That would be kind of taking you through the
pre-processing steps for single-cell RNA seq analysis. You would use this if you want to rerun
the data independently or if you have some kind of novel method or analysis that you want to run on
the data and you want to do the pre-processing from the very beginning using the raw data.
And that's all I have for this video and thank you guys for watching!
*chiptune music*