welcome to the data science with MATLAB
video series in this video we'll focus
on importing the data which is the first
step in the data science process in this
case we actually have a bunch of CSV
files and I have them in this storm
event data directory so you can see
there's a pile of them and they go again
from 1980 to 2017 and so if you're just
wanting to explore the data and start
bringing it in you can use the import
tool and so this is great for reading
one file and again just kind of looking
at what you even have alright so you can
interactively explore you can choose the
data types that you want to bring it in
as which we'll talk about later so if
you want in this case it's going to
bring it in as a table you can choose
individual data types or individual
names and so again this helps really
explore and choose how to work with some
of this data for example if MATLAB
wasn't able to read it it will replace
it with an an
and a great thing is that you can
actually generate code from this so if
you do this once and you have many files
like this you could actually generate a
function like this and then use that
over and over again to bring in the data
so there's also if you're reading just
one file there's a few nice functions
for this so for example read table and
so this just brings in that data just
like you saw previously with the import
tool and it just reads it into MATLAB
for me and so again this is great if you
have one file but in our case we have
multiple files spanning over those
different years and we need to figure
out there's all they're also organized
in the same exact way and so there's a
way that we can do this that's called a
datastore and so a datastore is a great
way again you know anytime you have
multiple files or extremely large files
or you know databases with a bunch of
data you may not be able to just read it
all into memory directly or you may not
want to and so a datastore is a great
way to manage that in MATLAB because it
just really points to the data and then
you can kind of choose how much data you
want to access
you know what types of data you can
control a lot of that and so we'll do
that for this example in our storm event
data and so I've created a datastore and
as you can see it's pointing to a number
of files and it gives me some shows me
some of the options that I have and so I
don't really want to take I don't want
to think about all of that right now
I want and that allowed to do some of
the work for me so I'm just going to get
a quick preview of the data and see what
types and what organization MATLAB has
decided based on the files I also want
to make sure that all of the files that
I want were represented and so I can do
this by checking different properties of
the datastore and so here I can take a
look at all the files listed and sure
enough they go from 1980 to 2017 and so
to start out I don't necessarily want to
look at everything right away I just
want to look at the most recent data so
I'm just gonna go back and select 2016
and 2017
so that takes care of the files or the
dates that I want that I'm interested in
I also want to choose which data
properties or which data variables that
I want and other properties about those
so there are 51 of these variables again
there's many representations of time I
really only need one or two of those and
so I can preview again and make sure I
have just the data that I need and take
a quick look so I've got the end
locations and time begin time sources
and our event narratives and so as you
can see there are many different data
types included here so we want to make
sure that we're using the best data
types for the job MATLAB will try to
make the best decision but you also have
control of this through the data store
itself and so for example I can create
categorical variables for the state and
the event and just ensure that the date
variables are using the right types all
right and so now I've got the right data
types I've got the right variables
selected and now I want to decide how
much data to bring in and how to work
with that so with a data store you have
several options there are so we saw a
preview that reads in just the first few
lines so you can get just a feel for a
quick feel for the data you can also
read a subset of the data so you can
just use read and that'll use the read
size which I'll show in a second you can
also read all so if you want to just
bring everything from the data store
into memory go for it or you could use a
tall array I'll actually show this a bit
later where you actually work with the
data out of memory and so you just
pretend like it's any old MATLAB
variable but it's actually pointing to
that out of memory data so for now I
want to just read in a subset so I can
carry on my analysis you know as I
usually do and so I want to check the
default read size which we can do in
this case it's 20,000 lines and so if I
use read as I'm doing here this just
brings in the first 20,000 rows the
second
call to read will you know take the next
twenty thousand rows and so on and so
that's a great way to just manage you
know taking small subsets and doing
analysis that way in this case since I
only used I only selected two years I
can actually just read all of that in I
know I have enough memory to manage that
and as it turns out there was one
hundred and five thousand or so lines so
that's a manageable set and I'll
continue through my pre-processing now
that I've read the data thanks for
watching in the next video we'll look at
pre-processing the data now that we've
imported it