Now let's step back a minute and think a little bit more
about CSV. We've talked about the fact that fields are delimited by
commas. So what happens if we have a field that actually has
a comma in it, like for example, this one. This particular Beatles
album was released on two different labels, one in New Zealand and
one in the US, so the way this data set has been
set up, those two different labels are simply separated by a comma
here. Now based on what we know so far about CSV, or
I should say what we've discussed so far in the
class about CSV, this would cause a problem for us
because our parser would interpret this as a field separator.
Now, the way that the CSV format actually handles this, or
the way that most applications that deal with CSV format
actually handle this, is to do something like the following. So
you can see here that this is the field in
this actual CSV file. Over here what I've done is simply
load it inside a Google sheet, but here's a raw
CSV file. And you can see that the way this
has been structured is for this particular line, this field
has been enclosed in quotes. Okay? So what that does is
indicates that you can ignore field delimiters from here to
here. So, we've got some choices in terms of what
we use for quotes, you could use double quotes or
you can use single quotes. Well, that would cause a problems
in other ways. You can see here that we have this
quote character here, single quote here. There's also one here, which
is actually used as an apostrophe for Sgt Pepper's Lonely Hearts
Club Band. So it would be extremely tedious if in our
Python programs we had to deal with all of these different
variations of exceptions. And the fact is that though we call
it CSV, or comma separated values, you can really use any
delimiter you want here, as long as that character is only used
for a field delimiter in rows in our dataset.
So as, so often is the case of software
development, this problem has been abstracted away and solved
for all of the different variations, the tedious details that
we might have to deal with in order to work with the format like CSV that has so
many variations, and asterisks as my friend, Will Cross
is fond of saying. This is the Python CSV module.
This module deals with CSV formats in a pretty complete
way. So, let's look at how we use this module. Now,
what I'm actually going to do here is use the DictReader
class from this module. This assumes that what we want to do
is read all of our data into dictionaries, which is
what we've been doing all along, and what we'll kind of
continue to do throughout the rest of the course. But
it has some other pretty cool features as well. For example,
it assumes that the first row of whatever file we are
going to read is actually a header row. And that those
are the names we want to use for fields. So, going
back to our CSV file, if I scroll up to the
top, we can see that this first row here is actually
all of the field labels that we would like for the
columns in this data set or the fields in this data
set. So, what this dictionary reader will do for us is as
it reads in rows, it will create a dictionary for
each row. The field names will be whatever it found in
that first row, and it remembers them as we read
through the data file. And the values then will in turn
be each of the associated values on each line of
the file. And again, it also handles things like dealing with
quote characters, dealing with quoted fields that may commas inside
of them, and so on. We don't have to worry about
that at all, using the CSV module. So let's take a
look at the rest of this code. Essentially we're just opening
up the data file. We're instantiating a DictReader from the CSV
module, and then we're simply looping through. Each time through here,
this class is going to produce a line for us. And
that line will actually be a dictionary, composed of the appropriate
fields for that particular line. So then if we scroll down,
what I'm going to do here is simply print out all of those
values, okay? So, let's take a look at running
this piece of code. And again, remember we're using
the CSV module. Okay, so if we run this,
the output we get, we'll just look at the second
to last one here, is a dictionary composed of
each of the labels that came out from the first
line of this file. And the field value for
each one of the fields for this particular row from
the data file. Okay? And it seamlessly handles for
us fields that may be quoted on a particular
line, and other nuances that we might see in
the CSV format, and conveniently stuffs everything into individual dictionaries
for us. So, whenever you're working with CSV files
in Python, it's best to use the CSV module,
because so many of the challenges of working with
this type of data have already been solved for us.