>> So we were just talking about how we can
use Python to work with data files like CSV files.
Let's try it from an actual notebook.
So I have here a notebook in front of me,
and because I want to read the data into a pandas DataFrame,
because when we're working with Python,
DataFrames are very useful for data science.
I'm going to need to import that pandas library again.
You can get used to that line of code, I'm thinking.
Let's say I have a nice CSV file containing airport information,
write the airport's name, city, and country.
You can actually see that actual row is in my CSV file itself,
as long with the names,
cities and country for a number of different airports.
So I can simply use the Read CSV command
to read that data into a DataFrame,
and then display the contents of that DataFrame.
So sure enough, you can see that
the column names were read successfully from the file.
You can see that index values
were created as Christopher was explaining.
Panda DataFrames, you always have an index for each row.
In this case, because it was none provided,
it generated them for me.
So everything looks great.
But when we are writing Python code,
we have to make sure that that code can handle
different variations in the data file that we're loading.
For example, I might have a record here,
wherever row for Heathrow London has an extra comma in it,
and that extra comma is going to mess up my code because
it now thinks there's four values in that row is a three.
So by default, if I just try to read that file,
it's actually going to crash,
and it won't put any data at all inside my Dataframe.
Well, I have a couple of options here,
I can now go digging through that file,
try to figure out where is the one row
of that's causing the error and try to fix it.
Yes, there will be some hints in the error message
to help you find which row has the problem.
But one of the neat things about data science is
sometimes it's okay if we've got a million rows,
maybe we can leave 12 of them out.
So maybe we just want to skip the bad rows.
So in that case,
we can use Python supports that as well.
So what we can do, is we can specify error bad lines equals false.
That simply means if you meet a row that you can't interpret,
skip that row to continue.
So if I run this one,
you'll see it successfully loads the records,
and that record from London for United Kingdom was simply skipped.
So that record was not loaded.
The other common situation you'll encounter when
reading data files with your Python code is,
the file may not contain
a row that tells you what the column headers are.
So here, I have some data,
but there's no row telling me what the column name should be.
By default, if you pass this in,
basically Python and the pandas
are going to get a little confused,
because they assume you are
passing in values for the column headers.
So it reads the first row of data,
and thinks those are the column headers.
So suddenly, I have a column header called Seattle-Tacoma,
and Seattle in USA.
So I need to make sure I have a way of saying, "No,
the first row in the file is actually data, it's not headers.
"That's why we have the option of specifying header equals none.
So if I specify header equal none when I load the file,
that's a way of saying there's no header row,
and I simply want you to assign values to those column headers.
If I don't tell then what values to assign,
it'll just give them a number value,
column 0, column 1, column 2.
Now you might have appreciated when
crystal was doing some queries against the Dataframe,
that sometimes it's nice to be
able to specify the columns by name.
It's easier for me to remember that city contains the city,
rather than column index 1 is the city.
Even if the header values aren't specified in the file,
you can specify it by using the names parameter,
what column names you would like to use.
So if I run this, you'll see it actually successfully said,
"Oh, you don't have a header row."
So here, specify a value name,
city, and country for my three columns.
The other scenario you're most likely to run into when working
with CSV files or Datafiles in general, is a missing value.
In this case, I have a data file.
For one of my records,
shuffle in the Netherlands,
there's no city specified.
You just see comma, comma.
Now if you read that,
the way Python pandas will display a missing value,
is it displays it as NaN,
so not a number, if you will.
So that indicates that there was no value
found for that particular value inside the record.
So you'll see that as well.
Now sometimes when you start really doing a lot of data science,
and you do a lot of cleaning of data,
you'll see that when you look at data science courses,
once you finish cleaning the data,
you might decide, "Hey, now that
the state is all tidied up and cleaned,
maybe I'd like to save a copy of it,
so the next time I'm working with it,
I don't have to do as much cleansing and rework."
So one of the other neat things we can do with pandas,
is we can actually write
the contents back out to a CSV file as well.
So if I have data in a DataFrame already,
then I can simply say, hey,
let's use to CSV,
and let's write that output to a data file.
Now, one of the things that's a little
interesting if I was to open that file up,
and I'll show it to you in a minute,
is the index values will be written in as well.
So if I don't want the index value,
that zero, one, two, three,
four to appear in a CSV file,
then I want to specify when I write the file,
that I want index equals false.
That's a way of saying please do not include the index values.
So let's go take a look at the actual data files created,
so you can see the difference between using index equals false,
or not specifying index equals false.
So here you can see,
here is the file I created by default.
You can see it added the index values from the Dataframe,
and you can see when I did specify index equals false,
that it simply added the row values itself without an index.
So there you have it. You now have
the ability to use your Python Code,
to move data to and from CSV files, and pandas DataFrames.