Alright, everybody let's have some super fun happy times today!
look my cute little face. Don't you like that?
So we're doing a how-to tutorial using PyMongo. So lets send some data
today from Python to Mongo and back. If you like the code it's going to be on my
Github which Mr. Fugu Data Science. If you like this and would like to see any my
other videos or get caught up on some material. Feel free at Mr. fugu Data
Science. Alright don't forget to "SUBscribe" and" turn on" your notification
bell". Alright we are sending data frames and CSVs
today from Python to MongoDB. We will then do some queries to show that
works and show you how to how you set that up .And then lastly, we'll be sending
data back from Mongo to Python. Wonderful, if you don't have Mongo installed yet I
suggest lets you pause the video and go to this link if you download it I hope
you downloaded the code here so you can follow along with me. and go on the
Mongo site if you're using "homebrew" with Mac like I am you can do a "brew install"
"MongoDB - community" let that download. Then you need to make a directory to
store your data. Because, otherwise later you're gonna end up having some errors
pop up and say what the heck's going on so you do: "sudo mkdir - P" and then
you do a slash data " /" DB and then you do follow that by a "Who am I" to figure out
who you're giving permissions for this file and then you'll do a "sudo Chown -
large "R" lowercase "V" and then you'll put the name of whoever who am is and then
you do a slash data / DB. and you're good to go and you now have two options to
start your service: you could first it'll be it'll be in the link it'll be in the
description after your download and they'll say do you want to have it
running in the background or after login or you turn on your computer or do you
want to turn it on and off each time and for me I'm doing it turn it off turn it
on each time. Where I'm using two terminals, the first one I start the
service calling "MongoD" the second when I open in a separate terminal and I say
all right run" Mongo" and then that's where I open up my on the command line
for me to just start doing my little operations. If you don't have PyMongo I
attached the link for the official documentation and you can scroll down
and pluck this and do that. You can also get the Anaconda version, pluck this and
take that. and if you need to pause the video just do it and get these taken
care of. The basic idea of MongoDB is that it's very powerful it is a "document
database" it's very flexible in the sense that it has "no schema" your files are
stored in the binary form of JSON called "BSON", each document you store will
be provided with its own individual ID tag. That will be "B underscore ID colon"
whatever the name is. A "collection" can be thought of as a" table" such as those that
you use in "MySQL and Postgres". One thing I would like to know is unlike those you
"do not have a support of a server-side joint". Alright, today we're going to do a
simplified version world we did not have any users and we're not logging into any
clusters if you do have that scenario or would like to create a user I provided
the links for both or you can create a config file as well as create a user. And in
alternate videos I've done that you should check out for "PSQL"
for instance. I set up some config files and each language is
different so not bear in mind with that. We're going to use PyMongo as "PYM" for
our interface and connection between Python and Mongo we will use "pandas" so we
can do data frame manipulations and save as CSV's. You can do the "OS" for looking
for files in our system or "importing CSV and JSON". I have this little function
that we've been using lately in other videos that you could check out, you know.
We'll go over today but, basically what we're walking over a directory looking
for either a specific file or a specific file extension that's the gist of what's
going on here. There's two parameters that are output the first one that you
can look at is say alright this create our data frame for the second one. Okay,
let's put together the: absolute path for the file we're looking for. Which is
this. So we're gonna say alright that's let's import this in and see what it looks
like, and we'll see that you have this unnamed "0" junk which is basically your
"row ID numbers" which I don't want. and I get rid of that with this where I
just take the first column and everything else. Now, it took care of that
and that's fine. To do any other processes I'm gonna dump this as a CSV
file with a new name and saying I fixed it. we're going to then need to store our
fake user information as a dictionary and you're referring to this as "Records"
there's another one called "columns" but that's not what we're using today. Here's
the first two entries of it we really have our "key" which is a "credit card
number" and then the "value" is the "number" and remember this is just all made-up
junk. I did in another video with generating fake data with "R" as
well as Python. We need to establish a connection, with the client so we're
calling "MongoDB localhost" because we're NOT logging in today. We're doing a
database that we created called, I created this prior which is called
"fake users" and then within this database you can create a collection on the fly
ad-hoc so you can create this and say alright we would like to call it "user
info" and remember this is our collection "aka table". So you can create this on the
fly and we'll say ok let's go inside of MongoDB and say show collections. Ok,
there's nothing right. Because, we haven't sent any attributes to this yet. We
haven't populated, so you say ok let's populate it. Let's run this again. Perfect.
Now it's in there. Let's say ok DB dot user underscore info dot let's find and
count all rows (db._info.find().count()). Perfect so that corresponds to this. So we got five
thousand eight hundred twenty six rows so that worked out perfect. Now, lets
makes sure nothing's goofy. Lets actually look and find one entry and see what it
actually looks like. So here's one entry and here's the "ID" let's generate it with
this, which sounds. Great so we know that we imported this in so what do we just
do and refresh. We took a data frame and we sent it to
MongoDB. Perfect, that's the first thing we've done: now we
will do a CSV from Python to MongoDB. So we stored that fixed file without the
row IDs, we're calling the first parameter for the function above where
we're calling the absolute path. We are then taking a look at these individually.
So we have the absolute path now this but now it's stored as a list and we
don't want that. So we store it as a strength so we could input it into a PD
dot read CSV, so pandas read CSV. So we're going to say all right what's going on
here. So we read this Oh, what's up with this right? So that looks kind of funny
is that going to be a problem. Let's find out we go to the CSV reader and we'll
say okay let's open the CSV and use the "r" parameter, which means read we use a
CSV dictionary reader, where we take our CSV that we opened. We create a header
with all of our column names and now we iterate through this reader and create a
dictionary. Where we then insert into our table. Which would be "user info table"
that we're creating. Okay, so let's let that go now let's check and see what
collections or tables we have now. Now, we have both. Now lets look at "user info"
table. Let's find one, perfect.
Then lets also count, so we have the same we're sending the same file just one
into CSV is a data frame. So that's why this is matching again for the rows
perfect. Now we can run a query of where we say let's find somebody with the
first name of "Bishop" and then that's call it from the table of "user info"
table with this query. And lets iterate through and see what we have as a printout
we only have one person with the name of Bishop. Okay, well what if we look
for Alex: look at that we have Alex Acosta,
Alex Guzman, a few other people named Alex. Okay so what we did was we said we
will query from Python to Mongo and see what we could get from that. So we know
that the interface works. and then we say all right how about we take some data
from Mongo and export it from Mongo
to Python. Okay, well then let's do this we'll do DB dot the table we're using
here which is user info table and once they find everything and exclude the ID
numbers. and then we'll put it into a data frame and we're saying this
parameter, this method "dot from dict" because the formatting that you're using
is going to be a dictionary formatting and we want pandas to be able to
recognize that and send it in. Alright, so then what happens if we keep this as one?
well you're only calling the ID numbers and we don't need that in this case. We
can also say, well lets take the first name of Alex again and exclude these IDs
and let's take this query and this put it into a list and then from that list
lets converted into a data frame and we're taking this dot from dictionary
again. and here you go. Perfect! So what we did was we did a query a query from
Mongo pulled that information into Python and then we were able to store that
information is data frames. Which is great and lastly if you need to take a
CSV file that you're using so you're taking your Mongo information and you
dump a CSV file then you're reading it into Python again. That's what this link
for this example is indicating and you can look into this on your own
leisure and figure out how to go about it and look at that code. As always I've
left my citations for anything that I've used today and for the reading you may
be interested in. and I would like to say "Please SUBscribe" to these videos "Turn On"
your notification bell, and leave a " Like" if you thought that this was useful
content. I look forward to hearing from their comments for any material you
would like me to cover, you may be stuck on, or have just pure interest in. And it
doesn't have to be computer science it could also be relative to data science
, statistics, mathematics or chemistry thank you for watching this video and I
appreciate it have a good one as always bye