Welcome, everybody. You're watching Mr. Fugu Data Science. Today, we will access
nested data: using Mongo lite.If you're unfamiliar; this is working in
conjunction with R to interface with MongoDB. We will do some queries,
aggregation, and MapReduce. All the code will be provided with the link in
description below, to my Github. So don't forget to subscribe, and turn on that
notification bell. And please like this video. If you don't like it:
throw me a comment, and tell me why before you decide to throw a thumbs down.
I would appreciate it. We need to use the tidy verse, knitter, markdown, and Mongo
Lite libraries. Here's refresh on what this data looks like. We used this in
previous examples in our MongoDB series. All right, so what's going on here? If I
was a recruiter and I had a database full of clients, and each client is a
candidate. Where we have a first name, last name, a list of their skills, the
state they live in, specialty, their experience level, and are
they willing to relocate. Alright and you say, well how is this nested? Well because,
to access the data for each candidate you have to call candidate dot, and then
whatever parameters you would like to access. Now, we need to set up a
Mongo connection. So I don't have any user credentials or anything like that,
SSH connections, nothing like that. If you have those, you need to refer to
the documentation for MongoDB and Mongo lite to set that up. We're not doing
that today. We have a collection; where my collection is called, recruiter clients
and my DB is called Berkeley. Now, we create our connection. But, you have to
pay attention here: and I wanted to write this I have a terminal open currently
for Mongo D. Which is creating my connection, and then I'm able to access
this. Because, I don't have it running in the background all the time. I have to
call it each time. That's just how I set up my interface. So
if you have the same situation, pay attention to that. Otherwise, you're gonna
end up with an error. So what we can do is we could call a Mongo connection and
we could find everything with a query. But, there's something interesting here.
so I do a dee plier 'dplyr' glimpse. and I have to call the candidate. This is similar to
if I was in the Mongo shell and I did whatever the DB collection dot and then
this is replacing the dot basically. So I called head. And this is all of my data
and that looks perfect. But, what happens if I did the same thing and I just
called n. Well, you have something goofy here. Let me ,let's
go back and close all of this real quick. So we could, oh wait not that one. so
let's close this. and that's real fast see what happens. So here you have an
issue: because it's saying well you have in length seven. But, you're only
calling one thing and that's why I have to call this candidate; to actually
access the data. Okay, that's the issue with this. When I'm calling anything that's
nested. Because, it recognizes that there's other things that are attached
to that. But, I also have something here from: the last example that we did. I also
did an update. and I allowed the modification to be shown. and so that's
what's going on here, that's what's showing. Enough of this playing around.
okay lets call our candidate. So just like if you're in the Mongo shell. Lets
look for a first name of someone named Brenda. and I say alright candidate
dot first name. Then I call the person named Brenda. And I say I want to do a
field, where I would like the first name to be true. and just call that. And this
is basically, your query. You can also write just like this, and it's going to
do the same thing. But, what's up with this? I have this ID number instead of
any actual data coming up. And that looks kind of weird, right this was throwing me
off with this stupid library. and I was like hmm.
I don't understand why I'm getting this printout. Well you have to do something
interesting. We can still run a regular count in it and it provides our 500
documents that's correct. But, I run my query, and I find all fields.
so it's just basically telling me, I want to run this query and get all of the
data associated with all the columns basically. Well I had to assign a
variable so I called it Brenda. and I need to call a candidate: because, this is
nested. So what else would be associated with Brenda? I could call the ID as well.
So we call the ID. We could pull the ID numbers, that we found right here.
That's pretty useful. Okay, what happens if we want to exclude some of
these fields? Well we could just say this just take the first and
last name; and you have to call true. and then we'll call the last name. For
instance; if you didn't need all of the columns. First name,
Oh we need the candidate. there we are.
So since this is nested, I had to do the candidate dot first name, which is true
and the same for the last name. If I wanted to show any particular fields.
Then we could query it. and say this look for only people who know spark. All right
so then I run my query, and I want all the fields. This right here is
letting me look only for people who know spark, which is here. Okay, perfect.
Then I want to find a candidate who has the skill of C++. But, not only C++; at
least C++ ,or including C++ I should say. So we're running the same thing ,here.
Except we don't have the the brackets. So you can see C++, C++ for all
of these. Okay but what if we have an ID that we want it to call? So we wanted one
particular person, and we know their ID at the time. That's okay, you can call the
ID, and then your colon, and then you'll need to use the dollar sign, and then oh
I D. And then you just call it if you have it, and print it out like this.
Alright, and if you notice something looks interesting for all this stuff
like we're doing. We have our single quote, then we have our curly brace, and
then we have to close it out with our right here single quote and a curly
brace. And then we'll call everything else. In the mongo shell, you don't
need the strings surrounding your operators. But, here you do. That's one of
the distinctions, you should pay attention to. Now,
what about or operators, can we run that? Yeah we can run or, and any of the
same stuff. You just have to set it up a little different. So we call it query and
we call the fields. and I'm gonna call all of them, and let's zoom out a little
bit, okay. We're calling all of our fields we have
to just remember to do our single quotations, and then call your curly
braces. and they put everything else that you want on the inside. The only
distinction: you have to make sure your operators are always enclosed by the
strings, and you always have to enclose these with and strings as well. Just as
well as these. Now, let's say let's find people, all the people who
have Python as a skill, okay. all right that's cool. So we run this query and we
use The all operator. But, then if you notice
something here: the use of this bracket, before without
the all, only printed out the spark. Whereas, here we're finding all
skills including Python. So kind of similar to the C++ one; that we did above.
We could do an aggregation: where we call the aggregate, and then this is all the
same. Except you got to remember as always you got to put in your single
quote. But, here you do not need to put the curly braces after the single quote.
You still have to put the string around every operator. That operator, this
operator, this I guess it may be an operator I don't know. and then you still
have the same notation: except you have to enclose then,
else, and if, for this conditional in strings. And everything else is the
same as MongoDB. So this aggregate is saying let's project our data. Taking the
first name, the last name, where we're creating: basically a count of all the
skills that each person has. and so you're saying this use a condition if;
our skills is an array, then let's take the size of that array. Basically count
the number of elements in that list. Otherwise if there isn't just return n/a.
Saying there is nothing there. But to call this, it's a little bit different. So
let's see what's going on okay so we could say what do we call it? Skills count.
okay so skills count, you use the dollar sign, which is basically
equivalent to the dot in Python or the terminal commands. And then now you see
that I have the ID's. I have a candidate, as usual, and then I have our number
skills that we just created. So in the terminal will be a little
bit different, because it would just all print out together. But here I have to do a
column bind. Where I'm basically concatenated two columns of data or two
data frames. and just putting it together as one, is basically what's going on here.
So I could put out the same output as I would have in the terminal. By just
running this by itself. and unlike the terminal you don't have to worry about
these, semicolons. So here we say, each one of these people has a specific
number of skills. Each person is different, just like in life right. We
could also do like a match, similar to regular expressions. Where we're saying
let's find the candidates with the specialty of machine learning. And lets
print all of those out. Where we're only doing as usual
remember the quotation around, the operators. and the single quotes here
around the aggregate. And you do not need with aggregates, the curly brace. Like you
would use in the queries. Then we run a MapReduce. So that here it's
MapReduce one word lower case. and you have to call like this map equals, reduce
equals. In Mongo you can or the terminal or Python. You can just call it any way
you want you know any function name. But, here you have to call it by these
otherwise I was getting an error. And I used almost the same notation except
when I was writing this in a pi Mongo (pymongo) or the Mongo shell. you'd have to do a
variable name and then equals. But you don't have to use that notation here. So
you just use your same functions, and you just place around them the single quotes
and nothing else. and you just run that. Then to get
your data back together. You have to do the if you want to put in a data frame
just do the C bind or C bind dot and then data frame, and you'll be good. Then
I have ah. this is interesting. so I mention this. For the
MapReduce. you have an ID and the value. so your value is all of the counts. But,
your ID is your state's here. Which is interesting. That threw me off. Okay and
then if you want to name these columns. Just designated by your variable names
here. So then you don't have to come back in and rename it after the fact. You just
do it all at once. Then our last example here: we're doing another Map
Reduce. Where we're taking a function and we're looking at, I still need to do this
variable name. I'm pretty sure. I used this function to
say that's look for our skills and let's print off this. All of
the skills that people could have. and count the number of occurrences that you
have for each skill. That's what's going on in this. Just as you would have as
always in your emit function with your map function. you always have an
emit. That's still the same as regular Mongo. and you still have this dot, and
then whatever you're calling. Those are still the same little things. Only thing
is you're using your single quotes to place that around your function, and you
don't need your variable equals here. You don't need that. That'll be the end of
this video I'd like to say thank you for watching. I hope I brought utility to
somebody. Please like, and subscribe. and if you subscribe turn on that
notification bell. Please, don't throw me any dislikes, you know. Throw me a comment
with the critique in the comments. That's more beneficial for everyone. Thanks for
watching I appreciate it see in the next video bye