So let's take a look at an example of using
the aggregation framework. To answer some questions about our data,
let's find out how we could use the aggregation framework,
to determine which user in our data set, has produced the
most tweets. Now, let's talk about the process we would
like the database to perform for us. So given the
way our data is laid out, the first thing we're
going to want to do is group the tweets by user.
Remember, each tweet, has the user as
a field within it. Then we're going to want to
count each user's tweets, and finally, select the
user with the most tweets. Now, let's rethink
this third step. What's probably going to be
most valuable, is not just seeing the one
person who's tweeted the most, but instead, counting
the number of tweets for each user, and
then sorting them. If we sort into descending order,
the person at the top, will be the one
with the most tweets. So our process is really
group all tweets by user, count each user's tweets,
sort into descending order, and then, select the user
at the top. Okay. So, with this process outlined,
let's take a look at how we would express
this in the aggregation framework. And, we'll use this as
an example, to launch our exploration of the
aggregation framework in MongoDB. Okay. So, aggregation queries
in MongoDB are issued using the aggregate command.
And, we'll talk about this in just a
bit. But aggregations are done with a pipeline.
And a pipeline is essentially a series of
stages, that are included as elements of an
array, that's passed through aggregate as a parameter.
Okay, so the first thing we need to do is group. Now, here we're going to use
the group operator in this first stage of our
aggregation pipeline. And the way we're going to group,
is based on a user's screen name. Let's briefly
go back, and take a look at the data.
Remember that, all of our tweets have a user
field. And that user field is actually a nested
document, that contains a screen name. Okay, so
if we go back to our aggregation query, we
can see that we're saying for the user
sub document, I want the screen name field. Okay,
so what's this about? Well, this isn't an
operator. Instead, what this means is, so even though
it's inside quotes, don't interpret this as a string.
That is to say, don't make the ID user.screen_name.
Rather, group together all documents where the value of
screen name for the user sub document. That's what
this dollar says should happen here. Where the value
of this is the same. So, all tweets that
have this, the same value for this field will
be grouped together. And then we need an accumulator
of some kind. There are a number of different
accumulator operators that we can use. What this means is,
for every document that has the same
value for this field, increment this value, count,
by one. That's the semantics of this.
So, this accomplishes those first 2 steps that
we talked about here, group tweets by user, and count each user's tweets. Then
the next thing that's going to happen, is we'll simply do a sort. And this says,
sort based on the count of the documents that are
passed into this stage, and sort into descending order. Okay,
what do I mean by the documents passed into this
stage? Well, the reason why this is called a pipeline, is
because each stage receives a set of input documents and
produces a set of output documents, so we'll talk a
little bit more about that later. The input documents to
this stage that uses this sort operator, are the documents output,
by the stage that uses the group operator. So
what we end up with, for this stage, is a
bunch of documents that have a underscore ID field,
and account field. And this stage of the pipeline, is
going to sort those documents, based on the value
of their count field. Okay, now, if you didn't catch
all that, don't worry about it. Because we are
going to dive into all the bits and pieces here,
as we move through this lesson. Let's go ahead and run
this code, and see what it produces. Now I'm simply going to
pipe this to the system program less, so that I can
see the very top of the results that are produced. Okay? The
results from an aggregation query are always a single document. It
is the result field of that return document that we're interested
in. And here we can see that what was outputted, are
exactly the documents that were passed to the sort stage or our
aggregation query. They each have an underscore ID field, and
a count associated with them. And they've been sorted into
descending order. Now, note that there's only 8. Or rather,
that the maximum number of tweets for an individual user is
8, okay? This data set is a snapshot from a
relatively short period of time. So that's why these counts are
fairly small. As we move through this lesson, we'll take
a closer look at the aggregation pipeline, at what it means
to have stages within that pipeline, and at the various aggregation
operators that are available to us, as well as other operators,
such as accumulator operators, that we can use in the group
stage, and other sorts of operators we use elsewhere in aggregation queries.