Okay now I'd like to talk about group in a little more detail. By now you should
understand that group's role is really to aggregate it's
input in some way based on the operators specified.
Here are a list of operators that we
can use in grouping documents together in the aggregation
framework. We've seen some several times. First simply selects
the first documented group. Last, the last documented group.
Max, min, and average, all do what you might
expect, based on a numeric value we're calculating as part
of a group. As one example, let's take a
look at average. So, here we have an aggregation pipeline.
And the idea here is to calculate the average
number of re-tweets for any tweets using a particular hashtag.
So we do an unwind on the hashtags array of
the entities field. Then we're going to group, based on the
text of the hashtag. So, this'll give us one
document for every hashtag used in a given tweet.
So then in the group stage here, we're going
to aggregate based on the text, so based on
the hashtag itself. And then I'm going to calculate
an average based on the field retweet_count, which is
a top-level field for tweets. Finally, we're going to
sort, based on the retweet_avg. Okay, so this gives us
an idea of where we might use average and what
it looks like. The syntax is very similar to what
we've seen with sum. Now what I'd like to do
is nudge this discussion of operators along just a little
bit, and I'd like to introduce a couple of additional
operators. One is push, and the other is addToSet. These
are extremely useful operators in a variety of different situations
because these are operators that actually deal with arrays. Okay?
So let's take a look at an example of where
we might use something like this. We're going to look
at an example involving addToSet. Essentially what addToSet does is,
as it's accumulation function for a grouping, it adds values to
an array, but does so by treating the array as
a set, that is it won't add the same value more
than once to the array we're accumulating in. Okay. So
here's an example. Now in this example, what I'm doing is
aggregating together all of the unique hashtags and grouping
by the user screen name. So this is essentially
all of the hashtags that an individual user has
used in their tweets, but we're ignoring hashtags that a
user has used in multiple tweets. So, here we're
going to use addToSet to do that. And the way that
this works is we're going to group, again we're going to
specify as our id to define the grouping based on
the screen name of the user creating the
tweets. Okay? Note that we're unwinding, based on
the hashtags array. So what we are going to do then is use addToSet, and for
each hashtag the actual text of the hashtag,
we're going to add that to this array,
unique_hashtags, that's part of the group document for
an individual user. Using addToSet ensures that no
matter how many times a given hashtag occurs, that is no matter how many times
a user used it in a given tweet. So, no matter how many times the group
stage sees that text, it will be added
one and only one time to this unique_hashtags
array. And then I'm simply going to do a sort based on ID. So let's run this.
Okay? And so, all we're getting out is a list of
pairings for a screen name, and a list of all the
unique hashtags they've used. Okay? So we can see that, for
this particular user....1, 2, 3, 4, 5, 6 different hashtags, used,
we don't know in how many different tweets, or how many
times. This is essentially the vocabulary of hashtags for this particular
user. And if we scroll through here we'll see, we'll see
the vocabulary of hastags for all of the users in this collection.
Now, what might be more satisfying here, is
if we were to use something like sort based
on the number of unique hashtags that individual
users have written as part of their tweets. But
that's when you require us to do something
we haven't talked about yet, which is using the
same operator in more than one group stage
in an aggregation pipeline. We'll look at that next.