All right. So now I would like to show you a
large collection, and the effect that indexing can have on performance. Now,
our collection of tweets is relatively small. So, instead of going
to revisit the open street map dataset, which you should remember from
the previous lesson. This will also serve as a nice transition
to the next lesson, which is a project using the Open Street
Map data. So for this example, I'm going to work in
the MongoDB shell, and the database in which this data is stored
is the OSM database. So I'll switch to using
that, and then let's take a look at what documents
look like in this database, and to do that,
we'll just do a find everything. Okay, so then if
we scroll up, You see what these documents look
like. There's location information, latitude/longitude. So all data in this
collection is tied to a specific location. And something I'd
like to point out is that some of the documents
in this collection actually have a TG field. And this
is shorthand for tag. So if you remember, we looked
before at how specific locations will be tagged from time
to time within this particular data set. And the way
these tags work in this collection is, there is a
tag field that is array valued and each of the
individual values in this array are a sub-document with a"
k" and" v" field. Much like we saw in the XML
version of this data set. Now, the reason why the data
is represented this way is because there could be multiple taggings
all with the same type or key. So storing them in
an array this way gives us the ability to do that without
one tag writing over another as we put them into MongoDB.
One alternative would be to have this be a field name out
here, and this be the value for that field. Okay, now
something I'd like to point out about this data set is there
are more than seven million documents in this particular collection. Now, we've
just talked about indexing, and how indexes can improve performance. So let's
take a look at a query and the amount of time it
takes for that query to come back. Right now, I don't have
an index on this tag field. So let's do a query and
see what our performance looks like. Now this particular collection that I've
loaded here has actually all of the Open Street Map data from
the city of Chicago again. And what I'm doing here is querying
for any nodes or any geographic locations that have been tagged with the
name Giordanos which is a famous Chicago pizza chain. So if you do this query
[BLANK_AUDIO]
You can see that it takes a little while to
come back. Let's do a pretty version of this query.
[BLANK_AUDIO]
Okay, so a couple of seconds right? For the query
to come back. So, if we're doing a single query,
[SOUND] not a big of a deal. The fact is,
in most applications, we're doing many queries. In some applications
hundreds, or maybe even thousands, or tens of thousands of
queries in a very short period of time. So, waiting
two seconds for a query to come back, given the
load that places on the database server, and. Although simultaneous queries
going on at the same time simply just doesn't work
for our applications. It's death to our application, as I mentioned
previously. So if we built an index on this collection, then
our performance improves significantly because instead of having to do a
table scan. As we saw before. I can simply go right
to the place on disc where these particular documents matching my
query are located. Now, going too far beyond where we're already
at with indexing is beyond the scope of this class. So
I'm just going to, essentially get you started with indexes
here. And encourage you to go look at the MongoDB
documentation or take a look at the free online courses
we offer on university.mongodb.com, see in the instructor notes for
a comprehensive treatment of all of these topics that we've
been discussing with regard to Mongo DB, including indexing. Okay,
so let's build my index, and the way I do
this is specifying the field on which I would like the
index created. And I do this simply by saying, make
it the case that there is an index on the
field tg in the collection nodes. Okay? Now, again, more
than seven million documents in this collection, so this is going to
take a little while to come back. It's going to take
a little while to build the index. Had we created the
index at first then as we were loading data into
the collection it would have been updated with each write to
the data base. But in this case we're building the index after
we've loaded all the data in so that I could work through
this example. So, we'll just skip ahead in the video to the
point where the index is actually created. This is going to take a
couple of minutes. Okay. So now with the index created, let's run
that query again and look at the performance difference. It was intimidate
really. We got back our documents right away. And so that illustrates
the performance differences in using an index versus not using an index.