Let's talk about database performance. Database performance in MongoDB is
driven by pretty much the same thing that every database is
driven by, which is are you going to use an
index to resolve your query or not? So this maybe a
review for a lot of you, but I want to take
you through the basics of indexing, and what it is and
why it's so effective. If you look at MongoDB or any
other database, it will store its data in these large files
on disk. There's no particular order for the
documents on the disk. They can be anywhere in
a database file. If you want to pull out
a particular document, you do a query. Now, what
the database is going to have to do
by default is scan through the entire collection to
find the data. This is called a table scan
in relational databases and a collection scan in MongoDB.
It is death to your performance. If the data
set is large, it'll be extremely slow. So, instead what
we do is we create an index, or maybe
more than one index. So how does indexing work? Well,
it's actually pretty straightforward. If something is ordered, like
for instance, this list of hashtags, even if it's very
long, it's very quick to find something in the
list because we can use binary search to do it.
So, finding something in a sorted list is really
quick. Now to create an index, we want to specify
a key. An index is simply an ordered list
of keys. Now we don't actually keep them linearly
ordered like this in MongoDB, we use something called
a B Tree, but conceptually you can imagine it
looking like this perfectly reasonable conceptual model. And if
I'm looking for like say Lucero, I can quickly search,
find the item and then have a pointer
to the actual document or documents inside the
collection that have this hashtag associated with them.
So this picture actually represents an index that is
composed of just hashtags. So we're using a
single field as the basis for our index. But
this is just a special case of the
more general idea of indexes in MongoDB. In mongoDB,
indexes are ordered lists of keys. You can have just
one, as we saw in the previous example, or we
could do something like this and construct an index out
of three keys. Let's say for example the hashtag, the date
on which a tweet was created, and the username of
the person creating the tweet. The order is important here. Because,
conceptually, the way the index is built, is that the
hashtag will be at the top, so here are my hashtags:
Colin Hay, Lucero and then Steve Earle near
the end of my index. And then, within
this, so let's say Colin Hay is conceptually
here, for all of those items, we'll have dates
as the second level of our index. And
what we're going to do within the Colin Hay portion
of the index is sort these by order
of date. So the documents then, that are identified
down to this level of date are first identified by
the hashtag colinhay and then sorted based on their date of
creation. And then finally, within each date, so for each
day on which a tweet was created with the hashtag colinhay,
we're going to have the usernames of all of those
users and those then will be sorted as well. So if
you provide me just the hashtag I can go into the
index and find all the let's say Luceros in this case.
And if you prefer to also provide me the date,
then I can break it down and find all the Lucero
tweets on that particular day. At the very bottom of this
of course, is going to be pointers to the actual data.
Now, in order for MongoDB to be utilized in index
you have to give it a left most set of items.
So, you can give it just the hashtag or you can
give it just the hashtag and the date or the hashtag,
the date, and the user name. For this particular index, if you
provide me just the date, I can't do much really with the
index because the date is down here at this level. So to
use an index, I need to start at the top, and this is
true whether or not I'm doing a query or I'm doing an
update or I'm doing a sort, because sorts also will use an
index to sort their values. So for instance, if I pull a
bunch of data out of the collection and want to sort it by
hashtag, with MongoDB, I can use this index to do
the sorting. Now, one other point I want to make is that
every time you want to insert something into the database
this index would need to be updated, and that updating is
going to take some time. So we use indexes to make
Reads faster, but Writes become a little bit slower if you
have an index because the index needs to be updated. So
you need to take that into consideration when you're thinking about
what indexes you might want for your particular
application. Indexes are not costless. They take space
on disks, they also take time to keep updated. So you don't want to have a index
on every single possible way you're going to
query the collection, you instead want to have
an index on the ways you're most likely
way you are going to query the collection.