BRENDAN KENNY: Hi, my name is Brendan Kenny.
I work with the Maps API team.
And I'm here to tell you as quickly as possible how you
can make Maps in your web browser work with the
Crossfilter library for interesting data
visualization.
So the Crossfilter library, if you go to their Github repo,
is a great library for helping you explore, exactly as they
say here, multivariate data sets.
So data sets where each of your data points has multiple
variables associated with it.
You want to see how they interdepend and interrelate.
So the example they have here has to do with airline flights
and the amount of time early that they might arrive.
And so you can see here a classic view where you have
multiple views into a data set.
And we can look and see how the different
histograms here compare.
So we can see that the time of day is relatively evenly
distributed throughout the day, but
almost nothing at night.
And you see how the distance and the arrival delay both
have this long tail out towards the right.
So what's great about modern computer technology, browser
technology is we can drop this in and actually have these
histograms be interactive.
So I can look here.
And I can use this brush tool to select subsets within one
of these data sets and see how that affects the other views
within the data.
So you can see, for instance, as I come out here to the long
tail, you get actually a large number of flights that arrive
quite a bit earlier.
Because they're so long, they have more
time to arrive early.
Whereas much shorter flights tend to stick right around
zero and have that longer tail.
And so actually, each of these is interactive.
And so I can select subsets within each, and see how that
affects the overall data.
And so this is great because not only can you find new
stories in here, because relationships within data
tends to not be even over the full distribution.
So things that might be true between relationships between
long flights and the arrival delay might not be the same
relationships that, exactly as I said, short trips have with
arrival delay.
And so if you have just summary statistics where you
see the full thing, you're not going to find those
relationships.
The other thing it allows you to do is allows more lay
people to come in and discover things about flights.
I don't know much about distance and how that relates
to arrival delay, really.
And so just by playing around with this, I can discover
things, and then approach things more methodically after
I know what I'm trying to look for.
So we want to look at how we can apply this to Maps.
So Crossfilter is really great for histograms
and graphs like this.
But there's no reason why we have to limit
ourselves to that.
Crossfilter is really just a data library.
It just spits out JavaScript values.
And so it's up to us to create this interface.
In fact, this interfaces is all D3.
It's not the Crossfilter library at all.
So I wanted to show you how you could hook
that up quite quickly.
So we're going to start off with an example data set.
I just created this randomly.
And I apologize for the boring variables.
But it's just a JavaScript array of JavaScript objects.
And in each of these objects, we have a latitude, a
longitude, and two variables-- a val 1, a val 2, that have
different distributions that I've generated.
And we want to see how they relate to each other and how
they relate spatially.
And so how we introduce that into Crossfilter is actually
quite simple.
The docs are a little daunting at first because Crossfilter
is made to be very general, which is great later on as you
start trying to expand the visualization or
Crossfiltering that you do.
Being so general is extremely helpful.
But it can make it a little daunting to start at first.
But here's a really simple way that you can start out.
And it's just about as basic as you can get.
So we create our filter by just calling the Crossfilter
function with our point data, with that array of points.
And then, we actually create the dimensions that we want to
explore on this data.
So by default, Crossfilter doesn't create any dimensions.
We have to manually make those, which is actually a
good thing, because that means we can pass any JavaScript
objects in there.
And they can contain functions.
They can have pointers into the DOM.
And we don't have to worry about Crossfilter
messing that up.
It also allows us to create dimensions, variables that
we're interested in, that are actually
derived from the data.
So maybe we combine two data points into one dimension that
we want to explore.
So that's all the general stuff you
can figure out later.
So for now, we're just going to make it really easy.
We're going to say we're interested in dimension of
variables that's just the val 1.
And this is a function that's mapped over all the points and
just returns from each point the val 1.
And so we created an array within Crossfilter
of all the val 1s.
And then, we make this grouping, which is a little
bit more abstract.
But if you think of it just as the histogram where we're
bucketing them and looking at the frequency of each bucket
within the data set, it becomes a little more clear.
So I know that val1 varies over 0 to 10.
And so I just want to bucket it by integer since it's
floating point.
I just want to floor it and see it.
So then, I'm going to create another dimension for val 2.
And I again return val 2.
But val 2 is a much wider range, I think 0 to 1,000.
And so I'm going to make larger buckets.
And so I'm going to floor for every 25.
I'm going to bucket it, but still keep
it at the same value.
So let's take a look at what that looks like.
And if I refresh the page, you can see that we have two
histograms here-- value 1, which has this almost normal
distribution maybe.
So here's what this looks like.
We have two histograms.
And you can see value 1 has this normal distribution.
Value 2 is a little bit more uniform.
And I totally ripped out the histogram and brush code from
the Crossfilter example and just adapted it to this data
set-- or not even adapted, I literally used it.
And so now, we can brush and see how the two things relate
to each other.
So that's great.
And you can see that this is already helpful.
If these were meaningful variables to us, then it would
be helpful.
But we want to see again the spatial component.
So what we do next is just add a map.
And so you saw before that that array of points each has
a latlong value.
So I can add a map.
And then, I can just, for each of those points, I can loop
over that and add a marker to the map.
And for each marker, I just set the
latlong from that point.
And now, I have a point associated with each value.
And so this isn't a view yet.
It's really just showing where those points are.
So I want to connect these two views to each other.
So I need to bind first the map to Crossfilter, and then
bind Crossfilter to the map.
So let's take a look how to do that.
So what I want to do is bind the current view in the map to
Crossfilter so I can filter the data based on what's
currently in view.
So to do this, I create two new dimensions, a latitude
dimension and a longitude dimension.
So the simple dimensions that we had before, or similar to
those, where they just return those values directly.
And now, what I can do is use a map event on bounds changed
to actually get the bounds, the most westward and most
eastward, and most northward to most southward.
And then, I can use those to actually filter those
dimensions.
So I pass these things.
And I say, this is pretty much just asking Crossfilter to
filter out all the data points that don't
fall within that range.
So that's exactly what we want.
We're now filtering on the bounds.
And now, when we refresh, we can see how that looks.
And so you can see as I move points in and off the screen,
the histograms below change.
And this is doing exactly what we were doing before.
We were selecting subsets of the data by brushing within
the histogram and seeing how that
affected the other histogram.
Now we're seeing, based on the points that are currently on
screen, what is the distribution of the values
that are associated with those points?
So you can imagine this is helpful.
As you zoom in, we can see, just like before where we
could see the long tail distance was associated with
certain patterns.
We could see spatially as we zoom into certain areas of the
United States, maybe certain values have different patterns
within that area.
You could imagine a different tool where you are actually
dragging and selecting a bounding box, or any sort of
thing like that.
You're selecting census blocks.
And you could affect the histograms below.
So now, we need it to go the other way where selecting
what's in the histogram drives what's on the screen.
So we do that by adding one final dimension.
And I just call it the ID dimension.
And there might be an easier way to do this, but this is
the first way I thought of doing it.
And it works OK.
So we'll use it for now.
But if anybody has better ideas, feel free to leave them
in the comments.
There's probably a more idiomatic way.
So in this, I just return an ID.
So this is a little subtle.
But this is a function that's mapped over our data with the
built-in map, array.map.
The second thing that you get is that the
index within the array.
And so I can just return the index of that point.
So I know which point it was that created the current thing
that's being filtered.
And so I make a grouping of it where it's
actually the full thing.
So I have 300 data points.
So this group is 300 buckets.
And each bucket only has one of one thing in it, or nothing
in it if a point has been filtered out.
And so when I created my markers, I put them in an
array also that mirrors exactly the point array.
And so what I can do is when I check a bucket with a
particular ID, I can just go in and take that ID, that same
ID, and set it to either visible or not if that bucket
is empty or not.
So that code is that is right down here,
where I update markers.
And so I get the grouping.
And I do grouping.all.
And this is something that--
this is where I'm not totally sure this is the best
way of doing it.
But the nice thing about .all is that it's not creating the
new array every time there's a change.
It's actually using the same array over and over again.
And so, as I said, I just loop over those points.
And I just check--
I use the key, which is the ID, to look up
into my markers array.
And then, I just check if anything is in that bucket, I
set that marker to visible.
And if not, it's set to invisible.
And so if I once more refresh the page, I come back here.
And you can see that it's still bound so my view affects
the Crossfilter.
And now it goes the other way, as well.
When I filter the Crossfilter using the other histograms, I
can see that it affects the markers currently
drawn on the screen.
So if I only want the value 2s that are within this range, it
will filter the histogram of value 1s associated with the
same points.
But it also affects the pointers that
are drawn on screen.
And so now, we have a fully bound thing.
And you can say, things only within this subset, looks like
this in value 2.
And it looks like this spatially.
And so once we have this, you can start doing
all sorts of things.
And it gets a little hairy when you have a bunch of
markers that are even more than this.
So this is about 300 and actually works quite well.
There's some performance things we can do with
de-bouncing and that sort of thing.
But actually, the Maps API handles it fairly well doing
exactly that.
And there's some really interesting things we can do
with D3 and the Maps API which both have kind of strong NVC
or data driven views.
And so there's more interesting things we can with
binding them together.
And then, we can take a look at things like WebGL, where we
can display many more points.
And we can express these filter ranges in a more
interesting way, rather than looping over each point.
And then checking, we can do some interesting things with
actually our geometry.
But those are all talks for another day.
So I hope this was helpful.
Check out the source code and feel free to make Maps to your
heart's content.
We can definitely do more and more interesting things here,
and there's no reason to limit ourselves to just a map or
just a histogram when we have so much better
things we could do.