DYLAN THOMAS: Hello, welcome to What In The World.
This is a regular show where enterprise Geo Googlers tell
you a little bit more about an interesting part of enterprise
Maps products.
My name is Dylan Thomas, I'm a Community Maps Manager.
And with me today is Graeme Merrall,
who's a Deployment Engineer.
Hi Graeme.
GRAEME MERRALL: Hi, Dylan.
DYLAN THOMAS: And today I think you're
going to tell us some really interesting, handy tips
about data integration.
GRAEME MERRALL: That's right.
So a lot of people when they go down the GIS road, generally
start using Google Earth to manage their data because it's
simple and it has a great interface.
But one of the restrictions with Google Earth it
that it uses a format called KML,
which is an XML format used by Google Earth.
It's a very rich format, but it generally
allows users to insert unstructured data along
with the content.
And the problem when you move to a system like Google Maps
Engine is that you require structured data to get access
to the attributes.
DYLAN THOMAS: So you might start with something simple
like an HTML table.
And it looks fine on the map, but then
when you're saying when you move into an enterprise system
you need structured data.
That becomes a problem at that point.
GRAEME MERRALL: That's correct.
And you can actually see in the example that I've got up here,
I've got some mountains from New Zealand,
which is my home country.
And if we click on this window, you
can see we get a nice piece of some HTML that pops up.
But it's actually represented in the KML access
as actually HTML that's completely unstructured.
DYLAN THOMAS: Got it.
So if we were going to look at this-- I mean,
you're saying it's HTML code.
Which is fine if you want to look at it,
but it's pretty horrible if you want to try and import that
into some other system.
GRAEME MERRALL: That's right.
You can actually see in this screenshot
I've got up, we actually can see the underlying KML
and what it looks like.
You can see it just looks like HTML.
And so if we import that into Maps Engine,
you can see all we get is the description field.
We're actually not able to do anything with it.
So we can't style it and filter it, or anything like that.
And that's obviously a problem, because the whole point
of wanting to get structured data into KML and into Maps
Engine is that you want to actually
be able to do that sort of filtering and styling.
So one of the things you can do is to change or alter the way
that you use to create or manage your KML.
And so there's two methods for doing that.
The first is to use something called extended data.
And extended data drops out this description field
by creating essentially a set of name value pairs
that you include inside your KML.
And then to replace that description window,
you create something called a Balloon Style.
Now the nice thing about that is you
just defined it once at the top of the document,
and then every other piece of content gets to reuse that.
So it makes it much simpler to manage and change it.
We can have a quick look at some KML, just show you that.
Here's one showing the Balloon Style I just briefly mentioned.
And you can see some placeholders in there
with square brackets.
And so that data is going to get transplanted up
from a placeholder, a placemark.
So you can see now we've got an extended data field.
We have a data name.
And you can see we have the display name, and the value.
And also for the height, we have the display name,
and the value.
DYLAN THOMAS: Ah, so it's structured,
but then the other part, too, is that you're only
defining that display style in one place.
So you're not replicating the display style
for every single placemark, you're just
doing it once-- the balloon, I should say.
GRAEME MERRALL: That's right, that's right.
So well, I would say there's, again, in the classic method,
if you wanted to change the format of the table,
you would have to change every single placemark.
And this one, you just change it once.
And it ripples through every single change.
Now if we look at Google Earth, we
can see that, actually, it doesn't look any different.
From a user point of view, it looks as good as it ever did.
DYLAN THOMAS: So we're done, right?
We've got structured data.
So what does it look like in the Maps Engine, then?
GRAEME MERRALL: Well, the problem
with our fantastic structured data
is it imports only as a string.
See, here's now the data that I've
converted using extended data, and imported into Maps Engine.
But now everything has gone in as a string.
So now the height field is actually a number,
it's 2797 meters, or what have you.
So because it's a string, you can't
perform basic comparisons, like greater than or less than.
So ideally you want to be able to import numbers as numbers
and dates as dates and strings as strings.
It just makes good sense.
So you have to search for an alternate style
to make that happen.
And that's using something called schema data.
And schema data inside KML allows
you to type your content.
And it's actually slightly simpler
than using extended data.
The way you do it is you still define your balloon style,
but then you create something at the top--
you can see highlighted in red --we
create something called a schema.
And you can see, now we've defined
something called a mountain name of type string,
and defined a height of type int.
And you can also see I've altered the display
name slightly for mountain height,
so it's actually-- you can actually
sort of pack in whatever sort of data you like in there.
You can even put [INAUDIBLE].
DYLAN THOMAS: Nice.
Can you store-- can you do dates as well, for example?
Are there other types there, or is it just numbers and strings?
GRAEME MERRALL: There's about eight different types
you can store.
So string, int, date, floats, real numbers,
all sorts of things.
We'll include a URL to point you to the documentation that
covers this, so you can read about the different types
that it's imported.
DYLAN THOMAS: Great
GRAEME MERRALL: So we can also now see a placemark,
and you can see we have extended data.
And now we have the schema data takes and simple data takes.
And you can see it's very simple.
It's just simple name, value, name, value.
DYLAN THOMAS: Much cleaner, yeah.
GRAEME MERRALL: All right, so it actually
makes that placemark very, very simple and clean to look at.
DYLAN THOMAS: And it all looks the same
in Google Earth, right?
There's no difference to the consumer?
GRAEME MERRALL: That's right.
So if you go back to Google Earth and turn this one on,
you can see again, it looks exactly the same.
I've changed the formatting, so I apologize slightly.
But you can see again, we've got [INAUDIBLE] 2797 meters,
as it should do.
If we look at the data types, there it is again, same thing.
So it's no different to the end user.
But if we go ahead and have a look now in Maps Engine,
I'll just go a bit through my search results,
we now have our content and our height
correctly in as a number.
So now we can start to do that comparison.
DYLAN THOMAS: Yeah, excellent.
And so you're recommending the use of a library
to do some of this work, right?
GRAEME MERRALL: That's correct.
So one of the things-- particularly
if you've already got a whole bunch of KML
where you've got content in the description field
--you need to get that out and turn it
into, say, an extended data field where you can import.
You need to actually-- obviously,
it's far simpler to do that programmatically.
So we have a great tool which is written by a Googler,
but it's open source, called pyKML.
And it makes it very easy to read and write and also enter
KML files.
Now one of the things that's nice about it,
is that you actually just reference the names.
So you can see about halfway down,
you can see the slide that begins with four.
And what it's doing is, I've said for every placemark,
and then KML-- the XML to find a point or a line that
says placemark.
And so I just use the word placemark
to say find every placemark in my KML file.
DYLAN THOMAS: It's almost like you're
reading KML at that point.
If you insert some angle brackets, it's the same thing.
GRAEME MERRALL: Yeah, it's almost
like reading KML in Python.
And then you can just open up-- you
just have a look at the description
field, which is in Python called description.
And it depends on your situation so you
can read the contents of that field.
And then you just write it back using extended data.
So I've said here in the line here,
that says Fragment.ExtendedData.append.
So [INAUDIBLE] in pyKML say create
a field called extended data and append some KML to it.
And if you flip back in the video
and have a look at the XML, or the KML for this,
you'll actually see it actually matches up very simply.
DYLAN THOMAS: Oh, excellent.
So in the description field, you said
this is sort of a custom, or case by case basis, right?
So when you actually get the description,
then, obviously you might need to use
another library or some special logic to take your content
and turn it into the structured data you want.
But at that point, you're just writing it out using pyKML.
GRAEME MERRALL: Yeah, that's right.
So how you actually crack open the description field
and read your data out is gonna pin entirely
on how you've got that in there.
If it's much tabular data, it's going
to be quite easy to use a simple HTML library to read that.
If it's just in this bullet point, so something
a little bit more free-form, you're
gonna have a bit more of a struggle to do that.
The alternative of course, is if the data
is stored in a database of some sort,
you can actually just write some pyKML
to actually create KML from scratch.
You don't need to crack open a KML file and edit it.
You can just dump it out from your database,
using the pyKML library to create
a brand new KML with all [INAUDIBLE].
DYLAN THOMAS: That's a really good idea, yeah.
Well, Graeme, I've got one other question for you then.
So you popped up the URL for pyKML briefly,
we'll put that in the video.
But is there a good source for getting some real world
examples of this in action, so our customers can take a look
and get a head start?
GRAEME MERRALL: Yeah, so the pyKML website
includes a lot of different examples
for reading, writing, and editing KML,
both simple and complex.
So you can manage very large structured data of KML
if you wish.
And also through the pyKML website
there's actually a mailing list as well.
So if you've got any questions that
aren't answered by the examples, from reading the documentation,
by all means, jump short.
If you jump on the mailing list, you'll
likely get your question answered there as well.
DYLAN THOMAS: Excellent, great.
Well Graeme, thanks very much indeed,
really useful information here.
And we will make sure we include the links in the video,
and we'll see you next time on What In The World.