[MUSIC PLAYING]
JUSTIN BRAATEN: Hi.
Welcome, everybody.
I'm Justin Braaten, developer advocate with the Google Earth
Engine team.
Today, I'd like to talk about harmonizing
50 years of Landsat imagery.
This is an exciting year for the Landsat program.
It's their 50th year of imaging.
And this is an incredible record of Earth change.
So what we're seeing here are a couple animations
of a Landsat time series from 1984 to the present.
We see things like, on the left there, shoreline changes
on Sagar Island in India.
On the right is forest change in British Columbia.
We see forest clear cuts and a big open-pit coal mine.
Lots of other changes, like lake level rise and fall,
agricultural irrigation in the desert.
We've got really dynamic rivers.
This is Ucayali River in Peru the left.
On the right, we see Las Vegas increasing in size over time.
And these are all annual composites,
cloud-free imagery from Landsat 5 through Landsat 8,
in this case.
And if you've worked with Landsat,
you've often heard this story about we've
got 50 years of imagery.
But often what you see in academic research and data sets
are the data starting in 1984 and going to the present.
So you're often missing that first 10 years in the '70s,
and so this story about 50 years of imagery
is not always what we actually get.
So I've been really interested in-- well,
I guess before I move on--
Landsat 5 has been imaging since 1984
and really the baseline for much of the work that we see.
But Landsat began imaging in 1972,
so there was three earlier satellites that
collected MSS data, MultiSpectral Scanner, and this
is what I've really been interested in is taking
the time series as far back as you can.
But that data from the '70s is quite different from the data
that Landsat 5 began collecting with the Thematic Mapper
sensor.
So the rest of the talk will be about trying to take this MSS
data from the '70s and making it look like TM,
so we can do an analysis on 50 years of data and not just
the 40 years.
And Earth is constantly changing.
There's lots of things happening in the '70s.
This is an example from Mount St. Helens, which you see right
there sort of in the center.
And up above it is where the volcanic blast occurred.
So all of that region that's kind of in yellow,
it was the result of the volcano.
And what we're seeing here is a three-band composite
of time in the '70s.
And so all the pixels that are color, like red, yellow, blues,
those are all changed.
Everything in grayscale is not changed,
and this is to emphasize that a lot was happening in the '70s
that we don't know about if we're not
going to look at that era.
Before I came to Google, I was working
at Oregon State University for a research
lab interested in doing forest change detection and model
infrastructure and things like that.
And when you look at a time series of Landsat, after--
depending on the forest type--
15 or 20 years, the satellite signal
is saturated after a clear cut, after a disturbance.
So you really don't have a lot of more information
after 20 years of growth.
And so there were some people at this research lab
that I was working at thought that if we could
map disturbances back into the '70s,
we would at least know when something happened and would
have some more information and more context
for what the present is like.
And they found success by including
disturbance information that could better predict
current forest conditions.
And this was just sort of a one-off paper,
and my supervisor thought it'd be great
if we could do this more often.
Can you please automate this process?
And so I came up with this system.
This original system was in R, so we
had to download all of the Landsat imagery.
It was still kind of a pain in the butt.
It was run on local computers, downloading all this imagery,
running this--
all these steps, but it worked.
It worked pretty well.
And I moved on from this project eventually.
But, when Earth Engine came around,
I was really excited to try to port this functionality
of taking MSS imagery and making it look like TM,
bring it into Earth Engine, automate it even further.
And so the result, at this point,
is that I have a notebook.
It's sort of a prototype.
It relies on the Earth Engine API
to do all this processing of MSS imagery.
And what you see, here, is the notebook.
The first half of it is just sort of parameterization.
Instead of Earth Engine authenticating to it,
initializing, the bottom part are four or five steps
to actually do the work.
And so, basically, you can set the parameters, hit
Run All on this notebook, and, in about half a day,
you get a 50-year time series of Landsat that
is harmonized through time.
And so some of the challenges about generating this data
are MSS, it only had four bands--
green, red, and two near-infrared bands.
So it's missing a lot of the information
that later sensors had.
In particular, SWIR bands--
ShortWave InfraRed-- and thermal, which
contain a lot of information.
Some other things that are funky about MSS compared to TM,
the later sensor, is the radiometric and spatial
resolution.
So, on the left, there, we have an example of MSS data.
The shading, there are 64 shades.
It was 6-bit data.
And, also, the pixel resolution was larger
than TM, which had 8-bit data and 30-meter pixels.
So we have these challenges of MSS
not looking like TM, both in terms of the data that it
collected spectrally and then sort of the degraded sensor
qualities.
But there's a cool feature about Landsat, the program,
that Landsat Satellites 4 and 5 carried both the MSS and the TM
sensor.
So we have this great data set of coincident imagery,
the exact same landscape at the exact same time.
We have an example of MSS and TM,
so we can develop a relationship to help
predict TM bands from MSS data.
So the first step of this process
to get MSS to look like TM is to generate a reference
image of MSS from the '80s.
So MSS that was on Landsats 4 and 5 is pretty good data.
So the first thing we do is make a cloud-free composite
from a few years in the '80s of MSS.
And it might look like this, for instance.
And then we take all of our other MSS data, which
is generally much noisier than TM and later sensors,
and we normalize it to this reference image,
just using random forest regression.
So we get images that all sort of look like each other
to start out with, but they still
don't have all of the bands that TM has.
The next step, then, is we have these images that look
like each other, relatively.
We remove the clouds from them and build
annual cloud-free composites.
So we have one image that represents sort
of the growing season per year.
And, now, we have this nice set of MSS images
that have no clouds in them per year.
Then we want to draw a relationship between the MSS
data and the TM to make MSS look like TM.
So we do that with these coincident images.
We take these pairs that were collected
on the exact same day, the same location.
We draw a random sample.
This is just three sort of examples
of what those might be.
Random sample.
We create a linear regression model with random forest.
That's an aggregate of all of the samples
that we have for these coincident image pairs.
And then we take our MSS images, run them through this model,
and out we get pseudo-TM data.
And so you've made this linear regression,
and now they look a lot like MS--
or, excuse me, MSS looks like TM,
but you still have this residual around your regression line.
So we can clean it up even a little bit further
by calculating the mean offset per pixel.
So this is an example of one pixel's time series.
And at the top, there, we're showing
that there was still this offset in Tasseled Cap Angle space.
And we can remove that offset and get them
lined up a little closer.
And so we do this on a per-pixel level.
And this is an example of what that offset might look like.
And you'll see a lot of linear features in there.
And this, it's sort of the result of MSS
having lower resolution data than TM.
TM is better able to resolve the edges for roads and clear-cuts,
streams.
So, now, we'll look at examples of the data, the TM,
the good data MSS, the data from earlier, the poorer data.
And, at this point, the data have
been run through the random forest regression model,
no offset correction yet, but you can
see they look quite similar.
What we're visualizing, here, is SWIR-1, NIR, and green as RGB.
And recall that MSS didn't have any SWIR bands.
So, yeah, we can see we've done a pretty good job of getting it
into the SWIR space, but it's blocky.
You can't see edges as well.
The stream is not as crisp.
Any time there's a transition between forest and non-forest,
it's sort of blocky and not well defined.
After we do the offset correction,
you can see that it becomes much more crisp.
We can see the stream outline.
We can see the utility line in the upper left come up
real clearly now and a little bit more diversity
in sort of the color.
And if we look at the histogram of this image,
we can see that MSS and TM, their distribution
is quite similar for all of the bands.
So MSS did not have a blue band, and you can see
that we predict it quite well.
It didn't have SWIR-1 and 2.
We do a little bit less well with predicting those,
but get the general trend.
In the Tasseled Cap Brightness, Greenness, Wetness,
our derivative bands, Tasseled Cap Wetness
has a large component from SWIR, so it's not as well predicted
as Greenness and Brightness.
But it all looks pretty close.
So if you're trying to map forest change from clear-cuts,
for instance, sort of large dramatic change,
it does a pretty good job.
This is it comparing the same image, coincident image
pair, TM to MSS.
They look quite similar.
Maybe, in the agricultural region,
MSS is a little more muddled.
This is looking at the comparison with SWIR-1, NIR,
and green.
Again, really similar.
Here's the coincident pair in two-color.
And two-color includes the blue band, which MSS didn't have,
but we can predict it really well, as you can see here.
And the last thing that we do to this data
is run it through LandTrendr to try
to clean up any extra interannual variability.
So this is an example of a single-pixel's time series.
It happens to be a forest.
At the top, there, the blue represents
a stable, mature forest.
And then it gets cut.
It turns red in color, and then it recovers.
But you can see there's this noise, year to year.
What LandTrendr does is it finds these sort of durable events
and segments and assigns vertices.
We can interpolate new values between these vertices
to get a really nice straight time series.
And so we can generate this synthetic time series of MSS
through to the present, where everything has
been normalized and segmented.
And now we can include 1972 to the present.
This is a lot of clear-cutting in the Pacific Northwest.
Blue is mature forest, reddish brown are bare ground,
and kind of greenish is recovering forest.
This is Mount St. Helens.
It erupted in 1980.
So we couldn't see before the blast unless you include MSS.
And we can see, here, that it's quite seamless from MSS
to TM era and after.
We saw this animation earlier.
It's the same location in British Columbia
with the open-pit coal mine.
And, by including the 1970s data,
we get really close to when it began to open up.
This is an example of the Olympic Peninsula
and Washington state forest harvests again.
This shows a region in the Forest Service.
Oh, let's see if we can show it here.
This region is Forest Service land.
They had a big policy change in 1994.
And you can see that, in the '80s and the '90s,
there was quite a bit of harvesting,
and then it stops and regrows.
They didn't allow harvesting in this section anymore.
And it's nice to be able to include MSS for a little bit
more context and a longer history
to understand before the policy was in effect.
This is showing wildfires in Yosemite National Park.
And I think the cool thing to note,
here, is all the harvesting that is
happening on the west side of the park in the '70s and '80s.
You know, again, without MSS, we wouldn't
be able to see that as well.
And so, at this point, we've just
been looking at synthetic time series data.
But what you can also do with the output
from running this code is all of the outputs
that LandTrendr gives you for segmentation, so
with these segmented outputs, we can create maps
of forest change, for instance.
So what we're seeing, here, is the year of forest disturbance,
greatest forest disturbance.
Yeah, so this has been sort of a pet project of mine.
The code is available for use on GitHub.
There's a notebook there and some instructions.
So, if you're interested, you can take a screenshot
or write down the URL.
There's not a lot of support I can give for it, well,
but there's an Issues section.
If you have questions, I might be able to help you out.
But I'm really happy for you to try it
in your region of interest, except that it doesn't
work well all over the world.
This is a map of availability of MSS data.
The blues are areas where this system will probably
work pretty well.
Areas outside of the blue, it's going to be challenging.
There's just not a lot of observations.
Thank you.
AUDIENCE: Woo!
[APPLAUSE]
[MUSIC PLAYING]