[MUSIC PLAYING]
ADRIANO MATOS: So I hope you guys are doing well.
My name is Adriano Matos.
I'm a first year master's student
studying natural resources and environmental science
at the University of Nevada Reno.
And today I'm going to be talking
to you guys about my research at the GigaFire Project,
as it's called, a large-scale fuels mapping
and modeling in the cloud computing environment.
So before I get into the weeds of the Google technologies
that I use, I think it's important, first,
that I talk about some background, how
this project came to be, as well as identify some key terms I
think are important to discuss so that we're all
on the same page.
So this project came to be from two agencies, the California
Air and Resources Board and the California
Department of Forestry and Fire Protection.
CARB, for short, is an agency that's
dedicated to looking at the harmful effects of air
pollution in California.
And Cal Fire is an agency that's dedicated to looking
at fire protection in the private wildlands of the state.
And both these agencies want to simplify
the process of quantifying carbon
emissions from, specifically, fuels treatments.
And what our fuels treatments?
They're basically ways to be able to make forests
more resilient to wildfires.
So if you look at the before image here,
you see that we have a forest that is densely packed,
lots of closely spaced trees, lots of foliage
and understory growth below the trees.
And if a fire were to come into this area,
it would be very intense and very severe.
So the idea behind the fuel treatment
is that we're essentially converting
what you see in the before image to what you see in the after
image.
And fuels treatments consist of things
like thinning, which involves removing trees, mastication,
which is another variation of thinning, just
using heavy machinery to be able to shred trees down,
prescribed burning and pile burning, which involves fires
in the understory to remove all that foliage,
to create what you see in this after image
so that when a fire does come into that area,
it's less severe and easier to control for firefighters.
SPEAKER 1: You can try to stay this way.
The more you move your mouth that way, [INAUDIBLE]..
ADRIANO MATOS: That's OK.
Sorry.
I tend to move a lot.
OK.
OK, so anyway, sorry.
So we want to be able to quantify carbon emissions
from these fuel treatments.
And we're doing this through a model
called the forest vegetation simulator, or FES for short.
It's a forest simulation model.
It models forest vegetation change in response
to natural succession, disturbance, and management.
Essentially, how it works is you pass in tree metrics, things
like the species, the diameter at breast height,
the age of the tree, site metrics
like elevation, slope, and aspect, and then the fuel
treatment scenarios.
That all goes in together into this model
to essentially create these predictions of what
these forests will look like after, say, 25 years.
So bringing everything together, we
want to run this model for the entire state of California
for all forested lands.
So this formula that I made up basically
shows how long it will take to actually run this extent.
There's approximately 133 billion meters squared
worth of forested lands in California.
And each FES run will consist of a 30 by 30 meter plot,
or as I call, pixel size here.
It's equated to 900 meters squared.
I'll be calling FES runs pixels now, just
to standardize the language.
FES will now be just synonymous to a 30 by 30 meter pixel size.
So for each FES run, we're running 10 treatments
and running 100 iterations of those treatments for one FES
run.
And then approximately, it takes 0.7 CPU seconds
to run one FES instance.
And that equates to approximately 29,000 CPU hours.
So this is definitely a big data problem.
This isn't something that we can run on any ordinary machine.
And we need all the computational resources
that we can get to be able to run
the entire state of California and make it timely.
So that's where Google Cloud comes into play.
I'm not going to go into everything
that Google Cloud has to offer.
If I did, I'd be here all day.
But essentially, it's a way for users
to be able to access Google's virtual resources.
It's the same resources that Google
uses for things like YouTube, and Gmail, and Google Search.
So the idea here is that we get access
to these virtual resources so that we
can deploy our applications at any scale that we want to.
And now I get into the weeds of the workflow.
So I know this might be a little intimidating,
but I'm going to break the workflow apart piece by piece
and then bring everything together at the end.
So we're going to start at the top left and work our way down.
So the first step is using Google Earth Engine,
containerized within that Cloud Run service.
So I'm going to explain what those two things mean.
Google Earth Engine, for those of you guys who don't know,
is a JavaScript/Python API.
It lets you analyze geospatial data.
And it's free to use.
As long as you have a Google account,
you have access to this array of data, from satellite imagery,
to weather data, to topographic data.
Essentially, as long as you have an understanding
of programming, you have ways to be able to export this data
and do really cool analysis on it.
So that's Google Earth Engine.
And it's housed within a Cloud Run service.
Now Cloud Run is Google Cloud's flavor
of containerization technology.
For those of us who don't know what Docker or containerization
is, it's basically a way to be able to take
your code or your application, as well as all the dependencies
and packages that are needed to be able to run that code.
And you put it in a box.
And by transporting that box to any machine,
you don't have to worry about reinstalling those dependencies
again and again or have to deal with version conflicts, which
is a common problem with data science and programming
languages.
So if you take anything out from what I'm saying,
Cloud Run is just basically able to let us deploy our own code
in the Cloud environment.
So with that being said, this Google Earth Engine script
takes a search area of interest.
In this case, it's that small, little, red square
you see within the state of California.
We'll eventually run the entire state.
But for the time being, we're just
running that small square for testing purposes.
So we take that search area, and we
use that search area as a cookie cutter
to extract out pixel values from a data set called tree map.
Now tree map is a 30 by 30 meter gridded data
set that was created by the US Forest Service.
It represents, basically, just plot identifiers.
And we use these plot IDs to be able to query
their database that contains all the tree metrics that we need.
So all we're doing is just using that little box
and extracting out the pixel values that represent the plot
IDs that we're interested in taking in the future.
So we take these plot IDs.
We'll just look at, specifically, 980, for example.
And we push it over to this next step
here, where we have a database that contains all the tree
metrics that we're interested in.
And we're going to use that number, 980,
here to query that 30 by 30 meter
plot for all the trees within that plot to push into FES.
So everything that was mentioned before, species, height, age,
all that stuff is extracted in this step.
Now we have a Compute Engine instance, which is just
basically a virtual machine.
If you don't know what a virtual machine is,
it's just a mini computer with its own operating
system and hardware.
And we have MongoDB installed on this virtual machine.
And we have our forest inventory analysis database,
which is the database that has all the tree metrics installed
within Mongo.
So all we're doing here is just simply doing
a query for all the tree metrics that we need
to be able to run into FES.
And then from there, we take all the metrics that we extracted,
and we go into the bread and butter of the workflow.
So it's very similar in the sense
that we're still using Cloud Run.
But now we have our FES step.
So we've taken the tree metrics in,
and we're pushing it into our Cloud Run service
to generate these predictions of what these trees will
look like after applying the fuel treatment scenarios.
Now the key thing that I want to point out
here is that with this Cloud Run service,
you'll notice that there's three copies of FES.
In this case, Cloud Run has a unique feature
where you can essentially parallelize your workflow.
So each FES instance here is getting
its own unique set of inputs.
And this allows us to be able to run the entire state
of California on a large scale.
You're only seeing three instances
of FES being run here.
But right now in our workflow, we
can run, concurrently, up to 1,500 instances of FES,
each of which have their own unique inputs.
And we can do that within seconds.
So we're basically able to generate these products
in seconds and make this really, really
timely to be able to get this extent that we
need to get done.
And then lastly, we take those-- well, second to last,
we take all those forest change products,
and we just write them into a database.
So it's the same technologies that we were using before.
We're using Compute Engine to install an instance of Mongo.
And we just have an empty database where we just
write those outputs into it.
And then from there, we can take those outputs
and do a geospatial query to be able to create
some cool products.
Now in this case, we're more concerned with the carbon
emissions.
So the raster on the left is just
a raster that represents the carbon emissions after applying
a fuel treatment scenario.
You don't really have to be concerned with exactly
what those pixel values mean.
It's just a representation of what we can actually generate.
But that's basically the gist of the workflow.
So to sum it up, from the beginning,
we take the search area.
We extract out these plot identifiers.
We use these plot identifiers to query a database
and take the tree metrics that we need
for our plots, or our pixels.
We push that into our FES step.
We generate these tree products.
And then we write them into a database,
and then do a query to be able to create cool end
products to visualize.
So some key takeaways-- this project
is entirely automated in the Google Cloud space.
So as soon as you input in the search area of interest
in the beginning, the whole workflow inherently
knows how to go from start to finish without any other user
input, which is a really nice feature.
We can also specialize FES.
FES doesn't have an inherent idea
of where the trees are located.
So by using this tree map data set--
I don't really have to go back at this point.
But by using the tree map data set and the search
area of interest, we can essentially
identify any area that we're interested in
and create these products based upon, again, whatever search
area that we're interested in.
We can externalize geospatial analysis
outside the scope of Google Earth Engine.
Google Earth Engine is great.
The fact that it's free to use for any user is astounding.
But we are limited in the sense that it is a shared resource.
So by being able to use what Cloud has to offer,
we can take what Google Earth Engine gives us
and go even further beyond.
As I mentioned before, this project
runs the applications in parallel.
That's what I was showing before with the first step.
And lastly, we can generate spatially explicit
uncertainties for all FES outputs.
So I mentioned briefly before-- and you guys probably
don't remember--
this iteration part.
So for a single pixel, a single 30 by 30
meter section of trees that we're interested in looking at,
we're actually doing a sample of those trees
within that same area 100 times.
And what that allows us to do is,
essentially, we can take the variance of those iterations
and create uncertainty maps to be
able to make sure that we have an honest prediction of
whether or not these forest change products are accurate.
So the next steps would be to optimize
the computational efficiency of our code to save costs.
Despite the fact that it works really well
and it's really fast, it is pricey.
So we're working on trying to port over the workflow
to our own high-performance computing environment
at our university to be able to save some money.
So it's one disadvantage.
It is costly.
But at the same time, it's a really great resource
to be able to create these products in a fast way.
The visualization step is technically not incorporated
into the workflow just yet.
We are just at the stage where we have all the outputs written
in the database.
And we mainly just do the query ourselves.
That will eventually be incorporated into the workflow
and will be automated.
And then the tree map raster, the raster
that we use to be able to grab the plot IDs that we need
to be able to do the query to take out those tree
metrics, that will actually be replaced entirely
with a generated synthetic tree list at every position
within our search area.
That's where the 100 iteration part comes into play.
That will make things a lot easier
for us to be able to create this uncertainty maps.
And yeah, that's basically it.
My poster session is tomorrow.
So if you guys have any other questions,
feel free to bother me or pick my brain with whatever
questions you may have.
Now this project was also a collaborative effort.
I was in charge of just programming it and putting it
up into the Cloud space.
But if you have any questions regarding synthetic tree lists
or uncertainty, my colleague Taylor Brown--
she unfortunately had her poster session on Monday--
but she'd be more than happy to answer questions
regarding synthetic tree lists.
And then my other colleague Ashley Cale,
she was in charge of the whole FES process.
I basically just poured that over into the Cloud space.
But if you have any questions about how FES works,
and how you actually define the fuel treatments,
and how to actually push in the inputs,
her poster session is tomorrow afternoon.
And she was directly involved in this project.
So she'll know everything that I'm talking about.
So feel free to bother her with specific questions regarding
FES if you have them.
And that's basically it.
I just want to acknowledge all my colleagues
at the lab who helped me make this project possible,
as well as a couple of Google engineers, Yvonne
Aquino and Payim Moussavi, who are
customer engineers at Google who gave me
a lot of insight on how to be able to parallelize
this workflow.
And that's it.
[MUSIC PLAYING]