[MUSIC PLAYING]
JEFF JOHNSON: Welcome back, developers.
Jeff here, and I'm joined once again by our friend Alpha
from Pivotal.
Alpha is here to tell us about how
to handle monitoring and logging to our Cloud Foundry
applications.
Last time we were together, we talked
about how to monitor our infrastructure,
but now we're talking about the Foundry.
What's next, Alpha?
ALPHA CHEN: The next step is connecting the Cloud Foundry
logging and metrics pipeline, also called Loggregator,
to Stackdriver.
JEFF JOHNSON: So this would be data
ranging from running application logs to health metrics
such as container CPU usage.
ALPHA CHEN: That's right, Jeff.
Loggregator is designed to hook up
to external data sinks, which Cloud Foundry calls nozzles.
The Stackdriver nozzle connects to the pipeline
through UDP today, potentially switching to TCP in the future,
and receives a [INAUDIBLE] stream
of data from Cloud Foundry to prevent backups in the system.
The nozzle can be scaled horizontally
as necessary to handle increased log traffic.
JEFF JOHNSON: That's why we, and I mean a great team
from Pivotal and Google, including us,
developed the Stackdriver nozzle.
This open-source project connects the Loggregator
to Stackdriver logging and metrics.
It is available on GitHub as part of our Stackdriver tools
BOSH release.
Alpha, can you show us how to get set up with the nozzle?
ALPHA CHEN: Let's do it.
First, we need to enroll our GCP project and the Stackdriver
metrics API, and ensure our quotes are
set to the appropriate limits as documented on our GitHub.
Deploying the nozzle can be done by modifying the BOSH release
or deploying the tile through PCF Operations Manager.
JEFF JOHNSON: Let's use the BOSH release for this.
ALPHA CHEN: Sounds good.
To deploy the Stackdriver nozzle, first upload the latest
release from our GitHub repo to our BOSH director,
and then add the release to the manifest.
Now create a pool of nozzles with some light configuration
to connect to Loggregator in our BOSH manifest.
JEFF JOHNSON: This sounds like a great time
for, and here's one I prepared earlier.
ALPHA CHEN: Let's paste it in for now.
We'll have a more detailed example in our GitHub linked
in the description below.
Let's also run the world's fastest BOSH deploy.
JEFF JOHNSON: Voila!
So, we've got logging.
Let's prove it.
Logs generated by our application
are streamed into the Stackdriver logging console.
You can see the logs appear in near real-time as they happen.
The logs we generated show up with detailed metadata
related to the GC instance and the Cloud Foundry application.
One great use of this is to filter down
to logs from a specific application.
Streaming logs can be useful, but often, you
want to perform aggregations and analysis on your data.
This can be done in real-time by creating a log sync
from Stackdriver into BigQuery.
ALPHA CHEN: Every message that CF generates
maintains a structure which we can use in our analysis.
Let's see what unique user agents have
hit our Cloud Foundry today.
JEFF JOHNSON: Sure.
Let me test my minimal BigQuery knowledge.
So we want to filter by HTTP start stop events.
And I'm going to add a simple GROUP BY statement.
OK, just find it down here.
Here's that structure we talked about.
And I'll just go ahead and select that value,
so we can see it.
And there you go.
There's a list of all the user agents
that hit our foundry today.
ALPHA CHEN: That's awesome!
So now we have a full logging pipeline
backed by CF, that Stackdriver, and Google BigQuery.
So let's take a look at those metrics
we've been talking about.
Why don't we open up Stackdriver monitoring
and take a look at our user-created dashboard?
JEFF JOHNSON: This is another cooking show moment,
but the creation process was just
Add Chart, click, click, click.
So we'll just go ahead and skip it.
ALPHA CHEN: CF exposes a ton of metrics
on the health of your Cloud Foundry and applications.
The Stackdriver nozzle handles these metrics
and sends them into Stackdriver monitoring.
This dashboard goes beyond what we can measure at the VM level
and gives us a look into the huge number of metrics
that Cloud Foundry exposes on each container.
JEFF JOHNSON: Cool.
So we can see information like CPU activity of an application.
And we can do that at the space, org, or even application level.
ALPHA CHEN: This sort of view lets operators and developers
better understand their applications
and make scaling decisions.
The user can also create alerts to detect abnormalities.
JEFF JOHNSON: And that's application monitoring
with Cloud Foundry and Stackdriver.
ALPHA CHEN: And thanks for watching our series on Cloud
Foundry and Stackdriver.
JEFF JOHNSON: For more information about Stackdriver,
and Cloud Foundry, and Google Cloud Platform,
check out the description.
[MUSIC PLAYING]
Check out our last video on host monitoring with Stackdriver,
and be sure to check out our playlists for more videos
on Pivotal Cloud Foundry.
Don't forget to subscribe.