in our last episode we covered best
practices for deploying and using cloud
monitoring today we'll start looking at
best practices that you can use to get
the most value out of cloud logging
we'll cover how to ingest and store logs
and we'll cover the best ways to query
logs and manage logging costs in our
next episode welcome to engineering for
reliability with google cloud
before we can do anything with logs at
scale our services need to create them
and they need to be ingested in cloud
logging
we covered the sources of logs in google
cloud in the previous episode but let's
quickly review
there's nothing you need to do to
control how logs are emitted from the
platform and network
but when writing your own logs there are
some best practices you should follow to
get the most value out of that data
this is where structured logging comes
in
consider a log entry that looks like
this
where the entire payload message is in a
single text payload field
if you want to get any data out of that
field you have to use regular
expressions or other string processing
methods
using a structured logging approach like
this to create log entries with the json
payload field that then has nested
fields under it allows you to query and
analyze log data much easier
every field within json payload can be
used to search in query logs just like
other fields
you can even add it to the logs field
explorer to enable easily grouping and
searching by that field later
but how do you get your application
locks to look like that
if your logs are written by the cloud
logging or the cloud apps agents use one
of the supported parsers to emit
structured logs or write your own parser
if needed
if you're using the cloud logging api
send the full log entry structure with
the json payload
finally if you're using either the cloud
logging library or a third-party login
library in your code
make sure you're using them in the way
that emits structured logs
we'll link to the documentation for each
of these in the episode notes
writing logs in a way that allows you to
make the best use of the data in them is
great but what's the best way to store
them to balance security retention and
access
let's have a look at log centralization
configuring retention managing access
and using exports
first keeping your logs in a central
place helps you manage all your logs
consistently and efficiently
we talked about using log buckets in the
previous episode but remember that for
each cloud project billing account
folder and organization
cloud logging automatically creates two
log buckets required and default
you can create additional buckets and
configure log syncs that send logs from
multiple projects to those buckets
it's often advisable to do this for
audit logs as an example
your business needs and requirements
will likely govern how long you need to
retain logs for
you can configure the retention period
for any user defined log bucket and also
for the default lock bucket
simply edit the bucket under log storage
and set your preferred retention period
up to 10 years
you can also control where the logs in
the bucket are stored you also have the
option to set the location to global if
you don't need to specify where your
logs are physically stored
there are two mechanisms you can use to
control access to logs and buckets the
first one is via iam permissions we'll
link to the detailed guide in the
episode nodes but a user will at least
need the logging viewer role to be able
to read logs in the project
the second mechanism to control log
access is log views which provides you
with an advanced and granular way to
control access to the logs in your log
buckets
for example consider a scenario in which
you store all of your organization's
logs in the central project
in this case you might want to control
which projects different users can view
logs from
using custom log views you can give one
user access to logs only from a single
project
while you give another user access to
logs from all the projects
finally you may need to export logs to a
third-party system such as splunk
you can do that using a push-based
method through the splunk http event
connector or pull-based method using the
splunk add-on for google cloud platform
we covered best practices for ingesting
and storing logs in cloud logging and
controlling log access retention and
exports
i hope these recommendations help you
keep your services reliable and your
users happy in our next episode we'll
tackle best practices for analyzing logs
and managing logging costs thanks so
much for watching today don't forget to
like and subscribe to never miss out on
more engineering for reliability with
google cloud see you soon