Welcome to Solving with
AWS Solutions, your deep dive
into reference architectures
built by AWS that are ready to
deploy instantly in the cloud.
I'm Ashley and today I'm joined
by Lilat to tell us about the
Centralised
Logging Implementation.
Organisations look to analyse
their logs in order to obtain
valuable insights that can help
you with troubleshooting,
security and compliance
and business insights.
This solution will provision a
secure infrastructure to
automate log aggregation from
multiple AWS accounts and
regions so you can spend more
time on actual log analysis.
Let's take a look.
OK Lilat, let's get started
with the landing page.
Can you walk me through that?
Sure.
So the landing page will help
you find the implementation
guide to help you learn more
about the solution.
You can also review the
architecture diagram,
which will explain all the
components that are deployed
by the solution.
As well as if you want
to dive deeper, you can look at
the source code here as well.
OK, let's talk about
the architecture here.
I see that we have
three key components.
Let's talk about it.
Let's talk
about component one,
log ingestion.
The solution creates the cloud
watch log destination in all
the regions by default but if
you wish to deploy that log
destination in very
specific regions, you can do so
by changing the parameter
inside the
CloudFormation stack.
You would need to create a
CloudWatch logs subscription
filter in each of
the spoke accounts.
Once you create
that subscription, all the log
events will come through
directly to
the primary account.
In component two, this is where
we focus on the log indexing.
The Kinesis data streams is
used as a target for the Cloud
Watch log destination.
All the log events make it to
the Kinesis streams that will
invoke a Lambda function with
that log event.
The function of this Lambda
function is to transform that
into an Amazon open
source service document.
Once that is transformed we
store that into Kinesis
Data Firehose, that Data
Firehose that indexes those,
these events or these documents
into the open source service.
OK, so the third component
about visualization looks like
I would use Kibana to be able
to access that data and Amazon
open search service.
Can you talk a little bit more
about that?
Sure.
So we wanted to make it easy
for you to see and
visualize this data.
So out of the box, we also give
you a demo spoke stack that you
can deploy in
your spoke account.
We create the CloudWatch logs
subscription inside this Cloud
Formation stack, as well as we
deploy our web server.
So I have deployed this a few
days ago and we have been
collecting data so that we
can visualize it.
Let me take you to
the Kibana dashboard.
Once we have the
Kibana dashboard, we are
visualizing all the index
documents that we have an
open source service.
So here we can see the cabana
dashboard is visualizing all
the open source documents that
we have indexed in the open
source service domain.
You can change your window from
today to do a week if you wish
to see the data here that will
expand into the whole week,
and you can easily visualize
your 200 code count and 404
code count, as well as number
of packets that are coming in
that are being accepted,
as well as being rejected.
If you want to dive deeper you
can also look at all the log
events at a very
specific event level.
Here you can easily identify
where is that event coming from
?
What is the source of of that
of that event?
If you want to filter it,
you can also use this search
box here to filter only
the 404 errors.
So you see, we wanted to make
it easy for you
to visualize,
explore, filter your data that
has been collected from
multiple regions
across your account.
Well, that's that's wonderful
and pretty simple.
Can you talk about how this is
really beneficial for my
business?
Sure.
So we wanted to make it easy
for you to visualize and
explore all of your data from
multiple accounts and multiple
regions in
your AWS environment.
This solution helps you collect
all that data and help you just
focus on the visualization and
exploration of that data.
Great, thanks so much for
walking me through this Lilat.
You're welcome.
Thanks for joining us as we
explored Centralised Logging,
a solution that lets you easily
deploy resources to essentially
log and visualize your AWS
environment and applications
across multiple accounts.
Check out the solution and many
more on the
AWS Solutions website.
See you soon!