I think the first one is, obviously, logging and metrics.
Those are extremely popular solutions for us.
And we don't stay stale with logging and metrics.
The environment changes every day.
When it comes to trying to capture log files, we try to capture metrics.
The best example is that now if you are using services and tools like Docker and Kubernetes,
if you are re-architecting your service, your system to have many more services that are
single-purpose, versus one service that is multi-purpose, you will have a system with
services coming and going like crazy.
And you want to be able to have a system, or a way to monitor those applications, in
an efficient way and hopefully in a way that you don't have to go and try to change configuration
files every day.
So we've really embraced that, and embraced this new way of building applications, and
we've done a lot when it comes to autodiscover various services that are running in a common
platform, or when it comes to trying to being able to monitor or tail log files that come
from systems like Docker and Kubernetes.
And in order to show you what we've done over the past year, I would like to welcome Monica
on stage to talk about it.
Hey, Monica.
(Applause).
>>MONICA SARBU: Thanks, Shay.
Hi, everyone.
A critical feature for any monitoring solution for Kubernetes is to be able to dynamically
adjust the configuration when something happens in your cluster — because you don't want
to change the configuration of the monitoring solution each time a new port or a new application
is deployed in Kubernetes.
In Beats, this is solved by the autodiscover feature.
And to demonstrate that the autodiscover feature, I would like to show you this new Kibana app
that we have been working on recently.
It is still a work in progress, it is called Infra-UI, and it is a great way to show you
an overview of your infrastructure or running services.
In the Kubernetes tab, you can see the different pods, and they are grouped by pods.
You can group by container, name space, node, and so on.
And also, you can filter them to select only a few pods by the same criteria.
If we click on one of these pods, we will be able to see this dedicated, curated dashboard
that gives you an overview of your pod.
And it gives you metrics, like CPU usage, memory usage, network traffic, and so on.
Metrics are collected with Metricbeat and also logs that are coming from that particular
pod that are collected with Filebeat.
I want to show you a simple deployment that
I did, it is a deployment for NGINX.
And for that, I need to set the filter to NGINX deploy.
And, as you can see, I have here two NGINX pods that are running on two different nodes.
And now, let’s scale my deployment to five NGINX pods.
And for that, I'm going to use the QCTI where I will scale the replicas to five.
It take a while for Kubernetes to create these new pods, so if we go back to the UI,
we will see it in a second.
Here we are, we have five pods.
And it is cool to see how they show up in real-time in this UI, but this is not sufficient
for monitoring Kubernetes.
Besides seeing a list of all of your ports, you want to see, to monitor the service running
in each of these pods.
In our example, NGINX and for that, we need to run the NGINX model for Metricbeat to collect
the metrics from NGINX.
You have to run the NGINX module in Filebeat to collect and parse the NGINX logs.
Okay.
So Beats 6.01 is coming with these manifest files that we are using for deploying in Metricbeat
and Filebeat that makes it easier for you to deploy Metricbeat and Filebeat on Kubernetes.
And basically, this is the Metricbeat manifest file, and here, what I did, I added this section
to configure autodiscover, and here, I instruct Metricbeat to start to listen for Kubernetes
events.
And when a new pod with a container in NGINX starts, then it enables the NGINX module to
be able to deploy the NGINX service inside that new pod.
And here it is, you can see, I am setting this config template to configure the NGINX
module.
And here, under the host, is constructing the url of the NGINX service based on the
properties of the pod.
And similarly, we are doing this for Filebeat as well.
And here, I'm configuring autodiscover, and when a new pod with container name in
NGINX is started, then NGINX module in Filebeat is started to be able to collect and parse
the NGINX logs.
If we start this, and then we would be able to see this under the under this dashboard
that I prepared.
So this is a dashboard that shows you an overview of the NGINX service.
And, as you can see here, I have five pods.
And also, I can see the logs that are collected from all this NGINX instances and, as you
can see here, I have the NGINX server logs and I also have the access logs.
And as you can see here, the access logs are already parsed.
This is because of the ingest pipeline from the NGINX module in Filebeat.
Now, let's go over to Metricbeat overview, and here you will see the aggregated metrics
that are coming from all of these NGINX instances, and you can see things like active connections,
request rate, growth rate, that are coming from all of these NGINX instances.
Or, we can select an NGINX host, and then we will be able to see all these metrics that
are for this specific NGINX host.
That's all.
Now back to Shay.
>>SHAY BANON: Thank you very much, Monica.
(Applause).