Welcome to This is My Architecture.
I'm Hayato Kiriyama from Amazon Web Services.
I'm joined by Masa, Security Engineer of Cookpad.
Thank you for joining us today.
Thank you for having me.
First of all, please tell us about Cookpad.
Yes. Cookpad is a community platform to share recipe ideas and cooking tips.
Since we have 100 million users worldwide,
keeping each user's information is an important task for our business.
I am in charge of security of this architecture,
running on AWS.
So your task in Cookpad is keeping
your services and users secure on AWS.
Exactly.
This is the architecture to accomplish that goal.
Could you please explain what it does?
This is an architecture to monitor security logs.
This architecture helps people who are concerned about security monitoring
and responding to instant, such us security breach.
For example, people who work in Internet Security Operation Centers.
You built this solution for security people.
How does it work?
This architecture consists of three parts:
Raw Collection, Retention and Analysis.
First, Raw Collection part is to gather log data.
From EC2 instance, to entry correct messages
and send them through Firehose, to S3.
From Soundcloud as monitor services, such as Cloudtrail,
they can put their logs into S3 bucket directly.
From 3rd Party Service, their logs
can be retrieved by AWS Lambda to S3.
So it deals with many different types of logs.
So what is the middle part for?
The second part is Retention part.
It saves all logs into S3 bucket,
and S3 life-cycle policy manages their retention period.
OK, all logs happening in S3, and managed by S3 life-cycle policy,
and the last part?
Lastly, third part is Analysis part.
For short term objectives,
Lambda function inspect log data, to raise a hazard
and notify it to [INAUDIBLE]
Also, to utilize log data for longer time,
Lambda function transfer log data to Graylog.
Graylog is an open source search engine,
to store log data to Elasticsearch service,
The ultimate objective is to send long period log data for audit.
It can be processed by Amazon Athena.
Thank you. I got an overview.
And what is the most innovative part of this architecture?
In security monitoring, security engineers should respond to a hazard quickly.
According to a survey of manager security services in general,
the latency of log processing should be under one or two minutes.
On the other hand, there are some processing systems
that handle browser traffic is an expensive number, usually.
This architecture achieves them both
low latency and low level integrity
by scale of AWS Services with low cost.
With low cost. So what cost of benefit did you see with this architecture?
This architecture processes 330 million logs per day.
It's about 270 gigabytes.
The Lambda function that handles these logs
requires only two dollars per day.
Only two dollars per day? Great!
So comparing the general log monitoring solution,
what makes this solution outstanding?
In general, many security log management
require to configure a log schema, before connecting logs.
This architecture does not require to configure a log schema in connection part.
Because S3 does not care of log schema.
I see. No schema makes it easy to cope with logs.
Lastly. Do you have any plan for future development?
Yes, we have a plan to develop a new anomic digital future
for security logs with Amazon Sagemaker.
Awesome. Thanks for sharing that.
My pleasure.
Today, we talked about security monitoring with Masa, from Cookpad.
Thanks for watching.
This is My Architecture.
[Thank you for watching For more information visit aws.amazon.com/this-is-my-architecture]
[AWS]