- Welcome to "This is My Architecture."
I'm Orit and with me Liron from Deep Instinct.
Liron, tell us about Deep Instinct.
- Well, Deep Instinct applies deep learning to cyber security.
We use it for real-time detection and prevention of zero data attacks.
- Cool, what are we going to talk about today?
- So providing the zero data attack prevention,
we will need to use Agent on clients in order to provide that.
We use to create an environment in order to provide millions of Agent
that will want to access the web and get information from the web.
Now, we use CloudFront in order to provide all this information.
Now, the Agent will get a request directly from CloudFront
and then CloudFront will use S3 in order to provide the information back.
Now, what we found, that we wanted to create a real-time statistics solution.
The problem is normally that CloudFront will provide logs, which logs are amazing-
they're good, but they're not good for real-time statistics.
In order to provide real-time statistics, we had to use Lambda@Edge
which can provide us a real-time solution for the request from CloudFront.
The idea was like that.
Every Agent will ask hundreds of millions of times a day
information from the CloudFront.
All the Agent together will request everything
and we can tap it in the middle of the request, use Lambda@Edge
at the Viewer request situation, take all this information
and inject it directly into DynamoDB in order to post it in real-time.
- From all the purpose build databases out there today,
why did you choose DynamoDB?
- It's easy, simple, we can change it on the fly
and change anything we want for the statistics
without any hindrance. - Sounds good.
Are there any other use cases you use Lambda@Edge for?
- Okay, so after we did this, we understood the major implementation
and changes that we can do onto our system.
What we try to do is create a rolling upgrade to the Agent.
Now, an Agent will request an upgrade and normally it will go to S3
to get the upgrade.
But, if we don't want millions of Agents to get the update at the same time,
we want to gradually distribute it around the world-
we can tackle the request at the Viewer Request
load anything to another DynamoDB and then see if the request became
for example, 10,000, 100,000 and then stop it.
Use the Viewer Request to respond to the Agent
there is no update right now.
- How do you uniquely identify your Agents?
- Okay, so in order to do that we actually used something called
Signed Request to the CloudFront.
Normal Signed Request is used for security measures
so other one than the Agent will not be able to go to the CloudFront.
What we did, we used to inject a unique identifier inside the request
in order to read it.
Thus the Viewer Request at the Lambda@Edge
will get the information of which Agent directly.
This unique information will provide us even a rate limit monitor
in order to provide, if we see the same request
with the same identifier, go from different IP addresses
then we can know this is a problem. Also, this will help us
get all the statistics to go to the DynamoDB,
because if customer is not under, we cannot see all the Agents.
With this unique identifier, we can recognize any Agent by itself
and provide any solution for it.
- Cool, it looks like you leverage Lambda@Edge
to collect real-time statistics in an efficient way
and also to intervene in the middle of the request before it reaches the region
and keep the details about the specific unique agent.
- Not only that, what we found out is that we can provide a real-time decision
in the Lambda@Edge, the Viewer Request.
What we found out that if we want to do any change to the information that gets
back into the Agent, we can intervene in something called Lambda@Edge
Origin Response.
Once a request come to the Viewer Request,
it will then go and ask S3 for the information,
but the Origin Response does not know
that the Viewer Request did this information.
To do that, we logged all the information we wanted into DynamoDB
and then the Origin Response will get this information back.
What we can do here is we can identify the Agent
because of the unique identifier,
we use it to save the state and know this Agent
request at a specific file and change it after we get it from the bucket.
- So gradual roll outs of Deep Instinct goes to Lambda@Edge as well?
- Yes, we can manipulate any updates, different updates for different customers
and everything according to that.
- Amazing.
Thanks Liron for sharing that with us and thank you for watching
'This is My Architecture.'