Hello, Cloud Gurus and welcome to AWS Service Spotlight. In this series,
we put the spotlight on specific AWS services,
walking through detailed technical demos to explore their features, benefits,
and costs.
I'm Nick Triantafillou and today we're going to talk about Amazon Detective.
Announced that re:Invent 2019 and made generally available in March 2020,
Amazon Detective is a service that makes it easy for customers to conduct fast
and efficient investigations into security issues across their AWS
accounts. It simplifies the process of analysis, investigation,
and identification of the root cause of potential security issues or
suspicious activities. In this episode,
I'll give a rundown of Detective, taking a look at its features and costs,
and I'll run through a demo to show you how it looks in action before providing
my final verdict. Let's get into it.
Amazon Detective automatically collects logs from your AWS resources.
It then uses machine learning,
statistical analysis and graph theory to build a linked set of data.
And it then presents this data to you in the form of useful graphs and tables so
that you can conduct your security investigations more efficiently.
Its main use is to determine issues you may not even be aware of by monitoring
the activity of your AWS accounts over time,
and then using its intelligence to see if anything that occurs differently to
that. Take a very quick example of running,
say 10,000 virtual servers in EC2,
and if just a single one of them gets hacked,
maybe a hacker starts mining Bitcoin on it,
Detective will let you know which instance it's happening on,
when it started happening within about 30 minutes of it starting,
which is really impressive. Let's talk about the costs.
Amazon Detective has a free 30 day trial. There's no restrictions at all.
You get to try all of its features in any region where Detective is offered.
After that you pay $2 per gigabyte for the first terabyte of data
ingested per account, per region, per month.
And then there's a sliding scale after that, where the price halves at 4,
5 and 10,000 gigabytes costing $1,
50 cents and 25 cents per gigabyte. I'll give you two quick examples.
500 GB of data injested per month is going to cost about a thousand
dollars per month.
Eight terabytes of data ingested per month is going to cost about $7,500 per
month. It may seem pricey, but the majority of the data being ingested,
are logs from various AWS services, which are just text files,
and you would have to really be doing something special to generate even one
gigabyte of text log files per month, let alone eight.
And if you are hitting those terabyte numbers,
then this service will probably save you more money than you're paying for the
service. Going over the benefits, Amazon Detective is fast and effective.
The way it can monitor what's going on across multiple accounts will save you
hours of investigation time. It also has some great visualizations,
which we'll show you right now. So let's go and take a look in our demo.
Okay,
we are here in our AWS console in the Amazon Detective service section.
All we have to do is come over here and click start investigating and it shows
how to set up the service. This one I've already had set up for a few days.
One important thing to note if you're going to try out Amazon Detective,
is that it also uses a different AWS service called Amazon
GuardDuty.
And it receives information from GuardDuty as well in regards to threats that
have been detected on your AWS account.
And the logs from that are some of the logs that come into Detective to help you
investigate the issue. So what I can do here is come up to search.
And this is where we can find a finding or an entity.
A finding is something that this service has intelligently found in your
account, and it thinks you should investigate. An entity
just refers to something in your account. So an EC2 virtual machine,
that is an entity. Even an IP address is an entity.
So because I SSH to my EC2 machines,
I can actually look up my own home IP address,
and that will be an entity that this service is tracking.
So what I can do here is come to choose type. We can look at EC2 instances,
IP addresses, roll sessions, users,
AWS accounts attached to this.
You can invite lots of your other AWS accounts and it all gets consolidated into
this primary account.
But what I'm going to do is have a look at a GuardDuty finding.
These are the things that you really want to investigate.
Now let's pick one of these at random. I'll go for this one here.
And this says that this particular IP address is performing an SSH
brute force attack against this EC2 instance in particular.
So what Detective is now going to let you see is all the information it has
collected about this particular incident. So if we come down,
we can see that it happened around this time. The severity is 40.
The finding direction is inbound,
which means something is happening to your server.
And not you doing something to someone else's server.
It shows you the exact AWS resource involved.
So imagine you have a hundred AWS accounts you're monitoring.
It can immediately show you it's happening on this account and it's happening on
this EC2 instance. If you come a bit further down,
it gives you more information about this EC2 instance.
It tells you when it was created, it's ARN, how it was created.
So this one was created by a CodeStar worker that I was using ,and we come
down a bit further, it shows you any other finding associated with this.
We just have the one finding.
So that's all is listed there for this particular EC2 instance.
Now we look at our behavior graph where you can see the service has actually
plotted your VPC flow volume.
So the amount of network traffic coming into your EC2 server over the past
couple of weeks, and this is a great way to detect if something has happened,
because if you've just got a baseline and then a spike,
you can imagine something unusual is going on.
And this is where the machine learning and the AI built into the service detects
that something's going on. Same here,
generally we don't have a lot of inbound traffic, but we have two spikes here.
So it thinks, well,
something has gone wrong and you can come down here and you can actually get
more details about this particular time period. We come down here,
we get more information. So was it a TCP or UDP packet?
Was it inbound? Was it accepted or rejected by our security groups?
And you can see,
I actually have almost 2000 pages of this kind of data of all the things coming
into the server. And if you remember,
I said every IP address ends up being an entity.
We could click a random one of these IP addresses and Detective will tell us
everything it.
Knows about it. So.
This IP address isn't calling any API services on my account.
We have a finding associated with it as well.
It looks like it has been observed for about three minutes a few days ago.
We can even look at any new behavior. It looks like there's none there.
And what resource interaction does it have? Well,
it communicated with this IP address and it was for
236 kilobytes inbound, and about half a megabyte outbound.
Now something I just recently enabled was a whole bunch of sample findings in
Detective, just so I can show you what other things you can look up.
And the best way to look at this is to actually bring up the GuardDuty console.
So if I come to services and load up GuardDuty, as I said,
GuardDuty is the partner service that can detect things going wrong,
which it then passes to Detective. So you can see here,
it has generated a whole bunch of different things.
So you can see all the investigations it's constantly doing.
It will detect if someone is trying to do a DNS denial of service towards your
EC2 instances, trying to brute force SSH, as we've seen.
if someone's running a Bitcoin mining tool,
if the traffic volume is unusual, and all these different kinds of things.
And even in this GuardDuty console right here, if we select one of these,
you can go to actions and you click on investigate.
And that brings you straight over to Detective as well. In this instance,
it's not working because it's just a sample, but in a real world example,
something like this will come up.
You can click on it.
And then you can delve deep into what the issue was,
find out where it came from, and hopefully either patch up your firewall,
change your security groups and try and prevent this issue from ever happening
again.
So I hope this has given you a great little introduction to how Amazon Detective
works, how it works together with Amazon GuardDuty,
and gives you the resources you need to investigate any security issues that are
occurring across your AWS accounts.
So what's the verdict.?Let's go over some pros and cons. As we saw in the demo,
Detective had great interactive visualizations,
which made it easier to drill down into either the root cause of an issue or the
location where the issue originated.
It was simple to use with its clean UI, which anyone could click through.
And it's a great time-saver being able to alert you to issues you might not even
know about,
and also tell you when they started occurring and where the issue is occurring
as well.
It's nearing Minority Report efficiency of stopping crimes before they even
happen.
The only cons I can see with Amazon Detective is that if you want to try it out
for a quick demo,
you actually need a partner service called Amazon GuardDuty running for at least
48 hours before you can enable Detective.
This can be a bit of an issue if you just want to do a quick evaluation of the
service. And again, I'll just mention the price as a con,
but it really depends on how much data you are ingesting into this service for
a regular user. One gigabyte of logs per month might even be excessive.
So it won't cost you much more than $2 a month.
Huge data driven organizations may face a heftier bill though.
So should you invest, evaluate or revisit? Well,
my recommendation is to evaluate.
This is definitely a service which could be useful for certain organizations and
maybe not so much use for others. And that's fine. I suggest you evaluate it,
pitch it against your own needs.
Try out the free 30 days and see if it finds anything interesting in your
account that you didn't even know about.
I know I discovered at least the one web server running on an EC2 instance that
I forgot about in one of my accounts. Now,
if you'd like to learn more about Amazon Detective,
be sure to check out my newly published Amazon Detective Deep Dive course
available now on A Cloud Guru. Okay, that's it for today,
keep being awesome Cloud Gurus,
and I hope to see you next time on the AWS Service Spotlight.