- [Host] Please join me in welcoming Bob Rudis.
- Thank you.
(applause)
So this is a talk for, I think it works for everyone
but it's kinda geared toward service providers
and large organizations so if you're
in one of those two categories, could you raise your hand?
Cool, hopefully as this package evolves
and maybe you might use R, actually how many people
use R too cause this is gonna be an R focused talk.
Good, there's, nice.
There's a nice tiny focus here.
So this is, what I've tried to do is take like
some of the practices we're doing internally, open-source
them and give tools to folks out there so we can
all make security better at organizations
like help organizations have better security outcomes.
And, so just quickly about me, for about 20 years
I had a real job like most of you
and then I did this thing called the
Data Breach Investigation's Report for a couple years.
And now I'm chief data scientist at Rapid7
and I don't have a real job cause I'm not actually out there
in the trenches having to do hunts, having to actually
keep organizations safe every single day.
I work at Rapid7 labs where we do lots of research
and we do lots of reports and I do lots of work work
but you guys are kind of the unsung heroes of security.
There's a really old book out about this stuff called
Data Driven Security, it's like five years old,
I feel real old because of that.
And if you also would like to play with some data,
opendata.rapid7.com, we basically give away all of our
scan data, so if you want to have at that it's all yours,
just sign up and go to there and we'll do that.
And for folks that use R and don't know my R stuff,
there's about 90 packages out there that are related
to cyber security stuff.
So have at that as well and keep out there.
And there's another one now.
So is this like my obligatory MITRE ATT&CK slide ache.
I've been talking about ATT&CK publicly
for about a year now.
Actually ATT&CK kind of kept me in cyber security,
I was getting really down about cyber for a while
and then when Attack came out I got like really invigorated
again cause it was filling a really missing gap everywhere.
And we've really embraced ATT&CK at Rapid7
especially within Rapid7 labs and our
manage detection and response practice where we kind of
deal with a whole bunch of organizations and dealing with
response for those folks.
And for a while we were hand coding, I think everyone's
probably felt this pain, hand coding our incidents
and the detections from the environment and from
the hunts people did to ATT&CK, starting in January
of this year, that is an automated process
very similar to the coupled other automated processes
that you've heard of here, so I love that theme
that I've heard today.
Whoever organized this again did an amazing job
cause like it cascades down really well.
You can do the automation, it actually will work
and it's actually pretty cool.
I will show you some of the results from that here.
If you wanna see ATT&CK based reports,
our quarterly threat reports starting especially with
this quarter, are all basically doing ATT&CK based metrics
and no other type of metrics that are out there
except for the stuff that we see
at our global honeypot network.
So ATT&CK is kind of used mostly on red and blue teams
right now, at least that's the way I've been seeing it used
as I talk to folks, you're out there in the trenches,
you're doing the work or defending, or you're in the
pentest team or the red team and you're actually kinda
go out there and do things.
But you know, those are two ways of actually using it,
but it actually, I don't see ATT&CK being used at the
layer 8 team a whole lot.
They're also not very colorful at the layer 8 team.
And so my goal with this one, and we do this
on a regular basis is to try to use ATT&CK at a
macro level to convey what's going on across all of our
customers and two each of our individual customers
and at an industry level.
So we have the ability to kind of give an idea of
what's happening, what's missing.
And I'll kinda go through some of that right now.
So this is, all the slides are online,
I'm not a big github fan but I think everybody else is,
so I stick myself everywhere so that you're not locked into
a closed platform.
Sorry, I'm one of those guys, yeah.
So it's all out there and all this stuff is out there
and you can grab it from there but we'll give you a
high level view of it now since we don't have a whole lot
of time to go into detail.
So why R, first of all?
It's why R, cause I like R and I program in R,
I program in everything but I kind of program in R.
So, but why am I doing this in R?
Why am I trying to get you use R to do this?
So this is R, its open-source just like all the other things
that you probably use, it's mission-built for doing
data science type tasks which include visualization
and other types of analysis.
In this particular case, it's got a very rich ecosystem
for doing data visualization and that's one of the things
I like to actually try to do with it.
And R isn't just R, much like Python or other tools,
R can interface with lots of everything so every one
of those lists up there plus the ellipses,
R can interface with pretty much any other
language out there and you can use it really effectively
and a lot more effectively than you can for those languages
from other languages.
So if you haven't tried R, I encourage you to give it a shot
but if you're gonna use R, you really need to use R Studio.
And I'm doing this, like I'm not even pimping R Studio,
it's a free product as well too.
And the reason why I'm saying to use this one, it really
makes R kind of like, and this is the way I kind of use it
and there's other folks that use it this way, it's kind of
the best DFIR investigations console ever made
and even though you don't know about it.
So you can do all sorts of stuff within it, it's a
source code editor, you could execute things line-by-line,
I'm gonna talk about notebooks in a second so if you're
a notebook fan, you can use stuff in a notebook format.
And if you wanna use your favorite language instead of R
in it, you can even in kind of ripple format as well too.
So our studio is a great environment for doing
all sorts of things, that kind of wraps R up and kind of
gives it even more super powers than it already has.
So more reasons for why I use R
so I can calm down a little bit now.
rmarkdown is one of the main reasons why I think R
and doing things within R context work really well.
rmarkdown, there's two flavors of it, a notebook format
so if you're one of those weird Jupyter notebook fans,
this is kind of like that.
I'm not a big notebook fan,
I think browsers are for browsing and you know,
real things are for real things.
Just a little thing, that's all.
But you can also code up reports that you can actually
write in here too, so one of the things you actually get
with this which I'm gonna show you, is like an already
pre-built canned report for actually generating some
initial high level visualizations that will get
expanded over time as the package increases.
So rmarkdown is really cool and it isn't limited to
just the R language.
You can do Python stuff in it, you can do Scala stuff in it.
You can do anything you want to in it.
It's very expandable and it kinda does
a whole lot of things.
It can also do application building, it's a really really
simple application building interface that ties code
and code logic and presentation together in a very
straightforward way so that you have the ability to
create interactive presentations, and one of the goals
of this as you'll see towards the end is
I'm gonna help build an app so that you can actually have
the ability to do this without coding.
So like why, what's the purpose of attckr?
So some of the purposes of it are to answer questions like
what does the distribution of tactics and techniques
look like across my customer base within a customer
in a particular industry, however you wanna
slice up the categories.
Where do we have technology gaps, either as us, a provider,
or as organizations.
Where do we have workforce gaps?
And I'll talk about that a little bit down.
Where should we invest our limited budget in certain
defense technologies or certain defense processes?
How does this thing compare with that thing?
And that could be anything like these tactics' usage
against these tactics' usage
or this attckr uses of tactics or this attckr,
you can do all this because we have the ability to kind of
put things into a particular format.
And with what's the dwell time trend as well too,
I'm a really big fan of dwell time.
In the DBIR we call that the detection deficit,
I have to call it dwell time because we call it dwell time
at work, so that's what I'm gonna do there.
So one of the things you get with this
is a kind of pre-built set of things that make it
easier to use the CTI format.
I think as the previous speaker just said, you can
stare at the CTI JSON, it's like looking into
the heart of the TARDIS, it's beautiful but it will
drive you mad.
And it's not very suited for data science tasks at all.
As I was telling Katie before, I'm really not a big fan
of excel but I'm a big fan of rectangular data.
So I have rectangularized a lot of the CTI data,
stuck it into here so you can use it.
Plus I provided other tools and mechanisms for you
to take that data and enable it to do other kind of
data science tasks with that or visualization tasks on there
so I'm not gonna through those all in here, I wanted to put
it here as like a teaser for what you can do with it
to have it out there.
But the thing about this is every time MITRE updates
the CTI format this goes out, grabs it,
rebuilds the package, re-tidies all the data
and puts it back out there so you have no work
on your end to kind of do go do this.
So what I did for stuff I'm gonna show you today
is we use like, we actually use MITRE,
the MITRE ATT&CK framework
and we code all of our incidents in it like I said,
some of it's through automation, some of it's through
the hunt data that the folks actually do themselves
for things.
This is a sample of JSON that I'm using that's kinda
the foundation of the data that I've got all the
visualizations based on here.
Don't get hung up on this JSON, I'm gonna support using
whatever you guys want.
So whether it's the STIX format or whether it's the new
MITRE ATT&CK format for doing like the sharing of these
things that's gonna be out there.
I'm hoping, I'm really hoping that like Varisk and ATT&CK
have some kind of lovechild that give us the ability
to have a really cool data set that gives us this
macro view of these things as well.
So v2 of attckr or v0.2 of attckr is like,
well what do you get out of the box one now,
and you're gonna get the ability to analyze dwell time
metrics, produce ATT&CK heatmaps in a way that I like to
produce them and I'm hoping to convince you of this.
An awesome way to look at what I like to call
Cumulative ATT&CK Tactics Distributions.
It's a fancy sounding thing that hopefully
won't be too complex.
The thing on the left is one giant html page
that got generated through an rmarkdown report.
What you see copied and pasted across all over there
are some of the expanded sections, so lets just go through
some of them instead of actually
trying to stare at that screen.
So this is all, you basically load up you create,
this is all done by actually just going into the package,
starting a new rmarkdown file that's based on this template,
giving it your data and then it magically
makes all these charts for you so you don't have to do
like literally any work except
get your data into the format.
So I took a sample of all of our incidents,
I created, actually I took all of our incidents,
I created distribution based upon all those incidents
and then I created a, like I selected a sample
of that distribution for putting into the package.
So you get real life incidents or at least
a picture of real life incidents based on the
distribution of incidents that we've actually seen
with me removing some things so I could have
some talking points today.
So one of them, one of the things you get to see
dwell time based upon this incident data set.
And the cool thing about this is like, the whole idea is
you wanna actually reduce dwell time, right?
So like you get the ability to show these reports,
have them available and also look at them over time.
So this is looking at those for the overall one,
this is looking at it by quarter, and then it's looking at
the overall and it's a little harder to see with the
colors but you basically get the over time view
of how you're doing with the dwell time.
So you can communicate that back to show how well
your technology is doing or how well your folks are doing
kinda working at these things.
You can generate that in a bunch of different ways
so there's a heatmap view of the dwell time as well.
And then you can actually look at it more specifically
and this is a better view than the other one
to kind of look at dwell time specifically by month
instead of by quarter.
So it's lots of slicing it different ways.
If you want more views, submit an issue.
I don't care where in those repos you do
if you want to submit an issue, I'll be glad to put it
out there, it's actually not that much work.
It's also not that hard to copy and paste the template
code to do it yourself.
So I love this heatmap versus the ones that I've seen
generally produced which shows the entire framework and then
you've got like little things colored here and there.
So what I've done is created a way to
with one call to one R function generate a heatmap
that's sorted by the most occurred technique within
that particular tactic and then put it out there.
You can do it proportion, like this way you can do it
proportionally, you can do it over time.
I'm gonna go back to this one real quick.
So what I've also done on here is I've deliberately
left out some things that we had because you get to
immediately see gaps.
Right, so you know that this is nowhere near as big as
the entire MITRE ATT&CK framework.
You also see that privilege escalation discovery
and xfill are missing from this particular one
and I did that on purpose.
I'll talk a little bit more on why that's important
in a minute, but being able to see where things aren't
there is almost as important as seeing where things are
or what things are where because you might have to deal
with what those gaps actually are out there.
So you can see it over time as well too, to see what
changed over a quarter.
And so I wanted to kind of go to this one
and almost land on this one.
So this is a cumulative ATT&CK distribution and the
number's a little harder to see, it's probably easier
online, you can grab the slides and look at it.
So what I've done is, it accumulates all of the tactics
that have occurred, all the techniques within a tactic
that have occurred and we basically sum all those up,
we get the percentage of those and we just add those over
since we actually have a really nice open-source of...
I can do this.
A really nice opensource kill chain that we can use to see
the things progress over to each individual category.
So what you're seeing, and I think there's a button...
So you can see there's gaps and privilege escalation
and lateral movement, and what we can do at work
which will eventually be in the package is
drill down into both what's in each individual category,
so what are you finding versus what aren't you finding.
But then you also get to see like what isn't there.
So why isn't that stuff there?
And it's a great conversation to have about maybe
your, the folks that your customers might have
technology gaps, like they might not have things that detect
things that are feeding into your environment
to be able to find these things.
You may have gaps in your log parsing for those things
or you may have incident responders that don't like to
deal with those types of types of events and just
kind of ignore them.
So this gives you a way to identify those areas and see
where they are and kinda maybe have some like,
more tools at your disposal to investigate
and dig into things a little bit better.
Then you can also do really cool things like for us,
do it by industry, so again it's a little harder to see
but this is basically a combined cumulative graph
that shows how each individuals' industry
tactics and techniques distribution looks compared
to each other, it's pretty much like the others,
a similar pattern but some of them look a little bit
differently, I did sort of jury-rig this one
to make it look differently.
But this way you can see what's happening across
all of those, have the ability to do more stuff.
So what are we gonna do?
We're gonna add more charts, we're gonna add more
discrete timestamp support, more functions,
hopefully get to the point where there's
zero coding skills required and give you some
event stream analysis so basically think MITRE ATT&CKs
with incidents and then do clickstream analysis and that
to be able to do really interesting distribution checks
against what might be the most prevalent thing
in your environment and why.
And I got 30 seconds.
- Nailed it, well please join me in thanking Bob.
(applause)