- So our next speakers are Jose Barajas
and Stephan Chenette from AttackIQ.
Excel spreadsheets have been a common
and sometimes derisive topic at ATT&CKcon in the past
but it turns out there actually are other tools out there
for working with ATT&CK.
Jupyter Notebooks can be a really powerful tool for working
with data and we use them ourselves with ATT&CK.
Next talk's gonna share some of the ways
they've been able to use this tool
to tell different kinds of stories with data.
So please welcome to the stage Jose and Stephan.
- Perfect, (indistinct).
Everybody hear me okay?
Perfect.
So I'm Stephan Chenette, I'm the founder and CTO
of AttackIQ.
We have who am I slides
but my focus is typically on setting the technology strategy
and direction for or scenario based
ATT&CK Emulation Platform.
The other thing I work on is working with our data analyst
to take all the data that our platform is accumulating
and help our customers and our clients
to really visualize it so that they can tell better stories.
And in fact, that was part of the influence for this talk.
- Cool, and then, hi everyone.
My name is Jose Barajas.
I'm a technical director here at AttackIQ.
I lead our sales engineering team.
I also lead the research on behalf of the company
that we do with minor ingenuity.
So those paper pamphlets you're seeing around,
we're a large part of that.
And I get the pleasure of leading that
on behalf of the company.
So with that, let's go ahead and get started
and walk you through how we help our customers
better communicate with their teams
through Jupyter Notebooks.
And to start, I've seen this theme
starting with Selena as part of the opening today
is storytelling seems to be a common theme
throughout this today.
And I think that just goes to show
how difficult communicating, right?
Whether it's with other analysts that you're working with,
your directors or the CSO or C-suite level,
or even maybe even in the board.
It's quite difficult so we've done quite a bit
in that area and wanna share a little bit
of how we've addressed that.
Telling a story is nice, telling a story is great,
but we're able to back it up
through empirical data evidence.
It just brings it home, right?
So it's nice to hear the nice story
and then we can show the analyst
all the details that actually back it up.
It puts it all together very beautifully.
And what I heard from the team at MITRE just now
is you guys are using Jupyter Notebooks today.
A lot of you in this room are probably
using Jupyter Notebooks today and I, myself,
throughout my career have used them.
It's an amazing way of just pulling in different data sets,
doing a quick analysis, sharing with other folks.
And today we just wanna show you how we've been able
to take the work that do as analysts
and do a bunch of technical things
but use that same mechanism to actually
express back to our teams that I mentioned earlier,
the findings and the stories that we wanna tell them.
Now, before we get into those details,
let's talk a little bit about Jupyter Notebooks
and what they are.
At the end of the day, for those of you that may not know,
Jupyter Notebooks is just an open source tool
that lets you put together code
to put together a bunch of processes.
And for myself, this is how I pull together data sets
from a bunch of different APIs.
Results as an example, minor data,
think of like mapping (indistinct) as an example.
And at the end, these Jupyter Notebooks
are super valuable because that's where I do my work
day to day.
And then it's easily nowadays,
can be shared through either Google Collabs.
They have a desktop app.
So it's really very accessible now.
And I think that's one of the most valuable things
that these Jupyter Notebooks have brought as of late.
And at the end of the day, they're made up
of these three simple components.
First of all, a text block.
Always perfect to set up the stage, right?
You let people know what we're talking about.
The text block's gonna allow you to do that.
Simple markdown is really what they are.
From there, we have the code blocks.
That's where we're gonna do all the work,
we're gonna pull in the results.
We're gonna pull in data from a bunch of different APIs,
do a bunch of calculations and finally visualize that
and put that together into something that tells a story.
And these Jupyter Notebooks, the reason why
they're so valuable is their notebooks.
The core reason why they were created is for scientists.
As you're doing a bunch of work,
you can now share it with someone else
and they just swap in their data
and they can do the same analysis.
And as you'll see with the presentation today,
that also means that as you're doing your day to day work,
the blocking and tackling that you have to do,
you can take that same data
and just create a quick visualization
to express a thing that you wanna express to your team.
And we'll start off with one example
which is gonna be focused on taking the work that we've done
and expressing it to the executive and CSO at that level.
So I'll hand that off to you Stephan.
- Perfect, thanks Jose.
Okay, so for the sake of a few of these examples,
imagine with me that your security team has been asked
to assess its security controls.
Assess them via different, either threat actors,
or as we heard in the earlier talk, various campaigns
with unknown named actors.
Now to do that, you're gonna use some type of tool
or platform to emulate a lot of that behavior.
We're very big in ATT&CK Flows, ATT&CK Graphs
not just looking at it from an atomic perspective
but actually looking at it as kind of a multi-asset
chain of behaviors.
So that behavior happens, part of assessing the controls
is taking note and putting all of that information,
some type of repository database, something accessible,
hopefully via an API.
So part of the Jupyter Notebook experience
is that you can access all that data.
So typically after you kind of run these exercises,
you maybe use a PowerPoint, dump it all to an Excel sheet,
it'd be fairly static as an experience
when you're presenting it.
Maybe that happens a week later and you're like,
we assessed our controls.
Here's how we're doing.
Our endpoint failed here or network control failed here.
Here's how we're doing against
FIN6 or this particular campaign but it's static.
I don't know when that presentation was given.
If I see it a month later, how old is that data?
Let me offer a different suggestion
in terms of how you can present it.
Now, Jose said he's used Jupyter Notebooks forever
as an analyst.
We're sitting on our computer, pulling in data,
going through it, interacting in our notebook,
in our workbook.
But what we've seen be very valuable is actually to use
Jupyter Notebooks as a presentation mechanism.
So amongst your team of five, 10, 15 analysts,
someone on the security team, whether it's the CTI team,
whether it's the threat (indistinct)
and really management team going through the data
and actually pulling it in real time.
And as Jose said, a Jupyter Notebooks has a text box
with markdown language.
So you can put a header, explain it, much like you would to
either an executive, someone on the operations team,
someone on the tactical level.
Has some code that you can pull data from,
whether it's the taxi server pulling
(indistinct) information,
whether it's the ATT&CK Emulation Platform
for all the results and then paint your visualizations.
Now, in this example, this is a Jupyter Notebook
that we highly abstracted, but we've seen used
multiple times where codes written
from the visualization perspective,
you can paint in a very high level picture
to say, an executive.
What does that ATT&CK Flow look like?
What were the multiple steps?
Where were our controls supposed to detect or prevent?
Where did they fail?
That's a story and I'm talking about at the CIO,
Chief Risk Officer auditor level.
No details other than ATT&CK Flow
and maybe you're mentioning what the techniques are
along the way, but fewer details is better.
Just understanding what exactly happened
and then using some type of data frame
and mapping on Plotly and looking at a table,
looking at threat actors and dumping it
within the Jupyter Notebook, as you're telling the story,
and you've had a text block to explain
what you're about to show to again, the right audience,
which is fewer details, to show these threat actors
or these particular campaigns.
And we were neither able to detect or prevent,
showing red or green.
And we've seen multiple times more and more details
at this level get added and they're like,
"I have a lot of questions."
And at this level, you wanna paint a picture
and get a certain emotional response, like any good story.
So red and green, in this typical sense,
for one of our customers just meant
we were neither able to prevent or detect it.
And showing this data with this Jupyter Notebook
and going along from the start of showing it
at an executive level will provoke and trigger emotion
that will hopefully go to action.
- Yeah.
And the goal with this here at the end of the day,
we're talking at the executive level.
I know for all of you in the room,
you see two green blocks in a red block.
And that seems way too simple.
At the end of the day, as analysts,
we just need to showcase where do we need help?
Where do we need support?
And while we have all that data to back it up,
that's all that we need to show at that executive level.
Now, with that said, we have to address that problem, right?
When we ask for resources, what are we gonna do with them?
And the great thing about this is all that context is there.
We're just exposing in different levels
depending on who we're talking to.
So let's take a look at how we've put together
a different view for the same data set
so that we can support the detection engineers
at the organization
and so that they can start affecting things at the org too.
Go ahead, Stephan.
- Sure.
So again, kind of drilling down
either as part of the same Jupyter Notebook
or a different one and you can imagine you have a whole
git repository of all these Jupyter Notebooks.
When someone comes over and say,
"hi, I have a certain question."
And instead of pulling out that intel report
or security assessment report that you wrote a month ago
to try to answer that question,
you say, "oh, let me pull out a Jupyter Notebook."
And it's meant to answer a certain set of questions
to a particular set of audience of a certain type,
whether it's strategic, operational or tactical
at a certain level.
So in this particular case,
we work with a lot of detection engineering teams.
And the question that they're asking
isn't just where are our gaps
or control capabilities failing or succeeding
to prove that but where are our detection logic failing?
And as you might run a particular ATT&CK Graph
or ATT&CK Flow, there is more detail
that can be dumped at this level.
And again, within, embedded inside of a Jupyter Notebook.
So you're looking, presenting just like I am right now
to you, but you've got a Jupyter Notebook in front of you
and you can be very interactive.
Jupyter Notebooks allow you in real time,
not just to change the code, but to have various
input boxes to allow parameters and then rerun the code.
Rerun just that text block, not the entire thing.
Output a visualization and see, beginning, middle event,
where in the ATT&CK Flow, where are detection rules failing?
Where were they succeeding?
Doing that at a very granular level, knowing timing,
when was it run on what machine was it run?
What was the sequence and the timing sequence
between the various atomic techniques
that were run within the ATT&CK Flow.
And this can also be shown at a tactics level
to show weaknesses amongst kind of the tactics.
Again, it's not always sequential left to right,
it could be done in parallel.
- And the huge advantage I've seen here,
when I put together PowerPoint slides
then I get asked, "well, what was the data source?
How did you arrive at that conclusion?
What was the logic that led you to say
that we're okay in this area?"
And because that's exactly where we function,
exactly where we work, all that information is there, right?
This is where I pulled the data.
This is where I did the mappings.
And this is how I arrived at that result.
So it lets you engage right then and there
with the analysts whenever you are presenting
to those folks.
- So this is our last example.
And again, one of the points that we want you
to take away from this is next time you're gonna look at
maybe presenting some of your data through a PowerPoint,
write it up in a Jupyter Notebook.
This is a historical performance
viewpoint of looking at some of the data.
So obviously, emulations have been run.
You have more knowledge about the controls,
what failed, you have more knowledge about
some of the detection, logic, what worked,
what didn't work, where in the ATT&CK Graph did it work,
did it not work?
Below it over time.
So as Jose said, you might ask for resources, more time,
more projects, more technologies, but in January,
where were we failing?
Did we improve in February?
Did we improve in March?
Looking at it from both high level,
either campaign or a threat actor perspective,
or looking at it a much more granular level
from a detection engineering perspective of specific rules
that were meant to catch atomic techniques
within that entire ATT&CK Flow.
And again, it might be that you can't catch
that atomic technique on its own but you can
when they're glued together.
- Yeah, and I think at the end of the day,
I think the call to action to you is
you're already using Jupyter Notebooks
in some shape, way or form.
Some analysts that you follow in ATT&CK
are using them already.
Think about using it as a way to present
'cause you can take what you're already doing,
manipulate it into what we've shown you
and then when those people are asking
for that very detailed information
of how you arrive at that, it's right there.
And that's been a very valuable thing for us.
So thank you for your time so far.
- Thanks.
- Thank you.
Thank you, and I think Jupyter Notebooks,
but even more broadly, we're all big fans of automation.
So I don't think we have time for many questions
but I just wanted to shoot a simple one.
You mentioned, you know, the power of automation,
kind of walked us through a story of an example.
In terms of, one of the first questions we have
in this space, is how do you determine
a good candidate for automation in terms of,
is it a particular analyst problem?
Is it a story I'm trying to tell?
How do you really try to enumerate and really dive into
that type of decision?
- Yeah.
I mean, I think that that comes down to
when we're working with our customers or clients,
we're asking them what their priorities are.
And sometimes at a very strategic level,
as we saw in the presentations earlier,
they might be worried about something in the news.
Maybe that's a perceived threat and it's not real.
And then we'll drill down and we'll kind of talk
both at the strategic, operational and tactical level,
make sure everyone's aligned.
That kind of sets the priorities of what gets tested.
What's the focus?
And then what can be automated to to get tested
so that you can verify what's working is working.
- Exactly, I mean, there seems to be a couple themes
of the day but know your audience,
like I said, that's fantastic, thank you.
- Thanks (indistinct).
Thanks everyone.
- [Jose] Thank you very much.