- Next up, we have a team from Temple University.
We have Aunshul Rege
Who's an associate professor there along
with PhD student, Rachel Bleiman.
- Great, hi everyone.
My name is Aunshul Rege
I'm an associate professor
with the Department of Criminal Justice at Temple.
And I'm here today
with my amazing graduate research assistant,
Rachel Bleiman.
And today we wanna share with you
how we are using PRE-ATT&CK
and ATTA&CK in the social sciences
for cyber security and cyber crime education and research.
We wanna thank Mitre for the opportunity to let us share
what we're doing with you all today.
And of course we want
thank our funders National Science Foundation
because without them we couldn't do what we're doing.
So on the agenda we're gonna talk about
how we're using PRE-ATT&CK
for cyber crime and security education.
Rachel is then gonna share with you
how we've been using ATT&CK
for some of our research datasets.
And then we're gonna end with a summary
and talk about where we'd like to go with, you know,
what we're doing in education and research.
So let's get started with that first component.
How are we using MITRE PRE-ATT&CK framework
in security education?
I teach a cybercrime course in the social sciences.
The emphasis here is on the human aspects
of cyber attacks and security
with an emphasis within that on social engineering.
The class is composed of students from multiple disciplines.
So we have not just social science students
but we also have computer science and engineering students.
We've got grouped them up in about eight different groups
that have both social and computer science students in them.
The objectives of one of the assignments that
they have is we're trying to map the PRE-ATT&CK framework
to social engineering case studies
to help students learn how these frameworks can be used
to conduct threat intelligence
but also to understand the limitations
of both the case studies and the framework.
So we've got about six social engineering case studies
with a lot of details
and students will be required to look at the overall mapping
to the PRE-ATT&CK framework,
the specific action on certain tactics
and techniques that they find
and then identify mitigation strategies identified
in the framework and also extend beyond that.
This was our first attempt
this semester to try something like this,
but we're hoping to learn and grow in this.
So students, like I said,
we're gonna be given six different case studies
within each study they would then map it all
to the overall framework
as you can see by the highlighted boxes here.
So if they find certain components, they would map it on.
But then the idea here is we want them
to think about things like
what is the proportion of mapping?
How much of the case study were you able
to successfully map onto the framework?
Because in this gets them to think about the implications.
What does this mean for the case study?
Right? Do we see certain patterns?
Can we identify if certain tactics
and techniques are useful?
And of course then what are the implications
for PRE-ATT&CK in the sense
that how well does it apply to social engineering?
Now we're aware that PRE-ATT&CK is gonna be restructured
and possibly merged with ATT&CK
but this is what we're doing.
In the fall semester,
we'll be adapting this project moving in the future.
So once they've done this overall mapping
they then have to zoom into a specific tactic
and try to identify things along the lines of techniques,
similar techniques that fall under that tactic to again,
identify relationships, see if they can come up
with any detection measures as identified in the framework.
And of course dive into the difficulty level
for the adversary.
But in addition to this,
we want them to think about mitigation
for social engineering.
So currently, if you look at, you know,
MITRE's ATT&CK website under mitigations
they have mitigations listed for enterprise and mobile.
And we are trying to get our students to think about well,
what would something similar look like
for PRE-ATT&CK for social engineering?
What would be those mitigation strategies?
And if none exists, students are then encouraged
to recommend mitigations.
And these might take the form of for instance, policy
or training and awareness, right?
Moving beyond that technical domain.
So the overall then objective here is
that students learn about threat intelligence,
about mapping, about identifying patterns and trends
and also about mitigation.
So these are efforts in the education space.
I'm now gonna pass it on to Rachel to talk about
what we're doing or the datasets.
So Rachel.
- Thanks Aunshul.
So through our cybersecurity in action research
and education lab website, which is sites.temple.edu/care
we offer free downloads of the course projects along
with two datasets that we've created
and we have recently mapped onto the MITRE ATT&CK Framework.
Both of these data sets are updated on a monthly basis.
The first data set we have, is a repository
of social engineering incidents.
We currently have 623 incidents recorded spanning
from 2011 up through August, 2020.
The second dataset consists
of ransomware attacks against critical infrastructure.
And this dataset has 747 incidents,
ranging from November of 2013 up through September of 2020.
All of the incidents that we gather
for both of these datasets is completely based
on publicly disclosed information
that we collect from Google alerts.
And a couple of months ago,
we received some feedback from people
who use the ransomware dataset
and they asked if we could map it onto the attack framework.
So we did that
and we decided to also map the social engineering dataset
onto the framework as well.
So on the table on the left here are some of the variables
that we record in the social engineering dataset,
including dates, who the target is and where they're located
what social engineering tactic is used.
The monetary cost of the attack, who the attacker is
and some information about the ploy of the attack.
About 50%
of the social engineering tactics in our dataset mapped
onto the attack techniques or software.
The primary techniques we were able to match we're phishing
and spear phishing, but there were also some instances
of malware that mapped onto the software and the framework.
And then we also mapped about 23%
of the attackers onto the attack group slash attacker.
So group 32 or Lazarus group appeared several times
and was the most common
while groups 59, 92 and 94 were also repeatedly mapped.
Moving onto the critical infrastructure ransomware dataset,
our variables record information about the date
and location of the attack, who is targeted,
what strain of ransomware is used
and details about the ransom.
And in our transition from the ninth iteration
to the 10th iteration of the dataset,
we began mapping onto the ATT&CK framework.
We first removed all incidents where the strain was recorded
as NotPetya as the ATT&CK Framework defines NotPetya
as a wiperware and not a ransomware.
We then mapped the ransomware strain variable
onto the attack software ID about 56%
of the strains mapped onto the tax software.
And these eight softwares are the ones
that appeared in our dataset repeatedly
and included Maze LockerGoga, NetWalker,
Ragnar Locker, Robinhood, Ryuk, Sam Sam and WannaCry.
And we did experience some challenges
and found some limitations
with mapping our dataset onto the attack framework.
In the social engineering dataset,
many of the social engineering tactics
do not currently exist in the framework such as whaling
and vishing.
And while our mapping results did show that about half
of the tactics mapped over the bulk
of our incidents in this dataset are phishing
and spear phishing attacks
which the framework does include
which raised that percentage for us.
We had very few techniques besides vishing
that maps onto the framework.
And in the ransomware dataset,
we were unable to map a portion of the strains.
Some strains that appear frequently the datasets such
as revil and has been around
for some time does not does not yet exist in the framework
and other strains that are newer,
such as ransom X are also not yet in the framework.
- Great. Thanks Rachel.
So I'm just gonna bring everything to a close here
and share with you some of, some of the reasons
why we're using this right in the education space.
I as an educator certainly found PRE-ATT&CK
to be extremely useful
because it helps her students develop the ability to map
and understand threat intelligence
and also the ability to understand challenges
and limitations.
I think what's really cool is we're trying
to map social engineering cases
which is not typically done.
So I think that's an interesting exercise in and of itself,
from the social science perspective.
What else will like about this?
Is it isn't technical enough?
So all disciplines can engage.
I have a lot of social science students
who can actually participate in this
and get a top level understanding of threat intelligence.
So I think that this is a really, really cool exercise
for my students to try out from a dataset perspective.
Interestingly, when Rachel and I started this,
we were struggling to get data because
for obvious reasons, companies don't wanna share it
or government doesn't wanna share it.
So we said, you know what, let's create our own dataset
and we'll share it with other folks in the academic domain.
And so we've had a lot of educators
and students of course request this data which was great
because it it's allowed us to fulfill sort of our objective
of sharing this with the wider academic community.
But interestingly we got a lot of requests from governments,
not just in the US but around the world
and also industry from all over the world
that was interested in, in getting a copy of our dataset,
said, hey, let's, you know, see if we can look at trends
and patterns across ransomware strains,
comparing the data to their own internal datasets
or raising awareness and for training.
And what was really cool is mapping it
onto MITRE really really improve the quality of our dataset.
It gave our data set a lot of credibility, and we got a lot
of responses saying, thank you for mapping this.
This is gonna help us out big time.
So that is,
ah you know,
the really cool thing is it was after we mapped it
onto the MITRE Framework we actually got,
the dataset was actually picked up by a Security Week
and also mentioned,
our lab was mentioned in Bleeping Computer
which was again, another big win for us.
Cause we're a team of trying to do something like this.
And it's lovely to see that our efforts are having an impact
and benefiting the wider community.
So one of the things that we wanna do is,
like I said we're aware that PRE-ATT&XK and ATT&CK are going
to be restructured merging.
And I know that PRE-ATTACK and ATT&CK
has helped me in the social science space,
right helped me with my education and research efforts.
But I would love to do, is to see
how we in the social science domain, through our education
and research efforts can contribute back
to PRE-ATT&CK and ATT&CK
And this is what our previous speaker mentioned
as well as how can we be a contributing?
We're trying to create data repositories.
I remember, you know, where do we go to get data?
Right. So we're looking at indictments,
we're looking at social engineering case studies.
We can conduct focus groups and interviews
with social engineers.
And we're also trying to weave it into
what we've best started at Temple is a
Collegiate Social Engineering Capture the Flag competition
and training event.
So can we use MITRE ATT&CK Frameworks to sort of shape
that exercise and in turn use student experiences
to then contribute back to this framework.
So if anyone's interested in any of this,
we are definitely seeking collaboration.
We'd love to hear not just from academic communities
but also from industry and government and of course MITRE.
So that's everything that I had.
We can I guess, take it up for questions
- Fantastic talk.
And like people are going nuts over this data set.
And like I said, I love that you touch at the end at like
how it was truly a grassroots effort to like,
you know, compile that and build it yourself.
I think the biggest question is like you said,
I love the idea of, like I said no matter
how technical things are it's all,
there's a human element to all this work.
My biggest question is you know,
have you seen themes in terms of the mitigations
that your students are putting together
for these attacks that you can share
with the wider community,
the things that we can do to kind of
you know, prevent, or at least mitigate
to some degree, these layer eight human-based behaviors?
- Yeah.
Yeah and that's a great question.
And you know, I'm gonna respond, respond in two parts here.
I guess, one is it's primarily been policy, you know,
appropriate behavior, what to click on what not to click on.
So a bit more on the education and training aspects of this.
But I love that because it's a multidisciplinary class,
you're getting computer scientists
who really think about this.
This is the next generation of workforce
of computer scientists who are gonna be developers
and defenders.
And so thinking about using these frameworks,
not just in the technical aspect,
but also on the human domain, right,
that for, I think it's really getting them to think about
that as they go out and design and defense systems.
So that's, I know it's impossible
to really give your and your question a,
an effective response in the short time we have
but I'm happy to carry on that conversation in this life.
- That makes a lot of sense.
Like you said, you can only build a wall so strong
if someone's just gonna open the door, what's the point
but yeah, with that, thank you.
Thanks again for a great presentation.
That was really excellent.
Except there's a lot of slack activity going on.
So definitely hang around and ask some questions,
but-- - Great. Thank you all.