>> DANIEL BURROUGHS: And what I'm talking about today, this is basically, a follow‑up
to a talk I gave a few years ago at DEF CON 18, about looking at information that's freely
available out there on the net, and doing some trending and analysis of it, and trying
to make something useful out of it.
So a little bit about my background, I'm currently ‑‑ I'm the director of technology at the Center
for Law Enforcement Technology training and research, which is a nonprofit research center.
They got spun out of work that I used to do when I was a professor at the University of
Central Florida.
I was there for about ten years and in the engineering program, taught computer engineering,
I developed the computer security curriculum there, and did embedded systems amongst some
other things, and eventually moved away from teaching and more into research.
And we ended up spinning out the research into an independent nonprofit center.
I'm also the CTO for Hoverfly Technologies and prior to this, I used to work as a research
associate up at the institute for security technology studies the Dartmouth College.
So over the course of the last 20 years, some of the things that I have worked on appear
on this list, and, you know, it took me quite a while to catch on to kind of what, like,
the common theme between all of the things I was working on.
I'm slow to pick up on these things at times, and eventually, as I started putting it together
and kind.
Realizing some of the same things I was doing, I realized all of this stuff from information
sharing that I'm working on now, to hardware sensor networks, to intrusion detections,
they rely on sensor fusion.
Everything that we are doing, all of those things that are ‑‑ that I listed up there,
they are all based on taking some sort of sensor, and using it to try to get some measure
of reality, but the sensor always has some limitations.
Sometimes it's a significant one.
Sometimes it's not so bad, but every sensor that we look at reality, including ourselves,
including when we view things, it's always got some sort of limitation, and it's one
particular view, and that influences the data we are seeing, and you can get ‑‑ we
have to work towards trying to get more meaningfulness out of the data that we have.
One of the ways that we do this, and one of the things ‑‑ the techniques that I find
most versatile, I would say, is sensor fusion, where we take multiple sensors, multiple ways
of looking at the same thing, and kind of put that together, with the hope that we can
take the limitations of one observation and cancel it out with a different observation
that has a different set of limitations.
So at least that's the hope.
At least, you know, if we can put two halfway decent things to go and get something that's
more than the sum of its parts.
So before I get kind of more into my stuff, I always feel like with the ‑‑ in this
particular subject, that I have to give an acknowledgment to the guy that inspired kind
of some of these thoughts in my head and it was actually at DEF CON, way back at DEF CON
13, Broward Horne gave this talk on Meme mining for fun and profit.
It's problem ‑‑ you know, all great ideas come out of a problem.
I guess a lot of ideas come out of trying to solve a problem too, but his was a really
good idea.
His problem was that he would find that he would, like, start learning some new technology,
some new tool or at least it was new to him and by the time he felt he had mastered it,
it was kind of on the way out or the market, the job market was just saturated with people
doing that now or it had just fallen by the wayside and nobody cared about it.
He was always kind of struggling, what should I spend my time learning to get ahead?
And he ended up kind of thinking about this as like, everything has this sort of saturation
curve where a trend starts happening and there's a little bit of chatter about it and eventually
it starts taking off, and everybody hears about it when it's big and growing and then
it gets boring and old.
You want to try those things earlier on and went through and did it.
This is a slide pulled out of his old presentation.
What he would do is he would look at new sources and forums and blogs, for information and
key words and kind of pull them out and see what was trending on there, with the idea
that that's kind of a precursor to seeing that early chatter about it, so something
can take off.
This one in this particular case, this is ‑‑ the red line shows how many times the word
"palladium" showed up in news reports and forums.
And the blue is the "price of palladium" and you can see clearly there's a lot of chatter
about it before the price spiked up and then the chatter dropped off before the price came
back down.
It's a really good indicator about predicting the future there, what's going on.
Anyway, that thought inspired me and when I was ‑‑ when I was teaching, I would
have students who could come to me and they would want to know, what skills do they need
to get a good job and all of that and I tried to apply what Broward had done in a similar
way by monitoring and observing trends and this is single variable observation.
It's doing some correlation and it started off looking at Craigslist data, just because
Craigslist is nicely available.
It's well organized by geographic location and you can go in in certain categories, like
where they have the job postings in there, it's categories by different types of jobs.
Craigslist is not the best place to look for jobs but it had some interesting properties
in that there are a lot of small companies that post on there or maybe trying new things,
a lot of entrepreneurial companies, start‑ups things like that are starting there, not so
much the big ones.
That tends to skew it a little bit more towards being a leading indicator, something that
is ‑‑ it will come out a bit ahead of the curve.
So some of the things that I ended up looking at, just because I found correlations in here
were jobs, items for sale and adult services.
And I didn't ‑‑ I'm not saying I looked for adult services on Craigslist.
It's just my research took me there.
(Laughter).
So, you know, the things I saw, looked like this.
This is an example.
This is just showing job postings by date.
And there is a ‑‑ this is showing ‑‑ the dips you see there, these are weekly trends,
it goes dead on the weekends.
There's a spike on a Monday and a spike on a Friday.
Okay, it's kind of boring.
It's sort of interesting, but not unexpected.
There were certain things that stood out.
In this particular case, one of the things that jumped out at me was Austin never had
a spike on a Friday.
It always dropped off.
It's kind of hard to see, but it's the orange line in there.
It never has a second spike in it.
I thought that was interesting.
The other thing, and this is what came out of the adult services was there was a correlation
between adult services being offered or bicycles for sale or a lot of items for sale.
This led to a couple of interesting discussions.
One of my favorite moments at DEF CON, when someone said, hey, I think I can help you
out, I'm from Austin and my sister is a prostitute.
(Laughter).
So that and then it led into a discussion of
things could you sell one time, like a bicycle or something that you can sell over and over
and over again.
So, okay, that's what I had done before, and we had looked at that.
There's some interesting stuff there, but I wanted to dig a bit deeper into the data
and look for more relationships and more correlations between data and hopefully be able to pull
in other sources and do some fusions on this.
I started looking for things like different cycles in like the job postings or correlations
in them.
Because at the time when I was working on them, keep in mind, I was really trying to
help out some of the students and figure out what skills they needed and what would really
help them get ahead.
There were definitely correlations in there.
You know, there are things that you would see, but nothing unexpected.
Nothing really interesting that jumped out in related skills.
Could you say if a job was going to have one particular tool set or skill set listed, there
are other ones likely to be listed with it as well.
Again, nothing really jumped out at me as being unexpected out of it, but eventually,
there were a couple of interesting things that showed up.
One was kind of funny and it was how often the words drug test or drug screen showed
up in a job advertisement correlated with the different skills in it.
And apparently, like ‑‑ (Laughter)
If you don't think you are going to pass a drug test, don't bother learning SAP, because
it won't do you any good.
If you want to develop IOS applications, you know, go knock yourself out.
(Laughter).
You know, I guess there's probably some logic here, like how corporate or uncorporate the
environment is, I suppose.
Another thing was looking at jobs that had benefits, and like retirement and health and
medical.
You know, the interesting one, the best one was COBOL but I think it was a bit of an outlier,
there were so few jobs with COBOL.
I guess to get any grizzled COBOL programmer to come work for you, you had to give them
a lot of benefits.
Python and Android, and HTML, you won't give them much in the way of benefits, I suppose.
As I was looking into this ‑‑ actually, this is much more recently.
This is earlier this year.
I came across this article.
This is actually out of the journal of "Psychology" where a psychologist, Dorothy Brambel.
She looked at the missed connections of Craigslist.
This is where people say, oh, I saw you as I was walking across the parking lot and tried
to catch your eye and they post it and hope they will make a connection.
This is organized by state.
This is where people had the most missed connections.
Walmart has a log on the South.
(Laughter).
You know, Oklahoma, it's the state fair.
Of course!
You know, it makes perfect sense.
And, you know, in Nevada, it's casinos.
And the one thing I just had to put this up there, one thing that just jumped out at me
like crazy was Indiana, it's at home.
(Laughter).
I don't know what they're doing in Indiana, but I'm pretty sure they are doing it wrong.
(Laughter).
So I was talking with a friend of mine about this stuff, Dave Kerbletski and he told me
about something that he had done in his neighborhood in Orlando, Florida.
They had a rash of crime, recently and they didn't know they had a rash of crime, until
the neighbors got talking to even other.
Everybody knew a little different incident that had happened.
He went and did some searching and found out that there was some open source data that
the sheriff's department would post about their CAD, their dispatch calls and he wrote
this little tool to do some geolocating on it and tweet it out and then you can subscribe
to it, and get tweets from this thing, like, really hyper local things for your neighborhood,
about what's going on there.
And it's actually one thing that's fun.
I pulled this up earlier today, and, like, you know, I was just noticing things.
This is in Orlando area.
You know, the first tweet that's on there, and I'm amazed at the ‑‑ you know, the
sheriff's office is putting this out, they are basically saying there's a designated
patrol area available, which means there's an area that nobody is patrolling it currently.
And this is down, like, in a real tourist trap part of Orlando.
That could be useful information to somebody, to know there are no cops there right now.
And then there's a few accidents and then I guess the people at the bottom, down on
Poppy Avenue would be happy to note that there's a fugitive from justice running around in
their area.
This kind of led us to look into more sources for data.
What they offered where we were, wasn't very ‑‑ wasn't very useful or organized.
We found out and started looking in places that kind of subscribe to the open gov system
and this is a movement to have more transparent government data.
Some cities publish huge amounts of data about what's going on in their city, the fire department,
the police department, live interesting data, and Seattle, Boston, Chicago, a number of
others.
These are three that I spent a bit of time looking at.
There's information about incidents that are going on, like, police, fire.
In Chicago, you can actually track where the snowplows are in the city.
You can track where garbage trucks are in realtime, from the city, which I just find
really kind of fascinating.
There's information about where bicycle racks, public toilets, land marks and even where
cameras are, where the city has all of its cameras posted, which that one I thought was
actually particularly interesting but you can really go on here and make a map of what
is an observable location throughout the city, and what is not an observable location.
Which, again, that could be useful information for somebody.
Here's something, the Seattle one is great.
They have their visualization tools built right into this thing and this is showing
a map showing police incidents over a period of time, around in part of Seattle and I pulled
up this area.
You notice, like most of it ‑‑ everything is kind of in that same yellow orange, except
for one big glowing red blob out there and, you know, over in Georgetown.
I don't know if anybody is from Seattle here.
But I'm like wondering what the heck is going on over in Georgetown.
And you look in a little bit closer and right next to it is the Boeing Propulsion Engineering
Labs, which makes me feel really good.
So coming back, to an area I know a bit more about, back in Orlando, we pulled up data
that had ‑‑ we pulled out traffic ticket, they don't publish data about who got ticket,
but you can see when a traffic stop occurred.
And I ‑‑ we looked at it and pulled data, that covered three roads in the area and this
is right out by the University of Central Florida.
These are three roads that run all east‑west and they are the three major roads just kind
of ‑‑ one is right into the university, and one is a bit north and one is a bit south
and they all have about the same amount of traffic on and they all have a similar traffic
pattern.
And when we went through and what this chart is showing here is this is each one of the
groupings is a ‑‑ is a week long period, five week days.
And then it's repeated over six weeks.
And one of the things I found really interesting was the chance of a traffic ticket occurring
on one of these roads, the order ‑‑ it was always likely at different times of the
day.
It always followed the same sort of pattern, particularly between this highway 50 and University
Boulevard.
The Highway 50 traffic stops all preceded the University Boulevard traffic stops and
when you go out there and you look at the traffic, the traffic pattern is not really
any different.
So if you start thinking about this and start putting together, well, why ‑‑ you know,
why do you always see one before the other, I don't have ‑‑ you know I don't have
hard evidence to back this up, but our belief is you are seeing an influence of the patrol
pattern of the police in the city.
So you are actually able to kind of get in there and through their information that they
are putting out sort of start tracking them.
It's kind of like, you know, there's a talk we went to earlier yesterday, I guess it was,
there's a great talk with Brendan O'Connor that was talking about tracking people by
seeing, like, information their devices are spitting out on wireless networks.
It's a similar concept, that they are putting out a lot of information here, that is ‑‑
that if you look at it the right way and you take the right pieces of data, and put it
to go, you can pull a lot more information out about what their ‑‑ about what they're
doing and what's going on.
So, you know, why ‑‑ so by this time, I have kind of changed kind of what I was
interested in doing and probably because I quit teaching and I left the university so
I don't have students anymore.
I'm not that interested in helping people find jobs.
So now I found it kind of interesting to look at these government entities and the police
and other things that were going on and also because I have worked with law enforcement
a lot.
It's kind of interesting to see how on one hand, they are very protective of their data,
but at the same time, they are putting out a lot of information that I'm not sure that
they quite realize how much that they are putting out there.
Frankly, I think it's actually kind of a good thing.
I like being able to have more information and being able to look back on them and like
I say, why should the NSA have all the fun on spying on people?
So the ‑‑ what's next with this, and there's ‑‑ there so much more I would
like to talk about, but these 20‑minute talks you have to be kind of fast in.
What I'm really interested in is actually expanding the model that we have been using
on this data to be analyzed.
We kind of built things that are very purpose driven, that the first net of analysis, we
did was very structured around the seeking out the jobs, doing that, and then kind of
got side tracked by the crime and going off that direction.
And I want to bring this back together and try to build a more robust model for analyzing
this data and throw some data mining at this, where so far a lot of what we have done has
been what I would say is like hypothesis based where I make a prediction about something
I should see in here and then a correlation and see whether it exists in the data or doesn't
exist.
And I'm sure there's a lot of relations in there that are things that I wouldn't expect
or I wouldn't find otherwise, I want to throw a bit of sort of data mining and kind of that
sort of blind either ‑‑ either AI or brute force approach to finding relations
throughout the data.
So I think I'm about out of time right now and I'm getting a nod from the back and so
I will wrap it up there and if there are any questions, I would happy to take a couple
until they cut me off.
(Applause).
Thank you.
Okay.
Thank you.