>> Announcer: Live from San Jose, California,
it's The Cube,
covering Big Data Silicon Valley 2017.
>> Hey, welcome back everybody.
Jeff Frick here with The Cube.
We are at Big Data SV in San Jose
at the Historic Pagoda Lounge,
part of Big Data week
which is associated with Strata + Hadoop.
We've been coming here for eight years
and we're excited to be back.
The innovation and dynamicism of big data
and evolutions now with machine learning
and artificial intelligence, just continues to roll,
and we're really excited to be here
talking about one of the nasty aspects
of this world, unfortunately, malware.
So we're excited to have Darren Chinen.
He's the senior director of data science and engineering
from Malwarebytes.
Darren, welcome.
>> Darren: Thank you.
>> So for folks that aren't familiar with the company,
give us just a little bit of background
on Malwarebytes.
>> So Malwarebytes is basically
a next-generation anti-virus software.
We started off as humble roots with our founder
at 14 years old getting infected with a piece of malware,
and he reached out into the community and,
at 14 years old, wrote his first,
with the help of some people, wrote his first
lines of code to remediate a couple of pieces of malware.
It grew from there and I think by the ripe old age of 18,
founded the company.
And he's now I want to say 26 or 27
and we're doing quite well.
>> It was interesting, before we went live
you were talking about his philosophy
and how important that is to the company
and now has turned into really a strategic asset,
that no one should have to suffer from malware,
and he decided to really offer a solution for free
to help people rid themselves of this bad software.
>> Darren: That's right.
Yeah, so Malwarebytes was founded under the principle that
Marcin believes that everyone has the right
to a malware-free existence
and so we've always offered a free version
Malwarebytes that will help you to remediate
if your machine does get infected with a piece of malware.
And that's actually still going to this day.
>> And that's now given you the ability
to have a significant amount of inpoint data,
transactional data, trend data,
that now you can bake back into the solution.
>> Darren: That's right.
It's turned into a strategic advantage for the company,
it's not something I don't think that we could have planned
at 18 years old when he was doing this.
But we've instrumented it so that we can get
some anonymous-level telemetry
and we can understand how malware proliferates.
For many, many years we've been positioned
as a second-opinion scanner and so we're able to see
a lot of things, some trends happening in there
and we can actually now see that in real time.
>> So, starting out as a second-position scanner,
you're basically looking at,
you're finding what others have missed.
And how can you,
what do you have to do to become the first line of defense?
>> Well, with our new product Malwarebytes 3.0,
I think some of that landscape is changing.
We have a very complete and layered offering.
I'm not the product manager,
so I don't think, as the data science guy,
I don't know that I'm qualified
to give you the ins and outs,
but I think some of that is changing
as we have, we've combined a lot of products
and we have a much more complete sweep
of layered protection built into the product.
>> And so, maybe tell us,
without giving away all the secret sauce,
what sort of platform technologies did you use
that enabled you to scale to these hundreds of millions
of in points,
and then to be fast enough at identifying
things that were trending that are bad
that you had to prioritize?
>> Right, so traditionally, I think AV companies,
they have these honeypots, right,
where they go and the collect a piece of virus
or a piece of malware,
and they'll take the MD5 hash of that
and then they'll basically insert that
into a definition's database.
And that's a very exact way to do it.
The problem is is that there's so much malware
or viruses out there in the wild,
it's impossible to get all of them.
I think one of the things that we did
was we set up telemetry
and we have a phenomenal research team
where we're able to actually
have our team catch entire families of malware,
and that's really the secret sauce to Malwarebytes.
There's several other levels
but that's where we're helping out in the immediate term.
What we do is we have,
internally, we sort of jokingly call it
a Lambda Two architecture.
We had considered Lambda long ago,
long ago and I say about a year ago
when we first started this journey.
But there's, Lambda is riddled with,
as you know, a number of issues.
If you've ever talked to Jay Kreps from Confluent,
he has a lot of opinions on that, right?
And one of the key problems with that is,
that if you do a traditional Lambda,
you have to implement your code in two places,
it's very difficult, things get out of sync,
you have to have replay frameworks.
And these are some of the challenges with Lambda.
So we do processing
in a number of areas.
The first thing that we did was we implemented Kafka
to handle all of the streaming data.
We use Kafka streams to do
inline stateless transformations
and then we also use Kafka Connect.
And we write all of our data both into HBase,
we use that, we may swap that out later
for something like Redis,
and that would be a thin speed layer.
And then we also move the data into S3
and we use some ephemeral clusters
to do very large-scale batch processing,
and that really provides our data lab.
>> When you call that Lambda Two,
is that because you're still working essentially
on two different infrastructures,
so your code isn't quite the same?
You still have to check the results on either
on either fork.
>> That's right, yeah, we didn't feel like it was,
we did evaluate doing everything in the stream.
But there are certain operations
that are difficult to do with purely streamed processing,
and so we did need a little bit,
we did need to have a thin,
what we call real time indicators,
a speed layer, to supplement
what we were doing in the stream.
And so that's the differentiating factor
between a traditional Lambda architecture
where you'd want to have everything in the stream
and everything in batch,
and the batch is really more of a truing mechanism
as opposed to, our real time is really directional,
so in the traditional sense,
if you look at traditional business intelligence,
you'd have KPIs that would allow you to gauge
the health of your business.
We have RTIs, Real Time Indicators,
that allow us to gauge directionally,
what is important to look at this day, this hour,
this minute?
>> This thing is burning up the charts,
>> Exactly. >> Therefore it's priority one.
>> That's right, you got it.
>> Okay.
And maybe tell us a little more,
because everyone I'm sure is familiar with Kafka
but the streams product from them is a little newer
as is Kafka Connect, so it sounds like
you've got, it's not just the transport,
but you've got some basic analytics
and you've got the ability to do the ETL
because you've got Connect that comes from
sources and destinations, sources and syncs.
Tell us how you've used that.
>> Well, the streams product is,
it's quite different than something like Spark Streaming.
It's not working off micro-batching,
it's actually working off the stream.
And the second thing is, it's not a separate cluster.
It's just a library, effectively a .jar file, right?
And so because it works natively with Kafka,
it handles certain things there quite well.
It handles back pressure and when you expand the cluster,
it's pretty good with things like that.
We've found it to be a fairly stable technology.
It's just a library and we've worked very closely
with Confluent to develop that.
Whereas Kafka Connect is really something
that we use to write out to S3.
In fact, Confluent just released a new,
an S3 connector direct.
We were using Stream X, which was a wrapper
on top of an HDFS connector
and they rigged that up to write to S3 for us.
>> So tell us,
as you look out,
what sorts of technologies
do you see as enabling you to build a platform
that's richer, and then
how would that show up in the functionality
consumers like we would see?
>> Darren: With respect to the architecture?
>> Yeah.
>> Well one of the things that we had to do is
we had to evaluate where we wanted to spend our time.
We're a very small team,
the entire data science and engineering team
is less than I think 10 months old.
So all of us got hired, we've started this platform,
we've gone very, very fast.
And we had to decide, how are we going to, a,
get, we've made this big investment,
how are we going to get value to our end customer quickly,
so that they're not waiting around
and you get the traditional big-data story where,
we've spent all this money
and now we're not getting anything out of it.
And so we had to make some of those strategic decisions
and because of the fact that the data
was really truly big data in nature,
there's just a huge amount of work that has to be done
in these open-source technologies.
They're not baked,
it's not like going out to Oracle
and giving them a purchase order
and you install it and away you go.
There's a tremendous amount of work,
and so we've made some strategic decisions
on what we're going to do in open-source
and what we're going to do with
a third-party vendor solution.
And one of those solutions that we decided
was workload automation.
So I just did a talk on this
about how Control-M from BMC
was really the tool that we chose
to handle a lot of the coordination,
the sophisticated coordination,
and the workload automation on the batch side,
and we're about to implement that
in a data-quality monitoring framework.
And that's turned out to be
an incredibly stable solution for us.
It's allowed us to not spend time
with open-source solutions that do the same things
like Airflow, which may or may not work well,
but there's really no support around that,
and focus our efforts on what we believe to be
the really, really hard problems to tackle
in Kafka, Kafka Streams, Connect, et cetera.
>> Is it fair to say that Kafka
plus Kafka Connect solves many of the old ETL problems
or do you still need some sort of orchestration tool
on top of it to completely commoditize,
essentially moving and transforming data
from OLTP or operational system
to a decision support system?
>> I guess the answer to that is,
it depends on your use case.
I think there's a lot of things that Kafka
and the stream's job can solve for you,
but I don't think that we're at the point
where everything can be streaming.
I think that's a ways off.
There's legacy systems that really don't
natively stream to you anyway,
and there's just certain operations
that are just more efficient to do in batch.
And so that's why we've, I don't think batch for us
is going away any time soon
and that's one of the reasons why workload automation
in the batch layer initially was so important
and we've decided to extend that, actually,
into building out a data-quality monitoring framework
to put a collar around
how accurate our data is on the real-time side.
>> Cuz it's really horses for courses,
it's not one or the other,
it's application-specific,
what's the best solution for that particular is.
>> Yeah, I don't think that there's,
if there was a one-size-fits-all it'd be a company,
and there would be no need for architects,
so I think that you have to look at your use case,
your company, what kind of data,
what style of data, what type of analysis do you need.
Do you really actually need the data in real time
and if you do put in all the work to get it in real time,
are you going to be able to take action on it?
And I think Malwarebytes was a great candidate.
When it came in, I said,
"Well, it does look like we can justify
"the need for real time data, and the effort
"that goes into building out a real-time framework."
>> Jeff: Right, right.
And we always say, what is real time?
In time to do something about it,
(all chuckle)
and if there's not time to do something about it,
depending on how you define real time,
really what difference does it make
if you can't do anything about it that fast.
So as you look out in the future
with IoT, all these connected devices,
this is a hugely increased attack surface
as we just read our essay a few weeks back.
How does that work into your planning?
What do you guys think about the future
where there's so many more connected devices
out on the edge and various degrees of intelligence
and opportunities to hi-jack, if you will?
>> Yeah, I think,
I don't think I'm qualified to speak
about the Malwarebytes product roadmap as far as IoT goes.
>> But more philosophically,
from a professional point of view,
cuz every coin has two sides,
there's a lot of good stuff coming from IoT
and connected devices,
but as we keep hearing over and over,
just this massive attack surface expansion.
>> Well I think, for us, the key is we're small
and we're not operating,
like I came from Apple where we operated
on a budget of infinity,
so we're not--
>> Have to build the infinity or the address infinity
(Darren laughs)
with an actual budget.
>> We're small and we have to make sure
that whatever we do creates value.
And so what I'm seeing in the future is,
as we get more into the IoT space
and logs begin to proliferate
and data just exponentiates in size,
it's really how do we do the same thing
and how are we going to manage that in terms of cost?
Generally, big data is very low in information density.
It's not like transactional systems
where you get the data,
it's effectively an Excel spreadsheet
and you can go run some pivot tables and filters
and away you go.
I think big data in general requires a tremendous amount
of massaging to get to the point where
a data scientist or an analyst can actually extract
some insight and some value.
And the question is, how do you massage that data
in a way that's going to be cost-effective
as IoT expands and proliferates?
So that's the question that we're dealing with.
We're, at this point, all in with cloud technologies,
we're leveraging quite a few of Amazon services,
server-less technologies as well.
We just are in the process of moving to the Athena,
to Athena,
as just an on-demand query service.
And we use a lot of ephemeral clusters as well,
and that allows us to actually run all of our ETL
in about two hours.
And so these are some of the things that we're doing
to prepare for this explosion of data
and making sure that we're in a position
where we're not spending a dollar to gain a penny
if that makes sense.
>> That's his business.
Well, he makes fun of that business model.
>> I think you could do it,
you want to drive revenue to sell dollars for 90 cents.
>> That's the dot com model, I was there.
>> Exactly, and make it up in volume.
All right, Darren Chenin,
thanks for taking a few minutes out of your day
and giving us the story on Malwarebytes,
sounds pretty exciting and a great opportunity.
>> Thanks, I enjoyed it.
>> Absolutely, he's Darren, he's George, I'm Jeff,
you're watching The Cube.
We're at Big Data SV at the Historic Pagoda Lounge.
Thanks for watching, we'll be right back
after this short break.
(upbeat techno music)