BRUNO AZIZA: Hi, everyone.
This is Bruno Aziza, and today I am talking to Cody.
Now, Cody is going to talk to us about real-time and anomaly
detection, one of the design patterns
that we are delivering to you this week.
If you don't remember what design patterns are
and our conversation with Robert last week on product
recommendation, these design patterns are a starting point.
They're there to help you deploy a solution for a given business
problem.
So anomaly detection is a big one,
and Cody has a ton of experience in that.
I've known him for a long time.
He was at Domo for seven years, and now has been here
at Google for a year.
So, Cody, take us through the genesis.
Why does real-time matter to our customers?
CODY IRWIN: Real-time, I think, matters to every company,
especially in this digital transforming age.
Like, everything is changing.
I think 2020 has shown us very clearly that companies that
can adapt quickly and understand customers and understand
competitors and the market in general
are those that ultimately win.
And we found that real-time really
is the solution to help them find answers quickly.
BRUNO AZIZA: So we talked about real-time.
Now, how does this real-time problem
relate to anomaly detection?
CODY IRWIN: So real-time, in and of itself, is super exciting.
Like, everyone you've met should be like, yeah,
of course I want real-time.
I want answers fast.
But the reality is, if we just do real-time, if we just
take a real-time stream and point it at a company,
at a problem, it could make things worse
unless there's some more intelligence built into it.
So we found that the sweet spot tends to be combining real-time
data streams with advanced analytics--
so AI, ML, in this case, anomaly detection.
So helping organizations find, in the moment,
those outliers, the unique things that they should really
care about.
And those could be good or bad.
It could be like, this is a risk.
Worry about this.
Or it could be like, hey, something good is happening.
This is different.
Do more of this.
So we found that to be really a sweet spot with real-time.
That's why [? protection ?] is critical.
BRUNO AZIZA: We have to tell people
that these design patterns originate
from customers and partners that tell us about their need.
And we go out and use the best practices we have
and build this guidance, sample code, architecture guidance,
so you can use those best practices at your company.
So where did this story on anomaly detection
start on your end?
CODY IRWIN: So we had a large teleco provider approach us
and say, hey, I have this problem.
And originally, it was actually more of a processing problem.
They're like, we have a lot of data.
We have this massive network.
We want to know what's different in our traffic,
but we can't even process the data.
So we started there.
And the awesome thing is, with tools like Dataflow, BigQuery,
we were able to meet their scaling needs.
So the requirements were 250,000 events
per second that had to be processed.
They kind of kicked off this journey.
Like, we took that pattern, we saw the problem.
And we've been kind of like wrapping it up in a pattern
and saying, OK, does anyone else have this need
for anomaly detection?
We've been amazed at how many companies are like, yeah,
I have a problem.
I could use some help.
It is real-time in nature.
It really has turned into kind of a rinse and repeat.
BRUNO AZIZA: That's excellent.
And that's the whole point for why we're
building these design patterns.
So we want to invite you to take advantage of them
right now by clicking on the link down below, and of course
suggesting more design patterns that you
might need for your rendition.
But before we go, I want to talk about partners,
and how they added to the ecosystem here.
One in particular, I think, Confluent,
that you've been working with.
Tell us more about that.
CODY IRWIN: At Google Cloud we love, love, love partners.
And in this case, Confluent has been a pretty key partner
in helping us bridge the divide between on-premise or other
clouds to GCP.
It's like, a lot of times people are like, yeah,
I have this problem.
I'd love for Google to help me solve it.
But the process for doing the integration sometimes
isn't super-clear.
A lot of companies are invested in Kafka.
Confluent is basically a managed version of Kafka,
and so they made it very, very easy for us
to create the integration and capitalize
on this functionality.
I will say as well, when it comes to the actual anomaly
detection, we found that a number of other companies
have a lot of great proprietary models
that get plugged in to this.
So our architecture is actually somewhat transferable.
The pipeline is pretty established.
Like, we know how to do that really, really well.
We know how to integrate models.
I'm kind of more generic models, but we
have partners that really can augment that and bring
specialty into the actual need.
So for some organizations, the patterns
are like the exact solution.
For others, the architecture is the solution.
And a model fits really well into an architecture.
BRUNO AZIZA: And so there's value for customers,
there's value for prospects listening to us,
there's value for partners.
There's value for everyone.
Again, if you are interested in this design pattern,
make sure you click on the link down below.
If you are interested in others that we might not
have built yet, don't be shy, and suggest which ones
you'd like us to build for you.
Cody, thank you so much for the time today.
I hope people are going to reach out to you.
We'll talk to you soon.
CODY IRWIN: Thanks, Bruno.