[MUSIC PLAYING]
BRUNO AZIZA: Welcome to another episode of "Data Journeys."
This is where we come to learn from data leaders that
have experienced amazing growth, amazing transformation,
and have lessons they're going to share with us.
Today, I have the pleasure to talk to Rene.
He's the tech lead at Bol.com.
And so, Rene, thank you so much for taking the time
to talk to us today.
Let's start from the top.
What is Bol.com?
RENE KROON: So Bol.com is the largest online retailer
in the Netherlands and Belgium.
We are out there for 20 years already.
And so far, we've managed to hold off
competitors like Amazon.
So very happy with that.
We're not just selling our own products.
We have over 45,000 partners on our platform.
And together, we can sell up to 1 million items a day,
for example, on Black Friday.
So that's who we are.
BRUNO AZIZA: And you sell about 30 million items a year.
So you've got a lot of volume, a lot of data.
So let's talk a little bit about the use cases
that you're solving today with data analytics from Google.
RENE KROON: BigQuery is everywhere in our company.
So if there's data, then it will end up in BigQuery.
So what happened is, last year, we
started with the COVID situation.
And it meant that our sales went through the roof.
And normally, we carefully prepare for the holiday season.
There's like a six-month preparation cycle.
And now everything exploded.
So, of course, this was a time where
it was extra important to have our analytics ready,
so that we can keep steering all our processes
and not disappoint our customers and, at the same time,
keeping our delivery promises.
BRUNO AZIZA: And your data was already growing fast.
But it's grown 60% year over year,
over 100 IT teams providing data into your BI environment.
So what are the types of use cases
you're solving with the platform?
What issues are you solving for?
RENE KROON: So one of my goals is to make BigQuery
a low-entry platform.
So lots of people solve the problems that they think
are important.
And I'm not even aware.
So we provide a low-entry platform,
where people can solve their day-to-day issues.
So savvy business analysts can check their own product
and go into the data set to check one anomaly.
But we also use it all the way at the top.
So the business intelligence curated
daily sales report, for example, comes to BigQuery.
And even parts of our general ledger
are now running in BigQuery.
So all data that is in services will
be going to BigQuery, if there's an extra analysis needed.
BRUNO AZIZA: So the data ends up anywhere
in marketing and finance and sales,
all the way to the executive boardroom.
Let's talk a little bit about the journey of the data.
Where does it start?
What does the stack look like today?
RENE KROON: So I think most of our data
is generated internally.
So there's one large stream that is, for example,
the clicks on the website.
That's what we call the clickstream.
That's one very large stream that has special treatment.
And then there are microservices.
So there are well over 1,000 services in the company.
And I think 200 of them provide a standardized BigQuery data
set.
And so with these data sets, we can combine everything,
so logistics information or preferences or items
bought together.
Like recommendations is something
that comes out of this combination of data sets.
BRUNO AZIZA: Recommendations like product recommendations?
RENE KROON: Yeah.
So we have a recommendations pipeline.
And if you click on the website, it
is able to update sometimes even before you hit the next page.
So in between page visits, you will
get updated recommendations.
And these are based on like historic data in BigQuery
like what people bought together, for example,
and then augmented in real time with what you are looking at.
That's also a valuable, I would say,
one of the most valuable flows.
BRUNO AZIZA: You have a fairly broad use
of the Google platform.
You have Pub/Sub, Cloud SQL.
You use Spanner.
You use Dataflow for real time.
You use Data PROC and BigQuery.
So how do these kind of blocks come together?
What do you start with?
And obviously, you explained that you end in BigQuery.
I'm just curious about the journey leading to that.
RENE KROON: Well, so we make, usually,
a distinction between IT teams that
are usually producing the data.
And then the data is in BigQuery and then
taken up by our business intelligence teams
and distributed to what we call the end users or business
users, who can actually correlate
the data to the business needs.
So what will happen is that a microservice
will decide that they want to publish a BigQuery data set.
They will use our middleware called Stanley.
And that allows them to provide a standardized data set.
So the data that is in this Stanley format
is recognizable by analysts.
And that will then end up on BigQuery.
And from there, depending on the sensitivity,
it can be freely accessible, or people need to ask for access.
And most of the postprocessing will
take place with Airflow or Composer
and scheduled queries from there.
BRUNO AZIZA: What are the types of tools that
are interfacing BigQuery today?
Are you using Tableau?
Are you using Data Studio?
And then how do you break it up across the departments?
Is it kind of free service for everyone?
Or have you figured out a way to make sure that people
follow a particular path?
RENE KROON: So what I just described with Stanley
is like the producing part, getting the data there.
And then visualizing and working with it is the second part.
I must say that the BigQuery editor has become really
good last year.
So sometimes people don't even need
to leave the Cloud Console.
But if they do want to provide a report,
then company-curated reports are delivered in Tableau.
And at the lower level, we're trying
to see if we can incorporate Data Studio,
so that people can create reports
with more ease at the department level, for example.
BRUNO AZIZA: That's great.
Tell me more a little bit about Stanley.
It's a code name for the middleware product
that you have to deliver the data?
So it's the data platform.
What's part of Stanley?
And how did you build it?
RENE KROON: Stanley had a predecessor.
And the predecessor was called Eddy.
And its goal was to deliver data to Hadoop.
And it was like a monolithic server, where
you can send an HTML message.
And it would end up in a table in a Hadoop cluster.
So for BigQuery, there was a project called Stanley.
And it consists of client SDK that
is incorporated in the server that
sends data and a small server that
can process the data on the other side
and put it in BigQuery.
And it has some additional functionality.
So to comply with the GDPR, for example, it can forget data.
A forget statement is sent to an entity.
BRUNO AZIZA: Did you retire Eddy then?
Or what happened to Eddy?
RENE KROON: I was actually the one shutting Eddy down
at some point.
So I guess, also, our cloud migration
was triggered at some point by our Hadoop cluster slowly
filling up.
And it was put there once by definitely skilled consultants.
But to replace it in-house, just starting
with the capacity calculation, is like a nightmare.
How many servers do we need to run?
And so we left that behind and went to the cloud.
BRUNO AZIZA: So we talked about data infrastructure
and how you get the data in the stage
that it can be used with high quality.
What are some of the use cases that you're
particularly proud of?
RENE KROON: Most proud of the general ledger.
Mostly because I was part of the recent audit, where
it passed without problems.
So what it does is that, every sale that
is made on our web shop--
that can be either an item sold by ourself
or one of our partners--
goes to a number of transactions.
Of course, there's the amount paid by the customer.
And depending on whether it's a partnership or not,
we have to pay it out, depending on whether
or not the usual logistics service.
We have to make additional payments.
And we combine this together.
So you have one sale on the web shop.
You have some logistic costs associated, some warehousing
costs, some partner commission.
And to put all these amounts into perspective
in a stable system, we use BigQuery.
And it does the job very well.
BRUNO AZIZA: So, Rene, we learned a lot
about your migration and your journey.
Lots of people are wondering, well,
what did Rene do that I must absolutely go do today?
So what are your three do's here,
things that people need to absolutely do tomorrow
to be as successful as you are?
RENE KROON: They should definitely
provide business users with their own projects.
So what we see is that we can provide a low-entry barrier.
But that's, of course, also the problem of who pays
and what's the risk of a bad query.
And so we solved that by giving business teams
their own projects.
They can use it as scratch space.
They can share data.
They can share queries.
It works very well for them.
And it's a scalable way for us to provide this.
We just give a team a workspace.
And the people in that team can go wild.
If they go too wild, we'll notify them
and see if we can help them.
BRUNO AZIZA: How do you police for that?
So I know you probably have a flat-rate system
today with BigQuery.
But how do you make sure that queries don't get out of hand,
cost doesn't get out of hand?
I mean, I know you're giving them the accountability
and the access.
But how do you govern all of that?
RENE KROON: Flat-rate plays a large part in it.
So flat-rate guarantees that we don't run
into surprises with our bills.
Inside the flat-rate, we set up a hierarchy.
So we have one set that's called business critical
that is related to our warehouse operations.
That one is always executed first.
And then two or three tiers lower
are what we call the business projects.
And so basically, they can use all idle capacity
at any moment.
And that's a safe way to provide maximum performance to them.
We actually monitor this with some bots and scripts.
So we have a chat room where a bot will notify us
if an alert is triggered, for example, in Grafana.
BRUNO AZIZA: So let's talk a little bit about Grafana.
Like how does that work?
So you've got alerts telling you when
queries are performing poorly.
How does that work?
How often do you get these alerts?
How did you orchestrate?
RENE KROON: So there are basically two ways
to go about this.
The first one is to use stick driver metrics.
That basically gives you some insight
in what's going on in projects.
So that's also how we started.
So we can use a funnel to easily export
the sector-- use the stick driver metric
and put an alert on it.
And they'll notify us in a schedule.
The first alert we made, for example,
was if our slow pull is full.
Well, on Monday morning, it's always
triggering because that's when everybody
is doing their weekly work.
And then the second part, where we
get a bit more sophisticated, is by using BigQuery's information
schema tables, which holds information on each job that
has been completed.
And that's where we actually can sort of spot the bad apples
or changes.
BRUNO AZIZA: So you have a system that, first of all,
you've taken care of the price lack of predictability
by going the flat-rate.
But also, you're now watching performance
and the use of the platform by creating real-time alerts.
Those are in real time, correct?
RENE KROON: Yeah.
It's real time.
So the good thing is that it's correlating with business use.
So if we are in the chat room with our sort
of administrators, then you will see
the alerts popping up throughout the day if something happens.
BRUNO AZIZA: That's excellent.
So that's our best practice number one.
Let's talk about now the opposite, which is something
that people need to avoid.
What did you learn through your journey?
And what should people watching us watch out for?
RENE KROON: I think using Data Studio in an enterprise setting
can be tricky.
So what's important to remember is
that Data Studio is not an official Google Cloud product.
But it's part of the Marketing Suite.
So basically, we have to treat it
as a sort of Sheets or Document.
And that makes it behave a lot different than what we're used
to with cloud provisioning.
BRUNO AZIZA: What about the area of security and governance?
What should people avoid?
RENE KROON: Yeah.
I have a book here.
It's "Cloud Native Transformation" from O'Reilly.
I read it back to back.
It's a few pages on good practice
and best practices on cloud.
I was very happy to see that we applied most of it,
except for do as much security as you can upfront.
And I can tell from experience that delaying it
will not make it less painful.
In fact, it will make it more painful.
So don't take a shortcut with security.
Just rather, figure out something else, if you can.
BRUNO AZIZA: If you had to start it over,
what would you start with from a security standpoint?
RENE KROON: Well, we started with the cloud
as sort of an experimental way.
So we put in data that was not sensitive at all.
And then at the later stage, we wanted
to add personal information, for example.
And then we ran into a lot of issues.
Because of the experimental nature,
we did not define the architecture very well.
So we had to reverse engineer it and then apply new measures.
But we did succeed.
But I think the lack of architecture
sort of killed us here.
BRUNO AZIZA: So basically, you start with nonsensitive data,
test your platform, and then you move up the level
of sensitivity as you learn.
What about data quality?
RENE KROON: Yeah.
So that's one of the do's, I would say.
So people start putting in data.
You have to load threshold.
It's working very well.
And then you have all these data sets.
And the people that work in finance,
they are used to crosscheck results and make sure
that the data is really correct.
But a lot of people are already happy that they
have access to the data and will just do a report
and go with it.
But that does not mean that the data itself is in good shape.
So there can be all kinds of discrepancies between BigQuery
and the original database.
So we also built a separate tool for that that
can check per hour or 30 minutes if there
are discrepancies between BigQuery
and the relational database.
So that we make sure that the reports on BigQuery
are reflecting the true state of things.
BRUNO AZIZA: How does that work?
RENE KROON: It basically is based on a time window.
So we first started with a sort of complete comparison.
But that doesn't work for data that
changes fast or frequently.
So it works with a bit of timing now.
And within that time window, it will
detect synchronization errors between the source
and the target.
And at first, it will hash a lot of records,
take a lot of records, see if they are correct.
If there is an error in that, if the checksums do not
match between remote and local, then it
will start zooming in until it will point you
to the record that is not correct.
BRUNO AZIZA: Is this a tool that you built yourself,
or did you buy the--
RENE KROON: No.
We built it ourselves.
But I would say the idea as the technology behind it
are not new.
But we wanted to provide an easy integration
endpoint for the teams.
BRUNO AZIZA: What did you use to build it?
RENE KROON: Most of our internal tooling
is running on Java or Kotlin these days.
BRUNO AZIZA: Excellent.
So you've got a system to check quality
that you've built yourself.
And that's going to allow you to scale to thousands of users.
Rene, well thank you so much for the time today.
We learned about data quality.
We learned about governance.
And we learned about a great book
that everybody needs to go out and get the "Cloud Native
Transformation" book from O'Reilly.
If you want to find out more about stories just
like this one, click on the link down below.
I hope that you can connect directly with Rene as well.
Until next time, I'm Bruno Aziza.
[MUSIC PLAYING]