[AUDIO LOGO]
BRUNO AZIZA: Hi, everyone, I'm Bruno Aziza,
and welcome to another episode of "Data Journeys."
This is the place we come to learn from data leaders--
their lessons, their do's and don'ts, their journey.
And today, we're talking to Edward from Autotrader UK.
They've got a really interesting use case-- three
petabytes of data, 4,000 dashboards, 34 data products
they've built, reinvented the way
they approach their data stack all together.
And so we're going to get right into that.
But before we do that, Edward, thank
you so much for taking the time to talk to us today.
EDWARD KENT: Thanks, Bruno.
BRUNO AZIZA: All right.
So at your company, more than half of your employees
are using Looker.
You got a lot of data.
So first of all, tell us a little bit
about what does Autotrader UK do?
EDWARD KENT: So Autotrader is the UK's largest
digital automotive marketplace.
So effectively, we connect buyers and sellers
of new and used vehicles through online classified adverts.
And we've been doing this for the last 40 years.
So we originally started as a print business.
We were founded in the 1970s.
And we've been evolving as times have gone by.
So we started our journey to digital in 1996
with our first website.
By the way, that was I think three years before Google even
existed.
And we are now a fully digital business.
In terms of some stats, we get in excess
of 60 million cross-platform visits each month.
We're around eight times larger than our nearest
competitor in the UK for the share
of cross-platform minutes.
And in the UK, more than 75% of all time
spent on all automotive classified sites
is on Autotrader.
BRUNO AZIZA: Eight times larger than the nearest competitor.
You've got 75% of all time spent on automotive classifieds
is on your site.
So you've got a really good sense of what's
happening in the industry.
Now, we'll find out today that over half of your employees
are using Looker.
So you've also been able to really activate
this data with your users.
But before we get there, tell us a little bit
about the unique challenges you have to work through
in order to get to this stage?
EDWARD KENT: The big one is the scale at which we operate.
We're one of the largest websites in the UK.
You mentioned some big stats there--
63 million cross-platform visits each month.
So just working with that sheer volume of data,
tracking user behavior, et cetera, at scale
has certainly been the challenge.
There's probably a couple of things
that are fairly unique to us in the industry as well.
One of the big ones is that we have
a lot of anonymous cross-device usage.
So people come to our site.
And they use it, but they don't necessarily
register and sign in.
And then they switch devices around.
They browse us on their smartphone, on the app.
Then they load up their desktop browser
and continue their journey there.
So tracking user behavior across these devices
is a real challenge.
We also struggle a little bit in that
we have two distinct groups of users
we want to do analytics about.
So we have consumers who come to our site.
They browse it.
They interact with adverts, et cetera.
They run searches.
But we also have customers who are the car
dealers who advertise with us.
They're the people who pay us money.
And the analytical needs of those two groups of people
are quite distinct.
BRUNO AZIZA: And you've got a pretty sophisticated stack.
You've got BigQuery, Bigtable, Looker.
And you've advanced your data products
in a way that serves these multiple communities.
Your migration had hundreds of stakeholders.
So take us a little bit through that.
How did you approach your migration
and how it relates to your data analytics strategy?
Did you just do a standard lift and shift or did
you take a different approach?
EDWARD KENT: We started by, first of all,
we built a data lake.
And then we started using Google BigQuery
to take advantage of that.
And that's where we started getting real traction.
So the speed at which we were able to model data, build stuff
out, and deploy that to users really
was the catalyst that really got us going.
Alongside though, we still had our old world.
So we had an on-premise Oracle data warehouse.
And that was driving most of our really high-profile,
high-risk reporting-- so things like financial reporting.
And that's where we needed to make sure
that we had a mature platform we could migrate to before we
started thinking about how we would go about moving that
to the Cloud.
So for the last couple of years, we've
been getting that migration underway.
And it is now complete.
We have powered off our on-premise servers.
And we are now fully in the Cloud
when it comes to data analytics.
But I guess the success behind that project
was the fact that we didn't just do a lift and shift.
So working in the Cloud means that we can approach things
differently.
BigQuery lets us work at a scale at which we simply just
wouldn't have been able to in Oracle.
So rather than just lifting and shifting our workloads
as they were, we basically rebuilt
them all from the ground up in our new data platform
stack using BigQuery, using DBT to model data,
Looker for dashboard analytics, and Airflow for orchestration.
BRUNO AZIZA: And tell me a little bit more
about the capabilities you deploy
using BigQuery that really change the game.
You talked about financial reporting, which is high risk.
It's got to be highly accurate.
I'm sure there's regulations.
There's all types of scrutiny around that.
So you've got to keep that very tight.
In addition to that, you've got a changing platform.
You've got a changing talent, maybe.
How did you work through that as you're evolving
to this new modern data stack?
EDWARD KENT: Yeah, so working with BigQuery and DBT
meant that, compared to our previous footsteps
into the Cloud-- so for example, working in the data lake,
working with Spark-- we were able to move much more quickly.
But with that comes risk.
So one of the key things that's given confidence to our move
to BigQuery has been building up testing
around data, testing around DBT models
and automated monitoring and observability that then sits
on top of that layer as well.
So that lets us know if anything doesn't look right
with what we've been modeling.
BRUNO AZIZA: And was the financial reporting
the main use case you drove, or were
are the others that you went through as you were evolving?
EDWARD KENT: There were a whole bunch of use cases.
So financial reporting was probably
the most high profile and high risk for us internally.
But we had hundreds of stakeholders both internally
and externally-- so people around the business
who typically would receive reports via email,
either every day, every week, every month.
And then some of those reports went out
to external stakeholders as well.
BRUNO AZIZA: So we talked about financial reporting here.
That's one of the use cases.
What were the other use cases?
And was there any difference in, now that you're
on GCP and BigQuery, you're able to deploy them differently
than you might have in on-prem world?
EDWARD KENT: Yeah.
So we had a whole bunch of stakeholders both internally
and externally effectively distributing
data about who our customers are,
how they're performing with us, how their adverts
are performing, how customers are
interacting with those customers, et cetera.
And yeah, we did approach things differently in the Cloud.
So we've been following this principle around the data
mesh and decentralization.
And that's really helping us.
So for example, we would model an entity, like a customer.
And we have a team who are invested
in who our customers are, what constitutes
a customer, what products those customers subscribe to,
et cetera.
But once we've got that, then other teams can then
use that definition to start building their own data
products.
And it's this concept of decentralization
that's much easier in the Cloud than it
would have been on prem.
So rather than a central team then
having to manage the customer definition that
is suitable for everyone in the business,
we can have a core customer entity
that other teams can then build off.
They can apply new requirements that
suit them that don't necessarily apply
to everyone in the business.
BRUNO AZIZA: That's an outstanding best practice
and a real-world example of how to actually deploy
a data mesh, which is a great transition
to our next question.
Once we talk about use cases, we want to know about
the do's and don'ts because this is not your first rodeo.
You've done this many times before.
And so we'll start with the positive.
If you were to talk to yourself at the beginning
of this journey, what would be the things that you would say,
Edward, you got to start with this?
Or you got to--
it's a best practice here or you've
got to start this team here or start with this use case?
What comes to mind when you think about these best
practices for you?
EDWARD KENT: So the first one for us
I think is decentralized.
And you alluded to this earlier.
And this lesson came to us a little bit late in the game.
But it stands to reason that if you
make data available through self serve
to people in your organization, then inevitably
they just want more and more and more.
And so what we found initially when
we had the central team effectively fulfilling
all these data requests was that we just
had this big backlog in the queue of data requests.
Could we just ingest this new data source?
Could we just model this new table?
Could we just build this new report?
And we just couldn't fulfill it fast enough.
And so decentralizing has been extremely helpful
where effectively, we make product teams
responsible for their own data, and we empower
them to build the data products that they
need by providing tooling to make it as easy as possible.
And in doing so, effectively, we distribute the workload.
But we also then distribute the knowledge.
People building data products have that domain knowledge.
They know more about the data they're
modeling than we ever could have done.
And so that's been really helpful.
And I'm talking decentralizing both on a skills
level within Teams, but also decentralizing
in terms of the data itself.
So rather than siloing data in one big project,
decentralizing it across multiple,
for us Google GCP projects or it could just
be multiple data sets, et cetera.
BRUNO AZIZA: And the infrastructure,
the way BigQuery is built does allow you to do that easily
by assigning projects.
And even on the cost management side,
you can do that very well.
I'm curious about the talent.
Did that change then the type of talent
those domains had to hire?
Did it change the type of talent you had to hire?
How did you manage that because a lot of that
is also a people equation?
EDWARD KENT: So we solved that by embedding data engineers
within product teams.
So rather than hiring an external talent,
we effectively just trained up people within those teams
to be able to use the tooling that we had.
BRUNO AZIZA: Excellent.
So you had your own data engineers
that actually were embedded in the business unit
so they can enable the construction of these data
products in connection with your team.
OK.
So that's a huge best practice.
What about the flip side of that?
What are the things that are the don'ts, the lessons that you
learned along the way that you'd say, hey, watch out,
when you started this use case or when you start on this
journey.
EDWARD KENT: So on the flip side,
I talk about decentralizing.
But one thing to be careful of is making sure
that you don't end up with multiple competing
copies of the same thing for no good reason.
And keeping on top of that has been
a bit of a challenge for us.
So it's very tempting for 10 different teams
to create 10 different customer entities.
And maybe there are good reasons for that.
And maybe they're a good practice
in terms of keeping that maintained, et cetera.
But often, it just leads to confusion.
So one big problem for us has been discoverability.
If someone wants to find out about the customers
and they go and search for our tables,
they might find 20, 30 tables that all
look like good contenders.
How do they know which one to go to?
And that's a challenge for us that we've probably
not yet resolved fully.
BRUNO AZIZA: So don't silo data.
Make sure you watch out because it has, of course, consequence
on confusion but also has consequence
on pure cost of having to store the same data multiple times.
Anything else that you think is an important consideration
when people are building for these data product structures
or this data mesh concept?
EDWARD KENT: Another big one I'd say
is automate your infrastructure configuration
instead of doing it manually.
So for us, we use Terraform.
But I know there are other solutions out there as well.
But if you want to be able to deploy stuff at scale
and have confidence that if you ever needed to rebuild it,
you could do so, use configuration
rather than manually deploying infrastructure.
BRUNO AZIZA: All right, so watch out for silos,
make sure you can automate as much as you can.
There's a last bit I'm curious about, this idea of adoption
of business intelligence across the organization.
It's pretty amazing that you have 4,000 dashboards
and more than half the company is running on Looker.
So I'm curious about how you got there.
What did you have before, and how did you
get to such a good adoption number?
EDWARD KENT: Yeah, so a couple of years ago,
the only way for people to consume data
was to effectively set up a report to be scheduled out
to them via email.
So Looker has transformed that.
It allows people to go and self-serve
what they need when they want it and to deep dive.
And they can explore data.
They can ask the questions that only
they need answers to directly without having
to have someone build a specific report to them.
In terms of getting to--
we've got I think over half the company now using
Looker on a regular basis.
And the reason for that is just by making
it the place to go for the data that they
need to do their jobs.
So we've got support staff working on telephones.
They get, for example, car dealers calling in.
Then our telephone analysts can use
Looker to then bring up information about that customer
in real time.
And they can use that to drive the conversation on the phone.
BRUNO AZIZA: Excellent.
And how did you build the talent to do that?
Did you find that you have to hire new people,
scale up people, train them on different things?
How did Looker change the game for you
in this idea of enabling more people to build data products?
EDWARD KENT: So we focused on upskilling internally.
So we run training courses at all levels.
So for people who just need to consume dashboard content,
we run training sessions for teams to show them
how to do so.
But we've also been upskilling people
on how to model the data under the hood.
So effectively, teams can build what they need without, again,
having to come to their central team.
BRUNO AZIZA: Well, thank you so much, Edward.
That was amazing.
We learned about your best practices
to embed data engineers inside your business teams
to make the data mesh a reality.
We learned about avoiding silos and automating
with a modern platform so you can effectively
scale the creation of your data products
across the organization.
I could stay here and talk to you for hours.
And I really hope that people can reach out to you.
In fact, I'd like to invite them to click down on the link
down here to get access to more stories like this one.
Edward, thank you so much for taking the time
to talk to us today.
EDWARD KENT: Thanks so much for having me.
BRUNO AZIZA: Until next time, I'm Bruno Aziza.
[AUDIO LOGO]