HARISH KURAM: Hello, everyone.
Please welcome to today's presentation
in the data analytics stream, scaling data driven insights
across a complex global organization
with Looker and BigQuery.
In today's presentation, we would
like to cover Iron Mountain business and technical drivers,
data analytics and visualization architecture that
was established, scalability principles that
were in play, GCP products and services that were used,
Looker value proposition for Iron Mountain, challenges
and recommendations made.
Before we get into the details, we
would like to introduce ourselves.
I'm Harish Kuram, principal data architect at SpringML.
I was the project manager for this Iron Mountain project.
I have around 25 years of experience
implementing and delivering technology projects
across many large companies.
JARRETT GARCIA: Hi, my name is Jarrett Garcia.
I'm the director of enterprise platforms at Iron Mountain.
I've been with Iron Mountain since December,
and I've been excited to join the Iron Mountain
team to establish a singular data platform.
HARISH KURAM: I would like to give you
a quick introduction to the Iron Mountain technical data
analytics project.
SpringML, in partnership with Google PSO,
worked together for 10 weeks this past March
and April to successfully deliver this project.
Iron Mountain provided the program leadership and SMEs;
SpringML provided the sprint planning, scrum management,
and implementation; while Google PSO
provided the product assembly, architecture, consultation,
and oversight.
I would now like to hand it over to Jarrett
to speak about Iron Mountain, the initiative that
led to this project, and the current state.
Over to you, Jarrett.
JARRETT GARCIA: Thanks, Harish.
So this is the typical who is Iron Mountain slide.
We're a global leader for storage and information
management services, we're in 50 countries,
and we service a ton of clients.
I'm not going to spend a lot of time here.
Hopefully you've heard of us and we
can move on to the next slide.
So today, what we wanted to talk about
is the global enterprise, the data platform
where what we're trying to solve is a localization problem
that Iron Mountain has had.
So we have a lot of different systems,
a lot of different countries that we're working with,
and each system in each country has
their own way of doing things.
It tends to lead to a lot of repetition
as well as a lot of customization.
It also gives our executive team a lot
of headaches because we're not able to necessarily look
at the company holistically.
So when we want to see a footprint
for a particular client, it can take several weeks for us
to be able to gather all that data
and then be able to present that information back to our client.
The goal of the global data platform
is to be able to centralize all of this.
The project that we began with Google
and with SpringML was geared to get this kicked off.
What we wanted to do is to take some of our operational systems
and land everything into a central data platform.
We also then wanted to write a bunch of different dashboards
and develop insights that would enable our leadership
to have a global viewpoint.
Moving on.
So there were three main things as we talk about our master
data initiative.
One is the technology.
So we're a Google company.
We love working with Google.
We love GCP.
We felt that BigQuery was the right platform for us.
BigQuery has a lot of benefits, and Harish
is going to go over some of those later on today.
But as a platform, being able to pull everything
in to a central area and then be able to utilize BigQuery
and all the things that come with it
really is going to help speed up that process.
The other thing that we had to consider as well
is a governance policy.
So what we wanted to do is to start to set up a data steward
model where we can formalize our data governance practices.
So our idea here is to democratize data
and to make it readily available to all Iron Mountain employees.
But we do have to consider specific things from a data
protection standpoint.
And then the other part of this is
it's a big cultural change for us, as well.
So the company wasn't used to thinking this way.
So what we wanted to do is to start
to show the entire company that this type of thinking
and efforts can happen quickly and pretty seamlessly.
We adopted an agile mindset, and we
were able to start to collect data
from different countries and different operational systems
and start to land it into the cloud.
And then as well, we partnered with SpringML with Looker
and with Google in order to build out those dashboards.
Moving on.
So I'm going to jump a little bit ahead.
Harish is going to go through all the details about how
we got to the end state.
But to give you a preview, we were extremely successful
in the project.
We have today, based off the success of this project,
we have 11 different systems feeding our enterprise data
platform.
We have data from 56 different countries
flowing into the platform.
And for us, five terabytes is--
it's a good start.
We've only been up and running for about six months,
so we're very happy with the progress that we're making.
The monstrosity that you see on the right is kind of a picture
that we have.
We're servicing different use cases, as well.
So we're helping the business from the records management
side, from the data management side.
We're working with our colleagues in infrastructure,
within project management and HR,
to be able to give them a lot more capabilities
than we've had in the past.
So with that, I'm going to go ahead and hand it
over back to Harish.
HARISH KURAM: Thanks, Jarrett.
Before I delve in deeper, I would
like to spend a minute or two talking about SpringML.
We are a premium Google Cloud Partner,
and we have delivered over 200 plus cloud data
analytics and visualization implementations.
We have four main capabilities on the Google Cloud
stream of our business, machine learning
and artificial intelligence, cloud migrations, data
analytics and visualization, and application development.
As a Google Cloud Partner, we have achieved credentials
for four specializations, data analytics, machine learning,
marketing analytics, and application development.
Now I would like to delve deeper into the Iron Mountain story.
I want to talk about Iron Mountain's
business and technical drivers for this and future projects.
Global operations has a requirement
to realize cost savings in record center operations.
There's a need for productivity monitoring
in record centers across the Iron Mountain footprint.
There is a need for consistency in building and activity
definitions across countries and operational systems.
There is a need for visibility into facility or building level
information.
There's a lack of clear insight for leaders
across multiple sites, regions, and countries
that needs to be addressed.
There are mostly file-based processes with challenges
related to file transfers.
There is a monolithic data source scape
with challenges related to reference
data definition for districts, buildings, and activities.
There are time consuming manual checks
with reference data provided by business.
The reports are defined in crystal reports provision
through business objects.
There is a need to standardize reporting tools.
Now I would like to talk about the architecture
of the solution we designed.
Some important features to note, an on-premise virtual machine
to upload the data extracts from multiple data sources
to Google Cloud Storage; file upload event-based invocation
of Google Cloud Composer DAG; upload
of data to BigQuery staging tables orchestrated by DAG;
invocation of SQL transformations
and merging of data orchestrated by DAG into BigQuery
and reporting data set; and finally, RCO reporting
dashboard that we built on Looker which I'll
present on the later slide.
The relevant scalability principles
in play for this engagement are BigQuery,
a highly scalable serverless data warehouse
setup with on-demand usage and growth;
recommendation for automatic scaling of Composer worker
nodes based on usage metrics; and finally, an event
driven ingestion pattern we've built with discussion
around moving to message driven Cloud Pub/Sub in future.
With that, I would like now to talk about the key TCP products
and services we implemented on this project.
Google Cloud Composer for workflow orchestration
that included processing and loading files into tables
and running secret transformations and mulch
queries.
BigQuery for loading, coding, and analyzing large amounts
of data cost effectively.
Google Storage for staging and archiving of file
extracts in CSV formats, facilitating
future retrieval if necessary.
Google Operations or Stackdriver for logging,
monitoring, and alerting activities
of the ingestion pipeline.
And finally, delivering insights,
reports, and dashboards using Looker.
I would like to spend a few minutes talking about business
intelligence platform, Looker, and how
it benefited Iron Mountain.
Implementing Looker helped us quickly
deliver insights and enable real time analytics in a matter
of two to three weeks.
It helped us implement governance and data integrity
by limiting table access, data access,
masking specific columns, and auditing.
Having the self-service feature enable the business analysts
to ask new questions about the data that is readily available,
and to troubleshoot problematic indicators
with Iron Mountain operations.
And most importantly, Looker leverages the GCP investment
by Iron Mountain.
It basically leverages BigQuery natively
to perform all the queries.
What I'm now presenting is the record center operations
dashboard that we developed for Iron Mountain.
Some of the insights the dashboard enables
are the ability to investigate dip in achievement percentage
by drilling to specific country, and district,
and down to the warehouse; collaborate with country
managers to identify specific bottlenecks
and take specific actions to bolster the workforce
or take corrective actions within the same day;
ability to explore the data and ask additional questions,
especially why some buildings are reporting high overtime
hours, for example; set up alerts
when achievement thresholds are not met so quick action can
be initiated.
Some of the challenges we found and the recommendations
we were able to provide are agreeing
upon data formats, business logic, and structure
ahead of time to minimize re-engineering, continue
to develop a diverse culture for maximum scalability
and portability by leveraging terraform for automated
deployments, and building a CICD pipeline for efficient code
deployments.
Data cleansing is best done upstream,
near the source of the data, to minimize complexity
in coding logic.
Review architectural decisions constantly and optimize
to implement the right fit.
For example, Cloud Dataflow was determined
to be an overkill for the simple transformations.
We switched to using BQ Load, as there were minimal or no SQL
transformations, or the parallel processing wasn't needed.
Identified growth opportunities.
We recommended improved security monitoring,
firewall logs, alerts, and so on, fault resolution,
and performance management for future projects.
And finally, I would like to close out
with the project outcomes we were able to achieve.
We completed proof of concept to ingest data
from Iron Mountain systems to BigQuery for data analysis.
We enabled enterprise data warehouse modernization
and reporting through data democratization.
Enabled near real-time handling of data ingestion
and on-demand visualization.
Helped reduce different tools being used at Iron Mountain
to reduce copying of data sets.
Enabled iron mountain for success
by beginning to create a unified GCP
standard across the organization,
and established an infrastructure baseline
for future data ingestion and visualization projects.
This concludes our presentation.
JARRETT GARCIA: Thank you.
HARISH KURAM: Thanks, everyone.