Hello and welcome on This is My Architecture.
My name is Rudy and I'm here today with Michel Poulalion.
Hello Michel.
Hello Rudy.
Thank you for having me and giving me the opportunity to share my work.
Welcome.
Before we start,
could you tell us a little bit about Veolia?
Yes.
Veolia, is the global leader for environment services.
The activity consists of three core businesses:
water cycle management, energy services management,
and the recovery and recycling of waste.
Some numbers : 170,000 employees around the world,
with a €26 billion turnover.
Ok.
You came here today to talk to us about a specific solution.
Could you tell us the purpose of this solution?
Yes.
The purpose is to identify,
and so it's a FinOps approach.
It's to identify the financial inefficiencies
when using AWS resources,
to identify and share them with a specific group of people
who can possibily
find a solution to these inefficiencies.
Ok.
If we dive into more details,
could you tell us how the solution works
in terms of services?
Yes, there are a lot of services,
but it's quite simple.
So you have two accounts.
We have a central account called "chapeau"
to invoke Lambda functions,
our operational functions, in general,
and a group of monitored accounts.
We have CloudWatch Events that runs everyday
at 7:30 a.m.
and that activates the different types of Lambda functions.
Each Lambda function, there are five of them today.
There will be more in the future.
There is a function that allows you to identify unutilized EC2,
so it uses
a remote access service
via a Describe EC2 API.
Ok.
How does it work?
Because here I see an IAM role.
I guess the Lambda function
needs specific permissions to access an instance account.
Indeed.
Indeed, I forgot to mention that.
There is an STS AssumeRole from the Lambda function
or from every Lamdba function
to get an authorized access to the scanned accounts
and this IAM role is limited to its minimum,
mainly for unutilized EC2,
to give permissions to Describe Instances
on an EC2 service.
A second function.
They all follow the same pattern,
mainly it's to identify the EBS that are unused,
the process is always the same, via a specific role for the EBS,
from Describe EBS,
always with an AssumeRole in the first place.
Small difference is that for unused EBS,
every function allows read-only access,
where the unused EBS don't delete the volume that is not being used.
Immediately.
So be careful with this.
It deletes the volume if there is no "Preserve" tag.
We created a tag called "Preserve".
If there is a "Preserve" tag,
it means that there is reason that explains why the EBS is unused.
It's the only function, in fact, that needs write permissions,
that is part of this IAM Role
to delete unused EBS
and that shouldn't be kept.
The RDS Unused function that identifies the RDS bases
on every account
with no activity for the last seven days.
Alright.
An ELB Unused function that will also scan the other ELB accounts.
Here, for the three core ELB families: the Classic
the Application, and the Network.
And, an S3 Metrics that is a bit peculiar,
which always uses a CloudWatch like the rest of its services,
and will collect every buckets with the size
and their size over time.
It allows you to have a history,
but also and mostly to see the lifecycles policies
that are set up in order to
use the correct storage classes that match our type of use.
Question.
I see a DynamoDB table.
How do you connect all these monitored accounts
using this solution?
The DynamoDB table is indeed a crucially important feature.
It shows every AWS account we have.
There is a line for every AWS account
and we have attributes and an attribute matching
a Lambda function we want to evoke.
We manage quite a lot of things via the Lambda services,
at least, for the FinOps, it's always true, true, true
for verification of efficiencies on the account.
Ok, it's clear.
What are the benefits that you...
I mean, the cost reductions
that you reached with this solution?
Overall, we were able to reduce costs compared to 2018
by 10% for our AWS use at constant perimeter.
Obviously, the Lambda scripts, these Lambda functions helped us get there.
It's difficult to say for the EC2.
However, we know that EBS allowed us to save $20,000 a year,
and it's the same for ELB,
because we had 80 ELB that were unused,
which equals to $20,000 a year as well.
Ok, interesting.
Have you experienced any kind of difficulties
implementing this solution?
I experienced some problems with S3 Metrics.
I wanted to use an API S3 on Boto 3, but it was not in the Boto 3
installed by default on Lambda.
I had to create a Layer with the last updated library Boto 3.
And from there, I was able to access the exact API I wanted.
And, the other problem... I mean, it was not a problem,
more something I noticed during the implementation,
it's that for the unused EC2 where we notice that
even when the CPU is not used,
the EC2 is still important.
For instance, an NTP server that doesn't use a lot of CPU,
but that uses the network.
So, I will implement a V2 for this Lambda,
so I can add the network criteria in the decision-making,
on how the EC2 is useful or not.
Ok, it's clear.
Great, thanks a lot.
Really interesting.
Thank you for sharing with us the architecture.
Thank you, Rudy.
It was fun.
And thank you for watching this episode of This is My Architecture.