Welcome to This Is My Architecture
I am Stefano Sandrini from AWS
Today Luca Di Lieto from Enel is with me.
Hi.
Luca, Enel is a company that needs no introduction
it's very famous in Italy and all over the world.
Anyway, would you like to say something about it?
Yes, Enel is the biggest international distributor
in the world, in terms of end users,
we have 74 million end clients,
half in Italy and the rest
in Spain and South America.
You are the product owner of a system
called 2BEAT within Enel.
Can you tell us something more about it?
Yes, 2BEAT is the system that remotely manages
the electronic meter installed by Enel
on the delivery points on the distribution network.
A particular use case of 2BEAT
is the management
of the data related to the load curves.
Can you talk about the architecture you have implemented?
Yes, data are recorded
by the electronic meter,
then collected by another device
the concentrator,
on our network,
which makes them available to the central system.
And we have designed the central system
with two tiers,
an acquisition tier
and a processing tier.
The acquisition tier is basically
made of EC2 instances with a custom code,
which, actually, connects to the devices,
gets the data from the device,
and forwards them to the SQS,
and then to the processing tier,
for the processing.
I guess that, once the data
enter the SQS queue,
they are taken
by the other processing tier.
Yes, in the processing tier
we have Beanstalk,
which actually does the EC2 instance provisioning,
installs a deamon, the SQSD,
which connects to SQS,
fetches the messages
and forwards them to the microservice
done by our code
installed directly on EC2 instances
provisioned by Beanstalk.
Our microservice uses,
as data store,
Dynamo DB,
both for inventory data
and for processing data,
and then generates a message
towards the external system
through an SQS queue.
I can see you use other services,
Redshift, S3 and Kinesis.
What do you use them for, exactly?
We use them to control processing
within this architecture.
We actually have BI tools
that connect to Redshift
and, through a feature, Redshift Spectrum,
is able to execute query on S3,
and the data on S3 arrive
from our microservices,
that push data to a Kinesis Data Stream,
that, through a Kinesis Firehose service,
saves the data to S3. The data we save on S3
are basically data
we save also in DynamoDB,
metrics data from the microservices
and also the data that arrives directly
from the devices we save on S3.
Luca, Enel is famous
for innovation and continuous research,
so I guess for 2BEAT
you are already thinking
how to innovate the system.
Yes, we have activated a POC directly in production,
to shorten the time
the datum spend
within the architecture.
OK.
So, basically, we have installed
on our devices, IoT Greengrass,
which lets us move
some of the application logic on the edge,
on the devices.
Basically today we use
a pull pattern to acquire data because we have our code
that connects to the devices,
Of course.
We are moving, or trying to move
a push pattern,
where our code on the devices
through Greengrass, intercepts the event
of datum publication,
forwards the datum to an IoT core,
which in turn forwards the data
towards the layer, the processing tier.
This is for the edge part.
What about the processing tier,
do you have any idea?
Yes, looking at the overall architecture,
we realized that we had
all the serverless services,
but one, Beanstalk.
Beanstalk works great,
but at a monitoring level
and at provisioning level
it is actually different form the others.
In order to harmonize the provisioning model
and the monitoring model,
we are trying to substitute
Elastic Beanstalk with a Lambda,
which, using the new Lambda feature,
Lambda polling SQS,
lets you have the same integration pattern with SQS,
message fetching on SQS,
the use of the same data store
and message generation
towards and external system through SQS.
Luca, thanks for sharing
this very interesting use case
and the architecture.
Thank you.
And thank you all for watching this episode
of This Is My Architecture.