Welcome to 'This is My Architecture.'
Today I'm joined by Simon from SBB.
- Hey, Simon. - Hello, Ramon.
Simon, tell us about SBB.
SBB is the Swiss Federal Railway Company
and its subsidiary, SBB Cargo, is the main freight company in Switzerland.
And that's where we had this project here.
Okay, so you brought us a nice architecture diagram here.
Maybe tell us, in the first place, what was the project about.
Every time a freight train is leaving,
one person has to walk all around the train for inspection.
And for a long train, it can take more than half an hour.
And with this project, we wanted to reduce this time,
and we do it by putting sensors on the train,
putting sensors along the track.
And with this data we collect,
we try to get a good view of the condition of the train
before the person that is going there.
And then they only can focus on very few critical parts
so the inspection is done within minutes.
Okay, very cool.
So all of this sensor data is flowing into your system,
maybe you can walk us through all of this.
Yes, so Producer, let's take the locomotive.
There is a lot of data
and this data we want to capture to bring into our digital twin.
And the Producer is sending data in any kind of format.
So the Lambda is just taking the data and forwards it into Kinesis.
On a locomotive, there, we get about three million samples a day.
But we are actually only interested in very few parts.
So we use Kinesis Analytics to reduce it,
to filter out to just the data we need,
and then we forward it into a new Kinesis stream.
The target is DynamoDB,
so that this Lambda is just managing the data in and out of it.
So Kinesis stream, the data is caught by the Lambda,
and then it's pushed in the right format into the DynamoDB.
On the other side, you have consumers, for example, an algorithm.
The algorithm is deciding, is this train now in a good condition or not?
And this algorithm wants the data,
so we have this interface, the API Gateway,
where it asks for the data,
but, finally, the data is then, again, through the same Lambda,
pushed to the gateway and then to the consumer.
Okay, and so, this is all highly automated
and it accelerates the process that you are going through many times a day, I would assume.
So what were the top priorities, designing the system.
So the business had a clear idea what they wanted to achieve,
but we had actually, in the project, no idea how we would get there.
So what data do we need, what are the algorithms we are going to use?
So I had to design a very flexible architecture.
And I knew it will have a lot of different data sources
so I want to have a standardized way
because the maintenance of the whole system should be low.
So for example, let's say, here, we have the data from the locomotive.
We have another data source sensors along the track.
And now we want to get this data in to the digital twin.
And what all we have to do is copy-paste this stream here.
So all what is in this dash box, we just copy-paste.
And it can do copy-paste easily
because everything what is here is infrastructure as code.
And so, we just copy the Lambda,
so the infrastructure, stream, Kinesis Analytics.
Now we only have to modify the Lambda
because here the data can come in a completely different way.
So the Lambda is getting the data,
and at the end, it's, again, pushed into the same Kinesis stream.
So here we have all the data normalized, harmonized,
and can be used, for example, in this case,
it's a digital twin.
And even if, for example, exchange digital twin need another concept,
you just connect it here to this stream.
And, yeah, we are very flexible in that way.
So it's very easy for you to add multiple of those Producer pipelines
with different formats of data.
And you mentioned Lambda for example,
and I see a few other serverless and fully managed service
on the blackboard here.
Why did you choose those types of services
in comparison to a more traditional, EC2-based architecture?
As mentioned, one of our priorities
is to have the least operations as possible
because we're in a very small team
and we wanted to concentrate really on the business logic
to bring the data to the business.
And we didn't want to manage servers, operating system, or anything like that
because AWS did the servicing for us.
And for example, Lambda, the data that is modified
or transformed,
that's just a few lines of code.
There is no advantage having a large setup for that.
So very little maintenance overhead, so to say.
Absolutely.
Awesome. Thank you for sharing, Simon.
You're welcome. Thank you.
And thanks for watching 'This is My Architecture.'