Welcome to 'This is My Architecture'.
Today, we are in Zurich, Switzerland,
and I'm joined by Alkis from Hexagon.
Welcome Alki. Thank you very much Christian and nice to be here.
Alkis tell us a little bit about Hexagon, and the project you've been working on.
Absolutely, Hexagon is a global leader in providing
geospatial data solutions and services in a range of industries.
Part of the company is focusing on building lighter hardware solutions
that can be either handheld devices or devices that can be mounted on a car
or on a plane.
These devices are primarily used by professional sales, architects, city servers,
or government agencies.
Our project, which is called Hexagon Digital Reality
is Hexagon's cloud space data store that stores the data
licensed by Hexagon or its partners.
It also provides a platform on which you can integrate with,
or push data from our hardware devices.
Great, so that sounds very interesting.
Let's dive into the architecture and explain what we are seeing here.
Absolutely.
On our left-hand side, we have our AWS data and streaming
and on our right-hand side, we have our AWS user experience and processing.
Both use the logic of streaming data back to the user
but also the user experience and the process assets
are hosted on EKS as you can see.
We went with the approach after migrating from Kops to EKS
because we want to focus on the manage services and take into consideration
and the benefits of scalability, autoscaling,
and the self mechanisms of EKS.
Perfect.
So you mentioned that you're generating a lot of data,
high precision data from the devices itself.
So I guess that's a lot of data you're dealing with.
How do you handle that?
So we created only one data account where we store all of our data on S3
and this becomes available through our EKS
and through VPC peering with our multiple EKS and AWS account
which is dev UAQAT.
And in that case, we actually make the data available both to our development
and the users using the AWS's backbone network securely and privately.
And I guess that's where you profit from low latency
and high bandwidth since you're using the AWS backbone
and also provide the application of these big data sets
multiple times to the different environments you're running.
You mentioned previously that customers have also the possibility to upload data
from their devices and from other sources.
Please walk me through how this works.
That's correct.
Both of these use cases trigger the same processing pipeline.
Either a user drags and drops or direct from the device,
what happens is our system provides a pre-sign Euro back to the user.
When that happens, the user actually uploads the item directly to S3.
When that becomes available on S3, when the uploads is finished,
an S3 event is triggered and sends a message into a queue on our SQS.
Okay, so you're having a mechanism to upload the files to S3
and have that event sitting in SQS.
So I guess there's some sort of microservice or service picking that up
in order to be processed.
Yes, we have a component running a container on EKS,
and that is listening to that queue.
So when the message becomes available there, then the logic of that begins.
Okay, so what happens next?
I guess that's not the processing itself,
so I guess you need them to process the data and store it somewhere again.
The next step is where that you're mentioning the processing pipeline.
So we have different output based on the uploaded file.
And in that case, we need different CPU or different memory.
So we created a group of other scaling groups based on that logic
and within that we are using EC2 instances.
Those EC2 instances are having templates
that are spinning up specific containers within those instances
that are pulling the images from ECR.
So when the message becomes available
and a component picks it up, it sends the information back to those EC2 instance
again using a messaging system.
When that happens its specific EC2 instance creates its own pipeline step
to that processing pipeline.
So I guess here you're profiting from the variety of different instance families
and instance types, because like you mentioned, you have, depending
on the output type different recommendations or requirements
regarding the CPU and the memory.
So what happens next, and where do you install the data,
and how do you make it available to the customer again?
So when the process is finished the EC2 instances are sending back
to the S3 and that through the same becomes available back to the user
and that processed item can either be placed on the map,
in our platform or be available for download. Okay.
That's very interesting.
So thanks for sharing this architecture with us today Alkis.
Thank you very much.
It was really nice to be here.
And thank you for watching 'This is My Architecture'.