Welcome to "This is My Architecture."
I'm Matt from AWS.
Today, I'm joined by Sangho Lee from Samsung.
- Thanks for joining. - Thank you for having me.
So we're going to talk about Samsung Health.
What is Samsung Health?
Samsung Health was released in 2015.
Our mission is to help people be healthier
and get more exercise.
We have about 35,000 requests per second from all over the world.
Great. So this is a mobile app?
A mobile application.
And we've used various AWS products
such as DynamoDB, ElastiCache, SQS
and so on to handle the traffic.
Great. Yeah, 35,000 requests per second.
You need an architecture that scales, right?
Yeah, exactly.
So what happens?
I assume traffic from the mobile app
comes into API Gateway first, is that right?
Sure.
We have API Gateway at the forefront
and the traffic is delivered by a VPC link
to securely connect API Gateway and internal processes.
Okay, so because you're using VPC link,
you have to use NLB,
and then you're going to route your traffic
from NLB to the application.
It looks like you're using containers
or Kubernetes specifically, is that right?
Yeah, exactly.
So why did you choose to go with the containers?
We improved the way we operate our cloud.
In the past, we used to operate our cloud
in the traditional way.
Okay.
It's like we configure the runtime environment
on each machine
and build some RPM files containing application binary files.
And then deploy them into cluster.
However, as the service was getting more complex,
and various new features were being added,
we realized it would be difficult to operate them efficiently
using the traditional way.
So we decided to apply Kubernetes for this project.
Got it.
So you're using EC2 Linux with application binaries as RPMs
and as you got more popular starting to deploy more faster,
it allowed you to really unlock your developers
by using Kubernetes.
- Yeah, right. - Now you're using EKS specifically.
So why did you use EKS and not for example Kubernetes on EC2?
There are many ways about that.
First, we don't have to set up Kubernetes control plane ourselves
because AWS manages it.
Yeah.
And in addition, EKS runs Kubernetes control plane instances
across multiple AZs to ensure high availability.
And also EKS automatically detects
and replaces on healthy control plane instances.
So we can focus on our developing logic.
Great.
Okay, so undifferentiated heavy lifting is the managed services
that EKS provides make it easier for you and your developers.
- Yeah, right. - All right.
So getting back to the architecture,
we have the request coming in through VPC link into NLB.
And then NLB has to route the request
to the...it looks like you have multiple clusters.
What happens here?
Yeah, we route the traffic by port forwarding from NLB side.
So we needed a way to connect multiple ports to EKS node.
So you have multiple services running in the clusters on different ports,
and you're using NLB port forwarding.
Yeah.
But then how does it work on the EKS side?
Yeah first, we consider the load balancer
which is created automatically by Kubernetes...
The Load Balancer service type?
Yeah, but we were unable to take any advantage of that.
This is because it can connect only one port.
However, in our architecture,
we needed a way to connect multiple ports
with just one NLB.
Got it.
So you had multiple services, multiple ports
so load balancer node type didn't work.
So what did you use instead?
So we figured that the best way for us
using NodePort on other Kubernetes service type...
- Ah, NodePort. Okay. - Yeah.
So this is how we do it.
We define the NodePort inside of Kubernetes cluster...
Yeah.
And then map the port to the network load balancer
using target group.
Okay.
And we also create different NodePort definition
for each application.
So we expose the port to the outside using NodePort,
and then deliver users request to the NodePort
through the target group from NLB.
Okay.
So in this way, we were able to implement Kubernetes architecture
that can handle multiple port forwarding
with just one network load balancer.
Interesting.
I think a lot of viewers are going to find that interesting.
You have multiple services but also multiple clusters.
How many clusters? How many applications do you have?
Actually, we have 15 applications on this cluster.
Wow! Okay.
So 15 applications and within each application,
multiple services, multiple ports,
using the port forwarding
and the combination with a NodePort service type.
That's really interesting.
So there's a lot going on here, you know, in the scale.
So there's a lot to monitor and a lot of logs.
How are you handling monitoring?
Metrics for infrastructure
as cloud usage and DynamoDB usage
are collected by AWS CloudWatch.
Okay.
And metrics for application
such as CPU usage of Kubernetes port,
and network throughput are collected by Prometheus.
And we integrate those
two kinds of metrics into Grafana Dashboards.
Interesting.
So you're using Prometheus for the application level monitoring,
and then you're using
CloudWatch for the infrastructure monitoring,
and then visualizing it all with Grafana.
Yeah, exactly.
Okay. And so that's monitoring.
There's also a lot of logs,
especially at this kind of scale.
- How do you handle logs? - You're right.
We use a Fluentd and Elasticsearch
and AWS CloudWatch logs.
We first installed Fluentd agent
as DaemonSet on the cluster node,
and then Fluentd agent
then log to AWS CloudWatch logs,
and then AWS CloudWatch store logs into Elasticsearch.
Okay, so you have DaemonSet
and use Fluentd for logs,
the logs then go into CloudWatch,
and you load them into Elasticsearch cluster.
And do you use Kibana or something?
Yeah, we, finally we can monitor the logs through Kibana.
Okay.
And the last thing I see X-Ray up here, what are you using X-Ray for?
Yeah, since our architecture is based on MSA,
we also use AWS X-Ray
for application performance management.
Interesting.
So you know what, originally, I thought that the network part
was the most interesting part of this architecture.
But actually, the way you use multiple tools for monitoring
and logging and application traces is equally interesting.
It's another very interesting part about how you're using Kubernetes.
Yeah.
It's a really interesting architecture,
and I really appreciate you sharing it with us today.
Thank you.
And thanks for watching "This is My Architecture."