Welcome to watching This is My Architecture.
-I'm the FanJie, architecture from AWS. And here are two guests from Momenta, Jeff and Michael.
- Hello both. - Hello.
- Jeff, could you introduce Momenta company to us?
- Momenta is an autonomous driving start-up company. Our core technology is deep-learning based perception, HD map,
and autonomous path-planning algorithm. Our products include multi-level autonomous driving solutions and the derivative big data services.
Our motto is to create brains for autonomous vehicles. - A cool company with a leading position in this field.
So the solution being demonstrated here is targeted for the issues in autonomous driving, right?
What issues does this solution solve exactly? - The solution we are talking about today is targeted for our in-house data flow issues.
It comprises 3 parts. The first part is to finish the acquisition process by collecting data and storing it into DynamoDB
through APIGateway. Secondly, sending the data to our partners through SQS. Lastly, part of the data goes to our filtering and training system.
This solution solves issues regarding data acquisition and sending messages to partners. Training content for later phase is also kept on the platform.
Can you give more details on the collection practice?
- Some data will be generated from the vehicles, including camera, video, photo and data from other sensors. Data files will be saved in S3 directly.
The description of the data file, the metadata, will be saved in DynamoDB through APIGateway. In fact, there is various type of data.
Data from the vehicles generate results through vehicle-carried algorithm models, the Inference. The more definite result triggers
one Lambda function through DynamoDB to write into SQS and another Lambda function to push to partners.
All the data triggers another Lambda function to write into ElasticSearch, and ES offers rich search features. The less definite result triggers
another Lambda function to enter the filtering and training platform. - So regarding these 3 routes,
All data goes through the above routes for retrieving and searching. The middle part is for the definite events on the vehicle-carried platform.
Those need to be further processed events will be pushed to the relevant partners and 3rd parties by the platform. What left are the uncertain events,
and those events will be handled by the training cluster, right? There are things interest me. Firstly, aside from this Lambda
in the middle having two queues, most Lambdas connect to service directly after processing the message,
then why a queue is added here? - Because we can not control the services from our partners, there might be some availability issues,
or the response time is too long. When such situations occur, they might block the data passage on this route. A queue
with a retry function allows data to wait in the queue and to be pushed into a dead message queue after a number of retries. The failed cases will be handled separately.
- You mentioned blocking. On which part does the blocking occur? - When Lambda is pushing data to partners directly,
this part will retry when failed. - This part where DynamoDB Streaming sending to Lambda.
If Lambda failed to process or the time is too long, then the streaming flow triggers new Lambda.
Therefore, what this DynamoDB triggers is an ordered queue. Understood. It's suitable to be used for decoupling and caching here.
After data is sent to S3 service, it will be used by training tasks. Are there anywhere else that also use the data from sensors or photo
or videos? On one hand, the URL of the original data file was sent to the partner when we push the original data.
We use S3 feature to generate a pre-signed link for this URL for the sake of security reason. This link is valid in a very short period
and un-guessable, only the designated partner can fetch it within the designated time. - I see. S3 is treated as a shared platform and
protection is formed through the security control on the sharing process. Okay. Since some of the events can‘t be predicted at the processing phrase,
how are they defined and how do they work after coming to the Lambda platform and the following training cluster platform?
-This is the filtering and training cluster, it is deployed on AWS GPU autoscaling group. We use Kubernetes to schedule tasks and to realize auto-scaling of the clusters.
-Therefore, it is a containerized platform and runs on the GPI instance. So, what instances do we use in here?
-We use different instances for Inference and training. Since Inference requires very little from GPU, we use P2 instance. Training is more demanding,
we use P2 or P3 instances depending on the training tasks. -Therefore it is dynamically scalable and expandable. And with different types of tasks,
you can use different types of instances to meet the requirement. Another question. As a matter of fact, many partners and customers can choose to
use their own data center for deep-learning. Because the cost of doing it at one's own data center is not very high in China. How do you look at doing the Training on the cloud?
What are your considerations regarding the TCO of doing Training on the cloud? -There are two things to consider. One, our data has come from the cloud.
Data is stored in S3. If to be downloaded to local and sent back to S3 after training, it's costly. The other, since we use many dynamic extended tasks
and acquires many GPU instances, when the tasks are few, we can release the unused instances. Thus not only we are free from limitation of the number of machines,
but also we can release redundant resources to save cost when the number is not sufficient. -Under many circumstances,
when analyzing and processing internet data( and exactly because it is from the internet), processing it on the cloud is the most reasonable way, even for a company focusing on deep-learning.
Because the cloud has a large bandwidth, elastic resources and elastic usage abilities, right? Okay. Thank you for your sharing. This is a fantastic architecture.
Thank you all for watching this edition of This is My Architecture.