Hello and welcome to another episode of 'This is My Architecture'. I am Sonika, and today I have Abhinav from Spyne.ai.
- Hello Abhinav. Thank you for joining us. - Hi. Thanks for having me.
Abhinav, can you tell us a bit about Spyne.ai?
Yeah, so at Spyne, we are solving global internet cataloging with computer vision at its core.
So we are using AI to help businesses shoot, edit, and publish their catalogs on marketplaces on scale at a fraction of cost and time.
Image processing, right? So I assume these are like many images.
Can you talk about what scale are you dealing with?
Yeah, so we process around 100K images every day.
Okay and who are your users? What category do they belong?
So our product is used by 80+ enterprises globally in food, automobile, fashion, and retail industries.
Okay and let's say this one user uploads the images, how does this platform help?
So users can upload images from a lot of sources like SDK, APIs, our application, as well as web.
Once we get the images they are passed on to our gateway, that's where the images come from,
and appropriately pass it to the Application Load Balancer where we decide, on the basis of the category of the image
and how exactly it is to be processed, on which ECS cluster would this image go through.
Okay, can you tell a tell us a bit more on the ECS cluster because you spoke about multiple categories, right?
Yeah, so every image comes with a lot of processing parameters and metadata and how exactly will the image be processed.
Okay? So once it gets to ECS we save the parameters and metadata to our RDS as well and our RDS actually powers our dashboard
where users can go view their catalogs as well as download them.
Okay, so once the category is done, what do you do next?
Okay, once we have all the info about the image - how exactly it will be processed..
We push those images to their appropriate queues.
Okay, and once the category is decided, the image moves to the appropriate queue, how do you process those images?
Okay, now we know what sort of model that image would go through
and what sort of CPU middleware would that image go through to have that sort of output.
So, then from SQS that image is consumed by the appropriate AI service.
- Okay, which means the processing layer? - Exactly.
So these instances... what are you exactly doing at this processing layer?
So we have divided the processing pipeline into three major categories.
It's 'Pre',
'Infer',
and 'Post'.
In 'pre', we do stuff like resizing, color correction. In 'post', we create images from the output that we get from the base,
and in 'infer', we read the images to their appropriate GPU models.
Okay, so I assume that these are different work types, right? So how do you decide what EC2 instance you will use?
Yeah, so for 'pre' and 'post', we let the image go through CPU instances because those are CPU-intensive workloads.
For 'inference', that image goes through GPU workloads, and for GPU, we use G5 Instances that are like A10 GPUs.
Okay, and you spoke about 'inference', right? So I assume these are like AI models.
- Exactly. - So how are you training these AI models?
So we have this training pipeline.
First, we upload the input data to S3 - the input training data.
Once we have the input training data we trigger SageMaker. Once we trigger the SageMaker, it gets the input data from S3.
It gets it and starts the training.
We check the losses and accuracy of how the training is going and once the training completes, it uploads the weights to S3.
Okay. So once the weights are there innthe S3, how do you update the processing layer?
Once we have the weights on S3, we have a basic benchmarking process through which the weights go through
and once it passes our benchmarks and accuracy, we trigger the SageMaker, it triggers the Lambda.
Okay. From there the Lambda synchronizes the weights from S3 to the processing services.
Okay, and what are you doing to make this, because you are doing image processing, so, I assume that training data is a lot.
So, once you have so much of training data, what exactly are you doing in SageMaker to optimize it?
In SageMaker, we are leveraging Spot Instances.
With Spot Instances, we have been able to reduce our cost to one-third of it
just by using Spot Instances,
and not just the cost point of it, with SageMaker, we have been able to distribute the resources and manage them efficiently and effectively within our team.
Oh, that's great. Thank you so much, Abhinav for sharing this architecture with us.
Thank you for watching, 'This is My Architecture'.