- Hello, welcome to This is my architecture. I'm Gianluca and this is Luca from Tooso.AI.
- Hi Gianluca. - Luca, what does Tooso.AI do?
- Yes, we are an artificial intelligence company based in San Francisco
and here in Milan. We are developing a semantic technology based
on a proprietary knowledge-graph. One of our first end-to-end applications
is within the field of digital commerce.
- Can you give us an example of semantic research?
- Yes. Our technology can identify, starting from a simple text
like "short blue jeans", the product specifications
and to return them to the user.
What use case are you introducing today?
This is our pipeline for data analytics acquisition
that our customers integrate into their apps or websites
and that allows us to train the search engine to optimise the results,
based also on the customer behaviour.
- Can you tell us more about how it works?
- The user accesses our client's website or application
and downloads a Java snippet from CloudFront to track events.
Whenever an event such as page view or product click occurs
our APIs call Route 53, which finds the endpoints closest to the user
and submits the request through the API getaway.
API gateway transfers the event to a small lambda function
for basic validation and privacy compliance.
Once this is done
we submit the event log with the request of the execution context
and the response from our lambda function to FireHose.
At the moment we use FireHose in a simple buffer mode
with 5 mins or 100 megs of g-zipped data max. When buffering is complete, FireHose transfers the GZIP file to the output S3 bucket.
- I can see other components here. What do you use them for?
- Yes, here we have SNS. We use it as message broker,
our S3 bucket is set up to submit an event when creating a new object,
for example from FireHose to SNS. Among the subscribers of this endpoint
we have our ETL lambda. This is the core of managing data.
We retrieve the logs generated by the Tooso applications,
we standardise them in an internal format we add our information
of the products selected by the client to create a persistent event database.
The lambda function transmits the result of these transformations
in another bucket S3 output so the data can then be processed
with the whole infrastructure spark and EMR.
- Luca I see that the whole architecture is serverless. What are the benefits
of this approach?
- The main advantage is that you can automatically scale
the whole data acquisition, saving and transformation pipeline.
So we can manage different loads without intervening
at the level of the architecture.
- Thanks Luca. What are the future developments
of your architecture?
- One of the next evolutions that we want to apply to the infrastructure
is to acquire data through API gateway and lambda
by using CloudFront instead of API gateway with a lambda at edge
so that we can guarantee low latency time globally.
- Luca, thanks a lot for being here. - Thank you.
- And thank you for following This is My Architecture.