Hi. Welcome to 'This is My Architecture,' coming to you from Zurich in Switzerland.
Today, I have Olivier from SonarSource with us.
- Hi, Olivier. - Hi, Ross.
What can you tell us about SonarSource?
So SonarSource was founded 13 years ago by three developers
and the goal was to provide code analysis solutions.
So today we have three products, SonarLint, SonarCloud, and SonarQube.
And we support 27 languages, program languages.
And today, we will speak about SonarCloud.
All right, great. That sounds great.
Can you take us through from the starting point
of what we see on the board here?
Yes, exactly. The starting point is developers.
As you know, developers write code and, say, push code into GitHub repo most of the time.
So when a code change is pushed into this repo,
it triggers an event
and we capture this event into this Lambda.
We do some triage. We do some analysis.
And then we create an analysis request.
It goes straight into this queue, an SQS queue.
Okay, why do you have SQS there?
Could you not speak directly to Fargate,
or even have Fargate picking up the message from GitHub?
No, we have many reasons to use SQS.
The first reason is to reverse the load.
We don't want to overwhelm the Fargate cluster the service.
And the second reason is for scaling
because this Fargate cluster is using autoscaling
and it's plugged directly into this queue.
So the Fargate cluster as a service will pull the queue
and then the first step is to clone the code.
I can imagine that can get pretty big.
You've got no control over how your custom repos could be.
Yes, exactly.
So the biggest codebase is nine gigabyte.
And, yes, it's pretty big.
And do you use any external storage from Fargate
in order to keep those repos or--
No. No, no. We stick to the ephemeral storage.
For now, it's good enough. All right.
All right, let's see what happens next.
Okay, so once we have the code, we can analyze this code.
And once it's analyzed,
we push an analysis report to this block.
And this EC2 instance is receiving reports through an API.
And then this report is stored into this database.
Once it's in the database, we have this component
implemented as a Fargate cluster.
And this component, we call it the compute engine,
will pull the database to extract reports.
Okay, so you're using this like a queuing mechanism--
Yes, exactly. It's temporary storage for the reports
we get from this component.
And the compute engine here is pulling the database to extract the reports.
Once a report is extracted, we do a lot of computations.
We compute a lot of code quality metrics.
And once it's done, we update the database.
And we create an analysis report here, ready to be consumed by users.
I would like to mention that we store also additional data here.
And this data is for optimization.
We do some search
and we need to have searching on an optimized format.
Okay, and is this Amazon Elasticsearch?
No, no. Not yet.
Today, it's an in-house Elasticsearch for a service.
But we actually plan to move to the Amazon version.
Okay, all right.
And then the next step from your users' perspective,
I assume they want to be able to see the results of all of this hard work.
Yes, once it's done,
they go to the UI
and they call the UI, or, simply, the API,
because you can extract the same data from these two different channels,
the UI and the API.
Okay, great.
And what does this look like in the real world?
How much is this being used?
For auto scan, specifically,
we analyze up to 150K analyzes per week.
And SonarCloud is part of an ecosystem.
We basically support around 300,000 organizations.
Those are some really big numbers.
That's been great, but what's next? What's coming next--
Many things. One of the first things we want to do is
to move to a managed solution service.
And a severely important point would be to remove
the queue we have in Aurora and to use a procurer, like an SQS queue.
Okay. Okay, get some optimization there.
Anything from a products perspective?
From a product perspective, for now, to scan, does not support compiled languages.
We support non-compiled languages such as TypeScript, Python, and so on.
And so, next step, to support Java and C#.
Olivier, thank you for sharing that. That was really interesting.
You are welcome.
And thank you for watching 'This is My Architecture.'