When you're coding,
if you don't remember to go and click that button,
because you're actually coding
and not focused on other stuff,
it'll do that automatic detection
for you and just eliminate distractions
at the right moment for when you need it.
Welcome to "Powered by Snowflake,"
where I interview technology leaders
building businesses and applications on Snowflake.
I'm your host, Daniel Myers.
And today I'm talking with Mason Mclead,
CTO of Software.com.
Software.com is a DevOps metrics platform.
Mason, how are you today?
I'm doing great, how are you?
Doing well. Doing well.
So I really want to dive in here and learn more about you,
your background, and what exactly is software.com.
Sure, yeah.
So as you introed, my name's Mason, I'm CTO at Software.com.
We're a DevOps metrics platform.
Basically what that means is we look
at all the data that's coming through the tools that you use
as a software team to build your products,
and look at where your bottlenecks are,
allow you to see where certain processes
aren't working as well as you would want,
and give you tools
and information to go and fix that and improve it over time
so that your entire organization can work more efficiently.
And I always think that a more efficient system
makes for happier and more productive developers.
That's awesome. That's awesome.
So tell me about the history of Software.com.
When was it founded and where are you today?
Yeah, so it was actually founded in 2018,
and it had a very specific focus
on building tools for developers as editor extensions.
So we take telemetry from the actual editor itself,
and give you back time reports and activity levels
and all sorts of really interesting data
that individuals can use to help their own productivity.
And what we've done over the years is expand that
into organization wide metrics,
pulling in from system-wide tools that allow you
to see the full picture as well as power automations
and features for individual developers to protect your time,
to help you focus and to make your day more enjoyable.
That's pretty exciting.
So it really seems like a data cloud like Snowflake,
really helps to power a platform like that.
Yeah, absolutely.
I mean, it's very much at the core of what we do.
I always describe it as a U-shaped platform.
So we pull in data, we process it,
we create insights out of it.
We power automations with it,
and we send it right back to you in its refined form.
And how we do that right now is with Snowflake.
That's awesome.
And it really, I mean,
the age old saying of there's data
and then there's the information,
the actionable data that you're able to take from that.
And so it seems like a
DevOps metrics platform
like this is able to glean that information out
and take action on it.
So that's awesome.
You know, can you tell me more and kind of show me a demo
of this platform that you've built?
Yeah, absolutely.
So I'm going to start with the bare bones
of what this whole thing is.
So to demo how we built this on top of Snowflake,
I'm going to walk you through the data flows,
starting from how we pull data from our editor plugins,
web hooks, third-party apps and our own events.
We publish all of that to a Kinesis collector.
We're using Snowplow Infrastructure for this,
and we've got registered JSON schemas.
We enrich that data.
And then it goes into S3 files,
which we then use Snowpipe from Snowflake to pull that in.
And we're able to get events
from the time that they occurred,
to being in Snowflake in less than a minute.
And the average is about 20 seconds from event
to it being able to be analyzed inside of Snowflake
with this infrastructure.
We also have a Snowflake loader
for stuff that doesn't need to be real time.
So everything that we collect goes into there
and that's every 15 minutes, which is still pretty quick.
And from there we are, basically we've set up
all of these events and we're doing millions
and millions a day that are coming in through these pipes
and go into a big database split up into all these tables.
And that powers two big things that we do here.
One is a real-time, or as you see here,
every-five-minute flow that can detect
what we call our flow mode.
So whenever a developer's in a highly productive state,
we're able to detect that
from the telemetry that we pull from the editor
and we call a Snowflake external function
when we're processing this data to send that out
to a Lambda that has our model that's built in-house.
It processes that.
If it detects a flow score above a certain threshold
it then calls another external function,
which goes to a backend service
and uses web sockets to push updates
to all the clients that have gone into flow mode.
And what that allows you to do
is mute Slack notifications, book time in your calendar.
You can go full screen in your editor if you want.
I actually have it to turn on
one of my favorite Spotify playlists.
We have Spotify controls.
We can have it control lights,
all sorts of automations that we can do
with our code time extension and our editor ops extension.
So it's like Slack ops or anything like that,
but for your editor specifically. And in that same process,
we push up-to-date telemetry stats
to your code time dashboard,
which you can access inside the editor.
So you don't have to leave your primary workspace
in order to get that data,
and that's on the individual developer side.
So lots of cool tools that we're able to power
straight from Snowflake.
And it was really fun to be able to build out
all of these different external functions
to hook all this together,
because I didn't really see no one else doing it.
But I'm like, if I can do an API call
to anywhere, then I can do anything.
So I just kind of hooked it all together
and made it work.
That's awesome, that's awesome.
And a platform like this,
it's really cool to see these like tangible examples,
like the Slack "do not disturb" and "calendar protect,"
and "editor go full screen."
So that is really cool to see.
And I think it gives a clear, tangible
example of what this platform can do.
When going through this,
can we see a live demo
of how you would set this up?
Yeah, absolutely.
And so we can switch over to Visual Studio Code.
So, we have extensions for VS Code, for Sublime,
for IntelliJ and their whole family of editors,
lots and lots of options.
So wherever you're coding, we're probably there.
And it installs for VS Code, little extension here,
you can click on that to see your code time.
This is our dashboard that's here
and you can enter flow mode here or, again,
as you saw in the diagram, when you're coding,
if you don't remember to go and click that button
because you're actually coding and not focused
on other stuff, it'll do that automatic detection
for you and just eliminate distractions
at the right moment for when you need it.
So just to see how that works,
you can just enter flow mode here manually.
See that icon goes up there.
For this webinar, of course, I've closed Slack,
but it has muted my Slack notifications.
It also puts a purple dot as my status
so that people on my team can see that I'm in flow mode
and know not to bother me.
And that's really helpful as well.
And in my calendar, if I opened it up
you would see that there's that meeting that I booked
for just myself, that protects code time.
So no one's going to be booking time
on my calendar for the next 30 minutes.
And you know, that's configurable, depending on
how long you want that to happen.
And we can have an auto expire,
so it doesn't stay there forever.
You can come back and then
set custom statuses, all sorts of cool things.
And then to look at what I talked about before,
where I have it set up to play Spotify.
It also says that whenever I come out of flow mode,
so it's detected that I'm done and I've come back.
So in the time that I was talking, the model just ran
and decided that I wasn't actually coding at that moment
and deactivated my flow mode so that communication
can start flowing again.
That's really, really powerful.
Yeah, so whenever you enable flow mode here,
I've got it to say, it's code time.
And then it tells Spotify to put on a specific album
and you can do all sorts of cool stuff
with our editor ops extension.
It's exciting to see
how this level of automation and the types of data
that you're able to glean
to really improve developer productivity in this way.
Can you talk about and explain how and why
did you choose Snowflake to build this solution?
Yeah, so going back
to here with our diagram, one of the things
that I saw when I first joined Software.com,
which was two years ago, was we were loading
all of this event data, and it's really detailed,
into a traditional transactional database.
And we put some queuing in front
to try to manage the process
and to give the database time to actually ingest.
But it just, it wasn't scaling.
And we're up to around 300,000 developers
across the world using this platform, as well
as several thousand using our organization-wide tools.
And that kind of scale just wasn't happening.
And we had a cluster of databases.
We had managed services.
We were trying everything to get that going.
It just wasn't.
So I had to rebuild it
in order to go at the scale that we're at now.
And we tried with a self-hosted
sort of OLTP-type database using a column store.
And the ingestion was just the bottleneck
of the whole thing.
We would go too fast and it would fall over.
And so, I'll go back on here to this,
setup. We tried for three months
to get our other self-hosted databases up and running.
And I had used Snowflake before at a previous job.
I was the VP of Engineering at Fair,
and scaled that organization up.
I was the first engineer to over 120 engineers.
And we had Snowflake on the backend there
for the traditional BI use cases that you would see.
So I knew that it could scale.
I knew it could handle terabytes of data,
and it was just time.
The three months that we had spent
trying to get the other thing to work,
it just wasn't bearing fruit.
So I called up Snowflake and, actually within three days
of our POC, I had our first Snowpipe running
and it has not stopped since.
So I went from three months of effort
to try to get our homegrown one to work
to three days and it was up.
And from that point on, when you see that kind of success
that fast, you don't turn back.
That's an awesome journey to go through.
I mean, you've experienced what it was like
before Snowflake, and so yeah,
that's an awesome story. Can you talk about
how has the features of Snowflake,
I know you mentioned that you use external functions, right?
What other features of Snowflake
do you leverage when building with Snowflake?
Yeah, so, to look at our organization data--
we process a lot of data--we bring that in.
So the fact that it can do highly parallel queries
and process immense amounts of data continually
and really quickly, and it's a cost effective way
to do it as well.
We're able to build huge star schemas
with all our facts and dimensions for all the organizations
that have signed up with us. Pre-build reporting tables.
And we've got two use cases right now
for the data shares, one where we have a vendor
that's sharing data with us
so that we can classify remote versus office work,
which is really interesting research
that we're doing right now on our data set.
And we're starting to partner with a researcher
at Carnegie Mellon
to look at specific context-switching costs for developers.
And so a data share for them with an anonymized data set
is what we're looking at for sharing
with some academic partners.
And that enables a lot of really fast,
very secure data sharing,
without a lot of work on either side.
So I'm glad that you mentioned data sharing.
I mean, it really is a game changer
when you can eliminate the need
for an API wrapper just to access data.
And if the whole purpose
is to interact with, query, and access data,
utilizing a data share is a powerful way to do that.
So it's interesting that you mentioned that.
So, from here, what is next?
Six months, one year down the road, what can developers
and your customers get excited about down the road?
Yeah, so looking at what we're doing
with our system wide data,
plus what you just saw for the automations,
it's really going to be bringing those two things together.
So we're able to show you high-level
how well you're delivering your code
to either the default branch
or all the way to production,
if we've got trackable releases.
We can show you the lead time per stage.
We can show you how your code's converting through that,
where you're spending your time
and lots of details in all of that.
The thing to look forward to is,
from these high-level metrics
and these overview viewpoints,
being able to influence and help
individual developer behavior
in a very passive way.
Because we're not trying to control anything.
We're trying to suggest
and let people know best practices,
is that we'll be able to save developers time.
We'll be able to suggest specific actions
at the right time based on the individual data stream,
based on the organization data stream.
So the individual behaviors can be impacted
and influenced in a way that improves the overall metrics.
And so it's not just another data platform
that you just look at
and then you decide what to do.
This is something that can power discreet experiences
that learn from, and then impact these overall metrics.
I love that.
I mean, I love the fact
that it's able to take this automated action.
I mean, you showed it right there
in the demo of really blocking out those distractions,
things like that.
And having increased productivity
is one of those things where it has real tangible value
to the business.
So absolutely this is a really,
really cool solution that you've built.
And I love the fact that it's built on Snowflake.
And so from here,
if people want to learn more about Software.com
and the solution that you've built, where should they go?
Yeah, so go to software.com.
You can see all of the metrics
that we gather there and you can onboard
and see your own metrics in less than 10 minutes as well.
So if you kind of look
at the infrastructure that we had built
and how the data can come in so fast,
we can show you your overview
in the first 10 minutes after you've signed up.
And you're able to dive into details about your lead time.
You can see your GitHub actions, customize your dashboards
and also monitor to see when your pull request
review time is going up.
You can send out an alert to Slack channel or to emails,
all sorts of things that we're starting to build out.
So come and check us out at software.com,
and start to empower your teams with DevOps metrics.
Perfect. And Mason,
I want to say thank you for joining me today.
This has been another episode of "Powered by Snowflake."
And if you're looking to build your next application
on Snowflake, check out developers.snowflake.com.
Thanks.