- So, we talked a lot about how Delta Lake
solves a lot of this complexity around managing data,
and how Delta engine improves the performances
of all your analytics workloads.
But, once you have all the data,
there's only a handful of analysts
who have the tools
to make sense of it in an organization.
Zooming out, data teams are small part of the equation.
In most organizations,
we have organizations like marketing, HR, finance,
they don't wanna use Python or SQL or Scala.
Today to consume data,
the rest of the organization
has to really jump through lots of hoops.
If you want data teams to truly unite,
we have to make it easier to consume that data
for the rest of the folks in these organizations.
Recently, a really fascinating open source project
has emerged, that's starting to solve these problems.
Redash, is an easy-to-use,
open source dashboarding,
and visualization service
for data scientists and SQL analysts.
With it, data teams are better able
to democratize sharing data,
across and within organizations.
The community behind this project is absolutely amazing.
This project started in 2013,
just like Databricks.
They have over 300 open source contributors,
and they're really passionate
about democratizing access to data across organizations,
and it's working in a really, really big way.
Every day, millions of users,
and thousands of organizations around the world
are using Redash to get insights,
and make data driven decisions.
They've seen a ton of success,
over 7,000 deployments around the world,
and the engineering team standing behind this great project
is absolutely fantastic.
And they've build something great
that aligns completely with the Databricks values.
So, it's my great pleasure,
to announce today
that Redash has joined Databricks,
and we're super excited
to align these two open source communities,
the Spark community with the Redash community.
To tell us more, about how this will work,
it is my great pleasure to introduce, Arik Fraimovich,
the Founder at Redash.
Arik,
- Thank you, Ali.
I'm really excited to be joining both Databricks
and the larger open source community of Spark.
Seven years ago,
I started Redash as a hackathon project,
and I didn't expect to be here today
sharing the story with such a large community.
Back then we just moved our data into Delta Lake,
and we needed somehow to use the data,
so, we tried some SQL Workbench tool,
and this was really missing collaboration.
We realized that what we need
is some web application where we can,
type in the query, get the results,
visualize, and share them,
and I decided to give it a try, that hackathon,
and that's what started Redash.
Working with Databricks earlier this year,
I realize how much alignment we have
on a lot of these core ideas
that I had when I started the project.
Make it easy to collaborate with others around the data,
and democratize data for all the teams.
But, beyond the product and vision alignment,
it's the amazing culture in Databricks,
that really makes me and the team excited
about joining Databricks.
But now, let's talk a bit about
what Redash actually is.
So, Redash gives you a SQL interface
to query your database in its natural syntax.
But we give you some tools to make it easier,
like the schema browser, auto complete and query snippets.
Now, once you have the data that you query
you can visualize it with wide variety of visualizations,
and then group these visualizations into dashboards.
You can also set up refresh schedule,
so that you get the most fresh data without waiting.
And when it comes to databases
Redash has you covered.
So, Redash supports over 40 types of databases,
data warehouses, data lakes,
and different types of API's.
So, it's likely that,
whatever data source your organization uses,
you can connect it to Redash.
Now, let's check out
how you can use Redash with Databricks.
Let's say that I'm a data analyst in a SaaS company,
and I'm looking to help the business understand
revenues usage and how to drive conversions better,
and reduce churn.
We have at least three types of data,
account data, payments data and usage data.
But, this data is coming from three different sources.
Accounts data is in the operational database,
payments is with the payments processor,
and only usage data is in our Delta Lake.
For many meaningful analysis of the data,
we will need to join the data from all three sources.
We can do this join in Redash,
but this has limits and performance issues.
What we really want is a single source with all the data,
and this is exactly what we can achieve with a Delta Lake.
Now, all of our data is in a single location,
and we can easily use the data together
to create more meaningful insights,
and have the most fresh data.
And now, with that intro,
let check out the actual demo.
Right, so here we have the Redash query screen,
and let me type in the query
to just show you a bit how it works.
So, a handout to complete helping me a bit here,
although it doesn't save me from the typos.
Now, here on the left,
we have the schema browser
that shows us the tables and their columns.
Now, we got the data back.
We can look at it but it's much nicer
to look at visualization.
It's easier to understand this way.
So, let's create one real quick.
So, here we see the trend
of our monthly recurring revenue over time,
and we can see that it's growing,
but we don't know why or how it's growing.
So, it probably makes sense
to see a breakdown of what's contributing (growth)
So, let's create a bit more complex query this time,
so, this one is a bit long,
so, I'm using query snippet for it.
Now query snippet is a very simple construct,
you have this keyword that triggers it,
and you have a definition of what the snippet will be,
and then when you trigger it,
it gets inserted in to your query.
I got the query back,
and this time,
I have much more information.
But again, a visualization can really help understand it.
So, here's a breakdown of our monthly recurring revenue,
and we can see that most of the time,
it's growing due to new monthly reoccurring revenue,
but we had some successful campaigns
around expansion in a few months,
and we have some regular expansion over time,
and unfortunately, we also have some churn over time.
Now, from the same set of data,
I can create some other visualizations,
like again, the growth of monthly
recurring revenue over time,
some counters,
and the average revenue per user over time.
Next, once we--
Sometimes we would like to look at these data
at the daily level.
So, here we have another visualization,
that's showing us the rolling window of 30 days
of the monthly recurring revenue at that day.
And that's useful when we want to track it
in more real time fashion.
For example, when we're running some campaign,
or we are aware of some issue
that we want to track more closely.
Once we have a few of these visualizations,
we would like to create a dashboard.
In these dashboards,
we see different facets of our monthly recurring revenue.
Like this breakdown that we saw before,
the growth for a time,
breakdown by different plan types,
and break down by geography.
Now, looking at the revenues is great,
but we also want to help our customers success team,
to help drive this revenue forward.
So, here we have a dashboard
for our customer success team
that shows them two types of accounts.
One type is promising trial accounts,
that's basically accounts that are currently
in their trial period,
didn't convert yet,
but showing good indication of usage,
so, we might want to reach out to them,
and help them switch over to a paid account.
On the other hand, we have the paid account
that showing a decline in usage,
where we would like to reach out to them,
and see if we can help them get back on track.
To help the customer success agents.
Each row here is a link
to this other customer insights dashboard,
and this dashboard is using a parameter
to filter out and show us only this account.
Here we can see some details about the account,
like their usage over time,
type of account
the number of widgets they created,
and we can definitely see the decline
and we can drill deeper to understand
why it might be
or just reach out to the customer.
Now, this was a really quick taste of Redash,
we have a more in depth session later today
that I recommend you checking out.
Thank you very much.