Your data sources are taking too long to respond and that's making your Data
Studio report is slow?
Maybe we can extract data to create a snapshot and make your report load
hell lot quicker.
Hi, this is Ahmad from Siavak with another Data Studio tutorial.
And today we're gonna learn how we can use Extract Data Sources to create a
snapshot of our data and speed up and improve the performance of our Data Studio
dashboards.
When a Data Studio report is viewed or a new filter or a date range is applied
for each chart on the report canvas for each of scorecard, for each table,
for each trendline,
Data Studio needs to retrieve data from the data source.
This means that it should send an API request to ask for data.
It needs to wait for the API,
for the tool's API to respond and send the requested data
back to Data Studio.
And only then it can visualize that data on the report canvas.
The process of retrieving data from an external API is what usually takes the
longest amount of time.
So if we can perform this step beforehand,
and if we can extract data from those data sources from Google Analytics,
from Facebook Ads, from Google Ads and have it ready for Data Studio,
we can make the data set instantly available to Data Studio.
So the only thing that it needs to do is the visualization part,
which usually happens very quickly. If not instantly.
Let's see how we can do that.
So here on this report.
I have a live connection to Google Analytics and I have a table here with date,
source/medium, three metrics,
and I have three scorecards at the top comparing to the previous period.
Now, when we say live data connection,
it means that if I change the date range at the top,
Data Studio needs to ask the data connector
to request data from Google Analytics API,
and then Google Analytics needs to process that request to see, okay,
what kind of data are we looking for? What kind of metrics,
dimensions for what date range process the data and then response
with a data set to make it available for Data Studio to refresh the
report and show us the new numbers. Let's see the process in action.
So for example,
let's say I want to add last year as the data range
for this report. And you can see that that when I hit apply,
the process takes a bit long.
It takes like right now it's 10 seconds is still waiting for
data. And I guess it takes maybe a bit longer. Okay.
We have our dataset, it took about 10,
15 seconds before the data was available to Data Studio and then instantly
older scorecards and table showed up. Now,
let's see how we can use extracted data source to extract the same data
set and make it available instantly for Data Studio.
So let me go back to edit mode and duplicate
these scorecards and the table,
and make a copy of them on the right,
that's it.
These are still connected to the sample demo account data source,
which is a direct connection to Google Analytics.
Now we're gonna add another data source. To create Extracted Data Source.
We need to click our data just like any other data source. And here
we should use this one. The Extracted Data Source is a bit different, uh,
compared to the other data connectors that are available to us in terms that the
other data connectors,
they connect directly to a tool or marketing or Analytics tool to extract data
from.
But Extract Data does not directly connect to a third party
tool.
What it does is that it takes one of the data sources that we already have
and create a snapshot of data from that data source. So for example,
here,
I'm going to choose the sample demo account as the data
source that I'm gonna use. And I'm gonna extract data from.
The next step would be to decide what dimensions and metrics do we want to
extract from the data source. We know that to create that table,
we needed the date, sorry,
date as the dimension,
we needed source/medium as the second dimension.
And if I remember correctly,
we were using users and new
users and sessions. We had revenue
transactions and goal completions.
Now that we have selected the dimensions and metrics that we want to be
extracted from this data source,
we need to also define what date range do we want this Extract Data
to use when creating a snapshot update. Now,
we need to think about it beforehand because we need to see what kind of date
ranges do we want the viewer of the report to be able to select when they're
looking at the reports. If you want to give them the ability to, for example,
compare this year to last year,
then we need to make sure that last year's data are included in this snapshot.
That's why for the purpose of this demonstration,
I'm going to set an advanced date range and set the start
date to today minus three years.
So it has to start from two years ago and the end date would be today
minus one day, which would be yesterday. If we want,
we can add a segment or a filter as well,
but I just don't want to do that right now.
Now that we've finished with the configuration,
we need to click save and extract.
So there's Extract Data connector goes back to Google Analytics and create a
snapshot of the data the way we configured it. But before that,
I'm sure that you've noticed the auto update section at the top.
And this is a functionality that actually allows us to repeat the
extraction of the data on a daily or weekly or a monthly basis.
This is so important because as you can see right now we are extracting from
1st of January, 2017 to 15th of August.
And if we do not turn the auto updates on the snapshot that's being
created will remain a static.
So regardless of the time that the users looks at the report, maybe tomorrow,
next week or two years later,
the data in the data set would be the same.
It will not reflect any session or user that could happen
after this end date. But if we turn on the auto update,
it's will repeat the data extraction on a daily basis.
And because we've set the date range as an advanced date range three years
before today, one day before today,
then each day see a start and end date will be recalculated
and new data will be added to the, to set.
So I'm gonna set it to daily.
And I can also identify which hour of the day do I want
extraction to happen. So consider this,
let's say you're creating this report for your client that arrives at her
office 9:00 AM each day, and wants to take a look at data.
Then you might want to set the data extraction to maybe 7:00 AM,
8:00 AM. So to make sure that every day at 9:00 AM,
when your client is going to open the report, a new,
fresh set of data is available in the data set.
I'm gonna click save an extract.
And now it takes a bit of time for Data Studio to create a snapshot of the
data. And then it just disappears.
The whole user interface disappears,
which means that the extraction was successful.
I preferred and hope there was some kind of a notification that yes,
your extraction was successful. You can now use the data, but there isn't,
but if it disappears, know, that it's fine.
It's just doing what it's supposed to do.
Now let's change the data source for all of these,
to the new Extracted data set that we created.
If you take a look here, you cannot find it.
It will not be updated in this list immediately.
What you need to do is that you need to refresh your report.
Okay, now, if I select all of these, which I cannot see,
but I know on there,
I can click on the data source and in new Extract Data,
data source appears here. And when I click, it
will be applied as the data sources of all of these components.
Now this is a default name.
So a good practice is to edit your Extracted Data Sources immediately
and give them a descriptive name like GA
Denmark counts. So, you know,
what kind of data included in that extract that data source.
Okay. It's showtime,
let's go over to the view mode and see how long does it take for each
of these data sources to return data when we apply any date range?
Okay.
So let's select a completely custom date range.
So for example, let's say I want from 1st of August,
2019 to end of July,
2020 and see which one returns data quicker.
So this one was already ready and it took a bit longer to
respond. Let's try another date range.
For example, this year to date,
let's apply again, this one's ready.
And just one second after that, the live data source is ready for us.
Now, in this example,
you can see that the difference between the performance of the live data source
and the Extracted Data Source are not that huge.
It's just one or two seconds,
or maybe it was just one second of delay.
And for the purpose of this demonstration, to be honest,
I expected and I hoped the live data to be a bit more slower,
just like it usually is, but it wasn't. But in real life,
I had situations that my clients had to wait one, two,
or three minutes before the live data sets,
returned any data for them to look at.
And it was a really bad user experience. That was actually the first time.
And the reason that we decided to switch many of our client reports to extract
the data sources or Big Query tables, so they load faster.
The performance of the live data source. It depends on many factors,
the server response time, the amount of data that you're requesting,
the actual dimensions or metrics,
are there any segments applied and are there any processing that needs to be
done Google Analytics side to be able to return data.
So if you want to ensure the maximum performance and absolute
reliability, to make sure that any time that your clients load your reports,
data is there and will be loaded super quickly.
You might want to use Extract Data Source because it actually creates
this snapshot and it stores it as I've had in a Big Query table in the
backend. So it actually uses the Big Query infrastructure to store
this snapshot of the data and make it available for Data Studio.
But before we wrap up,
we need to know about the limitations of Extracted Data Sources as well.
So first of all, as you,
so we needed to select which dimensions and metrics did we want to
extract from the data source.
So only the fields that we'd select will be extracted.
Some pre-planning is required here. And then again,
only the data for the selected date range will be extracted.
We can go back and edit our Extracted Data Source and apply another date range,
but we have to do it manually if we want. Next,
it can only extract data every day, week or month.
So depending on the time of the day that you look at your reports,
it might be up to 24 hours of delay.
So data is not absolutely fresh. And finally,
the Data Extract,
the extracted snapshot cannot be more than a hundred megabytes in volume.
So if you try to extract so many dimensions and metrics altogether for
a really long date range, then, uh,
it might actually time out and give you a warning that it cannot extract that
much data. In those cases,
It's good to know that for each chart or for each group of charts,
which use the similar dimensions and metrics,
you can create another Extract Data Source. It's free.
You can create as many as you want.
That's it.
I hope you enjoyed this video and you can use what you've learned to speed up
and improve the performance of your reports. If you're new to measureschool,
make sure to subscribe so you can get notified of the future videos that we
publish. And if you have any questions, please feel free to leave a comment.
This is Ahmed. See you in the next video.