PETE FRISELLA: Hi, everybody.
My name's Pete Frisella.
I'm a developer advocate on the Google Analytics team.
I'd like to introduce you to a new series we have called "Off
the Charts." This series is about getting into the deep
features of Google Analytics, understanding how it works,
things you can do with it, and how to use the feature itself.
So today, I'd like actually talk to you about getting
started with cost data upload, and this is all about
measuring non-Google paid campaigns.
So let's talk about what is cost data upload.
I'll kind of get into little bit more about what that means
and what cost data upload works, how it works and the
feature itself.
Then we'll go into the architectural overview, which
explains the deep details and also how you would actually go
about uploading data to Google Analytics.
And then finally, we'll talk a little bit about some
opportunities that are available from a developer
perspective and also from a business perspective.
So with that, we'll get started.
So what is cost data upload, and what does
cost data itself mean?
So this is all related to all the paid campaigns you may be
running out there, all the information that's related to
that paid campaign.
Whether it be ad costs, clicks, impressions, the name
of the campaign, all this itself relates to cost data.
It's what we refer to as cost data in Google Analytics.
So for example, if you are running a campaign, say with
Google AdWords, for example, you'll know this is pretty
familiar for you.
You've got cost data, and if you link that to Google
Analytics, which is what you should be doing, you actually
get these reports within Google Analytics that
highlight, for your paid campaigns, how they're
performing.
And they give you a lot more information around how to
optimize certain key words, maybe landing page
optimization.
Now, these things are important things, but to
really do this analysis, you need that cost data associated
with the campaigns that you're running in Google AdWords.
And this linking works really well today, and it takes care
of all that for you.
And it gives you some powerful tools to take care of analysis
around campaigns and user interaction, and how campaigns
are performing.
The thing is, though, for a lot of people, why we
encourage you to use Google AdWords, we love Google
AdWords, obviously.
There is other paid campaign services out there that people
obviously use, and that includes things like other
paid search services.
You might be running in a campaign on a social network.
You might be doing email marketing campaigns.
So all of these campaigns are running out there, and you're
getting users visiting your website or your property from
these paid campaigns, and that's great.
You're able to track what they're doing, the conversions
possibly that are taking place, goal completions.
And you're getting a good general sense of which paid
campaigns are driving traffic to your site.
This is great.
So a report like this, that I'm showing here right now, is
kind of common that you're looking at, for my particular
campaigns, how are they performing and what kind of
traffic is coming to my site because these campaigns?
Now, you see that first row there, row number one, is a
Google AdWords campaign.
And you'll notice while you have user interaction data
such as visits, and you have impressions and clicks cost
data, you don't get the same kind of visual in cost data
that you would have for AdWords.
You don't get the same data for the other paid campaigns
that you may have.
So for example, we have an AdWords network campaign
running, and we have an affiliate
network campaign running.
And you'll see, we know people are coming to the site, but we
don't have the cost data within GA to do that analysis.
There's kind of two things missing around this then.
For the particular paid campaign itself, being able to
analyze that in terms of the ROI and the metrics you might
be getting.
Click-through rate, cost per click, things like that, you
kind of miss out on that.
And then also, you don't really have the ability to
compare how these services are performing against each other.
So is the AdWords network driving more?
Better conversions, or is Google AdWords driving better
conversions?
So you're missing that kind of complete view, that complete
analysis, around your cost data across all your different
paid campaigns that you might be having for all online
advertising and digital marketing.
But cost data kind of solves that problem.
When we launched this feature about a month ago, we had some
success stories right off the bat with a couple customers,
working with partners of ours to get something in place and
a solution for them.
So, for example, we talked Next Analytics and Cardinal
Path, and they came up with a solution that
uses cost data upload.
And they worked with Natural Wellbeing.
And really, it was around fully automating this process
for them, so being able to take their cost data
automatically, export it, and upload it into Google
Analytics using this new feature.
And it worked really well for them in saving them a lot of
time and doing this full automation.
We also did the same thing with ShufflePoint and E-Nor.
They came up with a solution using cost data upload again
to help OEMPCWorld.
And this one was really about, again, fully automation.
And a lot of the analysis they were doing was kind of a
manual process where you had your Google AdWords data, you
had some other campaigns.
You had to manually bring that stuff together and do analysis
outside of GA.
So this new feature really allowed them to bring all that
into Google Analytics and to do that complete analysis
within the product itself.
So it saved them a lot of time, fully automated, a great
solution for them.
So I think it would be good just to start off with a quick
demo to show you how this actually works.
And then we're going to dive into details, and I can give
you a little bit more background on how these actual
steps that involved to do this.
Sometimes, it's just easier to show you how it works, so I'll
start off with a report.
This is within Google Analytics.
It's a custom report, and the example here is that I have a
paid search ad running, and it's for an ad network.
It's a CPC, and it's for the keyword
"dogbone." And I can see--
so I've tagged my URLs, and I'll get into that
a little bit later.
So I know that someone's clicked on the URL, and
they've visited my property, and I can see that there's
been two visits.
But just as how I described before, we don't have
any cost data here.
But the thing is, I actually do have the cost data.
The service provider that I'm using allows me to
export the cost data.
I just don't have it in GA, so that's really
what I want to do.
I want to have that same data available in GA so I can do
analysis and can carry out cost analysis.
So just something to note here, the source is Ad
Network, the medium is CPC, and the keyword is "dogbone."
So as I mentioned, I actually do have this cost data.
So you can imagine that I exported this from the paid
search provider and have it in CSV file.
And there's a header up here that has some dimensions that
I'll explain later.
But in row two, we can see the same values that I just showed
you in report are actually here.
We have Ad Network as a source, CPC is our medium, and
we have this keyword here, "dogbone."
And there's that cost data I was talking about.
We have impressions, we have clicks, and we have ad cost,
so I have this data.
It's there.
It's available, and there was a couple other
keywords that I ran also.
Now, let's actually upload it and see how this process
works, and then we'll see--
once it's available in the report, you'll see how it's
actually joined together.
So I have a little script.
It's a Python script, just to kind of test things and see
how it's working, how the feature works.
I'll start off with--
there's this new entity called custom data source.
I can also explain what this means in a little bit, but
basically, you upload the cost data to a custom data source,
and you specify which date you'd like to upload
the cost data for.
So, in this example, this cost data actually is from
yesterday, so I exported all of the cost data and metrics
from yesterday into a CSV file, and it's
available to upload.
So I'm going to select the custom data source that I want
to upload to.
And I'll show you how this process works.
So I have quite a few accounts here.
I want to upload to the My Store account.
I will select the web property, and then here's what
I'm talking about in terms of custom data sources.
I have three set up, and I'm just going to choose the first
one, the Ad Network custom data source.
So now I've selected it.
If I go back up, you'll see now that it's actually
selected, here, Ad Network, and then also the account and
the web property.
And then the upload data again is 2012, November 28.
So I will do an upload of cost data, which is
number three here.
I know the file that I showed you, what the name of it is.
Oops.
Spelled that wrong.
OK, so it's actually uploading right now to Google Analytics
to that customer data source, and if all goes successful,
we'll get a response back from the server
saying it was a success.
So it takes a little bit of time to process the data.
It's not going to show up instantly.
I will go back to the presentation.
It could take anywhere from a few minutes to up to 12 hours,
so you should see data within 12 hours.
Depending on the kind of load and your profile and web
property, it could take a little longer.
I'm going to actually go back to the presentation.
We've successfully uploaded cost data.
I'll come back a little bit later, and we'll see if it's
available in the report and if it's been joined
to our actual campaign.
In the meantime, let's talk about architectural overview,
how this feature works, and how you would actually go
about uploading cost data.
So there's quite a few steps involved, but they're all
pretty simple.
The first four are really around how to prepare for a
successful upload.
And the fifth is really around reporting and how you would do
analysis, the tools that are available for that within GA.
We'll start with tagging campaign URLS.
So obviously, the primary benefit to this feature is
that you're able to join that user interaction data that's
already in GA that you have, and you're able to join that
with the cost data that might sit somewhere externally.
So bringing those two together is really a huge benefit here.
But in order to do that, it's important that you use
campaign tracking parameters, so you should
be doing this anyways.
But the important part of this is that the campaign tracking
parameters, the values you use as part of the campaign,
should or have to match the values that you're uploading
through cost data upload feature.
So I'll explain a little bit more in detail what this
means, but basically, they have to match, and if they
match, then they'll actually get joined
together in that case.
So in this example here, I have an ad running.
It's not a Google ad.
It's another paid search provider.
The link there actually has UTM parameters as part of it,
these campaign tracking parameters, and I've specified
an ad network, CPC as a medium, the campaign is summer
sale, and again the term is "dogbone." So you've seen
these values throughout, and we'll kind of explain how this
all works now.
OK, so you've tagged your URLs.
All your destination URLs for all your paid
campaigns are tagged.
And again, this is not for Google AdWords.
Google AdWords has a different mechanism for
uploading cost data.
You just link your profile to your Analytics account, and
it'll take care of itself.
This is for non-Google paid campaigns, just something to
keep in mind.
But once you've tagged those URLs, the next thing you want
to do is actually get the data out of the external source and
prepare it as a CSV file to upload, so let's take a look
at what that means.
The first thing is what can you actually upload.
So there's quite a few different dimensions and three
metrics that you can provide values for
through cost data upload.
There's actually nothing new here in terms of dimensions
and metrics.
So these use existing AdWords dimensions and metrics that
you've seen and are probably familiar with, so
it's nothing new there.
So if you have any reports set up, like custom reports around
performance of paid campaigns, those don't have to be changed
once you start uploading cost data.
Any zero values that you had before will just automatically
be populated with this data that you've uploaded.
For the dimensions, there are two that are required, which
are source and medium.
And then for metrics, there's three of them,
and only one is required.
One of the three is required.
For any ones that are omitted, we'll just assume it's a value
of zero basically for those.
So this is the important part here.
We talked about the tagging of the URLs, and
this is why it's important.
This is how the data actually gets
joined and mapped together.
So you have all your user interaction data sitting in GA
already today, and you're doing that through normal
measurement that you've already done all the time, so
that's taken care of.
Now, you're uploading cost data.
And what's going to happen is the campaign tracking
parameters, the values you use there and the ones you upload,
if there's a match there, then the data
will get joined together.
This is why it's important that these things are synced
across the two.
Also, there is a specific case in terms of the case that is
required for these things.
The keyword is case insensitive.
So it doesn't matter if the keyword is capitalized or not,
they'll get joined regardless.
But the other four dimensions are case sensitive, so you've
got to be careful there and make sure these values are
consistent in that regard.
Also, it's up to you really to determine the naming
conventions that are used in the external system.
It's quite possible that the naming convention used by this
external source might be a little bit different and not
as matched up with the namings that we use for our
dimensions within GA.
So it's kind of up to you to look at what's available from
the external provider and map and see how that maps these
dimensions and the values you need to provide for these
dimensions.
It's kind of up to you to do that.
And usually, it's a one-step kind of process once you've
got it setup.
Then you don't have to really worry about it anymore, but
identifying those mappings at first is really important.
OK, so we know what we can upload.
Let's take a look at actually what you would upload.
So I showed you a CSV file before.
Here's another example.
So this is for aggregates, right?
You're uploading aggregate data on a daily basis.
That's usually the typical use case.
So best practice then is really to wait until the end
of the day and then upload cost data.
Or upload cost data for the previous day because you want
to upload it in an aggregate, and you upload it for a
specific date.
You say I'm going to upload it for yesterday or the day
before and so on.
Again, you'll notice here that the values I have for the
source and medium match the values that I use in my URLs
for my campaigns as part of the campaign tracking
parameters, and those should always be consistent.
And then you have your actual metrics here, the cost data
itself, the compressions, ads, and clicks.
In this case, I provided values for all three because I
had them available.
And then, of course, you saw the list of all the dimensions
that are available.
Those can be included also.
There is quite a few different constraints and requirements
around what needs to be provided and the values you
can provide.
These are all available on our developer site, which I
provide some links later for you guys.
OK, so we've tagged our campaign URLs with campaign
tracking parameters.
We've got our CSV file that we exported out from the external
source, and we've prepared it properly.
And now we have to create a data source.
Remember, we have to upload to a custom data source.
We have to create that, and that's actually done through
the web interface.
OK, so I'll show you how it actually is done right now,
and then I'll explain how exactly that this works.
So it's at the web property level.
And we're currently doing white listing, so some of you
may have this, some of you may not, but it will be all rolled
out to everybody in the next week or so I think.
So if you go to the web property level in the admin
panel, you'll see a tab called Custom Definitions.
So I already have three custom data sources
available in here.
I'm going to create a new one, so I'll show
you how it's done.
So a new custom data source, You can imagine just, say, I
want to have one for, let's say, email marketing, so I'll
put email marketing campaigns, specify the type.
We're uplinking cost data.
That's the type.
And then we link your profile.
Linking profiles will enable this cost data to be available
for those profiles.
You can imagine you might have 100 profiles here.
You can select which ones you want.
There's a little search box to make it a little bit easier.
And then you just select which profiles you would like the
cost data to show up in.
Click Create, and you'll see now I have a new custom data
source for email marketing campaigns.
And the important thing here is there's an ID, and you
actually need that ID to do uploads, and I'll show you how
that's done.
I'm going to go back to the presentation.
So let's look at what's a custom data source, and how
does it relate to Google Analytics model?
I think you're probably mostly familiar with this.
This is the way GA is set up today.
We have Web Properties.
That's where all the user interaction data is collected.
That's the level it's collected at.
And then we have profiles, and profiles basically act as
views of that data.
So you can have one maybe for marketing.
You might have one for sales, or certain filters apply that
you want to have different views of your web property
user interaction data.
What's been introduced with cost data upload is custom
data sources.
So this resource kind of acts as representing
external data sources.
You could have a single or multiple external data sources
per custom data source.
It's up to you.
You can have multiple custom data
sources on your web property.
I would say, as a best practice, I would create a
custom data source per external data source.
It's just a little bit easier to manage, and it allows you
to kind of separate data from across the different sources
that you might have.
It's really important that you link a profile.
So this, as I showed you before, when you link
profiles, it enables that data to be available
within those reports.
If you don't have that linking, then the cost data
will not show up in reports.
So in this case, we have an example here where we've
created a custom data source, and it's linked to two
profiles, profile one and profile three.
So those two will show data, and profile two will not show
any cost data from that custom data source.
In addition to that, we have what we call
daily upload resources.
These kind of represent the actual data you've uploaded to
GA on a per-date basis.
So you have one for each day, a resource,
and these are important.
They store information around which custom data source they
belong to and also metadata around recent changes that
have taken place with this particular resource.
So if someone uploaded to it, deleted it, you get that kind
of changelog history.
It's part of that resource.
You manage and create custom data sources within the UI,
and you actually upload cost data using an API.
You can actually also look at the history of a daily upload
resource within the UI.
I'll show you that, by going to the history.
There's a little history link here.
You can kind of get it.
For that custom data source, you can see all the changes
that have taking place.
And you can actually get-- per day, you can see all the
changes that have taken place.
OK, also from a creation standpoint, in deletion, you
need to be an admin.
So admins, anything with modification to a resource,
whether it's create or delete, is admin.
And viewers, just regular users, can view data, but they
can't do anything around creating
or deleting or uploading.
So that's important.
OK, so we showed you how to create a data
source, so let's recap.
We've got our URLs tagged with campaign tracking parameters.
We've got our CSV file that's prepared and which was
exported out from the external source.
We've created a custom data source
using the web interface.
Now, we actually want to upload the cost data, and we
use this API to carry this out.
So here's an example in Python.
It's pretty simple, actually.
You create a media file upload object, and you
specify the CSV file.
And then you actually just call the upload method for the
daily uploads resources.
And you specify a few parameters that are required,
and you execute that.
And it'll automatically upload the file for you and respond
with a 200 if successful.
If there was an error, typically--
I think when people first start out with this, they get
errors around the file formatting for the CSV.
The errors are really descriptive, so they really
help you out in terms of figuring out what
the problems are.
There's also a few other operations you can carry out.
You can list which daily uploads.
This gives you your history and changes.
You can look up for a year of history.
You can look at what's taken place.
And we also have a delete method.
So if you want to get rid of data completely, you can
actually delete it out of the daily resource.
And this is important because the way the cost data is
stored within GA, it's actually stored separately
outside of your user interaction data.
So this is great because you can kind of upload--
If you?
Want to delete something, it doesn't actually touch any of
your user interaction data.
So feel free to upload and delete without any worrying
about corrupting or having any effect on your user
interaction data that's already there.
All right, so once you've kind of done that, and you've
actually successfully uploaded cost data, and it's been
processed, you'll have the data
available in your reports.
So going back to the start of our presentation, now, the
magic's happened.
Before, we had zero values for the cost data.
And you can see now when you've done it properly, this
actual cost data will get joined.
In reports that you would normally see zeroes now show
you the campaign visits are now joined and associated with
that uploaded cost data, which gives you all these nice other
additional metrics, calculated metrics, around margin, ROI,
and things like that.
So now you can do your complete ROI analysis across
all of your different paid campaigns.
Reporting is available a few different ways.
We have the API, of course.
You can continue to use the core report in API to pull
metrics out.
Like I said before, the dimensions and measures
haven't changed, so any reports you might have been
running before will work just as they did before.
There's no changes to be made.
And then from the web interface, there's kind of two
main options, which are you can build a custom report or
there's a new cost analysis report under Traffic Sources.
The cost analysis report, this is what it looks like.
It's actually just for cost data upload.
It was created for that.
And then for custom reports, let's just see if our data
that I had uploaded has been processed.
So let's refresh this and, yes, you can see.
That was the cost data we uploaded at the start of the
presentation.
It has been joined to my campaign visits for that
particular keyword, and we can now see
impressions, clicks, and cost.
And of course, you could do--
there's some calculated metrics, like cost per click,
and you can add those to your custom report.
So great, it worked.
So some of the opportunities are, for developers, you have
an opportunity to build a connector.
There's kind of two steps to this, I would say.
There's the initial setup, which is kind of just creating
custom data source, but then there's two cases that you
really got to consider.
One is historical upload.
You'll notice when I did the upload, I
uploaded to actually yesterday.
It was data for the 28th, and you can actually upload data
all the way back to 2005.
So if you've been tagging your campaign URLs over the last
seven years, you can actually go back and export the data
and upload that.
It's a little bit different use case, because typically
when you export, you're going to export a large chunk of
data from your system, and then you have to individually
upload that for each day.
So that initial historic upload is a little bit
different use case, but the API really allows you to
automate all that.
And then going forward, there's an opportunity to
build a connector that pulls the data down from the
external source automatically and uploads it to GA.
If you don't really have the development resources or you'd
just like to get started, we do have four partners that
have built integrations using this feature.
Next Analytics, GA Data Uploader, Analytics Canvas,
and ShufflePoint all have great integrations.
And depending on your requirements, I think any one
of them will be able to solve a lot of different use cases.
So I'd definitely encourage you to check these out in our
App Gallery.
So to finish up here, we have just a few resources for you.
We'll include these links at the bottom of the video for
you, but all of this is available on our developer
site, which is developers.googl
e.com/analytics.
And we also have the App Gallery, which is
google.com/analytics/apps.
All right, thank you very much, and
you can start uploading.
I think if we have some questions,
we'll take some questions.
MALE SPEAKER: All right, so taking a
quick look at the Moderator.
We have a few questions.
One of them is from John, who says he's played around a
little bit with uploading cost data, and his campaigns don't
seem to be matching up.
Are the UTM parameters case sensitive?
PETE FRISELLA: Yeah, I think I touched on this earlier.
So the keyword is normalized to lowercase no matter what.
So that one, you don't even have to worry about case.
But for the other ones, you do have to worry about case
sensitivity.
Those will affect what gets matched up.
But again, because the data's stored separately, go ahead,
upload things, and you'll see if you have a problem, you
simply just delete the data.
It doesn't affect any of your user interaction data, so
there's no problem there.
MALE SPEAKER: And Andrew asks, "Are we going to be able to
apply advanced segments to uploaded cost data?"
PETE FRISELLA: So because you're uploading aggregate
totals for a day, it's not on a session basis.
So you can't actually do the advanced
segments like you would.
It actually acts exactly the same way that AdWords does
today, which is advanced segments would not apply for
this particular case.
All right, so thanks very much for taking the time and
joining me today about learning
about cost data upload.
I'll see you guys next time.
[MUSIC PLAYING]