>> KEITH: Good morning. I’m Keith Washington from the Ads Marketing team,
and I’m being joined today by my colleague, Amaya Garbayo, from the Google Media Lab team.
Welcome, Amaya.
>> AMAYA: Hi, Keith. It’s great to be here.
>> KEITH: Amaya, I’m excited to hear your insights on Google’s approach to brand measurement,
but first, tell me about the Media Lab team. Who’s on that team, and what do you do there?
>> AMAYA: We in the Media Lab come from diverse backgrounds: agencies, publishers, and client side
for advertisers like Coca-Cola, Kellogg, Bank of America, and many more.
And we are responsible for investing Google’s Marketing media dollars.
In particular, I work measuring the brand effectiveness of those dollars,
everything from campaign measurement to advanced analytics.
>> KEITH: I love that. But tell me, how’s that different from what other marketing teams
at other companies are doing?
>> AMAYA: In many other companies, the measurement and optimization department does just that: measuring.
But in the Google Media Lab, we use data to inform many other aspects of the media process,
from establishing media planning minimums to evaluating creative effectiveness
>> KEITH: Sounds invaluable.
What are some guiding principles your team uses that marketers watching today can take away with them?
>> AMAYA: There are 4 main strategies and tools that my team uses
to provide Google with the best data possible to drive the desired outcomes with our dollars.
First, prelaunch: build the best plan before we spend any money.
Second, during the campaign: we are tweaking the plan in market to maximize outcomes.
Then, the post campaign analysis, which evaluates what did we get for our money?
And then finally, we prepare Google for the future to come, doing in-depth channel analysis
or piloting new approaches to measure in a world without cookies.
>> KEITH: So when you’re in this pre launch phase, what are some of the things you take into account?
>> AMAYA: We look at 2 things:
one, the ability of the creative to deliver success in-market;
second, if the media weight is sufficient to communicate the message.
>> KEITH: And how does your team measure and achieve creative excellence?
>> AMAYA: Google tests in lab the vast majority of the creatives that are used in our media campaigns.
The program evaluates creative effectiveness is global, involves various research partners,
and is executed with the same rigor and methodology across all brands.
The creative testing program has a strong ability to predict the performance that we will see in market,
and hence, all our brands use the data to optimize creative
and ensure Google gets the best return on investment for our advertising money.
Our goal is to have 75% of the creative to be optimized,
and then use the 25% remaining for creative experimentation.
In the chart, you can see the progress made towards that goal in the last few years across all Google brands.
>> KEITH: Wow. And how does the program help improve over time?
>> AMAYA: The program provides not only scores to determine the best creatives to lift brand metrics,
but also guidance and best practices for our creative teams.
The best practices are tailored to the KPI that we want to impact.
For example, supersize all logos and text is very, very important for awareness,
but only somewhat important for purchase intent.
On the other hand, using close-up frames of the product is important across all KPIs
in the funnel to deliver success.
I have an example of how the program has helped improve results.
Something as simple as identifying the brand in the opening frame can increase awareness by 2 times.
Let’s take a look.
>> 2 CHAINZ: Let’s get one.
>> AWKWAFINA: Oh, yeah.
>> 2 CHAINZ: Ugh! Every time I wanna get a … I ran out of storage.
>> AWKWAFINA: Yesterday I had to delete a picture of my grandma.
Why? So I could take a picture of my grandma!
>> 2 CHAINZ: Now that’s cold.
>> AWKWAFINA: It’s ice cold. I just want a phone that gets me. You know?
>> 2 CHAINZ: Knows my schedule.
>> AWKWAFINA: Knows my taste.
>> 2 CHAINZ: Knows my music.
>> AWKWAFINA: I’m over the hype.
>> 2 CHAINZ: Super over it. Let’s say goodbye to this thing.
>> AWKWAFINA: How are you gonna do that?
Oh, are you just gonna—
Oh.
[Say goodbye to unhelpful phones.] Bye, phone!
[Starting at just $399] >> 2 CHAINZ: OK, I see what you're doing, Google.
>> AMAYA: Here is another example of the impact of the creative testing program
and how it helps tailor assets to the different video channels to maximize in-market effectiveness.
Television uses a reveal format, while online video works better with up-front branding,
showcasing the product up front to account for any skipping behavior.
Once each asset of the campaign has been tested,
the brand managers will get an email with an assessment of the likelihood of success for that campaign.
The final rounds of decisions are made based on this data to ensure we maximize the ROI of our dollars.
>> KEITH: Okay, so we’ve covered creative.
But what about media planning? That’s got to have a pretty big impact too, no?
>> AMAYA: You are a step ahead of me.
Google Media Lab has a tool called the pre-flight check that scores every campaign
and determines if it complies with best practices
and if it can be successful at delivering brand lift in market.
If the campaign fails like you saw in the first chart, agencies and internal teams
will get to work to tweak the campaign and ensure a passing check before launch.
Lastly, we periodically verify the signal quality of our programs.
If we have done our job well, campaigns that comply with both programs should get more success
in market than the ones that do not.
The chart that you see split campaigns using the 2 dimensions discussed:
creative quality on the vertical axis and media sufficiency on the horizontal axis.
Campaigns that have both passing creative and sufficient media weight are effective and efficient,
here in the upper right-hand corner, while campaigns with neither, in the bottom left,
struggle to deliver positive results. And even when they do, they are less efficient.
But perhaps more interesting is the fact that media weight is crucial for effectiveness
(bottom right corner), but creative is more important for efficiency.
Understanding the nuances in this chart helps brands make trade-offs
between effectiveness and efficient, when needed.
The biggest accomplishment of the creative testing and the pre-flight check programs
is that they help Google marketing maximize the chances of success before we spend a single penny.
Once the campaign is live, optimizations come into play, again to ensure we maximize outcomes.
>> KEITH: So, you’re able to make immediate changes in real time during the campaign? How does that work?
>> AMAYA: As we discussed at length in the previous section, the creative results heavily influence
creative selection before the campaign starts.
But even during the campaign, those results continue to be valuable to
upweight and downweight investment across all channels.
Similarly, an early read across various platforms will impact budget shifts
towards channels delivering better outcomes.
For example, in this campaign, the early success of YouTube
triggered shifts in budget to capitalize on the success.
>> KEITH: That sounds like a lot of moving parts. How do you ensure your team is able to keep up?
>> AMAYA: In order to make all these optimizations in a systematic manner, and as fast as possible,
the Media Lab has developed decision trees.
>> KEITH: Decision trees. Such a great term.
Would you please explain what decision trees are for the folks who may not be familiar?
>> AMAYA: The decision trees contemplate every possible outcome of all metrics we track in market
and offer a blueprint of actions and approval protocols that will make sure we make decisions swiftly.
>> KEITH: What happens after everything is said and done?
>> AMAYA: Google evaluates the outcomes and answers the question, “What did we get for our money?”
First, we need to understand “what happened?”
against our primary KPI and the goals we established before the campaign launch.
Second, we ensure we discuss only 3 material stories
that have deep implications for both the strategies and tactics.
Then, we determine a series of actions, the stakeholders responsible,
and the impact of those actions on upcoming campaigns, both short and long term.
>> KEITH: This all sounds pretty powerful. But how do you account for things like evolving
consumer expectations, concerns about privacy, and more people than ever shopping from home?
>> AMAYA: With an eye towards the future,
Google measurement also conducts meta-analysis across all campaigns, channel deep dives, does methodology reviews,
and monitors the shifts in consumer viewing habits, preparing Google marketing for what is to come.
In this particular example, we have analyzed the historical investment of our very own YouTube
and determined a few variables critical to drive success.
We utilized classification tree modeling, which is a supervised machine learning model
that helps us explore the structure of a data set while developing easy-to-visualize decision rules
for predicting the outcome variable, in this case, Brand Lift.
>> KEITH: Does the analysis reveal anything special about YouTube?
>> AMAYA: The analysis confirmed some of the known important variables like campaign length and frequency,
but also revealed endemic-to-the-platform elements
like building YouTube creative that are critical for success.
In addition to the improvements that were recapped, the analysis also revealed how a new YouTube product,
video ad sequencing, helped deliver success on the platform.
Video ad sequencing allows marketers to ensure that individual consumers will see a series of ads
in a specific order and the right number of times to help a full understanding
of complex messages within a given time period.
This allows marketers to do incredible tailoring with tons of flexibility,
which is unmatched by any other video channel in the industry.
For our brands, the campaigns that included VAS were 2 times more effective than the ones that did not.
>> KEITH: Oh, okay. I see how the combination of multiple formats builds frequency
without annoying consumers like myself. But can you give me an example of what that really looks like?
>> AMAYA: Of course.
A typical video ad sequence might open with the longer 30-second format early in the week
follow with an unskippable 15-second on Thursday, and close with a 6-second reminder on Sunday.
Video ad sequencing can be used with a single concept and various ad lengths, as the example we are showing.
Or you can introduce additional concept and have longer series.
The possibilities are endless, and video ad sequencing can be used to address many marketing challenges.
And, oh, by the way, in addition to better brand metrics, it increases consumers’ engagement with the ad
and is also more efficient than the standard buys.
>> KEITH: Got it. So with all this data, how do you keep all the senior stakeholders informed?
>> AMAYA: We have a scorecard that summarizes all that with only 6 metrics.
The metrics have been standardized and they are blessed globally to represent 100% of the investment.
We use 6 metrics to paint a full picture of the entire budget.
That gets delivered to our CMO and our most-senior leaders every quarter.
There are 3 metrics that we evaluate before the campaign starts:
the budget, the percent of budget behind past creative, and the percent of budget passing pre-flight check.
And then, we look at 3 metrics after the campaign concludes:
the percent of budget that lifted the primary KPI;
1 effectiveness metric, using the numbers of individuals lifted;
and 1 efficiency metric, the cost per individual lifted.
>> KEITH: Thanks, Amaya, for spending the time with us today, and thanks to all of you for watching.
I hope it was helpful to understand how Google measures brand
in order to drive a more effective and efficient marketing investment.