Hi everyone, I'm Krista Seiden,
Analytics Advocate at Google, and today I'm joined by
John Mesh, who is a product manager
on Google Optimize. In this quick tip video
Jon is going to walk us through an
overview of reporting in Optimized;
specifically how to read and understand reports
in Google Optimize.
So with that, I'm going to go ahead and
hand it over to Jon.
Thanks Krista.
So as you all know when you're running
an a/b test one of the most critical
things is to look at how the performance
of those different variants are
to make sure that you're picking out
the right one in the future.
So let's go ahead and look at one particular test here.
Now this test is running on the
Google Merchandise store; this is where you can
buy Google t-shirts and such.
And I just want to give you a quick fast overview
of what's happening here.
Now this is a redirect test and so users who
come to this original page,
the shop.googlemerchandisestore.com,
may be redirected to this YouTube brand page,
and to look a little bit more about how this is set up
let's look at the Targeting.
So people who come in from youtube.com are being
sent over to that redirected youtube
brand page, and obviously what we're
trying to do here is decrease the
number of balances and try to increase
purchases trying to make sure that
people are seeing that relevant content more quickly.
So let's go on over to the reporting page here.
Now in a previous video we covered a little bit about the
header but of course you can always go
back and forth between the details and
reporting or back to all of your experiments,
but within this page I want to focus on a
few specific things.
First let's cover this right side panel here real quick.
Obviously it's got a little bit of
information about the experiment but a
couple of key things to point out here,
one is this ID. This is the ID that
sent in from the web page to Google Analytics,
and this is also what you can use in
Advanced Segmentation,
you can also then go directly into
Google Analytics from this link right here
if you want to take a look at those points there.
And we'll cover that a little bit more later Quick Tip video.
But on to the main part of this report so you can
see we've got a few different cards here
within this page and what we've really
tried to do with Optimize is to make it
super simple at the top of the page
to see what's happening with your test even
if you don't really know much about statistics.
So you can see right here we're telling this
particular user of this test, you have results, it's now safe
to stop your experiment.
There's also no clear leader so you need to do a little
bit more deep diving, and then you can
also see that we've got some sort of
traffic overview of how many sessions
there were on any given day.
Now already I'm looking at a little bit of a warning
here because I'm saying there was kind
of a drop-off it so I might want to talk
to my team and understand a little bit
more about why there was such a drop there.
So that's the quick overview and
what we really do there is try to make
it again super simple, and we use a few
different rules in that logic to help
you understand what's happening there.
So we do things like suggest a two week
minimum runtime, we actually use some
advanced Bayesian analysis to understand
how likely it is that you might find
more value from your tests,
and things like that.
So let's move on down the page
a little bit here. So this next card here
is our improvement overview.
You can see on the left has all the variants
and then we have a couple of metrics that
show up here, bounces and then that part
is completed; and those were two
particular objectives that I set up
before the test. And you can see a couple
things here, obviously the original is
the baseline and so we're comparing to
that baseline, and you can see that this
YouTube brand page for bounces is
actually being reduced, which is a good
thing of course for bounces.
It's dropped by about thirty two percent,
but I do want to call your attention to this
little interval underneath,
and that tells you exactly, sort of the
opening of that likely improvement range.
We call that the credible interval and a
bayesian world and that tells you how
likely it could be that the improvement
falls in that range.
So it's important to pay attention to that.
So I've got obviously these two
different goals here this is my primary
objective and that's what this first
card is based off of, but I can also go
and add other objectives at least in Optimize 360.
So in this particular case
I want to go ahead and add one more
objective here of people who browsed
more than five pages because I'm really
interested in seeing how users behavior
was affected by this test,
and you can see that the median improvement here is
actually it's dropped by about 20 percent.
Now that's really interesting in
this particular case because that might
mean that they're actually finding what
they're looking for faster,
or am I just being that they're a little less
interested; hopefully it's the the former there.
So what we're actually going to do here
is go a little bit farther down the page
and look at this
detail card, and I'm going to look at
that objective that I was just showing you;
those gold two completions of people
who browsed more than five pages.
And this is telling me a lot more
detailed information here.
It tells me the
conversion rate of people who actually
completed that goal in each session,
it tells me the difference in the
improvement, and that's the same number
that you might see above. It also tells
me the probability of these things to
either beat the original or to be best overall.
So in this particular case it
looks like that original is still best
overall at about 91 percent probability.
Finally of course in the
bottom here you can actually look at
these conversion rates over time.
So in this particular case again you can see
there's something that kind of happened
here around December 6th and so we might
want to go and talk to others on the
team or do a little bit more analysis GA
to understand exactly what happened.
But that's just a real quick overview
of the reporting within Google Optimize.
Great. Thanks Jon, and stay tuned to future
Quick Tip videos where we'll go into
more detail about how to read and
understand these Optimize Reports in
Google Analytics as well as more
advanced statistics overview.
So thank you very much.