You have two option option a and b and
you don't know which option is
performing better. A B testing is a
technique that allows you to decide that.
Let me give you example. AtliQ
technologies is a software company owned
by my brother. They have this contact
button at the top right. Contact button
allows them to generate sales lead
because a website visitor. If they want
to develop a software they will click on
contact button. Now business manager in
AtliQ proposes that we should move the
contact button in the middle because
that way it will generate sales leads.
Now we live in a world of data and we
don't use intuition. So what AtliQ will
do is they will prepare an alternate
version of the website where the contact
button is in the middle
and they will run both the versions in
parallel for let's say 10 days
where the traffic is divided equally
between both the versions. Unless the
website is getting 10 000 users every
day now 5000 users are visiting version
A where the contact button is at top
right the remaining 5000 are visiting
version B where the contact button is in
the middle. At the end of the day they
calculate the sales lead lesson 10 days
version A got 300 sales lead version B
got 400 sales lead.
Now they can conclude that
version B is better because that allowed
them to generate more sales lead. Now it
is not as simple as it looks because you
need to consider number of factor. Factor
number one is is there a sampling bias.
what I mean by that is the 5000 users
who are visiting version B maybe they
were from the same area where AtliQ
already has an established name and that
would be a case of sampling bias you
need to
distribute your load randomly. So that
you know people from the random region
are visiting the participating in
version Aand B in an equal way that way
there is no bias.
You also need to make sure there are
enough samples. You can't run this
experiment for one hour and let's say
500 user visits your website
based on that you make a conclusion that
will also not work. We'll cover all of
these things in detail including
what is called statistical significance
in a separate video, but my point is you
want to make sure you are not arriving
at a conclusion just by a random chance.
Now A B testing is something that is
used by most of the website. Whenever
they make change in their layout..
for example, Amazon repetitively uses A B
testing and
I'm going to provide a link of a
research paper where they have shown how
they use A B testing to design the tabs.
You know the product category tabs in
Amazon website, Linkedin, Facebook any
website whenever they want to make a
layout change they use A B testing. A B
testing is used heavily in data science
and machine learning as well
Product recommendation on Amazon is a
classic case. Let's say you are a data
scientist working in Amazon
and you have built a machine learning
model that makes product recommendations.
So if you're visiting one product you
see some other product recommendation at
the bottom.
Now
you are deploying a new version of this
machine learning model. You won't go
ahead and just directly deploy that you
will do A B testing which means
you will deploy your algorithm on a
separate instance and you will have
percentage of your traffic going to the
new version
for some days.
It doesn't have to be 50 50 percent
maybe 70 percent traffic goes to the
current version 30 goes to the new
version then you measure the success of
the new version after certain amount of
days and again you have to take into
account the random chance and and all of
that is done by hypothesis testing, which
again will cover in a separate video. but
at the end of the day now you can make a
conclusion that
version b is better or not and based on
that you can deploy that to pretty much
all the users
Now you'd ask me
why only A B testing? Why not A B C or A B C D
testing? Well you can do it.
The contact button that we had at the
top right we can put it in the middle we
can put it in a top left and maybe run
three versions and call it A B C testing
and that's perfectly valid but A B
testing is more popular because when you
have two option
it becomes very easy to test things out
as you had more options your experiment
your experimental design gets very very
complicated. Hence a b testing till date
remains the very simple very powerful
technique and people use it
throughout the industry-
Facebook ad campaign for example, social
media ad campaign is another classical
example where
you will
run your ad campaign. Let's say I have
for example
50 000 budget for my product you know to
spend on social media campaign what I
might do is I will run maybe two ad
campaigns on facebook twenty five
thousand dollar each and
or let's say initial face ten thousand
dollar five thousand dollar each and I
will
design two separate campaigns which will
have
separate ad layout. Maybe my demographics
configuration is a little different and
by running this experiment, I can figure
out which ad campaign is performing the
best and then I can deploy rest of my
advertisement money on my ad campaign. So
it's a very powerful technique. I hope
you like this video. If you have any
question please post in a comment box
below.