you're talking about uncertainty
with such certainty
you're talking about what you don't know
and yet you're able to put numbers on
the extent to which you don't know it
which makes me think you do know it
so
the symbol is sigma
and it's something you'll just hear
thrown away frequently uh in reports
about scientific results where they're
saying basically how significant their
results are how important they are and
you'll hear people saying this is a
three sigma result or a five sigma
result or a one sigma result we've been
hearing a lot about it with regard to
the higgs and the uh maybe
evidence that's that's the higgs is
being seen at
cern at the large hadron collider in the
atlas and cms detectors
so it's for example in the context of
the large hadron collider where they're
inching towards detecting the higgs
boson they keep quoting slightly higher
values for this quantity sigma as a way
of indicating that they're getting
closer and closer to a detection of it
they won't quite say we've found it but
neither will they say we haven't found
it what they talk about is the evidence
for it being there in and they use this
kind of this symbol sigma or standard
deviation so whenever you make a
measurement
in any kind of science astronomy physics
whatever it is you make your measurement
but you should also always quote an
error bar on it because you never very
very rarely measure something precisely
usually there's some uncertainty with
your instrumentation or whatever so
really the sigma is the size of your
error bar it's how much uncertainty
there is around the measurement you
actually made imagine you've got a big
stat a stadium a football stadium full
of people
and you decide to do measure everyone's
height then you plot it you plot the
height uh the number of people have a
given height and you just you you make a
distribution of it right so that that
distribution will be peaked at some
particular height in other words
the majority of people will most people
will have this height and fewer people
will have you know they'll either be
taller or shorter there'll be some
maximum where the average is
okay so you've got your average let's
say it's six foot
and then now you can be asked can begin
to ask questions about the distribution
itself and one way you can do it is you
can say
how many people are there
within 33
of that average
on one side and 33 on the other side so
that's about 66
that's one sigma
so one sigma
is
takes up into account
sort of 66
of the data
around the mean so it's the mean plus 33
percent plus or minus 33 percent
if i go to two sigma you're then saying
now i want to think about uh around i
think two sigma is around 95 of the data
so you're including all these people and
you can count how many there are and
that those are in the two sigma regime
three sigma goes to uh 99.
uh seven percent i think it is
so that's taking you up to three sigmas
we're almost including everybody now
right but we're at three sigma
four sigma takes you to
99.99 percent okay so
that's that's surely that's everything
that's not good enough for particle
physics it's the five sigma that's the
important thing for particle physics
we'll come back to it in a second but
what five sigma means is you're
including 99.9999
of all the data so there's only like
uh one in one million
people not included in that 99.99
so that that's what sigma is it's not
very difficult it's it's kind of tells
you about the distribution around the
mean and that's all it does the mean is
the average and sick the number of sigma
tells you basically the fraction or the
percentage
of data which is accompanied is included
around that mean scientists are a bit
sloppy when they use these things
they're a bit careless and so the sigma
that you often hear quoted really should
be associated with something where
there's a particular kind of error the
most common kind of error is a thing
called a gaussian error
it's the the
gaussian distributions or a particular
form of mathematical distribution they
come up all over the place particularly
in uncertainties it's kind of the
converse
you make use of it in the converse way
in the science experiment so um
in other words you look for things that
are way out in this
at the tail of the distribution so that
means a large number of sigma it's
really these things are a measure of how
often you're prepared to be wrong in
that if you if you publish a result when
things get to this sort of 99
significance level um then that means
that well one time in 100 i'm going to
claim a result is true and actually it's
going to be false
so it's how often you're prepared to be
wrong so at one sigma it would be you
know you'd be wrong with a third of the
time and no scientist is prepared to be
put up with being wrong a third of the
time you just look stupid in this case
we're looking at where it's certain
we're looking at the collision of
protons and from those collisions let's
concentrate on one of the one of the
routes that has led to the possible
discovery of the higgs if it's there and
that is the production of photons so two
photons are being produced at the end of
the of the of this collision process
now
i have a model without a higgs
so let's have which is the standard
model without the higgs there are many
ways the protons can collide in this
model and produce two photons
and what i can do is i can end up with
what it was effectively a distribution
of the the the number of photons i
expect as a function of the energy of
the of the of the system
and now i can ask well if i include the
higgs
what what happens
and i can calculate what i expected to
happen but what i can also do is just do
the experiment i can just look for the
photons right i've got these amazing
detectors and i can just look for the
photons
and in particular what i can ask is at
any given energy which corresponds in
fact to the to the mass of a higgs
particle which we're
we're trying to test for i have some
average value that i expect there to be
for the number of photons produced
and then i have one sigma around the
average value here let's make up a
number let's say we expect to see a
thousand photons
and then one sigma would be including 66
percent
plus and minus so it would be a thousand
and whatever
that would be and then
a two sigma would be including 95
percent
and i look at the data and usually if
the data is lying within that band
then
i'm not too excited about it
it's consistent with the background
model you know you shouldn't always
expect to see an average right because
that's why you you have an average you
have some above and some below and the
average out
but every now and again maybe you'll see
something which is
up at three sigma
well three sigma is bigger than two
sigma
and then you can think oh
that's that's a bit bit higher but you
tend not to get too excited you might
think we'll keep an eye on that and
that's where we're at at the moment with
the higgs um in actual fact so the the
the data is at this kind of
2.8 sigma level
um
and and what people say those who are
waiting to you know before you call up
stockholm and and uh and book your
flights
two sigma's not uh three sigma is not
good enough in fact four sigma's not
good enough the 99.99
what they will have to see is evidence
of distribu of of events which are at
five sigma away from that average that
you would expect the model to have
without a higgs because when you're at
five sigma if you just think about it in
terms of your underlying model the model
without the higgs what you're saying is
i am seeing events occurring
where i would normally have expected
them to occur once in every i think it's
1.7 million times in other words it's
rare
but i'm built i'm finding lots of them
and that's what they'll do they'll learn
they'll aim to pick up lots not just one
but quite a few of them and when people
are quoting errors on some of these very
complicated experiments like those at
the lhc where the errors they're
measuring really aren't gaussian by a
very long way because things are
incredibly complicated there's all sorts
of sources of error systematic errors
uncertainties in the modelling and so on
almost certainly the errors are not
really gaussian but they sort of convert
it into what's the equivalent if the
errors were gaussian because that's
something that scientists sort of have
in their back of their heads that if you
know if something's significant at a one
sigma level you probably don't believe
it if it's a two sigma level you start
believing it by the time it's four or
five sigma you say that's pretty certain
so from what you've told me tell me if
this is an accurate statement then
if you're wanting to confirm something
that you already know
you want a low sigma if you're looking
for something new
you hope for a high sigma it's
effectively like that you i don't know
if you hope for it i mean you you should
just be saying well whatever the data
gives me it gives me but if you if your
theory is
you know bang on then of course all the
data that's coming in should be
perfectly consistent with what your
average expectation value is that you
should have very few
really far outliers
whereas if your theory is not the full
mccoy and is not explaining everything
that's going on in that
that regime
and there are other
important contributors to that process
then indeed you'll you as you get more
and more data
then more and more of these new events
will emerge that your theory isn't
accounting for and they'll appear in the
outlying regions of your distribution
large sigma how can you be near
near being sure about something when you
don't even know if it exists or not it's
actually i think you're looking at it
the wrong way around i think the way you
have to look at it if you're looking for
something like the higgs boson what
you're asking is how far am i away from
so you your starting position should
always be the status quo which is we've
never found a higgs boson so perhaps the
status quo is there is no higgs boson
and then you start getting more and more
evidence that actually maybe there is a
higgs boson and so you sort of start
getting further and further away from
that position of being consistent with
there not being a higgs boson so really
what you're measuring when you make a
measurement with the lhc is how far is
that measurement away from there not
actually being a higgs boson at all the
look elsewhere effect oh brady
no okay here we go
um
i might trip up on this
so what has happened here right they've
they've got the plots and um of the
events or the confidence levels versus
uh the pig's mass the range of higgs
masses
and i mentioned the one at about 126 gv
has been the the one which has got the
most significance
that's called a local
significance because you're looking at
126 gv
however
it could be that just random
fluctuations
that kind of event could have occurred
at any other energy scale it might have
occurred at 135 gv or
123 gv or
so that one itself could be an outlier
from another graph
yes i think that might be one way of
interpreting it and and so the
significance of
of assigning
that high sigma to that particular
energy scale then becomes diminished and
there's a statistical way of trying to
account for that and the look elsewhere
just means you're looking across the
band of
and the likelihood of this thing
appearing as you say as an outlier from
a different a different regime that's
yeah it's a neat way of saying it
you should be you should be here really
to day you're an astronomer not a
particle physicist as far as i know yeah
indeed do you use sigmas we use sigmas
astronomers are very bad at statistics i
have to say and so some astronomers use
statistics very sloppily i try not to
but astronomers do use sigmas but i'm
much more comfortable when you quote
something as one of these confidence
levels that you've you've measured
something
at the 90 confidence level which
basically means that you know i'm pretty
sure i've measured something but there's
a one in 10 chance i'm wrong the whole
of science is based on
in in some ways narrowing down the
uncertainties um
what people are taught when they're
coming to do physics at university and
now and at school or you should be
taught at school it's not the actual
value necessarily it's the uncertainty
in that value how confident are you
you're talking about uncertainty
with such certainty
you're talking about what you don't know
and yet you're able to put numbers on
the extent to which you don't know it
which makes me think you do know it
it's the classic thing right there are
known unknowns and there are unknown
unknowns there are known unknowns
because we know there are limitations to
our experiment we know that we only
recorded this much data and therefore
the most precision we could ever get is
this much and so on so there are those
things which are sort of we really can
quantify but then there are the unknown
unknowns which is that it's possible we
just set up our experiment wrong it's
possible that something in the
experiment we really don't understand
it's possible that there's some implicit
approximation we've made somewhere on
the line that actually has a bigger
effect than we thought it did and so
those kind of things those sort of
systematic errors where we really don't
know what they are they can always be
lurking there and so even when somebody
quotes a result as you know a six sigma
result and therefore hugely significant
if they've missed some systematic error
in there it can still go away and so
indeed we have to be a little careful
that when you quote an error and quote
something as being incredibly improbable
that you're wrong
there's always this possibility that
you've just missed something entirely in
the process