When a physicist says the word “information,” what do they mean?
Does it have anything to do with digital information?
Let’s dive real deep into this rabbit hole.
Hey Crazies.
I’ve been meaning to do a video on information for a while now.
It’s tricky though because the word “information” means different things to different people.
This is a much deeper question than it sounds.
It demands a longer video!
Over on my speed of light video, I said light was the fastest thing
and some of you had some complaints.
Abhijit pointed out there are distant parts of the universe moving away from us
faster than the speed of light.
I was also having a conversation with Przemyslaw in the comments
and it went a little something like this:
If all motion is relative, can’t I say stars and galaxies are moving around me faster than the speed of light?
No, they’re moving so slow, you can barely tell they’re moving at all.
Yeah, but they’re really far away, so isn't that still a lot of motion for them?
Well... uh...
The conversation made me realize this needed some serious clarification.
First of all, he’s correct if we look out at some distant star in space.
No, screw that, let’s be more specific.
There’s this star in the shoulder of Orion called Betelgeuse that’s pretty bright.
Technically, it’s pronounced bet-el-jaws-ah, but no one ever says it right.
If we watch Betelgeuse in the sky for, say, 6 hours,
It’ll sweep out a 90 degree angle, which doesn’t seem very far in the sky.
But, if we take into consideration that Betelgeuse is hundreds of light years away,
then the distance it appears to travel is for lack of a better word, astronomical.
That distance divided by the 6 hours is most definitely larger than the speed of light.
You could say something like:
But Betelgeuse didn’t really move in that 6 hours.
The Earth just rotated.
And you’d be right, but so is the person that says Betelgeuse moved.
Remember, relativity says all perspectives are on equal footing.
Neither one of these perspectives is wrong.
Betelgeuse is either not moving or moving faster than light and that's fine.
Wait wait, how is it fine that Betelgeuse is traveling faster than light?
Because information still isn’t traveling faster than light.
When we do physics on paper, we can draw the picture from several points of view.
It can give us this impression that we’re omnipresent or something.
But we have to remember that we are always somehow part of that picture.
We have to take those measurements we’re making predictions about.
The only reason we even know anything about Betelgeuse
is because it emits light that takes hundreds of years to get to Earth.
The fact that it looks like it’s moving faster than light to us
doesn’t matter because it’s always doing this hundreds of light years away.
The only interactions we have with Betelgeuse are those traveling photons,
which go at exactly the speed of light from everyone’s point of view.
Oh, photons and information are the same thing?
Why didn’t you just say?
Eh eh, stop jumping to conclusions.
Alight, so… information…
If we’re going to explain something away with it, we better know what it means.
Let’s start back before this word was even being used.
Back when the speed of light was hot debate.
Always the rebel, Galileo, was pretty sure the speed of light was finite.
That it took time to get from one place to another.
He even suggested measuring it using a couple mountains and a lantern,
but light is just too fast for experiment that simple.
Newton was firmly in the infinite speed camp.
He was convinced that light traveled from one place to another in no time at all
and even applied the idea to his universal law of gravity.
He said that, if the Sun just disappeared,
the Earth would instantly respond to its strange and sudden absence.
Luckily, by the end of that century,
people like Roemer and Huygens had taken measurements proving Newton wrong.
The speed of light is finite.
So, in the early 1700s, we knew light takes time to get places
and we were ready to start talking about information,
but we didn't.
The fact that light was finite was more of a nuisance than anything else.
It was just something you had to adjust for in your data when you looked into space.
It wasn’t really a part of scientific conversation until we started to realize
how big the universe actually was and where it came from,
and that took a while.
Beautiful images like this take hours or sometimes days of exposure,
which means we needed cameras to be invented first.
The oldest known photograph has been dated to 1826
and it was good timing because that’s when physics really started to change.
Several measurements of the speed of light,
discovery that light was an electromagnetic wave and the speed of light was constant,
special relativity, space-time diagrams, general relativity,
then in 1924, Edwin Hubble measured the distance to Andromeda
and found it was millions of light years away from Earth,
way passed anything anyone could have imagined at the time.
In 1927, Georges Lemaître predicted the universe was expanding much to Einstein’s dismay.
And then, Hubble returned in 1929 with more measurements showing
the universe was, in fact, expanding.
Why is this important?
We’re getting there. Be patient!
Let’s review: We know the speed of light is finite and the same for all observers.
We also know the universe is billions of light years across and billions of years old.
If light takes time to get places, then we’re not looking at the universe in real time.
The farther away something is, the farther back in time we’re looking.
So now it’s important to start talking about information.
Say we want to affect the Centauri system 4 light years away.
The fastest thing we can send there is light, which has a finite speed.
We have to wait 4 years for that light to get there.
The same is true if the Centauri system wants to affect us.
If we both send a signal at the same time, those signals would pass through each other here,
so that’s the earliest we could interact with each other.
Diagonal light paths in all directions form something we call a light cone.
We can only interact with the Centauri system where our light cones overlap.
This applies to everything, whether there are people or not.
The upper limit on the speed of interactions is the speed of light, including gravity.
Any speed you observe to be faster than light is okay
as long as that speed isn’t causally connecting events in space-time.
That super-fast star from earlier is okay because, from that point of view,
the rest of the universe is moving with it.
It’s not getting any closer to anything else.
The speed of light isn’t the speed of light it’s the speed of causality.
It’s the speed of information!
Without this upper limit, the universe would have effects before causes,
dogs and cats living together, mass hysteria!
Fine. it’s the speed of information, but what is information?
Oh, right!
Well, it can’t be matter or energy because we’ve seen those sometimes break the limit.
The modern form of the term “information” wasn’t really used
until we started communicating with technology.
In 1948, a huge paper was published on how we transmit data.
Let’s say you want to send this message.
Whether you send it by cable, radio wave, or some other method,
it has to be converted into some kind of signal a transmitter can send.
That signal is sent to a receiver, where it’s converted back into a message.
The more data the signal contains, the harder it is to send, the longer it takes to send,
and the more likely some of it will be lost along the way.
So you want that message to be converted into as little data as possible.
It’s why these different copies of the same picture have vastly different file sizes.
The same amount of information is contained using different amounts of data.
That same 1948 paper mention a quantity they call information entropy,
which tells you the smallest amount of data required to contain a certain amount of information.
For example, English is between 0.6 and 1.3 data bits per English character.
They decided to call it entropy because the equation is incredibly similar to physical entropy.
Back to the timeline!
Rudolf Clausius first used the word entropy in 1865,
but it didn’t look much like what we would eventually call information entropy
until Boltzmann and Gibbs got their hands on it in 1877.
See the resemblance?
They didn’t really understand what it meant though
because, at the time, no one was sure what matter was even made of.
The electron wasn’t discovered until 1897 and we wouldn’t see an atomic model that
looked anything like the modern atom until 1911.
It was around then we started to realize what was going on.
That “mixed-up-ness” is about how the energy inside an object’s atoms is organized.
The more ways that energy could be organized, the less information you actually have about it.
It keeps shifting between all the possibilities.
Over time, the universe tends to have more and more ways of organizing energy,
so the entropy of the universe is always going up.
That’s the second law of thermodynamics!
But we need to be careful comparing data information and physical information.
Data is something humans made up.
The physical world is not!
Inside of something like your body, you have all sorts of atoms
exchanging energy with each other and the surroundings.
Let’s consider something a little simpler.
The Sun is emitting something we call solar wind.
It’s basically just a bunch of particles like protons and electrons.
Let’s focus our attention on a single proton.
If it bumps into another proton, energy is exchanged and their entropy goes up.
When it’s not bumping into anything, its entropy stays steady.
But if it’s not interacting with anything, then it’s a quantum object,
which means there are things you can’t know about it.
The uncertainty principle says quantum particles are only particular about a few properties at a time.
All the others are pretty indefinite.
So, when that proton interacts with something, its properties will change.
Some of the definite ones might change value, some of them might become indefinite,
and some indefinite ones might become definite.
We can’t know for sure ahead of time.
Stupid quantum physics!
The physical information of the proton is some composite measure of where it is,
what it’s doing, how much mass, charge, and energy it has,
you know, its properties.
No matter what happens, less is knowable about the universe as a whole.
Information is always lost.
And the change in entropy just measures how much we lost.
I’m stretching this a little here, but what if we’re all made of information?
And, over time, that information fades away as it interacts with itself.
Let me know what you think in the comments.
And until next time, remember, it’s OK to be a little crazy.