In all of physics, Entropy is one of the most unique yet enigmatic concepts. And understandably
so. what does it actually mean?
The common interpretation is that it is a measure of disorder in the universe. And the
second law of thermodynamics states that the entropy can never decrease. In other words, the universe
is on an inexorable path to more and more disorder. And it is this one way property
of entropy that some scientists contend is the reason time also flows one way - forward.
And If entropy is responsible for something like flow of time itself, then is it a fundamental
property of nature? Or is there something deeper about entropy that really explains
what is going on at the core of reality?
Scottish physicist, James Clerk Maxwell came up with a thought experiment, called Maxwell’s
demon, that showed a mechanism where entropy of a closed system could be reversed. It would
violate the 2nd law. It was a paradox that no one could resolve it for over 100 years.
But when computer researchers at IBM finally did solve this paradox, they showed that entropy
was related to something that could be even more fundamental - and that is Information!
How is this possible? What is the link between entropy, information, and time? The answer
is coming up right now…
If you observe the rotation of the planets around the sun, and then reverse the video,
there is nothing about the reversed motion that is unusual. It would be permissible within
the laws of physics.
Similarly, as far as is known, all physical laws are time reversible. The only laws of
physics which appears to be time irreversible is the law of increasing entropy – the 2nd
law of thermodynamics.
Scrambling an egg, shattering a hard drive, putting sugar in coffee – these are all
actions such that if you saw them in reverse, you would immediately know that you are watching
something absurd, that could not happen. But the 2nd law of thermodynamics appears to be
the only law that reversing these processes would violate. Running the movie backward
would mean decreasing entropy.
The 2nd law says that in an isolated system the entropy can never decrease. But this doesn’t
tell us what entropy actually is. So what is entropy? The common definition is that
entropy is a measure of the disorder of a system. This is not quite precise. To get
a better understanding, let look at a simple visual example.
Let’s take a container with gas isolated on one side with a barrier. If we remove the
barrier, we expect the gas to expand into the full volume of the container. We have
just increased the entropy of the gas.
Why is there more disorder if the gas expands into a larger volume? The reason is that there
is a greater volume and thus more places that individual gas molecules could be. There are
more ways to arrange the gas molecules in a larger volume than in a smaller volume.
This greater possible number of arrangements is really a better description of entropy.
The insight was made by Austrian Physicist Ludwig Boltzmann as a definition of entropy
- that it is a way of counting the number of arrangements of atoms inside a system.
Now, what if we had a plunger on the right wall and pushed the gas back to half the volume
of the container. Wouldn’t this reduce the entropy back to what it was? No, because compressing
the gas like this requires putting work into the system. It would raise the temperature
of the gas. The gas would heat up and the entropy would increase making up for any decrease
in entropy because of the smaller volume.
Why does the entropy increase when the temperature increases? Again, this has to do with an increase
in the number of ways that the motions of molecules can be described. An increase in
temperature is due to an increase in motions or kinetic energy of molecules in the gas.
As the molecules move faster, the temperature increases. There are more ways to describe
this motion than if the molecules were moving slower.
Now the question is, could the gas spontaneously arrange itself so that it fills only half
the volume? You might say, well that can’t happen because it would violate the 2nd law
by decreasing the entropy. Yes, it would violate the 2nd law, but the truth is that
there is nothing in the laws of physics that really prevents this from happening, except
that it is very very unlikely.
The possibility that all the multiple trillions of molecules in the gas could arrange themselves
in one side of the container is about one in 10^150,000,000,000,000,000,000,000. This
number is so enormous that it is statistically impossible. To give you an idea of scale,
the total number of planck volumes, the smallest volume that could exist, that could fit in
the volume of the known universe of the universe is only 10^183. So, needless to say, the 2nd
law is not going to be violated anytime soon. But it also tells you that the 2nd law is
not an absolute law. It is a statistical law. In other words, it could be violated, but
it is just extremely unlikely.
But in 1867, Scottish physicist, James Clerk Maxwell devised a thought experiment that
showed that that the 2nd law could be violated, and entropy could indeed decrease. And this
thought experiment seemed perfectly plausible to scientists, and they could not figure out
why it was wrong for over a 100 years. British Physicist, Lord Kelvin later labeled this
"Maxwell’s Demon."
Imagine a chamber with a divider in the center. If we start out with a gas in one chamber,
this gas will distribute itself evenly between the two chambers. Now what if we put a door
between the two chambers that is controlled by a little demon.
The door is frictionless so adds no heat or energy to the enclosed system. The demon controls
the door such that he only lets the molecules travel in one direction. When a molecule comes
near the door from the right, he opens the door to let the molecule travel to the left
chamber.
He doesn’t let any of the molecules from the left chamber travel over to the right
chamber. You can imagine that over time, this demon makes it so all the gas is collected
on the left chamber, such that no molecules remain in the right chamber.
No work was added and the temperature remained the same, so the entropy decreased. This clearly
seems to violate the 2nd law.
You might say well, it’s a thought experiment with an imaginary demon. It can’t really
be done. But you have to remember, this was thought of at a time when the concept nano
robots and microprocessors did not exist. It is conceivable that in the near future
we could build a computer controlled device that could be the mechanical equivalent of
the demon.
So scientists could not disprove the idea behind this thought experiment for over a
hundred years. The solution finally came in 1982 by two scientists at, of all places,
IBM research, Rolf Landaur and Charles Bennett. They boiled the idea of the demon
down to a more fundamental idea behind entropy. And that is information.
They showed that In order for the demon to reduce the entropy, he has to gather information
about the movement of various molecules to determine when to open and close the door.
He is building a memory record. The demon is increasing the amount of information in
the system because he is part of the system. This increase in information is an increase
in entropy.
They showed that the decrease in entropy due to the volume decrease is exactly counterbalanced
by an increase in information in the demon’s brain. The 2nd law is not violated.
You might say, well, all I have to do is erase the memory of the demon. If this was a computer
hard drive, we can just reformat the hard drive, and there, now the hard drive
is low entropy again.
Aha, but Landower and Bennett showed that erasing information is not free. Erasing information
produces heat. It increases entropy. So you can't gather information in the Demon, and
then erase it to reduce the overall entropy. The increase in entropy from the heat of the
erasure is perfectly offset by the decrease in information in the hard drive.
This is not very much heat, only about 3 billionths of a trillionth of a joule per bit that is
erased.
But this also gives us a clue about the true nature of entropy. And that nature is that
it is really a measure of the information required to express the state of a system.
A system with Higher entropy requires more information to describe its microstates.
When you increase temperature by adding energy for example, there are more ways that this
energy can be shared with the gas molecules. This requires more information to figure out.
If we increase the volume, the molecules have more places where they can be, so this requires
more information to describe the microstates of the gas. This information requirement is
a better definition of entropy than simply disorder.
Now, how does time enter into the picture? How is time related to entropy and information?
Since all the laws of physics we know appear to be symmetrical with respect to time, there
doesn’t seem to be any reason time should be asymmetrical…Except through the idea
of entropy. Entropy is asymmetrical. It flows only in one direction - higher.
The feature of universe that changes with time Is entropy.
This inspired British Scientist Arthur Eddington to postulate that it is the increase in entropy
that is responsible for the flow of time. But does time go forward because entropy increases?
This is not well understood. It is
not a good theory because the prediction it would make is that time should go backward
when entropy decreases.
There are pockets of decreasing entropy all over the place. The inside of your refrigerator
is a place where entropy is decreasing. Your food is getting colder, but time doesn’t
flow backward inside your refrigerator.
Similarly, the earth at night experiences decreasing entropy. But time does not flow backward at
night.
So what actually causes time to flow forward? To answer that, let’s look at why entropy
increases.
Entropy increases not because the fundamental laws of physics says it does, but because
there are simply more ways that microstates can arrange themselves in the future than
in the past.
For example, there are more ways to arrange checker pieces on a checker board than when
neatly arranged at the start of the game.
We are much more likely to be in a state of higher entropy than lower entropy as time
passes. But since entropy is fundamentally a statistical law, based on higher and higher
amount of information needed to describe microstates, then could the flow of time also be a statistical
phenomenon? That is, time is much more likely to flow forward than to flow backward?
If so, this would leave open the idea that time flowing backward is not forbidden, but
just statistically impossible.
If entropy increases over time, that means that entropy must have been at its lowest
possible state at the start of the universe, at the big bang. This is sometimes referred
to as the "past hypothesis.'
This low entropy start is thought to be the primary reason why time flows forward.
But why was entropy so low at the start? Why did the universe start in a very ordered
state? Given all the energy and matter in the universe, and all the states it could
have been in, why did it start in this way - in this very small subset of all the different configurations
that it could have been in.
No one really knows the final answer, but there are some leading ideas.
Maybe there is a fundamental reason, but we haven’t discovered it yet. And we are trying
to explain something that is part of our ignorance.
Maybe It is just a fact. It is a property of the universe, like the fundamental constants.
Another idea is from physicists like as Alan Guth and Sean Carroll. if there is no upper
limit to entropy, that is, maybe entropy is infinite, then no matter where the universe started, it would have been in
the lowest entropy state. It has nowhere else to go except to a higher entropy state.
If time flows forward as entropy becomes higher, then we are a result of the 2nd law. We are
a consequence of the increasing entropy of the universe. If the universe was in thermal
equilibrium, we would not exist. The reason we have causality, evolution, biological processes,
thoughts, memories and even consciousness is the 2nd law.
Alan Guth points out that If information was not increasing in our brain, we would not
have memories or consciousness. This links entropy to consciousness itself.
And Entropy will continue to increase for billions, and probably trillions upon trillions
of years, when all the galaxies will have spread apart. All the stars will have burnt
out. Dark energy will have ripped apart all matter including all molecules, all atoms,
and even subatomic particles. At some point we will be in thermal equilibrium, at least
locally where ever we are in the universe.
And at this point, entropy may cease its inexorable forward march. And all will become quiet. But
space will not go away. And laws of physics probably won’t either.
What will remain is the vacuum energy, quantum fluctuations in the fabric of space time.
Given enough time, there will be enough random energy fluctuations within the vast space of
the universe that an atom forms spontaneously. A molecule forms spontaneously. Interstellar
gases could form. Stars and galaxies could form. The bigger the thing, the longer you
have to wait. But if you have an eternity to wait, it will happen at some point.
And statistics predicts that if you wait long enough, around 10^10^10^120 years, even a
universe could form spontaneously.
Guys, in this short video, I have only given you a sample of how entropy works.
But if you want to get a deeper understanding of this and other scientific subjects,
then your best bet is to head on over to Brilliant, today's sponsor.
Brilliant a problem solving website and app that allows you to dive into the world of science and math
often through the eyes of a pioneering scientist or mathematician. They offer a variety
of courses from scientific thinking to quantitative finance. There is no better way to learn than
by doing. It's the best way to train your brain. You can take your learning to the next level by
solving problems and working on puzzles in real time. And don't worry if you get stuck on a problem
or puzzle. They give you hints. The goal is always to learn and grow, not to feel silly.
I was really impressed by how well thought-out and curated their content is.
They have a special offer for Arvin Ash viewers right now. Head on over to brilliant.org/ArvinAsh
to sign up for free. Not only will you get a free trial, but the first 200 visitors will
get 20% off their subscription. Check it out, I think you’ll be impressed.
I'd like to thank my supporters on Patreon for their generous support. If you like this
video then please feel free to share it with your friends. I will see you in the next video
my friend.