View source for Entropy (statistical thermodynamics)

{{Short description|Concept}}
{{primary sources|date=July 2018}}
The concept '''entropy''' was first developed by German physicist [[Rudolf Clausius]] in the mid-nineteenth century as a thermodynamic property that predicts that certain spontaneous processes are irreversible or impossible. In [[statistical mechanics]], [[entropy]] is formulated as a statistical property using [[probability theory]]. The '''statistical entropy''' perspective was introduced in 1870 by Austrian physicist [[Ludwig Boltzmann]], who established a new field of [[physics]] that provided the descriptive linkage between the macroscopic observation of nature and the microscopic view based on the rigorous treatment of large ensembles of microscopic states that constitute [[thermodynamic system]]s.

== Boltzmann's principle ==
{{main|Boltzmann's entropy formula}}
Ludwig Boltzmann defined entropy as a measure of the number of possible microscopic states (''microstates'') of a system in [[thermodynamic equilibrium]], consistent with its macroscopic thermodynamic properties, which constitute the ''macrostate'' of the system. A useful illustration is the example of a sample of gas contained in a container. The easily measurable parameters volume, pressure, and temperature of the gas describe its macroscopic condition (''state''). At a microscopic level, the gas consists of a vast number of freely moving [[atom]]s or [[molecule]]s, which randomly collide with one another and with the walls of the container. The collisions with the walls produce the macroscopic pressure of the gas, which illustrates the connection between microscopic and macroscopic phenomena.

A microstate of the system is a description of the [[position (vector)|position]]s and [[momentum|momenta]] of all its particles. The large number of particles of the gas provides an infinite number of possible microstates for the sample, but collectively they exhibit a well-defined average of configuration, which is exhibited as the macrostate of the system, to which each individual microstate contribution is negligibly small. The ensemble of microstates comprises a statistical distribution of probability for each microstate, and the group of most probable configurations accounts for the macroscopic state. Therefore, the system can be described as a whole by only a few macroscopic parameters, called the ''thermodynamic variables'': the total [[energy]] {{mvar|E}}, [[volume]] {{mvar|V}}, [[pressure]] {{mvar|P}}, [[temperature]] {{mvar|T}}, and so forth. However, this description is relatively simple only when the system is in a state of equilibrium.

Equilibrium may be illustrated with a simple example of a drop of food coloring falling into a glass of water. The dye diffuses in a complicated manner, which is difficult to precisely predict. However, after sufficient time has passed, the system reaches a uniform color, a state much easier to describe and explain.

Boltzmann formulated a simple relationship between entropy and the number of possible microstates of a system, which is denoted by the symbol {{math|Ω}}. The entropy {{mvar|S}} is [[proportionality (mathematics)|proportional]] to the [[natural logarithm]] of this number:
<math display="block">S = k_\text{B} \ln \Omega</math>
The proportionality constant {{math|''k''<sub>B</sub>}} is one of the fundamental constants of physics and is named the [[Boltzmann constant]] in honor of its discoverer.

Boltzmann's entropy describes the system when all the accessible microstates are equally likely. It is the configuration corresponding to the maximum of entropy at equilibrium. The randomness or disorder is maximal, and so is the lack of distinction (or information) of each microstate.

Entropy is a thermodynamic property just like pressure, volume, or temperature. Therefore, it connects the microscopic and the macroscopic world view.

Boltzmann's principle is regarded as the foundation of [[statistical mechanics]].

== Gibbs entropy formula==
The macroscopic state of a system is characterized by a distribution on the [[microstate (statistical mechanics)|microstates]]. The entropy of this distribution is given by the Gibbs entropy formula, named after [[Josiah Willard Gibbs|J. Willard Gibbs]]. For a classical system (i.e., a collection of classical particles) with a discrete set of microstates, if {{math|''E''<sub>''i''</sub>}} is the energy of microstate {{mvar|i}}, and {{math|''p''<sub>''i''</sub>}} is the probability that it occurs during the system's fluctuations, then the entropy of the system is
<math display="block">S = -k_\text{B}\,\sum_i p_i \ln (p_i)</math>

<div style=" width: 320px; float: right; margin: 0 0 1em 1em; border-style: solid; border-width: 1px; padding: 1em; font-size: 90%">
'''Entropy changes for systems in a canonical state'''

A system with a well-defined temperature, i.e., one in thermal equilibrium with a thermal reservoir, has a probability of being in a microstate {{mvar|i}} given by [[Boltzmann's distribution]].

Changes in the entropy caused by changes in the external constraints are then given by:
<math display="block">\begin{align}
dS &= -k_\text{B}\,\sum_i dp_i \ln p_i\\
&= -k_\text{B}\,\sum_i dp_i (-E_i/k_\text{B}T -\ln Z)\\
&= \sum_i E_i dp_i / T \\
&= \sum_i [d (E_i p_i) - (dE_i) p_i] / T
\end{align}</math>
where we have twice used the conservation of probability, {{math|1=∑<sub>''i''</sub> ''dp<sub>i</sub>'' = 0}}.

Now, {{math|∑<sub>''i''</sub> ''d''(''E<sub>i</sub>'' ''p<sub>i</sub>'')}} is the expectation value of the change in the total energy of the system.

If the changes are sufficiently slow, so that the system remains in the same microscopic state, but the state slowly (and reversibly) changes, then {{math|∑<sub>''i''</sub> (''dE<sub>i</sub>'') ''p<sub>i</sub>''}} is the expectation value of the work done on the system through this reversible process, {{math|''dW''<sub>rev</sub>}}.

But from the first law of thermodynamics, {{math|1=''dE'' = ''δQ'' + ''δW''}}. Therefore,
<math display="block">dS = \frac{\delta\langle Q_\text{rev} \rangle}{T}</math>

In the [[thermodynamic limit]], the fluctuation of the macroscopic quantities from their average values becomes negligible; so this reproduces the definition of entropy from classical thermodynamics, given above. 
</div>

The quantity <math>k_\text{B}</math> is the [[Boltzmann constant]], a multiplier of the summation expression.  The [[summation]] is [[Dimensionless quantity|dimensionless]], since the value <math>p_i</math> is a probability and therefore dimensionless, and {{math|ln}} is the [[natural logarithm]]. Hence the [[SI unit]] on both sides of the equation is that of [[heat capacity]]:
<math display="block"> [S] = [k_\text{B}] = \mathrm{\frac {J} {K}}</math>

This definition remains meaningful even when the system is far away from equilibrium. Other definitions assume that the system is in [[thermal equilibrium]], either as an [[isolated system]], or as a system in exchange with its surroundings. The set of microstates (with probability distribution) over which the sum is found is called a [[statistical ensemble]]. Each type of [[statistical ensemble]] (micro-canonical, canonical, grand-canonical, etc.) describes a different configuration of the system's exchanges with the outside, varying from a completely isolated system to a system that can exchange one or more quantities with a reservoir, like energy, volume or molecules. In every ensemble, the [[thermodynamic equilibrium|equilibrium]] configuration of the system is dictated by the maximization of the entropy of the union of the system and its reservoir, according to the [[second law of thermodynamics]] (see the [[statistical mechanics]] article).

Neglecting [[correlation]]s (or, more generally, [[Statistical independence|statistical dependencies]]) between the states of individual particles will lead to an incorrect probability distribution on the microstates and hence to an overestimate of the entropy.<ref name="jaynes1965">{{cite journal | first = E.T. | last = Jaynes | title = Gibbs vs Boltzmann Entropies | journal = American Journal of Physics | page = 391 | year = 1965 | doi = 10.1119/1.1971557 }}</ref>  Such correlations occur in any system with nontrivially interacting particles, that is, in all systems more complex than an [[ideal gas]].

This {{mvar|S}} is almost universally called simply the ''entropy''.  It can also be called the ''statistical entropy'' or the ''thermodynamic entropy'' without changing the meaning.  Note the above expression of the statistical entropy is a discretized version of [[Shannon entropy]].  The [[von Neumann entropy]] formula is an extension of the Gibbs entropy formula to the [[Quantum mechanics|quantum-mechanical]] case.

It has been shown<ref name="jaynes1965" /> that the Gibbs entropy is equal to the classical "heat engine" entropy characterized by <math>dS = {\delta Q} / {T}</math>, and the [[Boltzmann distribution#Generalized Boltzmann distribution|generalized Boltzmann distribution]] is a sufficient and necessary condition for this equivalence.<ref name="Gao2019">{{cite journal |last1= Gao |first1= Xiang |last2= Gallicchio |first2= Emilio |first3= Adrian |last3= Roitberg  |date= 2019 |title= The generalized Boltzmann distribution is the only distribution in which the Gibbs-Shannon entropy equals the thermodynamic entropy |url= https://aip.scitation.org/doi/abs/10.1063/1.5111333|journal= The Journal of Chemical Physics|volume= 151 | issue= 3|pages= 034113|doi= 10.1063/1.5111333 |pmid= 31325924 |arxiv= 1903.02121 |bibcode= 2019JChPh.151c4113G |s2cid= 118981017 |access-date= }}</ref> Furthermore, the Gibbs entropy is the only entropy measure that is equivalent to the classical "heat engine" entropy under the following postulates:<ref name="Gao2022">{{cite journal |last1= Gao |first1= Xiang |date= March 2022 |title= The Mathematics of the Ensemble Theory |journal= Results in Physics|volume= 34|pages= 105230|doi= 10.1016/j.rinp.2022.105230 |bibcode= 2022ResPh..3405230G |s2cid= 221978379 |doi-access= free |arxiv= 2006.00485 }}</ref>
{{ordered list
| The probability density function is proportional to some function of the ensemble parameters and random variables.
| Thermodynamic state functions are described by ensemble averages of random variables.
| In the limit of infinite temperature, all the microstates have the same probability.
}}

=== Ensembles ===

The various ensembles used in [[statistical thermodynamics]] are linked to the entropy by the following relations:{{clarify|reason=What are the quantities that are being maintained constant between these different ensembles? Is this relationship only valid in the thermodynamic limit?|date=September 2013}}
<math display="block">S = k_\text{B} \ln \Omega_\text{mic} = k_\text{B} (\ln Z_\text{can} + \beta \bar E) = k_\text{B} (\ln \mathcal{Z}_\text{gr} + \beta (\bar E - \mu \bar N)) </math>
* <math>\Omega_\text{mic} </math> is the [[microcanonical ensemble|microcanonical partition function]]
* <math>Z_\text{can} </math> is the [[canonical ensemble|canonical partition function]]
* <math>\mathcal{Z}_\text{gr} </math> is the [[grand canonical ensemble|grand canonical partition function]]

== Order through chaos and the second law of thermodynamics ==
We can think of Ω as a measure of our lack of knowledge about a system. To illustrate this idea, consider a set of 100 [[coin]]s, each of which is either [[coin flipping|heads up or tails up]]. In this example, let us suppose that the macrostates are specified by the total number of heads and tails, while the microstates are specified by the facings of each individual coin (i.e., the exact order in which heads and tails occur). For the macrostates of 100 heads or 100 tails, there is exactly one possible configuration, so our knowledge of the system is complete. At the opposite extreme, the macrostate which gives us the least knowledge about the system consists of 50 heads and 50 tails in any order, for which there are {{val|100,891,344,545,564,193,334,812,497,256}} ([[combination|100 choose 50]]) ≈ 10<sup>29</sup> possible microstates.

Even when a system is entirely isolated from external influences, its microstate is constantly changing. For instance, the particles in a gas are constantly moving, and thus occupy a different position at each moment of time; their momenta are also constantly changing as they collide with each other or with the container walls. Suppose we prepare the system in an artificially highly ordered equilibrium state. For instance, imagine dividing a container with a partition and placing a gas on one side of the partition, with a vacuum on the other side. If we remove the partition and watch the subsequent behavior of the gas, we will find that its microstate evolves according to some chaotic and unpredictable pattern, and that on average these microstates will correspond to a more disordered macrostate than before. It is ''possible'', but ''extremely unlikely'', for the gas molecules to bounce off one another in such a way that they remain in one half of the container. It is overwhelmingly probable for the gas to spread out to fill the container evenly, which is the new equilibrium macrostate of the system.

This is an example illustrating the [[second law of thermodynamics]]:
: ''the total entropy of any isolated thermodynamic system tends to increase over time, approaching a maximum value''.

Since its discovery, this idea has been the focus of a great deal of thought, some of it confused. A chief point of confusion is the fact that the Second Law applies only to ''isolated'' systems. For example, the [[Earth]] is not an isolated system because it is constantly receiving energy in the form of [[sunlight]]. In contrast, the [[universe]] may be considered an isolated system, so that its total entropy is constantly increasing. (Needs clarification. See: [[Second law of thermodynamics#cite note-Grandy 151-21]])

== Counting of microstates ==
In [[classical mechanics|classical]] [[statistical mechanics]], the number of microstates is actually [[Uncountable set|uncountably infinite]], since the properties of classical systems are continuous. For example, a microstate of a classical ideal gas is specified by the positions and momenta of all the atoms, which range continuously over the [[real number]]s. If we want to define Ω, we have to come up with a method of grouping the microstates together to obtain a [[countable set]]. This procedure is known as [[coarse graining]]. In the case of the ideal gas, we count two states of an atom as the "same" state if their positions and momenta are within ''δx'' and ''δp'' of each other. Since the values of {{math|''δx''}} and {{math|''δp''}} can be chosen arbitrarily, the entropy is not uniquely defined. It is defined only up to an additive constant. (As we will see, the [[Entropy (classical thermodynamics)|thermodynamic definition of entropy]] is also defined only up to a constant.)

To avoid coarse graining one can take the entropy as defined by the [[H-theorem#Tolman's_H-theorem|H-theorem]].<ref>{{cite book |isbn=0-486-68455-5|title=Lectures on Gas Theory|last1=Boltzmann|first1=Ludwig|date=January 1995|publisher=Courier Corporation }}</ref>
<math display="block">S = -k_\text{B} H_\text{B} := -k_\text{B} \int  f(q_i, p_i) \, \ln f(q_i,p_i) \,dq_1 \, dp_1 \cdots dq_N \, dp_N</math>

However, this ambiguity can be resolved with [[quantum mechanics]]. The [[quantum state]] of a system can be expressed as a superposition of "basis" states, which can be chosen to be energy [[eigenstate]]s (i.e. eigenstates of the quantum [[Hamiltonian (quantum mechanics)|Hamiltonian]]). Usually, the quantum states are discrete, even though there may be an infinite number of them. For a system with some specified energy {{mvar|E}}, one takes {{math|Ω}} to be the number of energy eigenstates within a macroscopically small energy range between {{mvar|E}} and {{math|''E'' + ''δE''}}. In the [[thermodynamical limit]], the specific entropy becomes independent on the choice of {{math|''δE''}}.

An important result, known as [[Nernst's theorem]] or the [[third law of thermodynamics]], states that the entropy of a system at [[absolute zero|zero absolute temperature]] is a well-defined constant. This is because a system at zero temperature exists in its lowest-energy state, or [[ground state]], so that its entropy is determined by the [[Hamiltonian (quantum mechanics)|degeneracy]] of the ground state. Many systems, such as [[crystal|crystal lattices]], have a unique ground state, and (since {{math|1=ln(1) = 0}}) this means that they have zero entropy at absolute zero. Other systems have more than one state with the same, lowest energy, and have a non-vanishing "zero-point entropy". For instance, ordinary [[ice]] has a zero-point entropy of {{val|3.41|u=J/(mol⋅K)}}, because its underlying [[crystal structure]] possesses multiple configurations with the same energy (a phenomenon known as [[geometrical frustration]]).

The third law of thermodynamics states that the entropy of a [[perfect crystal]] at absolute zero ({{val|0|ul=K}}) is zero. This means that nearly all molecular motion should cease. The [[quantum harmonic oscillator|oscillator equation]] for predicting quantized vibrational levels shows that even when the vibrational [[quantum number]] is 0, the molecule still has vibrational energy:{{Citation needed|date=March 2021}}
<math display="block">E_\nu = h \nu_0 \left(n + \tfrac{1}{2}\right)</math>
where <math>h</math> is the [[Planck constant]], <math>\nu_0</math> is the characteristic frequency of the vibration, and <math>n</math> is the vibrational quantum number. Even when <math>n=0</math> (the [[zero-point energy]]), <math>E_n</math> does not equal 0, in adherence to the [[Heisenberg uncertainty principle]].

== See also ==
{{div col}}
* [[Boltzmann constant]]
* [[Configuration entropy]]
* [[Conformational entropy]]
* [[Enthalpy]]
* [[Entropy]]
* [[Entropy (classical thermodynamics)]]
* [[Entropy (energy dispersal)]]
* [[Entropy of mixing]]
* [[Entropy (order and disorder)]]
* [[Entropy (information theory)]]
* [[History of entropy]]
* [[Information theory]]
* [[Thermodynamic free energy]]
* [[Tsallis entropy]]
{{div col end}}

== References ==
{{reflist}}

{{DEFAULTSORT:Entropy (Statistical Thermodynamics)}}
[[Category:Thermodynamic entropy]]