View source for Quantum relative entropy

{{Short description|Measure of distinguishability between two quantum states}}
{{More references|date=February 2025}}{{Inline references|date=February 2025}}

In [[quantum information theory]], '''quantum relative entropy''' is a measure of distinguishability between two [[density matrix|quantum states]]. It is the quantum mechanical analog of [[relative entropy]].

== Motivation ==

For simplicity, it will be assumed that all objects in the article are finite-dimensional.

We first discuss the classical case. Suppose the probabilities of a finite sequence of events is given by the probability distribution ''P'' = {''p''<sub>1</sub>...''p''<sub>''n''</sub>}, but somehow we mistakenly assumed it to be ''Q'' = {''q''<sub>1</sub>...''q''<sub>''n''</sub>}. For instance, we can mistake an unfair coin for a fair one. According to this erroneous assumption, our uncertainty about the ''j''-th event, or equivalently, the amount of information provided after observing the ''j''-th event, is

:<math>\; - \log q_j.</math>

The (assumed) average uncertainty of all possible events is then

:<math>\; - \sum_j p_j \log q_j.</math>

On the other hand, the [[Shannon entropy]] of the probability distribution ''p'', defined by

:<math>\; - \sum_j p_j \log p_j,</math>

is the real amount of uncertainty before observation. Therefore the difference between these two quantities

:<math>\; - \sum_j p_j \log q_j - \left(- \sum_j p_j \log p_j\right) = \sum_j p_j \log p_j - \sum_j p_j \log q_j</math>

is a measure of the distinguishability of the two probability distributions ''p'' and ''q''. This is precisely the classical relative entropy, or [[Kullback–Leibler divergence]]:

:<math>D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} \!.</math>

'''Note''' 
#In the definitions above, the convention that 0·log&nbsp;0&nbsp;=&nbsp;0 is assumed, since <math>\lim_{x \searrow 0} x \log(x) = 0</math>. Intuitively, one would expect that an event of [[zero probability]] to contribute nothing towards entropy.
#The relative entropy is not a [[metric space|metric]]. For example, it is not symmetric. The uncertainty discrepancy in mistaking a fair coin to be unfair is not the same as the opposite situation.

== Definition ==

As with many other objects in quantum information theory, quantum relative entropy is defined by extending the classical definition from probability distributions to [[density matrix|density matrices]]. Let ''ρ'' be a density matrix. The [[von Neumann entropy]] of ''ρ'', which is the quantum mechanical analog of the Shannon entropy, is given by

:<math>S(\rho) = - \operatorname{Tr} \rho \log \rho.</math>

For two density matrices ''ρ'' and ''σ'', the '''quantum relative entropy of ''ρ'' with respect to ''σ''''' is defined by

:<math>
S(\rho \| \sigma) = - \operatorname{Tr} \rho \log \sigma - S(\rho) = \operatorname{Tr} \rho \log \rho - \operatorname{Tr} \rho \log \sigma = \operatorname{Tr}\rho (\log \rho - \log \sigma).
</math>

We see that, when the states are classically related, i.e. ''ρσ'' = ''σρ'', the definition coincides with the classical case, in the sense that if <math>\rho = S D_1 S^{\mathsf{T}}</math> and <math>\sigma = S D_2 S^{\mathsf{T}}</math> with <math>D_1 = \text{diag}(\lambda_1, \ldots, \lambda_n)</math> and <math>D_2 = \text{diag}(\mu_1, \ldots, \mu_n)</math> (because <math>\rho</math> and <math>\sigma</math> [[commuting matrices|commute]], they are [[Diagonalizable matrix|simultaneously diagonalizable]]), then <math>S(\rho \| \sigma) = \sum_{j = 1}^{n} \lambda_j \ln\left(\frac{\lambda_j}{\mu_j}\right)</math> is just the ordinary [[Kullback-Leibler divergence]] of the probability vector <math>(\lambda_1, \ldots, \lambda_n)</math> with respect to the probability vector <math>(\mu_1, \ldots, \mu_n)</math>.

=== Non-finite (divergent) relative entropy ===

In general, the ''support'' of a matrix ''M'' is the orthogonal complement of its [[kernel (matrix)|kernel]], i.e. <math>\text{supp}(M) = \text{ker}(M)^\perp </math>. When considering the quantum relative entropy, we assume the convention that &minus;''s''&nbsp;·&nbsp;log&nbsp;0&nbsp;=&nbsp;∞ for any ''s''&nbsp;>&nbsp;0. This leads to the definition that

:<math>S(\rho \| \sigma) = \infty</math>

when

:<math>\text{supp}(\rho) \cap \text{ker}(\sigma) \neq \{ 0 \}.</math>

This can be interpreted in the following way. Informally, the quantum relative entropy is a measure of our ability to distinguish two quantum states where larger values indicate states that are more different. Being orthogonal represents the most different quantum states can be. This is reflected by non-finite quantum relative entropy for orthogonal quantum states.  Following the argument given in the Motivation section, if we erroneously assume the state <math>\rho</math> has support in <math>\text{ker}(\sigma)</math>, this is an error impossible to recover from.

However, one should be careful not to conclude that the divergence of the quantum relative entropy <math>S(\rho\|\sigma)</math> implies that the states <math>\rho</math> and <math>\sigma</math> are orthogonal or even very different by other measures. Specifically, <math>S(\rho\|\sigma)</math> can diverge when <math>\rho</math> and <math>\sigma</math> differ by a ''vanishingly small amount'' as measured by some norm.  For example, let <math>\sigma</math> have the diagonal representation

<math>\sigma=\sum_{n}\lambda_n|f_n\rangle\langle f_n|</math>

with <math>\lambda_n>0</math> for <math>n=0,1,2,\ldots</math> and <math>\lambda_n=0</math> for <math>n=-1,-2,\ldots</math> where <math>\{|f_n\rangle, n\in \Z\}</math> is an orthonormal set.  The kernel of <math>\sigma</math> is the space spanned by the set <math>\{|f_{n}\rangle, n=-1,-2,\ldots\}</math>. Next let

<math>    \rho=\sigma+\epsilon|f_{-1}\rangle\langle f_{-1}| - \epsilon|f_1\rangle\langle f_1|</math>

for a small positive number <math>\epsilon</math>.  As <math>\rho</math> has support (namely the state <math>|f_{-1}\rangle</math>) in the kernel of <math>\sigma</math>, <math>S(\rho\|\sigma)</math> is divergent even though the trace norm of the difference <math>(\rho-\sigma)</math> is <math>2\epsilon</math> . This means that difference between <math>\rho</math> and <math>\sigma</math> as measured by the trace norm is vanishingly small as <math>\epsilon\to 0</math> even though <math>S(\rho\|\sigma)</math> is divergent (i.e. infinite). This property of the quantum relative entropy represents a serious shortcoming if not treated with care.

== Non-negativity of relative entropy ==

=== Corresponding classical statement ===

For the classical [[Kullback–Leibler divergence]], it can be shown that

:<math>D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} \geq 0,</math>

and the equality holds if and only if ''P'' = ''Q''. Colloquially, this means that the uncertainty calculated using erroneous assumptions is always greater than the real amount of uncertainty.

To show the inequality, we rewrite

:<math>D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} =  \sum_j (- \log \frac{q_j}{p_j})(p_j).</math>

Notice that log is a [[concave function]]. Therefore -log is [[convex function|convex]]. Applying [[Jensen's inequality]], we obtain

:<math>
D_{\mathrm{KL}}(P\|Q) = \sum_j (- \log \frac{q_j}{p_j})(p_j) \geq - \log ( \sum_j \frac{q_j}{p_j} p_j ) = 0.
</math>

Jensen's inequality also states that equality holds if and only if, for all ''i'', ''q<sub>i</sub>'' = (Σ''q<sub>j</sub>'') ''p<sub>i</sub>'', i.e. ''p'' = ''q''.

=== The result ===

[[Trace inequalities#Klein2016 10|Klein's inequality]] states that the quantum relative entropy

:<math>
S(\rho \| \sigma) = \operatorname{Tr}\rho (\log \rho - \log \sigma).
</math>

is non-negative in general. It is zero if and only if ''ρ'' = ''σ''.

'''Proof'''

Let ''ρ'' and ''σ'' have spectral decompositions

:<math>\rho = \sum_i p_i v_i v_i ^* \; , \; \sigma = \sum_i q_i w_i w_i ^*.</math>

So

:<math>\log \rho = \sum_i (\log p_i) v_i v_i ^* \; , \; \log \sigma = \sum_i (\log q_i)w_i w_i ^*.</math>

Direct calculation gives

:<math>S(\rho \| \sigma)= \sum_k  p_k \log p_k - \sum_{i,j} (p_i \log q_j) | v_i ^* w_j |^2</math>
:<math>\qquad \quad \;  = \sum_i p_i ( \log p_i - \sum_j \log q_j | v_i ^* w_j |^2)</math>
:<math>\qquad \quad \; = \sum_i p_i (\log p_i - \sum_j (\log q_j )P_{ij}),</math>

where ''P<sub>i j</sub>'' = |''v<sub>i</sub>*w<sub>j</sub>''|<sup>2</sup>.

Since the matrix (''P<sub>i j</sub>'')''<sub>i j</sub>'' is a [[doubly stochastic matrix]] and -log is a convex function, the above expression is

:<math>\geq \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij})).</math>

Define ''r''<sub>i</sub> = Σ<sub>''j''</sub>''q<sub>j</sub> P<sub>i j</sub>''. Then {''r''<sub>i</sub>} is a probability distribution. From the non-negativity of classical relative entropy, we have

:<math>S(\rho \| \sigma) \geq \sum_i p_i \log \frac{p_i}{r_i} \geq 0.</math>

The second part of the claim follows from the fact that, since -log is strictly convex, equality is achieved in

:<math>
\sum_i p_i (\log p_i - \sum_j (\log q_j )P_{ij}) \geq \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij}))
</math>

if and only if (''P<sub>i j</sub>'') is a [[permutation matrix]], which implies ''ρ'' = ''σ'', after a suitable labeling of the eigenvectors {''v<sub>i</sub>''} and {''w<sub>i</sub>''}.<ref>{{Cite book |last=Nielsen |first=Michael A. |url=http://www.cambridge.org/gb/academic/subjects/physics/quantum-physics-quantum-information-and-quantum-computation/quantum-computation-and-quantum-information-10th-anniversary-edition?format=PB&isbn=9781107002173#PkOdhEVJmYvWi4Bd.97 |title=Quantum computation and quantum information |last2=Chuang |first2=Isaac L. |date=2010 |publisher=Cambridge University Press |isbn=978-1-107-00217-3 |edition=10th anniversary |location=Cambridge}}</ref><sup>:513</sup>

== [[Trace inequalities#Joint convexity 2016 10|Joint convexity of relative entropy]] ==
The relative entropy is [[Trace inequalities#Joint convexity function 2016 10|jointly convex]]. For <math>0\leq \lambda\leq 1</math> and states <math>\rho_{1(2)}, \sigma_{1(2)}</math> we have

<math>D(\lambda\rho_1+(1-\lambda)\rho_2\|\lambda\sigma_1+(1-\lambda)\sigma_2)\leq \lambda D(\rho_1\|\sigma_1)+(1-\lambda)D(\rho_2\|\sigma_2)</math>

== [[Strong Subadditivity of Quantum Entropy#Mono2016 10|Monotonicity of relative entropy]] ==
The relative entropy decreases monotonically under [[Completely positive map|completely positive]] [[Trace (linear algebra)|trace]] preserving (CPTP) operations <math>\mathcal{N}</math> on density matrices,

<math>S(\mathcal{N}(\rho)\|\mathcal{N}(\sigma))\leq S(\rho\|\sigma)</math>.

This inequality is called '''monotonicity of quantum relative entropy''' and was first proved by [[Göran_Lindblad_(physicist)|Göran Lindblad]].

== An entanglement measure ==

Let a composite quantum system have state space

:<math>H = \otimes _k H_k</math>

and ''ρ'' be a density matrix acting on ''H''.

The '''relative entropy of entanglement''' of ''ρ'' is defined by

:<math>\; D_{\mathrm{REE}} (\rho) = \min_{\sigma} S(\rho \| \sigma)</math>

where the minimum is taken over the family of [[separable state]]s. A physical interpretation of the quantity is the optimal distinguishability of the state ''ρ'' from separable states.

Clearly, when ''ρ'' is not [[quantum entanglement|entangled]]

:<math>\; D_{\mathrm{REE}} (\rho) = 0</math>

by Klein's inequality.

== Relation to other quantum information quantities ==

One reason the quantum relative entropy is useful is that several other important quantum information quantities are special cases of it.  Often, theorems are stated in terms of the quantum relative entropy, which lead to immediate corollaries concerning the other quantities.  Below, we list some of these relations.

Let ''ρ''<sub>AB</sub> be the joint state of a bipartite system with subsystem ''A'' of dimension ''n''<sub>A</sub> and ''B'' of dimension ''n''<sub>B</sub>.  Let ''ρ''<sub>A</sub>, ''ρ''<sub>B</sub> be the respective reduced states, and ''I''<sub>A</sub>, ''I''<sub>B</sub> the respective identities.  The [[maximally mixed state]]s are ''I''<sub>A</sub>/''n''<sub>A</sub> and ''I''<sub>B</sub>/''n''<sub>B</sub>.  Then it is possible to show with direct computation that

:<math>S(\rho_{A} || I_{A}/n_A) = \mathrm{log}(n_A)- S(\rho_{A}), \;</math>

:<math>S(\rho_{AB} || \rho_{A} \otimes \rho_{B}) = S(\rho_{A}) + S(\rho_{B}) - S(\rho_{AB}) = I(A:B), </math>

:<math>S(\rho_{AB} || \rho_{A} \otimes I_{B}/n_B) = \mathrm{log}(n_B) + S(\rho_{A}) - S(\rho_{AB}) = \mathrm{log}(n_B)- S(B|A), </math>

where ''I''(''A'':''B'') is the [[quantum mutual information]] and ''S''(''B''|''A'') is the [[quantum conditional entropy]].

== References ==
{{Reflist}}

* {{cite journal | last=Vedral | first=V. | title=The role of relative entropy in quantum information theory | journal=Reviews of Modern Physics | publisher=American Physical Society (APS) | volume=74 | issue=1 | date=8 March 2002 | issn=0034-6861 | doi=10.1103/revmodphys.74.197 | pages=197–234|arxiv=quant-ph/0102094| bibcode=2002RvMP...74..197V | s2cid=6370982 }}
* Marco Tomamichel, "[https://link.springer.com/book/10.1007/978-3-319-21891-5 Quantum Information Processing with Finite Resources -- Mathematical Foundations]". [[arXiv:1504.00233]]

[[Category:Quantum mechanical entropy]]
[[Category:Quantum information theory]]