Information diagram

Template:Short description An information diagram is a type of Venn diagram used in information theory to illustrate relationships among Shannon's basic measures of information: entropy, joint entropy, conditional entropy and mutual information. Information diagrams are a useful pedagogical tool for teaching and learning about these basic measures of information. Information diagrams have also been applied to specific problems such as for displaying the information theoretic similarity between sets of ontological terms.

• Venn diagram showing additive and subtractive relationships among various information measures associated with correlated variables Template:Mvar and Template:Mvar. The area contained by both circles is the joint entropy Template:Tmath. The circle on the left (red and violet) is the individual entropy Template:Tmath, with the red being the conditional entropy H(X|Y). The circle on the right (blue and violet) is Template:Tmath, with the blue being H(Y|X). The violet is the mutual information Template:Tmath.
  Venn diagram showing additive and subtractive relationships among various information measures associated with correlated variables Template:Mvar and Template:Mvar. The area contained by both circles is the joint entropy Template:Tmath. The circle on the left (red and violet) is the individual entropy Template:Tmath, with the red being the conditional entropy $H(X|Y)$. The circle on the right (blue and violet) is Template:Tmath, with the blue being $H(Y|X)$. The violet is the mutual information Template:Tmath.
• Venn diagram of information theoretic measures for three variables Template:Mvar, and Template:Mvar. Each circle represents an individual entropy: Template:Tmath is the lower left circle, Template:Tmath the lower right, and Template:Tmath is the upper circle. The intersections of any two circles represents the mutual information for the two associated variables (e.g. Template:Tmath is yellow and gray). The union of any two circles is the joint entropy for the two associated variables (e.g. Template:Tmath is everything but green). The joint entropy Template:Tmath of all three variables is the union of all three circles. It is partitioned into 7 pieces, red, blue, and green being the conditional entropies H(x|y,z),H(y|x,z),H(z|x,y) respectively, yellow, magenta and cyan being the conditional mutual informations I(x;z|y),I(y;z|x) and I(x;y|z) respectively, and gray being the interaction information Template:Tmath. The interaction information is the only one of all that may be negative.
  Venn diagram of information theoretic measures for three variables Template:Mvar, and Template:Mvar. Each circle represents an individual entropy: Template:Tmath is the lower left circle, Template:Tmath the lower right, and Template:Tmath is the upper circle. The intersections of any two circles represents the mutual information for the two associated variables (e.g. Template:Tmath is yellow and gray). The union of any two circles is the joint entropy for the two associated variables (e.g. Template:Tmath is everything but green). The joint entropy Template:Tmath of all three variables is the union of all three circles. It is partitioned into 7 pieces, red, blue, and green being the conditional entropies $H(x|y,z),H(y|x,z),H(z|x,y)$ respectively, yellow, magenta and cyan being the conditional mutual informations $I(x;z|y),I(y;z|x)$ and $I(x;y|z)$ respectively, and gray being the interaction information Template:Tmath. The interaction information is the only one of all that may be negative.

References

Template:Settheory-stub