proofs
Provedentropy-cost-lawattention-entropystabilitylaw

Entropy–Cost identity: n·Σπ² = exp(log n − H₂), H₂ = −log Σπ² (Rényi-2 entropy)

For all n∈N\htmlData{mx=1}{n \in \htmlData{mx=0}{\mathbb{N}}} and π:Fin⁡ n→R\htmlData{mx=5}{\pi : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}, if 0<n0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{n} and 0<∑jπ(j)20 \mathrel{\htmlData{mx=5}{<}} \htmlData{mx=4}{\sum_{j} \htmlData{mx=3}{\htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})}^{2}}}, then

n⋅∑jπ(j)2=elog⁡n−−log⁡(∑jπ(j)2)\htmlData{mx=7}{\htmlData{mx=1}{\htmlData{mx=0}{n}} \cdot \htmlData{mx=6}{\sum_{j} \htmlData{mx=5}{\htmlData{mx=4}{\htmlData{mx=2}{\pi}(\htmlData{mx=3}{j})}^{2}}}} \mathrel{\htmlData{mx=19}{=}} \htmlData{mx=18}{e^{\htmlData{mx=17}{\htmlData{mx=9}{\log \htmlData{mx=8}{n}} - \htmlData{mx=16}{-\htmlData{mx=15}{\log \left(\htmlData{mx=14}{\sum_{j} \htmlData{mx=13}{\htmlData{mx=12}{\htmlData{mx=10}{\pi}(\htmlData{mx=11}{j})}^{2}}}\right)}}}}}
What the notation means · or point at any symbol
NNatural numbers
The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin
Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application
Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
⟹Implies
If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
<Less than
0 is strictly smaller than n. (0 < n)
(x : T)As real numbers
n read as an element of real numbers. Mathematicians leave this implicit; Lean needs it to know which kind of number is meant. ((n : ℝ))
·Multiplication
n times ∑ j, π(j)^2. ((n : ℝ) * ∑ j, π j ^ 2)
=Equality
n · (∑ j, π(j)^2) and exp(log(n) − −log(∑ j, π(j)^2)) are the same. ((n : ℝ) * ∑ j, π j ^ 2 = Real.exp (Real…)
∑Summation
Add up π(j)^2 for every j. (∑ j, π j ^ 2)
RReal numbers
The real numbers: every point on the number line. (ℝ)
πPi
The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
aᵇPower
π(j) raised to the power 2. (π j ^ 2)
eˣExponential function
e raised to the power log(n) − −log(∑ j, π(j)^2), where e ≈ 2.718. (Real.exp (Real.log n - (- Real.log (∑ j…)
logNatural logarithm
The logarithm of n to base e: the power e must be raised to to give n. (Real.log n)
-Subtraction
log(n) minus −log(∑ j, π(j)^2). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (Real.log n - (- Real.log (∑ j, π j ^ 2)))
−xNegation
The negative of log(∑ j, π(j)^2). (- Real.log (∑ j, π j ^ 2))

The uniform-proposal sampling cost factor 1 + χ² of an attention row equals exp(log n − H₂) exactly. Part of 'Entropy–Cost Law: attention compute cost is n e^{-H2}, so stability regularizers also buy speed'.

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
11 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements