Provedentropy-cost-lawattention-entropystabilitylaw

Cost factor ≤ exp(log n − H_∞): n·Σπ² ≤ n·max π

For all $\htmlData{mx=1}{n \in \htmlData{mx=0}{\mathbb{N}}}$ , $\htmlData{mx=5}{\pi : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ and $\htmlData{mx=1}{M \in \htmlData{mx=0}{\mathbb{R}}}$ , if $0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{n}$ , $0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{M}$ , $\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})}}$ , $\htmlData{mx=5}{\forall j,\ \htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})} \mathrel{\htmlData{mx=4}{\leq}} \htmlData{mx=3}{M}}$ and $\htmlData{mx=3}{\sum_{j} \htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})}} \mathrel{\htmlData{mx=4}{=}} 1$ , then

\htmlData{mx=7}{\htmlData{mx=1}{\htmlData{mx=0}{n}} \cdot \htmlData{mx=6}{\sum_{j} \htmlData{mx=5}{\htmlData{mx=4}{\htmlData{mx=2}{\pi}(\htmlData{mx=3}{j})}^{2}}}} \mathrel{\htmlData{mx=15}{\leq}} \htmlData{mx=14}{e^{\htmlData{mx=13}{\htmlData{mx=9}{\log \htmlData{mx=8}{n}} - \htmlData{mx=12}{-\htmlData{mx=11}{\log \htmlData{mx=10}{M}}}}}}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
⟹Implies: If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
<Less than: 0 is strictly smaller than n. (0 < n)
∀For all: The statement after the comma holds for every j. (∀ j, 0 ≤ π j)
∑Summation: Add up π(j) for every j. (∑ j, π j)
=Equality: ∑ j, π(j) and 1 are the same. (∑ j, π j = 1)
(x : T)As real numbers: n read as an element of real numbers. Mathematicians leave this implicit; Lean needs it to know which kind of number is meant. ((n : ℝ))
·Multiplication: n times ∑ j, π(j)^2. ((n : ℝ) * ∑ j, π j ^ 2)
≤At most: 0 is less than or equal to π(j). (0 ≤ π j)
RReal numbers: The real numbers: every point on the number line. (ℝ)
πPi: The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
aᵇPower: π(j) raised to the power 2. (π j ^ 2)
eˣExponential function: e raised to the power log(n) − −log(M), where e ≈ 2.718. (Real.exp (Real.log n - (- Real.log M)))
logNatural logarithm: The logarithm of n to base e: the power e must be raised to to give n. (Real.log n)
-Subtraction: log(n) minus −log(M). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (Real.log n - (- Real.log M))
−xNegation: The negative of log(M). (- Real.log M)

theorem ec-cost-le-exp-min-entropy {n : ℕ} (π : Fin n → ℝ) (M : ℝ) (hn : 0 < n) (hMpos : 0 < M) (h0 : ∀ j, 0 ≤ π j) (hM : ∀ j, π j ≤ M) (hsum : ∑ j, π j = 1) :
    (n : ℝ) * ∑ j, π j ^ 2 ≤ Real.exp (Real.log n - (- Real.log M))

Min-entropy controls the uniform-proposal cost; with the dilution bound (rsd-max-weight-bound) this recovers n·Σπ² ≤ e^R. Part of 'Entropy–Cost Law: attention compute cost is n e^{-H2}, so stability regularizers also buy speed'.

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 16 lines of Lean

Cost factor ≤ exp(log n − H_∞): n·Σπ² ≤ n·max π

How it was proved

More proved statements