proofs
Provedvariational-cost-lawinformationheadline-theoremfirst-principles

Information law: for a row drawn from a prior and any proposal, E[log(1+χ²)] ≥ I(J; row) and E[1+χ²] ≥ exp I(J; row), where J ~ π is a key drawn from the row

For all I,J∈Type⁡\htmlData{mx=1}{I, J \in \htmlData{mx=0}{\operatorname{Type}}}, lam:I→R\htmlData{mx=3}{\mathit{lam} : \htmlData{mx=0}{I} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, π:I→J→R\htmlData{mx=5}{\pi : \htmlData{mx=0}{I} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=1}{J} \mathrel{\htmlData{mx=3}{\to}} \htmlData{mx=2}{\mathbb{R}}} and r:J→R\htmlData{mx=3}{r : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, if ∀i, 0≤lam(i)\htmlData{mx=4}{\forall i,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{\mathit{lam}}(\htmlData{mx=1}{i})}}, ∑ilam(i)=1\htmlData{mx=3}{\sum_{i} \htmlData{mx=2}{\htmlData{mx=0}{\mathit{lam}}(\htmlData{mx=1}{i})}} \mathrel{\htmlData{mx=4}{=}} 1, ∀i,j, 0<π(i,j)\htmlData{mx=5}{\forall i, j,\ 0 \mathrel{\htmlData{mx=4}{<}} \htmlData{mx=3}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{i}, \htmlData{mx=2}{j})}}, ∀i, ∑jπ(i,j)=1\htmlData{mx=6}{\forall i,\ \htmlData{mx=4}{\sum_{j} \htmlData{mx=3}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{i}, \htmlData{mx=2}{j})}} \mathrel{\htmlData{mx=5}{=}} 1}, ∀j, 0<r(j)\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} and ∑jr(j)=1\htmlData{mx=3}{\sum_{j} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} \mathrel{\htmlData{mx=4}{=}} 1 (given Fintype⁡ I\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{I}}, Fintype⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{J}} and Nonempty⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Nonempty}}\,\htmlData{mx=1}{J}}), then

∑ilam(i)⋅∑jπ(i,j)⋅log⁡(π(i,j)∑klam(k)⋅π(k,j))≤∑ilam(i)⋅log⁡(∑jπ(i,j)2r(j))∧e∑ilam(i)⋅∑jπ(i,j)⋅log⁡(π(i,j)∑klam(k)⋅π(k,j))≤∑ilam(i)⋅∑jπ(i,j)2r(j)\begin{aligned} & \htmlData{mx=27}{\sum_{i} \htmlData{mx=26}{\htmlData{mx=4}{\htmlData{mx=2}{\mathit{lam}}(\htmlData{mx=3}{i})} \cdot \htmlData{mx=25}{\sum_{j} \htmlData{mx=24}{\htmlData{mx=8}{\htmlData{mx=5}{\pi}(\htmlData{mx=6}{i}, \htmlData{mx=7}{j})} \cdot \htmlData{mx=23}{\log \left(\htmlData{mx=22}{\frac{\htmlData{mx=12}{\htmlData{mx=9}{\pi}(\htmlData{mx=10}{i}, \htmlData{mx=11}{j})}}{\htmlData{mx=21}{\sum_{k} \htmlData{mx=20}{\htmlData{mx=15}{\htmlData{mx=13}{\mathit{lam}}(\htmlData{mx=14}{k})} \cdot \htmlData{mx=19}{\htmlData{mx=16}{\pi}(\htmlData{mx=17}{k}, \htmlData{mx=18}{j})}}}}}\right)}}}}} \\ & \quad \htmlData{mx=1}{\leq} \htmlData{mx=43}{\sum_{i} \htmlData{mx=42}{\htmlData{mx=30}{\htmlData{mx=28}{\mathit{lam}}(\htmlData{mx=29}{i})} \cdot \htmlData{mx=41}{\log \left(\htmlData{mx=40}{\sum_{j} \htmlData{mx=39}{\frac{\htmlData{mx=35}{\htmlData{mx=34}{\htmlData{mx=31}{\pi}(\htmlData{mx=32}{i}, \htmlData{mx=33}{j})}^{2}}}{\htmlData{mx=38}{\htmlData{mx=36}{r}(\htmlData{mx=37}{j})}}}}\right)}}} \\ & \htmlData{mx=0}{\land} \htmlData{mx=71}{e^{\htmlData{mx=70}{\sum_{i} \htmlData{mx=69}{\htmlData{mx=47}{\htmlData{mx=45}{\mathit{lam}}(\htmlData{mx=46}{i})} \cdot \htmlData{mx=68}{\sum_{j} \htmlData{mx=67}{\htmlData{mx=51}{\htmlData{mx=48}{\pi}(\htmlData{mx=49}{i}, \htmlData{mx=50}{j})} \cdot \htmlData{mx=66}{\log \left(\htmlData{mx=65}{\frac{\htmlData{mx=55}{\htmlData{mx=52}{\pi}(\htmlData{mx=53}{i}, \htmlData{mx=54}{j})}}{\htmlData{mx=64}{\sum_{k} \htmlData{mx=63}{\htmlData{mx=58}{\htmlData{mx=56}{\mathit{lam}}(\htmlData{mx=57}{k})} \cdot \htmlData{mx=62}{\htmlData{mx=59}{\pi}(\htmlData{mx=60}{k}, \htmlData{mx=61}{j})}}}}}\right)}}}}}}} \\ & \quad \htmlData{mx=44}{\leq} \htmlData{mx=86}{\sum_{i} \htmlData{mx=85}{\htmlData{mx=74}{\htmlData{mx=72}{\mathit{lam}}(\htmlData{mx=73}{i})} \cdot \htmlData{mx=84}{\sum_{j} \htmlData{mx=83}{\frac{\htmlData{mx=79}{\htmlData{mx=78}{\htmlData{mx=75}{\pi}(\htmlData{mx=76}{i}, \htmlData{mx=77}{j})}^{2}}}{\htmlData{mx=82}{\htmlData{mx=80}{r}(\htmlData{mx=81}{j})}}}}}} \end{aligned}
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
x ∈ TVariables
I, J range over Type: the statement must hold for every choice. ((I J : Type))
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to I. Lean writes f x; papers write f(x). (Fintype I)
Nonempty
Nonempty: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Nonempty)
⟹Implies
If I, then ℝ. (Between types, A → B is the type of functions from A to B.) (I → ℝ)
∀For all
The statement after the comma holds for every i. (∀ i, 0 ≤ lam i)
∑Summation
Add up lam(i) for every i. (∑ i, lam i)
=Equality
∑ i, lam(i) and 1 are the same. (∑ i, lam i = 1)
∧And
Both hold: (∑ i, lam(i) · (∑ j, π(i, j) · log(π(i, j)/(∑ k, lam(k) · …, and exp(∑ i, lam(i) · (∑ j, π(i, j) · log(π(i, j)/(∑ k, lam(k)…. (∑ i, lam i * ∑ j, π i j * Real.log (π i…)
RReal numbers
The real numbers: every point on the number line. (ℝ)
≤At most
0 is less than or equal to lam(i). (0 ≤ lam i)
<Less than
0 is strictly smaller than π(i, j). (0 < π i j)
πPi
The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
logNatural logarithm
The logarithm of π(i, j)/(∑ k, lam(k) · π(k, j)) to base e: the power e must be raised to to give π(i, j)/(∑ k, lam(k) · π(k, j)). (Real.log (π i j / ∑ k, lam k * π k j))
a/bDivision
π(i, j) divided by ∑ k, lam(k) · π(k, j). In Lean, dividing by 0 gives 0. (π i j / ∑ k, lam k * π k j)
·Multiplication
lam(k) times π(k, j). (lam k * π k j)
aᵇPower
π(i, j) raised to the power 2. (π i j ^ 2)
eˣExponential function
e raised to the power ∑ i, lam(i) · (∑ j, π(i, j) · log(π(i, j)/(∑ k, lam(k) · π…, where e ≈ 2.718. (Real.exp (∑ i, lam i * ∑ j, π i j * Rea…)

Shannon form of the variational cost law. For a mixture of positive rows π_i with weights λ_i (the posterior over rows given whatever side information the proposal was built from) and any proposal r: Σ_i λ_i Σ_j π_ij log(π_ij/p̄_j) ≤ Σ_i λ_i log(Σ_j π_ij²/r_j), where p̄ = Σ_i λ_i π_i is the posterior predictive, and the exponential of the left side bounds the expected cost factor. The left side is the mutual information between a sampled key and the row, so, in bits, log₂(cost) ≥ (bits needed to predict the sample from the row) − (bits the side information carries about the sample). Proof: Jensen for log per row, then Gibbs' inequality for the predictive. This is the general form of the argument in the polylog-query lower bound. First-principles reformulation, 2026-10-05.

How it was proved

Proved
5 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
93 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements