proofs
Provedsampleability-lawself-certifying-proposalimportance-samplingattention-compute-law

Exponential-family proposal: log Σπ²/r_θ = log Σπ²e^{−θf} + log Σe^{θf}, a sum of two log-sum-exps (convex in θ)

For all J∈Type⁡\htmlData{mx=1}{J \in \htmlData{mx=0}{\operatorname{Type}}}, π,f:J→R\htmlData{mx=3}{\pi, f : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}} and θ∈R\htmlData{mx=1}{\theta \in \htmlData{mx=0}{\mathbb{R}}}, if ∀j, 0<π(j)\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})}} (given Fintype⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{J}} and Nonempty⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Nonempty}}\,\htmlData{mx=1}{J}}), then

log⁡(∑jπ(j)2eθ⋅f(j)∑keθ⋅f(k))=log⁡(∑jπ(j)2⋅e−θ⋅f(j))+log⁡(∑keθ⋅f(k))\begin{aligned} & \htmlData{mx=21}{\log \left(\htmlData{mx=20}{\sum_{j} \htmlData{mx=19}{\frac{\htmlData{mx=4}{\htmlData{mx=3}{\htmlData{mx=1}{\pi}(\htmlData{mx=2}{j})}^{2}}}{\htmlData{mx=18}{\frac{\htmlData{mx=10}{e^{\htmlData{mx=9}{\htmlData{mx=5}{\theta} \cdot \htmlData{mx=8}{\htmlData{mx=6}{f}(\htmlData{mx=7}{j})}}}}}{\htmlData{mx=17}{\sum_{k} \htmlData{mx=16}{e^{\htmlData{mx=15}{\htmlData{mx=11}{\theta} \cdot \htmlData{mx=14}{\htmlData{mx=12}{f}(\htmlData{mx=13}{k})}}}}}}}}}}\right)} \\ & \quad \htmlData{mx=0}{=} \htmlData{mx=44}{\htmlData{mx=35}{\log \left(\htmlData{mx=34}{\sum_{j} \htmlData{mx=33}{\htmlData{mx=25}{\htmlData{mx=24}{\htmlData{mx=22}{\pi}(\htmlData{mx=23}{j})}^{2}} \cdot \htmlData{mx=32}{e^{\htmlData{mx=31}{-\htmlData{mx=30}{\htmlData{mx=26}{\theta} \cdot \htmlData{mx=29}{\htmlData{mx=27}{f}(\htmlData{mx=28}{j})}}}}}}}\right)} + \htmlData{mx=43}{\log \left(\htmlData{mx=42}{\sum_{k} \htmlData{mx=41}{e^{\htmlData{mx=40}{\htmlData{mx=36}{\theta} \cdot \htmlData{mx=39}{\htmlData{mx=37}{f}(\htmlData{mx=38}{k})}}}}}\right)}} \end{aligned}
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to J. Lean writes f x; papers write f(x). (Fintype J)
Nonempty
Nonempty: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Nonempty)
⟹Implies
If J, then ℝ. (Between types, A → B is the type of functions from A to B.) (J → ℝ)
x : TVariables
π, f range over J → ℝ: the statement must hold for every choice. ((π f : J → ℝ))
∀For all
The statement after the comma holds for every j. (∀ j, 0 < π j)
logNatural logarithm
The logarithm of ∑ j, π(j)^2/(exp(θ · f(j))/(∑ k, exp(θ · f(k)))) to base e: the power e must be raised to to give ∑ j, π(j)^2/(exp(θ · f(j))/(∑ k, exp(θ · f(k)))). (Real.log (∑ j, π j ^ 2 / (Real.exp (θ *…)
=Equality
log(∑ j, π(j)^2/(exp(θ · f(j))/(∑ k, exp(θ · f(k))))) and log(∑ j, π(j)^2 · exp(−(θ · f(j)))) + log(∑ k, exp(θ · f(k… are the same. (Real.log (∑ j, π j ^ 2 / (Real.exp (θ *…)
RReal numbers
The real numbers: every point on the number line. (ℝ)
<Less than
0 is strictly smaller than π(j). (0 < π j)
πPi
The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
aᵇPower
π(j) raised to the power 2. (π j ^ 2)
eˣExponential function
e raised to the power θ · f(j), where e ≈ 2.718. (Real.exp (θ * f j))
a/bDivision
exp(θ · f(j)) divided by ∑ k, exp(θ · f(k)). In Lean, dividing by 0 gives 0. (Real.exp (θ * f j) / ∑ k, Real.exp (θ *…)
·Multiplication
θ times f(j). (θ * f j)
∑Summation
Add up exp(θ · f(k)) for every k. (∑ k, Real.exp (θ * f k))
+Addition
log(∑ j, π(j)^2 · exp(−(θ · f(j)))) plus log(∑ k, exp(θ · f(k))). (Real.log (∑ j, π j ^ 2 * Real.exp (-(θ …)
−xNegation
The negative of θ · f(j). (-(θ * f j))

For a score-shaped proposal r_θ ∝ e^{θ f}, the log cost factor decomposes into two log-sum-exp terms, each convex in θ; gradient training of the proposal has no spurious local minima. Part of 'The Sampleability Law: a transformer can be trained to make its own attention cheap to sample, and the samples certify the saving' (self-certifying learned proposals inside the Attention Compute Law).

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
18 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements