Provedsampleability-lawself-certifying-proposalimportance-samplingattention-compute-law

Proposal gradient: d/dθ Σπ²/r_θ = −Σ r_θ (π/r_θ)² (log r_θ)′, an expectation under r_θ of the same importance ratio

For all $\htmlData{mx=1}{J \in \htmlData{mx=0}{\operatorname{Type}}}$ , $\htmlData{mx=3}{\pi : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}$ , $\htmlData{mx=5}{r, r' : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=1}{\mathbb{R}} \mathrel{\htmlData{mx=3}{\to}} \htmlData{mx=2}{\mathbb{R}}}$ , $\htmlData{mx=1}{\theta \in \htmlData{mx=0}{\mathbb{R}}}$ and $\htmlData{mx=13}{\mathit{hd} : \htmlData{mx=12}{\forall j,\ \htmlData{mx=11}{\htmlData{mx=0}{\operatorname{HasDerivAt}}\,\htmlData{mx=5}{t \mapsto \htmlData{mx=4}{\htmlData{mx=1}{r}(\htmlData{mx=2}{j}, \htmlData{mx=3}{t})}}\,\htmlData{mx=9}{\htmlData{mx=6}{r'}(\htmlData{mx=7}{j}, \htmlData{mx=8}{\theta})}\,\htmlData{mx=10}{\theta}}}}$ , if $\htmlData{mx=5}{\forall j,\ 0 \mathrel{\htmlData{mx=4}{<}} \htmlData{mx=3}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j}, \htmlData{mx=2}{\theta})}}$ (given $\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{J}}$ ), then

\htmlData{mx=39}{\htmlData{mx=0}{\operatorname{HasDerivAt}}\,\htmlData{mx=11}{t \mapsto \htmlData{mx=10}{\sum_{j} \htmlData{mx=9}{\frac{\htmlData{mx=4}{\htmlData{mx=3}{\htmlData{mx=1}{\pi}(\htmlData{mx=2}{j})}^{2}}}{\htmlData{mx=8}{\htmlData{mx=5}{r}(\htmlData{mx=6}{j}, \htmlData{mx=7}{t})}}}}}\,\htmlData{mx=37}{-\htmlData{mx=36}{\sum_{j} \htmlData{mx=35}{\htmlData{mx=25}{\htmlData{mx=15}{\htmlData{mx=12}{r}(\htmlData{mx=13}{j}, \htmlData{mx=14}{\theta})} \cdot \htmlData{mx=24}{\left(\htmlData{mx=23}{\frac{\htmlData{mx=18}{\htmlData{mx=16}{\pi}(\htmlData{mx=17}{j})}}{\htmlData{mx=22}{\htmlData{mx=19}{r}(\htmlData{mx=20}{j}, \htmlData{mx=21}{\theta})}}}\right)^{2}}} \cdot \left(\htmlData{mx=34}{\frac{\htmlData{mx=29}{\htmlData{mx=26}{r'}(\htmlData{mx=27}{j}, \htmlData{mx=28}{\theta})}}{\htmlData{mx=33}{\htmlData{mx=30}{r}(\htmlData{mx=31}{j}, \htmlData{mx=32}{\theta})}}}\right)}}}\,\htmlData{mx=38}{\theta}}

What the notation means · or point at any symbol

Type: Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
Fintype: Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application: Fintype applied to J. Lean writes f x; papers write f(x). (Fintype J)
⟹Implies: If J, then ℝ. (Between types, A → B is the type of functions from A to B.) (J → ℝ)
x : TVariables: r, r' range over J → ℝ → ℝ: the statement must hold for every choice. ((r r' : J → ℝ → ℝ))
∀For all: The statement after the comma holds for every j. (∀ j, 0 < r j θ)
RReal numbers: The real numbers: every point on the number line. (ℝ)
<Less than: 0 is strictly smaller than r(j, θ). (0 < r j θ)
HasDerivAt: HasDerivAt: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (HasDerivAt)
x ↦ f(x)Function (λ-abstraction): The function sending t to r(j, t). Lean writes fun x => …; papers write x ↦ …. (fun t => r j t)
∑Summation: Add up π(j)^2/r(j, t) for every j. (∑ j, π j ^ 2 / r j t)
πPi: The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
aᵇPower: π(j) raised to the power 2. (π j ^ 2)
a/bDivision: π(j)^2 divided by r(j, t). In Lean, dividing by 0 gives 0. (π j ^ 2 / r j t)
−xNegation: The negative of ∑ j, r(j, θ) · (π(j)/r(j, θ))^2 · (r'(j, θ)/r(j, θ)). (-∑ j, r j θ * (π j / r j θ) ^ 2 * (r' j…)
·Multiplication: r(j, θ) times (π(j)/r(j, θ))^2. (r j θ * (π j / r j θ) ^ 2)

theorem sc-cost-gradient {J : Type} [Fintype J] (π : J → ℝ) (r r' : J → ℝ → ℝ) (θ : ℝ) (hr : ∀ j, 0 < r j θ) (hd : ∀ j, HasDerivAt (fun t => r j t) (r' j θ) θ) :
    HasDerivAt (fun t => ∑ j, π j ^ 2 / r j t) (-∑ j, r j θ * (π j / r j θ) ^ 2 * (r' j θ / r j θ)) θ

For a differentiable positive one-parameter proposal family, the derivative of the cost factor is an expectation under the proposal of a function of the importance ratio, so the proposal is trainable from the attention samples at no extra forward cost. Part of 'The Sampleability Law: a transformer can be trained to make its own attention cheap to sample, and the samples certify the saving' (self-certifying learned proposals inside the Attention Compute Law).

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 19 lines of Lean

Proposal gradient: d/dθ Σπ²/r_θ = −Σ r_θ (π/r_θ)² (log r_θ)′, an expectation under r_θ of the same importance ratio

How it was proved

More proved statements