Proveddrift-lawdecodevariational-cost-lawreview-response
Bayes-optimal snapshot proposal for normalised attention: the least expected cost factor under a drift prior is (Σ_j e^{s_j}√E[e^{2δ_j}/Z_δ²])², attained by r_j ∝ e^{s_j}√E[e^{2δ_j}/Z_δ²]
For all , , , , and , if , , , and (given , and ), then
What the notation means · or point at any symbol
- Type
- Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (
Type) - x ∈ TVariables
- Ω, J range over Type: the statement must hold for every choice. (
(Ω J : Type)) - Fintype
- Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (
Fintype) - f(x)Function application
- Fintype applied to Ω. Lean writes f x; papers write f(x). (
Fintype Ω) - Nonempty
- Nonempty: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (
Nonempty) - ⟹Implies
- If Ω, then ℝ. (Between types, A → B is the type of functions from A to B.) (
Ω → ℝ) - ∀For all
- The statement after the comma holds for every ω. (
∀ ω, 0 ≤ lam ω) - ∑Summation
- Add up lam(ω) for every ω. (
∑ ω, lam ω) - =Equality
- ∑ ω, lam(ω) and 1 are the same. (
∑ ω, lam ω = 1) - ∧And
- Both hold: (∑ j, exp(s(j)) · √(∑ ω, lam(ω) · (exp(2 · δ(ω, j))/Z(ω)^2…, and (∑ ω, lam(ω) · (∑ j, (exp(s(j) + δ(ω, j))/Z(ω))^2/(exp(s(j…. (
(∑ j, Real.exp (s j) * Real.sqrt (∑ ω, …) - RReal numbers
- The real numbers: every point on the number line. (
ℝ) - ≤At most
- 0 is less than or equal to lam(ω). (
0 ≤ lam ω) - <Less than
- 0 is strictly smaller than Z(ω). (
0 < Z ω) - eˣExponential function
- e raised to the power s(j), where e ≈ 2.718. (
Real.exp (s j)) - √Square root
- The non-negative number whose square is ∑ ω, lam(ω) · (exp(2 · δ(ω, j))/Z(ω)^2). (
Real.sqrt (∑ ω, lam ω * (Real.exp (2 * …) - a/bDivision
- exp(2 · δ(ω, j)) divided by Z(ω)^2. In Lean, dividing by 0 gives 0. (
Real.exp (2 * δ ω j) / Z ω ^ 2) - ·Multiplication
- 2 times δ(ω, j). (
2 * δ ω j) - aᵇPower
- Z(ω) raised to the power 2. (
Z ω ^ 2) - +Addition
- s(j) plus δ(ω, j). (
s j + δ ω j)
theorem drift-bayes-optimal-snapshot-proposal-normalized {Ω J : Type} [Fintype Ω] [Fintype J] [Nonempty J] (lam : Ω → ℝ) (s : J → ℝ) (δ : Ω → J → ℝ) (Z : Ω → ℝ) (r : J → ℝ) (hlam : ∀ ω, 0 ≤ lam ω) (hlams : ∑ ω, lam ω = 1) (hZ : ∀ ω, 0 < Z ω) (hr : ∀ j, 0 < r j) (hrs : ∑ j, r j = 1) :
(∑ j, Real.exp (s j) * Real.sqrt (∑ ω, lam ω * (Real.exp (2 * δ ω j) / Z ω ^ 2))) ^ 2 ≤ ∑ ω, lam ω * ∑ j, (Real.exp (s j + δ ω j) / Z ω) ^ 2 / r j ∧ ∑ ω, lam ω * ∑ j, (Real.exp (s j + δ ω j) / Z ω) ^ 2 / (Real.exp (s j) * Real.sqrt (∑ ω', lam ω' * (Real.exp (2 * δ ω' j) / Z ω' ^ 2)) / ∑ i, Real.exp (s i) * Real.sqrt (∑ ω', lam ω' * (Real.exp (2 * δ ω' i) / Z ω' ^ 2))) = (∑ j, Real.exp (s j) * Real.sqrt (∑ ω, lam ω * (Real.exp (2 * δ ω j) / Z ω ^ 2))) ^ 2Corrects an overreach noted by an external reviewer (2026-10-05): drift-bayes-optimal-snapshot-proposal optimises the second moment of the UNNORMALISED numerator, and its optimiser is not invariant to a common score shift, so it is not the optimal decode proposal for normalised attention. This instance of the variational cost law keeps the random normaliser Z_δ inside the expectation. The optimiser now depends only on the attention rows, as it must.
How it was proved
- Proved
- 5 October 2026
- By
- Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
- Checked by
- Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
- Axioms
- Classical.choice, Quot.sound, propext
- Proof
- 74 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.