proofs
Proveddrift-lawdecodevariational-cost-lawreview-response

Bayes-optimal snapshot proposal for normalised attention: the least expected cost factor under a drift prior is (Σ_j e^{s_j}√E[e^{2δ_j}/Z_δ²])², attained by r_j ∝ e^{s_j}√E[e^{2δ_j}/Z_δ²]

For all Ω,J∈Type⁡\htmlData{mx=1}{\Omega, J \in \htmlData{mx=0}{\operatorname{Type}}}, lam:Ω→R\htmlData{mx=3}{\mathit{lam} : \htmlData{mx=0}{\Omega} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, s:J→R\htmlData{mx=3}{s : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, δ:Ω→J→R\htmlData{mx=5}{\delta : \htmlData{mx=0}{\Omega} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=1}{J} \mathrel{\htmlData{mx=3}{\to}} \htmlData{mx=2}{\mathbb{R}}}, Z:Ω→R\htmlData{mx=3}{Z : \htmlData{mx=0}{\Omega} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}} and r:J→R\htmlData{mx=3}{r : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, if ∀ω, 0≤lam(ω)\htmlData{mx=4}{\forall \omega,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{\mathit{lam}}(\htmlData{mx=1}{\omega})}}, ∑ωlam(ω)=1\htmlData{mx=3}{\sum_{\omega} \htmlData{mx=2}{\htmlData{mx=0}{\mathit{lam}}(\htmlData{mx=1}{\omega})}} \mathrel{\htmlData{mx=4}{=}} 1, ∀ω, 0<Z(ω)\htmlData{mx=4}{\forall \omega,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{Z}(\htmlData{mx=1}{\omega})}}, ∀j, 0<r(j)\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} and ∑jr(j)=1\htmlData{mx=3}{\sum_{j} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} \mathrel{\htmlData{mx=4}{=}} 1 (given Fintype⁡ Ω\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{\Omega}}, Fintype⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{J}} and Nonempty⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Nonempty}}\,\htmlData{mx=1}{J}}), then

(∑jes(j)⋅∑ωlam(ω)⋅(e2δ(ω,j)Z(ω)2))2≤∑ωlam(ω)⋅∑j(es(j)+δ(ω,j)Z(ω))2r(j)∧∑ωlam(ω)⋅∑j(es(j)+δ(ω,j)Z(ω))2es(j)⋅∑ω′lam(ω′)⋅(e2δ(ω′,j)Z(ω′)2)∑ies(i)⋅∑ω′lam(ω′)⋅(e2δ(ω′,i)Z(ω′)2)=(∑jes(j)⋅∑ωlam(ω)⋅(e2δ(ω,j)Z(ω)2))2\begin{aligned} & \htmlData{mx=25}{\left(\htmlData{mx=24}{\sum_{j} \htmlData{mx=23}{\htmlData{mx=5}{e^{\htmlData{mx=4}{\htmlData{mx=2}{s}(\htmlData{mx=3}{j})}}} \cdot \htmlData{mx=22}{\sqrt{\htmlData{mx=21}{\sum_{\omega} \htmlData{mx=20}{\htmlData{mx=8}{\htmlData{mx=6}{\mathit{lam}}(\htmlData{mx=7}{\omega})} \cdot \left(\htmlData{mx=19}{\frac{\htmlData{mx=14}{e^{\htmlData{mx=13}{2\htmlData{mx=12}{\htmlData{mx=9}{\delta}(\htmlData{mx=10}{\omega}, \htmlData{mx=11}{j})}}}}}{\htmlData{mx=18}{\htmlData{mx=17}{\htmlData{mx=15}{Z}(\htmlData{mx=16}{\omega})}^{2}}}}\right)}}}}}}\right)^{2}} \\ & \quad \htmlData{mx=1}{\leq} \htmlData{mx=49}{\sum_{\omega} \htmlData{mx=48}{\htmlData{mx=28}{\htmlData{mx=26}{\mathit{lam}}(\htmlData{mx=27}{\omega})} \cdot \htmlData{mx=47}{\sum_{j} \htmlData{mx=46}{\frac{\htmlData{mx=42}{\left(\htmlData{mx=41}{\frac{\htmlData{mx=37}{e^{\htmlData{mx=36}{\htmlData{mx=31}{\htmlData{mx=29}{s}(\htmlData{mx=30}{j})} + \htmlData{mx=35}{\htmlData{mx=32}{\delta}(\htmlData{mx=33}{\omega}, \htmlData{mx=34}{j})}}}}}{\htmlData{mx=40}{\htmlData{mx=38}{Z}(\htmlData{mx=39}{\omega})}}}\right)^{2}}}{\htmlData{mx=45}{\htmlData{mx=43}{r}(\htmlData{mx=44}{j})}}}}}} \\ & \htmlData{mx=0}{\land} \htmlData{mx=117}{\sum_{\omega} \htmlData{mx=116}{\htmlData{mx=53}{\htmlData{mx=51}{\mathit{lam}}(\htmlData{mx=52}{\omega})} \cdot \htmlData{mx=115}{\sum_{j} \htmlData{mx=114}{\frac{\htmlData{mx=67}{\left(\htmlData{mx=66}{\frac{\htmlData{mx=62}{e^{\htmlData{mx=61}{\htmlData{mx=56}{\htmlData{mx=54}{s}(\htmlData{mx=55}{j})} + \htmlData{mx=60}{\htmlData{mx=57}{\delta}(\htmlData{mx=58}{\omega}, \htmlData{mx=59}{j})}}}}}{\htmlData{mx=65}{\htmlData{mx=63}{Z}(\htmlData{mx=64}{\omega})}}}\right)^{2}}}{\htmlData{mx=113}{\frac{\htmlData{mx=89}{\htmlData{mx=71}{e^{\htmlData{mx=70}{\htmlData{mx=68}{s}(\htmlData{mx=69}{j})}}} \cdot \htmlData{mx=88}{\sqrt{\htmlData{mx=87}{\sum_{\omega'} \htmlData{mx=86}{\htmlData{mx=74}{\htmlData{mx=72}{\mathit{lam}}(\htmlData{mx=73}{\omega'})} \cdot \left(\htmlData{mx=85}{\frac{\htmlData{mx=80}{e^{\htmlData{mx=79}{2\htmlData{mx=78}{\htmlData{mx=75}{\delta}(\htmlData{mx=76}{\omega'}, \htmlData{mx=77}{j})}}}}}{\htmlData{mx=84}{\htmlData{mx=83}{\htmlData{mx=81}{Z}(\htmlData{mx=82}{\omega'})}^{2}}}}\right)}}}}}}{\htmlData{mx=112}{\sum_{i} \htmlData{mx=111}{\htmlData{mx=93}{e^{\htmlData{mx=92}{\htmlData{mx=90}{s}(\htmlData{mx=91}{i})}}} \cdot \htmlData{mx=110}{\sqrt{\htmlData{mx=109}{\sum_{\omega'} \htmlData{mx=108}{\htmlData{mx=96}{\htmlData{mx=94}{\mathit{lam}}(\htmlData{mx=95}{\omega'})} \cdot \left(\htmlData{mx=107}{\frac{\htmlData{mx=102}{e^{\htmlData{mx=101}{2\htmlData{mx=100}{\htmlData{mx=97}{\delta}(\htmlData{mx=98}{\omega'}, \htmlData{mx=99}{i})}}}}}{\htmlData{mx=106}{\htmlData{mx=105}{\htmlData{mx=103}{Z}(\htmlData{mx=104}{\omega'})}^{2}}}}\right)}}}}}}}}}}}}} \\ & \quad \htmlData{mx=50}{=} \htmlData{mx=141}{\left(\htmlData{mx=140}{\sum_{j} \htmlData{mx=139}{\htmlData{mx=121}{e^{\htmlData{mx=120}{\htmlData{mx=118}{s}(\htmlData{mx=119}{j})}}} \cdot \htmlData{mx=138}{\sqrt{\htmlData{mx=137}{\sum_{\omega} \htmlData{mx=136}{\htmlData{mx=124}{\htmlData{mx=122}{\mathit{lam}}(\htmlData{mx=123}{\omega})} \cdot \left(\htmlData{mx=135}{\frac{\htmlData{mx=130}{e^{\htmlData{mx=129}{2\htmlData{mx=128}{\htmlData{mx=125}{\delta}(\htmlData{mx=126}{\omega}, \htmlData{mx=127}{j})}}}}}{\htmlData{mx=134}{\htmlData{mx=133}{\htmlData{mx=131}{Z}(\htmlData{mx=132}{\omega})}^{2}}}}\right)}}}}}}\right)^{2}} \end{aligned}
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
x ∈ TVariables
Ω, J range over Type: the statement must hold for every choice. ((Ω J : Type))
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to Ω. Lean writes f x; papers write f(x). (Fintype Ω)
Nonempty
Nonempty: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Nonempty)
⟹Implies
If Ω, then ℝ. (Between types, A → B is the type of functions from A to B.) (Ω → ℝ)
∀For all
The statement after the comma holds for every ω. (∀ ω, 0 ≤ lam ω)
∑Summation
Add up lam(ω) for every ω. (∑ ω, lam ω)
=Equality
∑ ω, lam(ω) and 1 are the same. (∑ ω, lam ω = 1)
∧And
Both hold: (∑ j, exp(s(j)) · √(∑ ω, lam(ω) · (exp(2 · δ(ω, j))/Z(ω)^2…, and (∑ ω, lam(ω) · (∑ j, (exp(s(j) + δ(ω, j))/Z(ω))^2/(exp(s(j…. ((∑ j, Real.exp (s j) * Real.sqrt (∑ ω, …)
RReal numbers
The real numbers: every point on the number line. (ℝ)
≤At most
0 is less than or equal to lam(ω). (0 ≤ lam ω)
<Less than
0 is strictly smaller than Z(ω). (0 < Z ω)
eˣExponential function
e raised to the power s(j), where e ≈ 2.718. (Real.exp (s j))
√Square root
The non-negative number whose square is ∑ ω, lam(ω) · (exp(2 · δ(ω, j))/Z(ω)^2). (Real.sqrt (∑ ω, lam ω * (Real.exp (2 * …)
a/bDivision
exp(2 · δ(ω, j)) divided by Z(ω)^2. In Lean, dividing by 0 gives 0. (Real.exp (2 * δ ω j) / Z ω ^ 2)
·Multiplication
2 times δ(ω, j). (2 * δ ω j)
aᵇPower
Z(ω) raised to the power 2. (Z ω ^ 2)
+Addition
s(j) plus δ(ω, j). (s j + δ ω j)

Corrects an overreach noted by an external reviewer (2026-10-05): drift-bayes-optimal-snapshot-proposal optimises the second moment of the UNNORMALISED numerator, and its optimiser is not invariant to a common score shift, so it is not the optimal decode proposal for normalised attention. This instance of the variational cost law keeps the random normaliser Z_δ inside the expectation. The optimiser now depends only on the attention rows, as it must.

How it was proved

Proved
5 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
74 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements