proofs
Provedunbiased-attentionend-to-endsoftmaxeasy-side

Independent unbiased estimates: E[A·B] = ab and E[(AB − ab)²] = Va·Vb + a²Vb + b²Va (exact product of two finite spaces)

For all Ω1,Ω2∈Type⁡\htmlData{mx=1}{\Omega_{1}, \Omega_{2} \in \htmlData{mx=0}{\operatorname{Type}}}, w1:Ω1→R\htmlData{mx=3}{w_{1} : \htmlData{mx=0}{\Omega_{1}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, w2:Ω2→R\htmlData{mx=3}{w_{2} : \htmlData{mx=0}{\Omega_{2}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, A:Ω1→R\htmlData{mx=3}{A : \htmlData{mx=0}{\Omega_{1}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, B:Ω2→R\htmlData{mx=3}{B : \htmlData{mx=0}{\Omega_{2}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}} and a,b,Va,Vb∈R\htmlData{mx=1}{a, b, \mathit{Va}, \mathit{Vb} \in \htmlData{mx=0}{\mathbb{R}}}, if ∑xw1(x)=1\htmlData{mx=3}{\sum_{x} \htmlData{mx=2}{\htmlData{mx=0}{w_{1}}(\htmlData{mx=1}{x})}} \mathrel{\htmlData{mx=4}{=}} 1, ∑yw2(y)=1\htmlData{mx=3}{\sum_{y} \htmlData{mx=2}{\htmlData{mx=0}{w_{2}}(\htmlData{mx=1}{y})}} \mathrel{\htmlData{mx=4}{=}} 1, ∑xw1(x)⋅A(x)=a\htmlData{mx=7}{\sum_{x} \htmlData{mx=6}{\htmlData{mx=2}{\htmlData{mx=0}{w_{1}}(\htmlData{mx=1}{x})} \cdot \htmlData{mx=5}{\htmlData{mx=3}{A}(\htmlData{mx=4}{x})}}} \mathrel{\htmlData{mx=9}{=}} \htmlData{mx=8}{a}, ∑yw2(y)⋅B(y)=b\htmlData{mx=7}{\sum_{y} \htmlData{mx=6}{\htmlData{mx=2}{\htmlData{mx=0}{w_{2}}(\htmlData{mx=1}{y})} \cdot \htmlData{mx=5}{\htmlData{mx=3}{B}(\htmlData{mx=4}{y})}}} \mathrel{\htmlData{mx=9}{=}} \htmlData{mx=8}{b}, ∑xw1(x)⋅(A(x)−a)2=Va\htmlData{mx=10}{\sum_{x} \htmlData{mx=9}{\htmlData{mx=2}{\htmlData{mx=0}{w_{1}}(\htmlData{mx=1}{x})} \cdot \htmlData{mx=8}{(\htmlData{mx=7}{\htmlData{mx=5}{\htmlData{mx=3}{A}(\htmlData{mx=4}{x})} - \htmlData{mx=6}{a}})^{2}}}} \mathrel{\htmlData{mx=12}{=}} \htmlData{mx=11}{\mathit{Va}} and ∑yw2(y)⋅(B(y)−b)2=Vb\htmlData{mx=10}{\sum_{y} \htmlData{mx=9}{\htmlData{mx=2}{\htmlData{mx=0}{w_{2}}(\htmlData{mx=1}{y})} \cdot \htmlData{mx=8}{(\htmlData{mx=7}{\htmlData{mx=5}{\htmlData{mx=3}{B}(\htmlData{mx=4}{y})} - \htmlData{mx=6}{b}})^{2}}}} \mathrel{\htmlData{mx=12}{=}} \htmlData{mx=11}{\mathit{Vb}} (given Fintype⁡ Ω1\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{\Omega_{1}}} and Fintype⁡ Ω2\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{\Omega_{2}}}), then

(∑x∑yw1(x)⋅w2(y)⋅(A(x)⋅B(y))=a⋅b)∧(∑x∑yw1(x)⋅w2(y)⋅(A(x)⋅B(y)−a⋅b)2=Va⋅Vb+a2⋅Vb+b2⋅Va)\begin{aligned} & \left(\htmlData{mx=17}{\sum_{x} \htmlData{mx=16}{\sum_{y} \htmlData{mx=15}{\htmlData{mx=7}{\htmlData{mx=3}{\htmlData{mx=1}{w_{1}}(\htmlData{mx=2}{x})} \cdot \htmlData{mx=6}{\htmlData{mx=4}{w_{2}}(\htmlData{mx=5}{y})}} \cdot (\htmlData{mx=14}{\htmlData{mx=10}{\htmlData{mx=8}{A}(\htmlData{mx=9}{x})} \cdot \htmlData{mx=13}{\htmlData{mx=11}{B}(\htmlData{mx=12}{y})}})}}} \mathrel{\htmlData{mx=21}{=}} \htmlData{mx=20}{\htmlData{mx=18}{a} \cdot \htmlData{mx=19}{b}}\right) \\ & \htmlData{mx=0}{\land} \left(\right. \htmlData{mx=44}{\sum_{x} \htmlData{mx=43}{\sum_{y} \htmlData{mx=42}{\htmlData{mx=29}{\htmlData{mx=25}{\htmlData{mx=23}{w_{1}}(\htmlData{mx=24}{x})} \cdot \htmlData{mx=28}{\htmlData{mx=26}{w_{2}}(\htmlData{mx=27}{y})}} \cdot \htmlData{mx=41}{(\htmlData{mx=40}{\htmlData{mx=36}{\htmlData{mx=32}{\htmlData{mx=30}{A}(\htmlData{mx=31}{x})} \cdot \htmlData{mx=35}{\htmlData{mx=33}{B}(\htmlData{mx=34}{y})}} - \htmlData{mx=39}{\htmlData{mx=37}{a} \cdot \htmlData{mx=38}{b}}})^{2}}}}} \\ & \quad \htmlData{mx=22}{=} \htmlData{mx=57}{\htmlData{mx=52}{\htmlData{mx=47}{\htmlData{mx=45}{\mathit{Va}} \cdot \htmlData{mx=46}{\mathit{Vb}}} + \htmlData{mx=51}{\htmlData{mx=49}{\htmlData{mx=48}{a}^{2}} \cdot \htmlData{mx=50}{\mathit{Vb}}}} + \htmlData{mx=56}{\htmlData{mx=54}{\htmlData{mx=53}{b}^{2}} \cdot \htmlData{mx=55}{\mathit{Va}}}} \left.\right) \end{aligned}
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
x ∈ TVariables
Ω₁, Ω₂ range over Type: the statement must hold for every choice. ((Ω₁ Ω₂ : Type))
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to Ω₁. Lean writes f x; papers write f(x). (Fintype Ω₁)
⟹Implies
If Ω₁, then ℝ. (Between types, A → B is the type of functions from A to B.) (Ω₁ → ℝ)
∑Summation
Add up w₁(x) for every x. (∑ x, w₁ x)
=Equality
∑ x, w₁(x) and 1 are the same. (∑ x, w₁ x = 1)
∧And
Both hold: (∑ x, ∑ y, w₁(x) · w₂(y) · (A(x) · B(y))) = a · b, and (∑ x, ∑ y, w₁(x) · w₂(y) · (A(x) · B(y) − a · b)^2) = Va ·…. ((∑ x, ∑ y, w₁ x * w₂ y * (A x * B y) = …)
RReal numbers
The real numbers: every point on the number line. (ℝ)
·Multiplication
w₁(x) times A(x). (w₁ x * A x)
aᵇPower
A(x) − a raised to the power 2. ((A x - a) ^ 2)
-Subtraction
A(x) minus a. On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (A x - a)
+Addition
Va · Vb plus a^2 · Vb. (Va * Vb + a ^ 2 * Vb)

The corrected gradient-product statement after external review: ua-product-estimate-error needs |b̂| ≤ F, which MLMC-debiased estimates violate. For independent debiased estimates of E[uk], E[u], E[k] the product is exactly unbiased with this variance, so the query gradient (a covariance) is estimated unbiasedly with constant-factor overhead. Part of 'Unbiased randomized attention: cost x variance thresholds' (end-to-end composition round).

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
36 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements