proofs
Provedunbiased-attentionend-to-endsoftmaxeasy-side

Vector-valued SNIS: coordinates |v_jc| ≤ F, one set of N i.i.d. draws for all d coordinates ⇒ Σ_c MSE_c ≤ d · 16F²(Σπ²/r)/N

For all n,N,d∈N\htmlData{mx=1}{n, N, d \in \htmlData{mx=0}{\mathbb{N}}}, r,π:Fin⁡ n→R\htmlData{mx=5}{r, \pi : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}, v:Fin⁡ n→Fin⁡ d→R\htmlData{mx=9}{v : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=8}{\to}} \htmlData{mx=5}{\htmlData{mx=3}{\operatorname{Fin}}\,\htmlData{mx=4}{d}} \mathrel{\htmlData{mx=7}{\to}} \htmlData{mx=6}{\mathbb{R}}} and F∈R\htmlData{mx=1}{F \in \htmlData{mx=0}{\mathbb{R}}}, if 0<N0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{N}, ∀j, 0<r(j)\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}}, ∑jr(j)=1\htmlData{mx=3}{\sum_{j} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} \mathrel{\htmlData{mx=4}{=}} 1, ∀j, 0<π(j)\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})}}, ∑jπ(j)=1\htmlData{mx=3}{\sum_{j} \htmlData{mx=2}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{j})}} \mathrel{\htmlData{mx=4}{=}} 1 and ∀j,c, ∣v(j,c)∣≤F\htmlData{mx=7}{\forall j, c,\ \htmlData{mx=4}{\left\lvert \htmlData{mx=3}{\htmlData{mx=0}{v}(\htmlData{mx=1}{j}, \htmlData{mx=2}{c})} \right\rvert} \mathrel{\htmlData{mx=6}{\leq}} \htmlData{mx=5}{F}}, then

∑c∑f:Fin⁡ N→Fin⁡ n(∏kr(f(k)))⋅(∑iπ(f(i))r(f(i))⋅v(f(i),c)∑iπ(f(i))r(f(i))−∑jπ(j)⋅v(j,c))2≤d⋅(16⋅F2⋅(∑jπ(j)2r(j))N)\begin{aligned} & \htmlData{mx=59}{\sum_{c} \htmlData{mx=58}{\sum_{f : \htmlData{mx=3}{\htmlData{mx=1}{\operatorname{Fin}}\,\htmlData{mx=2}{N}} \mathrel{\htmlData{mx=7}{\to}} \htmlData{mx=6}{\htmlData{mx=4}{\operatorname{Fin}}\,\htmlData{mx=5}{n}}} \htmlData{mx=57}{\left(\htmlData{mx=13}{\prod_{k} \htmlData{mx=12}{\htmlData{mx=8}{r}(\htmlData{mx=11}{\htmlData{mx=9}{f}(\htmlData{mx=10}{k})})}}\right) \cdot \htmlData{mx=56}{\left(\htmlData{mx=55}{\htmlData{mx=45}{\frac{\htmlData{mx=32}{\sum_{i} \htmlData{mx=31}{\htmlData{mx=24}{\frac{\htmlData{mx=18}{\htmlData{mx=14}{\pi}(\htmlData{mx=17}{\htmlData{mx=15}{f}(\htmlData{mx=16}{i})})}}{\htmlData{mx=23}{\htmlData{mx=19}{r}(\htmlData{mx=22}{\htmlData{mx=20}{f}(\htmlData{mx=21}{i})})}}} \cdot \htmlData{mx=30}{\htmlData{mx=25}{v}(\htmlData{mx=28}{\htmlData{mx=26}{f}(\htmlData{mx=27}{i})}, \htmlData{mx=29}{c})}}}}{\htmlData{mx=44}{\sum_{i} \htmlData{mx=43}{\frac{\htmlData{mx=37}{\htmlData{mx=33}{\pi}(\htmlData{mx=36}{\htmlData{mx=34}{f}(\htmlData{mx=35}{i})})}}{\htmlData{mx=42}{\htmlData{mx=38}{r}(\htmlData{mx=41}{\htmlData{mx=39}{f}(\htmlData{mx=40}{i})})}}}}}} - \htmlData{mx=54}{\sum_{j} \htmlData{mx=53}{\htmlData{mx=48}{\htmlData{mx=46}{\pi}(\htmlData{mx=47}{j})} \cdot \htmlData{mx=52}{\htmlData{mx=49}{v}(\htmlData{mx=50}{j}, \htmlData{mx=51}{c})}}}}\right)^{2}}}}} \\ & \quad \htmlData{mx=0}{\leq} \htmlData{mx=76}{\htmlData{mx=60}{d} \cdot \left(\htmlData{mx=75}{\frac{\htmlData{mx=73}{\htmlData{mx=63}{16 \cdot \htmlData{mx=62}{\htmlData{mx=61}{F}^{2}}} \cdot \left(\htmlData{mx=72}{\sum_{j} \htmlData{mx=71}{\frac{\htmlData{mx=67}{\htmlData{mx=66}{\htmlData{mx=64}{\pi}(\htmlData{mx=65}{j})}^{2}}}{\htmlData{mx=70}{\htmlData{mx=68}{r}(\htmlData{mx=69}{j})}}}}\right)}}{\htmlData{mx=74}{N}}}\right)} \end{aligned}
What the notation means · or point at any symbol
NNatural numbers
The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
x ∈ TVariables
n, N, d range over natural numbers: the statement must hold for every choice. ((n N d : ℕ))
<Less than
0 is strictly smaller than N. (0 < N)
Fin
Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application
Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
⟹Implies
If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
x : TVariables
r, π range over Fin n → ℝ: the statement must hold for every choice. ((r π : Fin n → ℝ))
∀For all
The statement after the comma holds for every j. (∀ j, 0 < r j)
∑Summation
Add up r(j) for every j. (∑ j, r j)
=Equality
∑ j, r(j) and 1 are the same. (∑ j, r j = 1)
|x|Absolute value
The size of v(j, c) without its sign. (|v j c|)
≤At most
|v(j, c)| is less than or equal to F. (|v j c| ≤ F)
RReal numbers
The real numbers: every point on the number line. (ℝ)
πPi
The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
∏Product
Multiply together r(f(k)) for every k. (∏ k, r (f k))
aᵇPower
(∑ i, π(f(i))/r(f(i)) · v(f(i), c))/(∑ i, π(f(i))/r(f(i)))… raised to the power 2. (((∑ i, π (f i) / r (f i) * v (f i) c) /…)
-Subtraction
(∑ i, π(f(i))/r(f(i)) · v(f(i), c))/(∑ i, π(f(i))/r(f(i))) minus ∑ j, π(j) · v(j, c). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. ((∑ i, π (f i) / r (f i) * v (f i) c) / …)
a/bDivision
π(f(i)) divided by r(f(i)). In Lean, dividing by 0 gives 0. (π (f i) / r (f i))
·Multiplication
π(f(i))/r(f(i)) times v(f(i), c). (π (f i) / r (f i) * v (f i) c)

Closes the external review's 'the SNIS bound is scalar' item: the same draws serve every value coordinate, and the summed squared error over d coordinates is at most d times the scalar bound (a per-coordinate application of ua-snis-mse-bound, inlined). With a Euclidean norm bound ‖v_j‖₂ ≤ F the factor d can be removed by the same argument with F_c² summing to F²; stated here in the per-coordinate form. Part of 'Unbiased randomized attention: cost x variance thresholds' (end-to-end composition round).

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
171 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements