Provedunbiased-attentionrandomized-algorithmssoftmaxaggregate

Cancellation cost: (Σ g_i)² ≤ n Σ g_i² (the ratio n Σ g²/G² is the row-subsampling cost factor)

For all $\htmlData{mx=1}{n \in \htmlData{mx=0}{\mathbb{N}}}$ and $\htmlData{mx=5}{g : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ :

\htmlData{mx=4}{\left(\htmlData{mx=3}{\sum_{i} \htmlData{mx=2}{\htmlData{mx=0}{g}(\htmlData{mx=1}{i})}}\right)^{2}} \mathrel{\htmlData{mx=13}{\leq}} \htmlData{mx=12}{\htmlData{mx=6}{\htmlData{mx=5}{n}} \cdot \htmlData{mx=11}{\sum_{i} \htmlData{mx=10}{\htmlData{mx=9}{\htmlData{mx=7}{g}(\htmlData{mx=8}{i})}^{2}}}}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
⟹Implies: If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
aᵇPower: ∑ i, g(i) raised to the power 2. ((∑ i, g i) ^ 2)
≤At most: (∑ i, g(i))^2 is less than or equal to n · (∑ i, g(i)^2). ((∑ i, g i) ^ 2 ≤ (n : ℝ) * ∑ i, g i ^ 2)
∑Summation: Add up g(i) for every i. (∑ i, g i)
RReal numbers: The real numbers: every point on the number line. (ℝ)
(x : T)As real numbers: n read as an element of real numbers. Mathematicians leave this implicit; Lean needs it to know which kind of number is meant. ((n : ℝ))
·Multiplication: n times ∑ i, g(i)^2. ((n : ℝ) * ∑ i, g i ^ 2)

theorem ua-cancellation-lower {n : ℕ} (g : Fin n → ℝ) :
    (∑ i, g i) ^ 2 ≤ (n : ℝ) * ∑ i, g i ^ 2

For signed per-row gradient contributions, exact-row subsampling has relative second moment n Σ g_i²/(Σ g_i)² ≥ 1; heavy cancellation (small net gradient) is what makes aggregate estimation expensive. Part of 'Unbiased randomized attention: cost x variance thresholds', round 15 (aggregate conjecture refuted: exact-row subsampling).

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 13 lines of Lean

Cancellation cost: (Σ g_i)² ≤ n Σ g_i² (the ratio n Σ g²/G² is the row-subsampling cost factor)

How it was proved

More proved statements