Provedunbiased-attentionrandomized-algorithmssoftmaxaggregate

Exact-row subsampling: for outputs in [0,1], E[(n·o_I)²] = n Σ o_i² ≤ n Σ o_i (relative second moment ≤ n/A)

For all $\htmlData{mx=1}{n \in \htmlData{mx=0}{\mathbb{N}}}$ and $\htmlData{mx=5}{o : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ , if $\htmlData{mx=4}{\forall i,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{o}(\htmlData{mx=1}{i})}}$ and $\htmlData{mx=4}{\forall i,\ \htmlData{mx=2}{\htmlData{mx=0}{o}(\htmlData{mx=1}{i})} \mathrel{\htmlData{mx=3}{\leq}} 1}$ , then

\htmlData{mx=11}{\sum_{i} \htmlData{mx=10}{\htmlData{mx=2}{\frac{1}{\htmlData{mx=1}{\htmlData{mx=0}{n}}}} \cdot \htmlData{mx=9}{(\htmlData{mx=8}{\htmlData{mx=4}{\htmlData{mx=3}{n}} \cdot \htmlData{mx=7}{\htmlData{mx=5}{o}(\htmlData{mx=6}{i})}})^{2}}}} \mathrel{\htmlData{mx=19}{\leq}} \htmlData{mx=18}{\htmlData{mx=13}{\htmlData{mx=12}{n}} \cdot \htmlData{mx=17}{\sum_{i} \htmlData{mx=16}{\htmlData{mx=14}{o}(\htmlData{mx=15}{i})}}}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
⟹Implies: If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
∀For all: The statement after the comma holds for every i. (∀ i, 0 ≤ o i)
∑Summation: Add up 1/n · (n · o(i))^2 for every i. (∑ i, (1 / (n : ℝ)) * ((n : ℝ) * o i) ^ 2)
≤At most: 0 is less than or equal to o(i). (0 ≤ o i)
a/bDivision: 1 divided by n. In Lean, dividing by 0 gives 0. (1 / (n : ℝ))
RReal numbers: The real numbers: every point on the number line. (ℝ)
(x : T)As real numbers: n read as an element of real numbers. Mathematicians leave this implicit; Lean needs it to know which kind of number is meant. ((n : ℝ))
aᵇPower: n · o(i) raised to the power 2. (((n : ℝ) * o i) ^ 2)
·Multiplication: n times o(i). ((n : ℝ) * o i)

theorem ua-row-subsampling-second-moment {n : ℕ} (o : Fin n → ℝ) (h0 : ∀ i, 0 ≤ o i) (h1 : ∀ i, o i ≤ 1) :
    ∑ i, (1 / (n : ℝ)) * ((n : ℝ) * o i) ^ 2 ≤ (n : ℝ) * ∑ i, o i

Refutes the round-14 conjecture: picking O(1/μ_o) random rows and computing each exactly (cost n per row) estimates Σ_i o_i unbiasedly with constant relative variance in O(n/μ_o) time, regardless of score range. Aggregate hardness only bites when the signal μ_o = A/n is small or contributions cancel. Part of 'Unbiased randomized attention: cost x variance thresholds', round 15 (aggregate conjecture refuted: exact-row subsampling).

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 18 lines of Lean

Exact-row subsampling: for outputs in [0,1], E[(n·o_I)²] = n Σ o_i² ≤ n Σ o_i (relative second moment ≤ n/A)

How it was proved

More proved statements