proofs
Provedunbiased-attentionrandomized-algorithmssoftmaxaggregate

Pair sampling: global score range ≤ R ⇒ n²·Σ_{ij} π_ij² ≤ e^R

For all n∈N\htmlData{mx=1}{n \in \htmlData{mx=0}{\mathbb{N}}}, s:Fin⁡ n→Fin⁡ n→R\htmlData{mx=9}{s : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=8}{\to}} \htmlData{mx=5}{\htmlData{mx=3}{\operatorname{Fin}}\,\htmlData{mx=4}{n}} \mathrel{\htmlData{mx=7}{\to}} \htmlData{mx=6}{\mathbb{R}}} and R∈R\htmlData{mx=1}{R \in \htmlData{mx=0}{\mathbb{R}}}, if ∀i,j,i′,j′, s(i,j)−s(i′,j′)≤R\htmlData{mx=11}{\forall i, j, i', j',\ \htmlData{mx=8}{\htmlData{mx=3}{\htmlData{mx=0}{s}(\htmlData{mx=1}{i}, \htmlData{mx=2}{j})} - \htmlData{mx=7}{\htmlData{mx=4}{s}(\htmlData{mx=5}{i'}, \htmlData{mx=6}{j'})}} \mathrel{\htmlData{mx=10}{\leq}} \htmlData{mx=9}{R}}, then

n⋅n⋅∑i∑j(es(i,j)∑i′∑j′es(i′,j′))2≤eR\begin{aligned} & \htmlData{mx=21}{\htmlData{mx=4}{\htmlData{mx=2}{\htmlData{mx=1}{n}} \cdot \htmlData{mx=3}{n}} \cdot \htmlData{mx=20}{\sum_{i} \htmlData{mx=19}{\sum_{j} \htmlData{mx=18}{\left(\htmlData{mx=17}{\frac{\htmlData{mx=9}{e^{\htmlData{mx=8}{\htmlData{mx=5}{s}(\htmlData{mx=6}{i}, \htmlData{mx=7}{j})}}}}{\htmlData{mx=16}{\sum_{i'} \htmlData{mx=15}{\sum_{j'} \htmlData{mx=14}{e^{\htmlData{mx=13}{\htmlData{mx=10}{s}(\htmlData{mx=11}{i'}, \htmlData{mx=12}{j'})}}}}}}}\right)^{2}}}}} \\ & \quad \htmlData{mx=0}{\leq} \htmlData{mx=23}{e^{\htmlData{mx=22}{R}}} \end{aligned}
What the notation means · or point at any symbol
NNatural numbers
The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin
Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application
Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
∀For all
The statement after the comma holds for every i, j, i', j'. (∀ i j i' j', s i j - s i' j' ≤ R)
(x : T)As real numbers
n read as an element of real numbers. Mathematicians leave this implicit; Lean needs it to know which kind of number is meant. ((n : ℝ))
·Multiplication
n times n. ((n : ℝ) * n)
⟹Implies
If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
-Subtraction
s(i, j) minus s(i', j'). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (s i j - s i' j')
≤At most
s(i, j) − s(i', j') is less than or equal to R. (s i j - s i' j' ≤ R)
RReal numbers
The real numbers: every point on the number line. (ℝ)
aᵇPower
exp(s(i, j))/(∑ i', ∑ j', exp(s(i', j'))) raised to the power 2. ((Real.exp (s i j) / ∑ i', ∑ j', Real.ex…)
eˣExponential function
e raised to the power s(i, j), where e ≈ 2.718. (Real.exp (s i j))
a/bDivision
exp(s(i, j)) divided by ∑ i', ∑ j', exp(s(i', j')). In Lean, dividing by 0 gives 0. (Real.exp (s i j) / ∑ i', ∑ j', Real.exp…)
∑Summation
Add up exp(s(i', j')) for every j'. (∑ j', Real.exp (s i' j'))

Uniform sampling of (query, key) pairs estimates the batch partition function to constant relative error with O(e^R) samples. With the aggregate gadget: the exponent of constant-factor batch partition-function estimation at range a ln n is exactly min(a, 2) (SETH, polylog dimension; candidate), sharpening the Θ(log n) threshold of Charikar–Siminelakis 2019 (Thm 1.2, constant C(δ, α) = O(e^{e^{δ/c(δ)}})) to the exact constant and full curve. Part of 'Unbiased randomized attention: cost x variance thresholds', round 13 (aggregate / batch partition function).

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
37 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements