Provedretrieval-speed-dualityhead-triageattentionduality

Dilution: with score range ≤ R every attention weight is ≤ e^R/n

For all $\htmlData{mx=1}{n \in \htmlData{mx=0}{\mathbb{N}}}$ , $\htmlData{mx=5}{s : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ , $\htmlData{mx=1}{R \in \htmlData{mx=0}{\mathbb{R}}}$ and $\htmlData{mx=3}{j \in \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{n}}}$ , if $\htmlData{mx=9}{\forall i, j,\ \htmlData{mx=6}{\htmlData{mx=2}{\htmlData{mx=0}{s}(\htmlData{mx=1}{i})} - \htmlData{mx=5}{\htmlData{mx=3}{s}(\htmlData{mx=4}{j})}} \mathrel{\htmlData{mx=8}{\leq}} \htmlData{mx=7}{R}}$ , then

\htmlData{mx=9}{\frac{\htmlData{mx=3}{e^{\htmlData{mx=2}{\htmlData{mx=0}{s}(\htmlData{mx=1}{j})}}}}{\htmlData{mx=8}{\sum_{k} \htmlData{mx=7}{e^{\htmlData{mx=6}{\htmlData{mx=4}{s}(\htmlData{mx=5}{k})}}}}}} \mathrel{\htmlData{mx=14}{\leq}} \htmlData{mx=13}{\frac{\htmlData{mx=11}{e^{\htmlData{mx=10}{R}}}}{\htmlData{mx=12}{n}}}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to n. Lean writes f x; papers write f(x). (Fin n)
⟹Implies: If Fin(n), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin n → ℝ)
∀For all: The statement after the comma holds for every i, j. (∀ i j, s i - s j ≤ R)
eˣExponential function: e raised to the power s(j), where e ≈ 2.718. (Real.exp (s j))
a/bDivision: exp(s(j)) divided by ∑ k, exp(s(k)). In Lean, dividing by 0 gives 0. (Real.exp (s j) / ∑ k, Real.exp (s k))
-Subtraction: s(i) minus s(j). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (s i - s j)
≤At most: s(i) − s(j) is less than or equal to R. (s i - s j ≤ R)
RReal numbers: The real numbers: every point on the number line. (ℝ)
∑Summation: Add up exp(s(k)) for every k. (∑ k, Real.exp (s k))

theorem rsd-max-weight-bound {n : ℕ} (s : Fin n → ℝ) (R : ℝ) (hR : ∀ i j, s i - s j ≤ R) (j : Fin n) :
    Real.exp (s j) / ∑ k, Real.exp (s k) ≤ Real.exp R / n

The per-head ceiling on single-token retrieval; the effect behind Scalable-Softmax (Nakanishi 2025). Part of 'Retrieval–Speed Duality: the score range that lets a head retrieve is the range that makes it hard to accelerate'.

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 19 lines of Lean

Dilution: with score range ≤ R every attention weight is ≤ e^R/n

How it was proved

More proved statements