Provedunbiased-attentionrandomized-algorithmssoftmaxlower-bound

Two-point (Le Cam) bound: identical inputs outside an event of probability ≤ η ⇒ worst-case MSE ≥ (1 − η)Δ²/4

For all $\htmlData{mx=1}{m \in \htmlData{mx=0}{\mathbb{N}}}$ , $\htmlData{mx=5}{w, z_{1}, z_{2} : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{m}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ , $\htmlData{mx=5}{E : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{m}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\operatorname{Prop}}}$ and $\htmlData{mx=1}{o_{1}, o_{2}, \eta \in \htmlData{mx=0}{\mathbb{R}}}$ , if $\htmlData{mx=4}{\forall i,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{w}(\htmlData{mx=1}{i})}}$ , $\htmlData{mx=3}{\sum_{i} \htmlData{mx=2}{\htmlData{mx=0}{w}(\htmlData{mx=1}{i})}} \mathrel{\htmlData{mx=4}{=}} 1$ , $\htmlData{mx=6}{\sum_{i \in \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{filter}}(\htmlData{mx=1}{E})}} \htmlData{mx=5}{\htmlData{mx=3}{w}(\htmlData{mx=4}{i})}} \mathrel{\htmlData{mx=8}{\leq}} \htmlData{mx=7}{\eta}$ and $\htmlData{mx=12}{\forall i,\ \htmlData{mx=3}{\lnot \htmlData{mx=2}{\htmlData{mx=0}{E}(\htmlData{mx=1}{i})}} \mathrel{\htmlData{mx=11}{\implies}} \htmlData{mx=6}{\htmlData{mx=4}{z_{1}}(\htmlData{mx=5}{i})} \mathrel{\htmlData{mx=10}{=}} \htmlData{mx=9}{\htmlData{mx=7}{z_{2}}(\htmlData{mx=8}{i})}}$ (given $\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{DecidablePred}}\,\htmlData{mx=1}{E}}$ ), then

\begin{aligned} & \htmlData{mx=8}{\frac{\htmlData{mx=7}{(\htmlData{mx=2}{1 - \htmlData{mx=1}{\eta}}) \cdot \htmlData{mx=6}{(\htmlData{mx=5}{\htmlData{mx=3}{o_{1}} - \htmlData{mx=4}{o_{2}}})^{2}}}}{4}} \\ & \quad \htmlData{mx=0}{\leq} \htmlData{mx=31}{\max(\htmlData{mx=19}{\sum_{i} \htmlData{mx=18}{\htmlData{mx=11}{\htmlData{mx=9}{w}(\htmlData{mx=10}{i})} \cdot \htmlData{mx=17}{(\htmlData{mx=16}{\htmlData{mx=14}{\htmlData{mx=12}{z_{1}}(\htmlData{mx=13}{i})} - \htmlData{mx=15}{o_{1}}})^{2}}}}, \htmlData{mx=30}{\sum_{i} \htmlData{mx=29}{\htmlData{mx=22}{\htmlData{mx=20}{w}(\htmlData{mx=21}{i})} \cdot \htmlData{mx=28}{(\htmlData{mx=27}{\htmlData{mx=25}{\htmlData{mx=23}{z_{2}}(\htmlData{mx=24}{i})} - \htmlData{mx=26}{o_{2}}})^{2}}}})} \end{aligned}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to m. Lean writes f x; papers write f(x). (Fin m)
⟹Implies: If Fin(m), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin m → ℝ)
x : TVariables: w, z1, z2 range over Fin m → ℝ: the statement must hold for every choice. ((w z1 z2 : Fin m → ℝ))
DecidablePred: DecidablePred: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (DecidablePred)
x ∈ TVariables: o1, o2, η range over real numbers: the statement must hold for every choice. ((o1 o2 η : ℝ))
∀For all: The statement after the comma holds for every i. (∀ i, 0 ≤ w i)
∑Summation: Add up w(i) for every i. (∑ i, w i)
=Equality: ∑ i, w(i) and 1 are the same. (∑ i, w i = 1)
·Multiplication: 1 − η times (o1 − o2)^2. ((1 - η) * (o1 - o2) ^ 2)
a/bDivision: (1 − η) · (o1 − o2)^2 divided by 4. In Lean, dividing by 0 gives 0. ((1 - η) * (o1 - o2) ^ 2 / 4)
-Subtraction: 1 minus η. On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (1 - η)
≤At most: 0 is less than or equal to w(i). (0 ≤ w i)
¬Not: It is not the case that E(i). (¬ E i)
filter: Finset.univ.filter: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Finset.univ.filter)
RReal numbers: The real numbers: every point on the number line. (ℝ)
Prop: Prop: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Prop)
aᵇPower: o1 − o2 raised to the power 2. ((o1 - o2) ^ 2)
maxMaximum: The larger of ∑ i, w(i) · (z1(i) − o1)^2 and ∑ i, w(i) · (z2(i) − o2)^2. (max (∑ i, w i * (z1 i - o1) ^ 2) (∑ i, …)

theorem ua-two-point-lower-bound {m : ℕ} (w z1 z2 : Fin m → ℝ) (E : Fin m → Prop) [DecidablePred E] (o1 o2 η : ℝ) (hw : ∀ i, 0 ≤ w i) (hsum : ∑ i, w i = 1) (hE : ∑ i ∈ Finset.univ.filter E, w i ≤ η) (hsame : ∀ i, ¬ E i → z1 i = z2 i) :
    (1 - η) * (o1 - o2) ^ 2 / 4 ≤ max (∑ i, w i * (z1 i - o1) ^ 2) (∑ i, w i * (z2 i - o2) ^ 2)

For any estimator, if two worlds with outputs o₁, o₂ produce the same observations except on an event of probability ≤ η, then max over the worlds of the MSE is ≥ (1 − η)(o₁ − o₂)²/4. Part of 'Unbiased randomized attention: cost x variance thresholds', round 5 (approximate-score proposals and the sampling lower bound).

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 36 lines of Lean

Two-point (Le Cam) bound: identical inputs outside an event of probability ≤ η ⇒ worst-case MSE ≥ (1 − η)Δ²/4

How it was proved

More proved statements