Provedunbiased-attentionrandomized-algorithmssoftmaxhardness

Composed hard side: MSE ≤ 1/256 ⇒ thresholding at 3/8 decides the gap with probability ≥ 3/4

For all $\htmlData{mx=1}{m \in \htmlData{mx=0}{\mathbb{N}}}$ , $\htmlData{mx=5}{w, z : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{m}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ and $\htmlData{mx=1}{o \in \htmlData{mx=0}{\mathbb{R}}}$ , if $\htmlData{mx=4}{\forall i,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{w}(\htmlData{mx=1}{i})}}$ , $\htmlData{mx=3}{\sum_{i} \htmlData{mx=2}{\htmlData{mx=0}{w}(\htmlData{mx=1}{i})}} \mathrel{\htmlData{mx=4}{=}} 1$ and $\htmlData{mx=10}{\sum_{i} \htmlData{mx=9}{\htmlData{mx=2}{\htmlData{mx=0}{w}(\htmlData{mx=1}{i})} \cdot \htmlData{mx=8}{(\htmlData{mx=7}{\htmlData{mx=5}{\htmlData{mx=3}{z}(\htmlData{mx=4}{i})} - \htmlData{mx=6}{o}})^{2}}}} \mathrel{\htmlData{mx=12}{\leq}} \htmlData{mx=11}{\frac{1}{256}}$ , then

\left(\htmlData{mx=0}{\frac{1}{2}} \mathrel{\htmlData{mx=2}{\leq}} \htmlData{mx=1}{o} \mathrel{\htmlData{mx=17}{\implies}} \htmlData{mx=3}{\frac{3}{4}} \mathrel{\htmlData{mx=16}{\leq}} \htmlData{mx=15}{\sum_{i \in \htmlData{mx=11}{\htmlData{mx=4}{\operatorname{filter}}\left(\htmlData{mx=10}{i \mapsto \htmlData{mx=5}{\frac{3}{8}} \mathrel{\htmlData{mx=9}{<}} \htmlData{mx=8}{\htmlData{mx=6}{z}(\htmlData{mx=7}{i})}}\right)}} \htmlData{mx=14}{\htmlData{mx=12}{w}(\htmlData{mx=13}{i})}}\right) \mathbin{\htmlData{mx=36}{\land}} \left(\htmlData{mx=18}{o} \mathrel{\htmlData{mx=20}{\leq}} \htmlData{mx=19}{\frac{1}{4}} \mathrel{\htmlData{mx=35}{\implies}} \htmlData{mx=21}{\frac{3}{4}} \mathrel{\htmlData{mx=34}{\leq}} \htmlData{mx=33}{\sum_{i \in \htmlData{mx=29}{\htmlData{mx=22}{\operatorname{filter}}\left(\htmlData{mx=28}{i \mapsto \htmlData{mx=25}{\htmlData{mx=23}{z}(\htmlData{mx=24}{i})} \mathrel{\htmlData{mx=27}{<}} \htmlData{mx=26}{\frac{3}{8}}}\right)}} \htmlData{mx=32}{\htmlData{mx=30}{w}(\htmlData{mx=31}{i})}}\right)

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to m. Lean writes f x; papers write f(x). (Fin m)
⟹Implies: If Fin(m), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin m → ℝ)
x : TVariables: w, z range over Fin m → ℝ: the statement must hold for every choice. ((w z : Fin m → ℝ))
∀For all: The statement after the comma holds for every i. (∀ i, 0 ≤ w i)
∑Summation: Add up w(i) for every i. (∑ i, w i)
=Equality: ∑ i, w(i) and 1 are the same. (∑ i, w i = 1)
∧And: Both hold: 1/2 ≤ o → 3/4 ≤ (∑ i, w(i)), and o ≤ 1/4 → 3/4 ≤ (∑ i, w(i)). ((1 / 2 ≤ o → 3 / 4 ≤ ∑ i ∈ Finset.univ.…)
≤At most: 0 is less than or equal to w(i). (0 ≤ w i)
·Multiplication: w(i) times (z(i) − o)^2. (w i * (z i - o) ^ 2)
RReal numbers: The real numbers: every point on the number line. (ℝ)
aᵇPower: z(i) − o raised to the power 2. ((z i - o) ^ 2)
-Subtraction: z(i) minus o. On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (z i - o)
a/bDivision: 1 divided by 256. In Lean, dividing by 0 gives 0. (1 / 256)
filter: Finset.univ.filter: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Finset.univ.filter)
x ↦ f(x)Function (λ-abstraction): The function sending i to 3/8 < z(i). Lean writes fun x => …; papers write x ↦ …. (fun i => 3 / 8 < z i)
<Less than: 3/8 is strictly smaller than z(i). (3 / 8 < z i)

theorem ua-hard-side-decision {m : ℕ} (w z : Fin m → ℝ) (o : ℝ) (hw : ∀ i, 0 ≤ w i) (hsum : ∑ i, w i = 1) (hmse : ∑ i, w i * (z i - o) ^ 2 ≤ 1 / 256) :
    (1 / 2 ≤ o → 3 / 4 ≤ ∑ i ∈ Finset.univ.filter (fun i => 3 / 8 < z i), w i) ∧ (o ≤ 1 / 4 → 3 / 4 ≤ ∑ i ∈ Finset.univ.filter (fun i => z i < 3 / 8), w i)

Over any finite probability space, an estimator z of the row output o with E(z−o)² ≤ 1/256 satisfies P(z > 3/8) ≥ 3/4 when o ≥ 1/2 and P(z < 3/8) ≥ 3/4 when o ≤ 1/4 (Chebyshev + gap, composed in one statement). Part of 'Unbiased randomized attention: cost x variance thresholds' (open theory problem), round 2.

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 42 lines of Lean

Composed hard side: MSE ≤ 1/256 ⇒ thresholding at 3/8 decides the gap with probability ≥ 3/4

How it was proved

More proved statements