Provedunbiased-attentionrandomized-algorithmssoftmaxhardness

Hamming distance of 0/1 vectors is affine in the inner product: Σ(a−c)² = |a| + |c| − 2⟨a,c⟩

For all $\htmlData{mx=1}{d \in \htmlData{mx=0}{\mathbb{N}}}$ and $\htmlData{mx=5}{a, c : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{d}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ , if $\htmlData{mx=11}{\forall i,\ \htmlData{mx=6}{\htmlData{mx=2}{\htmlData{mx=0}{a}(\htmlData{mx=1}{i})} \cdot \htmlData{mx=5}{\htmlData{mx=3}{a}(\htmlData{mx=4}{i})}} \mathrel{\htmlData{mx=10}{=}} \htmlData{mx=9}{\htmlData{mx=7}{a}(\htmlData{mx=8}{i})}}$ and $\htmlData{mx=11}{\forall i,\ \htmlData{mx=6}{\htmlData{mx=2}{\htmlData{mx=0}{c}(\htmlData{mx=1}{i})} \cdot \htmlData{mx=5}{\htmlData{mx=3}{c}(\htmlData{mx=4}{i})}} \mathrel{\htmlData{mx=10}{=}} \htmlData{mx=9}{\htmlData{mx=7}{c}(\htmlData{mx=8}{i})}}$ , then

\begin{aligned} & \htmlData{mx=9}{\sum_{i} \htmlData{mx=8}{(\htmlData{mx=7}{\htmlData{mx=3}{\htmlData{mx=1}{a}(\htmlData{mx=2}{i})} - \htmlData{mx=6}{\htmlData{mx=4}{c}(\htmlData{mx=5}{i})}})^{2}}} \\ & \quad \htmlData{mx=0}{=} \htmlData{mx=28}{\htmlData{mx=18}{\htmlData{mx=13}{\sum_{i} \htmlData{mx=12}{\htmlData{mx=10}{a}(\htmlData{mx=11}{i})}} + \htmlData{mx=17}{\sum_{i} \htmlData{mx=16}{\htmlData{mx=14}{c}(\htmlData{mx=15}{i})}}} - \htmlData{mx=27}{2 \cdot \htmlData{mx=26}{\sum_{i} \htmlData{mx=25}{\htmlData{mx=21}{\htmlData{mx=19}{a}(\htmlData{mx=20}{i})} \cdot \htmlData{mx=24}{\htmlData{mx=22}{c}(\htmlData{mx=23}{i})}}}}} \end{aligned}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to d. Lean writes f x; papers write f(x). (Fin d)
⟹Implies: If Fin(d), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin d → ℝ)
x : TVariables: a, c range over Fin d → ℝ: the statement must hold for every choice. ((a c : Fin d → ℝ))
∀For all: The statement after the comma holds for every i. (∀ i, a i * a i = a i)
∑Summation: Add up (a(i) − c(i))^2 for every i. (∑ i, (a i - c i) ^ 2)
·Multiplication: a(i) times a(i). (a i * a i)
=Equality: a(i) · a(i) and a(i) are the same. (a i * a i = a i)
aᵇPower: a(i) − c(i) raised to the power 2. ((a i - c i) ^ 2)
-Subtraction: a(i) minus c(i). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (a i - c i)
RReal numbers: The real numbers: every point on the number line. (ℝ)
+Addition: ∑ i, a(i) plus ∑ i, c(i). (∑ i, a i + ∑ i, c i)

theorem ua-hamming-as-inner-product {d : ℕ} (a c : Fin d → ℝ) (ha : ∀ i, a i * a i = a i) (hc : ∀ i, c i * c i = c i) :
    ∑ i, (a i - c i) ^ 2 = ∑ i, a i + ∑ i, c i - 2 * ∑ i, a i * c i

So a scaled negative Hamming distance is an attention score after augmenting two coordinates. Part of 'Unbiased randomized attention: cost x variance thresholds' (open theory problem), round 2.

How it was proved

Proved: 4 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 13 lines of Lean

Hamming distance of 0/1 vectors is affine in the inner product: Σ(a−c)² = |a| + |c| − 2⟨a,c⟩

How it was proved

More proved statements