Provedunbiased-attentiongradientone-pass

IID product space: E[U(X_i)K(X_i)] = E[UK] and E[U(X_i)K(X_j)] = E[U]E[K] for i ≠ j

For all $\htmlData{mx=1}{\Omega \in \htmlData{mx=0}{\operatorname{Type}}}$ , $\htmlData{mx=1}{N \in \htmlData{mx=0}{\mathbb{N}}}$ , $\htmlData{mx=3}{w, U, K : \htmlData{mx=0}{\Omega} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}$ and $\htmlData{mx=3}{i, j \in \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{N}}}$ , if $\htmlData{mx=3}{\sum_{\omega} \htmlData{mx=2}{\htmlData{mx=0}{w}(\htmlData{mx=1}{\omega})}} \mathrel{\htmlData{mx=4}{=}} 1$ and $\htmlData{mx=0}{i} \mathrel{\htmlData{mx=2}{\neq}} \htmlData{mx=1}{j}$ (given $\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{\Omega}}$ ), then

\begin{aligned} & \left(\right. \htmlData{mx=25}{\sum_{f : \htmlData{mx=4}{\htmlData{mx=2}{\operatorname{Fin}}\,\htmlData{mx=3}{N}} \mathrel{\htmlData{mx=6}{\to}} \htmlData{mx=5}{\Omega}} \htmlData{mx=24}{\left(\htmlData{mx=12}{\prod_{m} \htmlData{mx=11}{\htmlData{mx=7}{w}(\htmlData{mx=10}{\htmlData{mx=8}{f}(\htmlData{mx=9}{m})})}}\right) \cdot (\htmlData{mx=23}{\htmlData{mx=17}{\htmlData{mx=13}{U}(\htmlData{mx=16}{\htmlData{mx=14}{f}(\htmlData{mx=15}{i})})} \cdot \htmlData{mx=22}{\htmlData{mx=18}{K}(\htmlData{mx=21}{\htmlData{mx=19}{f}(\htmlData{mx=20}{i})})}})}} \\ & \quad \htmlData{mx=1}{=} \htmlData{mx=37}{\sum_{\omega} \htmlData{mx=36}{\htmlData{mx=28}{\htmlData{mx=26}{w}(\htmlData{mx=27}{\omega})} \cdot (\htmlData{mx=35}{\htmlData{mx=31}{\htmlData{mx=29}{U}(\htmlData{mx=30}{\omega})} \cdot \htmlData{mx=34}{\htmlData{mx=32}{K}(\htmlData{mx=33}{\omega})}})}} \left.\right) \\ & \htmlData{mx=0}{\land} \left(\right. \htmlData{mx=62}{\sum_{f : \htmlData{mx=41}{\htmlData{mx=39}{\operatorname{Fin}}\,\htmlData{mx=40}{N}} \mathrel{\htmlData{mx=43}{\to}} \htmlData{mx=42}{\Omega}} \htmlData{mx=61}{\left(\htmlData{mx=49}{\prod_{m} \htmlData{mx=48}{\htmlData{mx=44}{w}(\htmlData{mx=47}{\htmlData{mx=45}{f}(\htmlData{mx=46}{m})})}}\right) \cdot (\htmlData{mx=60}{\htmlData{mx=54}{\htmlData{mx=50}{U}(\htmlData{mx=53}{\htmlData{mx=51}{f}(\htmlData{mx=52}{i})})} \cdot \htmlData{mx=59}{\htmlData{mx=55}{K}(\htmlData{mx=58}{\htmlData{mx=56}{f}(\htmlData{mx=57}{j})})}})}} \\ & \quad \htmlData{mx=38}{=} \htmlData{mx=79}{\left(\htmlData{mx=70}{\sum_{\omega} \htmlData{mx=69}{\htmlData{mx=65}{\htmlData{mx=63}{w}(\htmlData{mx=64}{\omega})} \cdot \htmlData{mx=68}{\htmlData{mx=66}{U}(\htmlData{mx=67}{\omega})}}}\right) \cdot \left(\htmlData{mx=78}{\sum_{\omega} \htmlData{mx=77}{\htmlData{mx=73}{\htmlData{mx=71}{w}(\htmlData{mx=72}{\omega})} \cdot \htmlData{mx=76}{\htmlData{mx=74}{K}(\htmlData{mx=75}{\omega})}}}\right)} \left.\right) \end{aligned}

What the notation means · or point at any symbol

Type: Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
Fintype: Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application: Fintype applied to Ω. Lean writes f x; papers write f(x). (Fintype Ω)
NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
⟹Implies: If Ω, then ℝ. (Between types, A → B is the type of functions from A to B.) (Ω → ℝ)
x : TVariables: w, U, K range over Ω → ℝ: the statement must hold for every choice. ((w U K : Ω → ℝ))
∑Summation: Add up w(ω) for every ω. (∑ ω, w ω)
=Equality: ∑ ω, w(ω) and 1 are the same. (∑ ω, w ω = 1)
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
x ∈ TVariables: i, j range over Fin N: the statement must hold for every choice. ((i j : Fin N))
≠Not equal: i and j are different. (i ≠ j)
∧And: Both hold: (∑ f, (∏ m, w(f(m))) · (U(f(i)) · K(f(i)))) = (∑ ω, w(ω) ·…, and (∑ f, (∏ m, w(f(m))) · (U(f(i)) · K(f(j)))) = (∑ ω, w(ω) ·…. ((∑ f : Fin N → Ω, (∏ m, w (f m)) * (U (…)
RReal numbers: The real numbers: every point on the number line. (ℝ)
∏Product: Multiply together w(f(m)) for every m. (∏ m, w (f m))
·Multiplication: U(f(i)) times K(f(i)). (U (f i) * K (f i))

theorem op-iid-coordinate-means {Ω : Type} [Fintype Ω] {N : ℕ} (w U K : Ω → ℝ) (hsum : ∑ ω, w ω = 1) (i j : Fin N) (hij : i ≠ j) :
    (∑ f : Fin N → Ω, (∏ m, w (f m)) * (U (f i) * K (f i)) = ∑ ω, w ω * (U ω * K ω)) ∧ (∑ f : Fin N → Ω, (∏ m, w (f m)) * (U (f i) * K (f j)) = (∑ ω, w ω * U ω) * (∑ ω, w ω * K ω))

Coordinate expectations on the exact product space Fin N → Ω with weights Π w(f m), via Fintype.prod_sum. The building block of the one-pass gradient estimator. Part of 'Unbiased randomized attention', gradient round.

How it was proved

Proved: 5 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 41 lines of Lean

IID product space: E[U(X_i)K(X_i)] = E[UK] and E[U(X_i)K(X_j)] = E[U]E[K] for i ≠ j

How it was proved

More proved statements