proofs
Provedvariance-share-lawtraining-energystochastic-attentionlaw

Whole-step Variance-Share Law: (σ² + Σ_k A_k/b_k)(c₀ + Σ_k c_k b_k) ≥ (√(σ²c₀) + Σ_k √(A_k c_k))²

For all K∈N\htmlData{mx=1}{K \in \htmlData{mx=0}{\mathbb{N}}}, A,c,b:Fin⁡ K→R\htmlData{mx=5}{A, c, b : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{K}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}} and s2,c0∈R\htmlData{mx=1}{s_{2}, c_{0} \in \htmlData{mx=0}{\mathbb{R}}}, if ∀k, 0≤A(k)\htmlData{mx=4}{\forall k,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{A}(\htmlData{mx=1}{k})}}, ∀k, 0≤c(k)\htmlData{mx=4}{\forall k,\ 0 \mathrel{\htmlData{mx=3}{\leq}} \htmlData{mx=2}{\htmlData{mx=0}{c}(\htmlData{mx=1}{k})}}, ∀k, 0<b(k)\htmlData{mx=4}{\forall k,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{b}(\htmlData{mx=1}{k})}}, 0≤s20 \mathrel{\htmlData{mx=1}{\leq}} \htmlData{mx=0}{s_{2}} and 0≤c00 \mathrel{\htmlData{mx=1}{\leq}} \htmlData{mx=0}{c_{0}}, then

(s2⋅c0+∑kA(k)⋅c(k))2≤(s2+∑kA(k)b(k))⋅(c0+∑kc(k)⋅b(k))\begin{aligned} & \htmlData{mx=15}{\left(\htmlData{mx=14}{\htmlData{mx=4}{\sqrt{\htmlData{mx=3}{\htmlData{mx=1}{s_{2}} \cdot \htmlData{mx=2}{c_{0}}}}} + \htmlData{mx=13}{\sum_{k} \htmlData{mx=12}{\sqrt{\htmlData{mx=11}{\htmlData{mx=7}{\htmlData{mx=5}{A}(\htmlData{mx=6}{k})} \cdot \htmlData{mx=10}{\htmlData{mx=8}{c}(\htmlData{mx=9}{k})}}}}}}\right)^{2}} \\ & \quad \htmlData{mx=0}{\leq} \htmlData{mx=36}{\left(\htmlData{mx=25}{\htmlData{mx=16}{s_{2}} + \htmlData{mx=24}{\sum_{k} \htmlData{mx=23}{\frac{\htmlData{mx=19}{\htmlData{mx=17}{A}(\htmlData{mx=18}{k})}}{\htmlData{mx=22}{\htmlData{mx=20}{b}(\htmlData{mx=21}{k})}}}}}\right) \cdot \left(\htmlData{mx=35}{\htmlData{mx=26}{c_{0}} + \htmlData{mx=34}{\sum_{k} \htmlData{mx=33}{\htmlData{mx=29}{\htmlData{mx=27}{c}(\htmlData{mx=28}{k})} \cdot \htmlData{mx=32}{\htmlData{mx=30}{b}(\htmlData{mx=31}{k})}}}}\right)} \end{aligned}
What the notation means · or point at any symbol
NNatural numbers
The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin
Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application
Fin applied to K. Lean writes f x; papers write f(x). (Fin K)
⟹Implies
If Fin(K), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin K → ℝ)
x : TVariables
A, c, b range over Fin K → ℝ: the statement must hold for every choice. ((A c b : Fin K → ℝ))
x ∈ TVariables
s2, c0 range over real numbers: the statement must hold for every choice. ((s2 c0 : ℝ))
∀For all
The statement after the comma holds for every k. (∀ k, 0 ≤ A k)
aᵇPower
√(s2 · c0) + (∑ k, √(A(k) · c(k))) raised to the power 2. ((Real.sqrt (s2 * c0) + ∑ k, Real.sqrt (…)
√Square root
The non-negative number whose square is s2 · c0. (Real.sqrt (s2 * c0))
+Addition
√(s2 · c0) plus ∑ k, √(A(k) · c(k)). (Real.sqrt (s2 * c0) + ∑ k, Real.sqrt (A…)
≤At most
0 is less than or equal to A(k). (0 ≤ A k)
<Less than
0 is strictly smaller than b(k). (0 < b k)
RReal numbers
The real numbers: every point on the number line. (ℝ)
·Multiplication
s2 times c0. (s2 * c0)
∑Summation
Add up √(A(k) · c(k)) for every k. (∑ k, Real.sqrt (A k * c k))
a/bDivision
A(k) divided by b(k). In Lean, dividing by 0 gives 0. (A k / b k)

One budget law for every sampled operation in a training step (sampled attention heads, token-sampled weight-gradient matmuls à la Adelman & Silberstein NeurIPS 2021, ...): energy-to-target is bounded below by a Cauchy–Schwarz sum over operations. Part of 'Variance-Share Law: compute-optimal sampling budgets for stochastic attention in training'.

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
25 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements