proofs
Provedvariance-share-lawtraining-energystochastic-attentionlaw

Per-operation optimality: variance share = cost share, (A_k/b_k)/σ² = c_k b_k/c₀ ⇔ b_k² c_k σ² = A_k c₀

For all A,c,b,s2,c0∈R\htmlData{mx=1}{A, c, b, s_{2}, c_{0} \in \htmlData{mx=0}{\mathbb{R}}}, if 0<b0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{b}, 0<s20 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{s_{2}} and 0<c00 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{c_{0}}, then

Abs2=c⋅bc0  ⟺  b2⋅c⋅s2=A⋅c0\htmlData{mx=4}{\frac{\htmlData{mx=2}{\frac{\htmlData{mx=0}{A}}{\htmlData{mx=1}{b}}}}{\htmlData{mx=3}{s_{2}}}} \mathrel{\htmlData{mx=10}{=}} \htmlData{mx=9}{\frac{\htmlData{mx=7}{\htmlData{mx=5}{c} \cdot \htmlData{mx=6}{b}}}{\htmlData{mx=8}{c_{0}}}} \mathrel{\htmlData{mx=21}{\iff}} \htmlData{mx=16}{\htmlData{mx=14}{\htmlData{mx=12}{\htmlData{mx=11}{b}^{2}} \cdot \htmlData{mx=13}{c}} \cdot \htmlData{mx=15}{s_{2}}} \mathrel{\htmlData{mx=20}{=}} \htmlData{mx=19}{\htmlData{mx=17}{A} \cdot \htmlData{mx=18}{c_{0}}}
What the notation means · or point at any symbol
RReal numbers
The real numbers: every point on the number line. (ℝ)
x ∈ TVariables
A, c, b, s2, c0 range over real numbers: the statement must hold for every choice. ((A c b s2 c0 : ℝ))
<Less than
0 is strictly smaller than b. (0 < b)
=Equality
A/b/s2 and c · b/c0 are the same. ((A / b) / s2 = c * b / c0)
⟺If and only if
A/b/s2 = c · b/c0 holds exactly when b^2 · c · s2 = A · c0 holds. ((A / b) / s2 = c * b / c0 ↔ b ^ 2 * c *…)
a/bDivision
A divided by b. In Lean, dividing by 0 gives 0. (A / b)
·Multiplication
c times b. (c * b)
aᵇPower
b raised to the power 2. (b ^ 2)

At the optimum every sampled operation's share of gradient variance equals its share of step cost; equality in the whole-step bound. Part of 'Variance-Share Law: compute-optimal sampling budgets for stochastic attention in training'.

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
11 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements