proofs
Provedsecond-order-bias-lawstochastic-attentiondepthlaw

Bias vs noise: bias ≤ (M/2)V and noise sd G√V ⇒ bias ≤ noise × M√V/(2G)

For all M,G,V,bias∈R\htmlData{mx=1}{M, G, V, \mathit{bias} \in \htmlData{mx=0}{\mathbb{R}}}, if 0<G0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{G}, 0≤V0 \mathrel{\htmlData{mx=1}{\leq}} \htmlData{mx=0}{V} and bias≤M2⋅V\htmlData{mx=0}{\mathit{bias}} \mathrel{\htmlData{mx=5}{\leq}} \htmlData{mx=4}{\htmlData{mx=2}{\frac{\htmlData{mx=1}{M}}{2}} \cdot \htmlData{mx=3}{V}}, then

bias≤G⋅V⋅(M⋅V2G)\htmlData{mx=0}{\mathit{bias}} \mathrel{\htmlData{mx=13}{\leq}} \htmlData{mx=12}{\htmlData{mx=4}{\htmlData{mx=1}{G} \cdot \htmlData{mx=3}{\sqrt{\htmlData{mx=2}{V}}}} \cdot \left(\htmlData{mx=11}{\frac{\htmlData{mx=8}{\htmlData{mx=5}{M} \cdot \htmlData{mx=7}{\sqrt{\htmlData{mx=6}{V}}}}}{\htmlData{mx=10}{2\htmlData{mx=9}{G}}}}\right)}
What the notation means · or point at any symbol
RReal numbers
The real numbers: every point on the number line. (ℝ)
x ∈ TVariables
M, G, V, bias range over real numbers: the statement must hold for every choice. ((M G V bias : ℝ))
<Less than
0 is strictly smaller than G. (0 < G)
≤At most
0 is less than or equal to V. (0 ≤ V)
a/bDivision
M divided by 2. In Lean, dividing by 0 gives 0. (M / 2)
·Multiplication
M/2 times V. (M / 2 * V)
√Square root
The non-negative number whose square is V. (Real.sqrt V)

Bias/noise → 0 as V → 0: in smooth nets stochastic attention behaves like extra SGD noise, not systematic error. Part of 'Second-Order Bias Law: unbiased stochastic attention stays (almost) unbiased through smooth depth'.

How it was proved

Proved
4 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
16 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements