proofs
Provedalphabaguettealgorithm-ideab5

Skipping softmax mass B ≤ δ·A changes the log-sum-exp loss by at most δ

For all A,B,δ∈R\htmlData{mx=1}{A, B, \delta \in \htmlData{mx=0}{\mathbb{R}}}, if 0<A0 \mathrel{\htmlData{mx=1}{<}} \htmlData{mx=0}{A}, 0≤B0 \mathrel{\htmlData{mx=1}{\leq}} \htmlData{mx=0}{B} and B≤δ⋅A\htmlData{mx=0}{B} \mathrel{\htmlData{mx=4}{\leq}} \htmlData{mx=3}{\htmlData{mx=1}{\delta} \cdot \htmlData{mx=2}{A}}, then

0≤log⁡(A+B)−log⁡A∧log⁡(A+B)−log⁡A≤δ0 \mathrel{\htmlData{mx=7}{\leq}} \htmlData{mx=6}{\htmlData{mx=3}{\log (\htmlData{mx=2}{\htmlData{mx=0}{A} + \htmlData{mx=1}{B}})} - \htmlData{mx=5}{\log \htmlData{mx=4}{A}}} \mathbin{\htmlData{mx=17}{\land}} \htmlData{mx=14}{\htmlData{mx=11}{\log (\htmlData{mx=10}{\htmlData{mx=8}{A} + \htmlData{mx=9}{B}})} - \htmlData{mx=13}{\log \htmlData{mx=12}{A}}} \mathrel{\htmlData{mx=16}{\leq}} \htmlData{mx=15}{\delta}
What the notation means · or point at any symbol
RReal numbers
The real numbers: every point on the number line. (ℝ)
x ∈ TVariables
A, B, δ range over real numbers: the statement must hold for every choice. ((A B δ : ℝ))
<Less than
0 is strictly smaller than A. (0 < A)
≤At most
0 is less than or equal to B. (0 ≤ B)
∧And
Both hold: 0 ≤ log(A + B) − log(A), and log(A + B) − log(A) ≤ δ. (0 ≤ Real.log (A + B) - Real.log A ∧ Rea…)
·Multiplication
δ times A. (δ * A)
logNatural logarithm
The logarithm of A + B to base e: the power e must be raised to to give A + B. (Real.log (A + B))
-Subtraction
log(A + B) minus log(A). On ℕ, Lean's subtraction stops at 0: 2 − 5 = 0. (Real.log (A + B) - Real.log A)
+Addition
A plus B. (A + B)

Idea B5. If the kept vocabulary mass is A > 0 and the skipped mass B satisfies 0 ≤ B ≤ δA, then 0 ≤ log(A + B) − log A ≤ δ: the cross-entropy computed from kept logits underestimates the exact loss by at most δ.

How it was proved

Proved
3 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
22 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements