Provedalphabaguettealgorithm-ideaa3

Shared computation: the gradient of Σₖ ℓₖ(h(θ)) is (Σₖ ℓₖ′)·h′(θ)

For all $\htmlData{mx=1}{K \in \htmlData{mx=0}{\mathbb{N}}}$ , $\htmlData{mx=3}{h : \htmlData{mx=0}{\mathbb{R}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}$ , $\htmlData{mx=1}{h', \theta \in \htmlData{mx=0}{\mathbb{R}}}$ , $\htmlData{mx=7}{\ell : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{K}} \mathrel{\htmlData{mx=6}{\to}} \htmlData{mx=3}{\mathbb{R}} \mathrel{\htmlData{mx=5}{\to}} \htmlData{mx=4}{\mathbb{R}}}$ , $\htmlData{mx=5}{\ell' : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{K}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}$ , $\htmlData{mx=5}{\mathit{hh} \in \htmlData{mx=4}{\htmlData{mx=0}{\operatorname{HasDerivAt}}\,\htmlData{mx=1}{h}\,\htmlData{mx=2}{h'}\,\htmlData{mx=3}{\theta}}}$ and $\htmlData{mx=12}{\mathit{h{\ell}} : \htmlData{mx=11}{\forall k,\ \htmlData{mx=10}{\htmlData{mx=0}{\operatorname{HasDerivAt}}\,\htmlData{mx=3}{\htmlData{mx=1}{\ell}(\htmlData{mx=2}{k})}\,\htmlData{mx=6}{\htmlData{mx=4}{\ell'}(\htmlData{mx=5}{k})}\,\htmlData{mx=9}{\htmlData{mx=7}{h}(\htmlData{mx=8}{\theta})}}}}$ :

\htmlData{mx=16}{\htmlData{mx=0}{\operatorname{HasDerivAt}}\,\htmlData{mx=8}{t \mapsto \htmlData{mx=7}{\sum_{k} \htmlData{mx=6}{\htmlData{mx=1}{\ell}(\htmlData{mx=2}{k}, \htmlData{mx=5}{\htmlData{mx=3}{h}(\htmlData{mx=4}{t})})}}}\,\htmlData{mx=14}{\left(\htmlData{mx=12}{\sum_{k} \htmlData{mx=11}{\htmlData{mx=9}{\ell'}(\htmlData{mx=10}{k})}}\right) \cdot \htmlData{mx=13}{h'}}\,\htmlData{mx=15}{\theta}}

What the notation means · or point at any symbol

NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
RReal numbers: The real numbers: every point on the number line. (ℝ)
⟹Implies: If ℝ, then ℝ. (Between types, A → B is the type of functions from A to B.) (ℝ → ℝ)
x ∈ TVariables: h', θ range over real numbers: the statement must hold for every choice. ((h' θ : ℝ))
Fin: Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
f(x)Function application: Fin applied to K. Lean writes f x; papers write f(x). (Fin K)
HasDerivAt: HasDerivAt: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (HasDerivAt)
∀For all: The statement after the comma holds for every k. (∀ k, HasDerivAt (ℓ k) (ℓ' k) (h θ))
x ↦ f(x)Function (λ-abstraction): The function sending t to ∑ k, ℓ(k, h(t)). Lean writes fun x => …; papers write x ↦ …. (fun t => ∑ k, ℓ k (h t))
∑Summation: Add up ℓ(k, h(t)) for every k. (∑ k, ℓ k (h t))
·Multiplication: ∑ k, ℓ'(k) times h'. ((∑ k, ℓ' k) * h')

theorem dedup-shared-cotangent-sum {K : ℕ} (h : ℝ → ℝ) (h' θ : ℝ) (ℓ : Fin K → ℝ → ℝ) (ℓ' : Fin K → ℝ) (hh : HasDerivAt h h' θ) (hℓ : ∀ k, HasDerivAt (ℓ k) (ℓ' k) (h θ)) :
    HasDerivAt (fun t => ∑ k, ℓ k (h t)) ((∑ k, ℓ' k) * h') θ

Idea A3. When K duplicate prefixes share one computed activation h(θ), backpropagating once with the summed cotangent Σₖ ℓₖ′ gives exactly the gradient of the sum of their losses. This is the exactness of dedup in the backward pass.

How it was proved

Proved: 3 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: Classical.choice, Quot.sound, propext
Proof: 13 lines of Lean

Shared computation: the gradient of Σₖ ℓₖ(h(θ)) is (Σₖ ℓₖ′)·h′(θ)

How it was proved

More proved statements