Provedalphabaguettealgorithm-ideaa3

A causal model gives identical outputs on identical relocated prefixes

For all $\htmlData{mx=1}{\alpha \in \htmlData{mx=0}{\operatorname{Type}}}$ , $\htmlData{mx=7}{F : (\htmlData{mx=0}{\mathbb{N}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{N}}) \mathrel{\htmlData{mx=6}{\to}} \htmlData{mx=3}{\mathbb{N}} \mathrel{\htmlData{mx=5}{\to}} \htmlData{mx=4}{\alpha}}$ , $\htmlData{mx=3}{s, s' : \htmlData{mx=0}{\mathbb{N}} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{N}}}$ and $\htmlData{mx=1}{a, a', r \in \htmlData{mx=0}{\mathbb{N}}}$ , if $\htmlData{mx=19}{\forall s, s', t,\ (\htmlData{mx=8}{\forall i \leq \htmlData{mx=7}{t},\ \htmlData{mx=2}{\htmlData{mx=0}{s}(\htmlData{mx=1}{i})} \mathrel{\htmlData{mx=6}{=}} \htmlData{mx=5}{\htmlData{mx=3}{s'}(\htmlData{mx=4}{i})}}) \mathrel{\htmlData{mx=18}{\implies}} \htmlData{mx=12}{\htmlData{mx=9}{F}(\htmlData{mx=10}{s}, \htmlData{mx=11}{t})} \mathrel{\htmlData{mx=17}{=}} \htmlData{mx=16}{\htmlData{mx=13}{F}(\htmlData{mx=14}{s'}, \htmlData{mx=15}{t})}}$ and $\htmlData{mx=12}{\forall i \leq \htmlData{mx=11}{r},\ \htmlData{mx=4}{\htmlData{mx=0}{s}(\htmlData{mx=3}{\htmlData{mx=1}{a} + \htmlData{mx=2}{i}})} \mathrel{\htmlData{mx=10}{=}} \htmlData{mx=9}{\htmlData{mx=5}{s'}(\htmlData{mx=8}{\htmlData{mx=6}{a'} + \htmlData{mx=7}{i}})}}$ , then

\htmlData{mx=8}{\htmlData{mx=0}{F}(\htmlData{mx=6}{i \mapsto \htmlData{mx=5}{\htmlData{mx=1}{s}(\htmlData{mx=4}{\htmlData{mx=2}{a} + \htmlData{mx=3}{i}})}}, \htmlData{mx=7}{r})} \mathrel{\htmlData{mx=18}{=}} \htmlData{mx=17}{\htmlData{mx=9}{F}(\htmlData{mx=15}{i \mapsto \htmlData{mx=14}{\htmlData{mx=10}{s'}(\htmlData{mx=13}{\htmlData{mx=11}{a'} + \htmlData{mx=12}{i}})}}, \htmlData{mx=16}{r})}

What the notation means · or point at any symbol

Type: Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
x : TVariables: s, s' range over ℕ → ℕ: the statement must hold for every choice. ((s s' : ℕ → ℕ))
x ∈ TVariables: a, a', r range over natural numbers: the statement must hold for every choice. ((a a' r : ℕ))
NNatural numbers: The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
⟹Implies: If ℕ, then ℕ. (Between types, A → B is the type of functions from A to B.) (ℕ → ℕ)
x ↦ f(x)Function (λ-abstraction): The function sending i to s(a + i). Lean writes fun x => …; papers write x ↦ …. (fun i => s (a + i))
∀For all: The statement after the comma holds for every i. (∀ i ≤ t, s i = s' i)
+Addition: a plus i. (a + i)
f(x)Function application: s applied to i. Lean writes f x; papers write f(x). (s i)
=Equality: s(i) and s'(i) are the same. (s i = s' i)

theorem dedup-causal-relocated-prefix {α : Type} (F : (ℕ → ℕ) → ℕ → α) (hF : ∀ s s' t, (∀ i ≤ t, s i = s' i) → F s t = F s' t) (s s' : ℕ → ℕ) (a a' r : ℕ) (h : ∀ i ≤ r, s (a + i) = s' (a' + i)) :
    F (fun i => s (a + i)) r = F (fun i => s' (a' + i)) r

Idea A3 (prefix-dedup training). If a model's output at position t depends only on positions ≤ t (causality, with positions counted from the sample start), then two samples starting at a and a' whose first r+1 tokens agree produce the same output at relative position r. Their activations can be computed once.

How it was proved

Proved: 3 October 2026
By: Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by: Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms: None
Proof: 12 lines of Lean

A causal model gives identical outputs on identical relocated prefixes

How it was proved

More proved statements