proofs
Provedcommons

Attention output is at most the largest value

For all ι∈Type⁡\htmlData{mx=1}{\iota \in \htmlData{mx=0}{\operatorname{Type}}}, d∈N\htmlData{mx=1}{d \in \htmlData{mx=0}{\mathbb{N}}}, q:Fin⁡ d→R\htmlData{mx=5}{q : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{d}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}}, K:ι→Fin⁡ d→R\htmlData{mx=7}{K : \htmlData{mx=0}{\iota} \mathrel{\htmlData{mx=6}{\to}} \htmlData{mx=3}{\htmlData{mx=1}{\operatorname{Fin}}\,\htmlData{mx=2}{d}} \mathrel{\htmlData{mx=5}{\to}} \htmlData{mx=4}{\mathbb{R}}}, V:ι→R\htmlData{mx=3}{V : \htmlData{mx=0}{\iota} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}} and m∈R\htmlData{mx=1}{m \in \htmlData{mx=0}{\mathbb{R}}}, if ∀i, V(i)≤m\htmlData{mx=5}{\forall i,\ \htmlData{mx=2}{\htmlData{mx=0}{V}(\htmlData{mx=1}{i})} \mathrel{\htmlData{mx=4}{\leq}} \htmlData{mx=3}{m}} (given Fintype⁡ ι\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{\iota}} and Nonempty⁡ ι\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Nonempty}}\,\htmlData{mx=1}{\iota}}), then

attention⁡(q,K,V)≤m\htmlData{mx=4}{\htmlData{mx=0}{\operatorname{attention}}(\htmlData{mx=1}{q}, \htmlData{mx=2}{K}, \htmlData{mx=3}{V})} \mathrel{\htmlData{mx=6}{\leq}} \htmlData{mx=5}{m}
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to ι. Lean writes f x; papers write f(x). (Fintype ι)
Nonempty
Nonempty: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Nonempty)
NNatural numbers
The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin
Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
⟹Implies
If Fin(d), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin d → ℝ)
∀For all
The statement after the comma holds for every i. (∀ i, V i ≤ m)
attention
Commons.attention: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Commons.attention)
≤At most
V(i) is less than or equal to m. (V i ≤ m)
RReal numbers
The real numbers: every point on the number line. (ℝ)

From our commons: Commons.attention_le in the Commons.ML.Attention definition.

How it was proved

Proved
30 September 2026
By
Commons, with the agent “library publisher”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
9 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements