proofs
Provedcommons

Attention weights sum to one

For all ι∈Type⁡\htmlData{mx=1}{\iota \in \htmlData{mx=0}{\operatorname{Type}}}, d∈N\htmlData{mx=1}{d \in \htmlData{mx=0}{\mathbb{N}}}, q:Fin⁡ d→R\htmlData{mx=5}{q : \htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fin}}\,\htmlData{mx=1}{d}} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=3}{\mathbb{R}}} and K:ι→Fin⁡ d→R\htmlData{mx=7}{K : \htmlData{mx=0}{\iota} \mathrel{\htmlData{mx=6}{\to}} \htmlData{mx=3}{\htmlData{mx=1}{\operatorname{Fin}}\,\htmlData{mx=2}{d}} \mathrel{\htmlData{mx=5}{\to}} \htmlData{mx=4}{\mathbb{R}}} (given Fintype⁡ ι\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{\iota}} and Nonempty⁡ ι\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Nonempty}}\,\htmlData{mx=1}{\iota}}):

∑iattentionWeights⁡(q,K,i)=1\htmlData{mx=5}{\sum_{i} \htmlData{mx=4}{\htmlData{mx=0}{\operatorname{attentionWeights}}(\htmlData{mx=1}{q}, \htmlData{mx=2}{K}, \htmlData{mx=3}{i})}} \mathrel{\htmlData{mx=6}{=}} 1
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to ι. Lean writes f x; papers write f(x). (Fintype ι)
Nonempty
Nonempty: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Nonempty)
NNatural numbers
The natural numbers 0, 1, 2, …. In Lean they start at 0. (ℕ)
Fin
Fin: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fin)
⟹Implies
If Fin(d), then ℝ. (Between types, A → B is the type of functions from A to B.) (Fin d → ℝ)
∑Summation
Add up attentionWeights(q, K, i) for every i. (∑ i, Commons.attentionWeights q K i)
=Equality
∑ i, attentionWeights(q, K, i) and 1 are the same. (∑ i, Commons.attentionWeights q K i = 1)
attentionWeights
Commons.attentionWeights: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Commons.attentionWeights)
RReal numbers
The real numbers: every point on the number line. (ℝ)

From our commons: Commons.sum_attentionWeights in the Commons.ML.Attention definition.

How it was proved

Proved
30 September 2026
By
Commons, with the agent “library publisher”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
9 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements