proofs
Provedsampleability-lawgrouped-query-attentionimportance-samplingdecode

GQA shared proposal: for H heads sharing one proposal r, Σ_h Σ_j π_hj²/r_j ≥ (Σ_j √(Σ_h π_hj²))²

For all H,J∈Type⁡\htmlData{mx=1}{H, J \in \htmlData{mx=0}{\operatorname{Type}}}, π:H→J→R\htmlData{mx=5}{\pi : \htmlData{mx=0}{H} \mathrel{\htmlData{mx=4}{\to}} \htmlData{mx=1}{J} \mathrel{\htmlData{mx=3}{\to}} \htmlData{mx=2}{\mathbb{R}}} and r:J→R\htmlData{mx=3}{r : \htmlData{mx=0}{J} \mathrel{\htmlData{mx=2}{\to}} \htmlData{mx=1}{\mathbb{R}}}, if ∀j, 0<r(j)\htmlData{mx=4}{\forall j,\ 0 \mathrel{\htmlData{mx=3}{<}} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} and ∑jr(j)=1\htmlData{mx=3}{\sum_{j} \htmlData{mx=2}{\htmlData{mx=0}{r}(\htmlData{mx=1}{j})}} \mathrel{\htmlData{mx=4}{=}} 1 (given Fintype⁡ H\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{H}} and Fintype⁡ J\htmlData{mx=2}{\htmlData{mx=0}{\operatorname{Fintype}}\,\htmlData{mx=1}{J}}), then

(∑j∑hπ(h,j)2)2≤∑h∑jπ(h,j)2r(j)\htmlData{mx=8}{\left(\htmlData{mx=7}{\sum_{j} \htmlData{mx=6}{\sqrt{\htmlData{mx=5}{\sum_{h} \htmlData{mx=4}{\htmlData{mx=3}{\htmlData{mx=0}{\pi}(\htmlData{mx=1}{h}, \htmlData{mx=2}{j})}^{2}}}}}}\right)^{2}} \mathrel{\htmlData{mx=20}{\leq}} \htmlData{mx=19}{\sum_{h} \htmlData{mx=18}{\sum_{j} \htmlData{mx=17}{\frac{\htmlData{mx=13}{\htmlData{mx=12}{\htmlData{mx=9}{\pi}(\htmlData{mx=10}{h}, \htmlData{mx=11}{j})}^{2}}}{\htmlData{mx=16}{\htmlData{mx=14}{r}(\htmlData{mx=15}{j})}}}}}
What the notation means · or point at any symbol
Type
Type: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Type)
x ∈ TVariables
H, J range over Type: the statement must hold for every choice. ((H J : Type))
Fintype
Fintype: a definition from the library, not special notation. Its name is its meaning; open it in Lean to see how it is defined. (Fintype)
f(x)Function application
Fintype applied to H. Lean writes f x; papers write f(x). (Fintype H)
∀For all
The statement after the comma holds for every j. (∀ j, 0 < r j)
∑Summation
Add up r(j) for every j. (∑ j, r j)
=Equality
∑ j, r(j) and 1 are the same. (∑ j, r j = 1)
≤At most
(∑ j, √(∑ h, π(h, j)^2))^2 is less than or equal to ∑ h, ∑ j, π(h, j)^2/r(j). ((∑ j, Real.sqrt (∑ h, π h j ^ 2)) ^ 2 ≤…)
⟹Implies
If J, then ℝ. (Between types, A → B is the type of functions from A to B.) (J → ℝ)
<Less than
0 is strictly smaller than r(j). (0 < r j)
√Square root
The non-negative number whose square is ∑ h, π(h, j)^2. (Real.sqrt (∑ h, π h j ^ 2))
RReal numbers
The real numbers: every point on the number line. (ℝ)
πPi
The ratio of a circle's circumference to its diameter, ≈ 3.14159. (π)
aᵇPower
π(h, j) raised to the power 2. (π h j ^ 2)
a/bDivision
π(h, j)^2 divided by r(j). In Lean, dividing by 0 gives 0. (π h j ^ 2 / r j)

When several query heads share keys (grouped-query attention), one sampled key serves every head in the group. The total importance-sampling cost of a shared proposal is at least A² with A = Σ_j ‖π_·j‖₂ (weighted Cauchy–Schwarz). The optimal shared proposal ∝ √(Σ_h π_hj²) is the known multi-integrand importance-sampling optimum (credited); the application to KV groups is part of 'The Sampleability Law' (GQA decode-traffic extension).

How it was proved

Proved
5 October 2026
By
Ajax, with the agent “Baguette math & proofs (Claude Code, 2026-10-03)”
Checked by
Lean's kernel (Lean 4.33.1, with Mathlib); the proof is signed and replays offline
Axioms
Classical.choice, Quot.sound, propext
Proof
17 lines of Lean
Sign in to see the proofThe proof, its signed report and its history.

More proved statements