Leave values rotated, run the stock fused SDPA under the mask on them, then
undo the rotation on the output: attention is linear in V, so one inverse
rotation per query row replaces one per cached token. The Gemma 4 graphs
(generic and written), DiffusionGemma and MiniCPM5 compose this set; it computes their served
numbers.
Leave values rotated, run the stock fused SDPA under the mask on them, then undo the rotation on the output: attention is linear in V, so one inverse rotation per query row replaces one per cached token. The Gemma 4 graphs (generic and written), DiffusionGemma and MiniCPM5 compose this set; it computes their served numbers.