The KV4 kernel set over the composition's affine set plain: the 4-bit group-64 head-dim-256 reads (kv4Head256Kernels, the decode kernel at decodeBlock tokens per threadgroup) for one sequence, and plain's kernels for a row layout holding more.
plain
kv4Head256Kernels
decodeBlock
The KV4 kernel set over the composition's affine set
plain: the 4-bit group-64 head-dim-256 reads (kv4Head256Kernels, the decode kernel atdecodeBlocktokens per threadgroup) for one sequence, andplain's kernels for a row layout holding more.