How one decode step interacts with a cache under compiled decode
(returned by prepareDecodeStep, consumed by compiled-decode.ts):
"concat": the compiled graph fetches concat(active prefix, new kv)
— same values as today's write-then-slice — and the WRITE happens
outside the graph right after (writeDecodeStep), keeping the buffer
single-referenced at its slice_update so mlx donates it in place.
"ring": the write happens IN-graph (slice_update_dynamic at
writePos) and the fetch is the full updated buffer — the rotating
steady state, where attention reads the whole ring in ring order
(a concat would permute KV positions and change summation order).
The updated buffers come back as closure outputs; adoptDecodeStep
swaps them in.
How one decode step interacts with a cache under compiled decode (returned by prepareDecodeStep, consumed by compiled-decode.ts):