mlx-bun library API
    Preparing search index...

    How one decode step interacts with a cache under compiled decode (returned by prepareDecodeStep, consumed by compiled-decode.ts):

    • "concat": the compiled graph fetches concat(active prefix, new kv) — same values as today's write-then-slice — and the WRITE happens outside the graph right after (writeDecodeStep), keeping the buffer single-referenced at its slice_update so mlx donates it in place.
    • "ring": the write happens IN-graph (slice_update_dynamic at writePos) and the fetch is the full updated buffer — the rotating steady state, where attention reads the whole ring in ring order (a concat would permute KV positions and change summation order). The updated buffers come back as closure outputs; adoptDecodeStep swaps them in.
    interface DecodeStepPlan {
        activeLen: number;
        fetch: "concat" | "ring";
        writePos: number;
    }
    Index
    activeLen: number

    Valid prefix length for "concat" fetches (== offset).

    fetch: "concat" | "ring"
    writePos: number

    Write position on axis 2 (== ring index for rotating caches).