mlx-bun library API
    Preparing search index...

    Attention over a cache's positions as of one phase-named call. The cache picked the kernel when it was built and builds the mask from the offsets, row layout and sliding window it owns. The caller supplies only queries.

    The read owns its tensor handles. It stays valid after later appends, trims, row changes or disposal of its cache. Holding it across the cache's next append can make that append copy storage instead of writing in place, so dispose it when the forward that made it is done. It does no attention work before attend; disposing an unused read only releases handles.

    interface AttentionRead {
        attend(q: MlxArray, scale: number): MlxArray;
        dispose(): void;
    }
    Index
    • Queries q [B, Hq, L, Dk] (borrowed, positional encoding applied, Hq a multiple of the stored Hkv) to an owned output [B, Hq, L, Dv]. B and L must match the call that made this read, and the result is defined only for that call's phase. Callable any number of times: layers that share a donor's keys and values attend the donor's read with their own queries.

      Parameters

      Returns MlxArray