mlx-bun library API
    Preparing search index...

    Type Alias SharedKv

    SharedKv:
        | {
            keys: MlxArray;
            kind: "plain";
            offset: number;
            offsetArr?: MlxArray;
            restoreValues?: (output: MlxArray) => MlxArray;
            values: MlxArray;
            vRotated?: boolean;
        }
        | {
            bits: number;
            groupSize: number;
            keys: QuantizedTensor;
            kind: "quant";
            offset: number;
            offsetArr?: MlxArray;
            values: QuantizedTensor;
        }
        | {
            attention: KvAttentionView;
            kind: "view";
            offset: number;
            offsetArr?: MlxArray;
        }

    Fetched KV handed from a donor layer to its sharers (and to the speculative drafter). Owned by forwardLayers for the pass. offsetArr is an owned pre-write position snapshot. Donor and sharing attention layers keep it until the fetched KV is released, even if the cache advances or replaces its borrowed row-position array.

    Type Declaration

    • {
          keys: MlxArray;
          kind: "plain";
          offset: number;
          offsetArr?: MlxArray;
          restoreValues?: (output: MlxArray) => MlxArray;
          values: MlxArray;
          vRotated?: boolean;
      }
      • keys: MlxArray
      • kind: "plain"
      • offset: number
      • OptionaloffsetArr?: MlxArray
      • OptionalrestoreValues?: (output: MlxArray) => MlxArray
      • values: MlxArray
      • OptionalvRotated?: boolean

        TurboQuant deferred-V: values are still in the rotated (FWHT) domain — every consumer must un-rotate its attention OUTPUT (tq.unrotateValues) after sdpa. Attention is linear in V, so the result is the same decode, just transformed once per query row instead of once per cached token.

    • {
          bits: number;
          groupSize: number;
          keys: QuantizedTensor;
          kind: "quant";
          offset: number;
          offsetArr?: MlxArray;
          values: QuantizedTensor;
      }
    • {
          attention: KvAttentionView;
          kind: "view";
          offset: number;
          offsetArr?: MlxArray;
      }

    Settings
    Member Visibility