mlx-bun library API
    Preparing search index...

    Interface KvCacheServiceOptions

    interface KvCacheServiceOptions {
        codecs?: CacheCodecProvider;
        identity?: unknown;
        kvBudgetBytes?: number;
        pressure?: (heldBytes: number) => boolean;
        promptCacheBytes?: number;
        retention?: RetentionPolicy;
        runtime?: RuntimeConfig;
        saved?: SavedStateOptions;
    }

    Hierarchy

    • Pick<KvCacheSettings, "kvBudgetBytes" | "codecs" | "identity" | "runtime">
      • KvCacheServiceOptions
    Index
    codecs? identity? kvBudgetBytes? pressure? promptCacheBytes? retention? runtime? saved?
    codecs?: CacheCodecProvider

    The snapshot form of the composition's caches (default the graph's stateCodecs, else the built-in codecs).

    identity?: unknown

    Artifact, implementation, state ABI and codec identity captured at load: part of every checkpoint key.

    kvBudgetBytes?: number

    Aggregate projected KV bytes the admitted requests may hold (--kv-budget); absent: unlimited.

    pressure?: (heldBytes: number) => boolean

    Whether live memory is over what optional state may use, given the bytes the prefix store holds or has queued for writing.

    promptCacheBytes?: number

    RAM bytes of reusable prefixes (--prompt-cache), 0 keeping none; absent: no prefix store.

    retention?: RetentionPolicy
    runtime?: RuntimeConfig

    The scope paged state is built in.

    saved?: SavedStateOptions

    Settings
    Member Visibility

    On This Page
    Properties