mlx-bun library API
    Preparing search index...

    Interface PromptCacheEntry

    A reusable prefix and the backend/method state that covers its tokens. The caller owns a returned hit. After releasing or returning its state, it must release any retained backing through retain.

    interface PromptCacheEntry {
        attachments?: CheckpointAttachment[];
        caches: Cache[];
        ns: string;
        retain?: () => void;
        tokens: number[];
    }

    Hierarchy (View Summary)

    Index
    attachments? caches ns retain? tokens
    attachments?: CheckpointAttachment[]
    caches: Cache[]
    ns: string

    Namespace key — adapter spec for LoRA requests ("" = base model). KV computed under one adapter must never seed another's prefill.

    retain?: () => void

    Release hook for entries restored from the SSD cold tier: unmaps the backing COW file mapping. MUST run only after caches are disposed (they may alias file pages); every dispose path here honors that, and callers who take() an entry carry the thunk with it.

    tokens: number[]

    Settings
    Member Visibility

    On This Page
    Properties