mlx-bun library API
    Preparing search index...

    Interface WhisperModelHost

    ModelHost for Whisper checkpoints, plus what the composing host needs beyond the contract: preload and close.

    interface WhisperModelHost {
        policy: ResidencyPolicy;
        acquire(id: string, options?: AcquireOptions): Promise<ModelLease>;
        close(): Promise<void>;
        defaultFor(operation: ModelOperation): Promise<string | undefined>;
        pin(id: string): void;
        plan(id: string): Promise<ResidencyPlan>;
        preload(id?: string): Promise<void>;
        resident(): readonly ResidentModel[];
        serve(id: string, options?: { signal?: AbortSignal }): Promise<void>;
        stats(id: string): ModelStats;
        unload(
            id: string,
            options?: { flush?: boolean; force?: boolean },
        ): Promise<void>;
        unpin(id: string): void;
    }

    Hierarchy (View Summary)

    Index
    policy
    acquire close defaultFor pin plan preload resident serve stats unload unpin
    policy: ResidencyPolicy

    • Makes id the model defaultFor("generate") answers, loading it beside the resident ones when it fits, else in place of the least recently used unpinned one (its state is saved and resumes on return). Rejects with not-found for an id that is not a local model, not-switchable on a host that holds one model, and otherwise as acquire does.

      Parameters

      • id: string
      • Optionaloptions: { signal?: AbortSignal }

      Returns Promise<void>

    • Drains, optionally flushes state (default true), then releases. Rejects with in-use while a lease is held unless force; a model that is not resident resolves.

      Parameters

      • id: string
      • Optionaloptions: { flush?: boolean; force?: boolean }

      Returns Promise<void>

    Settings
    Member Visibility

    On This Page
    Properties
    Methods