mlx-bun library API
    Preparing search index...

    Interface WhisperModelHostOptions

    interface WhisperModelHostOptions {
        admit?: (bytes: number) => Promise<void>;
        backend?: WhisperBackend;
        catalog: Pick<ModelCatalog, "list" | "find">;
        configured?: { directory: string; id: string };
        events?: Pick<EventBus, "publish">;
        exclusive?: Exclusive;
        idleUnloadSec?: number;
        log?: (line: string) => void;
        now?: () => number;
        resident?: boolean;
        timers?: {
            clearTimeout(handle: unknown): void;
            setTimeout(fn: () => void, ms: number): unknown;
        };
    }
    Index
    admit? backend? catalog configured? events? exclusive? idleUnloadSec? log? now? resident? timers?
    admit?: (bytes: number) => Promise<void>

    Called with the bytes a checkpoint needs before it loads, so a host that also generates can make room (drain a model) first.

    backend?: WhisperBackend

    Real weights by default.

    catalog: Pick<ModelCatalog, "list" | "find">

    Resolves ids and paths to checkpoint directories, and lists what is local.

    configured?: { directory: string; id: string }

    The --whisper-model checkpoint: served under this id from this directory, and the default for transcribe.

    events?: Pick<EventBus, "publish">

    Receives model.load, model.unload and model.memory events; publishing never waits for a subscriber.

    exclusive?: Exclusive

    Wraps every decode call (the full server passes the generation gateway's exclusive lock).

    idleUnloadSec?: number

    Seconds a model stays loaded after its last lease. Default 0: released right after every take.

    log?: (line: string) => void
    now?: () => number

    Monotonic milliseconds for the reported load times.

    resident?: boolean

    Start with the configured model pinned (--whisper-resident).

    timers?: {
        clearTimeout(handle: unknown): void;
        setTimeout(fn: () => void, ms: number): unknown;
    }

    Idle-unload scheduling; tests supply a fake clock.

    Settings
    Member Visibility

    On This Page
    Properties