mlx-bun library API
    Preparing search index...

    Interface MlxGroupMethodHost

    interface MlxGroupMethodHost {
        cache: RequestStateCache;
        kvMaintenance: KvMaintenance | undefined;
        prefillChunkSize: number;
        prefillTailSplit: boolean;
        rows: readonly Row[];
        runtime: RuntimeConfig;
        filterRows(keep: number[]): void;
        finish(row: Row, reason: "length" | "stop"): void;
        join(row: Row): void;
        publish(
            row: Row,
            token: number,
            logprobs?: TokenLogprobs,
        ): Promise<boolean | void>;
    }
    Index
    cache kvMaintenance prefillChunkSize prefillTailSplit rows runtime
    filterRows finish join publish
    cache: RequestStateCache

    The composition's cache service: every row's state comes from it and returns to it.

    kvMaintenance: KvMaintenance | undefined

    The row-layout maintenance of the composition's KV scheme (where a row's composed caches join row layouts); absent under bf16.

    prefillChunkSize: number
    prefillTailSplit: boolean

    The composition's prefillTailSplit: a prompt's last token prefills alone.

    rows: readonly Row[]
    runtime: RuntimeConfig

    Settings
    Member Visibility

    On This Page
    Properties
    Methods