mlx-bun library API
    Preparing search index...

    Interface BatchStats

    interface BatchStats {
        cachedTokens: number;
        decodeMs: number;
        fill?: FillStats;
        finishReason: "length" | "stop";
        generatedTokens: number;
        prefillMs: number;
        promptTokens: number;
        spec?: {
            acceptanceLengths?: number[];
            accepted: number;
            acceptedByPos?: number[];
            drafted: number;
            draftedByPos?: number[];
            forwardsSaved?: number;
            phaseMs?: SpecPhaseMs;
            rejected?: number;
            rounds?: number;
            targetCalls: number;
            tokensPerForward?: number;
        };
    }
    Index
    cachedTokens decodeMs fill? finishReason generatedTokens prefillMs promptTokens spec?
    cachedTokens: number

    Prompt tokens served from the prompt cache: a joiner's prefill starts from the longest usable cached prefix the cache service acquires. 0 on a cold prefill.

    decodeMs: number

    Wall-clock from the first emitted token to finish.

    fill?: FillStats
    finishReason: "length" | "stop"
    generatedTokens: number
    prefillMs: number

    Wall-clock from admission (leaving the pending queue) to the first emitted token — the row's prefill span on the batch lane.

    promptTokens: number
    spec?: {
        acceptanceLengths?: number[];
        accepted: number;
        acceptedByPos?: number[];
        drafted: number;
        draftedByPos?: number[];
        forwardsSaved?: number;
        phaseMs?: SpecPhaseMs;
        rejected?: number;
        rounds?: number;
        targetCalls: number;
        tokensPerForward?: number;
    }

    Type Declaration

    • OptionalacceptanceLengths?: number[]
    • accepted: number
    • OptionalacceptedByPos?: number[]
    • drafted: number
    • OptionaldraftedByPos?: number[]
    • OptionalforwardsSaved?: number
    • OptionalphaseMs?: SpecPhaseMs

      Round phase wall time, summed over the request's rounds. Present only under MLX_BUN_SPEC_PHASE_TIMING=1, which forces an evaluation at each phase boundary (diagnostic: it removes the overlap production decode relies on, so the sum is not production round time).

    • Optionalrejected?: number
    • Optionalrounds?: number
    • targetCalls: number
    • OptionaltokensPerForward?: number

    Settings
    Member Visibility

    On This Page
    Properties