mlx-bun library API
    Preparing search index...

    Interface GenerateStats

    interface GenerateStats {
        cachedTokens: number;
        cacheTokens: number[];
        decodeMs: number;
        decodeTps: number;
        fill?: FillStats;
        finishReason?: "length" | "stop";
        generatedTokens: number;
        prefillMs: number;
        prefillTps: number;
        promptTokens: number;
        spec?: {
            acceptanceLengths?: number[];
            accepted: number;
            acceptedByPos?: number[];
            drafted: number;
            draftedByPos?: number[];
            forwardsSaved?: number;
            phaseMs?: SpecPhaseMs;
            rejected?: number;
            rounds?: number;
            targetCalls: number;
            tokensPerForward?: number;
        };
    }
    Index
    cachedTokens cacheTokens decodeMs decodeTps fill? finishReason? generatedTokens prefillMs prefillTps promptTokens spec?
    cachedTokens: number

    Prompt tokens skipped via a pre-warmed cache.

    cacheTokens: number[]

    Exact token sequence whose KV now lives in the cache (prompt + every decoded token that was forwarded, including a trailing EOS the pipeline forwarded before reading it). For PromptCache.put().

    decodeMs: number
    decodeTps: number
    fill?: FillStats

    Token fast-forwarding telemetry (MLX_BUN_FILL=strict). Present only when a fill table was actually armed for this generation.

    finishReason?: "length" | "stop"

    An observed terminal cause takes precedence over the token-count limit.

    generatedTokens: number
    prefillMs: number
    prefillTps: number
    promptTokens: number
    spec?: {
        acceptanceLengths?: number[];
        accepted: number;
        acceptedByPos?: number[];
        drafted: number;
        draftedByPos?: number[];
        forwardsSaved?: number;
        phaseMs?: SpecPhaseMs;
        rejected?: number;
        rounds?: number;
        targetCalls: number;
        tokensPerForward?: number;
    }

    Speculative-decoding telemetry (serve --draft-model path only). draftedByPos/acceptedByPos: per-draft-position round counts (index = position within a round's block) — the per-position acceptance signal.

    Type Declaration

    • OptionalacceptanceLengths?: number[]
    • accepted: number
    • OptionalacceptedByPos?: number[]
    • drafted: number
    • OptionaldraftedByPos?: number[]
    • OptionalforwardsSaved?: number
    • OptionalphaseMs?: SpecPhaseMs

      Round phase wall time, summed over the request's rounds. Present only under MLX_BUN_SPEC_PHASE_TIMING=1, which forces an evaluation at each phase boundary (diagnostic: it removes the overlap production decode relies on, so the sum is not production round time).

    • Optionalrejected?: number
    • Optionalrounds?: number
    • targetCalls: number
    • OptionaltokensPerForward?: number

    Settings
    Member Visibility

    On This Page
    Properties