mlx-bun library API
    Preparing search index...

    Interface SamplingSession<Scores, DeviceToken, Result>

    interface SamplingSession<Scores, DeviceToken, Result> {
        capturesLogprobs: boolean;
        independent?: { sample(scores: Scores): DeviceToken };
        isPlainGreedy: boolean;
        needsHistory: boolean;
        positional?: {
            commit(tokens: readonly number[]): void;
            sample(
                scores: Scores,
                steps: readonly number[],
                inputs?: DeviceToken,
            ): DeviceToken;
        };
        unread?: { sample(scores: Scores, step: number): DeviceToken };
        commitDevice(token: DeviceToken): void;
        commitNumbers(tokens: readonly number[]): void;
        dispose(): void;
        sample(logits: Scores, step: number): Result;
        seedHistory(tokens: readonly number[]): void;
    }

    Type Parameters

    • Scores
    • DeviceToken
    • Result
    Index
    capturesLogprobs: boolean
    independent?: { sample(scores: Scores): DeviceToken }

    Optional stateless operation shared across compatible rows and positions. It borrows scores and returns owned device tokens, without advancing history.

    isPlainGreedy: boolean
    needsHistory: boolean
    positional?: {
        commit(tokens: readonly number[]): void;
        sample(
            scores: Scores,
            steps: readonly number[],
            inputs?: DeviceToken,
        ): DeviceToken;
    }

    Optional per-position operation for ONE request whose selection depends on no grammar or probability capture: position p's draw depends only on its scores, its step and its processor history. Borrows [N,V] scores and returns owned device tokens [N] for the given steps, without advancing history. A request with processor history (needsHistory) also passes the window's inputs [N] (borrowed: the pending token, then the drafts): position p reads its history extended by inputs[1..p], the history sample() would hold there had every earlier position drawn its draft. Draws are those sample() would make at each step: seeded sampling keys by (seed, step), never by a shared stream, so unused positions cost nothing. commit takes the tokens a walk over the window used, in order, and advances history as sample() would have advanced it drawing each.

    unread?: { sample(scores: Scores, step: number): DeviceToken }

    Optional draw left on the device for a request whose selection depends on its processor history but on no grammar or probability capture: borrows [1,V] scores and returns the owned device token sample() would draw at step, committing it to history unread when sample() would commit it.