Interface MlxDecodeStep

interface MlxDecodeStep {
    close(): void | Promise<void>;
    step(
        token: MlxArray,
        state: Cache[],
    ): { evalWith: MlxArray[]; logits: MlxArray };
}

Index

Methods

Methods

close

close(): void | Promise<void>
Release per-run workspaces after pending outputs are dropped and before cache disposal. Wait for native work if releasing requires completion.

Returns void | Promise<void>
- Defined in packages/inference/src/generation/bindings/autoregressive.ts:22

step

step(
token: MlxArray,
state: Cache[],
): { evalWith: MlxArray[]; logits: MlxArray }
Consume one pending token and advance state once. Returned arrays belong to the caller. A step that throws fails the generation.
Parameters
- token: MlxArray
- state: Cache[]
Returns { evalWith: MlxArray[]; logits: MlxArray }
- Defined in packages/inference/src/generation/bindings/autoregressive.ts:19