Interface MlxGroupedMethod

interface MlxGroupedMethod {
    runningTokens?: number;
    advance(work?: MlxForwardWork): Promise<void>;
    dispose(): void;
    filterRows(keep: readonly number[], discard: boolean): void;
    prepare(row: Row): MlxGroupPreparation;
}

Index

Properties

Methods

advance dispose filterRows prepare

Properties

`Optional` `Readonly`runningTokens

runningTokens?: number

Maximum target tokens in the next iteration, including candidates.

Methods

advance

advance(work?: MlxForwardWork): Promise<void>
Parameters
- Optionalwork: MlxForwardWork
Returns Promise<void>
- Defined in packages/inference/src/execution/batch-types.ts:60

dispose

dispose(): void
Returns void
- Defined in packages/inference/src/execution/batch-types.ts:62

filterRows

filterRows(keep: readonly number[], discard: boolean): void
Parameters
- keep: readonly number[]
- discard: boolean
Returns void
- Defined in packages/inference/src/execution/batch-types.ts:61

prepare

prepare(row: Row): MlxGroupPreparation
Parameters
- row: Row
Returns MlxGroupPreparation
- Defined in packages/inference/src/execution/batch-types.ts:59