Class DraftVocabularyHead

Rows of a quantized head (affine packs along the input axis, so a row gather is exact). Owns its gathered arrays; borrows nothing afterwards.

Index

Constructors

Properties

Methods

dispose project

Constructors

constructor

new DraftVocabularyHead(
head: QuantizedHeadState,
ids: readonly number[],
): DraftVocabularyHead
Parameters
- head: QuantizedHeadState
- ids: readonly number[]
Returns DraftVocabularyHead
- Defined in packages/inference/src/generation/speculative/draft-vocab.ts:63

Properties

`Readonly`ids

`Readonly`size

size: number

Methods

dispose

dispose(): void
Returns void
- Defined in packages/inference/src/generation/speculative/draft-vocab.ts:79

project

project(hidden: MlxArray): MlxArray
hidden [..., H] → logits [..., N] over the listed ids.
Parameters
- hidden: MlxArray
Returns MlxArray
- Defined in packages/inference/src/generation/speculative/draft-vocab.ts:75