Class TwoModelProvider

Server-lifetime owner of the draft machinery (the loaded draft model). Its grouped rows are how every request drafts, one request or several.

Implements

DraftProvider

Index

Properties

grouped id model weightsBytes

Methods

Properties

`Readonly`grouped

grouped: GroupedDraftProvider

`Readonly`id

id: string

Human-readable id (registry id / path tail) for logs + cache namespacing.

`Readonly`model

model: RuntimeModel

`Readonly`weightsBytes

weightsBytes: number

Methods

dispose

dispose(): void
The draft graph releases what it built (constants, Neural Engine programs), then its weights go.

Returns void
Implementation of DraftProvider.dispose
- Defined in packages/inference/src/generation/speculative/sources/two-model.ts:72

`Static`load

load(modelDir: string, targetVocabSize?: number): Promise<TwoModelProvider>
Load the draft model from a snapshot dir. Vocab-size mismatch with the target is a WARNING (mlx-lm parity, server.py:363-368); the tokenizer- family hard check lives at the server (it owns both tokenizers).
Parameters
- modelDir: string
- OptionaltargetVocabSize: number
Returns Promise<TwoModelProvider>
- Defined in packages/inference/src/generation/speculative/sources/two-model.ts:49