Class Dflash2Provider

Server-lifetime owner of the draft machinery (the loaded draft model). Its grouped rows are how every request drafts, one request or several.

Implements

DraftProvider

Index

Properties

gamma grouped id weightsBytes

Methods

Properties

`Readonly`gamma

gamma: number

`Readonly`grouped

grouped: GroupedDraftProvider

`Readonly`id

id: string

Human-readable id (registry id / path tail) for logs + cache namespacing.

`Readonly`weightsBytes

weightsBytes: number

Methods

dispose

dispose(): void
Returns void
Implementation of DraftProvider.dispose
- Defined in packages/inference/src/generation/speculative/sources/dflash2-source.ts:85

`Static`load

load(modelDir: string, options: { bits: number }): Promise<Dflash2Provider>
bits: the drafter's projection width (0 keeps BF16). Persisted context is keyed by the drafter and that width; the target's own identity scopes it to one residual basis.
Parameters
- modelDir: string
- options: { bits: number }
Returns Promise<Dflash2Provider>
- Defined in packages/inference/src/generation/speculative/sources/dflash2-source.ts:72