OptionaladaptersOptionalcachePre-warmed KV caches (e.g. from the prompt cache). cache[0].offset prompt tokens are treated as already prefilled; only the suffix is forwarded. Caller keeps ownership — generate() will not dispose.
OptionalcacheApplication session affinity; independent of numerical cache identity.
OptionalcheckpointPeriodic, safe decode-boundary checkpoint hook. The cache covers
cacheTokens; pendingToken has been sampled but not emitted.
Optionalcurvev2 log-prob transfer-curve sampler. When set, the drawn monotone curve REPLACES temperature+softmax entirely (stochastic, seeded) — see packages/inference/src/sampling/curve.ts. Identity curve ≡ temperature 1.
OptionaldecodeResolved host policy. Direct compatibility calls resolve from their captured binding when this is absent.
OptionaleosOptionalfillToken fast-forwarding (packages/inference/src/generation/fill/,
02d723a:docs/design/speculative-decoding.md §7 "Token fast-forwarding"):
a per-request table of DETERMINED token spans
(compiled from the request's tools + the chat template). When the
stream enters one, the engine appends the whole span itself with ONE
multi-token forward and resumes sampling after it.
This is context extension, NOT speculation: nothing is drafted, verified, or rolled back, and no comparison is made against what the model would have produced. Injection bypasses the sampler, which is a documented behavior-policy deviation at temperature > 0 — hence opt-in (MLX_BUN_FILL=strict) and the request refusals in shouldUseFill. Serial lane only: the batch scheduler and the spec loop never read it.
OptionalfrequencyOptionalfrequencyOpenAI-style frequency penalty: subtracted once PER OCCURRENCE of the token in the recent window. May be negative.
OptionalgrammarGrammar-constrained decoding (packages/inference/src/sampling/grammar.ts): a compiled GrammarController that masks invalid tokens to -inf each step. L2-class (oMLX oracle). When set, the decode loop takes a slightly different shape: it eager-reads the token id (acceptToken needs a JS number, which the pipelined loop defers), advances the matcher, and awaits the async mask precompute (which overlaps the GPU forward). Non-grammar requests keep the fast pipelined loop.
OptionalhlgHLG tone-curve sampling. When enabled, replaces temperature's flat slope
with the piecewise curve (temperature becomes the mid gain). Off/undefined
⇒ the plain temperature path, unchanged. 02d723a:docs/archive/hlg-sampling.md.
Optionalimagebool [L] marking image tokens (bidirectional attention among them).
MUST be unset when the prompt contains any audio — audio prompts run
fully causal (02d723a:docs/design/generic-model-support.md §6.6).
OptionalinitialNumber of completion tokens already emitted before this resumed run.
OptionalinitialResume an interrupted autoregressive generation from a durable KV
checkpoint. cache must cover every token in promptTokens exactly;
initialPendingToken is the already-sampled next token, so no cache
rewind is required (important for recurrent/untrimmable layers).
OptionallogitAdditive per-token-id logit bias, applied before any penalty
(mlx-lm's logit_bias: {tokenId: bias}).
OptionallogprobsCapture each emitted token's log-probability (mlx_lm.server logprobs).
The distribution matches mlx-lm generate_step exactly: full-vocab
log-softmax of the logits AFTER logits processors, BEFORE the sampler's
temperature/top-p/top-k/min-p/XTC (generate.py L409-422). Off by default —
the hot path pays nothing when unset.
OptionalmaxOptionalminmin-p: keep tokens whose probability ≥ minP · p(top token). mlx-lm's
min_p. Applied after top-p, before XTC/top-k (reference chain order).
OptionalminMinimum number of tokens min-p may never filter out. Default 1.
Optionalmultimodalbool [L] marking ALL multimodal soft tokens (image | audio) for per-layer-input id zeroing (e2b/e4b), decoupled from imageMask so audio-only prompts (no bidirectional mask) still zero their positions. When unset, zeroing falls back to imageMask (the legacy vision shape). Caller keeps ownership.
OptionalonOptionalonFired once during prefill, at snapshotAt tokens (default: the full
prompt) and BEFORE any further KV is written — the one moment the
caches hold exactly that prefix (post-wrap rings can't be rewound
later). The server's prompt-boundary cache snapshot hangs here.
OptionaloriginalOriginal request prompt length used for protocol usage accounting when
promptTokens also contains replayed completion tokens.
OptionalprefillOptionalpresenceOptionalpresenceOpenAI-style presence penalty: subtracted ONCE from a token's logit if it occurred at all in the recent window. May be negative.
OptionalpromptVision path: pre-merged (unscaled) input embeddings [1, L, hidden] covering the whole prompt; prefilled in one shot (no chunking). Caller keeps ownership.
OptionalrepetitionOptionalrepetitionOptionalseedOptionalseedWhether the request supplied its seed explicitly. Durable generation checkpoints use this to distinguish a reproducible user policy from the server-generated seed that must be recovered from checkpoint metadata.
OptionalsignalCooperatively cancel at prefill/decode boundaries; pending native work finishes before owned resources are released.
OptionalsnapshotToken count at which onPrefillDone fires (the STABLE cache boundary — chat prompts end in a generation primer, e.g. 12B's thought-channel tokens, that the NEXT turn's re-render does not contain; snapshotting there would make the entry untrimmable-and-divergent). Prefill is split at this index when it falls mid-prompt.
OptionaltemperatureOptionaltopOptionaltopCapture the top-k (token id, logprob) pairs per emitted token from the
same distribution (mlx_lm.server top_logprobs). 0/unset = off.
OptionaltopOptionalvisionDiffusionGemma image-text-to-text: channel-first pixel_values [1,3,H,W].
When set, promptTokens are the spliced (<|image|>-expanded) ids and the
denoising engine prefills the merged vision features. Caller keeps ownership.
OptionalxtcXTC ("exclude top choices"): with this probability per step, drop every token above the threshold EXCEPT the least likely of them.
OptionalxtcToken ids XTC never removes (mlx-lm's server passes EOS + "\n").
OptionalxtcProbability a token must exceed to be an XTC removal candidate. [0, 0.5].
Mounted LoRA adapter ids to apply (resolved/validated by AdapterManager.resolveSpec). Residuals sum in order. Set on the model's LoraState for exactly the duration of this generation — a plain field, safe because the generation queue is serialized.