Optional ReadonlycomposedPresent on a cache that composes the storage it holds (see ComposedStorage).
OptionalminimumEarliest prefix whose retained representation can resume generation. Irreversible transitions may preserve bytes while invalidating older precision boundaries. Reuse policy must not trim below this offset.
Optional ReadonlyprefillA preparation method commits each request's own precision boundary. Splitting a forward for a sibling must not move that boundary.
Optional ReadonlyropeCompiled-decode trace adapters expose the RoPE offset as an int32 array input here; real caches leave it unset (static int path).
Optional Readonlystatestate() returned temporary views that the caller must release.
OptionalbytesPhysical cache storage consumed by one additional token for one row. Recurrent caches return 0 because their state does not grow with the sequence. Optional for stateless/training adapters that are never admitted or persisted.
OptionalcaptureOptionalcaptureCan trim(n) drop the last n tokens? (Ring caches lose trimability
once wrapped.)
OptionalmaxMaximum committed positions before this state changes precision.
Decode: one new position per row (S = 1), with no prefill padding in
effect.
Window: S positions per row, for prefill chunks and tails, speculative
verify windows and committed spans. Padding prepared through
PaddedPrefillCache holds: padded positions change neither state, each
row's convolution tail ends at its last real position, and each row's offset
advances by its real count. Outputs at padded positions are unspecified.
Stable storage identity for compatibility guards and persistence. REQUIRED: every capability predicate (isPlainKvCache, isRotating*, packages/inference/src/state/persistence.ts codecs) is a string compare on this value, so a missing override used to fail OPEN into wrong routing (PRs #42/#43: BatchedRotatingCache shipped without one and batched joins dropped running rows). Wrappers report their own kind plus the inner kind.
OptionalspecSpeculative verify-round support for NON-trimmable recurrent caches
(SSMCache — gated-DeltaNet conv + recurrent state). A cache exposing
these is spec-eligible without isTrimmable(): the serve loop arms a
round before the verify forward (the layer snapshots pre-round state —
free, MLX arrays are immutable — and records its position-local kernel
inputs), then resolves it: commit on full accept, or rollback(keep)
which restores the snapshot and bit-exactly REPLAYS the first keep
window tokens through the recurrence (identical arithmetic prefix ⇒
identical state). Trimmable caches leave these unset and keep trim().
OptionalspecOptionalspecDrop the last n tokens (future writes overwrite them). bypass (default
false → unchanged behavior) skips the isTrimmable guard for the
speculative-decode rollback case (trim the last n ≤ γ tips right after a
1 concat write); rotating caches then physically shrink the buffer tail.
Optionalbypass: boolean
Recurrent state for one gated DeltaNet layer. It owns its read the way AttentionCache does: the block hands over its projected inputs and receives the recurrence output. The cache zeroes padded positions, runs the causal convolution over its stored convolution state, keeps each row's convolution tail, runs the gated-delta recurrence with the kernels in
kernels/delta, replaces both states and advances its offsets. A training cache runs the differentiable recurrence behind the same calls.Both calls have these obligations:
qkv[B, S, convDim] is thein_proj_qkvoutput before convolution, anda,b[B, S, Hv] are the rawin_proj_aandin_proj_boutputs. The call consumes all three: the cache disposes them, or keeps them for an armed speculative round. The caller must not use or dispose them after the call.layeris borrowed and must stay valid until an armed speculative round resolves, because rollback replays through it.z) and the output projection itself.specRoundBegin), the call records the pre-call states and its inputs, sospecRoundRollback(keep)restores and replays the accepted prefix inside the cache. At most one call per armed round.S.