Rows in the layout, or null before the first append.
ReadonlydsaReadonlykvReadonlymaxReadonlyroleReadonlyropeExact logical bytes of valid f32 compressed state.
Append and return caller-owned chronological views.
The latent attention read (LatentAttentionCache.appendLatent): append,
then caller-owned chronological views of all stored state. The cache
retains its own full state.
Physical cache storage consumed by one additional token for one row. Recurrent caches return 0 because their state does not grow with the sequence. Optional for stateless/training adapters that are never admitted or persisted.
Copy one logical row into an independently-owned serial cache.
Caller-owned chronological views of all valid compressed state.
Caller-owned view of one row with its synthetic left padding removed.
Keep selected rows and normalize removable common left padding.
Can trim(n) drop the last n tokens? (Ring caches lose trimability
once wrapped.)
Merge serial or already-batched rows into this empty cache.
Stable storage identity for compatibility guards and persistence. REQUIRED: every capability predicate (isPlainKvCache, isRotating*, packages/inference/src/state/persistence.ts codecs) is a string compare on this value, so a missing override used to fail OPEN into wrong routing (PRs #42/#43: BatchedRotatingCache shipped without one and batched joins dropped running rows). Wrappers report their own kind plus the inner kind.
Speculative verify-round support for NON-trimmable recurrent caches
(SSMCache — gated-DeltaNet conv + recurrent state). A cache exposing
these is spec-eligible without isTrimmable(): the serve loop arms a
round before the verify forward (the layer snapshots pre-round state —
free, MLX arrays are immutable — and records its position-local kernel
inputs), then resolves it: commit on full accept, or rollback(keep)
which restores the snapshot and bit-exactly REPLAYS the first keep
window tokens through the recurrence (identical arithmetic prefix ⇒
identical state). Trimmable caches leave these unset and keep trim().
Keep each accepted prefix in compressed form. Right alignment lets the next shared append reuse the existing MLA/DSA attention geometry.
Live cache arrays to materialize at a prefill boundary.
The DSA index keys stored so far (LatentAttentionCache.storedIndexKeys).
Drop the last n tokens (future writes overwrite them). bypass (default
false → unchanged behavior) skips the isTrimmable guard for the
speculative-decode rollback case (trim the last n ≤ γ tips right after a
1 concat write); rotating caches then physically shrink the buffer tail.
Staticprojected
Capability contract for cache layouts that own their dynamic-row batching. The scheduler uses this for every storage family (plain and quantized KV, sliding windows, recurrent state, and layouts such as GLM's compressed MLA/DSA state) instead of teaching the scheduler their tensor layout.
mergeRowswrites into an empty cache and does not consume inputs: its sources are a running layout or one serial cache, then one serial cache per joining row. Extracted rows are independent owned serial caches.