Zero-copy in-memory clone of a cache list at its CURRENT state — new cache instances holding slice VIEWS of the live tensors (mlx arrays are immutable: continued generation replaces the source caches' internal arrays, it never mutates the shared buffers, so views are stable). Built for the prompt-boundary prompt-cache snapshot (multi-turn agent traffic: the reply's decode→encode roundtrip drift makes prompt+gen entries untrimmable at context > sliding window — a prompt-only entry is ALWAYS an exact prefix of the next turn, immune to drift). Must be called while the caches hold EXACTLY the state to snapshot (post-wrap rings cannot be rewound later). Same per-kind dispatch as snapshotCache/loadKvCache.
Tier policy: a pending write is not a lower-tier copy. This controls eviction only; persistence completion does not itself remove an entry.
ReadonlydurabilityReadonlymaxObserver after publication, independent of persistence policy.
The backend measures pressure; this store chooses which optional snapshots to release. No storage I/O runs at a reclaim boundary.
ProtectedpromoteReadonlyspillReadonlywriteDrop session affinity without deleting reusable checkpoints.
Evict entries older than idleMs. Tiered residency policy first ensures SSD coverage. No-op without a cold tier. Returns entries removed from RAM.
Read-only lookup of an immutable, cache-owned checkpoint.
Longest usable prefix length available for prompt WITHOUT taking
(the SSD tier compares this against its own best before restoring).
Start cold reads during request preparation, coalescing identical IO. Active request interest keeps a just-restored donor resident until the caller has consumed it. Cancellation releases interest through the lease.
OptionalsessionId: stringInsert (or reinsert) an entry; evicts LRU entries over the byte cap (spilling each to the cold tier first, when one is attached). If the entry itself exceeds the cap it is spilled + disposed, not stored.
Give optional retained state back to the allocator at an execution boundary. This never rejects or changes the active request.
Release LRU entries until the caller's live-memory check is satisfied. Unlike background eviction, pressure relief must finish the SSD write and release the buffers now, not enqueue clones that keep them alive. Call under the engine lock, after taking the request's reusable prefix.
Find the entry with the longest usable common prefix of prompt
across BOTH tiers. Every RAM entry lends zero-copy views and retains its
donor, including exact-boundary recurrent state and method attachments.
The caller owns the returned views —
dispose them or put() an extended entry, honoring retain either way.
Usable prefix = common prefix capped at prompt.length - 1 (at least one token must be forwarded to produce logits). Entries longer than the prefix need cache.trim(); ring caches lose trimability once wrapped — those entries only match in full.
Tier order: the cold tier wins only with a STRICTLY longer usable prefix (restore reads storage and materializes native arrays). A cold entry with an untrimmable divergent tail (ring post-wrap, SSM) is dropped and the RAM candidate (or a fresh prefill) serves. hits/misses count RAM candidacy only (cold restores are counted by the tier itself), preserving the /stats meaning.
OptionalsessionId: string
One cache with RAM and SSD storage. Inference sees the same put/take interface. Recency and capacity choose RAM residents; a pending write is never treated as a completed demotion.