Class SsdCacheStore

constructor

• new SsdCacheStore(opts: SsdStoreOptions): SsdCacheStore

  Parameters

  • opts: SsdStoreOptions

  Returns SsdCacheStore

stats

entries

• get entries(): number

  Returns number

longestDurablePrefixTokens

• get longestDurablePrefixTokens(): number

  Longest token prefix whose atomic cache file is present in the index.

  Returns number

maxBytes

• get maxBytes(): number

  Returns number

totalBytes

• get totalBytes(): number

  Returns number

evictToCap

• evictToCap(): void

  Enforce the byte cap: unlink oldest-mtime entries until under it.

  Returns void

find

• find(
      prompt: number[],
      ns?: string,
  ): { entry: SsdIndexEntry; prefixLen: number } | null

  Best USABLE stored entry for prompt in ns: the longest common prefix (capped at prompt.length−1 — at least one token must forward). An entry LONGER than the matched prefix is returned only when it can actually seed the prompt: the divergent tail must be trimmable after restore. Untrimmable entries (ring post-wrap, SSM) that would need a trim are SKIPPED here — before this gate, the big [prompt+gen] file always outranked the usable boundary snapshot, got restored, and was thrown away by take()'s backstop (the 2026-07-06 restart-0 + wasted 84 s restore-then-re-prefill path). Pure index lookup; no I/O.

  Parameters

  • prompt: number[]
  • ns: string = ""

  Returns { entry: SsdIndexEntry; prefixLen: number } | null

findExact

• findExact(
      tokens: number[],
      ns?: string,
  ): { entry: SsdIndexEntry; prefixLen: number } | null

  Exact checkpoint selection for a known session, independent of residency.

  Parameters

  • tokens: number[]
  • ns: string = ""

  Returns { entry: SsdIndexEntry; prefixLen: number } | null

findGenerationCheckpoint

• findGenerationCheckpoint(
      prompt: number[],
      key: string,
      cacheNs?: string,
  ): SsdIndexEntry | null

  Longest durable in-flight generation for an identical request. The checkpoint namespace is separate from the ordinary prompt-cache tier, so incomplete assistant output can never seed an unrelated prefill.

  Parameters

  • prompt: number[]
  • key: string
  • cacheNs: string = ""

  Returns SsdIndexEntry | null

hasDurablePrefix

• hasDurablePrefix(tokens: number[], ns?: string): boolean

  True when the SSD index can seed every token in this prefix. A longer trimmable descendant is equivalent to an exact boundary snapshot.

  Parameters

  • tokens: number[]
  • ns: string = ""

  Returns boolean

remove

• remove(path: string): void

  Parameters

  • path: string

  Returns void

removeGenerationCheckpoints

• removeGenerationCheckpoints(key: string): void

  Parameters

  • key: string

  Returns void

restore

• restore(
      entry: SsdIndexEntry,
      model: { makeCache(): Cache[] },
  ): LoadedKvCache | null

  Restore an entry via packages/inference/src/state/persistence.ts's STREAMED COPY (bounded host transient — live entry + one tensor; no mapping survives the call, the caches own their bytes). Bumps LRU mtime. On ANY failure the file is dropped and null returned.

  Parameters

  • entry: SsdIndexEntry
  • model: { makeCache(): Cache[] }

  Returns LoadedKvCache | null

restoreAsync

• restoreAsync(
      entry: SsdIndexEntry,
      model: { makeCache(): Cache[] },
  ): Promise<LoadedKvCache | null>

  Read on the dedicated CPU queue, then let the model codec adopt state.

  Parameters

  • entry: SsdIndexEntry
  • model: { makeCache(): Cache[] }

  Returns Promise<LoadedKvCache | null>

scan

• scan(): number

  Startup recovery: rebuild the in-memory index from disk. Corrupt headers are unlinked; .tmp orphans reaped; a file whose header names another model/tokenizer/fingerprint is left in place and not indexed. Only OUR fingerprint dir is touched. Returns entries indexed.

  Returns number

store

• store(
      tokens: number[],
      caches: Cache[],
      ns?: string,
      attachments?: CheckpointAttachment[],
  ): boolean

  Persist an entry. Synchronous (the tier calls it on the idle serial lane between requests); atomic via packages/inference/src/state/persistence.ts's tmp+fsync+rename. An entry bigger than the cap is refused; disk/write failure is a warn-once soft-fail — serving never stalls on the cold tier. Existing entries in the same ns whose tokens are a PREFIX of the new entry are superseded (the agent-conversation pattern: one growing entry, not N generations of it). Returns true if stored.

  Parameters

  • tokens: number[]
  • caches: Cache[]
  • ns: string = ""
  • Optionalattachments: CheckpointAttachment[]

  Returns boolean

storeAsync

• storeAsync(
      tokens: number[],
      caches: Cache[],
      ns?: string,
      runStep?: <T>(step: () => T) => Promise<T>,
      attachments?: CheckpointAttachment[],
  ): Promise<boolean>

  Non-blocking store (the write-behind persistence path): same file, index, and supersede semantics as store(), but the tensor flush yields the event loop between tensors so serving interleaves. Caller passes zero-copy CLONES it owns (a consistent snapshot no matter what the live entry does meanwhile) and disposes them after. runStep optionally wraps owner-thread snapshot preparation. Hashing, packing and disk writes execute on the CPU worker.

  Parameters

  • tokens: number[]
  • caches: Cache[]
  • ns: string = ""
  • OptionalrunStep: <T>(step: () => T) => Promise<T>
  • Optionalattachments: CheckpointAttachment[]

  Returns Promise<boolean>

storeGenerationCheckpoint

• storeGenerationCheckpoint(
      tokens: number[],
      caches: Cache[],
      checkpoint: {
          cacheNs: string;
          generatedTokens: number;
          key: string;
          originalPromptTokens: number;
          pendingToken: number;
          seed: number;
          seedWasExplicit: boolean;
      },
      runStep?: <T>(step: () => T) => Promise<T>,
  ): Promise<boolean>

  Persist one in-flight generation atomically. A completed newer checkpoint supersedes older checkpoints for the same request only after its rename succeeds, so a crash during a write leaves the prior point.

  Parameters

  • tokens: number[]
  • caches: Cache[]
  • checkpoint: {
        cacheNs: string;
        generatedTokens: number;
        key: string;
        originalPromptTokens: number;
        pendingToken: number;
        seed: number;
        seedWasExplicit: boolean;
    }

    • cacheNs: string

    • generatedTokens: number

    • key: string

    • originalPromptTokens: number

    • pendingToken: number

    • seed: number

      Actual sampler seed, including a server-generated default. Required to continue the random stream exactly after a restart.

    • seedWasExplicit: boolean

      True when seed was part of the client request rather than generated by the server.

  • OptionalrunStep: <T>(step: () => T) => Promise<T>

  Returns Promise<boolean>

tokenPrefixes

• tokenPrefixes(): Iterable<readonly number[]>

  Header-only provenance for the serving tokenizer; no tensor restore.

  Returns Iterable<readonly number[]>