mlx-bun library API
    Preparing search index...

    Interface ColdTier

    The cold (SSD) tier the RAM cache tiers over. Structural — this module never imports ssd-cache (which imports us); the server binds SsdCacheStore + the model into this shape (apps/mlx-bun/src/engine/cache-services.ts). Lookup and restore failures return null; store returns false on failure. Operations that require persistence retain their donor if storing fails.

    interface ColdTier {
        find(
            prompt: number[],
            ns: string,
        ): { handle: unknown; prefixLen: number } | null;
        findExact?(
            tokens: number[],
            ns: string,
        ): { handle: unknown; prefixLen: number } | null;
        restore(
            handle: unknown,
        ):
            | {
                attachments?: CheckpointAttachment[];
                caches: Cache[];
                retain: () => void;
                tokens: number[];
            }
            | null;
        restoreAsync?(
            handle: unknown,
        ): Promise<
            | {
                attachments?: CheckpointAttachment[];
                caches: Cache[];
                retain: () => void;
                tokens: number[];
            }
            | null,
        >;
        restoreObjectAsync?(
            handle: unknown,
        ): Promise<
            | {
                attachments?: CheckpointAttachment[];
                caches: Cache[];
                retain: () => void;
                tokens: number[];
            }
            | null,
        >;
        store(
            tokens: number[],
            caches: Cache[],
            ns: string,
            attachments?: CheckpointAttachment[],
        ): boolean | void;
    }
    Index
    find findExact? restore restoreAsync? restoreObjectAsync? store

    • Longest stored usable prefix for prompt/ns — index-only, no I/O. handle is the tier's opaque entry token, passed back to restore.

      Parameters

      • prompt: number[]
      • ns: string

      Returns { handle: unknown; prefixLen: number } | null

    • Parameters

      • tokens: number[]
      • ns: string

      Returns { handle: unknown; prefixLen: number } | null

    • Restore GPU-visible state. The default SSD backend copies into owned arrays. An optional backing release runs after consumers dispose their arrays. Null on any failure.

      Parameters

      • handle: unknown

      Returns
          | {
              attachments?: CheckpointAttachment[];
              caches: Cache[];
              retain: () => void;
              tokens: number[];
          }
          | null

    • Parameters

      • handle: unknown

      Returns Promise<
          | {
              attachments?: CheckpointAttachment[];
              caches: Cache[];
              retain: () => void;
              tokens: number[];
          }
          | null,
      >

    • Tensor objects have no target layers; restore with their own schema.

      Parameters

      • handle: unknown

      Returns Promise<
          | {
              attachments?: CheckpointAttachment[];
              caches: Cache[];
              retain: () => void;
              tokens: number[];
          }
          | null,
      >

    Settings
    Member Visibility

    On This Page
    Methods