Function bindGraphCaches

bindGraphCaches<
    Graph extends
        { cacheLayers: readonly MlxCacheLayer[]; cachePaging?: boolean },
>(
    graph: Graph,
    composition: {
        kv: KvStorageScheme;
        pagedKv: { blockSize: number; direct?: boolean } | null;
    },
): Graph & { stateCodecs?: CacheCodecProvider; makeCache(): Cache[] }
Bind request-state construction at load; the graph only declares layout.
Type Parameters
- Graph extends { cacheLayers: readonly MlxCacheLayer[]; cachePaging?: boolean }
Parameters
- graph: Graph
- composition: { kv: KvStorageScheme; pagedKv: { blockSize: number; direct?: boolean } | null }
Returns Graph & { stateCodecs?: CacheCodecProvider; makeCache(): Cache[] }
- Defined in packages/inference/src/models/cache-composition.ts:39