Interface ComposedStorage

The storage a composed cache holds and converts (the bf16-first KV legos a graph's makeCache builds for its composition's KV scheme). The row layouts, clones and saved files take the storage, never the lego; whoever hands storage back to serial use composes it with over.

interface ComposedStorage {
    over(storage: Cache): Cache;
    release(): Cache;
    storage(): Cache;
}

Index

Methods

over release storage

Methods

over

over(storage: Cache): Cache
This lego's composition over storage, which the result owns: storage that already holds the conversion start converts before this returns, and quantized storage reads with this lego's kernels.
Parameters
- storage: Cache
Returns Cache
- Defined in packages/inference/src/contracts/mlx/cache.ts:313

release

release(): Cache
Hands the storage held now to the caller and leaves this lego empty.

Returns Cache
- Defined in packages/inference/src/contracts/mlx/cache.ts:315

storage

storage(): Cache
The storage held now, borrowed: bf16 before the transition, quantized or encoded after.

Returns Cache
- Defined in packages/inference/src/contracts/mlx/cache.ts:309

Interface ComposedStorage

Index

Methods

Methods

over

Parameters

Returns Cache

release

Returns Cache

storage

Returns Cache

Settings

On This Page