Class DSAIndexCache

Optional DSA index-key state owned by one MLACache.

Index

Constructors

Properties

batchSize data headDim offset

Accessors

Methods

append dispose fetch restoreState state trim validateAppend

Constructors

constructor

new DSAIndexCache(geometry: DSAIndexGeometry): DSAIndexCache
Parameters
- geometry: DSAIndexGeometry
Returns DSAIndexCache
- Defined in packages/inference/src/state/glm52-cache.ts:112

Properties

batchSize

batchSize: number | null = null

data

data: MlxArray | null = null

`Readonly`headDim

headDim: number

offset

offset: number = 0

Accessors

byteLength

get byteLength(): number
Returns number
- Defined in packages/inference/src/state/glm52-cache.ts:145

Methods

append

append(value: MlxArray, batch: number, tokens: number): void
Parameters
- value: MlxArray
- batch: number
- tokens: number
Returns void
- Defined in packages/inference/src/state/glm52-cache.ts:127

dispose

dispose(): void
Returns void
- Defined in packages/inference/src/state/glm52-cache.ts:186

fetch

fetch(): MlxArray
Returns MlxArray
- Defined in packages/inference/src/state/glm52-cache.ts:136

restoreState

restoreState(data: MlxArray, offset: number): void
Adopt persisted compressed index-key state after validating it.
Parameters
- data: MlxArray
- offset: number
Returns void
- Defined in packages/inference/src/state/glm52-cache.ts:158

state

state(): MlxArray[]
Returns MlxArray[]
- Defined in packages/inference/src/state/glm52-cache.ts:153

trim

trim(n: number): void
Parameters
- n: number
Returns void
- Defined in packages/inference/src/state/glm52-cache.ts:171

validateAppend

validateAppend(value: MlxArray, batch: number, tokens: number): void
Parameters
- value: MlxArray
- batch: number
- tokens: number
Returns void
- Defined in packages/inference/src/state/glm52-cache.ts:117