Class Glm52DecoderLayer

Index

Constructors

Properties

config denseBenchmarkPrefillTokens dsa inputNorm layer mla mlp postAttentionNorm weights

Methods

forward forwardAsync

Constructors

constructor

new Glm52DecoderLayer(
    config: Glm52Config,
    weights: Glm52WeightSource,
    layer: number,
    hasDsa: boolean,
    expertBackend?: Glm52ExpertExecutionBackend | null,
    pilot?: Glm52PilotTracker | null,
    denseBenchmarkPrefillTokens?: number,
): Glm52DecoderLayer
Parameters
- config: Glm52Config
- weights: Glm52WeightSource
- layer: number
- hasDsa: boolean
- expertBackend: Glm52ExpertExecutionBackend | null = null
- pilot: Glm52PilotTracker | null = null
- denseBenchmarkPrefillTokens: number = 0
Returns Glm52DecoderLayer
- Defined in packages/inference/src/models/glm52/model.ts:493

Properties

`Readonly`config

config: Glm52Config

`Readonly`denseBenchmarkPrefillTokens

denseBenchmarkPrefillTokens: number = 0

`Readonly`dsa

dsa: Glm52DsaIndexer | null

`Readonly`inputNorm

inputNorm: MlxArray

`Readonly`layer

layer: number

`Readonly`mla

mla: Glm52Mla

`Readonly`mlp

mlp: Glm52Mlp

`Readonly`postAttentionNorm

postAttentionNorm: MlxArray

`Readonly`weights

weights: Glm52WeightSource

Methods

forward

forward(
    input: MlxArray,
    cache: LatentAttentionCache,
    dsaState: Glm52DsaSelectionState | null,
    reads: Glm52PhaseReads,
): MlxArray
Parameters
- input: MlxArray
- cache: LatentAttentionCache
- dsaState: Glm52DsaSelectionState | null
- reads: Glm52PhaseReads
Returns MlxArray
- Defined in packages/inference/src/models/glm52/model.ts:634

forwardAsync

forwardAsync(
    input: MlxArray,
    cache: LatentAttentionCache,
    dsaState: Glm52DsaSelectionState | null,
    reads: Glm52PhaseReads,
): Promise<MlxArray>
Parameters
- input: MlxArray
- cache: LatentAttentionCache
- dsaState: Glm52DsaSelectionState | null
- reads: Glm52PhaseReads
Returns Promise<MlxArray>
- Defined in packages/inference/src/models/glm52/model.ts:656