Class GemmaMoeLayerWeights

Norms and scale owned by one written layer; model features are supplied as concrete attention and feed-forward operations by the file.

Hierarchy (View Summary)

GemmaLayerWeights
- GemmaMoeLayerWeights

Index

Constructors

Properties

attn experts inputNorm layerScalar layerType mlp postAttnNorm postFfNorm postFfNorm1 postFfNorm2 preFfNorm preFfNorm2 prefix router

Methods

forwardAttn scaled

Constructors

constructor

new GemmaMoeLayerWeights(
    weights: Weights,
    config: ModelConfig,
    prefix: string,
    attn: Attention,
): GemmaMoeLayerWeights
Parameters
- weights: Weights
- config: ModelConfig
- prefix: string
- attn: Attention
Returns GemmaMoeLayerWeights
Overrides GemmaLayerWeights.constructor
- Defined in packages/inference/src/models/gemma4/written-blocks.ts:99

Properties

`Readonly`attn

attn: Attention

`Readonly`experts

experts: SwitchGLU

`Readonly`inputNorm

inputNorm: RMSNorm

`Readonly`layerScalar

layerScalar: MlxArray

`Readonly`layerType

layerType: string

`Readonly`mlp

mlp: GemmaMlpWeights

`Readonly`postAttnNorm

postAttnNorm: RMSNorm

`Readonly`postFfNorm

postFfNorm: RMSNorm

`Readonly`postFfNorm1

postFfNorm1: RMSNorm

`Readonly`postFfNorm2

postFfNorm2: RMSNorm

`Readonly`preFfNorm

preFfNorm: RMSNorm

`Readonly`preFfNorm2

preFfNorm2: RMSNorm

`Readonly`prefix

prefix: string

`Readonly`router

router: Router

Methods

forwardAttn

forwardAttn(
    x: MlxArray,
    cache: Cache | null,
    phase: Gemma4Phase,
    donor: DonorRead | null,
): { donor: DonorRead | null; h: MlxArray }
Parameters
- x: MlxArray
- cache: Cache | null
- phase: Gemma4Phase
- donor: DonorRead | null
Returns { donor: DonorRead | null; h: MlxArray }
Inherited from GemmaLayerWeights.forwardAttn
- Defined in packages/inference/src/models/gemma4/written-blocks.ts:56

scaled

scaled(x: MlxArray): MlxArray
Parameters
- x: MlxArray
Returns MlxArray
Inherited from GemmaLayerWeights.scaled
- Defined in packages/inference/src/models/gemma4/written-blocks.ts:67