Class GegluMLP

Dense GeGLU feed-forward: down(geglu(gate(x), up(x))) over quantized linears, with the activation form its owner chose at construction (compiled).

Hierarchy (View Summary)

GegluMLP
- GemmaMlpWeights

Index

Constructors

Properties

compiled down gate up

Methods

Constructors

constructor

new GegluMLP(
    weights: Weights,
    config: ModelConfig,
    prefix: string,
    compiled: boolean,
): GegluMLP
Parameters
- weights: Weights
- config: ModelConfig
- prefix: string
- compiled: boolean
Returns GegluMLP
- Defined in packages/inference/src/layers/geglu.ts:68

Properties

`Readonly`compiled

compiled: boolean

`Readonly`down

down: QuantizedLinear

`Readonly`gate

gate: QuantizedLinear

`Readonly`up

up: QuantizedLinear

Methods

forward

forward(x: MlxArray): MlxArray
Parameters
- x: MlxArray
Returns MlxArray
- Defined in packages/inference/src/layers/geglu.ts:74