Class MtpModule

The one MTP decoder block: fc-merge → attention → swiglu MLP → norm.

Implements

RecurrentMtpModule

Index

Constructors

Properties

attn decode fc finalNorm inputNorm mlpDown mlpGate mlpUp postAttnNorm preFcNormEmbedding preFcNormHidden window

Constructors

constructor

new MtpModule(
    weights: Weights,
    config: ModelConfig,
    resources: DisposableStack,
): MtpModule
Parameters
- weights: Weights
- config: ModelConfig
- resources: DisposableStack
Returns MtpModule
- Defined in packages/inference/src/models/qwen/mtp.ts:44

Properties

`Readonly`attn

attn: Qwen3Attention<MtpLinear>

`Readonly`decode

decode: MlxRowTable<RecurrentMtpForward> = ...

One position per row; the module has one draft kernel set, at every width.

`Readonly`fc

fc: MtpLinear

`Readonly`finalNorm

finalNorm: RMSNorm

`Readonly`inputNorm

inputNorm: RMSNorm

`Readonly`mlpDown

mlpDown: MtpLinear

`Readonly`mlpGate

mlpGate: MtpLinear

`Readonly`mlpUp

mlpUp: MtpLinear

`Readonly`postAttnNorm

postAttnNorm: RMSNorm

`Readonly`preFcNormEmbedding

preFcNormEmbedding: RMSNorm

`Readonly`preFcNormHidden

preFcNormHidden: RMSNorm

`Readonly`window

window: RecurrentMtpForward = ...

Several positions per row.