Class FullAttention

Hierarchy

Attention
- FullAttention

Index

Constructors

Properties

hasKv headDim isSliding kNorm kProj nHeads nKvHeads oProj qNorm qProj reads ropeBase ropeFreqs useKEqV vNorm vProj windowSize

Methods

forward forwardKEqV forwardSeparate forwardShared rope

Constructors

constructor

new FullAttention(
    weights: Weights,
    config: ModelConfig,
    prefix: string,
): FullAttention
Parameters
- weights: Weights
- config: ModelConfig
- prefix: string
Returns FullAttention
Overrides Attention.constructor
- Defined in packages/inference/src/models/gemma4/written-blocks.ts:17

Properties

`Readonly`hasKv

hasKv: boolean

`Readonly`headDim

headDim: number

`Readonly`isSliding

isSliding: boolean

`Readonly`kNorm

kNorm: RMSNorm | null

`Readonly`kProj

kProj: QuantizedLinear | null

`Readonly`nHeads

nHeads: number

`Readonly`nKvHeads

nKvHeads: number

`Readonly`oProj

oProj: QuantizedLinear

`Readonly`qNorm

qNorm: RMSNorm

`Readonly`qProj

qProj: QuantizedLinear

`Readonly`reads

reads: Gemma4ReadStrategy = gemma4CacheRead

How a donor layer reads its cache (the cache's own read).

`Readonly`ropeBase

ropeBase: number | null

`Readonly`ropeFreqs

ropeFreqs: MlxArray | null

`Readonly`useKEqV

useKEqV: boolean

`Readonly`vNorm

vNorm: RMSNorm | null

`Readonly`vProj

vProj: QuantizedLinear | null

`Readonly`windowSize

windowSize: number

Methods

forward

forward(
    x: MlxArray,
    cache: Cache | null,
    phase: Gemma4Phase,
    _donor: DonorRead | null,
): { donor: DonorRead; out: MlxArray }
x [B, L, hidden] (borrowed) to an owned [B, L, hidden]. A donor layer appends its keys and values to cache through the read phase names and returns that read (donor, owned by the caller, who keeps it for the layers that share it); a KV-sharing layer attends donor, the read its donor layer made in this forward, and returns null.
Parameters
- x: MlxArray
- cache: Cache | null
- phase: Gemma4Phase
- _donor: DonorRead | null
Returns { donor: DonorRead; out: MlxArray }
Overrides Attention.forward
- Defined in packages/inference/src/models/gemma4/written-blocks.ts:18

forwardKEqV

forwardKEqV(
    x: MlxArray,
    cache: Cache,
    phase: Gemma4Phase,
): { donor: DonorRead; out: MlxArray }
Parameters
- x: MlxArray
- cache: Cache
- phase: Gemma4Phase
Returns { donor: DonorRead; out: MlxArray }
Inherited from Attention.forwardKEqV
- Defined in packages/inference/src/models/gemma4/blocks.ts:223

forwardSeparate

forwardSeparate(
    x: MlxArray,
    cache: Cache,
    phase: Gemma4Phase,
): { donor: DonorRead; out: MlxArray }
Parameters
- x: MlxArray
- cache: Cache
- phase: Gemma4Phase
Returns { donor: DonorRead; out: MlxArray }
Inherited from Attention.forwardSeparate
- Defined in packages/inference/src/models/gemma4/blocks.ts:212

forwardShared

forwardShared(x: MlxArray, donor: DonorRead): { donor: null; out: MlxArray }
Parameters
- x: MlxArray
- donor: DonorRead
Returns { donor: null; out: MlxArray }
Inherited from Attention.forwardShared
- Defined in packages/inference/src/models/gemma4/blocks.ts:232

rope

rope(x: MlxArray, offset: number | MlxArray): MlxArray
A number offset takes the static fast::rope; an array offset (set by compiled-decode trace adapters) takes the dynamic variant — same kernel, offset read from the array (bit-exactness asserted in 02d723a:tests/unit/compile.test.ts).
Parameters
- x: MlxArray
- offset: number | MlxArray
Returns MlxArray
Inherited from Attention.rope
- Defined in packages/inference/src/models/gemma4/blocks.ts:193