Optionalprojections: QwenDeltaProjections<L>ReadonlyaReadonlyconvReadonlyconvReadonlydtReadonlyepsReadonlyheadReadonlyheadReadonlyinReadonlyinReadonlyinReadonlyinReadonlykeyReadonlynormReadonlynumReadonlynumReadonlyoutPer-model seam; null retains the oracle graph (weightless rms_norm, then a
scalar multiply). When set, the scale rides in as the norm's weight: MLX's
kernel writes w * T(x * inv), the same bf16 product, in one kernel. bf16 only.
ReadonlyrecurrenceWhat this block lends its recurrent cache on every call: its recurrence weights and the glue between the convolution and the recurrence.
Readonlyvaluex [B, S, hidden] (borrowed) to an owned [B, S, hidden], reading cache
for phase.
ProtectedheadsThe glue between the convolution and the recurrence: silu, the split into
q, k and v heads, and the q/k norms and scale (inv_scale = head_k_dim ** -0.5;
q *= inv_scale², k *= inv_scale), folded into the norm weight when
qkScale is set. Borrows convolved; the cache owns the heads.
Protectedrms
Gated-DeltaNet linear-attention block (mlx-lm GatedDeltaNet): projections, then the recurrence its cache runs (
GatedDeltaCache), then the gated norm with z and the output projection.loadLinearchooses the projection layer (default QuantizedLinear); the graph is the same.