Qwen3 attention (per-head q/k RMSNorm over head_dim before full-head RoPE, GQA). qwen3 and qwen3_moe share it; they differ only in the rope_theta default used when the config omits one.
rope_theta
reads is how the block reads its cache: the cache's own read, fixed when the block is built.
reads
Readonly
x [B, L, hidden] (borrowed) to an owned [B, L, hidden], appending to cache and attending through the read phase names.
x
cache
phase
Qwen3 attention (per-head q/k RMSNorm over head_dim before full-head RoPE, GQA). qwen3 and qwen3_moe share it; they differ only in the
rope_thetadefault used when the config omits one.