Optionalprojections: QwenAttentionProjections<L>ReadonlyheadReadonlykReadonlykReadonlynReadonlynReadonlyoReadonlyqReadonlyqReadonlyreadsReadonlyropeReadonlyropeReadonlyscaleReadonlyvx [B, L, hidden] (borrowed) to an owned [B, L, hidden], appending to
cache and attending through the read phase names. Media forwards
supply their own positions in mrope.
Protectedforward
Full (softmax) attention with output gate + q/k norm + partial RoPE.
loadLinearchooses the projection type (the MTP companion loads dense or quantized heads);readsis how the block reads its cache (the cache's own read unless a graph injects its kernels). The graph itself is the same.