Decoder state one decode owns (self-attention K/V and the cross K/V from the encoder output). The graph that made it is the only one that reads it; the transcription loop only advances, reorders and releases it.
Readonly
Tokens already in the self-attention cache.
Beam reorder: keep rows indices (take along the batch axis).
indices
Decoder state one decode owns (self-attention K/V and the cross K/V from the encoder output). The graph that made it is the only one that reads it; the transcription loop only advances, reorders and releases it.