ReadonlyappendKnown tokens: this graph declares no committed spans, so they read as a window.
ReadonlycacheOne cache per layer, sharing one mask memo. Their TurboQuant read is
deferred (deferredTurboQuantKernels): attention runs on the rotated
values and the read un-rotates its output.
ReadonlyconfigReadonlydecodeOne new token per sequence: ids [sequences, 1].
ReadonlyembedUnscaled token embeddings, for media prompts that splice embeddings.
ReadonlyfinalReadonlylayersReadonlylmReadonlylogitsThe one output head at every width.
ReadonlyloraReadonlymultiReadonlymultiReadonlyprefixReadonlypromptOne whole chunk of a prompt; rows are sequences times the chunk size. The GPU is this graph's one unit.
ReadonlypromptOptional chunk bindings when a file distinguishes chunks from prompt tails.
ReadonlysequenceFull-sequence scoring operations, indexed by total token rows.
ReadonlyverifyA speculative window of depth + 1 positions per sequence. This graph
declares no hidden-layer taps, so capture is always null.
ReadonlyweightsAn ordinary autoregressive graph: every method over every cache lego.
The separate quantized head, segmented backward, prefix-shared ORPO and flash training attention.
Builds no constants and opens no weights: nothing to release.
ProtectedforwardRun decoder layers [aIdx, bIdx) on hidden h, returning the residual
stream after the last layer in the range (NO finalNorm). The input h is
NEVER disposed — the caller owns it (for segmented backward h is a leaf
whose lifetime is managed by the autograd closure); intra-range
intermediates ARE disposed. The segmented-backward and prefix-shared
training paths' building block (02d723a:docs/design/orpo-training.md
§5.3). MiniCPM5 is plain full-attention with no KV-sharing, so each layer
reads its own stateless training cache cache[i].
The scoring and training forward (MlxSequenceForward, trainForward
over the training caches): the window read at every width. The training
caches were composed with the run's training attention, so a run that
selects flash attention reads with it here.
The graph's operations and construction facts. Pooled embedding support is declared by its operation alone. Other operations accompany the promises in
graphCapabilities; composition checks their pairing when it binds (declaredGraph).