ReadonlyappendKnown tokens: the committed read under knownSpans, the window read otherwise.
ReadonlycacheOne cache per layer, the full-attention caches sharing one mask memo:
the cache the composition's KV scheme names for each attention layer
(composedFullAttentionCache). Their TurboQuant read is dense
(denseTurboQuantKernels): the read decodes keys and values, then runs the
fused SDPA.
ReadonlyconfigReadonlydecodeOne new token per sequence: ids [sequences, 1], so rows are sequences.
ReadonlyembedUnscaled token embeddings, for media prompts that splice embeddings.
ReadonlyembeddingPrepared embeddings without explicit positions, indexed by total rows.
ReadonlyfaReadonlyfinalReadonlyknownMain's qualified committed spans, bound once when the generic graph loads.
ReadonlylayersReadonlylmnull when tied: the output head reuses embed_tokens (embed.asLinear).
ReadonlylogitsThe same output head at every declared numeric row.
ReadonlyloraReadonlymediaReadonlymediaReadonlymediaReadonlymixedEach group's attention reads its own caches and phase. Declared packed operations share the nonverify MLP rows; verification keeps its own geometry. Prompt groups above four rows evaluate each layer and recurrent state.
ReadonlymixedMixed groups that retain their own projection geometry. The scheduler routes each group independently when a verify group is present.
ReadonlymultiReadonlymultiReadonlyprefixBase path for LoRA target keys (weights carry the language_model prefix).
ReadonlypromptOne whole chunk of a prompt; rows are sequences times the chunk size, and each layer's output and recurrent state is evaluated before the next. The GPU is this graph's one unit.
ReadonlypromptOptional chunk bindings when a file distinguishes chunks from prompt tails.
ReadonlysequenceFull-sequence scoring operations, indexed by total token rows.
ReadonlytiedReadonlyverifyA speculative window: each sequence's last accepted token and its drafted
tokens, right-padded by the caller to the round's depth, ids
[sequences, depth + 1], so rows are sequences times (depth + 1). The graph
appends all depth + 1 positions; dropping the rejected suffix is the
caller's speculative transaction, begun before this call.
ReadonlyweightsPrepared image and video embeddings with request-owned mRoPE positions, layer taps for drafts. Its few full-attention layers materialize their scores at long context, so prefill chunks shrink to bound that workspace.
media embeddings/embeddings+positions: bind one request's prepared
embeddings into an input whose prefill is indivisible.
Releases the mRoPE frequencies a vision forward built; the weights are its owner's.
Ports over this graph's live caches that draft sources may consume.
ProtectedforwardThe decoder over h0 (consumed), every block reading its cache for
phase, on the fixed layer view the calling operation names.
bounded evaluates each layer's output and
recurrent state before the next. capture receives each layer's residual
output after the layer runs; the caller hands it the final hidden.
media: the lazy encoders this graph loads from the checkpoint at modelDir.
The graph's operations and construction facts. Pooled embedding support is declared by its operation alone. Other operations accompany the promises in
graphCapabilities; composition checks their pairing when it binds (declaredGraph).