One request's exact-token speculative acceptance walk. The backend owns logits and sampling; this state owns only the accepted prefix and stop facts. A group advances each live walk independently, regardless of batch size.
Readonly
Supply one target sample after its sampler has updated request history and grammar. Once done, this request needs no further target samples.
One request's exact-token speculative acceptance walk. The backend owns logits and sampling; this state owns only the accepted prefix and stop facts. A group advances each live walk independently, regardless of batch size.