Interface MlxBatchExecutionGroupOptions

Drain signal: while true, no NEW rows are admitted (running rows finish, the lock is released) so a waiting serial-lane request can run — mlx-lm's drain_batch. Pair with kick() when it flips back to false.

The composition's cache service (state/cache-service): admission under its KV budget (a joiner that would push the admitted rows' projected KV over --kv-budget WAITS in the queue, FIFO and head-of-line, until rows finish; one that can never fit is rejected), each row's state at admission (a reused prefix, a resumed generation, or fresh state), and the state rows offer back when they finish. Runs under the gateway mutex domain. Default: a service over the graph's composition that reuses and keeps nothing and admits without a budget.

Max rows in the running batch (mlx-lm --decode-concurrency).

Scheduling work budget for extending an existing preparation cohort.

Joiner prefill chunk length (default 2048, the serial loop's constant).