mx.clear_cache — release the allocator's cached buffers back to the
OS. mlx-lm calls this after every prefill chunk (and every 256 decode
steps); without it the first decode step after a long prefill pays a
one-shot allocator-reclaim stall (~800 ms after an 8k prefill —
measured; the root cause of the
context-scaling decode gap).
mx.clear_cache — release the allocator's cached buffers back to the OS. mlx-lm calls this after every prefill chunk (and every 256 decode steps); without it the first decode step after a long prefill pays a one-shot allocator-reclaim stall (~800 ms after an 8k prefill — measured; the root cause of the context-scaling decode gap).