Configuration reference

Generated at build time from apps/mlx-bun/src and packages/*/src without running them; tests and scripts are not scanned. These are the current settings on main; released versions can differ.

Read: value is the raw string; flag treats 1 as on, 0 as off, and anything else as the default; number takes a finite number, else the fallback; process.env is read directly rather than through the runtime configuration. Default is the fallback at each read site, so a key read at several sites lists each; computed means the code at the source link decides.

Serve options

The serve flags and their descriptions are in the CLI reference. This table adds what the parser enforces: literal defaults, numeric ranges, and accepted values.

Flag	Default	Accepts	Source
`--thinking`		`on` \| `off` \| `1` \| `0` \| `true` \| `false`	cli/serve.ts:42
`--kv-quant` (note)	`off`	`off` \| `config` \| `4` \| `8`	cli/serve.ts:48
`--prompt-cache` (note)		number ≥ 0	cli/serve.ts:54
`--ssd-cache-max`		number ≥ 0	cli/serve.ts:67
`--ssd-demote-idle`		number ≥ 0	cli/serve.ts:67
`--generation-checkpoint`		integer ≥ 1	cli/serve.ts:67
`--temperature`		number in [0, 5]	cli/serve.ts:73
`--temp`		number in [0, 5]	cli/serve.ts:73
`--top-p`		number in [0, 1]	cli/serve.ts:74
`--top-k`		number in [0, 1000000]	cli/serve.ts:74
`--hlg-sampling`		`on` \| `off` \| `1` \| `0` \| `true` \| `false`	cli/serve.ts:75
`--hlg-width`	`4`	number in [0, 100]	cli/serve.ts:85
`--hlg-shoulder`	`4`	number in [0, 100]	cli/serve.ts:85
`--hlg-toe`	`6`	number in [0, 100]	cli/serve.ts:86
`--hlg-pivot-offset`	`6`	number in [0, 100]	cli/serve.ts:86
`--draft-kind`		`dspark` \| `deepspec` \| `assistant` \| `two-model` \| `ngram` \| `mtp`	cli/serve.ts:100
`--mtp`		`on` \| `off` \| `1` \| `0` \| `true` \| `false`	cli/serve.ts:116
`--paged-kv-block-size`		integer ≥ 1	cli/serve.ts:135
`--memory-budget`		number ≥ 0	cli/serve.ts:140
`--model-budget`		number ≥ 0	cli/serve.ts:141
`--context-length`		integer ≥ 1	cli/serve.ts:143
`--host`	`127.0.0.1`		cli/serve.ts:144
`--kv-budget`		number ≥ 0	cli/serve.ts:146
`--max-tokens`		number in [1, 10000000]	cli/serve.ts:147
`--port`	`8080`	integer in [0, 65535]	cli/serve.ts:153
`--batch`	`8`	integer ≥ 1	cli/serve.ts:155
`--decode-concurrency`	`8`	integer ≥ 1	cli/serve.ts:155

Keys set by `serve`

When serve starts a model host it writes these keys from its flags; a written value replaces the environment’s.

Key	From	Source
`MLX_BUN_FORCE_WIRE`	`--force-wire`	cli/serve-host.ts:95
`MLX_BUN_ALLOW_PRIVATE_MEDIA`	`--allow-private-media`	cli/serve-host.ts:96

App keys

MLX_BUN_* keys read under apps/mlx-bun/src and in the module and host-library packages it installs (packages/module-*, packages/app-*), including environment mirrors of flags.

Key	Read	Default	Source
`MLX_BUN_CACHE_RETENTION`	value	unset (only `cost-size` changes it)	engine/cache-services.ts:107
`MLX_BUN_EVAL_KV_QUANT`	value	unset (only `1` changes it)	cli/inference.ts:156, cli/inference.ts:158
`MLX_BUN_EVAL_THINK`	value	unset (only `1` changes it)	cli/inference.ts:173
`MLX_BUN_GRAMMAR`	value	unset (only `0` changes it)	server/request-prep.ts:136
`MLX_BUN_HOME` (note)	value	computed	app-services/src/home.ts:19, module-memory/src/paths.ts:8
`MLX_BUN_JOB_PARENT_PIPE`	flag	set at jobs/runner.ts:115	cli/job-entry.ts:41
`MLX_BUN_JOBS_DB`	value	set at jobs/runner.ts:113	cli/job-entry.ts:46
`MLX_BUN_JOBS_DIR`	value	set at jobs/runner.ts:114	cli/job-entry.ts:47
`MLX_BUN_LANE_DEBUG`	value	unset (only `1` changes it)	server/inference-request.ts:56
`MLX_BUN_MEMORY_BATCH`	value	`1`	module-memory/src/model.ts:154
`MLX_BUN_MIC_CAPTURE`	value	computed	module-transcription/src/mic-capture.ts:28
`MLX_BUN_PAGED_KV` (note)	value	unset (only `1` changes it)	cli/serve.ts:134
`MLX_BUN_PI_DEBUG`	value	computed	module-chat/src/pi-backend.ts:171, module-chat/src/pi-backend.ts:628, module-chat/src/pi-backend.ts:654, module-chat/src/pi-backend.ts:742, module-chat/src/pi-backend.ts:823
`MLX_BUN_RD_CONTEXT_LIMIT`	value	computed	cli/serve.ts:175
`MLX_BUN_SESSION_CACHE`	value	unset (only `0` changes it)	server/routes.ts:71
`MLX_BUN_SHUTDOWN_TIMEOUT_MS`	value	computed	cli/serve-options.ts:104
`MLX_BUN_SSD_LAYOUT`	value	unset (only `blocks` changes it)	engine/cache-services.ts:95
`MLX_BUN_SSD_PREFETCH`	value	unset (only `0` changes it)	engine/cache-services.ts:98
`MLX_BUN_SSD_SEGMENTED`	value	unset (only `0` changes it)	engine/cache-services.ts:96
`MLX_BUN_SSD_SPILL_QUEUE_GB`	value	computed	engine/cache-services.ts:88
`MLX_BUN_SSD_WRITEBEHIND`	value	unset (only `0` changes it)	engine/cache-services.ts:97
`MLX_BUN_WIKI`	value	computed	storage/paths.ts:59, module-memory/src/vault.ts:14

Library tuning

These keys are library tuning and diagnostics, not app options; none is promoted to a flag. Grouped by the package that reads them.

@mlx-bun/inference

Key	Read	Default	Source
`MLX_BUN_ALLOW_PRIVATE_MEDIA`	value	set by `serve` from `--allow-private-media` (cli/serve-host.ts:96)	input/media-fetch.ts:72
`MLX_BUN_ANE_DYLIB`	process.env	computed	runtime/native.ts:16
`MLX_BUN_BATCH_EXTEND`	value	unset (only `0` changes it)	state/batched-mask.ts:385, state/batched-quant.ts:333
`MLX_BUN_BATCH_NO_PIPELINE`	value	unset (only `1` changes it)	execution/batch-group.ts:173
`MLX_BUN_BATCH_STEP_TRACE`	value	unset (only `1` changes it)	execution/batch-group.ts:174
`MLX_BUN_DSPARK_MINCONF`	value	computed	generation/speculative/sources/dflash-source.ts:53
`MLX_BUN_EARLY_FIRST_TOKEN`	flag	on	execution/batch-group.ts:374, execution/batch-group.ts:558, generation/autoregressive.ts:565
`MLX_BUN_EXPERT_IO_DYLIB`	value	computed	artifacts/experts/io.ts:71
`MLX_BUN_EXPERT_IO_DYLIB`	process.env	computed	runtime/native.ts:14
`MLX_BUN_EXPERT_OFFLOAD`	value	computed	artifacts/expert-offload.ts:88
`MLX_BUN_EXPERT_TRACE`	value	computed	runtime/expert-trace.ts:65
`MLX_BUN_FILL`	value	`off`	generation/autoregressive.ts:81
`MLX_BUN_FILL`	value	computed	generation/fill/session.ts:122
`MLX_BUN_FILL_APPEND_CHUNK_SIZE`	number	`0`	generation/fill/session.ts:137
`MLX_BUN_FILL_CANDIDATES`	number	`24`	generation/fill/session.ts:148
`MLX_BUN_FILL_INDEX_MAX`	number	`131072`	generation/fill/session.ts:149
`MLX_BUN_FILL_K`	number	`8`	generation/fill/session.ts:146
`MLX_BUN_FILL_MAX_SPAN`	number	`32`	generation/fill/session.ts:131
`MLX_BUN_FILL_TRACE` (note)	flag	off	generation/fill/session.ts:157
`MLX_BUN_FILL_TRACE` (note)	value	computed	generation/fill/session.ts:168
`MLX_BUN_FORCE_WIRE`	value	set by `serve` from `--force-wire` (cli/serve-host.ts:95)	generation/autoregressive.ts:120, generation/denoising.ts:27, generation/scopes.ts:59
`MLX_BUN_FRAME_EXTRACT`	value	computed	input/vision/video-frames.ts:16
`MLX_BUN_FRAME_EXTRACT`	process.env	computed	runtime/native.ts:15
`MLX_BUN_GRAMMAR`	value	unset (only `0` changes it)	sampling/grammar.ts:629
`MLX_BUN_GRAMMAR_DEBUG`	value	unset (only `1` changes it)	execution/batch-group.ts:927
`MLX_BUN_GRAMMAR_DRAFT_TOKENS`	number	`3`	execution/gateway-binding.ts:100
`MLX_BUN_HOME` (note)	value	computed	artifacts/identity.ts:50
`MLX_BUN_MEDIA_PREFIX_CACHE`	flag	on	execution/gateway-binding.ts:158
`MLX_BUN_P2R_SYNC`	value	unset (only `1` changes it)	execution/batch-group.ts:493, execution/batch-group.ts:545, generation/autoregressive.ts:520
`MLX_BUN_P2R_TRACE`	value	unset (only `1` changes it)	runtime/trace.ts:158
`MLX_BUN_PREFILL_MEM_LOG`	value	unset (only `1` changes it)	generation/autoregressive.ts:510
`MLX_BUN_RD_PREFILL_CHUNK` (note)	value	computed	generation/bindings/prefill-policy.ts:20, models/composition.ts:21
`MLX_BUN_RD_PREFILL_CHUNK` (note)	number	`2048`	generation/bindings/prefill-policy.ts:21, models/composition.ts:22
`MLX_BUN_SAMPLER_WINDOW_FILTERS`	flag	on	sampling/step.ts:196
`MLX_BUN_SPEC_DEVICE_ROUND`	flag	on	generation/speculative/round.ts:130
`MLX_BUN_SPEC_DRAFT_FILTER`	flag	on	generation/speculative/draft-policy.ts:38
`MLX_BUN_SPEC_DRAFT_TEMPERATURE`	value	computed	generation/speculative/draft-policy.ts:35
`MLX_BUN_SPEC_DRAFT_VOCAB`	value	computed	generation/speculative/draft-vocab.ts:43
`MLX_BUN_SPEC_LAYER_PROFILE`	flag	off	generation/speculative/round.ts:155
`MLX_BUN_SPEC_OP_INVENTORY`	flag	off	generation/speculative/round.ts:134
`MLX_BUN_SPEC_PHASE_TIMING`	flag	off	generation/speculative/round.ts:18
`MLX_BUN_TOKEN_MASK`	value	unset (only `metal` changes it)	sampling/grammar.ts:227
`MLX_BUN_TURBOQUANT_FUSED_DECODE`	flag	on	state/turboquant-codec.ts:8

@mlx-bun/mlx

Key	Read	Default	Source
`MLX_BUN_LIBMLXC`	process.env	computed	native.ts:15

@mlx-bun/quantize

Key	Read	Default	Source
`MLX_BUN_CALIBRATION_JSONL`	value	computed	calibration.ts:39

@mlx-bun/training

Key	Read	Default	Source
`MLX_BUN_CCE_BWD_BLOCK_EPS`	value	`1e-5`	trainer.ts:221
`MLX_BUN_CCE_BWD_FILTER_EPS`	value	`1e-5`	trainer.ts:220
`MLX_BUN_CCE_BWD_NOSTEEL`	value	unset (only `1` changes it)	trainer.ts:216
`MLX_BUN_CCE_BWD_SCALAR`	value	unset (only `1` changes it)	trainer.ts:217
`MLX_BUN_CCE_BWD_SKIP_P2`	value	unset (only `1` changes it)	trainer.ts:219
`MLX_BUN_CCE_BWD_SKIP_P2DQ`	value	unset (only `1` changes it)	trainer.ts:218
`MLX_BUN_CCE_LANE`	value	unset (only `1` changes it)	trainer.ts:213
`MLX_BUN_CCE_NOSTEEL`	value	unset (only `1` changes it)	trainer.ts:212
`MLX_BUN_CCE_SCALAR`	value	unset (only `1` changes it)	trainer.ts:214
`MLX_BUN_CCE_SG_SKIPDQ`	value	unset (only `1` changes it)	trainer.ts:215
`MLX_BUN_FLASH_MIN_M`	value	`1024`	trainer.ts:211
`MLX_BUN_MEM_LOG`	value	unset (only `1` changes it)	trainer.ts:15
`MLX_BUN_SEG_HEAD`	value	`checkpoint`	segmented.ts:73
`MLX_BUN_SEG_HEAD_CHUNK`	value	`512`	segmented.ts:85
`MLX_BUN_SEG_MEM_LOG`	value	unset (only `1` changes it)	segmented.ts:62
`MLX_BUN_TRAIN_ATTN`	value	unset (only `flash` changes it)	trainer.ts:601

Notes

--fused-sdpa, --kv-quant: --fused-sdpa on|off picks the read kernels of affine-quantized KV when the model loads: on, the fused quantized-KV attention tiles for windows under a causal mask; off, the unfused path for every read. Without it, fused SDPA is on only when the KV choice (--kv-quant, or --l2’s model-config KV) is config.
MLX_BUN_RD_PREFILL_CHUNK: Unset, the model’s prefill policy picks the chunk: 2048 tokens, halved for linear-attention models at long context to bound the attention workspace. The 2048 shown for its number read applies only to a set, non-numeric value.
--prompt-cache: --prompt-cache is in GiB (N × 2^30 bytes), but the cap without it is 8e9 bytes (decimal 8 GB, about 7.45 GiB).
MLX_BUN_PAGED_KV, --paged-kv: --paged-kv or MLX_BUN_PAGED_KV=1 turns paging on; no flag turns an environment 1 off, and either satisfies --paged-kv-block-size.
MLX_BUN_HOME: Root of everything mlx-bun writes by default: models/, adapters/, exports/, datasets/, db/, jobs/, chats, wiki, skills, logs and saved credentials. Unset, ~/.mlx-bun. Explicit paths win; the Hugging Face cache holds only downloads.
MLX_BUN_FILL_TRACE: 1 turns fill tracing on. Any other value except empty, 0, and true is a JSONL file path and also turns it on; true turns on neither.

mlx-bun is MIT licensed and built for Apple Silicon.