mlx-bun

Local LLMs on Apple Silicon · No Python

MLX inference.
In TypeScript.

A TypeScript/Bun library for local MLX inference, with a signed executable serving OpenAI/Anthropic-compatible APIs. Bit-exact logit tests against mlx-lm for validated configurations.

curl -fsSL https://mlx-bun.dev/install.sh | sh

or brew install joshuarossi/tap/mlx-bun · bunx mlx-bun · from source →

mlx-bun — zsh

$ mlx-bun

↓ First-run starter · MiniCPM5-1B

↻ serving on http://localhost:8080

↗ opening /#/chat

› what runs on my machine here?

The model runs on your Mac’s GPU through MLX and Metal. Local inference needs no hosted-model API key or Python service.

bit-exactmlx-bun ≡ mlx-lm (bf16 KV)mlx-bun ≡ mlx-optiq (mixed-precision KV)Numerical contracts and tested scope →

Build with it

Library

Inference in your process

Load a model and generate tokens inside a Bun application through mlx-bun/engine. Use the executable as a local worker when your app runs elsewhere.

Correctness

Compare the logits

Pinned mlx-lm and mlx-optiq oracles, artifact-specific fixtures, and explicit cache and execution settings. Read the coverage and limits alongside the results.

Execution

Reuse computed work

Continuous batching, speculative methods, specialized kernels, and RAM/SSD cache reuse. Measured comparisons identify where each combination helps.

Also in the box: a browser chat app, training tools, personal memory, and experimental tools.

Inspect the evidence

The benchmark ledger records results on M1 Max and M4 Pro, including default batching, context growth, and SSD cache reuse. Each comparison identifies its model, settings, and machine conditions. Remaining regressions and unsupported combinations are recorded alongside successful results.

MLX, mlx-c, mlx-lm, and mlx-optiq provide the runtime foundations and numerical references. Read how mlx-bun uses them and the correctness contract.

Keep reading

Installation Four ways in: Homebrew, curl, bunx, or source.

Quickstart First run, the chat UI, and the OpenAI-compatible API.

The HTTP API Chat completions, tools, vision, prompt caching, LoRA.

TypeScript library Load models and generate tokens inside a Bun application.

Correctness Reference implementations, tested configurations, and limits.

How it compares Where mlx-bun sits next to mlx-lm and mlx-optiq.