Introduction
mlx-bun runs local models on Apple Silicon through MLX. It includes a browser chat app, an OpenAI/Anthropic-compatible HTTP server, and a TypeScript library for embedding inference in Bun applications. The signed executable includes its runtime; inference does not require Python.
The server uses continuous batching for eligible requests. Model execution, sampling, scheduling, and cache storage have separate responsibilities, including when only one request is active. See server configuration for supported combinations.
Choose your starting point
Section titled “Choose your starting point”- Use local chat: follow installation and the quickstart.
- Build an application: read the library API or HTTP API.
- Evaluate the engine: inspect the correctness contract and measured comparisons.
Supported environment
Section titled “Supported environment”The native engine requires an Apple Silicon Mac running macOS 14 or later. Source and npm usage require Bun; the standalone bundle includes it. Use the model roster for validated architectures and artifact formats. Compatibility depends on the exact model and configuration.