mlx-bun library API
    Preparing search index...

    Interface QuantizeOptions

    interface QuantizeOptions {
        bits: 4 | 8;
        calibrationMix?: string;
        candidateBits?: number[];
        dtype?: ConvertDtype;
        groupSize: 32 | 64;
        mode?: string;
        nCalibration?: number;
        probeSource?: ProbeSource;
        quantizePredicate?: (base: string, fullShape: number[]) => boolean;
        reference?: string;
        targetBpw?: number;
        weightTransform?: WeightTransform;
    }
    Index
    bits calibrationMix? candidateBits? dtype? groupSize mode? nCalibration? probeSource? quantizePredicate? reference? targetBpw? weightTransform?
    bits: 4 | 8
    calibrationMix?: string

    Calibration data mixture spec ("optiq" or a JSONL path).

    candidateBits?: number[]

    Candidate bit-widths the knapsack may choose from (e.g. [4, 8]).

    dtype?: ConvertDtype

    mlx_lm.convert --dtype: dtype of the non-quantized tensors and of the quantization scales/biases. Absent keeps bf16 scales/biases and the source dtype for everything else.

    groupSize: 32 | 64
    mode?: string

    Quantization scheme; v1 only exercises "affine".

    nCalibration?: number

    Number of calibration samples (forward passes per layer×bit probe).

    probeSource?: ProbeSource

    Dependency used to provide the loadable model for sensitivity sweeps.

    quantizePredicate?: (base: string, fullShape: number[]) => boolean

    Extra per-module gate on top of the shape-eligibility check: return false to keep a module full-precision even though its shape qualifies (e.g. the DeepSpec drafter's confidence_head — threshold-sensitive and tiny). Excluded modules are recorded as false in the quantization block (mlx's "not quantized" convention). Shape-ineligible modules never reach this.

    reference?: string

    Reference dtype for sensitivity scoring (e.g. "bf16"). Informational only — the native path dequantizes the running model's own weights as the self-contained bf16 source.

    targetBpw?: number

    Target bits-per-weight for sensitivity+knapsack mixed precision.

    weightTransform?: WeightTransform

    Optional offline basis transform applied before quantization.

    Settings
    Member Visibility

    On This Page
    Properties