Gradient clipping: scale grads so their global L2 norm ≤ this before the optimizer step (0 = off). Standard ORPO/DPO stability guard — without it a single high-norm batch makes AdamW take a huge step and the loss diverges.
LoRA-input dropout (PEFT-style), 0 = off. Training-only regularizer for small preference sets. Recompute-safe: the mask is keyed by (step, layer) so segmented/checkpointed recompute reproduces it.
LoRA+: LR multiplier for the B leaves (A stays at the base LR). >1 speeds the B-driven early learning (B is zero-init). 1 = off.
Intra-layer MLP split (lever 4): with gradCheckpoint on (graphs declaring gradCheckpoint),
checkpoint the attention and MLP sub-blocks separately with the post-attn
residual as the boundary, so the backward recompute holds max(attn,MLP)+hMid
instead of the whole layer. Numerically identical to gradCheckpoint alone —
a finer memory↔compute split. No effect unless gradCheckpoint is on.
Token-chunk size for the ORPO head (0 = off). >0 computes the B=1
response-only LM head in token-chunks, each rematerialized in the backward,
bounding the dominant [M, vocab] logits term to [chunkSize, vocab].
Exact (numerically identical). Non-segmented path only (segmentSize takes
precedence). See 02d723a:docs/design/orpo-training.md → chunked fused log-prob.
Route the fused linear-CE head through the flash-CCE Metal kernel: in-kernel
quantized logits + online softmax (fwd) and dh accumulation (bwd) — neither
[M,vocab] nor a dequantized head touches HBM. Lowest memory + fastest on
large vocab (e4b 262k / CPM 130k). Implies the fused head; the Apple-CCE coeff
filter is on by default in the backward. B=1, non-segmented path.
Use the FUSED linear cross-entropy head (Liger/CCE-style): one CustomVjp with
an analytic softmax−onehot backward — no autograd through the head, no
retained [M,vocab] logits in either direction. Composes with
orpoChunkSize as the token-chunk size (defaults to 512 when unset). B=1,
non-segmented path only; graphs declaring an LM head (lmHead). Value bit-exact vs the
full-logits head, grads in the bf16 class. See
02d723a:docs/design/orpo-training.md → fused linear-CE head.
λ — weights ONLY the odds-ratio term (the SFT-NLL term stays unweighted).
Shared prompt-prefix ORPO (lever 7): one forward over [prompt; chosen; rejected]
with a block-sparse mask + block-wise RoPE, so the shared prompt is encoded ONCE
(token cost 2(P+R) → P+2R). B=1, non-segmented path; graphs declaring prefixShared.
Falls back to the two-forward orpoLoss for rows whose chosen/rejected prompts
differ. Composes with orpoFlashCe/orpoFusedCe (each branch routes through the
[M,V]-free head). See packages/training/src/prefix-shared.ts.
rsLoRA: scale the LoRA update by α/√rank instead of α, so per-layer rank scaling (by_bits/by_kl) changes capacity, not step size. Saved in the adapter config so inference applies the same per-layer scale.
Keep ALL eval-step checkpoints (full mountable adapters under
checkpoints/step-
Segmented backward: layers per segment (0 = off). Streams the SFT backward
segment-by-segment so only one segment's activations are live at a time —
beats per-layer checkpointing at long context, where naive checkpointing
holds every layer's recompute activations at once and crashes. B=1 SFT
only (the responseOnlyCe path); mutually exclusive with gradCheckpoint.
Numerically identical to off. See 02d723a:docs/design/orpo-training.md.
ORPO L_SFT scope (sft_scope). "full" (default; paper/TRL-faithful):
the chosen-NLL term is the token-mean cross-entropy over the FULL
prompt+response (only padding excluded), computed from the same chosen
forward — matching TRL's ORPOTrainer chosen_nll_loss. "response":
L_NLL = -ℓw (response-only), bit-exact to pre-sft_scope runs. The
odds-ratio ℓw/ℓr terms are response-only length-normalized means in BOTH
modes (that also matches TRL). Applies to every ORPO path (naive /
chunked / fused / flash / prefix-shared / segmented). orpo only.
Cap validation examples evaluated per eval step (0 = all). Val accuracy is correct/total over a FIXED subset — bounds the denominator AND the per-eval cost (full val over thousands of pairs is minutes each).
Warm-start: path to an existing adapter dir (with adapters.safetensors) whose LoRA weights initialize this run instead of the random/zero init. Continues training from a checkpoint's weights; the optimizer state and LR schedule restart. Rank/targets must match the checkpoint. "" = off (fresh init).
Gradient checkpointing: recompute layer activations in the backward pass to bound memory at long context (mlx-lm grad_checkpoint; default in optiq). Numerically identical to off — pure memory↔compute trade.