Head dims MLX 0.32.2's fused multi-query SDPA kernel serves for L > 8
queries on M1–M4 GPUs (mlx/backend/metal/scaled_dot_product_attention.cpp
use_fallback: 192 and 256 always fall back off-NAX; has_fused_kernel
lists 64/72/80/96/128 for the full kernel). Every other head dim runs
mlx/fast.cpp's composed fallback, which materializes the bf16 scores
tensor [heads, L, S] plus a bool mask per full-attention layer. The
decode kernel (L ≤ 8) fuses 256, so this term is a prefill-only cost.
Head dims MLX 0.32.2's fused multi-query SDPA kernel serves for L > 8 queries on M1–M4 GPUs (mlx/backend/metal/scaled_dot_product_attention.cpp
use_fallback: 192 and 256 always fall back off-NAX;has_fused_kernellists 64/72/80/96/128 for the full kernel). Every other head dim runs mlx/fast.cpp's composed fallback, which materializes the bf16 scores tensor [heads, L, S] plus a bool mask per full-attention layer. The decode kernel (L ≤ 8) fuses 256, so this term is a prefill-only cost.