The windows MLX's fused SDPA hands to its unfused fallback for want of a
vector kernel: at most 8 causal queries per row (above that its full
attention kernel serves), more than 32 query vectors per KV head (the
vector kernel's limit), no more queries than keys. q [B, H, L, D], keys
[B, KVH, N, D].
The windows MLX's fused SDPA hands to its unfused fallback for want of a vector kernel: at most 8 causal queries per row (above that its full attention kernel serves), more than 32 query vectors per KV head (the vector kernel's limit), no more queries than keys. q [B, H, L, D], keys [B, KVH, N, D].