The MLP with mlx_lm activations.py's compiled swiglu, at every sequence length: separate kernels differ numerically on MLX 0.32.2 at larger prefill shapes.
Readonly
MLP: returns mlp(x); the caller adds the residual when given.
Optional
The MLP with mlx_lm activations.py's compiled swiglu, at every sequence length: separate kernels differ numerically on MLX 0.32.2 at larger prefill shapes.