nn.gelu_approx: 0.5 * x * (1 + tanh(√(2/π) * (x + 0.044715 * x³))) — composed
exactly as mlx's python source, INCLUDING operand order: x**3 is mx.power
(NOT x·x·x — they round differently in bf16), and every scalar·array product
puts the SCALAR FIRST (0.5 * x, not x * 0.5). Multiply is commutative so
this is bit-identical, but it makes the compiled geglu graph's operand string
(Multiply(C, V)) match the oracle's fused kernel exactly. Scalars promote
weakly to x's dtype.
nn.gelu_approx: 0.5 * x * (1 + tanh(√(2/π) * (x + 0.044715 * x³))) — composed exactly as mlx's python source, INCLUDING operand order: x**3 is mx.power (NOT x·x·x — they round differently in bf16), and every scalar·array product puts the SCALAR FIRST (
0.5 * x, notx * 0.5). Multiply is commutative so this is bit-identical, but it makes the compiled geglu graph's operand string (Multiply(C, V)) match the oracle's fused kernel exactly. Scalars promote weakly to x's dtype.