silu(gate(x))·up(x) for 1..8 activation rows. Gate and up share their axis-1 geometry (rows a multiple of 8, columns of 256); their bit widths may differ. Borrows x and the weights; returns owned lazy output with x's leading dimensions.
silu(gate(x))·up(x) for 1..8 activation rows. Gate and up share their axis-1 geometry (rows a multiple of 8, columns of 256); their bit widths may differ. Borrows x and the weights; returns owned lazy output with x's leading dimensions.