A residual stream in another basis than the trained model's: the TurboQuant
R1 fold (packages/quantize/src/rotate.ts) stores h·R1, R1 = diag(s)·M, with s
the seed's lane-0 sign vector and M MLX's hadamard_transform/√n, and moves
the final-norm gain into the head.
A residual stream in another basis than the trained model's: the TurboQuant R1 fold (packages/quantize/src/rotate.ts) stores h·R1, R1 = diag(s)·M, with s the seed's lane-0 sign vector and M MLX's hadamard_transform/√n, and moves the final-norm gain into the head.