Stops gradient flow through a (identity forward; ∂=0). DPO reference logprobs are stop_gradient'd before the policy forward.
a
Stops gradient flow through
a(identity forward; ∂=0). DPO reference logprobs are stop_gradient'd before the policy forward.