DPO diagnostics (no grad): loss, accuracy (mean of adv>0), reward margin (mean of β·(logrC-logrR)) over the batch. Computes its own references.
DPO diagnostics (no grad): loss, accuracy (mean of adv>0), reward margin (mean of β·(logrC-logrR)) over the batch. Computes its own references.