Combine the prompt-span mean logp (promptMean, over nPrompt positions) and
the response mean logp (lw, over nResp positions) into the FULL-scope chosen
NLL: -(nPrompt·promptMean + nResp·lw) / (nPrompt + nResp) — the token-mean
CE over all prompt+response supervised positions (nn.CrossEntropyLoss
reduction="mean" over the label-shifted non-pad positions). Graph-preserving
(pure ops on the two means), so the head gradient flows to BOTH spans. Does
NOT dispose promptMean/lw (caller owns); returns [1] (caller owns).
Combine the prompt-span mean logp (
promptMean, over nPrompt positions) and the response mean logp (lw, over nResp positions) into the FULL-scope chosen NLL:-(nPrompt·promptMean + nResp·lw) / (nPrompt + nResp)— the token-mean CE over all prompt+response supervised positions (nn.CrossEntropyLoss reduction="mean" over the label-shifted non-pad positions). Graph-preserving (pure ops on the two means), so the head gradient flows to BOTH spans. Does NOT dispose promptMean/lw (caller owns); returns [1] (caller owns).