B=1 response-only length-normalized log-prob → [1]. The supervised input
positions are exactly { t in [0, T) : mask[t+1] == 1 } (the same set the
full-logits seqLogp sums), which for prompt+response data is the contiguous
span [startT, startT+M); the LM head runs only on that slice. Grads to
prompt-position LoRA still flow — the response hidden states depend on the
prompt through causal attention, captured upstream in h.
B=1 response-only length-normalized log-prob → [1]. The supervised input positions are exactly
{ t in [0, T) : mask[t+1] == 1 }(the same set the full-logits seqLogp sums), which for prompt+response data is the contiguous span[startT, startT+M); the LM head runs only on that slice. Grads to prompt-position LoRA still flow — the response hidden states depend on the prompt through causal attention, captured upstream inh.