B=1 masked CE that applies the LM head only at the supervised (response)
span. The supervised input positions are the contiguous range
[promptLen-1, len-1) (each predicts the next, response, token). The LM head
is position-independent and prompt positions get zero loss weight, so this
is numerically identical to maskedCe for B=1 — but it never materializes
logits at prompt positions. Gradients to prompt-position LoRA params still
flow: the response hidden states depend on the prompt through causal
attention, which is fully captured upstream in h.
B=1 masked CE that applies the LM head only at the supervised (response) span. The supervised input positions are the contiguous range [promptLen-1, len-1) (each predicts the next, response, token). The LM head is position-independent and prompt positions get zero loss weight, so this is numerically identical to maskedCe for B=1 — but it never materializes logits at prompt positions. Gradients to prompt-position LoRA params still flow: the response hidden states depend on the prompt through causal attention, which is fully captured upstream in
h.