Research paper
Forward-Only Fine-Tuning Induces a Reversal Asymmetry in GPT-2
Behavioural and mechanistic evidence across three model scales
GPT-2 Small, Medium and Large fine-tuned on forward-only questions answer every forward question but far fewer reverse ones, even though the fact is in the prompt. Every wrong reverse answer is empty or the forward answer: a learned directional rule, not a missing association.