Click card for metadata
Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA
Language models are increasingly taught from synthetic question answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the resulting pairs are used to fine tune, distill, or compress knowledge into another model. We show that this generation step is not neutral preprocessing. It is an implicit policy that both selects which evidence becomes training signal and d...