A new theorem shows how a learner can pick up the rules of a language from correct examples alone, by generating its own near-miss contrasts. Dr Rakesh Sengupta has authored a paper titled A Reverse-Gold Theorem: Positive-Data Identification by Substrate-Bounded Contrastive Learners which is in pre-print and under review at the Journal of Machine Learning Research (JMLR).
Children learn their language mostly from hearing correct sentences, and are rarely told what is wrong. A well-known 1967 result by E. Mark Gold showed that, mathematically, learning from correct examples alone is impossible for rich classes of languages. This paper shows a way around that limit. The learner creates its own “near-miss” examples by making small structured changes to what it hears, and learns to prefer the real sentence over its altered versions. Under conditions the paper makes precise, it can then learn to tell well-formed sentences from ill-formed ones, from a predictable amount of data. The same idea, called contrastive learning, is widely used to train modern AI systems, so the result also helps explain when such training can and cannot work.
