A novel large language model named Talkie has been trained exclusively on English language sources published before 1931, including etiquette manuals, encyclopedias, and poetry. This unique approach creates a 'vintage LLM' that researchers hope will serve as a test bed for understanding how data diversity shapes AI behavior.
By limiting the training corpus to a century-old dataset, scientists aim to study what these models can and cannot predict or create, offering insights into the role of historical context in machine learning. The initiative provides a fresh perspective on AI development, highlighting how the choice of training data influences model capabilities.
The project underscores the importance of data curation in AI and may help researchers identify limitations and biases inherent in modern LLMs trained on contemporary internet text.