*Machina Mirabilis: An Experiment in Retro-Futurism*

A recent experiment, submitted to the r/MachineLearning community, has sparked interest in the potential of Large Language Models (LLMs) to discover fundamental principles of physics. The goal of this project, dubbed "Machina Mirabilis," is to train an LLM from scratch on text data prior to 1900 and see if it can independently develop concepts of quantum mechanics and relativity.

*Background and Methodology*

The experiment utilizes a dataset of approximately 1 million books and articles published before 1900. This dataset is significantly larger than typical training datasets used for LLMs, which are often comprised of modern texts. The goal is to create a model that is free from the influence of 20th-century physics and can, therefore, develop its understanding of the world without the benefit of established knowledge.

The model, a variant of the transformer architecture, is trained using a combination of masked language modeling and next sentence prediction tasks. This training process is designed to help the model develop a deep understanding of language and its relationships.

*Results*

The results of the experiment are intriguing, if not entirely surprising. After several days of training, the model began to exhibit signs of understanding complex scientific concepts. It started to develop a vocabulary and syntax that was consistent with modern scientific writing. More impressively, the model began to discuss and apply concepts that are central to quantum mechanics and relativity, including wave-particle duality and time dilation.

*Implications and Limitations*

The success of the Machina Mirabilis experiment raises important questions about the potential of LLMs to discover fundamental principles of physics. While the model's performance is impressive, it is essential to note that the results are not necessarily a direct application of quantum mechanics and relativity. Instead, the model has likely developed a novel, albeit incomplete, understanding of these concepts.

The limitations of this experiment are clear. The model's understanding of physics is limited to the level of sophistication present in the texts it was trained on. It is unlikely that the model will develop a complete or accurate understanding of quantum mechanics and relativity, at least not without further training and fine-tuning.

*Conclusion*

The Machina Mirabilis experiment is a fascinating example of the potential and limitations of LLMs. While the results are encouraging, they also highlight the need for more research into the capabilities and limitations of these models. As researchers continue to push the boundaries of LLMs, it will be essential to consider the implications of these models on our understanding of the world and the potential for them to discover novel scientific principles.

The full experiment can be found on the r/MachineLearning community, and we look forward to seeing the continued development of this research.