Pinned post
Beyond pre-training, here's how I imagine most learning will work.
1. AI models / systems will maintain large collections of retrievable knowledge. This will include facts like "the capital of California is Sacramento" and tactics like "when playing Monopoly, buy a bunch of properties early on" or "make sure default arguments in Python are not mutable". The system will retrieve relevant facts / insights / tactics to perform tasks.
2. AI will ingest all types of data (books, codebases, news articles, streaming sensory input). This will happen at training time as well as while deployed to perform tasks.
3. As the system encounters new data, it will reason heavily to make sense of the data, especially to make sense of the data in the context of the system's existing knowledge. If the data is hard to understand, e.g., a technical research paper, this process will be very computationally intensive, and the output of the reasoning will be distilled into a new set of insights, tactics, and facts that are stored in the system's knowledge (along with references to the raw data that the insights were derived from). If the new knowledge contradicts existing retrieved facts, more reasoning will occur to attempt to resolve the contradiction, and existing facts in the knowledge collection will be updated (along with context about the reason for the update).
This has some nice properties.
- A way to remove or fix incorrect knowledge as the AI gathers new information.
- Some interpretability about what the model knows and its strategies for performing tasks. Also the ability to audit / check what the model knows.
- A way to benefit from training time reasoning as opposed to just reasoning at inference time and then discarding hard-earned insights.
- This kind of training time reasoning has the potential to be much more data efficient, but far more compute intensive.
This process of training time reasoning, knowledge storage, and retrieval could be emergent given the right architecture, but I expect it to be baked in initially.
1. AI models / systems will maintain large collections of retrievable knowledge. This will include facts like "the capital of California is Sacramento" and tactics like "when playing Monopoly, buy a bunch of properties early on" or "make sure default arguments in Python are not mutable". The system will retrieve relevant facts / insights / tactics to perform tasks.
2. AI will ingest all types of data (books, codebases, news articles, streaming sensory input). This will happen at training time as well as while deployed to perform tasks.
3. As the system encounters new data, it will reason heavily to make sense of the data, especially to make sense of the data in the context of the system's existing knowledge. If the data is hard to understand, e.g., a technical research paper, this process will be very computationally intensive, and the output of the reasoning will be distilled into a new set of insights, tactics, and facts that are stored in the system's knowledge (along with references to the raw data that the insights were derived from). If the new knowledge contradicts existing retrieved facts, more reasoning will occur to attempt to resolve the contradiction, and existing facts in the knowledge collection will be updated (along with context about the reason for the update).
This has some nice properties.
- A way to remove or fix incorrect knowledge as the AI gathers new information.
- Some interpretability about what the model knows and its strategies for performing tasks. Also the ability to audit / check what the model knows.
- A way to benefit from training time reasoning as opposed to just reasoning at inference time and then discarding hard-earned insights.
- This kind of training time reasoning has the potential to be much more data efficient, but far more compute intensive.
This process of training time reasoning, knowledge storage, and retrieval could be emergent given the right architecture, but I expect it to be baked in initially.
We're missing techniques for "training-time reasoning." Right now there's a lot of progress on inference-time reasoning, which is incredibly cool (I use o3 all the time).View quoted post on X
If I think about how I learn stuff, e.g., when reading a technical paper, it's very compute intensive. Most ("most" is actually not an exaggeration here) sentences / diagrams / equations don't make sense to me at first. I have to think hard about what the authors meant, how they derived those results from earlier parts of the paper, how to relate it to other things I know, even questioning whether the statement is correct or if the authors made a mistake.
It takes a lot of reasoning to learn. This doesn't just apply to technical concepts. If I'm reading a news article, in order to make sense of it, I have to think hard about how to interpret each piece of data. How strong is the evidence? Does it contradict other things I believe?Does their reasoning make sense?
Or learning to code. In our training sets, we have incredible open-source codebases like the Linux Kernel or Google Chrome along with their commit histories, issue descriptions, etc. There is so much depth here. If a human were to sit down for a long time and deeply study those codebases, they could learn a massive amount about software engineering. We aren't anywhere near squeezing all of the juice out of these datasets.
To replicate this, we need to put way more compute into the process of learning. Intuitively, that process doesn't feel like learning via next-token prediction or RL or generating synthetic data to train on.
