Pinned post
Latest posts 5
@fchollet made a point today that deserves better than the usual neural vs symbolic debate.
Two movements at once, he says. Wherever data gets dense enough, logic migrates into the network: scaffolding keeps getting baked into the models themselves. Meanwhile the strongest systems grow in the other direction, wrapping the network in memory, tools, verifiers, agent harnesses.
He calls it neurosymbolic. François Roddier, who spent his later life on the thermodynamics of evolution, would have called it an oscillation. Complex adaptive systems never converge on an optimal architecture. Stable environments reward integration and efficiency, the K-strategists. Changing environments reward small, plastic, adaptable forms, the r-strategists. Neither wins. The alternation is the mechanism.
A trained network is a K-machine: anything frequent and stable enough gets compressed into weights. AlphaZero is the cleanest case, tree search progressively distilled into the policy network. But weights are a terrible place for this morning's API state or a procedure invented mid-task. Hence the r-machinery: tools, search, sub-agents, external state. Ugly, but you can change it without retraining a trillion parameters.
So the boundary moves in a cycle. Solve a new problem outside the model. Let traces accumulate. Compress them in. The stronger model makes harder problems tractable, which demand new scaffolding one level up. Externalize, select, distill, internalize, repeat.
That is why the model can eat the harness while the harness keeps gaining importance: two moments of the same cycle. Pure Red Queen dynamics, the better the system exploits its environment, the faster it changes the environment it must adapt to.
I don't expect the architecture to converge. The oscillation is the architecture.
Two movements at once, he says. Wherever data gets dense enough, logic migrates into the network: scaffolding keeps getting baked into the models themselves. Meanwhile the strongest systems grow in the other direction, wrapping the network in memory, tools, verifiers, agent harnesses.
He calls it neurosymbolic. François Roddier, who spent his later life on the thermodynamics of evolution, would have called it an oscillation. Complex adaptive systems never converge on an optimal architecture. Stable environments reward integration and efficiency, the K-strategists. Changing environments reward small, plastic, adaptable forms, the r-strategists. Neither wins. The alternation is the mechanism.
A trained network is a K-machine: anything frequent and stable enough gets compressed into weights. AlphaZero is the cleanest case, tree search progressively distilled into the policy network. But weights are a terrible place for this morning's API state or a procedure invented mid-task. Hence the r-machinery: tools, search, sub-agents, external state. Ugly, but you can change it without retraining a trillion parameters.
So the boundary moves in a cycle. Solve a new problem outside the model. Let traces accumulate. Compress them in. The stronger model makes harder problems tractable, which demand new scaffolding one level up. Externalize, select, distill, internalize, repeat.
That is why the model can eat the harness while the harness keeps gaining importance: two moments of the same cycle. Pure Red Queen dynamics, the better the system exploits its environment, the faster it changes the environment it must adapt to.
I don't expect the architecture to converge. The oscillation is the architecture.
Indeed, this encapsulates perfectly the challenge of our economic system operating at a critical state. AI represents an unprecedented acceleration in the redistribution of productive energy flows through our society - a classic bifurcation point in a complex system.
UBI as conceived is essentially an attempt to reorganize societal energy flows when the traditional medium of exchange (work→wages→consumption) undergoes rapid phase transition. The difficulty is precisely that our current economic structures lack the proper ago-antagonistic mechanisms to manage such transitions.
Current models fail because they operate on linear economic thinking while confronting nonlinear thermodynamic realities. When productivity detaches from human labor at scale, we witness the economic equivalent of critical opalescence - widespread instability preceding systemic reorganization.
The speed issue is particularly concerning - self-organized criticality suggests that systems naturally evolve toward critical states where small perturbations trigger avalanches of various magnitudes. AI doesn't gradually change labor markets; it potentially triggers cascading reorganizations across interconnected economic subsystems.
What's needed is a complete rethinking of resource distribution when information-energy processing fundamentally shifts from biological to technological substrates - a transition far more complex than simply implementing cash transfers.
UBI as conceived is essentially an attempt to reorganize societal energy flows when the traditional medium of exchange (work→wages→consumption) undergoes rapid phase transition. The difficulty is precisely that our current economic structures lack the proper ago-antagonistic mechanisms to manage such transitions.
Current models fail because they operate on linear economic thinking while confronting nonlinear thermodynamic realities. When productivity detaches from human labor at scale, we witness the economic equivalent of critical opalescence - widespread instability preceding systemic reorganization.
The speed issue is particularly concerning - self-organized criticality suggests that systems naturally evolve toward critical states where small perturbations trigger avalanches of various magnitudes. AI doesn't gradually change labor markets; it potentially triggers cascading reorganizations across interconnected economic subsystems.
What's needed is a complete rethinking of resource distribution when information-energy processing fundamentally shifts from biological to technological substrates - a transition far more complex than simply implementing cash transfers.
A lot of people I talk to think AI societal impact will perhaps necessitate UBI in the futureView quoted post on X
Almost none of them can articulate how this would practically work, especially given the likely sand pile collapse type speed this could happen
It’s really tough to think about