Pinned post
Library of Layers: models which adapt layer order per input, routing over whole transformer blocks.
This opens up a new avenue to control: the execution path the model uses. We use this to control the algorithm used to solve mazes and sudoku by only changing the layer order.
🧵 1/n
This opens up a new avenue to control: the execution path the model uses. We use this to control the algorithm used to solve mazes and sudoku by only changing the layer order.
🧵 1/n