What if we thought of layers as flexible building blocks that can be rearranged and parsed in an optimal order conditioned on the input itself? Our new work explores this question👇
Big congrats to @arahmani_AR, @SeanMcleish, @micahgoldblum on this great work!
Big congrats to @arahmani_AR, @SeanMcleish, @micahgoldblum on this great work!
Library of Layers: models which adapt layer order per input, routing over whole transformer blocks.View quoted post on X
This opens up a new avenue to control: the execution path the model uses. We use this to control the algorithm used to solve mazes and sudoku by only changing the layer order.
🧵 1/n