Pinned post
Our Cayley-Encoder for improving LLMs' final representations was accepted to NeurIPS 2026, together with 3 new workshop papers! π
π₯ Main track: Improving LLM Final Representations with Inter-Layer Geometry.
Most LLM-based predictors use only the final layer, but the intermediate layers carry useful, complementary signal. We introduce the Cayley-Encoder, a lightweight GNN that treats the LLM's layers as nodes in a Cayley graph over SL(2, β€β), vand produces improved final representation for the LLM.
Across 13 tasks and 9 LLMs, the Cayley-Encoder:
π improves accuracy by up to 40 points over the last layer, adding at most 0.1% extra parameters,
π¬ lets a 14M-parameter LLM beat the last layer of a 200Γ larger one,
π― beats full-data baselines with just 8 labeled examples per class,
β‘ outperforms LoRA fine-tuning in 32 of 39 settings, with 6β12Γ fewer trainable parameters.
π Topology-Injected Masked Fine-Tuning for Graph-Conditioned Language Models, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
TIM-FT is a new method for aligning GFM embeddings into LLM sequences. It places graph embeddings directly at the token positions where an entity is mentioned, keeps the sequence length the same, needs no vocabulary expansion, and improves performance when only 15% of entities have a graph embedding. (Available soon π )
π Graph-Conditioned Language Models for Security Systems, NeurIPS Workshop on ML Systems.
SecBFF brings TIM-FT to security monitoring, where graph builds always lag behind the live event stream. It adds a single 530K-parameter projector and leaves the serving path unchanged. (Available soon π )
π Predicting Cable Dynamics with Physical Attention Bias, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
A cable has two distances between any two points: along its length and through space. Adding both as attention biases cuts prediction error by 15% and more than halves length drift on cables the model has never seen.
Congrats and thanks to all my collaborators!
#NeurIPS2026
π₯ Main track: Improving LLM Final Representations with Inter-Layer Geometry.
Most LLM-based predictors use only the final layer, but the intermediate layers carry useful, complementary signal. We introduce the Cayley-Encoder, a lightweight GNN that treats the LLM's layers as nodes in a Cayley graph over SL(2, β€β), vand produces improved final representation for the LLM.
Across 13 tasks and 9 LLMs, the Cayley-Encoder:
π improves accuracy by up to 40 points over the last layer, adding at most 0.1% extra parameters,
π¬ lets a 14M-parameter LLM beat the last layer of a 200Γ larger one,
π― beats full-data baselines with just 8 labeled examples per class,
β‘ outperforms LoRA fine-tuning in 32 of 39 settings, with 6β12Γ fewer trainable parameters.
π Topology-Injected Masked Fine-Tuning for Graph-Conditioned Language Models, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
TIM-FT is a new method for aligning GFM embeddings into LLM sequences. It places graph embeddings directly at the token positions where an entity is mentioned, keeps the sequence length the same, needs no vocabulary expansion, and improves performance when only 15% of entities have a graph embedding. (Available soon π )
π Graph-Conditioned Language Models for Security Systems, NeurIPS Workshop on ML Systems.
SecBFF brings TIM-FT to security monitoring, where graph builds always lag behind the live event stream. It adds a single 530K-parameter projector and leaves the serving path unchanged. (Available soon π )
π Predicting Cable Dynamics with Physical Attention Bias, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
A cable has two distances between any two points: along its length and through space. Adding both as attention biases cuts prediction error by 15% and more than halves length drift on cables the model has never seen.
Congrats and thanks to all my collaborators!
#NeurIPS2026
1:10