Twitxy Saved

Maya Bechler-Speicher

Verified account This account is verified. Learn more

@mayabechlerspei

AI Research Scientist, FAIR @ Meta Superintelligence Labs working on AI Verification.
125 Following 2.2K Followers 96 Posts

Latest posts 5

Pinned post

Maya Bechler-Speicher Verified account @mayabechlerspei
Our Cayley-Encoder for improving LLMs' final representations was accepted to NeurIPS 2026, together with 3 new workshop papers! πŸŽ‰

πŸŽ₯ Main track: Improving LLM Final Representations with Inter-Layer Geometry.
Most LLM-based predictors use only the final layer, but the intermediate layers carry useful, complementary signal. We introduce the Cayley-Encoder, a lightweight GNN that treats the LLM's layers as nodes in a Cayley graph over SL(2, β„€β‚™), vand produces improved final representation for the LLM.
Across 13 tasks and 9 LLMs, the Cayley-Encoder:
πŸ“ˆ improves accuracy by up to 40 points over the last layer, adding at most 0.1% extra parameters,
πŸ”¬ lets a 14M-parameter LLM beat the last layer of a 200Γ— larger one,
🎯 beats full-data baselines with just 8 labeled examples per class,
⚑ outperforms LoRA fine-tuning in 32 of 39 settings, with 6–12Γ— fewer trainable parameters.

πŸ“„ Topology-Injected Masked Fine-Tuning for Graph-Conditioned Language Models, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
TIM-FT is a new method for aligning GFM embeddings into LLM sequences. It places graph embeddings directly at the token positions where an entity is mentioned, keeps the sequence length the same, needs no vocabulary expansion, and improves performance when only 15% of entities have a graph embedding. (Available soon πŸ‘€ )

πŸ“„ Graph-Conditioned Language Models for Security Systems, NeurIPS Workshop on ML Systems.
SecBFF brings TIM-FT to security monitoring, where graph builds always lag behind the live event stream. It adds a single 530K-parameter projector and leaves the serving path unchanged. (Available soon πŸ‘€ )

πŸ“„ Predicting Cable Dynamics with Physical Attention Bias, NeurIPS Workshop on Symmetry and Geometry in Neural Representations.
A cable has two distances between any two points: along its length and through space. Adding both as attention biases cuts prediction error by 15% and more than halves length drift on cables the model has never seen.

Congrats and thanks to all my collaborators!
#NeurIPS2026
1:10
2 replies 3 reposts 17 likes 908 views View on X
Maya Bechler-Speicher Verified account @mayabechlerspei
Accepted to NeurIPS 2026!πŸ’ƒ Go improve your LLM's final representation with Cayley-Encoders!! 🀩
@NeurIPSConf
Maya Bechler-Speicher Verified account @mayabechlerspei Jun 4
πŸ€” Why do we still rely on the final layer of an LLM, when different layers encode different information? πŸ€”

In our new work, β€œImproving LLM Final Representations with Inter-Layer Geometry” (ICLR 2026 Workshop on Geometry-grounded Representation Learning and Generative Modeling) we show that actually, LLMs do not have one β€œbest” layer.
We introduce the Cayley-Encoder: an efficient and effective geometric encoder that learns one strong representation from all layer representations of the LLM, without biasing the representation toward any specific layer.
While adding at most 0.1% learned parameters to the LLM, the Cayley-Encoder achieves large empirical gains over LoRA fine-tuning, final-layer representations, expensive attention-based aggregation, and methods that optimize specific layers for the task.
View quoted post on X